Smart city integrated management system and method based on multi-mode communication

Abstract

The invention discloses a smart city integrated management system and method based on multi-modal communication, and relates to the technical field of data management. The system comprises a historical emergency analysis module, a historical multi-modal data analysis and priority calculation module, a real-time emergency identification and event matching module and a data processing priority generation module. The historical emergency analysis module constructs a historical emergency mapping table by acquiring and analyzing historical event records; the historical multi-modal data analysis and priority calculation module evaluates the use priority of various types of multi-modal data in historical events and calculates offset; the real-time emergency identification and event matching module receives the real-time event notification and identifies a real-time emergency; and the data processing priority generation module generates a data processing priority and outputs the data processing priority to related personnel. According to the method, the smart city can be efficiently supported to cope with emergencies, the multi-modal data processing flow is optimized, and the event response speed and the decision-making efficiency are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data management, and in particular to a smart city integrated management system and method based on multimodal communication. Background Art

[0002] With the continuous advancement of urbanization, the development of smart cities has gradually become a core goal of modern urban management. The core of smart city systems lies in achieving comprehensive perception and precise management of urban operations through the collection, integration, and analysis of multimodal data. Multimodal data comes from a wide range of sources, including traffic monitoring, environmental sensors, social media, fire alarm systems, and encompasses various types, including video, audio, sensor signals, and text. Efficiently processing this massive and diverse data is a major challenge facing smart city systems.

[0003] While existing smart city management systems already possess basic data collection and analysis capabilities, they still face limitations in data processing and resource allocation. Specifically, most current systems utilize pre-defined fixed priority strategies (e.g., prioritization by data type or device importance), making it impossible to flexibly adjust the order of data processing based on dynamic urban events (e.g., emergencies, peak traffic flows, and environmental disasters). For example, in emergency scenarios like fires and traffic accidents, firefighting sensors or emergency communication data require an immediate response. However, due to fixed priorities, existing systems still process traffic camera or environmental monitoring data according to conventional procedures, resulting in delays in critical information and impacting the efficiency of emergency decision-making. Summary of the Invention

[0004] The purpose of the present invention is to provide a smart city integrated management system and method based on multimodal communication to solve the problems raised in the above background technology.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions: A smart city integrated management method based on multimodal communication includes the following steps: Step S100. Obtaining historical emergency event record information from the database, analyzing the historical emergency event record information, thereby extracting historical emergency event features; and matching the historical emergency event features with the historical emergency events to construct a historical emergency event mapping table; Step S200. For each historical emergency event, obtain the corresponding historical multimodal data and the usage record of the historical multimodal data, analyze them in combination with the historical emergency event mapping table, and evaluate the usage priority of the historical multimodal data; and compare and analyze the usage priority of the historical multimodal data with the fixed priority of the preset data processing to obtain the corresponding priority offset; Step S300. Receive real-time dynamic event notification information, extract real-time dynamic event features from the real-time dynamic event notification information, combine with the historical emergency mapping table, identify the real-time emergency event; and match the historical emergency events in the database based on the real-time emergency event identification results, and associate the real-time emergency event with the matched historical emergency events; Step S400. Based on the historical emergencies associated with the real-time emergencies, the usage priority and priority offset of the historical multimodal data of the historical emergencies are obtained, and combined with the fixed priority of the preset data processing, the data processing priority of the real-time multimodal data of the real-time emergencies is generated, and the data processing priority of the real-time multimodal data is output to the relevant personnel, who will perform corresponding processing.

[0006] Furthermore, step S100 includes: S101. Obtain historical emergency event record information from a database. For each historical emergency event record information, extract features according to preset feature dimensions, and standardize the extracted features to obtain a historical emergency event feature set Li, where Li={li1,li2,...,lik}, where li1 represents the first preset dimension feature of the i-th historical emergency event, li2 represents the second preset dimension feature of the i-th historical emergency event, and so on. lik represents the k-th preset dimension feature of the i-th historical emergency event, where k represents a preset feature dimension. The preset feature dimensions include time features, geographic features, event type features, etc., where time features include the duration of the historical emergency event and the time period of the event; geographic features include the location of the event and the type of region; and event type features include the type of event and the scope of event impact. S102. According to the number i of the historical emergency event feature set Li, find the historical emergency event with the same number i, so as to obtain the mapping relationship between the historical emergency event feature set Li and the historical emergency event with the number i; summarize the mapping relationship between all historical emergency event feature sets and the corresponding historical emergency events, so as to construct a historical emergency event mapping table.

[0007] Furthermore, step S200 includes: S201. For each historical emergency, obtain corresponding historical multimodal data and usage records of the historical multimodal data; the historical multimodal data refers to data related to the historical emergency and in various forms, such as text data, sensor data, video and image data, and audio data; the usage records of the historical multimodal data refer to relevant records of using the historical multimodal data when analyzing the historical emergency; based on the usage records of the historical multimodal data, calculate the usage frequency coefficient P of each type of historical multimodal data for each historical emergency, and the corresponding calculation formula is: P=(Fd×Td) / (F×T), where Fd represents the number of times the historical multimodal data d is used, Td represents the average usage time of the historical multimodal data d; m represents the total number of historical multimodal data numbers; F represents the average number of times all historical multimodal data of the historical emergency are used, and T represents the average usage time of all historical multimodal data of the historical emergency; S202. Combined with the historical emergency mapping table, obtain the historical emergency feature set of historical emergency events, calculate the matching degree M of each historical multimodal data of each historical emergency event, and the specific calculation formula is: M=wd·Sim(Li,Dd), where wd is the weight of each historical multimodal data d, and the value is between 0 and 1; Sim(Li,Dd) is the similarity measure between the corresponding historical emergency feature set Li and the historical multimodal data Dd, which represents the degree of association between event features and data, and is specifically calculated by feature matching, distance measurement, etc.; Combined with the usage frequency coefficient P and matching degree M of each historical multimodal data of each historical emergency event, comprehensively calculate the usage priority score S1 of each historical multimodal data of the historical emergency event, and the corresponding calculation formula is: S1= α×P+β×M, where α and β represent coefficients, and α+β=1; summarize the usage priority score S1 of each type of historical multimodal data of historical emergencies, thereby obtaining the corresponding usage priority score interval, and divide the usage priority score interval into N subintervals according to the number N of fixed priorities for preset data processing, each subinterval corresponding to a usage priority; for the usage priority score S1 of the historical multimodal data of historical emergencies, obtain the usage priority sequence U={u1,u2,...,un} of each type of historical multimodal data, where u1 represents the usage priority of the first type of historical multimodal data, u2 represents the usage priority of the second type of historical multimodal data, and so on, and un represents the usage priority of the nth type of historical multimodal data; S203. According to the fixed priority of the preset data processing, each historical multimodal data of the historical emergency is marked with a corresponding fixed priority, thereby obtaining a fixed priority sequence G for each historical multimodal data of the historical emergency, and G={g1,g2,...,gn}, wherein g1 represents the fixed priority of the first type of historical multimodal data; g2 represents the fixed priority of the second type of historical multimodal data, and so on, gn represents the fixed priority of the nth type of historical multimodal data; the fixed priority sequence G is compared and analyzed with the usage priority sequence U to obtain the corresponding priority offset sequence X, and X=GU.

[0008] Furthermore, step S300 includes: S301. Receive real-time dynamic event notification information, analyze the real-time dynamic event notification information according to the analysis method of historical emergency record information, thereby extracting real-time dynamic event features and forming a real-time dynamic event feature set R. The real-time dynamic event feature set R is sequentially calculated with the historical emergency feature set of each historical emergency event in the historical emergency mapping table, and the largest similarity is selected as the corresponding similarity result Sim(R,Li); S302. Compare the similarity result Sim(R, Li) with the similarity threshold S0. If Sim(R, Li) ≥ S0, mark the real-time dynamic event as a real-time emergency event; based on the similarity result Sim(R, Li), associate the historical emergency event corresponding to the historical emergency event feature set Li with the real-time emergency event.

[0009] Furthermore, step S400 includes: S401. Based on historical emergencies associated with the real-time emergency, obtain a usage priority sequence U and a priority offset sequence X for historical multimodal data of the historical emergencies. Combined with a preset fixed priority sequence G for data processing, filter out the historical multimodal data category numbers that are 0 in the priority offset sequence X. For the remaining historical multimodal data category numbers that are not 0 in the priority offset sequence X, generate several data processing priority sequences RU based on the priority offset sequence X and the corresponding fixed priority sequence G. S402. All data processing priority sequences RU are output to relevant personnel, who then perform data processing simultaneously based on different data processing priority sequences RU and the preset fixed priority sequence X, thereby obtaining corresponding data processing results and corresponding response times. Based on the data processing results and response time, the relevant personnel select an optimal data processing priority sequence as the data processing priority sequence for real-time multimodal data of real-time emergencies.

[0010] A smart city integrated management system based on multimodal communication, comprising: a historical emergency event analysis module, a historical multimodal data analysis and priority calculation module, a real-time emergency event identification and event matching module, and a data processing priority generation module; The historical emergency event analysis module obtains the record information of historical emergency events from the database, analyzes the record information of historical emergency events, and extracts the characteristics of historical emergency events; and matches the historical emergency event characteristics with the historical emergency events to construct a historical emergency event mapping table; The historical multimodal data analysis and priority calculation module obtains the corresponding historical multimodal data and the usage record of the historical multimodal data for each historical emergency event, analyzes it in combination with the historical emergency event mapping table, and evaluates the usage priority of the historical multimodal data. It also compares and analyzes the usage priority of the historical multimodal data with the fixed priority of the preset data processing to obtain the corresponding priority offset. The real-time emergency event identification and event matching module receives real-time dynamic event notification information, extracts real-time dynamic event features from the real-time dynamic event notification information, and identifies real-time emergency events by combining them with the historical emergency event mapping table. The module then matches the real-time emergency event identification results with the historical emergency events in the database and associates the real-time emergency event with the matched historical emergency events. The data processing priority generation module obtains the usage priority and priority offset of the historical multimodal data of the historical emergency events based on the historical emergency events associated with the real-time emergency events, and combines the fixed priority of the preset data processing to generate the data processing priority of the real-time multimodal data of the real-time emergency events, and outputs the data processing priority of the real-time multimodal data to the relevant personnel, who will perform corresponding processing.

[0011] Furthermore, the historical emergency event analysis module includes a feature extraction unit and a mapping table construction unit; The feature extraction unit obtains the record information of historical emergencies from the database, analyzes the record information of historical emergencies, and thus extracts the features of historical emergencies; the mapping table construction unit matches the features of historical emergencies with historical emergencies and constructs a historical emergency mapping table.

[0012] Furthermore, the historical multimodal data analysis and offset calculation module includes a historical multimodal data analysis unit and an offset calculation unit; The historical multimodal data analysis unit obtains the corresponding historical multimodal data and the usage record of the historical multimodal data for each historical emergency event, and analyzes it in combination with the historical emergency event mapping table to evaluate the usage priority of the historical multimodal data; the offset calculation unit compares and analyzes the usage priority of the historical multimodal data with the fixed priority of the preset data processing to obtain the corresponding priority offset.

[0013] Furthermore, the real-time emergency event identification and event matching module includes a real-time emergency event identification unit and an event matching unit; The real-time emergency event identification unit receives real-time dynamic event notification information, extracts real-time dynamic event features from the real-time dynamic event notification information, and identifies real-time emergency events in combination with the historical emergency event mapping table; the event matching unit matches the historical emergency events in the database based on the recognition results of the real-time emergency events, and associates the real-time emergency events with the matched historical emergency events.

[0014] Furthermore, the data processing priority generation module includes a priority analysis unit and a data processing and selection unit; The priority analysis unit obtains the usage priority and priority offset of the historical multimodal data of the historical emergency events based on the historical emergency events associated with the real-time emergency events, and generates the data processing priority of the real-time multimodal data of the real-time emergency events in combination with the fixed priority of the preset data processing; the data processing and selection unit outputs the data processing priority of the real-time multimodal data to the relevant personnel, who then perform corresponding processing.

[0015] Compared with the prior art, the present invention has the following advantages: by analyzing historical emergencies and their multimodal data, the present invention constructs a historical emergency mapping table, which can conduct in-depth analysis of the characteristics of historical events, thereby better identifying the occurrence of real-time emergencies; by combining the usage priority of historical data with the identification of real-time events, it can provide more accurate priority sorting for real-time dynamic events, thereby improving the response speed and processing effect of real-time emergencies. By comprehensively considering the usage frequency, usage duration and matching degree of historical multimodal data with historical emergencies, the present invention provides a mechanism for dynamically evaluating data usage priority; compared with the prior art, the present invention can automatically generate a priority score for multimodal data by analyzing the usage history and similarity of data, ensuring that the most relevant and effective data is processed first, thereby improving the efficiency and accuracy of data processing. The present invention introduces an offset mechanism based on the fixed priority of data processing and the usage priority, and fine-tunes the data processing priority by calculating the priority offset; this mechanism can ensure that the priority adjustment is more accurate in complex real-time event processing, thereby improving the flexibility and adaptability of the system; compared with the traditional static priority sorting method, the priority offset mechanism of the present invention can adapt to dynamically changing event requirements and provide a more refined management and scheduling solution. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings: Figure 1 It is a module schematic diagram of a smart city integrated management system based on multimodal communication of the present invention. DETAILED DESCRIPTION

[0017] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0018] See also Figure 1 , the present invention provides a technical solution: A smart city integrated management system based on multimodal communication, comprising: a historical emergency event analysis module, a historical multimodal data analysis and priority calculation module, a real-time emergency event identification and event matching module, and a data processing priority generation module; The historical emergency event analysis module obtains the record information of historical emergency events from the database, analyzes the record information of historical emergency events, and extracts the characteristics of historical emergency events; and matches the historical emergency event characteristics with the historical emergency events to construct a historical emergency event mapping table; The historical multimodal data analysis and priority calculation module obtains the corresponding historical multimodal data and the usage record of the historical multimodal data for each historical emergency event, analyzes it in combination with the historical emergency event mapping table, and evaluates the usage priority of the historical multimodal data. It also compares and analyzes the usage priority of the historical multimodal data with the fixed priority of the preset data processing to obtain the corresponding priority offset. The real-time emergency event identification and event matching module receives real-time dynamic event notification information, extracts real-time dynamic event features from the real-time dynamic event notification information, and identifies real-time emergency events by combining them with the historical emergency event mapping table. The module then matches the real-time emergency event identification results with the historical emergency events in the database and associates the real-time emergency event with the matched historical emergency events. The data processing priority generation module obtains the usage priority and priority offset of the historical multimodal data of the historical emergency events based on the historical emergency events associated with the real-time emergency events, and combines the fixed priority of the preset data processing to generate the data processing priority of the real-time multimodal data of the real-time emergency events, and outputs the data processing priority of the real-time multimodal data to the relevant personnel, who will perform corresponding processing.

[0019] The historical emergency event analysis module includes a feature extraction unit and a mapping table construction unit; The feature extraction unit obtains the record information of historical emergencies from the database, analyzes the record information of historical emergencies, and thus extracts the features of historical emergencies; the mapping table construction unit matches the features of historical emergencies with historical emergencies and constructs a historical emergency mapping table.

[0020] The historical multimodal data analysis and offset calculation module includes a historical multimodal data analysis unit and an offset calculation unit; The historical multimodal data analysis unit obtains the corresponding historical multimodal data and the usage record of the historical multimodal data for each historical emergency event, and analyzes it in combination with the historical emergency event mapping table to evaluate the usage priority of the historical multimodal data; the offset calculation unit compares and analyzes the usage priority of the historical multimodal data with the fixed priority of the preset data processing to obtain the corresponding priority offset.

[0021] The real-time emergency event recognition and event matching module includes a real-time emergency event recognition unit and an event matching unit; The real-time emergency event identification unit receives real-time dynamic event notification information, extracts real-time dynamic event features from the real-time dynamic event notification information, and identifies real-time emergency events in combination with the historical emergency event mapping table; the event matching unit matches the historical emergency events in the database based on the recognition results of the real-time emergency events, and associates the real-time emergency events with the matched historical emergency events.

[0022] The data processing priority generation module includes a priority analysis unit and a data processing and selection unit; The priority analysis unit obtains the usage priority and priority offset of the historical multimodal data of the historical emergency events based on the historical emergency events associated with the real-time emergency events, and generates the data processing priority of the real-time multimodal data of the real-time emergency events in combination with the fixed priority of the preset data processing; the data processing and selection unit outputs the data processing priority of the real-time multimodal data to the relevant personnel, who then perform corresponding processing.

[0023] A smart city integrated management method based on multimodal communication includes the following steps: Step S100. Obtaining historical emergency event record information from the database, analyzing the historical emergency event record information, thereby extracting historical emergency event features; and matching the historical emergency event features with the historical emergency events to construct a historical emergency event mapping table; Step S200. For each historical emergency event, obtain the corresponding historical multimodal data and the usage record of the historical multimodal data, analyze them in combination with the historical emergency event mapping table, and evaluate the usage priority of the historical multimodal data; and compare and analyze the usage priority of the historical multimodal data with the fixed priority of the preset data processing to obtain the corresponding priority offset; Step S300. Receive real-time dynamic event notification information, extract real-time dynamic event features from the real-time dynamic event notification information, combine with the historical emergency mapping table, identify the real-time emergency event; and match the historical emergency events in the database based on the real-time emergency event identification results, and associate the real-time emergency event with the matched historical emergency events; Step S400. Based on the historical emergencies associated with the real-time emergencies, the usage priority and priority offset of the historical multimodal data of the historical emergencies are obtained, and combined with the fixed priority of the preset data processing, the data processing priority of the real-time multimodal data of the real-time emergencies is generated, and the data processing priority of the real-time multimodal data is output to the relevant personnel, who will perform corresponding processing.

[0024] Step S100 includes: S101. Obtain historical emergency event record information from a database. For each historical emergency event record information, extract features according to preset feature dimensions, and standardize the extracted features to obtain a historical emergency event feature set Li, where Li={li1,li2,...,lik}, where li1 represents the first preset dimension feature of the i-th historical emergency event, li2 represents the second preset dimension feature of the i-th historical emergency event, and so on. lik represents the k-th preset dimension feature of the i-th historical emergency event, where k represents a preset feature dimension. The preset feature dimensions include time features, geographic features, event type features, etc., where time features include the duration of the historical emergency event and the time period of the event; geographic features include the location of the event and the type of region; and event type features include the type of event and the scope of event impact. S102. According to the number i of the historical emergency event feature set Li, find the historical emergency event with the same number i, so as to obtain the mapping relationship between the historical emergency event feature set Li and the historical emergency event with the number i; summarize the mapping relationship between all historical emergency event feature sets and the corresponding historical emergency events, so as to construct a historical emergency event mapping table.

[0025] Step S200 includes: S201. For each historical emergency, obtain corresponding historical multimodal data and usage records of the historical multimodal data; the historical multimodal data refers to data related to the historical emergency and in various forms, such as text data, sensor data, video and image data, and audio data; the usage records of the historical multimodal data refer to relevant records of using the historical multimodal data when analyzing the historical emergency; based on the usage records of the historical multimodal data, calculate the usage frequency coefficient P of each type of historical multimodal data for each historical emergency, and the corresponding calculation formula is: P=(Fd×Td) / (F×T), where Fd represents the number of times the historical multimodal data d is used, Td represents the average usage time of the historical multimodal data d; m represents the total number of historical multimodal data numbers; F represents the average number of times all historical multimodal data of the historical emergency are used, and T represents the average usage time of all historical multimodal data of the historical emergency; S202. Combined with the historical emergency mapping table, obtain the historical emergency feature set of historical emergency events, calculate the matching degree M of each historical multimodal data of each historical emergency event, and the specific calculation formula is: M=wd·Sim(Li,Dd), where wd is the weight of each historical multimodal data d, and the value is between 0 and 1; Sim(Li,Dd) is the similarity measure between the corresponding historical emergency feature set Li and the historical multimodal data Dd, which represents the degree of association between event features and data, and is specifically calculated by feature matching, distance measurement, etc.; Combined with the usage frequency coefficient P and matching degree M of each historical multimodal data of each historical emergency event, comprehensively calculate the usage priority score S1 of each historical multimodal data of the historical emergency event, and the corresponding calculation formula is: S1= α×P+β×M, where α and β represent coefficients, and α+β=1; summarize the usage priority score S1 of each type of historical multimodal data of historical emergencies, thereby obtaining the corresponding usage priority score interval, and divide the usage priority score interval into N subintervals according to the number N of fixed priorities for preset data processing, each subinterval corresponding to a usage priority; for the usage priority score S1 of the historical multimodal data of historical emergencies, obtain the usage priority sequence U={u1,u2,...,un} of each type of historical multimodal data, where u1 represents the usage priority of the first type of historical multimodal data, u2 represents the usage priority of the second type of historical multimodal data, and so on, and un represents the usage priority of the nth type of historical multimodal data; S203. According to the fixed priority of the preset data processing, each historical multimodal data of the historical emergency is marked with a corresponding fixed priority, thereby obtaining a fixed priority sequence G for each historical multimodal data of the historical emergency, and G={g1,g2,...,gn}, wherein g1 represents the fixed priority of the first type of historical multimodal data; g2 represents the fixed priority of the second type of historical multimodal data, and so on, gn represents the fixed priority of the nth type of historical multimodal data; the fixed priority sequence G is compared and analyzed with the usage priority sequence U to obtain the corresponding priority offset sequence X, and X=GU.

[0026] In this embodiment, it is assumed that the types of historical multimodal data are text data, sensor data, video and image data, and audio data, and the corresponding fixed priorities of preset data processing are: sensor data (fixed priority 1, the highest priority), video and image data (fixed priority 2), text data (fixed priority 3), audio data (fixed priority 4, the lowest priority), and the corresponding fixed priority sequence G = {g1, g2, g3, g4} = {text data, sensor data, video and image data, audio data} = {3, 1, 2, 4}; Assume that a historical emergency event occurs. By analyzing the usage priority score S1 of the historical multimodal data of this historical emergency event, we can obtain the corresponding usage priority sequence U={u1,u2,u3,u4}={text data, sensor data, video and image data, audio data}={2,3,1,4}; compare and analyze the fixed priority sequence G with the usage priority sequence U to obtain the corresponding priority offset sequence X, and X=GU={1,-2,1,0}.

[0027] Step S300 includes: S301. Receive real-time dynamic event notification information, analyze the real-time dynamic event notification information according to the analysis method of historical emergency record information, thereby extracting real-time dynamic event features and forming a real-time dynamic event feature set R. The real-time dynamic event feature set R is sequentially calculated with the historical emergency feature set of each historical emergency event in the historical emergency mapping table, and the largest similarity is selected as the corresponding similarity result Sim(R,Li); S302. Compare the similarity result Sim(R, Li) with the similarity threshold S0. If Sim(R, Li) ≥ S0, mark the real-time dynamic event as a real-time emergency event; based on the similarity result Sim(R, Li), associate the historical emergency event corresponding to the historical emergency event feature set Li with the real-time emergency event.

[0028] Step S400 includes: S401. Based on historical emergencies associated with the real-time emergency, obtain a usage priority sequence U and a priority offset sequence X for historical multimodal data of the historical emergencies. Combined with a preset fixed priority sequence G for data processing, filter out the historical multimodal data category numbers that are 0 in the priority offset sequence X. For the remaining historical multimodal data category numbers that are not 0 in the priority offset sequence X, generate several data processing priority sequences RU based on the priority offset sequence X and the corresponding fixed priority sequence G. S402. All data processing priority sequences RU are output to relevant personnel, who then perform data processing simultaneously based on different data processing priority sequences RU and the preset fixed priority sequence X, thereby obtaining corresponding data processing results and corresponding response times. Based on the data processing results and response time, the relevant personnel select an optimal data processing priority sequence as the data processing priority sequence for real-time multimodal data of real-time emergencies.

[0029] In this embodiment, it is assumed that the fixed priority sequence of historical emergency events associated with the real-time emergency event is: G = {g1, g2, g3, g4} = {text data, sensor data, video and image data, audio data} = {3, 1, 2, 4}; the corresponding historical emergency usage priority sequence is: U={u1,u2,u3,u4}={text data, sensor data, video and image data, audio data}={2,3,1,4}; Compare and analyze the fixed priority sequence G with the usage priority sequence U to obtain the corresponding priority offset sequence X, where X=GU={1,-2,1,0}; According to the priority offset sequence X, the historical multimodal data type number of 0 is filtered out, so the audio data is filtered out; the historical multimodal data type number that is not 0 in the remaining priority offset sequence X is combined with the corresponding fixed priority sequence G according to the priority offset sequence X to generate several data processing priority sequences RU; according to the sign of the priority offset sequence X, the corresponding fixed priority sequence G is adjusted. For example, the sign of the priority offset corresponding to the video and image data is positive, so the priority in the corresponding fixed priority sequence G needs to be advanced by one, and the step size of each adjustment is 1, so the priority corresponding to the video and image data is 1; the adjusted fixed priority sequence G1={3, 1, 1, 4}, since the priority corresponding to the sensor data is 1, which is repeated with the priority corresponding to the video and image data, the priority corresponding to the sensor data needs to be adjusted; according to the priority offset sequence X, it can be seen that the offset sign of the priority corresponding to the sensor data is negative, so the corresponding fixed priority sequence G is negative. The priority in the priority sequence G needs to be moved back by one, so the adjusted fixed priority sequence G2={3, 2, 1, 4} is obtained, thereby obtaining the real-time data processing priority sequence RU={3, 2, 1, 4}; the same analysis is performed on the historical multimodal data type numbers that are not 0 in other priority offset sequences X, thereby obtaining other real-time data processing priority sequences: RU={2, 3, 1, 4}; all real-time data processing priority sequences and preset fixed priority sequences are summarized to obtain a real-time data processing priority sequence list; RU={3, 2, 1, 4} and RU={2, 3, 1, 4} and the fixed priority sequence G={3, 1, 2, 4} are output to relevant personnel, and the relevant personnel perform data processing according to the corresponding data processing priority sequence to obtain the corresponding data processing results and response time, and select an optimal data processing priority sequence according to the data processing results and response time as the data processing priority sequence for the current real-time emergency.

[0030] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0031] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art will be able to modify the technical solutions described in the aforementioned embodiments or substitute equivalents for some of the technical features. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A smart city integrated management method based on multimodal communication, characterized by: The method comprises the following steps: Step S100. Obtaining historical emergency event record information from the database, analyzing the historical emergency event record information, thereby extracting historical emergency event features; and matching the historical emergency event features with the historical emergency events to construct a historical emergency event mapping table; Step S200. For each historical emergency event, obtain the corresponding historical multimodal data and the usage record of the historical multimodal data, analyze them in combination with the historical emergency event mapping table, and evaluate the usage priority of the historical multimodal data; and compare and analyze the usage priority of the historical multimodal data with the fixed priority of the preset data processing to obtain the corresponding priority offset; Step S300. Receive real-time dynamic event notification information, extract real-time dynamic event features from the real-time dynamic event notification information, combine with the historical emergency mapping table, identify the real-time emergency event; and match the historical emergency events in the database based on the real-time emergency event identification results, and associate the real-time emergency event with the matched historical emergency events; Step S400. Based on the historical emergencies associated with the real-time emergencies, the usage priority and priority offset of the historical multimodal data of the historical emergencies are obtained, and combined with the fixed priority of the preset data processing, the data processing priority of the real-time multimodal data of the real-time emergencies is generated, and the data processing priority of the real-time multimodal data is output to the relevant personnel, who will perform corresponding processing.

2. The method for comprehensive smart city management based on multimodal communication according to claim 1, characterized in that: The step S100 includes: S101. Obtain historical emergency event records from a database. For each historical emergency event record, extract features based on preset feature dimensions. Standardize the extracted features to obtain a historical emergency event feature set Li, where Li = {li1, li2, ..., lik}, where li1 represents the first preset dimension feature of the i-th historical emergency event, li2 represents the second preset dimension feature of the i-th historical emergency event, and so on. lik represents the k-th preset dimension feature of the i-th historical emergency event, where k represents the preset feature dimension. S102. According to the number i of the historical emergency event feature set Li, find the historical emergency event with the same number i, so as to obtain the mapping relationship between the historical emergency event feature set Li and the historical emergency event with the number i; summarize the mapping relationship between all historical emergency event feature sets and the corresponding historical emergency events, so as to construct a historical emergency event mapping table.

3. The method for comprehensive smart city management based on multimodal communication according to claim 2, characterized in that: The step S200 includes: S201. For each historical emergency event, obtain corresponding historical multimodal data and usage records of the historical multimodal data; the historical multimodal data refers to data related to the historical emergency event and having multiple forms; the usage records of the historical multimodal data refer to relevant records of using the historical multimodal data when analyzing the historical emergency event; based on the usage records of the historical multimodal data, calculate the usage frequency coefficient P of each type of historical multimodal data for each historical emergency event, and the corresponding calculation formula is: P=(Fd×Td) / (F×T), where Fd represents the number of times the historical multimodal data d is used, Td represents the average usage time of the historical multimodal data d; m represents the total number of historical multimodal data numbers; F represents the average number of times all historical multimodal data of the historical emergency event are used, and T represents the average usage time of all historical multimodal data of the historical emergency event; S202. Combined with the historical emergency mapping table, the historical emergency feature set of the historical emergency is obtained, and the matching degree M of each historical multimodal data of each historical emergency is calculated. The specific calculation formula is: M=wd·Sim(Li,Dd), where wd is the weight of each historical multimodal data d; Sim(Li,Dd) is the similarity measure between the corresponding historical emergency feature set Li and the historical multimodal data Dd, which represents the degree of association between the event feature and the data; combined with the usage frequency coefficient P and matching degree M of each historical multimodal data of each historical emergency, the usage priority score S1 of each historical multimodal data of the historical emergency is comprehensively calculated, and the corresponding calculation formula is: S1=α×P+β×M, where α and β represent represents the coefficient, and α+β=1; summarize the usage priority score S1 of each type of historical multimodal data of historical emergencies, thereby obtaining the corresponding usage priority score interval, and divide the usage priority score interval into N subintervals according to the number N of fixed priorities for preset data processing, and each subinterval corresponds to a usage priority; for the usage priority score S1 of the historical multimodal data of historical emergencies, obtain the usage priority sequence U={u1,u2,...,un} of each type of historical multimodal data, where u1 represents the usage priority of the first type of historical multimodal data, u2 represents the usage priority of the second type of historical multimodal data, and so on, and un represents the usage priority of the nth type of historical multimodal data; S203. According to the fixed priority of the preset data processing, each historical multimodal data of the historical emergency is marked with a corresponding fixed priority, thereby obtaining a fixed priority sequence G for each historical multimodal data of the historical emergency, and G={g1,g2,...,gn}, wherein g1 represents the fixed priority of the first type of historical multimodal data; g2 represents the fixed priority of the second type of historical multimodal data, and so on, gn represents the fixed priority of the nth type of historical multimodal data; the fixed priority sequence G is compared and analyzed with the usage priority sequence U to obtain the corresponding priority offset sequence X, and X=GU.

4. The method for comprehensive smart city management based on multimodal communication according to claim 3, characterized in that: The step S300 includes: S301. Receive real-time dynamic event notification information, analyze the real-time dynamic event notification information according to the analysis method of historical emergency record information, thereby extracting real-time dynamic event features and forming a real-time dynamic event feature set R. The real-time dynamic event feature set R is sequentially calculated with the historical emergency feature set of each historical emergency event in the historical emergency mapping table, and the largest similarity is selected as the corresponding similarity result Sim(R,Li); S302. Compare the similarity result Sim(R, Li) with the similarity threshold S0. If Sim(R, Li) ≥ S0, mark the real-time dynamic event as a real-time emergency event; based on the similarity result Sim(R, Li), associate the historical emergency event corresponding to the historical emergency event feature set Li with the real-time emergency event.

5. The method for comprehensive smart city management based on multimodal communication according to claim 4, characterized in that: The step S400 includes: S401. Based on historical emergencies associated with the real-time emergency, obtain a usage priority sequence U and a priority offset sequence X for historical multimodal data of the historical emergencies. Combined with a preset fixed priority sequence G for data processing, filter out the historical multimodal data category numbers that are 0 in the priority offset sequence X. For the remaining historical multimodal data category numbers that are not 0 in the priority offset sequence X, generate several data processing priority sequences RU based on the priority offset sequence X and the corresponding fixed priority sequence G. S402. All data processing priority sequences RU are output to relevant personnel, who then perform data processing simultaneously based on different data processing priority sequences RU and the preset fixed priority sequence X, thereby obtaining corresponding data processing results and corresponding response times. Based on the data processing results and response time, the relevant personnel select an optimal data processing priority sequence as the data processing priority sequence for real-time multimodal data of real-time emergencies.

6. A smart city integrated management system based on multimodal communication, applied to a smart city integrated management method based on multimodal communication according to any one of claims 1 to 5, characterized in that: The system includes: a historical emergency event analysis module, a historical multimodal data analysis and priority calculation module, a real-time emergency event identification and event matching module, and a data processing priority generation module; The historical emergency event analysis module obtains the record information of historical emergency events from the database, analyzes the record information of historical emergency events, thereby extracting the characteristics of historical emergency events; and matches the historical emergency event characteristics with the historical emergency events to construct a historical emergency event mapping table; The historical multimodal data analysis and priority calculation module obtains the corresponding historical multimodal data and the usage record of the historical multimodal data for each historical emergency event, analyzes it in combination with the historical emergency event mapping table, and evaluates the usage priority of the historical multimodal data; and compares and analyzes the usage priority of the historical multimodal data with the fixed priority of the preset data processing to obtain the corresponding priority offset; The real-time emergency event identification and event matching module receives real-time dynamic event notification information, extracts real-time dynamic event features from the real-time dynamic event notification information, and identifies real-time emergency events in combination with the historical emergency event mapping table; matches historical emergency events in the database based on the identification results of the real-time emergency events, and associates the real-time emergency events with the matched historical emergency events; The data processing priority generation module obtains the usage priority and priority offset of the historical multimodal data of the historical emergency events based on the historical emergency events associated with the real-time emergency events, and combines the fixed priority of the preset data processing to generate the data processing priority of the real-time multimodal data of the real-time emergency events, and outputs the data processing priority of the real-time multimodal data to relevant personnel, who then perform corresponding processing.

7. The smart city integrated management system based on multimodal communication according to claim 6, characterized in that: The historical emergency event analysis module includes a feature extraction unit and a mapping table construction unit; The feature extraction unit obtains the record information of historical emergencies from the database, analyzes the record information of historical emergencies, and thus extracts the features of historical emergencies; The mapping table construction unit matches historical emergency event features with historical emergency events to construct a historical emergency event mapping table.

8. The smart city integrated management system based on multimodal communication according to claim 6, characterized in that: The historical multimodal data analysis and offset calculation module includes a historical multimodal data analysis unit and an offset calculation unit; The historical multimodal data analysis unit obtains the corresponding historical multimodal data and the usage record of the historical multimodal data for each historical emergency event, and analyzes it in combination with the historical emergency event mapping table to evaluate the usage priority of the historical multimodal data; the offset calculation unit compares and analyzes the usage priority of the historical multimodal data with the fixed priority of the preset data processing to obtain the corresponding priority offset.

9. The smart city integrated management system based on multimodal communication according to claim 6, characterized in that: The real-time emergency event recognition and event matching module includes a real-time emergency event recognition unit and an event matching unit; The real-time emergency event identification unit receives the real-time dynamic event notification information, extracts the real-time dynamic event features from the real-time dynamic event notification information, and identifies the real-time emergency event by combining the features with the historical emergency event mapping table; The event matching unit matches the historical emergency events in the database according to the recognition result of the real-time emergency event, and associates the real-time emergency event with the matched historical emergency event.

10. The smart city integrated management system based on multimodal communication according to claim 6, characterized in that: The data processing priority generation module includes a priority analysis unit and a data processing and selection unit; The priority analysis unit obtains the usage priority and priority offset of the historical multimodal data of the historical emergency events based on the historical emergency events associated with the real-time emergency event, and generates the data processing priority of the real-time multimodal data of the real-time emergency event in combination with the fixed priority of the preset data processing; the data processing and selection unit outputs the data processing priority of the real-time multimodal data to relevant personnel, who then perform corresponding processing.

Images