Community multi-mode emergency event grading response system

Abstract

The invention relates to the technical field of community emergency management, and discloses a community multi-mode emergency event grading response system. The multi-modal data acquisition module acquires and divides community multi-modal sensing data, the event feature extraction module respectively analyzes and generates event feature sets, the grading evaluation module fuses features to match emergency event grades, the response decision module calls a target response scheme according to the grades, and the central processing unit verifies related parameters to generate a final response instruction. The system is further provided with a resource scheduling module for optimizing emergency resource allocation, an instruction feedback module for triggering equipment response action, and a data archiving module for storing data to form an event record chain. According to the system, multi-modal data deep processing, accurate grading evaluation and intelligent response decision making are realized, the community emergency management efficiency and the intelligent level are improved, and the community safety is guaranteed.

Description

Technical Field

[0001] The present invention relates to the technical field of community emergency management, and in particular to a community multimodal emergency event hierarchical response system. Background Art

[0002] In modern society, communities, as the fundamental units of human life, are crucial for their safety. However, as communities continue to expand in size and become increasingly complex in their functions, the frequency and complexity of various emergency incidents are also increasing, posing numerous challenges to traditional emergency management approaches.

[0003] In terms of data collection, community emergency management systems previously acquired data in a single format, relying primarily on simple sensors or manual reporting. For example, relying solely on smoke sensors to monitor fire hazards made it difficult to fully grasp the on-site situation. A single sensor couldn't capture multimodal information like video and audio, preventing emergency response personnel from intuitively understanding key information such as the severity of the incident, the scope of involvement, and the status of personnel. This resulted in a lack of sufficient data support for emergency decision-making, resulting in inaccurate decisions and potentially delaying rescue efforts.

[0004] From the perspective of event feature analysis, traditional systems process collected data in a simplistic manner and lack the ability to deeply mine the data's value. For structured data, such as temperature and humidity data collected by sensors, simple threshold determinations are performed, making it difficult to analyze data trends over time and predict the course of events. Furthermore, unstructured data, such as community surveillance video and audio, remains largely underutilized, making it difficult to extract effective spatial features and accurately determine the specific location and environmental conditions of an incident, making it difficult to effectively target emergency responses.

[0005] Previous emergency response classification standards were not scientifically sound. Most were based on a single factor or simple rules, such as the severity of a fire, without considering multiple factors such as the risk of casualties, the potential for property damage, and the scope of the incident. This resulted in a mismatch between emergency response measures and the actual severity of the incident, leading to either an overreaction resulting in a waste of resources or an underreaction that failed to effectively control the situation.

[0006] Traditional community emergency response systems have fixed and inflexible response strategies. Once an emergency occurs, they can only follow a limited number of pre-set plans, unable to dynamically adjust to the real-time changes and specific circumstances of the incident. For example, when handling a traffic accident within a community, it is impossible to quickly develop an optimized rescue and traffic diversion plan based on factors such as the damage to the vehicles at the scene, the extent of injuries, and surrounding road congestion.

[0007] In terms of emergency resource dispatch, the traditional model inefficiently matches community emergency resources with incident needs. Emergency resources within the community are scattered and lack effective integration and management. When an emergency occurs, it's difficult to quickly and accurately locate and deploy appropriate resources to the scene. For example, firefighting equipment may be located far from the fire scene, and dispatch routes are not properly planned, resulting in delayed resource deployment and impacting rescue effectiveness.

[0008] Data management and feedback mechanisms are also weak links in traditional community emergency management systems. Previous systems often neglected the storage and analysis of relevant data after emergency response. Data was often stored in fragmented and inconsistent formats, making it difficult to form a complete chain of events, hindering subsequent review and summary of events and the accumulation of experience. Furthermore, during the emergency response process, there was a lack of effective feedback mechanisms, making it impossible to track the execution of response instructions in real time, making it difficult to adjust strategies based on actual results. Summary of the Invention

[0009] The purpose of the present invention is to provide a community multimodal emergency event hierarchical response system to solve the problems raised in the above background technology.

[0010] To achieve the above objectives, the present invention provides the following technical solution: a community multimodal emergency event hierarchical response system, the system comprising: A multimodal data acquisition module is used to obtain multimodal sensor data within the community and divide the multimodal sensor data into structured data streams and unstructured data streams based on preset data classification rules; An event feature extraction module, configured to perform temporal feature analysis on the structured data stream to generate a first event feature set, and perform spatial feature analysis on the unstructured data stream to generate a second event feature set; A hierarchical assessment module is configured to fuse the first event feature set and the second event feature set according to a preset event level mapping table, match them to the corresponding emergency event level, and use the emergency event level as an assessment parameter for the current event; A response decision module is used to call a target response plan in a preset response strategy library based on the emergency event level and use the target response plan as an execution parameter of the current event; a central processing unit, configured to send the multimodal sensing data to the event feature extraction module, send the first event feature set and the second event feature set to the hierarchical evaluation module, and perform a logic check on the evaluation parameters and the execution parameters to generate a final response instruction; Preferably, the event feature extraction module performs spatial feature analysis on the unstructured data stream, including: Dividing the video frame sequence in the unstructured data stream into a key frame group and a non-key frame group, and performing local texture feature extraction on the key frame group based on a preset convolution kernel group to generate a spatial feature matrix; Performing spectrum slicing processing on the audio waveform data in the unstructured data stream, extracting frequency domain energy distribution features within each slice and constructing a frequency domain feature vector; The spatial feature matrix and the frequency domain feature vector are cross-modal associated encoded to generate the second event feature set.

[0011] Preferably, the preset data classification rules include a basic data type set and an extended data type set; the basic data type set includes a video data identifier, an audio data identifier, and a sensor data identifier; the extended data type set includes a text data identifier and a geographic location data identifier, and each data identifier corresponds to an independent data parsing protocol.

[0012] Preferably, the system further comprises a communication interface, wherein the communication interface is used to realize communication connection between the multimodal data acquisition module, the event feature extraction module, the hierarchical assessment module and the response decision module and the community monitoring network respectively; The multimodal data acquisition module classifies the multimodal sensor data based on the preset data classification rule, including: Receiving original data packets from the community monitoring network in real time through the communication interface, and matching header labels of the original data packets according to data identifiers in the basic data type set to separate basic data segments; Traversing the extension field of the original data packet according to the data identifier in the extended data type set to extract the extended data segment; The basic data segment and the extended data segment are aligned according to timestamps and then written into a structured data buffer and an unstructured data buffer respectively.

[0013] Preferably, when the preset event level mapping table adopts a linear grading model, the emergency event level is a linear mapping result of the weighted fusion value of the first event feature set and the second event feature set; When the preset event level mapping table adopts a nonlinear grading model, the emergency event level is a discrete label set that classifies the nonlinear transformation results of the first event feature set and the second event feature set through an activation function.

[0014] Preferably, it further comprises a resource scheduling module connected to the central processing unit, wherein the resource scheduling module is connected to the community emergency resource database via the communication interface; The resource scheduling module is used to match the available equipment list from the community emergency resource database according to the equipment call requirements in the final response instruction, and generate an equipment deployment topology diagram to optimize the resource allocation path.

[0015] Preferably, the resource scheduling module generates a device deployment topology diagram including: Obtaining a three-dimensional map model of the community, and marking the real-time location coordinates of each device in the list of available devices in the three-dimensional map model; Calculating the shortest feasible path from the real-time location of each device to the target emergency area based on a path planning algorithm, and prioritizing the list of available devices according to path length; The shortest feasible path and the priority ranking are superimposed on the three-dimensional map model to generate a dynamically updated device deployment topology map.

[0016] Preferably, when the central processing unit performs logical verification on the evaluation parameters and the execution parameters, a dual verification mechanism of redundant verification rules and conflict detection rules is adopted, wherein the redundant verification rules are used to confirm the integrity of the parameters, and the conflict detection rules are used to eliminate logical contradictions between the parameters.

[0017] Preferably, the system further comprises a command feedback module connected to the central processing unit, the command feedback module being used to convert the final response command into a device control signal, and to send the device control signal to the target emergency device through the communication interface to trigger a response action.

[0018] Preferably, the system also includes a data archiving module connected to the central processing unit, and the data archiving module is used to store the multimodal sensing data, the first event feature set, the second event feature set, the emergency event level and the final response instruction, and generate a complete emergency event record chain according to the event timeline.

[0019] Compared with the prior art, the present invention has the following beneficial effects: In terms of data collection and processing, the multimodal data acquisition module can acquire rich multimodal sensor data within the community and divide it into structured and unstructured data streams based on preset classification rules. This design changes the single nature of traditional data collection. For example, in fire warning scenarios, it can not only collect structured sensor data such as smoke and temperature, but also obtain unstructured data such as on-site video and audio. The event feature extraction module analyzes the temporal and spatial features of the two types of data respectively to generate a comprehensive set of event features. In this way, the system can more deeply explore the information behind the data, accurately grasp the essential characteristics of the event, and provide a solid data foundation for subsequent hierarchical evaluation and decision-making.

[0020] The hierarchical assessment module matches the fused event feature set to the corresponding emergency event level based on a preset event level mapping table. Whether using a linear or nonlinear hierarchical model, both can scientifically and rationally determine the event level based on a comprehensive set of factors. For example, when addressing public health incidents within a community, the module considers both the changing trend in the number of infections (structured data characteristics) and the extent of the epidemic's spread across different areas of the community (unstructured data spatial characteristics) to arrive at an accurate event level. This avoids the one-sidedness of traditional hierarchical methods and ensures that emergency responses are more closely aligned with the actual severity of the incident.

[0021] The response decision module invokes targeted response plans from a pre-set response strategy library based on the emergency level, ensuring swift and targeted action for each emergency level. For example, when responding to a small fire (a lower emergency level), nearby small-scale firefighting equipment and personnel are deployed for initial firefighting. However, when facing a large fire (a higher emergency level), a large-scale fire emergency plan is activated, deploying more specialized firefighting forces and large-scale firefighting equipment. This precise, level-based decision-making effectively avoids over- or under-response and improves the efficiency and effectiveness of emergency response.

[0022] The central processing unit performs logical checks on both evaluation and execution parameters, employing a dual verification mechanism that combines redundancy check rules with conflict detection rules. This mechanism ensures parameter integrity and logical consistency, preventing decision-making errors caused by parameter errors. In actual emergency response scenarios, inconsistent parameters from multiple data sources or missing parameters may occur. This dual verification mechanism promptly identifies and resolves these issues, ensuring the accuracy and reliability of the final response instructions.

[0023] The resource scheduling module matches the device call requirements in the final response command to the available equipment list in the community emergency resource database and generates a device deployment topology to optimize resource allocation paths. For example, in the post-earthquake rescue scenario, the resource scheduling module can quickly locate resources such as medical emergency equipment and excavation equipment within the community. Using a path planning algorithm, it calculates the shortest feasible path and prioritizes them, effectively deploying resources to the affected area. This significantly improves the utilization of emergency resources and reduces rescue time.

[0024] The command feedback module converts the final response command into a device control signal and sends it to the target emergency equipment, triggering the response action and ensuring rapid command execution. Simultaneously, the data archiving module stores multimodal sensor data, event feature sets, emergency event levels, and final response commands, generating a complete emergency event record chain based on the event timeline. This not only facilitates real-time monitoring and adjustments during event handling but also provides rich data support for subsequent event review, experience summary, and emergency plan optimization, promoting the continuous improvement of community emergency management capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is a working principle diagram of the community multimodal emergency incident hierarchical response system according to the present invention; Figure 2 A diagram showing the working principle of the event feature extraction module for processing unstructured data streams; Figure 3 This is a working principle diagram of the multimodal data acquisition module dividing data based on the communication interface; Figure 4 This is a diagram showing the working principle of the central processing unit logic check. DETAILED DESCRIPTION

[0026] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0027] See also Figures 1-4 This invention provides a hierarchical community multimodal emergency response system designed to achieve efficient monitoring, accurate assessment, and rapid response to community emergencies. The system primarily consists of a multimodal data acquisition module, an event feature extraction module, a hierarchical assessment module, a response decision module, and a central processing unit.

[0028] During system operation, the multimodal data acquisition module first comes into play, acquiring multimodal sensor data from across the community. This data comes from a wide range of sources, including video data collected by surveillance cameras, audio data collected by microphones, and data collected by various sensors (such as temperature and smoke sensors). It may also include text data (such as community announcements and resident feedback) and geolocation data. Based on pre-set data classification rules, the multimodal data acquisition module divides the acquired multimodal sensor data into structured and unstructured data streams for subsequent targeted processing.

[0029] Next, the event feature extraction module begins its work. It analyzes the temporal features of the structured data stream, such as the temporal trends of sensor data and the time intervals between events, to generate the first event feature set. For the unstructured data stream, the event feature extraction module analyzes the spatial features to generate the second event feature set. These two event feature sets contain key information related to the emergency event and serve as an important basis for subsequent event assessment and decision-making.

[0030] The hierarchical assessment module fuses the first and second event feature sets according to a preset event level mapping table. This fusion more comprehensively reflects the characteristics of the emergency event. The fused features are then matched against the preset event level mapping table to determine the emergency level corresponding to the current event, which is then used as the assessment parameter for the current event.

[0031] The response decision module uses the target response plan from the preset response strategy library based on the emergency event level determined by the grading assessment module. This library contains a variety of response plans for different emergency event levels. These plans are developed based on past experience and professional analysis, ensuring scientific and effective results. The response decision module uses the selected target response plan as the execution parameter for the current event.

[0032] The central processing unit (CPU) plays a core coordination and control role in the entire system. It is responsible for sending multimodal sensor data to the event feature extraction module for feature extraction. Simultaneously, it sends the first and second event feature sets to the grading assessment module to provide data support for event grading. Furthermore, the CPU performs logical checks on the evaluation and execution parameters, employing a dual verification mechanism of redundancy check rules and conflict detection rules to ensure parameter integrity and eliminate logical inconsistencies between parameters, ultimately generating accurate and reliable final response instructions.

[0033] Embodiment 1: In this embodiment, the specific process of the event feature extraction module performing spatial feature analysis on the unstructured data stream is further described. When the unstructured data stream contains video frame sequences and audio waveform data, the module will perform the following operations.

[0034] First, divide a video frame sequence into keyframe and non-keyframe groups. Keyframes are frames that contain important information and represent the main content and changes in the video. This division reduces data processing and improves efficiency. The division method can be determined based on the degree of change in the video content. For example, when objects in the video are moving rapidly or the scene changes significantly, the corresponding frames can be classified as keyframes.

[0035] Then, local texture features are extracted from the keyframes using a preset convolution kernel set. This set of carefully designed convolution kernels allows different kernels to extract different types of texture features, such as edge features and texture orientation features. Through the convolution operation, rich local texture information is extracted from the keyframes, generating a spatial feature matrix. This spatial feature matrix contains important information such as the texture and shape of objects in the video, which is crucial for identifying the type and severity of emergency events.

[0036] For audio waveform data, the event feature extraction module performs spectrum slicing. The audio spectrum is divided into multiple slices according to a specific frequency range. Within each slice, the frequency domain energy distribution features are extracted. Frequency domain energy distribution features reflect the energy intensity of the audio in different frequency bands. Different types of sounds have different energy distribution characteristics in the frequency domain. For example, a fire alarm has higher energy in certain frequency bands, while the energy distribution of crowd noise is more dispersed. By extracting these frequency domain energy distribution features and constructing frequency domain feature vectors, the characteristics of the audio can be effectively represented.

[0037] Finally, the generated spatial feature matrix and frequency domain feature vectors undergo cross-modal correlation coding. This step fuses the data features of the two different modalities, video and audio, so that they complement each other and more comprehensively describe the characteristics of the emergency event. Through cross-modal correlation coding, a second event feature set is generated, providing richer and more accurate information for subsequent hierarchical assessment.

[0038] Example 2: This embodiment describes in detail the preset data classification rules and the process of the multimodal data acquisition module dividing the multimodal sensor data based on the rules.

[0039] The preset data classification rules include a set of basic data types and an extended set of data types. The basic data type set includes video data identifiers, audio data identifiers, and sensor data identifiers. These identifiers are key to identifying different types of basic data. For example, a video data identifier can be a specific code or tag that distinguishes video data from other types of data. Each data identifier corresponds to a separate data parsing protocol, ensuring that different types of data are correctly parsed and processed according to their respective rules.

[0040] When dividing multimodal sensor data, the multimodal data acquisition module receives raw data packets from the community monitoring network in real time through the communication interface. The communication interface acts as a bridge for data transmission, ensuring that the raw data packets are accurately transmitted to the multimodal data acquisition module.

[0041] After receiving the original data packet, the data identifier in the basic data type set is matched against the original data packet's header tag. The data packet's header tag contains basic information about the data. By matching the data identifier in the header tag, the basic data segments can be accurately separated. For example, if the data identifier in the header tag indicates that the packet contains video data, the corresponding video data segment can be separated from the original data packet.

[0042] The extended fields of the original data packet are traversed based on the data identifiers in the extended data type set. The extended fields store additional data, such as text data and geographic location data. By traversing the extended fields, the extended data segments can be extracted. For example, if the extended data type set contains a text data identifier, the corresponding text data is searched in the extended field and extracted.

[0043] The separated basic data segments and extended data segments are aligned by timestamp. Timestamps record the time when the data was generated. Aligning by timestamp ensures temporal consistency among different types of data, facilitating subsequent analysis and processing. After alignment, the basic data segments are written to the structured data buffer, and the extended data segments are written to the unstructured data buffer. The structured and unstructured data buffers are used to temporarily store different types of data, facilitating subsequent feature extraction and processing.

[0044] Example 3: When the preset event level mapping table uses a linear grading model, the emergency event level is the linear mapping result of the weighted fusion values ​​of the first and second event feature sets. When performing weighted fusion, it is necessary to assign appropriate weights to the different feature sets based on their importance. For example, if the first event feature set (such as the temporal features reflected by sensor data) is more critical for determining the event level in certain emergency events, it can be assigned a higher weight. Conversely, if the second event feature set (such as the spatial features reflected by video and audio data) is more important, it can be assigned a higher weight. By appropriately assigning weights, the two feature sets are fused to obtain a weighted fusion value.

[0045] This weighted fusion value is then matched to a preset event level through linear mapping. Linear mapping can be achieved by setting specific mapping rules, for example, assigning different emergency event levels to the weighted fusion value range. If the weighted fusion value falls within a specific range, it is mapped to the corresponding emergency event level, such as low, medium, or high.

[0046] When the preset event level mapping table uses a nonlinear classification model, the emergency event level is a set of discrete labels categorized by the nonlinear transformation results of the first and second event feature sets using an activation function. The activation function performs a nonlinear transformation on the input feature set, enabling the model to learn more complex feature relationships. Common activation functions include the Sigmoid function and the ReLU function.

[0047] First, the first and second event feature sets are input into a nonlinear classification model. An activation function is then applied to the feature sets, resulting in a nonlinear transformation. The transformed results are then classified using a classification algorithm to produce a discrete label set. This discrete label set represents the corresponding emergency level. For example, a discrete label set might include labels such as "minor incident," "normal incident," and "serious incident," each representing a different emergency level. Through the model's classification operation, the current event is accurately classified into the corresponding level.

[0048] Embodiment 4: This embodiment describes in detail the working process of the resource scheduling module, including the specific steps of connecting to the community emergency resource database and generating a device deployment topology map.

[0049] The resource scheduling module is connected to the central processing unit and establishes a connection with the community emergency resource database through a communication interface. The community emergency resource database stores a large amount of information related to emergency equipment, including the type, quantity, location, status, etc. of the equipment. After receiving the final response instruction issued by the central processing unit, the resource scheduling module begins to match the list of available equipment from the community emergency resource database according to the equipment call requirements in the instruction. The resource scheduling module will filter and match the equipment information in the database. If the final response instruction requires the call of fire-fighting equipment, the resource scheduling module will search for all available fire-fighting equipment in the database, including fire extinguishers, fire hydrants, fire trucks, etc., and organize the qualified equipment information into a list of available equipment.

[0050] When generating the device deployment topology, the resource scheduling module first obtains a 3D community map model. This model intuitively displays the community's geographic layout, building locations, and other information. The real-time location coordinates of each device in the available device list are annotated on the 3D map model, allowing users to clearly see each device's specific location within the community.

[0051] Next, the shortest feasible path from the real-time location of each device to the target emergency area is calculated based on the path planning algorithm. The Dijkstra algorithm is used here. Its core idea is to start from the starting point and continuously expand outward to find the shortest path. , where V represents the set of all vertices in the graph, i.e., the locations in the community map; E represents the set of all edges in the graph, i.e., the roads connecting the locations. Let s be the starting vertex (device location), t be the target vertex (target emergency area location), and d(V) represent the shortest distance from the starting vertex s to the vertex V. Initially, d(s) = 0. For other vertices, , The iterative formula of the algorithm is:

[0052] in, =W(V,u) represents the weight of this edge, which is the distance or cost from V to u. By continuously updating the value of d(u), we can eventually find the shortest distance from S to t, which is the shortest feasible path.

[0053] After calculating the shortest path, the list of available devices is prioritized according to the path length. Devices with shorter paths have higher priority because they can reach the target emergency area faster and improve the efficiency of emergency response. represents the path length from the i-th device to the target emergency area, according to Sort the devices in ascending order. The sorting formula can be simply expressed as:

[0054] Where n is the number of available devices, Indicates the path length of the first device (with the shortest path) after sorting, Indicates the path length to the second device, and so on.

[0055] Finally, the calculated shortest feasible path and priority ranking are overlaid onto the 3D map model to generate a dynamically updated device deployment topology. This device deployment topology provides real-time information on device locations, paths to target emergency areas, and device priorities. During emergency response, personnel can use this topology to quickly understand device distribution and scheduling, rationally arrange device usage, and improve the efficiency of emergency resource allocation.

[0056] Example 5: The central processing unit (CPU) uses a dual verification mechanism, combining redundancy check rules and conflict detection rules, to logically verify the evaluation and execution parameters. Redundancy check rules are used to confirm parameter integrity. The evaluation and execution parameters contain a wealth of information, such as the emergency level and the specifics of the response plan. Using these redundancy check rules, the CPU verifies that these parameters contain the necessary information and whether any missing or erroneous data exists. If a parameter is found to be missing critical information, the CPU will issue a warning or take appropriate measures to supplement it.

[0057] Conflict detection rules are used to eliminate logical inconsistencies between parameters. Logical conflicts may exist between evaluation and execution parameters. For example, a low-level emergency event might be accompanied by a response plan that utilizes a significant number of resources for a high-level event. This is clearly illogical. The central processing unit uses conflict detection rules to examine and analyze the logical relationships between parameters. If a conflict is detected, the parameters are adjusted or regenerated to ensure logical consistency.

[0058] The command feedback module is connected to the central processing unit (CPU). Its primary function is to convert the final response command generated by the CPU into a device control signal. The final response command is an abstract command that must be converted into a specific device control signal in order to be recognized and executed by the target emergency device. The command feedback module converts the final response command based on the communication protocol and control requirements of each device. If the final response command requires the activation of a fire alarm device, the command feedback module converts it into a control signal that complies with the fire alarm device's communication protocol.

[0059] The converted device control signal is sent to the target emergency device via the communication interface, triggering a response action. The communication interface ensures that the device control signal is accurately transmitted to the target emergency device. Upon receiving the device control signal, the target emergency device will perform the corresponding action specified by the signal, such as activating an alarm or spraying water to extinguish the fire.

[0060] The data archiving module, also connected to the central processing unit, is responsible for storing multimodal sensor data, the primary event feature set, the secondary event feature set, the emergency event level, and the final response instructions. This data is a crucial record of the entire emergency response process and is crucial for subsequent analysis and summary. The data archiving module generates a complete chain of emergency event records based on the event timeline, organizing and storing data related to each emergency event in chronological order. This allows convenient access to relevant data from the data archiving module when querying and analyzing a specific emergency event, providing strong support for subsequent emergency management and decision-making.

[0061] Example 6: Community environments are complex and diverse, presenting a wide range of potential emergencies, such as fire, theft, and sudden illness. Multimodal data acquisition modules play a crucial role in such environments. In a fire emergency scenario, images of flames and smoke captured by surveillance cameras constitute unstructured video data, while environmental data collected in real time by smoke and temperature sensors constitutes structured data. The multimodal data acquisition module rapidly categorizes this data into structured and unstructured data streams according to pre-set data classification rules. For video data, the module accurately separates video data segments by matching the raw packet header tags based on the video data identifiers in the basic data type set. Sensor data is similarly separated based on the corresponding identifiers. Furthermore, if someone at the scene sends a voice distress message via community communication equipment, the audio data is also collected and classified. If relevant records exist for text data identifiers in the extended data type set, such as brief descriptions of the incident entered by community staff, these data are also accurately extracted. These data are aligned by timestamp and stored in the structured and unstructured data buffers, respectively, providing timely and accurate data support for subsequent processing.

[0062] After receiving this data, the event feature extraction module performs temporal feature analysis on the structured sensor data. The smoke sensor data is analyzed over time. If the smoke concentration rises sharply within a short period of time, this trend becomes an important temporal feature. If the temperature sensor data continues to rise, it will also be included in the first event feature set. Unstructured video data is divided into keyframe groups and non-keyframe groups. Frames with noticeable flame flickering and smoke diffusion are identified as keyframes. Local texture features are extracted from these keyframes using a preset convolution kernel group. Information such as the edge contours of the flame and the diffusion texture of the smoke is converted into a spatial feature matrix. The audio data is processed through spectral slicing, and the energy distribution characteristics of the fire alarm sound in specific frequency bands are extracted to construct a frequency domain feature vector. This is then cross-modally correlated with the spatial feature matrix to generate the second event feature set. These feature sets comprehensively and meticulously characterize the characteristics of the fire event.

[0063] After receiving the first and second event feature sets, the hierarchical assessment module performs an evaluation based on a preset event level mapping table. If a linear hierarchical model is used, it combines features such as the flame spread rate (derived from video feature analysis) and the rate of smoke concentration rise (sensor data features), assigning appropriate weights to the different features for weighted fusion. The emergency event level is then determined through a linear mapping. If the fire spreads rapidly and the smoke concentration rises quickly, the weighted fusion value is larger, corresponding to a higher emergency event level; otherwise, it corresponds to a lower level. If a nonlinear hierarchical model is used, an activation function is used to perform a nonlinear transformation on the fused features. A classification algorithm is then used to accurately categorize the event into emergency event levels represented by discrete label sets such as "small fire," "medium fire," and "large fire."

[0064] The response decision module uses the emergency event level assigned by the hierarchical assessment module to call a target response plan from a pre-set response strategy library. If the fire is assessed as a "minor fire," the response plan might include notifying nearby security personnel to bring fire extinguishers to handle the situation. If it is a "medium fire," some features of the community firefighting system are activated, such as localized sprinkler systems, and specialized firefighting teams are notified. In the case of a "major fire," the firefighting system is fully activated, surrounding residents are evacuated, and fire trucks are directed to quickly enter the community for rescue operations.

[0065] The central processing unit plays a central role in the entire process. It accurately transmits multimodal sensor data to the event feature extraction module, sends the generated feature set to the hierarchical assessment module, and performs rigorous logical verification of the assessment and execution parameters. In one fire simulation scenario, the redundancy check rule discovered that the response plan omitted a step to notify surrounding residents to evacuate, prompting a prompt to supplement it. The conflict detection rule detected a mismatch between the number of firefighting resources called and the fire level and made adjustments to ensure parameter integrity and logical consistency, ultimately generating an accurate final response instruction.

[0066] The resource scheduling module works according to the equipment call requirements in the final response instruction. In a fire scenario, the available firefighting equipment list, such as fire extinguishers, fire hoses, fire trucks, etc., is matched from the community emergency resource database. After obtaining the community's three-dimensional map model, the real-time location coordinates of these devices are marked, and the path planning algorithm is used to calculate the shortest feasible path from the equipment location to the fire area. If there are multiple fire trucks, they are prioritized according to the length of the path, and the fire truck with the shortest path is dispatched first. The shortest feasible path and priority sorting information are superimposed on the three-dimensional map model to generate a dynamically updated equipment deployment topology map, which facilitates the commander to reasonably dispatch resources and improve rescue efficiency.

[0067] The command feedback module converts the final response command into a device control signal and sends it to the target emergency equipment through the communication interface. At the fire scene, the control signal triggers operations such as the sounding of the fire alarm and the activation of the fire sprinkler system. The data archiving module fully records the multimodal sensor data of the entire fire emergency event process, the feature sets generated at each stage, the determined emergency event level, and the final response command, and generates a complete emergency event record chain according to the event timeline, providing a detailed data foundation for subsequent review and summary of experience and optimization of the system. In the theft emergency scenario, the various modules of the system also work closely together. Video surveillance data, door and window sensor data, etc. are collected, feature extracted, graded and evaluated, and ultimately achieve accurate response and effective processing, fully demonstrating the system's adaptability and efficiency in different types of emergency events.

[0068] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "includes," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0069] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A community multimodal emergency incident hierarchical response system, characterized by: include: A multimodal data acquisition module is used to obtain multimodal sensor data within the community and divide the multimodal sensor data into structured data streams and unstructured data streams based on preset data classification rules; An event feature extraction module, configured to perform temporal feature analysis on the structured data stream to generate a first event feature set, and perform spatial feature analysis on the unstructured data stream to generate a second event feature set; A hierarchical assessment module is configured to fuse the first event feature set and the second event feature set according to a preset event level mapping table, match them to the corresponding emergency event level, and use the emergency event level as an assessment parameter for the current event; A response decision module is used to call a target response plan in a preset response strategy library based on the emergency event level and use the target response plan as an execution parameter of the current event; A central processing unit is used to send the multimodal sensing data to the event feature extraction module, send the first event feature set and the second event feature set to the hierarchical evaluation module, and perform logical verification on the evaluation parameters and the execution parameters to generate a final response instruction.

2. The hierarchical response system according to claim 1, wherein: The event feature extraction module performs spatial feature analysis on the unstructured data stream, including: Dividing the video frame sequence in the unstructured data stream into a key frame group and a non-key frame group, and performing local texture feature extraction on the key frame group based on a preset convolution kernel group to generate a spatial feature matrix; Performing spectrum slicing processing on the audio waveform data in the unstructured data stream, extracting frequency domain energy distribution features within each slice and constructing a frequency domain feature vector; The spatial feature matrix and the frequency domain feature vector are cross-modal associated encoded to generate the second event feature set.

3. The hierarchical response system according to claim 1, wherein: The preset data classification rules include a basic data type set and an extended data type set; the basic data type set includes video data identifiers, audio data identifiers and sensor data identifiers; the extended data type set includes text data identifiers and geographic location data identifiers, and each data identifier corresponds to an independent data parsing protocol.

4. The hierarchical response system according to claim 3, wherein: The system further includes a communication interface, which is used to realize communication connection between the multimodal data acquisition module, the event feature extraction module, the hierarchical assessment module and the response decision module and the community monitoring network respectively; The multimodal data acquisition module classifies the multimodal sensor data based on the preset data classification rule, including: Receiving original data packets from the community monitoring network in real time through the communication interface, and matching header labels of the original data packets according to data identifiers in the basic data type set to separate basic data segments; Traversing the extension field of the original data packet according to the data identifier in the extended data type set to extract the extended data segment; The basic data segment and the extended data segment are aligned according to timestamps and then written into a structured data buffer and an unstructured data buffer respectively.

5. The hierarchical response system according to claim 1, wherein: When the preset event level mapping table adopts a linear grading model, the emergency event level is a linear mapping result of the weighted fusion value of the first event feature set and the second event feature set; When the preset event level mapping table adopts a nonlinear grading model, the emergency event level is a discrete label set that classifies the nonlinear transformation results of the first event feature set and the second event feature set through an activation function.

6. The hierarchical response system according to claim 1, wherein: Also included is a resource scheduling module connected to the central processing unit, the resource scheduling module being connected to a community emergency resource database via the communication interface; The resource scheduling module is used to match the available equipment list from the community emergency resource database according to the equipment call requirements in the final response instruction, and generate an equipment deployment topology diagram to optimize the resource allocation path.

7. The hierarchical response system according to claim 6, wherein: The resource scheduling module generates a device deployment topology diagram including: Obtaining a three-dimensional map model of the community, and marking the real-time location coordinates of each device in the list of available devices in the three-dimensional map model; Calculating the shortest feasible path from the real-time location of each device to the target emergency area based on a path planning algorithm, and prioritizing the list of available devices according to path length; The shortest feasible path and the priority ranking are superimposed on the three-dimensional map model to generate a dynamically updated device deployment topology map.

8. The hierarchical response system according to claim 1, wherein: When the central processing unit performs logic verification on the evaluation parameters and the execution parameters, a dual verification mechanism of redundant verification rules and conflict detection rules is adopted. The redundant verification rules are used to confirm the integrity of the parameters, and the conflict detection rules are used to eliminate logical contradictions between the parameters.

9. The hierarchical response system according to claim 1, wherein: It also includes a command feedback module connected to the central processing unit, which is used to convert the final response command into a device control signal and send the device control signal to the target emergency device through the communication interface to trigger a response action.

10. The hierarchical response system according to claim 1, wherein: It also includes a data archiving module connected to the central processing unit, which is used to store the multimodal sensing data, the first event feature set, the second event feature set, the emergency event level and the final response instruction, and generate a complete emergency event record chain according to the event timeline.

Images