A method, system, medium, and product for analyzing a fire rescue process

Abstract

The application discloses a kind of analysis method, system, medium and product of fire rescue process, belong to data management field, the method is: acquisition in fire rescue process Multi-source rescue data;The multi-source rescue data is time-synchronized and is registered to space, and fusion rescue data is obtained;Utilize the event identification model of preestablished and the fusion rescue data are matched to obtain rescue key event sequence Feature;The rescue key event sequence is filled into preestablished summary template, and summary text is generated;Based on the fusion rescue data, rescue key event sequence and summary text, calculate multidimensional evaluation index, and obtain analysis report. By implementing the application, the problem of fire rescue review analysis difficulty can be solved.

Description

Technical Field

[0001] This invention belongs to the field of data management and relates to an analysis method, system, medium, and product for fire rescue processes. Background Technology

[0002] Firefighting and rescue are high-risk and high-intensity emergency operations. Each rescue operation accumulates valuable practical experience, which plays an important supporting role in the construction of fire brigades and personnel training. With the development of technologies such as the Internet of Things, big data, and artificial intelligence, a technical foundation has been provided for the data recording and analysis of the firefighting and rescue process. Using technical means to digitally manage the rescue process has become a trend in the industry.

[0003] Currently, rescue process records mainly consist of traditional paper records and video recordings. The analysis technology adopts a case management system from the field of emergency management. However, traditional paper records mainly rely on manual completion after the fact, which is prone to missing key information and errors due to being filled in from memory. Although video recordings are objective, they lack structured information, require manual review, and are inefficient. Existing case management systems are mainly data storage tools, lacking intelligent analysis capabilities. They require manual confirmation of key processes and extraction of lessons learned. Post-event summaries mainly rely on the commander's subjective recollection and judgment, lacking systematic analysis methods and quantitative indicators, making it difficult to objectively evaluate the rescue effect. Experiences are mainly in the form of written reports, lacking vivid scene reconstruction and operable knowledge extraction, making it difficult for novice firefighters to learn from them. Summary of the Invention

[0004] This application provides an analysis method, system, medium, and product for fire rescue processes, which can solve the problem of difficulties in fire rescue debriefing analysis.

[0005] To achieve the above objectives, in a first aspect, the present invention provides a method for analyzing the fire rescue process, comprising: Collect multi-source rescue data during the fire and rescue process; The multi-source rescue data is synchronized in time and registered in space to obtain fused rescue data; The key rescue event sequence is obtained by feature matching of the fused rescue data using a preset event recognition model; The sequence of key rescue events is filled into a preset minutes template to generate minutes text; Based on the fused rescue data, key rescue event sequences, and minutes, multi-dimensional evaluation indicators are calculated to obtain an analysis report.

[0006] Compared to existing technologies, the embodiments of this application have the following beneficial effects: Collecting multi-source rescue data enables comprehensive acquisition of heterogeneous information on-site, avoiding information loss and incomplete recording caused by a single data source; merging rescue data is obtained by performing time synchronization and spatial registration calculations on multi-source rescue data, eliminating time deviations and spatial coordinate differences between different devices, and providing a unified spatiotemporal benchmark for subsequent cross-modal data analysis; a preset event recognition model is used to perform feature matching on the merging rescue data to obtain a sequence of key rescue events, realizing the transformation from raw continuous data streams to structured actions, replacing the inefficient mode of manual backtracking; then, the sequence of key rescue events is filled into a preset minutes template to generate minutes text, automatically organizing discrete event points into a coherent natural language narrative. This significantly improves the efficiency and standardization of battlefield record generation. Subsequently, based on the fusion of rescue data, key rescue event sequences, and record text, multi-dimensional evaluation indicators are calculated to obtain an analysis report, achieving a leap from qualitative description to quantitative assessment and ensuring the objectivity and data support of the review conclusions. The above five steps form a complete closed loop of "full-domain perception, spatiotemporal fusion, intelligent identification, automatic narration, and quantitative assessment." Among them, the fusion of multi-source data provides complementary evidence for event identification, and the structured output of event sequences provides direct input for record generation and indicator calculation. This layer-by-layer data flow mechanism solves the core technical problem of the difficulty in fire rescue review analysis caused by data dispersion and unstructuredness in existing technologies, realizing automated and intelligent in-depth review of the entire rescue process.

[0007] In some embodiments of the first aspect of this application, the collection of multi-source rescue data during the fire rescue process includes: The multi-source rescue data is collected through various recording devices; wherein, the multi-source rescue data includes: video data, audio data, location data, and sensor data; the video data includes: drone video data, vehicle-mounted camera video data, law enforcement recorder video data, and surveillance video data; the audio data includes: voice communication data and environmental sound data; the location data includes vehicle and personnel location trajectory data; the sensor data includes: environmental parameter data and personnel and equipment status data.

[0008] Compared to existing technologies, the above embodiments have the following beneficial effects: By collecting multi-source rescue data including video, audio, location, and sensor data through various recording devices, a comprehensive data perception network is constructed. The combined collection of data from drones, vehicles, law enforcement recorders, and surveillance videos achieves complementary coverage of the fire scene panorama and first-person perspective, effectively eliminating blind spots. At the same time, the collection of voice communication data and environmental sound data not only records command instructions but also captures key acoustic features such as explosions and collapses, providing auditory dimension evidence for event judgment. Furthermore, through the collection of vehicle and personnel location trajectory data, the deployment path and movement dynamics of rescue forces are accurately reconstructed, while the acquisition of environmental parameter data and personnel and equipment status data reflects the changes in the fire situation and the life safety status of combat units in real time. This multi-dimensional, fine-grained data collection method ensures the integrity and richness of the original data archive, laying a solid data foundation for subsequent high-precision fusion analysis and event recognition.

[0009] In some embodiments of the first aspect of this application, the step of performing time synchronization and spatial registration on the multi-source rescue data to obtain fused rescue data includes: Add a timestamp to each data record in the multi-source rescue data and perform cross-correlation algorithm alignment to obtain time-synchronized data; Feature point matching and homography transformation are performed on video data from different perspectives in the time synchronization data, and the data from each sensor are mapped to a unified spatial coordinate system to obtain fused rescue data.

[0010] Compared to existing technologies, the above embodiments have the following beneficial effects: By adding timestamps to each data record and aligning it using cross-correlation algorithms, time-synchronized data is obtained. Signal processing principles are used to eliminate clock drift between heterogeneous devices, ensuring strict correspondence of multimodal data on the millisecond-level time axis. Furthermore, feature point matching and homography transformation are performed on video data from different perspectives, achieving geometric correction and spatial stitching of multi-camera images and constructing a unified panoramic visual space. Simultaneously, unified spatial coordinate mapping is performed on sensor data, anchoring dispersed values ​​such as temperature and air pressure to specific physical locations. This dual registration process of time and space eliminates the data silo effect, enabling precise association between actions in video footage and sensor readings at specific locations and voice commands at specific times. This results in highly consistent fused rescue data, significantly improving the accuracy of subsequent cross-modal feature matching and the spatial precision of event localization.

[0011] In some embodiments of the first aspect of this application, the step of using a preset event recognition model to perform feature matching on the fused rescue data to obtain a sequence of key rescue events includes: Target detection and behavior recognition are performed on the video data in the fused rescue data to obtain various visual events; Speech recognition and keyword detection are performed on the audio data in the fused rescue data to obtain each speech event; Threshold comparison and pattern recognition are performed on the sensor data in the fused rescue data to obtain the events of each sensor; The location data in the fused rescue data is used to determine the location and calculate the velocity to obtain each trajectory event; By summarizing the visual events, voice events, sensor events, and trajectory events, a sequence of key rescue events is obtained.

[0012] Compared to existing technologies, the above embodiments have the following beneficial effects: visual events are obtained by performing target detection and behavior recognition on video data, and key tactical action features such as water spraying and demolition are automatically extracted using computer vision technology; simultaneously, speech events are obtained by performing speech recognition and keyword detection on audio data, transforming unstructured speech streams into structured labels containing semantic information; further, sensor events are obtained by performing threshold comparison and pattern recognition on sensor data, and abnormal states such as high temperature warnings and insufficient air pressure are automatically captured through numerical fluctuation patterns; and trajectory events are obtained by performing position judgment and speed calculation on location data, quantifying spatiotemporal behaviors such as personnel entering dangerous areas or vehicles arriving at the scene; finally, various events are summarized to obtain a sequence of key rescue events. This modal parallel extraction and aggregation processing method fully utilizes the complementary advantages of different data sources, avoids the problem of missed detection caused by interference (such as smoke obscuring video and noise interfering with audio) in single-modal recognition, and generates a high-confidence and comprehensive structured event sequence.

[0013] In some embodiments of the first aspect of this application, the step of summarizing the visual events, voice events, sensor events, and trajectory events to obtain a sequence of key rescue events includes: The visual events, voice events, sensor events, and trajectory events are sorted in chronological order to obtain an initial event sequence; According to the preset sequential constraint rules, in the initial event sequence, visual events, voice events, sensor events and trajectory events corresponding to the same event are correlated and time-corrected to obtain the corrected event sequence. Based on the preset context reasoning rules, the corrected event sequence is supplemented with reasoning for missing events to obtain the rescue key event sequence.

[0014] Compared with existing technologies, the above embodiments have the following beneficial effects: by sorting various types of events in chronological order to obtain an initial event sequence, a preliminary temporal logic for event occurrence is established; further, event association and temporal correction are performed according to preset sequential constraint rules, and business logic knowledge such as "water supply must be before fire extinguishing" is used to automatically correct the disordered event sequence caused by sensor delay or recognition error, thereby improving the logical self-consistency of the event sequence; at the same time, based on preset contextual reasoning rules, missing events are inferred and supplemented in the corrected event sequence, and causal associations such as "breaking down the door" and "arriving at the scene" are used to intelligently infer intermediate events that were not directly detected when some sensor data is missing. This post-processing mechanism based on rule constraints and contextual reasoning not only improves the robustness of event recognition, but also effectively fills data blind spots, and finally obtains a logically rigorous and complete sequence of key rescue events.

[0015] In some embodiments of the first aspect of this application, filling the sequence of key rescue events into a preset minutes template to generate minutes text includes: Extract the information of each event from the sequence of key rescue events and fill it into the preset event description templates to obtain the event description structure text; Using a natural language generation model, the event description texts are converted into natural language to obtain a first summary text. The first summary text is then associated with the corresponding video clips, video frames, and audio clips of each event in the fused rescue data to generate a multimedia summary, which serves as the summary text.

[0016] Compared to existing technologies, the above embodiments have the following beneficial effects: By extracting event information and filling it into a preset event description template to obtain structured text, standardized event data is quickly transformed into semi-structured descriptions that conform to human reading habits, ensuring the standardization and uniformity of the minutes content; furthermore, the structured text is transformed into the first minutes text in natural language through a natural language generation model, and the automatic conversion from data to fluent narrative is achieved using deep learning technology, significantly reducing the time cost of manually writing minutes; at the same time, the first minutes text is associated with corresponding video clips, video frames, and audio clips to generate multimedia minutes, establishing a two-way index link between textual narration and original audiovisual evidence, enabling reviewers to instantly retrieve on-site evidence while reading the text. This integrated generation method of text, images, and sound greatly enhances the readability, traceability, and training value of battlefield minutes.

[0017] In some embodiments of the first aspect of this application, the step of calculating multi-dimensional evaluation indicators and obtaining an analysis report based on the fused rescue data, the sequence of key rescue events, and the minutes text includes: Based on the fused rescue data, key rescue event sequences, and minutes, the deviation between the execution time and the preset time for each task event is calculated as a time evaluation indicator; the ratio between the execution time of each rescuer and the total rescue time, as well as the consumption of personnel and equipment, is calculated as a resource evaluation indicator; the rescue rate of trapped personnel, and the rate of decrease in fire temperature and smoke concentration during the firefighting phase are calculated as effectiveness evaluation indicators; the frequency and type of risk events, and the working time of each rescuer in the dangerous area are statistically analyzed as risk evaluation indicators; the time deviation between each decision time point and the occurrence time of the corresponding decision response event, as well as the completion rate of each decision, are extracted as decision evaluation indicators. An analysis report is generated by combining the aforementioned time assessment indicators, resource assessment indicators, effect assessment indicators, risk assessment indicators, and decision assessment indicators.

[0018] Compared to existing technologies, the above embodiments have the following beneficial effects: Time evaluation indicators are obtained by calculating the deviation between task execution time and preset time, quantifying response speed and task delay levels, and intuitively reflecting rescue efficiency; Resource evaluation indicators are obtained by further calculating the proportion of personnel working time and equipment consumption, revealing the balance of manpower load and the efficiency of material use, providing data support for optimizing resource allocation; Effectiveness evaluation indicators are obtained by calculating the rescue rate of trapped personnel and the rate of decrease in fire temperature and smoke concentration, replacing subjective evaluation with the changing trends of objective physical quantities, accurately measuring the actual effectiveness of fire fighting and rescue; Risk evaluation indicators are obtained by statistically analyzing the number of risk events and the working time in dangerous areas, quantifying the exposure of safety hazards in the operational process; Decision evaluation indicators are obtained by extracting decision time deviation and completion rate, assessing the timeliness and execution of command response; Finally, an analysis report is generated by comprehensively considering the five types of indicators. This multi-dimensional quantitative evaluation system breaks down the complex rescue process into calculable mathematical indicators, avoiding the subjective assumptions based on personal experience in traditional debriefings, and achieving a scientific, objective, and comprehensive performance evaluation of the entire rescue process.

[0019] Secondly, the present invention also provides an analysis system for fire rescue processes, comprising: a data acquisition module, a fusion module, an event extraction module, a text generation module, and an analysis module; The data acquisition module is used to collect multi-source rescue data during the fire rescue process; The fusion module is used to perform time synchronization and spatial registration of the multi-source rescue data to obtain fused rescue data; The event extraction module is used to perform feature matching on the fused rescue data using a preset event recognition model to obtain a sequence of key rescue events; The text generation module is used to fill the sequence of key rescue events into a preset minutes template to generate minutes text; The analysis module is used to calculate multi-dimensional evaluation indicators and obtain an analysis report based on the fused rescue data, the sequence of key rescue events, and the minutes text.

[0020] Compared with existing technologies, the above embodiments of this application have the following beneficial effects: Collecting multi-source rescue data enables comprehensive acquisition of heterogeneous information on-site, avoiding information loss and incomplete recording caused by a single data source; fusion rescue data is obtained by performing time synchronization and spatial registration calculations on multi-source rescue data, eliminating time deviations and spatial coordinate differences between different devices, and providing a unified spatiotemporal benchmark for subsequent cross-modal data analysis; a preset event recognition model is used to perform feature matching on the fusion rescue data to obtain a sequence of key rescue events, realizing the transformation from raw continuous data streams to structured actions, replacing the inefficient mode of manual backtracking; then, the sequence of key rescue events is filled into a preset minutes template to generate minutes text, automatically organizing discrete event points into a coherent natural language narrative. This process significantly improves the efficiency and standardization of battlefield record generation. Subsequently, based on the fusion of rescue data, key rescue event sequences, and record text, multi-dimensional evaluation indicators are calculated to obtain an analysis report, achieving a leap from qualitative description to quantitative assessment and ensuring the objectivity and data support of the review conclusions. The above five steps form a complete closed loop of "full-domain perception, spatiotemporal fusion, intelligent identification, automatic narration, and quantitative assessment." Among them, the fusion of multi-source data provides complementary evidence for event identification, and the structured output of event sequences provides direct input for record generation and indicator calculation. This layer-by-layer data flow mechanism solves the core technical problem of the difficulty in fire rescue review analysis caused by data dispersion and unstructuredness in existing technologies, realizing automated and intelligent in-depth review of the entire rescue process.

[0021] Thirdly, the present invention also provides a computer program product, including a computer program or instructions, characterized in that, when the computer program or instructions are executed, they implement the analysis method of any one of the fire rescue processes of the present invention.

[0022] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the analysis method for any of the fire rescue processes of this invention. Attached Figure Description

[0023] Figure 1 This is a flowchart illustrating a method for analyzing a fire rescue process provided in some embodiments of the present invention.

[0024] Figure 2 This is a schematic diagram of the structure of a fire rescue process analysis system provided in some embodiments of the present invention.

[0025] Figure 3This is a schematic diagram of multi-source data acquisition provided in some embodiments of the present invention. Detailed Implementation

[0026] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] Example 1: Please refer to Figure 1 To address the difficulty of post-fire rescue analysis, an embodiment of the present invention provides a method for analyzing the fire rescue process, comprising steps S1 to S5: Step S1: Collect multi-source rescue data during the fire rescue process.

[0028] Furthermore, step S1 can be implemented through the following preferred embodiment, including step S11: S11: Collect the multi-source rescue data through various recording devices; wherein, the multi-source rescue data includes: video data, audio data, location data, and sensor data; the video data includes: drone video data, vehicle-mounted camera video data, law enforcement recorder video data, and surveillance video data; the audio data includes: voice communication data and environmental sound data; the location data includes vehicle and personnel location trajectory data; the sensor data includes: environmental parameter data and personnel and equipment status data.

[0029] In specific implementation, refer to Figure 3 The diagram illustrates a multi-source data acquisition method. This application automatically acquires multi-source data during the rescue process, including video, audio, GPS, sensor data, and communication records, among others. I. Video Data: (1) Drone video: panoramic video of the fire scene; resolution of 1920×1080 or higher; frame rate of 25-30fps; includes GPS coordinates and timestamps; (2) Vehicle-mounted camera: records the driving process and the situation on site. It is a multi-angle camera (front, rear, and side). It is activated when dispatching police and automatically triggers recording. (3) Law enforcement recorder: A first-person perspective camera worn by firefighters to record the on-site operation process; equipped with waterproof, shockproof and high-temperature resistant design; (4) Fixed monitoring: These are surveillance cameras inside and outside the building, which can be accessed through the fire protection network system and provide multi-angle viewing angles; II. Audio Data: (1) Walkie-talkie communication: Record all walkie-talkie conversations, including the speaker ID and timestamp, and automatically transcribe them into text; (2) Telephone recording: Recording the conversations between the command center and the scene, the person reporting the incident, and the person in charge of the unit; (3) On-site ambient sound: On-site sound is collected by law enforcement recorder and can be used to analyze the fire (such as explosion sound, collapse sound, etc.). III. GPS Track Data Collection: (1) Vehicle GPS: Records the driving trajectory of the fire truck, with a sampling frequency of 1Hz, including speed and direction information; (2) Personnel positioning: Record the movement trajectory of firefighters at the fire scene, especially those entering the building, through devices such as mobile phones, walkie-talkies, and positioning tags; IV. Data Acquisition from Various Sensors: (1) An air respirator, including: Gas pressure: Remaining gas volume; Usage time: Entry and exit times; Alarm message: Low voltage alarm; (2) Protective clothing sensors, including: External temperature: Surface temperature of the protective suit; Internal temperature: the body surface temperature of firefighters; Humidity: Humidity inside the protective suit; (3) Fire truck sensors, including: Water pressure and flow rate: parameters of the water supply system; Foam mixing ratio: a parameter for foam fire extinguishing; Vehicle status: Engine and pump status; (4) Environmental sensors: Temperature, smoke concentration and toxic gas concentration; wind direction and speed; building structural stress, etc. (5) System log collection: Command system logs include: alarm reception time, dispatch decision, task allocation, instruction issuance and plan adjustment records, etc. Communication system logs include: communication connection establishment and disconnection, data transmission records, and communication fault records; Equipment usage log includes: equipment issuance and return, equipment malfunctions and repairs, and equipment consumption statistics; The above data was collected automatically and is described below: Triggering mechanism: Data collection is automatically initiated when an alarm is received; video recording begins automatically when the vehicle starts; and recording is automatically initiated when personnel enter a dangerous area. Background data collection: Data collection is carried out automatically in the background without interfering with the normal work of firefighters, and features low power consumption and high reliability; Real-time transmission: Data is transmitted to the command center in real time, supporting 4G / 5G wireless transmission and using local storage + cloud backup.

[0030] In this preferred embodiment, multi-source rescue data, including video, audio, location, and sensor data, is collected by various recording devices to construct a comprehensive data perception network. The combined collection of data from drones, vehicles, law enforcement recorders, and surveillance videos achieves complementary coverage of the fire scene panorama and first-person perspective, effectively eliminating blind spots. Simultaneously, the collection of voice communication data and environmental sound data not only records command instructions but also captures key acoustic features such as explosions and collapses, providing auditory evidence for event assessment. Furthermore, the collection of vehicle and personnel location trajectory data accurately reconstructs the deployment paths and movement dynamics of rescue forces, while the acquisition of environmental parameter data and personnel and equipment status data reflects real-time changes in the fire situation and the life safety status of combat units. This multi-dimensional, fine-grained data collection method ensures the integrity and richness of the original data archive, laying a solid data foundation for subsequent high-precision fusion analysis and event identification.

[0031] Step S2: Perform time synchronization and spatial registration on the multi-source rescue data to obtain fused rescue data.

[0032] Furthermore, step S2 can be implemented through the following preferred embodiments, including steps S21-S22: S21: Add a timestamp to each data record in the multi-source rescue data and perform cross-correlation algorithm alignment to obtain time-synchronized data; S22: Perform feature point matching and homography transformation on the video data from different perspectives in the time synchronization data, and perform unified spatial coordinate mapping on the data from each sensor to obtain fused rescue data.

[0033] In practical implementation, the fusion includes temporal and spatial alignment fusion, including: (1) Time synchronization of different data sources, including unified time base, all devices use NTP (Network Time Protocol) to synchronize clocks and are calibrated regularly to avoid clock drift; each data record records a timestamp in the format of "year-month-day hour:minute:second:millisecond", and the time zone is unified to the local time zone; Time alignment algorithm: For data with inaccurate timestamps, cross-correlation algorithms are used for alignment, such as aligning video and walkie-talkie recordings using audio signals.

[0034] (2) Spatial registration of data from different perspectives, including coordinate system one, uniformly using WGS84 geographic coordinate system or building local coordinate system; spatial registration of videos from different cameras, using methods such as feature point matching and homography transformation to generate panoramic views or 3D scenes; and in the registration space, sensor positions are marked, recording the installation position of each sensor, and mapping sensor data into space; (3) Integrate multi-source data into a unified model, including a spatiotemporal data model, with time and space as indexes, and associate each spatiotemporal point with multiple types of data (video frames, sensor readings, personnel locations, etc.), and establish the relationship between different data, such as a video segment corresponding to a walkie-talkie call; Sensor data is stored using time-series databases (such as InfluxDB), video files are stored using object storage (such as MinIO), and metadata and relationships are stored using relational databases.

[0035] In this preferred embodiment, time-synchronized data is obtained by adding timestamps to each data record and aligning it using a cross-correlation algorithm. This eliminates clock drift between heterogeneous devices by utilizing signal processing principles, ensuring strict correspondence of multimodal data on the millisecond-level time axis. Furthermore, feature point matching and homography transformation are performed on video data from different perspectives to achieve geometric correction and spatial stitching of multi-camera images, constructing a unified panoramic visual space. Simultaneously, unified spatial coordinate mapping is performed on the data from each sensor, anchoring scattered values ​​such as temperature and air pressure to specific physical locations. This dual registration process of time and space eliminates the data silo effect, enabling precise association between behaviors in video footage and sensor readings at specific locations and voice commands at specific times. This results in highly consistent fused rescue data, significantly improving the accuracy of subsequent cross-modal feature matching and the spatial precision of event localization.

[0036] Step S3: Use a preset event recognition model to perform feature matching on the fused rescue data to obtain a sequence of key rescue events.

[0037] Furthermore, step S3 can be implemented through the following preferred embodiments, including steps S31-S35: S31: Perform target detection and behavior recognition on the video data in the fused rescue data to obtain various visual events; S32: Perform speech recognition and keyword detection on the audio data in the fused rescue data to obtain each speech event; S33: Perform threshold comparison and pattern recognition on the sensor data in the fused rescue data to obtain the events of each sensor; S34: Perform position determination and velocity calculation on the position data in the fused rescue data to obtain each trajectory event; S35: Summarize the visual events, voice events, sensor events, and trajectory events to obtain the sequence of key rescue events.

[0038] In this preferred embodiment, visual events are obtained by performing target detection and behavior recognition on video data, and key tactical action features such as water spraying and demolition are automatically extracted using computer vision technology. Simultaneously, speech events are obtained by performing speech recognition and keyword detection on audio data, transforming unstructured speech streams into structured tags containing semantic information. Sensor events are further obtained by performing threshold comparison and pattern recognition on sensor data, and abnormal states such as high temperature warnings and insufficient air pressure are automatically captured through numerical fluctuation patterns. Trajectory events are obtained by performing position judgment and speed calculation on location data, quantifying spatiotemporal behaviors such as personnel entering dangerous areas or vehicles arriving at the scene. Finally, the various events are summarized to obtain a sequence of key rescue events. This modal parallel extraction and aggregation processing method fully utilizes the complementary advantages of different data sources, avoids the problem of missed detection caused by interference (such as smoke obscuring video and noise interfering with audio) in single-modal recognition, and generates a high-confidence and comprehensive structured event sequence.

[0039] Furthermore, step S35 can be implemented through the following preferred embodiments, including steps S351-S352: S351: Sort the visual events, voice events, sensor events and trajectory events in chronological order to obtain an initial event sequence; S352: According to the preset sequential constraint rules, in the initial event sequence, visual events, voice events, sensor events and trajectory events corresponding to the same event are correlated and time-series corrected to obtain a corrected event sequence. Based on the preset context reasoning rules, the corrected event sequence is supplemented with reasoning for missing events to obtain the rescue key event sequence.

[0040] During event identification, preset templates can be used to assist in identifying key events, such as: (1) Scheduling events: Fire alarm received: A fire alarm has been received; Police dispatch: It has been decided to deploy a team; Dispatch: Fire trucks leave the fire station; Arrival: Arriving at the fire scene; (2) On-site deployment events: Command vehicle in position: The command vehicle is parked in place; Water supply system established: Water supply begins; Setting up a cordon: Complete the on-site security measures; (3) Firefighting Operation Events: Firefighting begins: Water cannons start spraying water; Breaking down doors and windows to enter a building; Fire control: The fire has been initially brought under control; Extinguishing open flames: The open flames are completely extinguished; (4) Rescue operation events: Trapped personnel located: Confirm the location of the trapped personnel; Begin search and rescue: Enter the building for search and rescue; Rescued Personnel: The trapped personnel were successfully rescued; Medical care: Providing first aid to the wounded; (5) Risk events: The fire suddenly spread: the fire got out of control; Building collapse: The collapse of a building's structure; Personnel injured: Firefighters injured; Equipment failure: failure of critical equipment; (6) Closing events: Begin cleanup: Remove any remaining sources of ignition; Personnel evacuation: Firefighters evacuated the fire scene; Return to base: Return to the fire station; Different identification methods are used for different types of data, such as: (1) Video-based event recognition can be achieved using computer vision technology: For example, when performing target detection, models such as YOLO and Faster R-CNN are used to detect targets such as fire trucks, firefighters, flames, and smoke; When performing behavior recognition: Models such as 3D CNN and Two-Stream network are used to identify the behavior of firefighters (such as running, breaking down, spraying water, carrying, etc.).

[0041] Scene change detection includes detecting changes in fire intensity (expansion, reduction, extinguishing) and changes in buildings (collapse, damage).

[0042] The rules for triggering events are defined, such as: Firefighting action initiated: Water gun spraying action detected; Rescue incident: Firefighters were detected carrying people out of the building; Fire control event: The area of ​​the flames continues to decrease.

[0043] (2) Audio-based event recognition: Speech recognition, which transcribes walkie-talkie conversations into text, can use ASR (Automatic Speech Recognition) technology, including keyword detection, such as detecting keywords or phrases like "start attack," "found trapped personnel," and "request reinforcements." It can also identify the speaker's identity (commander, squad leader, etc.) through pre-trained speaker models and recognize environmental sounds such as explosions, collapses, and alarms, which can be used as indicators of risk events.

[0044] (4) Sensor-based event recognition, including: Threshold detection, such as temperature exceeding the threshold → high temperature warning event; air pressure below the threshold → low pressure alarm event for breathing apparatus; water pressure drop → abnormal water supply event; Pattern recognition identifies characteristic patterns in sensor data, such as a rapid temperature rise followed by a fire spread. Anomaly detection is used to detect anomalies in sensor data, such as GPS signal loss leading to a missing person event. (5) GPS-based event recognition, analyzing GPS trajectory recognition events, including: Location determination: Vehicle arrives at fire location → Event arrives; Personnel entering the building → Building entry event; Personnel leaving the building → Evacuation event; Speed ​​judgment: Vehicle moving from a stationary position → Deployment event; The event of a vehicle moving from motion to a standstill → arrival; Trajectory Analysis: Analysts tracked the movement of personnel within the building; Identify search and rescue routes and evacuation routes; (6) Multimodal fusion recognition, which improves recognition accuracy by fusing multiple modalities: Evidence Fusion: There may be multiple pieces of evidence for the same event. For example, evidence for the event "firefighting began" includes: Video: Water gun spraying detected; Audio: "Start attack" command heard; Sensor: Water pressure rises; By fusing multiple pieces of evidence, confidence can be increased.

[0045] There are time constraints between events. For example, "water supply system establishment" must be done before "fire extinguishing begins". When identifying events, the identification results need to be corrected in combination with time constraints.

[0046] Contextual reasoning uses contextual information to infer events that are not directly identified. For example, if a "door breaking" behavior is detected and there was a previous "arrival at the scene" event, it is inferred to be a "door breaking and entry" event.

[0047] In this preferred embodiment, an initial event sequence is obtained by sorting various types of events in chronological order, establishing a preliminary temporal logic for event occurrence. Further, event association and timing correction are performed based on preset sequential constraints. Utilizing business logic knowledge such as "water supply must precede firefighting," the system automatically corrects event sequence errors caused by sensor delays or recognition errors, improving the logical consistency of the event sequence. Simultaneously, based on preset contextual reasoning rules, the corrected event sequence is supplemented with inferences for missing events. Using causal relationships such as "breaking down the door" and "arriving at the scene," it intelligently infers intermediate events that were not directly detected, even when some sensor data is missing. This post-processing mechanism based on rule constraints and contextual reasoning not only improves the robustness of event recognition but also effectively fills data blind spots, ultimately resulting in a logically rigorous and complete sequence of critical rescue events.

[0048] Step S4: Fill the sequence of key rescue events into the preset minutes template to generate minutes text.

[0049] Furthermore, step S4 can be implemented through the following preferred embodiments, including steps S41-S42: S41: Extract the information of each event in the sequence of key rescue events and fill it into the preset event description template to obtain the event description structure text; S42: Using a natural language generation model, the event description structure text is converted into natural language to obtain a first summary text. The first summary text is then associated with the corresponding video clips, video frames, and audio clips of each event in the fused rescue data to generate a multimedia summary, which serves as the summary text.

[0050] In practice, the first step in generating the minutes is to construct a timeline, organize the identified events into a timeline, arrange all events in chronological order, cluster events according to their type, cluster events that are close in time and semantically related, and divide the events into hierarchical structures, such as first-level events and second-level events.

[0051] Secondly, the minutes content is generated. Each event type has a description template, such as: "Arrived at the scene at 12:15, the command vehicle parked at the entrance of Zhongheng Shopping Center". Through key information extraction, key information is extracted from multi-source data and filled into the template, such as time, location, personnel, actions, and results.

[0052] Next, natural language processing techniques are used to generate conjunctions and transition sentences from the templated information, forming fluent text.

[0053] The last two steps are multimedia association and formatted output: The associations include video clip association, where each event is associated with a corresponding video clip; image association, where keyframes are extracted from the video as images; audio clip association, where related walkie-talkie conversations are associated; and the generation of data charts, such as graphs of sensor data.

[0054] Formatted output includes: Text reports: such as Word and PDF format text reports; Interactive Timeline: An interactive timeline in web page format, clickable to view details; Video editing: Automatically trim key segments to generate high-quality videos; Presentation: Reporting materials in PPT format.

[0055] In this preferred embodiment, structured text is obtained by extracting event information and filling it into a preset event description template. This quickly transforms standardized event data into a semi-structured description that conforms to human reading habits, ensuring the standardization and uniformity of the minutes content. Furthermore, a natural language generation model is used to transform the structured text into a first minutes text in natural language. Deep learning technology is used to achieve automatic conversion from data to fluent narrative, significantly reducing the time cost of manually writing minutes. At the same time, the first minutes text is associated with corresponding video clips, video frames, and audio clips to generate a multimedia minutes. A two-way index link between the text narrative and the original audiovisual evidence is established, allowing reviewers to instantly access on-site evidence while reading the text. This integrated text, image, and audio generation method greatly enhances the readability, traceability, and training value of battlefield minutes.

[0056] Step S5: Based on the fused rescue data, the sequence of key rescue events, and the minutes text, calculate multi-dimensional evaluation indicators and obtain an analysis report.

[0057] Furthermore, step S5 can be implemented through the following preferred embodiments, including steps S51-S52: S51: Based on the fused rescue data, the sequence of key rescue events, and the minutes, calculate the deviation between the execution time and the preset time for each task event as a time evaluation indicator; calculate the ratio between the execution time of each rescuer and the total rescue time, as well as the consumption of personnel and equipment, as a resource evaluation indicator; calculate the rescue rate of trapped personnel, as well as the rate of decrease in fire temperature and smoke concentration during the firefighting phase, as an effectiveness evaluation indicator; statistically analyze the number and type of risk events, as well as the working time of each rescuer in the dangerous area, as a risk evaluation indicator; extract the time deviation between each decision time point and the occurrence time of the corresponding decision response event, as well as the completion rate of each decision, as a decision evaluation indicator. S52: Generate an analysis report by integrating the time assessment indicators, resource assessment indicators, effect assessment indicators, risk assessment indicators, and decision assessment indicators.

[0058] When conducting a post-mortem analysis, multiple dimensions should be considered, including: (1) Time dimension analysis, which analyzes the time efficiency of the rescue process, including response time analysis, such as the time from receiving the alarm to dispatch, the time from dispatch to arrival and the time from arrival to start the rescue; task execution time analysis, including: the actual execution time of each task vs. the estimated time, and the identification of delayed tasks and their causes; critical path analysis, including the identification of the key task sequence that affects the total time and the analysis of bottlenecks on the critical path; and time comparison analysis, including comparison with historical cases and comparison with standard plans.

[0059] (2) Resource dimension analysis, used to analyze resource utilization efficiency, including: Personnel utilization analysis, such as personnel utilization rate: actual working time / total time; load balancing: variance of workload for each person; skill matching: matching of personnel capabilities with task requirements; Equipment usage analysis: equipment utilization rate, equipment failure rate, and rationality of equipment configuration; Material consumption analysis: the consumption of materials such as water and foam, their matching degree with the scale of the fire, and the efficiency of material use.

[0060] (3) Effectiveness dimension analysis, used to evaluate the effectiveness of the rescue, including: Task completion status: The completion status of each task and the reasons for any unfinished tasks; Rescue success rate: the rate of rescue of trapped personnel, the effectiveness of fire control, and the extent of property damage; Quality rating: A comprehensive score based on multiple indicators, compared with outstanding cases.

[0061] (4) Risk dimension analysis, used to analyze risks during the rescue process, including: Risk event statistics: the types and frequency of risk events, as well as the severity of the risk events; Personnel safety analysis: personnel injury status, working hours in hazardous areas, and implementation of protective measures; Risk response assessment: Whether the risk warning was timely and whether the response measures were effective.

[0062] (5) Decision dimension analysis, used to analyze the quality of command decisions, including: Decision-making timeliness: Response time for critical decisions and the impact of decision delays on rescue efforts; Decision correctness: Whether the decision conforms to the plan and norms, and the evaluation of the decision's effectiveness; Decision adjustments: the number of times and reasons for dynamic adjustments, as well as the timeliness and effectiveness of the adjustments.

[0063] (6) Lessons learned, including: Successful experience identification: Identify aspects that were done well, such as the significant effectiveness of drones in advance reconnaissance; Problem identification: Identify existing problems, such as insufficient pressure in municipal fire hydrants; Root cause analysis: Tracing the root cause of a problem, using methods such as the 5 Whys analysis and fishbone diagrams; Improvement suggestion generation: Propose improvement suggestions for the problem, such as suggesting the addition of a mobile water pump.

[0064] In this preferred embodiment, a time evaluation index is obtained by calculating the deviation between the task execution time and the preset time, which quantifies the response speed and the degree of task delay, and intuitively reflects the rescue efficiency. Further calculation of the proportion of personnel working time and equipment consumption yields a resource evaluation index, revealing the balance of manpower load and the efficiency of material use, providing data support for optimizing resource allocation. Simultaneously, the rescue rate of trapped personnel and the rate of decrease in fire temperature and smoke concentration are calculated to obtain an effectiveness evaluation index, replacing subjective evaluation with the changing trends of objective physical quantities, accurately measuring the actual effectiveness of firefighting and rescue. In addition, the number of risk events and the working time in dangerous areas are statistically analyzed to obtain a risk evaluation index, quantifying the exposure of safety hazards during the operation. The decision-making time deviation and completion rate are extracted to obtain a decision evaluation index, assessing the timeliness and execution of command response. Finally, an analysis report is generated by integrating these five types of indicators. This multi-dimensional quantitative evaluation system breaks down the complex rescue process into calculable mathematical indicators, avoiding the subjective assumptions based on personal experience in traditional debriefings, and achieving a scientific, objective, and comprehensive performance evaluation of the entire rescue process.

[0065] As a supplement, after each debriefing, the case studies are transformed into structured data: (1) Basic information, including: time, location, weather, building type, structure, number of floors, area, fire type, cause of fire and scale of fire, etc.; (2) Rescue information, including: dispatched teams, personnel, vehicles, equipment, task breakdown, personnel allocation, rescue process and key events, etc.; (3) Results information, including: casualties, property damage, rescue time, resource consumption and rescue effect assessment, etc.; (4) Lessons learned, including: successful experiences, existing problems and suggestions for improvement; (5) Multimedia materials, including: videos, pictures, audio, battlefield notes and debriefing reports, etc.

[0066] When storing, the cases are tagged, and multi-dimensional tags are added to each case: Architectural tags: such as high-rise buildings, underground spaces, timber structures, brick-concrete structures, etc.; Fire labels: such as electrical fires, oil fires, chemical fires, forest fires, etc. Rescue tags: such as personnel search and rescue, high-altitude rescue, demolition rescue, chemical and biological treatment, etc. Difficulty level labels: such as easy, medium, complex, very complex; Result labels: such as success, partial success, failure; (3) Intelligent search can be added within the system, supporting multiple search methods: Keyword search: Enter keywords to retrieve relevant cases; Conditional search: Filter by building type, fire type, time range, etc. Similar case retrieval: Input the characteristics of the current fire scene, calculate the similarity with historical cases, and recommend the Top-K most similar cases; Semantic retrieval: Input a natural language description, use NLP technology to understand the semantics, and retrieve semantically relevant cases; (4) Case recommendations, including: Scene-triggered recommendation: Upon receiving an alarm, similar cases are automatically recommended based on the characteristics of the fire scene for commanders to refer to; Learning recommendations: Based on the firefighters' learning history and skill gaps, suitable learning cases are recommended; Recommended Exercises: Based on the exercise plan and training objectives, suitable exercise cases will be recommended.

[0067] The system also allows configuration of training material generation, including: (1) Scene recreation: Transforming real cases into practice scenarios: Architectural scenarios: Reconstructing buildings using 3D modeling technology, or creating virtual buildings using VR / AR technology; Fire situation: Reconstruct the fire development process, smoke diffusion, and temperature distribution; Personnel distribution: Reconstruct the location of trapped personnel and the deployment of firefighters; (2) Character customization: Supports rehearsals for different roles: If a user plays the role of a commander and practices command decisions, the system plays other roles, responding to the commander's instructions and being used to evaluate the quality of the command decisions. For example, if a user plays the role of a squad leader and practices tactical execution, the system provides the fire situation and the squad leader makes tactical decisions. If the user plays the role of a combatant, they will practice specific command operations and use VR equipment to simulate a real combat environment.

[0068] (3) Difficulty adjustment: Adjust the difficulty according to the trainee's level. Simplified mode: Offers more tips and guidance, suitable for beginners; Standard mode: Based on real-life cases, suitable for experienced firefighters; Challenge Mode: Increases unexpected situations and disruptions, suitable for advanced firefighters.

[0069] (4) Intelligent evaluation: Automatically evaluates the effectiveness of the exercise. Decision evaluation includes: comparing the trainees' decisions with real cases, comparing the trainees' decisions with the optimal solution, and pointing out the advantages and disadvantages of the decisions; Operational evaluation includes assessing the standardization and efficiency of operations; Overall Score: An evaluation report is generated based on a comprehensive score of multiple indicators.

[0070] (5) Feedback and improvement, providing personalized feedback: Error pointing out: Identify errors and shortcomings in the drill; Correct demonstration: Play a demonstration of the correct operation; Knowledge push: Push relevant knowledge and skills training; Improvement suggestions: Provide targeted improvement suggestions.

[0071] In summary, compared with existing technologies, the above embodiments of this application have the following beneficial effects: Collecting multi-source rescue data enables comprehensive acquisition of heterogeneous information on-site, avoiding information loss and incomplete recording caused by a single data source; merging rescue data is obtained by performing time synchronization and spatial registration calculations on multi-source rescue data, eliminating time deviations and spatial coordinate differences between different devices, and providing a unified spatiotemporal benchmark for subsequent cross-modal data analysis; using a preset event recognition model to perform feature matching on the merging rescue data yields a sequence of key rescue events, realizing the transformation from raw continuous data streams to structured actions, replacing the inefficient mode of manual backtracking; then, the sequence of key rescue events is filled into a preset minutes template to generate minutes text, automatically organizing discrete event points into coherent natural language. The narrative significantly improves the efficiency and standardization of battlefield record generation. Subsequently, based on the fusion of rescue data, key rescue event sequences, and record text, multi-dimensional evaluation indicators are calculated to obtain an analysis report, achieving a leap from qualitative description to quantitative assessment and ensuring the objectivity and data support of the review conclusions. The above five steps form a complete closed loop of "full-domain perception, spatiotemporal fusion, intelligent identification, automatic narration, and quantitative assessment." Among them, the fusion of multi-source data provides complementary evidence for event identification, and the structured output of event sequences provides direct input for record generation and indicator calculation. This layer-by-layer data flow mechanism solves the core technical problem of the difficulty in fire rescue review analysis caused by data dispersion and unstructuredness in existing technologies, realizing automated and intelligent in-depth review of the entire rescue process.

[0072] Example 2: Please refer to Figure 2Based on the same inventive concept, the present invention discloses an analysis system for fire rescue process, comprising: a data acquisition module M1, a fusion module M2, an event extraction module M3, a text generation module M4, and an analysis module M5; The data acquisition module M1 is used to collect multi-source rescue data during the fire rescue process.

[0073] Furthermore, the data acquisition module M1 includes: a data acquisition unit; The acquisition unit is used to acquire the multi-source rescue data through various recording devices; wherein the multi-source rescue data includes: video data, audio data, location data, and sensor data; the video data includes: drone video data, vehicle-mounted camera video data, law enforcement recorder video data, and surveillance video data; the audio data includes: voice communication data and environmental sound data; the location data includes vehicle and personnel location trajectory data; and the sensor data includes: environmental parameter data and personnel and equipment status data.

[0074] In this preferred embodiment, multi-source rescue data, including video, audio, location, and sensor data, is collected by various recording devices to construct a comprehensive data perception network. The combined collection of data from drones, vehicles, law enforcement recorders, and surveillance videos achieves complementary coverage of the fire scene panorama and first-person perspective, effectively eliminating blind spots. Simultaneously, the collection of voice communication data and environmental sound data not only records command instructions but also captures key acoustic features such as explosions and collapses, providing auditory evidence for event assessment. Furthermore, the collection of vehicle and personnel location trajectory data accurately reconstructs the deployment paths and movement dynamics of rescue forces, while the acquisition of environmental parameter data and personnel and equipment status data reflects real-time changes in the fire situation and the life safety status of combat units. This multi-dimensional, fine-grained data collection method ensures the integrity and richness of the original data archive, laying a solid data foundation for subsequent high-precision fusion analysis and event identification.

[0075] The fusion module M2 is used to perform time synchronization and spatial registration of the multi-source rescue data to obtain fused rescue data.

[0076] Furthermore, the fusion module M2 includes: a time alignment unit and a spatial alignment unit; The time alignment unit is used to add a timestamp to each data record in the multi-source rescue data and perform cross-correlation algorithm alignment to obtain time-synchronized data. The spatial alignment unit is used to perform feature point matching and homography transformation on video data from different perspectives in the time synchronization data, and to perform unified spatial coordinate mapping on the data from each sensor to obtain fused rescue data.

[0077] In this preferred embodiment, time-synchronized data is obtained by adding timestamps to each data record and aligning it using a cross-correlation algorithm. This eliminates clock drift between heterogeneous devices by utilizing signal processing principles, ensuring strict correspondence of multimodal data on the millisecond-level time axis. Furthermore, feature point matching and homography transformation are performed on video data from different perspectives to achieve geometric correction and spatial stitching of multi-camera images, constructing a unified panoramic visual space. Simultaneously, unified spatial coordinate mapping is performed on the data from each sensor, anchoring scattered values ​​such as temperature and air pressure to specific physical locations. This dual registration process of time and space eliminates the data silo effect, enabling precise association between behaviors in video footage and sensor readings at specific locations and voice commands at specific times. This results in highly consistent fused rescue data, significantly improving the accuracy of subsequent cross-modal feature matching and the spatial precision of event localization.

[0078] The event extraction module M3 is used to perform feature matching on the fused rescue data using a preset event recognition model to obtain a sequence of key rescue events.

[0079] Furthermore, the event extraction module M3 includes: a first detection unit, a second detection unit, a third detection unit, a fourth detection unit, and a summarizing unit; The first detection unit is used to perform target detection and behavior recognition on the video data in the fused rescue data to obtain various visual events. The second detection unit is used to perform speech recognition and keyword detection on the audio data in the fused rescue data to obtain each speech event; The third detection unit is used to perform threshold comparison and pattern recognition on the sensor data in the fused rescue data to obtain the events of each sensor. The fourth detection unit is used to determine the location and calculate the speed of the location data in the fused rescue data to obtain each trajectory event; The aggregation unit is used to aggregate the visual events, voice events, sensor events, and trajectory events to obtain a sequence of key rescue events.

[0080] In this preferred embodiment, visual events are obtained by performing target detection and behavior recognition on video data, and key tactical action features such as water spraying and demolition are automatically extracted using computer vision technology. Simultaneously, speech events are obtained by performing speech recognition and keyword detection on audio data, transforming unstructured speech streams into structured tags containing semantic information. Sensor events are further obtained by performing threshold comparison and pattern recognition on sensor data, and abnormal states such as high temperature warnings and insufficient air pressure are automatically captured through numerical fluctuation patterns. Trajectory events are obtained by performing position judgment and speed calculation on location data, quantifying spatiotemporal behaviors such as personnel entering dangerous areas or vehicles arriving at the scene. Finally, the various events are summarized to obtain a sequence of key rescue events. This modal parallel extraction and aggregation processing method fully utilizes the complementary advantages of different data sources, avoids the problem of missed detection caused by interference (such as smoke obscuring video and noise interfering with audio) in single-modal recognition, and generates a high-confidence and comprehensive structured event sequence.

[0081] Furthermore, the summarizing unit includes: a sorting subunit and an associated gap-filling subunit; The sorting subunit is used to sort the visual events, voice events, sensor events and trajectory events in chronological order to obtain an initial event sequence. The associated missing event subunit is used to perform event association and timing correction on visual events, voice events, sensor events and trajectory events corresponding to the same event in the initial event sequence according to preset sequential constraint rules, to obtain a corrected event sequence, and to perform reasoning and supplementation on the corrected event sequence based on preset context reasoning rules to obtain the rescue key event sequence.

[0082] In this preferred embodiment, an initial event sequence is obtained by sorting various types of events in chronological order, establishing a preliminary temporal logic for event occurrence. Further, event association and timing correction are performed based on preset sequential constraints. Utilizing business logic knowledge such as "water supply must precede firefighting," the system automatically corrects event sequence errors caused by sensor delays or recognition errors, improving the logical consistency of the event sequence. Simultaneously, based on preset contextual reasoning rules, the corrected event sequence is supplemented with inferences for missing events. Using causal relationships such as "breaking down the door" and "arriving at the scene," it intelligently infers intermediate events that were not directly detected, even when some sensor data is missing. This post-processing mechanism based on rule constraints and contextual reasoning not only improves the robustness of event recognition but also effectively fills data blind spots, ultimately resulting in a logically rigorous and complete sequence of critical rescue events.

[0083] The text generation module M4 is used to fill the sequence of key rescue events into a preset minutes template to generate minutes text.

[0084] Furthermore, the text generation module M4 includes: a filling unit and a post-processing unit; The filling unit is used to extract information about each event in the sequence of key rescue events and fill it into a preset event description template to obtain the event description structure text. The post-processing unit is used to convert the event description structure text into natural language using a natural language generation model to obtain a first summary text, and associate the first summary text with the corresponding video clips, video frames and audio clips of each event in the fused rescue data to generate a multimedia summary, which serves as the summary text.

[0085] In this preferred embodiment, structured text is obtained by extracting event information and filling it into a preset event description template. This quickly transforms standardized event data into a semi-structured description that conforms to human reading habits, ensuring the standardization and uniformity of the minutes content. Furthermore, a natural language generation model is used to transform the structured text into a first minutes text in natural language. Deep learning technology is used to achieve automatic conversion from data to fluent narrative, significantly reducing the time cost of manually writing minutes. At the same time, the first minutes text is associated with corresponding video clips, video frames, and audio clips to generate a multimedia minutes. A two-way index link between the text narrative and the original audiovisual evidence is established, allowing reviewers to instantly access on-site evidence while reading the text. This integrated text, image, and audio generation method greatly enhances the readability, traceability, and training value of battlefield minutes.

[0086] The analysis module M5 is used to calculate multi-dimensional evaluation indicators and obtain an analysis report based on the fused rescue data, the sequence of key rescue events, and the minutes text.

[0087] Furthermore, the analysis module M5 includes: an indicator calculation unit and an analysis output unit; The indicator calculation unit is used to calculate the deviation between the execution time of each task event and the preset time based on the fused rescue data, the sequence of key rescue events, and the minutes text, as a time evaluation indicator; calculate the ratio between the execution time of each rescuer and the total rescue time, as well as the consumption of personnel and equipment, as a resource evaluation indicator; calculate the rescue rate of trapped personnel, as well as the rate of decrease in fire temperature and smoke concentration during the firefighting phase, as an effectiveness evaluation indicator; statistically analyze the number and type of risk events, as well as the working time of each rescuer in the dangerous area, as a risk evaluation indicator; and extract the time deviation between each decision time point and the occurrence time point of the corresponding decision response event, as well as the completion rate of each decision, as a decision evaluation indicator. The analysis output unit is used to integrate the time assessment indicators, resource assessment indicators, effect assessment indicators, risk assessment indicators, and decision assessment indicators to generate an analysis report.

[0088] In this preferred embodiment, a time evaluation index is obtained by calculating the deviation between the task execution time and the preset time, which quantifies the response speed and the degree of task delay, and intuitively reflects the rescue efficiency. Further calculation of the proportion of personnel working time and equipment consumption yields a resource evaluation index, revealing the balance of manpower load and the efficiency of material use, providing data support for optimizing resource allocation. Simultaneously, the rescue rate of trapped personnel and the rate of decrease in fire temperature and smoke concentration are calculated to obtain an effectiveness evaluation index, replacing subjective evaluation with the changing trends of objective physical quantities, accurately measuring the actual effectiveness of firefighting and rescue. In addition, the number of risk events and the working time in dangerous areas are statistically analyzed to obtain a risk evaluation index, quantifying the exposure of safety hazards during the operation. The decision-making time deviation and completion rate are extracted to obtain a decision evaluation index, assessing the timeliness and execution of command response. Finally, an analysis report is generated by integrating these five types of indicators. This multi-dimensional quantitative evaluation system breaks down the complex rescue process into calculable mathematical indicators, avoiding the subjective assumptions based on personal experience in traditional debriefings, and achieving a scientific, objective, and comprehensive performance evaluation of the entire rescue process.

[0089] In summary, compared with existing technologies, the embodiments of this application have the following beneficial effects: Collecting multi-source rescue data enables comprehensive acquisition of heterogeneous information at the scene, avoiding information loss and incomplete recording caused by a single data source; merging rescue data is obtained by performing time synchronization and spatial registration calculations on multi-source rescue data, eliminating time deviations and spatial coordinate differences between different devices, and providing a unified spatiotemporal benchmark for subsequent cross-modal data analysis; using a preset event recognition model to perform feature matching on the merging rescue data yields a sequence of key rescue events, realizing the transformation from raw continuous data streams to structured actions, replacing the inefficient mode of manual backtracking; then, the sequence of key rescue events is filled into a preset minutes template to generate minutes text, automatically organizing discrete event points into coherent natural language. The narrative significantly improves the efficiency and standardization of battlefield record generation. Subsequently, based on the fusion of rescue data, key rescue event sequences, and record text, multi-dimensional evaluation indicators are calculated to obtain an analysis report, achieving a leap from qualitative description to quantitative assessment and ensuring the objectivity and data support of the review conclusions. The above five steps form a complete closed loop of "full-domain perception, spatiotemporal fusion, intelligent identification, automatic narration, and quantitative assessment." Among them, the fusion of multi-source data provides complementary evidence for event identification, and the structured output of event sequences provides direct input for record generation and indicator calculation. This layer-by-layer data flow mechanism solves the core technical problem of the difficulty in fire rescue review analysis caused by data dispersion and unstructuredness in existing technologies, realizing automated and intelligent in-depth review of the entire rescue process.

[0090] Example 3: This invention also provides a computer program product, including a computer program or instructions, capable of running on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, it causes the at least one computing device to execute the analysis method for any of the fire rescue processes of this invention.

[0091] Example 4: This invention also provides a computer-readable storage medium storing at least one executable instruction that, when executed on a fire rescue process analysis system, causes the fire rescue process analysis system to perform one of the fire rescue process analysis methods described in any of the above method embodiments.

[0092] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this application may be practiced without these specific details. Similarly, for the purpose of simplification and aiding understanding of one or more aspects of the invention, in the above description of exemplary embodiments of this application, various features of the embodiments are sometimes grouped together in a single embodiment, figure, or description thereof. The claims, which follow the detailed description, are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of this application.

[0093] Those skilled in the art will understand that the modules in the system of the embodiments can be adaptively changed and placed in one or more systems different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components, except that at least some of such features and / or processes or units are mutually exclusive.

Claims

1. A method for analyzing the fire rescue process, characterized in that, include: Collect multi-source rescue data during the fire and rescue process; The multi-source rescue data is synchronized in time and registered in space to obtain fused rescue data; The key rescue event sequence is obtained by feature matching of the fused rescue data using a preset event recognition model; The sequence of key rescue events is filled into a preset minutes template to generate minutes text; Based on the fused rescue data, key rescue event sequences, and minutes, multi-dimensional evaluation indicators are calculated to obtain an analysis report.

2. The method for analyzing a fire rescue process as described in claim 1, characterized in that, The collection of multi-source rescue data during the fire rescue process includes: The multi-source rescue data is collected through various recording devices; wherein, the multi-source rescue data includes: video data, audio data, location data, and sensor data; the video data includes: drone video data, vehicle-mounted camera video data, law enforcement recorder video data, and surveillance video data; the audio data includes: voice communication data and environmental sound data; the location data includes vehicle and personnel location trajectory data; the sensor data includes: environmental parameter data and personnel and equipment status data.

3. The method for analyzing a fire rescue process as described in claim 2, characterized in that, The process of synchronizing and spatially registering the multi-source rescue data to obtain fused rescue data includes: Add a timestamp to each data record in the multi-source rescue data and perform cross-correlation algorithm alignment to obtain time-synchronized data; Feature point matching and homography transformation are performed on video data from different perspectives in the time synchronization data, and the data from each sensor are mapped to a unified spatial coordinate system to obtain fused rescue data.

4. The method for analyzing a fire rescue process as described in claim 2, characterized in that, The step of using a preset event recognition model to perform feature matching on the fused rescue data to obtain a sequence of key rescue events includes: Target detection and behavior recognition are performed on the video data in the fused rescue data to obtain various visual events; Speech recognition and keyword detection are performed on the audio data in the fused rescue data to obtain each speech event; Threshold comparison and pattern recognition are performed on the sensor data in the fused rescue data to obtain the events of each sensor; The location data in the fused rescue data is used to determine the location and calculate the velocity to obtain each trajectory event; By summarizing the visual events, voice events, sensor events, and trajectory events, a sequence of key rescue events is obtained.

5. The method for analyzing a fire rescue process as described in claim 4, characterized in that, The summarization of the aforementioned visual events, voice events, sensor events, and trajectory events yields a sequence of key rescue events, including: The visual events, voice events, sensor events, and trajectory events are sorted in chronological order to obtain an initial event sequence; According to the preset sequential constraint rules, in the initial event sequence, visual events, voice events, sensor events and trajectory events corresponding to the same event are correlated and time-corrected to obtain the corrected event sequence. Based on the preset context reasoning rules, the corrected event sequence is supplemented with reasoning for missing events to obtain the rescue key event sequence.

6. The method for analyzing a fire rescue process as described in claim 1, characterized in that, The step of filling the sequence of key rescue events into a preset minutes template to generate minutes text includes: Extract the information of each event from the sequence of key rescue events and fill it into the preset event description templates to obtain the event description structure text; Using a natural language generation model, the event description texts are converted into natural language to obtain a first summary text. The first summary text is then associated with the corresponding video clips, video frames, and audio clips of each event in the fused rescue data to generate a multimedia summary, which serves as the summary text.

7. The method for analyzing a fire rescue process as described in claim 1, characterized in that, Based on the fused rescue data, key rescue event sequences, and minutes, multi-dimensional evaluation indicators are calculated to obtain an analysis report, including: Based on the fused rescue data, key rescue event sequences, and minutes, the deviation between the execution time and the preset time for each task event is calculated as a time evaluation indicator; the ratio between the execution time of each rescuer and the total rescue time, as well as the consumption of personnel and equipment, is calculated as a resource evaluation indicator; the rescue rate of trapped personnel, and the rate of decrease in fire temperature and smoke concentration during the firefighting phase are calculated as effectiveness evaluation indicators; the frequency and type of risk events, and the working time of each rescuer in the dangerous area are statistically analyzed as risk evaluation indicators; the time deviation between each decision time point and the occurrence time of the corresponding decision response event, as well as the completion rate of each decision, are extracted as decision evaluation indicators. An analysis report is generated by combining the aforementioned time assessment indicators, resource assessment indicators, effect assessment indicators, risk assessment indicators, and decision assessment indicators.

8. An analysis system for fire rescue processes, characterized in that, include: Data acquisition module, fusion module, event extraction module, text generation module, and analysis module; The data acquisition module is used to collect multi-source rescue data during the fire rescue process; The fusion module is used to perform time synchronization and spatial registration of the multi-source rescue data to obtain fused rescue data; The event extraction module is used to perform feature matching on the fused rescue data using a preset event recognition model to obtain a sequence of key rescue events. The text generation module is used to fill the sequence of key rescue events into a preset minutes template to generate minutes text; The analysis module is used to calculate multi-dimensional evaluation indicators and obtain an analysis report based on the fused rescue data, the sequence of key rescue events, and the minutes text.

9. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed, they implement the method for analyzing a fire rescue process as described in any one of claims 1-7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements an analysis method for a fire rescue process as described in any one of claims 1-7.

Images