Intelligent smoke heat training monitoring system and method based on multi-modal fusion

Abstract

The invention provides an intelligent smoke heat training monitoring system and method based on multi-modal fusion, and the system comprises a multispectral detection module which is used for outputting infrared, visible light and fusion images in real time, and can penetrate smoke to recognize a fire source and a personnel contour; the position sensing module is used for outputting personnel position coordinates in real time; the high and low temperature monitoring module is used for monitoring the temperature of a key area and superposing data to a video picture; the voice acquisition module comprises a directional microphone array supporting noise reduction and sound source positioning, and is used for acquiring the voice of the trainee and the environment sound; the communication module is used for transmitting audio, video and temperature data to a command center server; and the main control module is used for coordinating and managing the work of each module. According to the system, the safety, scientificity and actual combat level of fire-fighting training are greatly improved, key technical support is provided for danger early warning and tactical decision, and meanwhile, the deployment and operation and maintenance cost of the system is remarkably reduced.

Description

Technical Field

[0001] This invention relates to the field of intelligent fire training equipment technology, specifically to an intelligent smoke and heat training monitoring system and method based on multimodal fusion. Background Technology

[0002] Real smoke and fire training, an indispensable part of modern fire training systems, provides firefighters with a near-realistic combat experience by highly replicating the high temperatures, dense smoke, darkness, and chaos of a fire scene. This training method not only hones firefighters' physical skills but also cultivates their psychological resilience, teamwork, and rapid decision-making abilities in extreme environments.

[0003] Traditional fire training equipment has played a vital role in cultivating basic firefighter skills, but due to inadequate safety monitoring systems, it has led to numerous accidents resulting in firefighter injuries and deaths during training in recent years. These tragedies expose systemic flaws in traditional training models regarding real-time risk monitoring, early warning and response mechanisms, and the management of protective equipment. Analyzing these accident cases can yield valuable lessons and point to pathways for improvement in both technology and management to upgrade training safety. Specifically, traditional fire training equipment (such as simulated combustion devices and smoke and heat training chambers) largely relies on visible light cameras or manual observation to assess training effectiveness. However, fire scenarios contain a wealth of "invisible" critical information—for example, the high-temperature zone of flames, the high-temperature heat source during smoldering, and the heat accumulation within the building structure. This information cannot be directly obtained through visible light but directly impacts fire development and rescue decisions. Furthermore, during training, commanders often struggle to observe the positions of trainees and the fire source in real time due to smoke obscuring their view, relying solely on manual experience, which presents safety hazards and inefficiencies. However, existing monitoring equipment mostly consists of single visible light cameras, which cannot accurately identify targets in smoky environments and lack environmental data and voice interaction capabilities, making it difficult to meet the needs of complex training scenarios. In summary, traditional fire training suffers from the following problems: 1. One of the core objectives of fire training is to familiarize trainees with the fire scene environment in a controlled environment. However, in traditional training, temperature monitoring often relies on point sensors (such as thermocouples), which have problems such as limited coverage and delayed data updates. Once the temperature rises abnormally (such as simulating excessive fuel leading to uncontrolled fire), it may cause safety hazards if it is not detected in time. 2. Defects in the positioning system: The reliability of existing UWB / RFID positioning technology drops sharply in high temperature, high humidity and smoky environments, and the positioning error can reach more than 50cm. 3. Data synchronization issues: Millisecond-level time discrepancies exist in multi-source data such as temperature, video, and location, affecting the timeliness of hazard warnings; 4. The effectiveness evaluation of fire training should be based on objective data rather than subjective experience: In the traditional way, instructors often evaluate the training by observing the actions of the trainees (such as the angle of water gun spray and the timing of demolition), but there is a lack of quantitative basis for key indicators such as "whether the fire is effectively controlled" and "whether the high-temperature danger zone is avoided". Summary of the Invention

[0004] The purpose of this invention is to provide an intelligent smoke and heat training monitoring system and method based on multimodal fusion. This system and method greatly improve the safety, scientific nature, and practicality of fire training, provide key technical support for hazard warning and tactical decision-making, and significantly reduce the deployment and maintenance costs of the system.

[0005] To achieve the above objectives, the present invention provides an intelligent smoke and heat training and monitoring system based on multimodal fusion, the system comprising: The multispectral detection module integrates an infrared thermal imaging camera, a 1080P visible light camera, and an image fusion processor to output infrared, visible light, and fused images in real time, and can penetrate smoke to identify fire sources and personnel outlines. The location sensing module is used to output the personnel's location coordinates in real time; High and low temperature monitoring module: This module has a built-in temperature sensor to monitor the temperature of key areas and overlay the data onto the video screen. The voice acquisition module includes a directional microphone array that supports noise reduction and sound source localization, used to acquire the voice of trainees and ambient sounds; The communication module is used to transmit audio, video, and temperature data to the command center server. The main control module is used to coordinate and manage the work of each module.

[0006] Preferably, the position sensing module is installed on the person's feet and configured to communicate with the main control module via Bluetooth.

[0007] Preferably, the system also includes a protective housing that surrounds all modules, the protective housing being IP67 waterproof, dustproof, and heat-resistant.

[0008] On the other hand, the present invention provides a method for monitoring using the above-mentioned intelligent smoke and heat training monitoring system based on multimodal fusion, the method comprising: Based on the Kriging interpolation algorithm, temperature field reconstruction is performed by fusing point-based and interview temperature data, achieving temperature field reconstruction within the training space at intervals of 0.1m. 3 Temperature modeling of the region; simultaneously, real-time generation of the two-dimensional temperature field distribution of the entire training space and dynamic monitoring of temperature gradient changes; A thermal diffusion prediction model based on a long short-term memory network (LSTM) is used to perform sub-second dynamic risk prediction of temperature changes over the next 30 seconds by integrating multimodal sensor data. High-precision positioning is achieved based on inertial navigation, and a four-level temperature sensing network of "point-line-surface-volume" is constructed to realize the reconstruction of the three-dimensional temperature field to display the personnel's location in real time. At the same time, when the voice module captures abnormal keywords, the control system illuminates the personnel positioning beacon. A quantitative assessment is conducted on the efficiency of fire source location, the effectiveness of fire suppression, the rationality of hazard avoidance, the safety of trainees, and the accuracy and timeliness of intelligent marking of hazardous areas, and an assessment report is generated.

[0009] Preferably, temperature field reconstruction includes: Data is collected, specifically infrared temperature data, using a multispectral detection module thermal imager to obtain the continuous two-dimensional temperature distribution of the training space; simultaneously, fiber optic temperature measurement points are deployed in blind spots and high-temperature sensitive areas to obtain fiber optic point data. Data preprocessing is performed, using moving average or wavelet transform to eliminate environmental noise in infrared data, and the timestamps of infrared and fiber optic data are aligned to synchronize time. A spatial correlation model of the temperature field is established based on the spatial variability function, and interpolation weights are assigned according to the confidence levels of point data and area data to optimize the fusion results; The training space is divided into 0.1m grids. 3 The grid; The Kriging interpolation algorithm is used to generate the temperature value of each grid node and fill in the infrared-uncovered areas. The two-dimensional temperature field is displayed using pseudo-color images or isotherms to achieve visual output; at the same time, thresholds are set to monitor sudden temperature gradient changes for dynamic monitoring.

[0010] Preferably, the temperature field reconstruction also includes recording temperature change curves to analyze the fire spread pattern.

[0011] Preferably, dynamic risk prediction includes: Input multimodal data, including 0.1m data reconstructed using the Kriging interpolation algorithm. 3 Temperature field data in the grid temperature distribution, airflow data obtained by the ultrasonic anemometer, and temperature, humidity, and air pressure data obtained by the sensor; Data preprocessing is performed to align multimodal data in time and use Z-score standardization to eliminate dimensional differences, thereby achieving data normalization; Set a sliding time window, using 10 seconds of historical data as the input window and 30 seconds of prediction as the output window; In the Long Short-Term Memory (LSTM) network, three hidden layers are used to handle spatiotemporal dependencies. The Attention mechanism is used to dynamically focus on high-temperature sensitive areas, and the smoothness of the temperature field is predicted based on the MAE+ gradient penalty term. Real-time prediction and correction are performed. When the measured data arrives, the prediction bias is corrected by Kalman filtering to achieve residual compensation. At the same time, the model parameters are updated incrementally every hour. The system outputs risk predictions, including thermal diffusion simulation based on the coupled calculation of the predicted temperature field and airflow field, marking overheated areas when the temperature in a certain area is too high, and issuing audible and visual alarms when the trainees are deemed to be at risk.

[0012] Preferably, the historical 10-second data of the sliding time window includes 100 frames of temperature field and 100 sets of airflow / environmental data. Meanwhile, in the prediction of the next 30 seconds, a rolling prediction is output every 0.1 seconds.

[0013] Preferably, high-precision positioning means maintaining a positioning accuracy of less than 10cm in the absence of GPS.

[0014] Preferably, the quantitative evaluation of training effectiveness includes: The efficiency of fire source location was evaluated by comparing the time it took for trainees to arrive at the high-temperature area with the actual location of the fire source in the thermal image. Analyze the matching degree between the water gun spray coverage area and the high temperature point to determine whether the fire extinguishing operation is accurate and assess the fire extinguishing effectiveness. Monitor the time trainees spend in high-temperature areas and their movement paths to assess the rationality of their avoidance of heat hazards; The test system was tested to determine whether it could automatically issue an early warning and alert the trainer when it detected that a firefighter was experiencing shortness of breath due to high temperature and the infrared module showed that his body surface temperature had reached 145°C. The test aimed to determine whether the voice module could immediately mark dangerous areas on a thermal image when it recognized shouts or calls for help.

[0015] Through the above technical solution, the intelligent smoke and heat training monitoring system based on multimodal fusion provided by this invention integrates multispectral imaging, distributed temperature sensing, inertial navigation and positioning, and voice analysis equipment. Utilizing temperature field reconstruction technology and employing the Kriging interpolation algorithm to fuse point (fiber optic) and area (infrared array) temperature data, it achieves temperature monitoring every 0.1m within the training space. 3The system models the temperature of the area and generates a two-dimensional temperature field distribution throughout the entire training space in real time, dynamically monitoring temperature gradient changes. Simultaneously, it achieves high-precision, low-latency prediction of temperature diffusion in fire smoke and heat training using an LSTM network, providing crucial technical support for hazard warning and tactical decision-making. This system addresses the core pain points of traditional fire smoke and heat training monitoring equipment in areas such as positioning accuracy, temperature monitoring, data synchronization, and safety protection. While improving the safety, scientific rigor, and practicality of training, it significantly reduces system deployment and maintenance costs, making it particularly suitable for firefighter training monitoring and evaluation in complex environments such as smoke and heat training devices and simulated building fire training facilities.

[0016] Other features and advantages of the embodiments of the present invention will be described in detail in the following detailed description section. Attached Figure Description

[0017] The accompanying drawings are provided to further illustrate embodiments of the present invention and form part of the specification. They are used together with the following detailed description to explain the embodiments of the present invention, but do not constitute a limitation thereof. In the drawings: Figure 1 This is a schematic diagram of the structure of the intelligent smoke and heat training and monitoring system based on multimodal fusion provided by the present invention; Figure 2 This is a flowchart of temperature field reconstruction in the intelligent smoke and heat training and monitoring method based on multimodal fusion provided by the present invention; Figure 3 This is a flowchart of the LSTM thermal diffusion prediction model in the intelligent smoke and heat training and monitoring method based on multimodal fusion provided by the present invention. Figure 4 This is a schematic diagram of the external appearance and installation of an intelligent visual monitoring device according to one embodiment of the present invention; Figure 5 This is a schematic diagram of an intelligent visual layout according to one embodiment of the present invention; Figure 6 This is a schematic diagram of an indoor smoke and fire characteristics training facility according to one embodiment of the present invention. Detailed Implementation

[0018] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the scope of the present invention.

[0019] It should be noted that the acquisition, transmission, storage, use, and processing of data in the technical solution of this application all comply with relevant laws and regulations. In the embodiments of this application, certain existing industry solutions such as software, components, and models may be mentioned. These should be considered exemplary, intended only to illustrate the feasibility of implementing the technical solution of this application, and do not imply that the applicant has already used or necessarily used such solutions.

[0020] See Figure 1 This invention provides an intelligent smoke and heat training monitoring system based on multimodal fusion, the system comprising: The multispectral detection module integrates an infrared thermal imaging camera, a 1080P visible light camera, and an image fusion processor. It is used to output infrared, visible light, and fused images in real time, and can penetrate smoke to identify fire sources (temperature threshold can be set) and personnel outlines. The location sensing module is used to output the personnel's location coordinates in real time; High and low temperature monitoring module: This module has a built-in temperature sensor to monitor the temperature of key areas and overlay the data onto the video screen. The voice acquisition module includes a directional microphone array that supports noise reduction and sound source localization, used to acquire the voice of trainees and ambient sounds; The communication module is used to transmit audio, video and temperature data to the command center server. Specifically, it can be a 4G / 5G module or a Wi-Fi transmission module. The main control module is used to coordinate and manage the work of each module.

[0021] In this embodiment, in order to accurately sense the location of personnel and avoid causing inconvenience to personnel when wearing the device, thereby affecting rescue efficiency, the location sensing module is preferably installed on the personnel's feet and configured to communicate with the main control module via Bluetooth.

[0022] When using this intelligent smoke and heat training and monitoring system based on multimodal fusion, in order to effectively protect it in the fire rescue environment and prevent water, dust and high temperature in the rescue environment from damaging the system, the system preferably also includes a protective shell that wraps all modules. This protective shell is IP67 waterproof, dustproof and high temperature resistant.

[0023] According to the above technical solution, the system's thermal infrared monitoring can visualize the temperature distribution in real time as a thermal image by detecting the infrared energy radiated from the surface of an object (which is positively correlated with temperature). For example, in simulated electrical fire training, thermal infrared can clearly show the localized high temperature (potentially reaching hundreds of degrees Celsius) at the point of electrical short circuit, helping trainees quickly locate the fire source; in building fire training, it can monitor the heat conduction process inside walls and furniture, revealing the potential risk of "no open flame on the surface but high internal temperature." This capture of "invisible" thermal features makes the training more closely resemble the complexity of real fires.

[0024] In practical use, the system employs standardized interfaces and a distributed deployment scheme for its sensors (fiber optic, infrared, IMU, etc.), supporting rapid installation and disassembly (deployment time for a single training chamber < 4 hours). The modular design significantly reduces deployment difficulty. Furthermore, the equipment boasts an IP67 protection rating (high temperature resistance, dustproof and waterproof), adapting to long-term, high-intensity training needs. Additionally, the system incorporates built-in sensor health monitoring modules (such as optical loss detection for fiber optic temperature sensors and zero-bias stability analysis for IMUs), providing early warnings of equipment anomalies (such as fiber optic breakage and IMU drift), preventing data loss due to sensor failures. This results in a more than 40% increase in mean time between failures (MTBF) and a 30% reduction in maintenance costs, achieving self-diagnosis of faults and extended lifespan.

[0025] Furthermore, the present invention also provides a method for monitoring using the above-mentioned intelligent smoke and heat training monitoring system based on multimodal fusion, the method comprising: Based on the Kriging interpolation algorithm, temperature field reconstruction is performed by fusing point-based and interview temperature data, achieving temperature field reconstruction within the training space at intervals of 0.1m. 3 Temperature modeling of the region; simultaneously, real-time generation of the two-dimensional temperature field distribution of the entire training space and dynamic monitoring of temperature gradient changes; A thermal diffusion prediction model based on a long short-term memory network (LSTM) is used to perform sub-second dynamic risk prediction of temperature changes over the next 30 seconds by integrating multimodal sensor data. High-precision positioning is achieved based on inertial navigation (high-precision positioning means maintaining a positioning accuracy of less than 10cm in the absence of GPS), and a four-level temperature sensing network of "point-line-surface-volume" is constructed to realize the reconstruction of the three-dimensional temperature field to display the personnel's location in real time. At the same time, when the voice module captures abnormal keywords (such as "help"), the control system illuminates the personnel positioning beacon. A quantitative assessment is conducted on the efficiency of fire source location, the effectiveness of fire suppression, the rationality of hazard avoidance, the safety of trainees, and the accuracy and timeliness of intelligent marking of hazardous areas, and an assessment report is generated.

[0026] For details, see Figure 2The aforementioned temperature field reconstruction includes: data acquisition; for infrared temperature data, acquiring a continuous two-dimensional temperature distribution in the training space using a multispectral detection module thermal imager; simultaneously, deploying fiber optic temperature measurement points in blind spots and high-temperature sensitive areas to obtain fiber optic point data; data preprocessing, employing moving average or wavelet transform to eliminate environmental noise in the infrared data, and aligning the timestamps of the infrared and fiber optic data for time synchronization (requiring hardware synchronization or software compensation); establishing a spatial correlation model of the temperature field based on the spatial variogram, and allocating interpolation weights according to the confidence levels of point and area data to optimize the fusion result; and mesh generation, dividing the training space into 0.1m grids. 3 The system uses a grid; Kriging interpolation is employed to generate temperature values ​​for each grid node and fill in areas not covered by infrared sensors; the two-dimensional temperature field is displayed using pseudo-color maps or isotherms for visualization; simultaneously, thresholds are set to monitor abrupt temperature gradient changes for dynamic monitoring and alarm functionality. In this way, thermal infrared monitoring leverages the advantages of area-array temperature sensing capabilities, utilizing temperature field reconstruction technology and Kriging interpolation to fuse point (fiber optic) and area (infrared array) temperature data, achieving temperature sensing across every 0.1m of the training space. 3 Temperature modeling of the area. Furthermore, it can generate a two-dimensional temperature field distribution across the entire training area in real time, dynamically monitoring temperature gradient changes. For example, in simulating a warehouse fire, thermal infrared can quickly identify localized high-temperature areas (such as smoldering points inside stacks). If the temperature in a certain area exceeds a preset threshold (such as 600℃), the system can automatically trigger an alarm and activate ventilation and fire suppression systems to prevent damage to the training box structure or burns to personnel. In addition, by recording temperature change curves, the fire spread pattern can be analyzed, providing data support for post-training analysis.

[0027] In this invention, the intelligent smoke and heat training and monitoring method based on multimodal fusion utilizes a thermal diffusion prediction model based on a Long Short-Term Memory (LSTM) network. By fusing multimodal sensor data (temperature field + airflow + environmental parameters), it achieves sub-second-level prediction of temperature changes over the next 30 seconds, with prediction errors controlled within ±5℃. Specifically, as shown... Figure 3 As shown, the above dynamic risk prediction includes: inputting multimodal data, including 0.1m data reconstructed by the Kriging interpolation algorithm. 3Temperature field data from the grid temperature distribution, airflow data from the ultrasonic anemometer, and temperature, humidity, and air pressure data from sensors are used for data preprocessing. Multimodal data are time-aligned and Z-score normalization is applied to eliminate dimensional differences, achieving data normalization. A sliding time window is set, with 10 seconds of historical data as the input window and 30 seconds of prediction as the output window (the historical 10-second data in this sliding time window includes 100 frames of temperature field and 100 sets of airflow / environmental data; simultaneously, a rolling prediction is output every 0.1 seconds in the 30-second prediction). A three-layer hidden layer (256 neurons) is used in the Long Short-Term Memory (LSTM) network. (128 / 64) To handle spatiotemporal dependencies, the Attention mechanism is used to dynamically focus on high-temperature sensitive areas (such as the location of battery cells), and the smoothness of the predicted temperature field is constrained by the MAE+ gradient penalty term. Real-time prediction and correction are performed. When measured data arrives, the prediction bias is corrected by Kalman filtering to achieve residual compensation. At the same time, the model parameters are updated incrementally every hour (GPU acceleration is required). Risk prediction is output, including thermal diffusion simulation based on the coupled calculation of the predicted temperature field and airflow field (such as the Fluent simplified model), marking the over-temperature area when the temperature of a certain area is too high, and sound and light alarms when the trainees are judged to be at risk.

[0028] Meanwhile, to promote the scientific approach to skills enhancement, this implementation method preferably includes the following quantitative assessment methods for training effectiveness: evaluating the efficiency of fire source location by comparing the time trainees arrive at the high-temperature area with the actual location of the fire source in the thermal image; analyzing the matching degree between the water gun spray coverage area and the high-temperature point to determine the accuracy of fire extinguishing operations and assess the targeted nature of fire extinguishing; monitoring the trainees' dwell time and movement path in the high-temperature area to assess the rationality of their avoidance of heat hazards; testing whether the system can automatically issue an early warning when it detects that a firefighter is experiencing shortness of breath due to high temperature and the infrared module shows that their body surface temperature has reached 145°C, thus alerting the trainer; and testing whether the voice module can immediately mark the danger zone in the thermal image when recognizing shouts or calls for help. Furthermore, the above data can be automatically generated into reports by the backend system, helping instructors accurately identify training shortcomings (such as a team member's insufficient sensitivity to smoldering fire sources), develop personalized improvement plans, and achieve a shift from "experience-driven" to "data-driven" approaches.

[0029] The following is a specific implementation method to illustrate the intelligent smoke and heat training monitoring system and method based on multimodal fusion provided by the present invention: First, the hardware is installed. Fiber optic temperature sensors (DTS) are laid along the structural components of the training device. Ultrasonic anemometers and temperature and humidity sensors are installed within the training device. A training intelligent visual monitoring device is fixed in the center of the ceiling inside the training device via a top flange, ensuring full coverage of the field of view. The intelligent visual monitoring device uses a 384×288 infrared focal plane array and visible light multispectral detection technology to assist fire commanders in quickly locating fire points and displaying real-time images in complex and changing fire environments. The detection module transmits the detected heat source information via Wi-Fi network to the command terminal, providing a clear view of the dynamic distribution of the fire. Structurally, with the mounting bracket, it can be installed anywhere on the training container, and is secure and not easily fallen. The viewing angle of the intelligent visual monitoring device can be freely adjusted. The appearance and installation location of the intelligent visual monitoring device are as follows. Figure 4 As shown. Based on Figure 5 and Figure 6 It can be seen that simply placing one intelligent visual monitoring device at each end of the training device, which is 12 meters long, 3 meters wide, and 2 meters high, is sufficient to completely cover the area inside the box.

[0030] Next, configure the system operation process: 1. System initialization: Complete initial alignment of inertial navigation (<5s); Establish a benchmark for temperature sensing networks; 2. Real-time monitoring: Location data is updated every 50ms; The temperature field is refreshed every 100ms; Real-time transmission of infrared images to the command and control center tablet; 3. Dynamic evaluation: By utilizing row temperature field reconstruction technology and employing the Kriging interpolation algorithm to fuse point (fiber optic) and area (infrared array) temperature data, temperature data within a 0.1m radius of the training space can be reconstructed. 3 Temperature modeling of the region; A heat diffusion prediction model based on Long Short-Term Memory (LSTM) network is used to achieve sub-second prediction of temperature changes in the next 30 seconds by fusing multimodal sensor data (temperature field + airflow + environmental parameters). The system provides real-time safety assessments of trainees and issues alerts when a threat to their personal safety is deemed imminent. Real-time assessment of the rationality of tactical actions; 4. Data backup: Real-time recording of data from various sensors, backup to local storage and cloud storage for subsequent learning and tracking, and data support for post-training analysis.

[0031] In summary, the intelligent smoke and heat training monitoring system based on multimodal fusion provided by this invention integrates multispectral imaging, distributed temperature sensing, inertial navigation and positioning, and voice analysis equipment. Utilizing temperature field reconstruction technology and employing the Kriging interpolation algorithm to fuse point (fiber optic) and area (infrared array) temperature data, it achieves precise temperature monitoring every 0.1m within the training space. 3 The system models the temperature of the area and generates a two-dimensional temperature field distribution throughout the entire training space in real time, dynamically monitoring temperature gradient changes. Simultaneously, it achieves high-precision, low-latency prediction of temperature diffusion in fire smoke and heat training using an LSTM network, providing crucial technical support for hazard warning and tactical decision-making. This system addresses the core pain points of traditional fire smoke and heat training monitoring equipment in areas such as positioning accuracy, temperature monitoring, data synchronization, and safety protection. While improving the safety, scientific rigor, and practicality of training, it significantly reduces system deployment and maintenance costs, making it particularly suitable for firefighter training monitoring and evaluation in complex environments such as smoke and heat training devices and simulated building fire training facilities.

[0032] In addition, modern fire training emphasizes "starting from real combat," and in real fires, temperature changes are one of the core factors affecting tactical decisions. For example, in indoor building fires, high-temperature smoke layers can compress downwards, leading to extreme fires; in chemical fires, the differences in combustion temperatures of different substances directly affect the selection of extinguishing agents. Traditional training kits struggle to simulate such complex temperature dynamics, while thermal infrared monitoring can reproduce the temperature evolution process of a real fire scene by pre-setting multi-zone, staged heating programs (such as simulating initial smoldering → mid-stage open flame → late-stage flashover). Trainees need to dynamically adjust tactics (such as evacuation or changing the jet angle) based on changes in temperature distribution in the thermal image (such as a sudden expansion of a high-temperature zone indicating a flashover risk). This "temperature-oriented" training model helps trainees develop an intuitive understanding of the thermal environment of a fire scene and improves their ability to respond in real fire situations.

[0033] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0034] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0035] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0036] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0037] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0038] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0039] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0040] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0041] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A multi-modal fusion-based intelligent smoke heat training monitoring system, characterized in that, The system comprises: A multi-spectral detection module integrating an infrared thermal imaging camera, a 1080P visible light camera, and an image fusion processor, for real-time output of infrared, visible light, and fused images, and capable of penetrating smoke to identify fire sources and personnel contours; A position sensing module for real-time output of personnel position coordinates; A high and low temperature monitoring module with a built-in temperature sensor for monitoring the temperature of critical areas and superimposing data onto video images; A voice collection module comprising a directional microphone array supporting noise reduction and sound source localization, for collecting the voices of training personnel and environmental sounds; A communication module for transmitting audio and video and temperature data to a command center server; A master control module for coordinating and managing the work of each module.

2. The intelligent smoke-heat training monitoring system based on multi-modal fusion according to claim 1, characterized in that, The position sensing module is installed on the personnel's feet and is configured to communicate with the master control module via Bluetooth. 3.The intelligent smoke-heat training monitoring system based on multi-modal fusion according to claim 2, characterized in that, The system further comprises a protective shell wrapped around all modules, which is IP67 waterproof, dustproof, and high-temperature resistant.

4. A method for monitoring using the intelligent smoke-heat training monitoring system based on multi-modal fusion according to any one of claims 1-3, characterized in that, The method comprises: According to the Kriging interpolation algorithm, the point and surface temperature data are fused to reconstruct the temperature field, so as to realize the temperature modeling of every 0.1m 3 area in the training space; meanwhile, the two-dimensional temperature field distribution of the whole training space is generated in real time, and the temperature gradient change is dynamically monitored; A thermal diffusion prediction model based on a long short-term memory network (LSTM) for sub-second dynamic risk prediction of future 30-second temperature changes by fusing multi-modal sensor data; High-precision positioning based on inertial navigation, and construction of a "point-line-surface-body" four-level temperature sensing network for three-dimensional temperature field reconstruction to display personnel positions in real time, while the system is controlled to light up personnel positioning beacons when abnormal keywords are captured by the voice module; Quantitative evaluation of fire source positioning efficiency, fire extinguishing targeting, risk avoidance rationality, safety of training personnel, and accuracy and timeliness of intelligent marking of dangerous areas to generate an evaluation report.

5. The intelligent smoke-heat training monitoring method based on multi-modal fusion according to claim 4, characterized in that, The temperature field reconstruction comprises: Data collection: for infrared temperature data, continuous two-dimensional temperature distribution of the training space is obtained by a thermal imager of the multi-spectral detection module; at the same time, optical fiber temperature points are arranged in dead angles and high-temperature sensitive areas to obtain optical fiber point data; Data preprocessing: environmental noise in the infrared data is eliminated by using moving average or wavelet transform, and the timestamps of the infrared and optical fiber data are aligned to synchronize time; Establishment of a temperature field spatial correlation model based on a spatial variation function, and allocation of interpolation weights based on the confidence of point data and surface data to optimize the fusion result; The training space is divided into 0.1m 3 grid by grid division. Generation of temperature values for each grid node by using the Kriging interpolation algorithm and filling of infrared uncovered areas; Visualization output by displaying the two-dimensional temperature field through pseudo-color maps or isotherms; at the same time, threshold monitoring of temperature gradient mutations is performed for dynamic monitoring.

6. The intelligent smoke heat training monitoring method based on multi-modal fusion according to claim 5, characterized in that, The temperature field reconstruction further comprises recording temperature change curves for analyzing fire spread patterns.

7. The intelligent smoke heat training monitoring method based on multi-modal fusion according to claim 5, characterized in that, The dynamic risk prediction comprises: Input multi-modal data, including 0.1m 3 Temperature field data in the grid temperature distribution, airflow data obtained by an ultrasonic anemometer, and temperature and humidity and air pressure data obtained by a sensor Data preprocessing: time alignment of multi-modal data and elimination of dimensional differences by using Z-score standardization to realize data normalization; Setting of a sliding time window: historical 10-second data is set as the input window, and future 30-second prediction is set as the output window; In the long short-term memory network (LSTM), three hidden layers are used to process the spatio-temporal dependency, the attention mechanism is used to dynamically focus on the high-temperature sensitive area, and the MAE+gradient penalty term is used to constrain the smoothness of the predicted temperature field; Real-time prediction and correction are performed, when the measured data arrives, the prediction deviation is corrected by Kalman filtering to realize residual compensation; at the same time, the model parameters are updated incrementally every hour; Risk prediction is output, including heat diffusion simulation based on the coupling calculation of the predicted temperature field and the airflow field, marking the over-temperature area when the temperature in a certain area is too high, and sound and light alarm when it is judged that the training personnel are at risk.

8. The intelligent smoke heat training monitoring method based on multi-modal fusion according to claim 7, characterized in that, The history of the sliding time window 10-second data includes 100 frames of temperature field and 100 groups of airflow / environmental data, and at the same time, in the future 30-second prediction, the rolling prediction is output every 0.1 second.

9. The intelligent smoke-heat training monitoring method based on multi-modal fusion according to claim 4, characterized in that, The high-precision positioning is to maintain a positioning accuracy of less than 10 cm in a GPS-free environment.

10. The intelligent smoke heat training monitoring method based on multi-modal fusion according to claim 4, characterized in that, The quantitative evaluation of the training effect includes: By comparing the time when the training personnel arrive at the high-temperature area with the actual position of the heat source in the thermal image, the heat source positioning efficiency is evaluated; The matching degree of the water gun spraying coverage area and the high-temperature point is analyzed to determine whether the fire extinguishing operation is accurate and to evaluate the fire extinguishing pertinence; The training personnel's stay time and moving path in the high-temperature area are monitored to evaluate the rationality of their risk avoidance to heat danger; The system tests whether it can give a warning and remind the trainee when it finds that the firefighter is out of breath due to high temperature, and the infrared module shows that the body surface temperature reaches 145℃; The voice module is tested whether it can immediately mark the dangerous area in the thermal image when it recognizes the shouting or distress call.

Images