An emergency fire fighting event identification method and system for industrial enterprises

Abstract

The application relates to the technical field of intelligent fire fighting, in particular to an emergency fire fighting event identification method and system for industrial enterprises. The application proposes the following scheme: visible light, thermal infrared and gas spectrum multi-modal sensing modules are used to collect field sensing data, brightness change, temperature gradient and spectral line intensity features are extracted under a unified space-time reference frame, a multi-modal feature map is constructed and a fire suspicion area is extracted. For the suspected area, a thermal plume field model is constructed based on thermal convection and smoke plume dynamics mechanism, multi-modal prediction trajectories are generated under the assumption of real fire, the prediction trajectories are spatially and temporally registered with the actual observation data, a cross-modal consistency score is calculated to determine the authenticity of the fire. The application can integrate multi-source physical information, realize anti-fact disambiguation of pseudo fire such as welding flash, steam and hot exhaust air, and significantly improve the accuracy and response time of fire identification in complex industrial environments.

Description

Technical Field

[0001] This application relates to the field of smart fire protection technology, and in particular to an emergency fire incident identification method and system for industrial enterprises. Background Technology

[0002] Existing industrial fire monitoring relies heavily on single-modal sensing methods, such as flame detection based on visible light video or anomaly identification based on infrared temperature. These methods are often affected by reflections, thermal radiation, steam atomization, and dust interference in complex industrial scenarios, leading to false alarms or missed alarms. Especially when there are highly similar phenomena such as welding sparks, hot exhaust, or steam venting, the discriminative features of a single modality cannot effectively distinguish between actual combustion and non-combustion phenomena, resulting in frequent false alarms and impacting production continuity and the reliability of emergency decision-making. While some multimodal fusion solutions can integrate information from different sensors, they fail to achieve consistent constraints on the responses of each modality at the physical level. They still suffer from problems such as ambiguous judgment criteria, long response delays, and insufficient reliability, making it difficult to meet the needs of high-risk industrial sites for accurate early fire identification.

[0003] To address the above issues, this application presents a method and system for identifying emergency fire incidents in industrial enterprises. Summary of the Invention

[0004] The technical problem this application aims to solve is to address the shortcomings of existing technologies by providing an emergency fire incident identification method and system for industrial enterprises. This method utilizes multimodal sensing modules (visible light, thermal infrared, and gas spectroscopy) to collect on-site sensing data. Under a unified spatiotemporal reference framework, it extracts brightness variations, temperature gradients, and spectral line intensity features to construct a multimodal feature map and extract suspected fire areas. For suspected areas, a thermal plume flow field model is constructed based on thermal convection and plume dynamics mechanisms. Under the assumption of a true fire, a multimodal predicted trajectory is generated, and the predicted trajectory is spatially and temporally registered with actual observation data. A cross-modal consistency score is calculated to determine the authenticity of the fire.

[0005] To achieve the above objectives, this application provides the following technical solution:

[0006] An emergency fire incident identification method for industrial enterprises is applied to a disaster prevention and monitoring platform. The platform is used to collect and process multimodal sensing data from the industrial enterprise site. The platform includes a visible light sensing module, a thermal infrared sensing module, and a gas spectral sensing module. The method includes:

[0007] Multimodal sensing data from the visible light sensing module, thermal infrared sensing module, and gas spectral sensing module are acquired to extract suspected fire areas;

[0008] The multimodal sensing data corresponding to the suspected fire area is input into a preset thermal plume flow field model to generate a multimodal prediction trajectory under the assumption of a real fire. The thermal plume flow field model is constructed based on the prior mechanism of thermal convection and plume dynamics.

[0009] The cross-modal consistency score of the multimodal sensing data corresponding to the suspected fire area is calculated based on the multimodal prediction trajectory.

[0010] If the cross-modal consistency score is greater than or equal to the preset judgment threshold, the target event is determined to be a real fire and an evidence pair including spatiotemporal thermal plume and spectral alignment results is output; otherwise, it is determined to be a false fire and the corresponding pseudotype label is output.

[0011] The acquisition of multimodal sensing data from the visible light sensing module, the thermal infrared sensing module, and the gas spectral sensing module includes:

[0012] The visible light image sequence acquired by the visible light sensing module, the thermal infrared temperature field sequence acquired by the thermal infrared sensing module, and the gas spectral response data acquired by the gas spectral sensing module are synchronized in time and calibrated in space to establish a unified spatiotemporal coordinate reference framework.

[0013] Within the spatiotemporal coordinate reference frame, feature extraction and joint analysis are performed on the multimodal sensing data to obtain a multimodal feature map that characterizes brightness changes, temperature gradients, and spectral line intensity changes.

[0014] Feature extraction and joint analysis are performed on the multimodal sensing data to obtain multimodal feature maps characterizing brightness changes, temperature gradients, and spectral line intensity changes, including:

[0015] Dynamic texture analysis is performed on visible light image sequences to extract pixel brightness change rate, edge oscillation frequency, and optical flow direction consistency parameters to obtain image modal features;

[0016] The temporal temperature gradient, spatial thermal conductivity, and non-Gaussianity index of local temperature distribution are calculated for the thermal infrared temperature field sequence in order to identify the heat plume region with continuous heating and buoyancy characteristics, and to obtain temperature mode characteristics.

[0017] The positions, half width at half maximum (FWHM), and intensity change rates of absorption and radiation peaks are extracted from the gas spectral response data within the characteristic bandwidth range. Similarity matching is performed with the preset fuel spectral line model to distinguish the characteristic spectral lines of combustion products from the scattering spectral lines of steam or hot gas flow, thereby obtaining the gas modal characteristics.

[0018] A multimodal fusion tensor is constructed based on the features of each modality, and a multimodal feature map is calculated based on the multimodal fusion tensor.

[0019] Calculating the multimodal feature map based on the multimodal fusion tensor includes:

[0020] The multimodal fusion tensor is standardized and dimensionally unified according to the three dimensions of space, time and mode, and a common metric domain with brightness change, temperature gradient and spectral line intensity as the main components is constructed. The main components are then assigned initial weights according to the field environmental parameters.

[0021] Robust tensor decomposition is performed on the common metric domain to obtain a low-rank component representing cross-modal common variation and a sparse component representing instantaneous noise, reflection scintillation and local jet interference, and the low-rank component is used as a candidate uniformity response field.

[0022] Within a preset sliding time window, the intermodal correlation matrix and canonical correlation coefficient of the low-rank components are calculated, and the correlation intensity map and time abrupt change map are obtained by combining the statistics of the change point test.

[0023] Physical feasibility screening is performed on the candidate consistent response fields to obtain a physical consistency response map. The physical feasibility screening includes: retaining voxel regions that simultaneously satisfy thermal convection and plume dynamics, voxel regions whose upward direction is consistent with the direction of visible light motion, voxel regions whose spectral bandwidth intensity continuously increases over time, and voxel regions that pass the mass conservation and energy balance approximation test.

[0024] The physical consistency response map, the correlation intensity map, and the time abrupt change map are weighted and combined to obtain a primary feature map, wherein the weights of the weighted combination are adjusted according to wind speed, temperature, and humidity.

[0025] Spatiotemporal connectivity filtering and morphological constraints are applied to the primary feature map to obtain a multimodal feature map.

[0026] The area suspected of being a fire includes:

[0027] In the multimodal feature map, a comprehensive response intensity map is calculated based on the brightness change rate, temperature gradient amplitude, and spectral line intensity change rate, wherein the comprehensive response intensity map is used to characterize the degree of synergistic enhancement of multimodal features within a spatial region;

[0028] The comprehensive response intensity map is processed by multi-threshold segmentation and region growing algorithm to extract candidate response regions, wherein the multi-threshold segmentation is used to determine the initial range of response intensity, and the region growing algorithm is used to expand the response pixel group of the initial range.

[0029] Temporal consistency detection is performed on the candidate response regions, and the co-correlation coefficients of brightness change direction, temperature rise rate and spectral line change trend are calculated. Regions corresponding to positive correlation of the co-correlation coefficients are retained to obtain temporal consistency regions.

[0030] The time-consistency area is dynamically stabilized by combining on-site environmental parameters, and the suspected fire area is output through differential smoothing.

[0031] The thermal plume flow field model includes an input layer, a hidden layer, and an output layer, wherein:

[0032] The input layer is used to receive multimodal sensing data corresponding to suspected fire areas, establish feature sequences based on input parameters, and calculate the fitting value of the feature sequences.

[0033] The hidden layer, based on the mechanistic priors of thermal convection and plume dynamics, performs coupled calculations on the fitted values. The mechanistic priors include the buoyancy driving equation, the momentum conservation equation, and the energy balance equation. The hidden layer solves for the rising velocity field, temperature distribution field, and plume tip diffusion radius of the thermal plume in the time series through numerical discretization. During the calculation process, it is corrected by the ambient wind speed, pressure difference, and obstacle distribution to obtain the thermal plume flow field parameters.

[0034] The output layer is used to generate a multimodal predicted trajectory under the assumption of a real fire, based on the thermal plume flow field parameters calculated from the hidden layer.

[0035] The generated multimodal predicted trajectory under the assumption of a real fire includes:

[0036] Based on the hot plume flow field parameters, the upward velocity distribution of the hot plume central axis, the plume tip height, and the spatiotemporal evolution curve of temperature are calculated, and the radiation intensity distribution of the hot plume surface is obtained based on the energy conservation relationship.

[0037] The radiation intensity distribution is projected onto the visible light and thermal infrared observation planes, and combined with the camera field of view, distance attenuation coefficient and sensor spectral response characteristics to generate corresponding visible light brightness prediction sequences and infrared radiation brightness prediction sequences.

[0038] Based on the temperature distribution field and fuel type information, the emission intensity of characteristic spectral lines of combustion products and their rate of change over time are calculated using a preset radiative transfer model, resulting in the time response curve of the spectral bandwidth.

[0039] The visible light brightness prediction sequence, infrared radiance prediction sequence, and time response curve of spectral bandwidth are synchronized and registered under a unified spatiotemporal coordinate system to generate a multimodal prediction trajectory.

[0040] The cross-modal consistency score of the multimodal sensing data corresponding to the suspected fire area is calculated based on the multimodal predicted trajectory, including:

[0041] The multimodal predicted trajectory is spatially registered and temporally aligned with the corresponding multimodal sensing data to establish an observation correspondence.

[0042] Based on the observed correspondence, the consistency measurement results of each mode are calculated, wherein the consistency measurement results of each mode include the consistency measurement results of the visible light mode, the consistency measurement results of the thermal infrared mode, and the consistency measurement results of the gas spectral mode;

[0043] The cross-modal consistency score is calculated based on the consistency metric results.

[0044] An emergency fire incident identification system for industrial enterprises, the system comprising:

[0045] The multimodal sensing and acquisition module is used to acquire visible light images, thermal infrared temperature fields, and gas spectral response data at the industrial site, and to perform time synchronization and spatial calibration of the sensing data for each modality.

[0046] The feature fusion and parsing module is used to extract and jointly analyze features from calibrated multimodal sensing data, construct a multimodal fusion tensor, and generate multimodal feature maps to characterize brightness changes, temperature gradients, and spectral line intensity changes.

[0047] The thermal plume flow field modeling module is used to receive multimodal sensing data corresponding to suspected fire areas, calculate thermal plume flow field parameters based on the mechanism prior of thermal convection and plume dynamics, and generate multimodal prediction trajectories under the assumption of a real fire.

[0048] The cross-modal consistency determination module is used to calculate the cross-modal consistency score based on the multimodal predicted trajectory and the actual observation data, determine whether the target event is a real fire or a false fire based on a preset threshold, and output the corresponding evidence pair or pseudo-type label.

[0049] The feature fusion and parsing module includes:

[0050] The modal feature extraction unit is used to perform dynamic texture analysis, temperature gradient calculation and spectral feature extraction on visible light image sequences, thermal infrared temperature field sequences and gas spectral response data, respectively, to obtain brightness change parameters, temperature distribution parameters and spectral intensity change parameters.

[0051] The multimodal tensor construction unit constructs a multimodal fusion tensor based on the brightness variation parameters, temperature distribution parameters, and spectral intensity variation parameters, and performs spatial, temporal, and dimensional unification processing on the multimodal fusion tensor.

[0052] The feature correlation calculation unit is used to calculate the correlation coefficient and change trend between modes based on the multimodal fusion tensor, and generate a multimodal feature map to characterize the cooperative change relationship between brightness, temperature and spectral lines.

[0053] Compared with the prior art, the beneficial effects of this application are:

[0054] This application constructs physically consistent predicted trajectories in multi-source sensing data by using multi-modal consistency determination based on thermal convection and plume dynamics mechanisms, and performs counterfactual comparison accordingly. This fundamentally reduces false fire interference, improves the accuracy and timeliness of fire identification in industrial sites, and provides interpretable and traceable fire determination basis for disaster prevention and monitoring platforms. Attached Figure Description

[0055] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0056] Figure 1 An exemplary application scenario diagram provided for an embodiment of this application;

[0057] Figure 2 A schematic diagram of a processor module provided in an embodiment of this application;

[0058] Figure 3 This is a flowchart illustrating an emergency fire incident identification method for industrial enterprises, provided as an embodiment of this application. Detailed Implementation

[0059] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0060] The term "embodiment" as used herein means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0061] This embodiment does not address an abstract platform solution, but rather a recurring and persistent identification dilemma in industrial settings that is difficult to resolve reliably:

[0062] When welding sparks fall directly into the monitoring field of view, when steam rises along the process pipe corridor, when dust forms bright particles under strong light, and when hot exhaust creates periodic upward airflows in the ceiling corners, the bright-moving-rising pattern presented in the video image and temperature field is extremely similar to the visible characteristics of early flames and plumes. At the same moment, the gas sensing subsystem may only detect slight bandwidth disturbances. Traditional single-modal alarm logic often misjudges this situation; high-sensitivity alarm systems typically experience high false alarm rates, while high-threshold alarm systems may miss detections, thus repeatedly diluting the valuable early detection window.

[0063] Taking a parallel assembly line for welding and inspection as an example, short-duration intense flicker gives video detectors a near-flame edge texture; in the fine chemical and battery electrode drying areas, steam and solvent vapor exhibit an upward plume-like pattern under backlighting; in the crushing, polishing, and spraying sections, the particle size distribution of dust causes a high-density scattering dot pattern in front of the lens; and near smelting, heat treatment, and large air ducts, hot exhaust air forms a stable upward channel in localized areas. These scenarios share a common characteristic:

[0064] While exhibiting high similarity in visible light and thermal infrared modes, they differ fundamentally in two dimensions: the presence of radiation / absorption spectral characteristics necessary for combustion and the spatiotemporal evolution of thermal convection and plume dynamics. It is precisely this apparent convergence coupled with differing mechanisms that makes black-box models relying on statistical correlations vulnerable during cross-workshop migrations.

[0065] The processing logic of this application is based on the aforementioned scenario:

[0066] If an event is indeed a fire, then under the assumption of a real fire, the plume rise velocity, plume tip expansion scale, and temperature-time curve derived from thermal convection and plume dynamics should corroborate the morphology and temperature rise observed in visible light and infrared light. Similarly, if actual combustion is involved, the characteristic bandwidth should exhibit an intensity evolution coordinated with the temperature field within a reasonable time delay. Conversely, if the morphology is merely due to welding sparks, steam, dust, or hot exhaust, this necessary cross-modal consistency will show anomalies in at least one mode.

[0067] It is understandable that this embodiment does not aim for more features or more complex classifications, but rather establishes a closed loop from the perspective of inferable mechanisms and verifiable data:

[0068] First, under the assumption of real fire, based on the simplified mechanism of thermal buoyancy and energy balance, short-term predicted trajectories of multimodal observables are generated. Then, the predictions are aligned and compared with actual observations to form a cross-modal consistency score. Finally, the score is used as the primary basis for determining authenticity, and evidence pairs of spatiotemporal thermal plume alignment maps and spectral line alignment maps are output for relevant personnel to quickly review and archive.

[0069] It is important to note that in the chaotic, crowded, and complex industrial environments with varying light and heat flows, this embodiment follows two principles in terms of application scenarios and deployment methods:

[0070] First, the scene selection does not depend on a fixed device or partition, but is geared towards common working conditions with reflection path constraints and viewpoint occlusion limitations: such as the semi-enclosed field of view formed by pipe corridors and beams and columns, strong reflections caused by light sources, rising channels in the corner area of ​​the ceiling, and heat dissipation through narrow gaps on the back of equipment.

[0071] Secondly, the sensing side maintains minimum completeness: visible light and thermal infrared are used to observe morphological and temperature evolution, and gas spectra are used to observe whether there is a bandwidth response that matches combustion; after time synchronization and external parameter calibration, the three are registered in a unified coordinate framework to ensure that the mechanism prediction quantity and the sensing observation quantity can be compared one-to-one.

[0072] It should be noted that this application does not require complex full-spectrum gas spectral data to be provided on-site. For some workshops, the presence of combustion-related bands can be determined solely by the intensity time series of the critical bandwidth. For locations where gas modes are not yet available for measurement, the application can still operate under the visible-infrared coupling mechanism, and when the uncertainty of the consistency score increases, it can actively extend the observation window or prompt for the acquisition of a third mode. This application can be smoothly deployed in typical workshops such as assembly lines, dust processing, drying and curing, heat treatment, and chemical batching, and can be put into application through unified time base and spatial calibration without changing the existing installation positions of cameras and infrared detectors.

[0073] refer to Figure 1 , Figure 1 This is an exemplary application scenario diagram provided for an embodiment of this application.

[0074] Figure 1 The application scenarios shown include industrial sites, disaster prevention and monitoring platforms, processors, and controllers, among which:

[0075] Industrial sites are used to represent production operation environments with typical multi-source sensing conditions and potential fire risks, such as welding workshops, painting and drying areas, chemical plant areas, or dust processing sections.

[0076] The disaster prevention and monitoring platform is connected to the processor to receive and centrally manage multimodal sensing data from the industrial site. The processor is used to execute the algorithm logic of this application, including multimodal feature extraction, thermal plume flow field modeling, counterfactual trajectory generation, and cross-modal consistency determination, in order to identify the authenticity of fire events.

[0077] The controller generates corresponding emergency control commands based on the processor's output. When a real fire is detected, it triggers an audible and visual alarm, activates or shuts down fire-fighting devices, or reports the incident to the safety management system. When a false fire is detected, it records the event information and maintains the current operating status.

[0078] It is understood that the processor in this application can be understood as an electronic processing unit used to execute the algorithm instructions configured in the disaster prevention and monitoring platform. Specific implementations may include a central processing unit (CPU), graphics processing unit (GPU), digital signal processor (DSP), field-programmable gate array (FPGA), or other embedded processing modules with computing power support, which can implement functions such as parsing and feature extraction of multi-source data through software, firmware, or hardware circuits.

[0079] Similarly, the controller in this application can be understood as an execution control unit used to execute control instructions and coordinate with field devices based on the calculation results of the processor. Specific implementations may include programmable logic controllers (PLCs), industrial control computers (IPCs), distributed control systems (DCS) control modules, smart gateways, or other devices with output control capabilities. These devices can interact with fire protection facilities, sprinkler systems, smoke exhaust systems, valve shut-off devices, and evacuation indicator equipment via fieldbus, Ethernet, or wireless communication for signal exchange and status control.

[0080] It should be noted that the specific form of the processor and controller can be selected and combined according to the configuration and application requirements of the factory disaster prevention system, and this application does not limit it; in different implementations, the processor and controller can be independent hardware units, or they can be integrated into the same industrial control platform or edge computing node, which will not be elaborated here.

[0081] refer to Figure 2 , Figure 2 A schematic diagram of a processor module provided in an embodiment of this application.

[0082] In one example, this application embodiment provides an emergency fire incident identification system for industrial enterprises, the system comprising:

[0083] The multimodal sensing and acquisition module is used to acquire visible light images, thermal infrared temperature fields, and gas spectral response data at the industrial site, and to perform time synchronization and spatial calibration of the sensing data for each modality.

[0084] The feature fusion and parsing module is used to extract and jointly analyze features from calibrated multimodal sensing data, construct a multimodal fusion tensor, and generate multimodal feature maps to characterize brightness changes, temperature gradients, and spectral line intensity changes.

[0085] The thermal plume flow field modeling module is used to receive multimodal sensing data corresponding to suspected fire areas, calculate thermal plume flow field parameters based on the mechanism prior of thermal convection and plume dynamics, and generate multimodal prediction trajectories under the assumption of a real fire.

[0086] The cross-modal consistency determination module is used to calculate the cross-modal consistency score based on the multimodal predicted trajectory and the actual observation data, determine whether the target event is a real fire or a false fire based on a preset threshold, and output the corresponding evidence pair or pseudo-type label.

[0087] The feature fusion and parsing module includes:

[0088] The modal feature extraction unit is used to perform dynamic texture analysis, temperature gradient calculation and spectral feature extraction on visible light image sequences, thermal infrared temperature field sequences and gas spectral response data, respectively, to obtain brightness change parameters, temperature distribution parameters and spectral intensity change parameters.

[0089] The multimodal tensor construction unit constructs a multimodal fusion tensor based on the brightness variation parameters, temperature distribution parameters, and spectral intensity variation parameters, and performs spatial, temporal, and dimensional unification processing on the multimodal fusion tensor.

[0090] The feature correlation calculation unit is used to calculate the correlation coefficient and change trend between modes based on the multimodal fusion tensor, and generate a multimodal feature map to characterize the cooperative change relationship between brightness, temperature and spectral lines.

[0091] It is understood that the system of this application can be configured in the processor.

[0092] Next, with reference to the accompanying drawings, we will further describe an emergency fire incident identification method for industrial enterprises provided by an embodiment of this application. Figure 3 The method shown is applied to a disaster prevention and monitoring platform, which is used to collect and process multimodal sensing data from industrial enterprise sites. The disaster prevention and monitoring platform includes a visible light sensing module, a thermal infrared sensing module, and a gas spectral sensing module. The method includes:

[0093] S1: Acquire multimodal sensing data from the visible light sensing module, thermal infrared sensing module, and gas spectral sensing module, and extract suspected fire areas;

[0094] In this embodiment, the synchronous acquisition of multimodal sensing data is achieved through timestamp unification and spatial extrinsic parameter calibration, ensuring that different modal data are analyzed correspondingly under the same spatiotemporal reference. Subsequently, the comprehensive response intensity is calculated by the rate of change of brightness, the amplitude of temperature gradient, and the rate of change of spectral line intensity, and regions with prominent responses are identified as suspected fire areas using a combination of multi-threshold segmentation and region growing algorithms.

[0095] Understandably, this step is used to separate spatial regions with potential combustion characteristics from complex backgrounds, avoiding false alarms in scenarios involving welding sparks, steam, or dust reflections. Multimodal fusion makes the recognition results more robust, and the threshold and response intensity weights can be flexibly adjusted according to different production environments, thereby significantly reducing the probability of false alarms while maintaining sensitivity.

[0096] S2: Input the multimodal sensing data corresponding to the suspected fire area into the preset thermal plume flow field model to generate a multimodal predicted trajectory under the assumption of a real fire.

[0097] The thermal plume flow field model mentioned above is constructed based on the prior mechanisms of thermal convection and plume dynamics.

[0098] In this embodiment, the thermal plume flow field model is numerically discretized based on buoyancy-driven, momentum conservation, and energy balance equations, and corrected by incorporating ambient wind speed, pressure difference, and obstacle distribution. The input parameters of the thermal plume flow field model include brightness changes, temperature rise rates, and spectral line intensity changes extracted from visible light, infrared, and spectral modes. Through mechanistic prior constraints, the thermal plume flow field model can calculate the rise velocity field, plume tip diffusion radius, and temperature distribution evolution of the flame or plume over time, thereby generating predicted trajectories for each mode under the assumption of a real fire.

[0099] S3: Calculate the cross-modal consistency score of the multimodal sensing data corresponding to the suspected fire area based on the multimodal prediction trajectory;

[0100] In this embodiment, the cross-modal consistency score is calculated by comparing the predicted trajectory with the actual observation data in a unified spatiotemporal coordinate system. Specifically, for the visible light mode, the structural similarity and morphological matching degree between the predicted brightness sequence and the actual brightness sequence are calculated; for the infrared mode, the root mean square error and the degree of agreement on the temperature rise trend between the predicted and actual temperature distributions are calculated; for the gas spectral mode, the deviations between the predicted and actual spectral lines in terms of peak position, full width at half maximum (FWHM), and rate of change of intensity are calculated. The results of each mode are weighted and fused according to the field environmental parameters to obtain a comprehensive consistency score. The essence of this step is to refute the real fire hypothesis through data. Only when the actual observation and the mechanism prediction are consistent across multiple modes can the event be considered to have real combustion characteristics, thereby avoiding misjudgment of visually similar false fires.

[0101] S4: If the cross-modal consistency score is greater than or equal to the preset judgment threshold, the target event is determined to be a real fire and an evidence pair including spatiotemporal thermal plume and spectral alignment results is output; otherwise, it is determined to be a false fire and the corresponding pseudotype label is output.

[0102] In this embodiment, the threshold for judging cross-modal consistency scores can be adaptively adjusted according to the risk level of different industrial scenarios. For example, a lower threshold can be used in chemical industrial areas to improve sensitivity, while a higher threshold can be used in welding areas to control the false alarm rate. When the score reaches the threshold, the spatiotemporal thermal plume and spectral line alignment results are presented in the form of image overlay or report, providing safety personnel with a traceable basis for judgment. If the score is lower than the threshold, it is automatically marked as a false fire and its modal characteristics are recorded for subsequent environmental self-calibration of the model.

[0103] Before detailing the specific technical aspects of the steps, this application's embodiments need to reiterate:

[0104] The emergency fire incident identification principle proposed in this application is not based on a simple improvement of traditional algorithms, but rather on a redefinition of the real-world observation mechanism in industrial settings. Industrial-grade fire detection has long relied on threshold determination of single signal features. Whether it's a sudden change in video brightness, the rate of infrared temperature rise, or an exceedance of gas concentration limits, these are essentially result-oriented detection paths—that is, inferring events from observed consequences. However, in complex real-world scenarios, such consequence features are often not unique.

[0105] Any process involving temperature rise, brightness change, or gas disturbance can trigger fire-like symptoms, ultimately leading to misjudgment. The technical starting point of this application is to find verifiable intermediate constraints from the causal chain of physical behavior, so that the identification logic no longer relies solely on the similarity of the results, but is based on the self-consistency of mechanistic consistency.

[0106] In practice, the identification process no longer involves a one-way observation-judgment, but instead constructs a reflexive verification mechanism:

[0107] Once potential fire characteristics are detected, a hypothetical thermal plume flow field model is first established based on the mechanistic constraints of thermal convection and plume dynamics. Then, using this model as a condition, the inevitable changing trends of each observation mode in the event of a real fire are deduced. Subsequently, this predicted trend is compared with the real-time observation signals on a unified spatiotemporal reference, thus forming a counterfactual judgment criterion. If the observed behavior and the mechanistic prediction remain intrinsically consistent, the event is considered to possess the dynamic characteristics of real combustion; if there is a significant deviation, it is determined to be caused by non-combustion disturbances.

[0108] In engineering practice, the operation of this application is manifested as a parallel processing architecture of temporal matching and mechanism verification. The visible light mode focuses on capturing morphological evolution and motion texture, the thermal infrared mode reflects temperature gradient and thermal conduction diffusion characteristics, and the gas spectral mode supplements the spectral response information of combustion products. After the three are synchronized in time and calibrated in space, a multi-dimensional coupled observation vector can be formed. On this basis, a thermal plume constraint with energy conservation, buoyancy drive, and momentum transfer as the core is introduced, making the prediction results physically interpretable. By comparing the temporal consistency between the prediction and the observation frame by frame, the dynamic rationality of the event can be judged, which can effectively eliminate false fire features caused by non-combustion interferences such as welding, electric sparks, and hot exhaust, and is especially suitable for scenarios with high reflectivity, strong disturbances, and limited viewing angles.

[0109] It is worth noting that the judgment criteria established in this embodiment are not customized for a single device or scenario, but rather an abstract model of the physical basis of fire phenomena. Therefore, the identification logic of this application is not only applicable to workshop video surveillance or infrared detection environments, but can also be extended to disaster prevention monitoring platforms with gas monitoring and spectral sampling capabilities. Through the two-way mapping relationship between mechanism priors and data observation, the method can maintain high judgment consistency under different operating conditions, different deployment densities, and even different sensing accuracies.

[0110] Those skilled in the art will understand that the specific sampling frequency and sensor type of multimodal data can be adjusted according to the field conditions, as long as they can simultaneously acquire the minimum features such as brightness changes, temperature gradients and spectral responses, without affecting the substance of the technical solution of this application.

[0111] Next, we will further elaborate on the technical aspects of the method in this application regarding areas suspected of being affected by fire.

[0112] Understandably, in complex industrial environments, accurately locating potential fire zones requires necessary preprocessing and structuring of the acquired multimodal sensing data. Otherwise, noise, reflections, and temporal drift in the raw data will directly obscure the true combustion characteristics. Therefore, before the data enters the fire analysis stage, a processing framework that unifies different modal information across spatiotemporal dimensions needs to be established, enabling observations from visible light, thermal infrared, and gas spectra to be compared and verified within the same reference frame.

[0113] In one example, acquiring multimodal sensing data from the visible light sensing module, the thermal infrared sensing module, and the gas spectral sensing module includes:

[0114] S1.1: Time synchronization and spatial calibration are performed on the visible light image sequence acquired by the visible light sensing module, the thermal infrared temperature field sequence acquired by the thermal infrared sensing module, and the gas spectral response data acquired by the gas spectral sensing module to establish a unified spatiotemporal coordinate reference framework.

[0115] Specifically, in the time dimension, a unified reference clock is used to time the sensing links, and timestamp alignment is completed through hardware pulses or network time protocols. The fixed and variable delays of each sensing path are recorded for delay compensation in the post-processing stage. For modes with different frame sampling frequencies, the sequences are resampled and interpolated to ensure that aligned frames with the same number represent the same observation time. In the spatial dimension, intrinsic parameter calibration and distortion correction are completed for the visible light and thermal infrared imaging units respectively. Then, extrinsic parameters are extracted through calibration targets or environmental steady-state features within a common field of view, and both are projected onto the same world coordinate system. For the spectral sensing unit, wavelength calibration and dark current and drift correction are first completed. Then, the geometric correspondence between the observation ray and the measured area is established based on the installation pose and sampling optical path, thereby anchoring the spectral sampling voxels to the region index consistent with the imaging field of view.

[0116] S1.2: Under the spatiotemporal coordinate reference frame, feature extraction and joint analysis are performed on the multimodal sensing data to obtain a multimodal feature map for characterizing brightness changes, temperature gradients and spectral line intensity changes;

[0117] Specifically, multimodal feature maps can be understood as a structured data representation that collaboratively encodes the changing trends of different physical modes on a unified spatial framework, rather than a simple image overlay. Its core lies in mapping the brightness dynamics of the visible light mode, the temperature gradient distribution of the thermal infrared mode, and the spectral line intensity evolution of the gas spectral mode to the same set of spatiotemporal voxels with mutually referential spatial and temporal scales. In this way, the originally independent perceptual dimensions of different modes are spatially mapped one-to-one and temporally unfolded synchronously, allowing their respective changing relationships to be directly observed and quantified.

[0118] In one specific implementation, dynamic texture analysis of visible light image sequences does not directly extract pixel brightness differences, but is based on a comprehensive analysis of temporally continuous inter-frame energy changes and local orientation field consistency. Visible light imaging is affected by both ambient light and device reflection; simple brightness fluctuations cannot accurately reflect the presence of a flame. Therefore, the brightness change rate distribution of each pixel in the frame sequence is first calculated, and the local motion direction and amplitude are calculated in conjunction with optical flow estimation to distinguish between random flickering and hot plume motion with a stable upward trend. In the spatial domain, Fourier analysis is performed on the brightness oscillation frequency of the edge region to extract the low-frequency continuous component and suppress the high-frequency flicker component, so as to highlight the gradual change characteristics of the flame and plume edges. At the same time, by calculating the optical flow orientation consistency parameter, the angle distribution of motion vectors in adjacent regions is statistically analyzed. If a clustering trend forms in the vertical direction, it indicates the existence of buoyancy behavior, which is then used as an important feature for determining the flame dynamics.

[0119] In another specific implementation, the temperature field sequence processing in the thermal infrared mode employs an analysis method combining gradient and diffusion characteristics. For each frame of the temperature distribution map, the rate of temperature change in the time direction is first calculated, and the difference results of consecutive frames are smoothed using Gaussian filtering to obtain a stable temporal temperature gradient. Then, the thermal conductivity diffusivity in the spatial direction is calculated, and the local temperature difference coefficient is used to reflect the diffusion trend of heat. Flame or combustion behavior in thermal infrared images typically manifests as a continuously rising temperature accompanied by an upward diffusion gradient field, while non-combustion disturbances are mostly non-steady-state and have uncertain diffusion directions. To further enhance the distinction between the two, a non-Gaussianity index for temperature distribution is introduced, namely, calculating the skewness and kurtosis of local temperature values ​​to describe the concentration and anomalous sharpness of the temperature distribution. When the temperature distribution in a certain area deviates from the normal characteristic and exhibits a continuously rising asymmetric distribution, it can be identified as a region where a heat plume may exist. By simultaneously considering temporal continuity, spatial diffusivity, and the anomalousness of the distribution pattern, temperature mode characteristics can effectively reflect the continuous energy release characteristics of flames, providing robust temperature evidence for subsequent mechanism verification.

[0120] In another specific implementation, the processing of gas spectral modes first involves extracting the positions, full width at half maximum (FWHM), and intensity change rates of absorption and radiation peaks within the characteristic bandwidth. The positions of characteristic peaks in the spectral signal reflect energy level transitions of specific gas components, while combustion products have unique emission line distributions. Therefore, by matching with a preset fuel spectral model, the presence of a real combustion process can be determined. To combat spurious peak interference caused by scattering from ambient vapors, hot air currents, or dust, this implementation calculates the intensity gradient difference and spectral linewidth change trends between adjacent bands. If synchronous enhancement and a stable FWHM occur, it is considered a combustion-related peak; otherwise, it is considered background noise. Simultaneously, a rate-of-intensity analysis is performed on the spectral intensity over time. The intensity growth slope is obtained through linear fitting within a short time window, determining whether it maintains a time-corresponding relationship with the thermal infrared temperature rise trend.

[0121] In another specific implementation, a multimodal fusion tensor is constructed based on each modal feature, and a multimodal feature map is calculated based on the multimodal fusion tensor, wherein:

[0122] The multimodal fusion tensor is standardized and dimensionally unified according to the three dimensions of space, time and mode, and a common metric domain with brightness change, temperature gradient and spectral line intensity as the main components is constructed. The main components are then assigned initial weights according to the field environmental parameters.

[0123] Robust tensor decomposition is performed on the common metric domain to obtain a low-rank component representing cross-modal common variation and a sparse component representing instantaneous noise, reflection scintillation and local jet interference, and the low-rank component is used as a candidate uniformity response field.

[0124] Within a preset sliding time window, the intermodal correlation matrix and canonical correlation coefficient of the low-rank components are calculated, and the correlation intensity map and time abrupt change map are obtained by combining the statistics of the change point test.

[0125] Physical feasibility screening is performed on the candidate consistent response fields to obtain a physical consistency response map. The physical feasibility screening includes: retaining voxel regions that simultaneously satisfy thermal convection and plume dynamics, voxel regions whose upward direction is consistent with the direction of visible light motion, voxel regions whose spectral bandwidth intensity continuously increases over time, and voxel regions that pass the mass conservation and energy balance approximation test.

[0126] The physical consistency response map, the correlation intensity map, and the time abrupt change map are weighted and combined to obtain a primary feature map, wherein the weights of the weighted combination are adjusted according to wind speed, temperature, and humidity.

[0127] Spatiotemporal connectivity filtering and morphological constraints are applied to the primary feature map to obtain a multimodal feature map.

[0128] In some optional implementations, after obtaining the three types of modal features, a multimodal fusion tensor is constructed based on their spatial correspondence. This tensor uses a unified spatial voxel as the basic unit, and each voxel contains the rate of change of brightness, temperature gradient, rate of change of spectral line intensity, and correlation factors between modes. To achieve dimensional uniformity among different modes, each modal feature is first standardized to zero mean and unit variance, and weighting coefficients are set according to sensor sensitivity and signal-to-noise ratio, so that modes with high signal reliability occupy a larger weight in the fusion. Subsequently, the co-responsivity is calculated in the tensor dimension, that is, the consistency of the changes of each modal feature over time is statistically analyzed. The comprehensive energy distribution characteristics of the fire area can be depicted through the co-responsivity field. Then, combined with the principal component decomposition of the tensor, the high-dimensional features are compressed into a three-dimensional feature map, where each channel represents the coupling strength of brightness, heat, and spectral response, respectively. The resulting multimodal feature map can intuitively show the common evolution relationship of different physical quantities in space, and can maintain a stable feature response even under complex conditions such as reflection, steam interference, and occlusion, providing a reliable input basis for subsequent screening of suspected fire areas and determination of mechanism consistency.

[0129] In yet another example, the extraction of suspected fire areas includes:

[0130] S1.3: In the multimodal feature map, a comprehensive response intensity map is calculated based on the brightness change rate, temperature gradient amplitude, and spectral line intensity change rate, wherein the comprehensive response intensity map is used to characterize the degree of synergistic enhancement of multimodal features within the spatial region;

[0131] Those skilled in the art will understand that the comprehensive response intensity map can be understood as a measurement of the coupled response of visible light, thermal infrared, and spectral modes within a unified spatial framework. Its core lies in measuring the synergistic enhancement relationship between the rate of change in brightness, the magnitude of the temperature gradient, and the rate of change in spectral line intensity across the spatiotemporal dimensions. The specific calculation methods for the rate of change in brightness, the magnitude of the temperature gradient, and the rate of change in spectral line intensity, as well as how to calculate the comprehensive response intensity map, can all be implemented using existing multimodal fusion algorithms, and will not be elaborated upon here.

[0132] S1.4: The comprehensive response intensity map is processed by multi-threshold segmentation and region growing algorithm to extract candidate response regions, wherein the multi-threshold segmentation is used to determine the initial range of response intensity, and the region growing algorithm is used to expand the response pixel group of the initial range;

[0133] Specifically, multi-threshold segmentation does not use a single fixed threshold. Instead, it determines multiple thresholds based on the histogram distribution of the comprehensive response and background statistics. It prioritizes locking high-confidence core seeds and then expands outward to a transition layer of medium intensity to reduce the disruption of connectivity caused by noise. Threshold selection incorporates scene-adaptive rules: when background disturbances are strong, the low-level threshold is increased to avoid large-scale false triggering; when the overall response within the observation window is low, the high-level threshold is decreased to preserve early weak signs.

[0134] In this embodiment, the region growth adopts either an eight-neighbor criterion or a three-dimensional voxel twenty-six-neighbor criterion, and uses the combined metric of comprehensive response intensity, directional consistency and persistence as growth conditions; during the growth process, the anisotropy within the neighborhood is constrained, so that the expansion is preferentially carried out along the upward and diffusion directions, and the lateral expansion is controlled, so as to better fit the spatial morphology of the thermal plume.

[0135] S1.5: Perform time consistency detection on the candidate response region, calculate the co-correlation coefficient of brightness change direction, temperature rise rate and spectral line change trend, retain the region corresponding to the positive correlation of the co-correlation coefficient, and obtain the time consistency region;

[0136] Specifically, the goal of time consistency detection is to distinguish between instantaneous high response and continuous coupled evolution. Therefore, a three-channel time-series trajectory is established within each candidate, and three types of relationship indicators—direction, amplitude, and trend—are calculated.

[0137] The directional relationship focuses on the unidirectional relationship between brightness motion and the principal temperature gradient; the amplitude relationship focuses on the relative stability of the temperature rise rate and the spectral intensity growth rate; the trend relationship evaluates the reliability of continuous growth through monotonicity within a short window and the number of inflection points. These relationships converge into a co-correlation coefficient, which measures the degree of consistent coupling of the three channels along the time axis.

[0138] In this embodiment, the estimation of the co-correlation coefficient is performed using a sliding window. Within each window, baseline removal and drift correction are applied to the three channels to avoid the influence of slowly varying backgrounds. For discontinuities caused by short-term missing measurements or occlusion, interpolation through adjacent windows is used for repair, and a maximum allowable percentage of missing measurements is set. Windows exceeding this percentage are not included in the statistics. To prevent the illusion of high correlation caused by short, intense solder joints or air masses, this application also includes hard constraints on minimum duration and minimum effective frame percentage. The co-correlation coefficient is only adopted when the effective frame percentage within the window reaches the set lower limit and the monotonicity threshold is satisfied.

[0139] S1.6: Combine the on-site environmental parameters to perform dynamic stability assessment on the time consistency area, and output the suspected fire area through differential smoothing;

[0140] Specifically, industrial sites are subject to background factors such as wind direction changes, equipment start-up and shutdown, and illuminance fluctuations. Even with good temporal consistency, natural fluctuations such as boundary tremors and slight intensity drops may still occur. Therefore, dynamic filtering driven by environmental parameters is introduced during the stabilization assessment phase: the tolerance for upward fluctuations is adjusted based on wind speed and direction, short-term attenuation of spectral channels is compensated based on humidity, and the baseline drift of infrared channels is corrected based on ambient temperature. Subsequently, differential smoothing is performed on the area, centroid trajectory, and comprehensive intensity sequence of the temporal consistency region within a short time window to eliminate high-frequency jitter and preserve the true growth trend.

[0141] Next, we will further elaborate on the technical content of the thermal plume flow field model in this application.

[0142] In one example, the thermal plume flow field model includes an input layer, a hidden layer, and an output layer, wherein:

[0143] The input layer is used to receive multimodal sensing data corresponding to suspected fire areas, establish feature sequences based on input parameters, and calculate the fitting value of the feature sequences.

[0144] The hidden layer, based on the mechanistic priors of thermal convection and plume dynamics, performs coupled calculations on the fitted values. The mechanistic priors include the buoyancy driving equation, the momentum conservation equation, and the energy balance equation. The hidden layer solves for the rising velocity field, temperature distribution field, and plume tip diffusion radius of the thermal plume in the time series through numerical discretization. During the calculation process, it is corrected by the ambient wind speed, pressure difference, and obstacle distribution to obtain the thermal plume flow field parameters.

[0145] The output layer is used to generate a multimodal predicted trajectory under the assumption of a real fire, based on the thermal plume flow field parameters calculated from the hidden layer.

[0146] Understandably, the basic principle of the aforementioned thermal plume flow field model lies in using the mechanisms of thermal convection and plume dynamics to perform counterfactual deduction of the heat flow evolution process within a suspected fire area, thereby generating the spatiotemporal trajectory of each mode under the assumption of a real fire. This model, by constructing the dynamic evolution relationship between the rising velocity field, temperature distribution field, and diffusion radius of the thermal plume over time, characterizes the physical processes of airflow uplift, thermal diffusion, and gas composition changes caused by combustion behavior, making subsequent matching with actual observation data clearly physically interpretable.

[0147] In terms of model construction, the brightness change rate, temperature gradient, and spectral line intensity from multimodal sensing data are used as input parameters. After time serialization, these parameters are input to the model's input layer to establish an input tensor containing temporal features. The hidden layer incorporates prior knowledge based on buoyancy-driven equations, momentum conservation equations, and energy balance equations, and uses finite difference or finite volume methods to discretize and solve the approximate field distribution of plume flow and heat transfer. Environmental wind speed, pressure gradient, and obstacle spatial distribution parameters are incorporated into the calculation process to correct the spatiotemporal distribution of rise velocity and diffusion radius, enabling the model to adapt to turbulent disturbances and local ventilation effects present in industrial environments. The output layer generates multimodal predicted trajectories, including temperature, velocity, density, and radiation intensity, that vary over time based on the flow field parameters obtained from the hidden layer.

[0148] It should be noted that the structure of the thermal plume flow field model is not limited to a three-layer form, and can also be implemented in other forms, such as embedding the mechanism prior into a lattice-based fluid dynamics model, a lattice Boltzmann model based on particle distribution functions, or a hybrid modeling method combining a neural differential equation framework.

[0149] Those skilled in the art will understand that any model constructed that can generate a physically consistent thermal plume distribution and evolution trajectory based on input multimodal observation data under the assumption of a real fire can be regarded as an alternative implementation of this application, and this application does not limit it.

[0150] In one example, generating the multimodal predicted trajectory under the assumption of a real fire includes:

[0151] S2.1: Based on the hot plume flow field parameters, calculate the upward velocity distribution of the hot plume central axis, the plume tip height, and the spatiotemporal evolution curve of temperature, and obtain the radiation intensity distribution of the hot plume surface based on the energy conservation relationship.

[0152] Specifically, the central axis of the thermal plume is used to characterize the dominant direction of the ascent channel, while the plume tip height reflects the dynamic boundary of heat carrying and environmental mixing. Both are intrinsically linked to the spatiotemporal evolution of temperature. To ensure the predictive trajectory is comparable, the ascent velocity, diffusion scale, and temperature accumulation should be expanded with a consistent time step, using the thermal plume flow field parameters as a link. Based on this, the radiation intensity distribution for visible and infrared modes should be derived, establishing a comparable mapping between observed and mechanistic quantities.

[0153] In this embodiment, the central axis can be determined by the mainstream integral curve of the velocity field. The plume height is given by the furthest point of the temperature isopleth boundary in the vertical direction, and local backflow is smoothed in a small range to avoid artificial height. The temperature curve is modeled using a combination of incremental accumulation and mixing attenuation within a fixed time window: incremental accumulation represents the continuous input of combustion, and mixing attenuation reflects the environmental dilution effect; both are corrected by adjusting the ambient wind speed, obstacle gap, and pressure difference. The radiation intensity distribution is given by a temperature-radiation mapping table, the parameters of which are obtained by field calibration at two or more points, and an equivalent emissivity correction is applied to the high reflectivity area of ​​the metal surface to obtain the radiation intensity profile of the thermal plume surface at each radial position.

[0154] S2.2: Project the radiation intensity distribution onto the visible light and thermal infrared observation planes, and combine the camera field of view, distance attenuation coefficient, and sensor spectral response characteristics to generate corresponding visible light brightness prediction sequences and infrared radiation brightness prediction sequences;

[0155] Specifically, the signal on the observation plane is the result of the combined effects of volume radiation and geometric imaging. Without projection transformation and response correction, the mechanistic spatial quantities are difficult to align directly with the image plane quantities. To form a prediction consistent with the sensor readings, the three-dimensional radiation intensity field needs to be mapped to pixel coordinates after being clipped by the view frustum, integrated by the line of sight, and attenuated by distance, while also considering factors such as lens transfer, sensor quantization, and spectral bandpass.

[0156] In this embodiment, projection uses an extrinsic parameter matrix to transform world coordinates to camera coordinates, and then maps them to the pixel plane using a pinhole imaging model. Distance attenuation is normalized by power based on the distance of the sampling point from the camera center along the line of sight, and projection area correction is introduced for large-angle lines of sight. The visible light and thermal infrared channels are loaded with their spectral response curves and integration time and gain calibration tables, respectively, and the line time difference caused by the rolling shutter is corrected with intra-frame linearity. For the visible light channel, considering the semi-transparent characteristics of the plume boundary, a semi-transparent superposition weight is applied to the boundary voxels in the line-of-sight integration to reproduce the brightness levels of the edge transition. For the infrared channel, under high-temperature and sparse plume conditions, the sampling density along the line is increased to reduce the low brightness caused by undersampling.

[0157] S2.3: Based on the temperature distribution field and fuel type information, the emission intensity of the characteristic spectral lines of the combustion products and their rate of change over time are calculated using a preset radiative transfer model to obtain the time response curve of the spectral bandwidth;

[0158] Specifically, comparable quantities for gaseous modes come from the emission or absorption behavior of combustion products in specific wavelength bands, the amplitude and rate of change of which are influenced by temperature, path length, and component concentration. By linking temperature distribution with fuel family profiles using a unified radiative transfer framework, time response curves consistent with the sampling optical path and bandwidth settings can be generated, making spectral predictions and camera predictions comparable on the same time axis.

[0159] In this embodiment, radiative transfer is described using bandwidth integral form. The inputs are the temperature field, the estimated effective optical path length, and the spectral parameters corresponding to the fuel type. The spectral parameters are provided by a fuel family library and can be calibrated on-site using a reference combustion source. To reduce interference from vapor and aerosol scattering, narrow bands with weaker water vapor interference are prioritized in band selection, and switchable subbands are added for cross-validation. For different fuel mixing scenarios, the equivalent mixing response is calculated based on the weighted distribution of the temperature field along the optical path. The rate of change of time is obtained by steady-state fitting of a short time series, and occasional spikes are suppressed using robust regression.

[0160] S2.4: Synchronize and register the visible light brightness prediction sequence, infrared radiance brightness prediction sequence, and spectral bandwidth time response curve under a unified spatiotemporal coordinate system to generate a multimodal prediction trajectory.

[0161] In one example, the cross-modal consistency score of the multimodal sensing data corresponding to the suspected fire area is calculated based on the multimodal predicted trajectory, including:

[0162] The multimodal predicted trajectory is spatially registered and temporally aligned with the corresponding multimodal sensing data to establish an observation correspondence.

[0163] Based on the observed correspondence, the consistency measurement results of each mode are calculated, wherein the consistency measurement results of each mode include the consistency measurement results of the visible light mode, the consistency measurement results of the thermal infrared mode, and the consistency measurement results of the gas spectral mode;

[0164] The cross-modal consistency score is calculated based on the consistency metric results.

[0165] Understandably, the cross-modal consistency score is used to measure the degree of agreement between predicted trajectories and actual observations in a multimodal space. Essentially, it is a similarity evaluation constrained by physical mechanisms, rather than a simple numerical fitting or image comparison. By calculating this score, it is possible to comprehensively evaluate, within a unified metric domain, whether the observations of each modality simultaneously satisfy the coupling change patterns under the assumption of a real fire, thereby determining whether the anomalies in the region originate from actual combustion behavior.

[0166] In this embodiment, to ensure the accuracy of consistency calculations, the multimodal predicted trajectories generated by the thermal plume flow field model are first spatially registered and temporally aligned with the multimodal sensing data collected on-site. Spatial registration is based on a unified extrinsic parameter matrix and calibration parameters, mapping the imaging planes of different viewpoints or sensors to ensure that each predicted voxel or pixel can establish a one-to-one relationship with the corresponding point in the actual observation data. For temporal alignment, the time nodes of the predicted sequence and the observation sequence are matched using timestamps and frame sequence information. For cases with frame rate differences, interpolation or resampling is used to ensure consistent temporal resolution. After alignment, an observation relationship table containing corresponding points of the multimodal data is established to provide input for subsequent consistency measurements.

[0167] Furthermore, consistency metrics are calculated independently within each mode and then fused under unified rules. The consistency metric for the visible light mode focuses on the spatial structural similarity between brightness distribution and dynamic texture, calculating a similarity coefficient by comparing the trends, texture gradients, and motion directions of the predicted and observed brightness sequences. The consistency metric for the thermal infrared mode centers on the spatiotemporal gradient matching of the temperature field, examining the deviations between the predicted and observed temperature rise curves in terms of growth rate, peak time, and diffusion morphology. The consistency metric for the gas spectral mode is based on spectral feature point matching, calculating the degree of overlap between the predicted and observed absorption and radiation peak positions, full width at half maximum (FWHM), and intensity change rate. The consistency results for each mode are fused into an overall cross-modal consistency score using weighted correlation coefficients, with the weights dynamically adjusted based on the field environment and sensor signal-to-noise ratio.

[0168] For example, the following numerical example is provided to illustrate how to convert the cross-modal consistency score. This example is only used to explain the relationship between the computational link and the dimensions. The selected parameters and values ​​are illustrative and do not represent the actual calibration results or engineering recommendations.

[0169] At a certain moment, the mechanism deduction module, under the assumption of real fire, provides three types of predicted quantities within the same spatial window: the infrared mode predicts an average temperature rise of 18 degrees Celsius, the visible light mode predicts a main plume direction of 62 degrees and a plume projection area of ​​2.4 square meters, and the spectral mode predicts a carbon monoxide characteristic peak intensity of 0.36 (based on the dimensionless reading after equipment normalization). At the same moment, the sensor's actual measurements show: an average temperature rise of 15 degrees Celsius in the infrared window, visible light detection shows a main plume direction of 70 degrees and a projection area of ​​2.1 square meters, and spectral detection shows a carbon monoxide characteristic peak intensity of 0.33. In order to uniformly convert the deviations of different dimensions into a consistency score of 0 to 1, this embodiment adopts a method of deviation normalization followed by linear deduction and truncation: First, an allowable scale is set for each quantity to express that the deviation can be regarded as consistent within the scale. Here, it is shown that the infrared temperature rise allowable scale is 10 degrees Celsius, the visible light direction allowable scale is 40 degrees, the area allowable scale is 3.0 square meters, and the spectral peak intensity allowable scale is 0.30.

[0170] In this example, the absolute deviation of the infrared temperature rise is 18 minus 15, which equals 3 degrees Celsius. Dividing this by the allowable scale of 10 yields a normalized deviation of 0.30. Consistency is calculated by subtracting the normalized deviation from 1, resulting in an infrared consistency of 1.00 minus 0.30 equaling 0.70. Next, regarding visible light consistency, the direction and area are combined into a single visible light consistency. The direction deviation is 70 minus 62, which equals 8 degrees. Dividing this by the allowable direction scale of 40 yields 0.20. The area deviation is 2.4 minus 2.1, which equals 0.3 square meters. Dividing this by the allowable area scale of 3.0 yields 0.10. To better reflect the constraint that direction better characterizes the dynamic morphology of the plume, this embodiment uses a direction weight of 0.6 and an area weight of 0.4. Therefore, the comprehensive normalized deviation of visible light is 0.6 multiplied by 0.20 plus 0.4 multiplied by 0.10, resulting in 0.16. Thus, the visible light consistency is 1.00 minus 0.16 equaling 0.84. Finally, we examine the spectral consistency. The peak intensity deviation is 0.36 minus 0.33, which equals 0.03. Dividing this by the tolerance scale of 0.30 gives 0.10. Therefore, the spectral consistency is 1.00 minus 0.10, which equals 0.90. At this point, all three modes have been converted to the same dimension, the 0-1 interval, and a larger value indicates a better agreement between the measured values ​​and the counterfactual predictions.

[0171] Furthermore, during cross-modal fusion, the three consistency scores need to be combined into a single comprehensive consistency score for threshold determination. Illustrated, considering that smoke obstruction may reduce visible light stability, and that infrared and spectral data are more sensitive to the "chemical / thermal characteristics of real fire," this embodiment illustrates a weighting of 0.4 for infrared, 0.2 for visible light, and 0.4 for spectral data. The comprehensive consistency score is obtained by weighted summation: first, the infrared contribution (0.4 x 0.70) equals 0.28; the visible light contribution (0.2 x 0.84) equals 0.168; and the spectral contribution (0.4 x 0.90) equals 0.36. Adding these three together yields 0.808, therefore the comprehensive consistency score is 0.808. If the alarm threshold configured in the system is 0.75, then 0.808 is greater than 0.75, satisfying the consistency determination condition, and the identification result of a suspected real fire incident is output.

[0172] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. An industrial enterprise-oriented emergency fire fighting event identification method applied to a disaster prevention monitoring platform, characterized in that, The disaster prevention monitoring platform is used to collect and process multimodal sensing data from industrial enterprise sites. The platform includes a visible light sensing module, a thermal infrared sensing module, and a gas spectral sensing module. The method includes: Multimodal sensing data from the visible light sensing module, thermal infrared sensing module, and gas spectral sensing module are acquired to extract suspected fire areas; The multimodal sensing data corresponding to the suspected fire area is input into a preset thermal plume flow field model to generate a multimodal prediction trajectory under the assumption of a real fire. The thermal plume flow field model is constructed based on the prior mechanism of thermal convection and plume dynamics. The cross-modal consistency score of the multimodal sensing data corresponding to the suspected fire area is calculated based on the multimodal prediction trajectory. If the cross-modal consistency score is greater than or equal to the preset judgment threshold, the target event is determined to be a real fire and an evidence pair including spatiotemporal thermal plume and spectral alignment results is output; otherwise, it is determined to be a false fire and the corresponding pseudotype label is output. The acquisition of multimodal sensing data from the visible light sensing module, the thermal infrared sensing module, and the gas spectral sensing module includes: The visible light image sequence acquired by the visible light sensing module, the thermal infrared temperature field sequence acquired by the thermal infrared sensing module, and the gas spectral response data acquired by the gas spectral sensing module are synchronized in time and calibrated in space to establish a unified spatiotemporal coordinate reference framework. Within the spatiotemporal coordinate reference frame, feature extraction and joint analysis are performed on the multimodal sensing data to obtain a multimodal feature map for characterizing brightness changes, temperature gradients, and spectral line intensity changes; Feature extraction and joint analysis are performed on the multimodal sensing data to obtain multimodal feature maps characterizing brightness changes, temperature gradients, and spectral line intensity changes, including: Dynamic texture analysis is performed on visible light image sequences to extract pixel brightness change rate, edge oscillation frequency, and optical flow direction consistency parameters to obtain image modal features; The temporal temperature gradient, spatial thermal conductivity, and non-Gaussianity index of local temperature distribution are calculated for the thermal infrared temperature field sequence in order to identify the heat plume region with continuous heating and buoyancy characteristics, and to obtain temperature mode characteristics. The positions, half width at half maximum (FWHM), and intensity change rates of absorption and radiation peaks are extracted from the gas spectral response data within the characteristic bandwidth range. Similarity matching is performed with the preset fuel spectral line model to distinguish the characteristic spectral lines of combustion products from the scattering spectral lines of steam or hot gas flow, thereby obtaining the gas modal characteristics. A multimodal fusion tensor is constructed based on the features of each modality, and a multimodal feature map is calculated based on the multimodal fusion tensor; Calculating the multimodal feature map based on the multimodal fusion tensor includes: The multimodal fusion tensor is standardized and dimensionally unified according to the three dimensions of space, time and mode, and a common metric domain with brightness change, temperature gradient and spectral line intensity as the main components is constructed. The main components are then assigned initial weights according to the field environmental parameters. Robust tensor decomposition is performed on the common metric domain to obtain a low-rank component representing cross-modal common variation and a sparse component representing instantaneous noise, reflection scintillation and local jet interference, and the low-rank component is used as a candidate uniformity response field. Within a preset sliding time window, the intermodal correlation matrix and canonical correlation coefficient of the low-rank components are calculated, and the correlation intensity map and time abrupt change map are obtained by combining the statistics of the change point test. Physical feasibility screening is performed on the candidate consistent response fields to obtain a physical consistency response map. The physical feasibility screening includes: retaining voxel regions that simultaneously satisfy thermal convection and plume dynamics, voxel regions whose upward direction is consistent with the direction of visible light motion, voxel regions whose spectral bandwidth intensity continuously increases over time, and voxel regions that pass the mass conservation and energy balance approximation test. The physical consistency response map, the correlation intensity map, and the time abrupt change map are weighted and combined to obtain a primary feature map, wherein the weights of the weighted combination are adjusted according to wind speed, temperature, and humidity. Spatiotemporal connectivity filtering and morphological constraints are applied to the primary feature map to obtain a multimodal feature map.

2. The emergency fire incident identification method for industrial enterprises according to claim 1, characterized in that, The area suspected of being a fire includes: In the multimodal feature map, a comprehensive response intensity map is calculated based on the brightness change rate, temperature gradient amplitude, and spectral line intensity change rate, wherein the comprehensive response intensity map is used to characterize the degree of synergistic enhancement of multimodal features within a spatial region; The comprehensive response intensity map is processed by multi-threshold segmentation and region growing algorithm to extract candidate response regions, wherein the multi-threshold segmentation is used to determine the initial range of response intensity, and the region growing algorithm is used to expand the response pixel group of the initial range. Temporal consistency detection is performed on the candidate response regions, and the co-correlation coefficients of brightness change direction, temperature rise rate and spectral line change trend are calculated. Regions corresponding to positive correlation of the co-correlation coefficients are retained to obtain temporal consistency regions. The time-consistency area is dynamically stabilized by combining on-site environmental parameters, and the suspected fire area is output through differential smoothing.

3. The emergency fire incident identification method for industrial enterprises according to claim 1, characterized in that, The thermal plume flow field model includes an input layer, a hidden layer, and an output layer, wherein: The input layer is used to receive multimodal sensing data corresponding to suspected fire areas, establish feature sequences based on input parameters, and calculate the fitting value of the feature sequences. The hidden layer, based on the mechanistic priors of thermal convection and plume dynamics, performs coupled calculations on the fitted values. The mechanistic priors include the buoyancy driving equation, the momentum conservation equation, and the energy balance equation. The hidden layer solves for the rising velocity field, temperature distribution field, and plume tip diffusion radius of the thermal plume in the time series through numerical discretization. During the calculation process, it is corrected by the ambient wind speed, pressure difference, and obstacle distribution to obtain the thermal plume flow field parameters. The output layer is used to generate a multimodal predicted trajectory under the assumption of a real fire, based on the thermal plume flow field parameters calculated from the hidden layer.

4. The emergency fire incident identification method for industrial enterprises according to claim 3, characterized in that, The generated multimodal predicted trajectory under the assumption of a real fire includes: Based on the hot plume flow field parameters, the upward velocity distribution of the hot plume central axis, the plume tip height, and the spatiotemporal evolution curve of temperature are calculated, and the radiation intensity distribution of the hot plume surface is obtained based on the energy conservation relationship. The radiation intensity distribution is projected onto the visible light and thermal infrared observation planes, and combined with the camera field of view, distance attenuation coefficient and sensor spectral response characteristics to generate corresponding visible light brightness prediction sequences and infrared radiation brightness prediction sequences. Based on the temperature distribution field and fuel type information, the emission intensity of characteristic spectral lines of combustion products and their rate of change over time are calculated using a preset radiative transfer model, resulting in the time response curve of the spectral bandwidth. The visible light brightness prediction sequence, infrared radiance prediction sequence, and time response curve of spectral bandwidth are synchronized and registered under a unified spatiotemporal coordinate system to generate a multimodal prediction trajectory.

5. The emergency fire incident identification method for industrial enterprises according to claim 1, characterized in that, The cross-modal consistency score of the multimodal sensing data corresponding to the suspected fire area is calculated based on the multimodal predicted trajectory, including: The multimodal predicted trajectory is spatially registered and temporally aligned with the corresponding multimodal sensing data to establish an observation correspondence. Based on the observed correspondence, the consistency measurement results of each mode are calculated, wherein the consistency measurement results of each mode include the consistency measurement results of the visible light mode, the consistency measurement results of the thermal infrared mode, and the consistency measurement results of the gas spectral mode; The cross-modal consistency score is calculated based on the consistency metric results.

6. An emergency fire incident identification system for industrial enterprises, used to implement the emergency fire incident identification method for industrial enterprises as described in any one of claims 1-5, characterized in that, The system includes: The multimodal sensing and acquisition module is used to acquire visible light images, thermal infrared temperature fields, and gas spectral response data at the industrial site, and to perform time synchronization and spatial calibration of the sensing data for each modality. The feature fusion and parsing module is used to extract and jointly analyze features from calibrated multimodal sensing data, construct a multimodal fusion tensor, and generate multimodal feature maps to characterize brightness changes, temperature gradients, and spectral line intensity changes. The thermal plume flow field modeling module is used to receive multimodal sensing data corresponding to suspected fire areas, calculate thermal plume flow field parameters based on the mechanism prior of thermal convection and plume dynamics, and generate multimodal prediction trajectories under the assumption of a real fire. The cross-modal consistency determination module is used to calculate the cross-modal consistency score based on the multimodal predicted trajectory and the actual observation data, determine whether the target event is a real fire or a false fire based on a preset threshold, and output the corresponding evidence pair or pseudo-type label.

7. The emergency fire incident identification system for industrial enterprises according to claim 6, characterized in that, The feature fusion and parsing module includes: The modal feature extraction unit is used to perform dynamic texture analysis, temperature gradient calculation and spectral feature extraction on visible light image sequences, thermal infrared temperature field sequences and gas spectral response data, respectively, to obtain brightness change parameters, temperature distribution parameters and spectral intensity change parameters. The multimodal tensor construction unit constructs a multimodal fusion tensor based on the brightness variation parameters, temperature distribution parameters, and spectral intensity variation parameters, and performs spatial, temporal, and dimensional unification processing on the multimodal fusion tensor. The feature correlation calculation unit is used to calculate the correlation coefficient and change trend between modes based on the multimodal fusion tensor, and generate a multimodal feature map to characterize the cooperative change relationship between brightness, temperature and spectral lines.

Images