Fire intelligent early warning method and system based on multi-modal data

Abstract

This invention discloses a fire intelligent early warning method and system based on multimodal data, belonging to the field of fire early warning technology. The method includes the following steps: acquiring the accuracy characterization parameters of thermal imaging sensors to evaluate monitoring accuracy, and optimizing the accuracy of sensors that do not meet preset requirements; using environmental gas phase parameters to correct infrared radiation attenuation in thermal imaging data, eliminating meteorological interference to restore the true temperature field; dynamically dividing the target area based on the corrected data, distinguishing between normal and abnormal pixel areas, and initially marking abnormal areas with an adaptively matched fire discrimination threshold; introducing smoke features from visible light images to perform secondary marking of specific abnormal areas, reducing false alarms through multimodal cross-validation; and adopting differentiated strategies for different status areas: dynamically adjusting the monitoring frequency for potential risk areas, and implementing graded early warnings for confirmed fire areas based on the fire spread trend.

Description

Technical Field

[0001] This invention relates to the field of fire early warning technology, and in particular to a fire intelligent early warning method and system based on multimodal data. Background Technology

[0002] Current intelligent fire early warning technologies mainly rely on image recognition using single-modal sensors or fixed thresholds, which suffers from problems such as response lag, susceptibility to environmental interference, and high false alarm rates. Thermal imaging technology, due to its advantages of temperature sensitivity, smoke penetration, and all-weather operation, has been gradually introduced into the field of fire monitoring. However, current thermal imaging fire detection methods still have the following shortcomings: First, the accuracy of thermal imaging sensors drifts over time (e.g., changes in pixel response rate, increased blind pixels), leading to inaccurate temperature measurements if not calibrated in time; second, the attenuation effect of environmental meteorological factors (e.g., water vapor, aerosols) on infrared radiation is not effectively compensated for, making it difficult to identify fire points at long distances; third, most methods use fixed temperature or area thresholds for fire assessment, which cannot adapt to different monitoring distances and seasonal changes, easily leading to missed or false alarms; fourth, relying solely on thermal imaging makes it difficult to distinguish between high-temperature interference sources (e.g., industrial equipment, vehicle lights) and actual flames.

[0003] Although there are existing patents for fire prediction, such as the invention patent with publication number CN119647962A which discloses a fire risk assessment and early warning system, including: (1) classifying potential fire risk factors into physical factors and environmental factors and collecting relevant data; (2) constructing a risk assessment data learning sample; (3) classifying fire risk levels; (4) establishing a fire risk calculation and assessment model; (5) comparing the risk score obtained by the established model with the classified fire risk level to obtain the building risk level. For example, the invention patent with publication number CN116611562A which discloses a smart park fire early warning management system and method based on the Internet of Things, includes: a data acquisition module, a data matching module, a prediction module, an early warning module, an electrical quantity monitoring module, a data preprocessing module and an analysis and processing module. By comparing the real-time data of the environmental acquisition points with the real-time curves of historical data and the trajectories of various historical curves, and matching them through a set matching formula, the known fire occurrence situation corresponding to the historical data with a high degree of matching with the real-time data is found to predict the current fire situation. However, none of these solutions address the accuracy assurance and environmental adaptive correction of the thermal imaging sensors themselves, making it difficult to achieve accurate and dynamic early fire warnings in complex environments. Therefore, how to evaluate and optimize the accuracy of thermal imaging sensors, correct infrared radiation by combining meteorological data, dynamically divide regions, and integrate visible light characteristics have become key issues that urgently need to be addressed in current intelligent fire early warning technologies. Summary of the Invention

[0004] To address the technical challenges of low accuracy, poor environmental adaptability, and frequent false alarms and missed alarms in fire early warning systems caused by sensor performance degradation, meteorological interference, fixed thresholds, and limitations of single-modality approaches, this invention provides a fire intelligent early warning method and system based on multimodal data, achieving accurate perception and intelligent early warning of fire risks in complex environments. The technical solution is as follows: On the one hand, a fire intelligent early warning method based on multimodal data is provided. This method includes: acquiring the accuracy characterization parameters of a thermal imaging sensor and evaluating its monitoring accuracy; determining whether the monitoring accuracy evaluation result meets preset monitoring requirements; if it does, performing adaptive matching of the fire discrimination threshold; if it does not, optimizing the monitoring accuracy of the thermal imaging sensor and re-evaluating the monitoring accuracy until the monitoring accuracy evaluation result meets the preset monitoring requirements; based on the environmental gas phase parameters collected in real time by meteorological sensors in the target area, performing infrared radiation intensity attenuation correction on the thermal imaging data collected by the thermal imaging sensor to obtain corrected thermal imaging data; and based on the corrected thermal imaging data... The target area is dynamically divided into regions, generating normal and abnormal pixel regions. Normal pixel regions are marked as first-state regions. Based on the dynamic region classification results and the adaptive matching results of the fire discrimination threshold, the state of each abnormal pixel region is dynamically marked, generating second and third-state regions. Based on the smoke features extracted from the visible light image of the target area, the third-state region is further marked, dividing it into first-state and second-state regions. For the first-state region, the monitoring frequency is dynamically adjusted based on the potential risks of the target area. For the second-state region, a graded fire early warning is implemented based on the fire spread trend.

[0005] On the other hand, a fire intelligent early warning system based on multimodal data is provided. This system is applied to a fire intelligent early warning method based on multimodal data. The system includes: a monitoring accuracy control module, a radiation intensity correction module, a regional status discrimination module, and a fire classification early warning module. The monitoring accuracy control module acquires the accuracy characterization parameters of the thermal imaging sensor and performs a monitoring accuracy assessment. It determines whether the monitoring accuracy assessment result meets the preset monitoring requirements. If it does, it performs adaptive matching of the fire discrimination threshold; if not, it optimizes the monitoring accuracy of the thermal imaging sensor and re-performs the monitoring accuracy assessment until the monitoring accuracy assessment result meets the preset monitoring requirements. The radiation intensity correction module performs infrared correction on the thermal imaging data collected by the thermal imaging sensor based on the environmental gas phase parameters collected in real time by the meteorological sensor in the target area. Radiation intensity attenuation correction yields corrected thermal imaging data. A region status discrimination module dynamically classifies the target area based on the corrected thermal imaging data, generating normal and abnormal pixel regions. Normal pixel regions are marked as first-state regions. Based on the dynamic region classification results and the adaptive matching result of the fire discrimination threshold, the status of each abnormal pixel region is dynamically marked, generating second and third-state regions. A fire classification and early warning module performs secondary labeling of the third-state region based on the smoke features extracted from the visible light image of the target area, dividing it into first-state and second-state regions. For first-state regions, the monitoring frequency is dynamically adjusted based on the potential risk of the target area. For second-state regions, a graded fire warning is issued based on the fire spread trend.

[0006] Beneficial effects The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following: 1. This invention improves the accuracy, adaptability, and timeliness of fire early warning by monitoring, evaluating, and dynamically optimizing the accuracy characterization parameters of thermal imaging sensors, correcting thermal imaging data by combining environmental gas phase parameters, dynamically partitioning and marking the target area, using visible light smoke characteristics for secondary verification of suspected areas, and dynamically adjusting the monitoring frequency and executing graded early warnings for different state areas. This effectively reduces the probability of false alarms and missed alarms, takes into account the monitoring needs and fire response efficiency in different scenarios, and thus realizes a full-process, intelligent, and refined intelligent fire early warning based on multimodal data, providing reliable technical support for early fire detection and rapid response.

[0007] 2. This invention, based on the comparison between the blind pixel rate of the thermal imaging sensor and a preset threshold, performs targeted dynamic adjustment of the blind pixel compensation intensity and hierarchical dynamic adjustment of the response gain coefficient. The blind pixel compensation is precisely controlled according to the blind pixel compensation requirement and the monitoring accuracy deviation to suppress discrete bad pixel interference. The response gain is adapted to different accuracy deviation scenarios through four-level gradient levels, balancing signal amplification and noise suppression, thereby accurately compensating for the sensor's monitoring accuracy gap, effectively optimizing the imaging quality and temperature measurement accuracy of the thermal imaging sensor, avoiding the impact of insufficient accuracy on fire identification, and thus achieving adaptive improvement and stable control of the monitoring accuracy of the thermal imaging sensor. This provides high-precision data support for subsequent fire early warning work and ensures the reliability and accuracy of the early warning system.

[0008] 3. This invention preprocesses the visible light image corresponding to the suspected fire area in the third state, extracts the feature vector of smoke texture and edge contour fusion, and performs cosine similarity matching between it and the standard feature vector of the preset smoke template. Based on the matching result, the suspected area is marked with a secondary state, thereby accurately distinguishing between real fires and false suspected areas, effectively reducing the false alarm rate of fire warnings, improving the accuracy of suspected fire identification, and thus realizing accurate verification and secondary confirmation of fire status. This further ensures the accuracy and reliability of fire warnings and provides accurate basis for subsequent graded warnings and responses. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 A flowchart of a fire intelligent early warning method based on multimodal data provided in this application embodiment; Figure 2 A flowchart illustrating the fire warning area status determination process for the intelligent fire warning method based on multimodal data provided in this application embodiment; Figure 3 This is a schematic diagram of the structure of a fire intelligent early warning system based on multimodal data, provided in an embodiment of this application. Detailed Implementation

[0011] Embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of the present disclosure are shown in the drawings, it should be understood that embodiments of the present disclosure may be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure.

[0012] It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure. In the description of the embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "this embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc., may refer to different or the same objects.

[0013] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0014] like Figure 1 The diagram shows a flowchart of a fire intelligent early warning method based on multimodal data provided in this application embodiment. The method includes the following steps: acquiring the accuracy characterization parameters of the thermal imaging sensor and performing a monitoring accuracy assessment; determining whether the monitoring accuracy assessment result meets the preset monitoring requirements; if it does, performing adaptive matching of the fire discrimination threshold; if it does not, performing monitoring accuracy optimization on the thermal imaging sensor and re-performing the monitoring accuracy assessment until the monitoring accuracy assessment result meets the preset monitoring requirements; and performing infrared radiation intensity attenuation correction on the thermal imaging data collected by the thermal imaging sensor based on the environmental gas phase parameters collected in real time by the meteorological sensor in the target area to obtain the corrected thermal imaging data.

[0015] Figure 2 This is a flowchart illustrating the fire warning area status determination process of the intelligent fire early warning method based on multimodal data provided in this application embodiment. Based on corrected thermal imaging data, the target area is dynamically divided into region categories, generating normal pixel regions and abnormal pixel regions. Normal pixel regions are marked as first-state regions. Based on the dynamic region category division results and the adaptive matching results of the fire discrimination threshold, the status of each abnormal pixel region is dynamically marked, generating second-state regions and third-state regions. The first region is a fire-free region, the second region is a confirmed fire region, and the third region is a suspected fire region. Based on the smoke features extracted from the visible light image of the target area, the third-state region undergoes secondary labeling, dividing it into first-state regions and second-state regions. For the first-state regions, the monitoring frequency is dynamically adjusted based on the potential risk of the target area. For the second-state regions, a graded fire early warning is implemented based on the fire spread trend.

[0016] This invention first acquires the accuracy characterization parameters of the thermal imaging sensor and conducts a monitoring accuracy assessment. When the assessment results do not meet the preset monitoring requirements, the sensor's monitoring accuracy is optimized and iteratively evaluated, thereby ensuring the basic reliability and accuracy of thermal imaging data acquisition and providing a stable and reliable data prerequisite for subsequent fire identification. By introducing environmental gas phase parameters collected in real time by a meteorological sensor to correct the infrared radiation intensity attenuation of the thermal imaging data, interference errors caused by environmental factors on infrared signal transmission can be effectively eliminated, thereby improving the authenticity and accuracy of thermal imaging data and avoiding misjudgment or missed detection of fires due to environmental interference. Based on the corrected high-quality thermal imaging data, the target area is dynamically divided into regions, distinguishing between normal pixel areas and abnormal pixel areas and marking them as first, second, and third state areas, respectively. This is combined with fire... The adaptive matching result of the judgment threshold enables accurate initial judgment of the state of abnormal areas, thereby achieving refined hierarchical identification of the fire status of the target area. This distinguishes between three different scenarios: no fire, confirmed fire, and suspected fire, to adapt to differentiated handling logic. Then, based on the smoke features extracted from visible light images, the suspected fire third-state area is marked with a secondary state, further eliminating false fires and identifying real fires. At the same time, for the no-fire first-state area, the monitoring frequency is dynamically adjusted according to potential risks to balance monitoring efficiency and resource consumption. For the confirmed fire second-state area, a graded fire warning is implemented according to the fire spread trend, thereby significantly improving the accuracy, timeliness, and pertinence of fire warnings. This achieves low false alarm rate, high reliability, and efficient response of intelligent fire warning under multimodal data fusion, effectively improving the overall performance of early fire monitoring and warning.

[0017] Accuracy characterization parameters include pixel responsivity, noise equivalent temperature difference, blind pixel rate, and response nonuniformity coefficient. The steps for assessing monitoring accuracy and determining whether the assessment results meet the preset monitoring requirements include: obtaining the current measured values ​​of each accuracy characterization parameter of the thermal imaging sensor, and calculating the monitoring accuracy coefficient of the thermal imaging sensor according to the preset weights corresponding to each accuracy characterization parameter. The specific formula is as follows: In the formula, This represents the monitoring accuracy coefficient of the thermal imaging sensor. The normalized pixel responsivity is defined as the sensitivity of a thermal imaging sensor pixel to incident radiation power, i.e., the output signal voltage generated per unit of radiation power input. It is typically obtained by pointing the sensor at a precisely temperature-controlled blackbody radiation source and measuring the rate of change of its output signal at different blackbody temperatures. The noise-equivalent temperature difference (NETD), after normalization, is a core indicator for evaluating the system's temperature sensitivity. It represents the minimum temperature difference between the target and the background required for the system to achieve a signal-to-noise ratio (SNR) of 1. Its calculation is based on the ratio of sensor noise to response rate. Specifically, it involves first measuring the root mean square (RMS) noise voltage of the sensor against a uniform background, then combining this with the system's response rate to a unit temperature change, and finally calculating the NETD value. A smaller NETD value indicates a stronger ability to detect subtle temperature differences. The normalized blind pixel rate refers to the percentage of invalid pixels with abnormal responses in a thermal imaging sensor out of the total number of pixels. According to the national standard (GB / T17444-2013), the specific determination method is as follows: pixels with a response rate lower than half of the average response rate of all effective pixels are determined as dead pixels, and pixels with a noise voltage greater than twice the average noise voltage of all effective pixels are determined as overheated pixels. The blind pixel rate is obtained by adding the number of these two types of blind pixels and dividing by the total number of pixels in the detector. The response nonuniformity coefficient, after normalization, is a parameter describing the consistency of the response of each pixel to uniform irradiation. It reflects the degree of difference in sensitivity between pixels. It is obtained by collecting the response values ​​of all effective pixels under uniform blackbody irradiation conditions and quantifying it by calculating the ratio of the standard deviation of these response values ​​to the average response value (i.e., the coefficient of variation). The specific calculation formula is: Response Nonuniformity = (Standard Deviation of Response Values ​​of All Effective Pixels / Average Response Values ​​of All Effective Pixels) × 100%. The smaller this value, the better the response consistency among the pixels of the detector, and the less the image quality is affected by fixed pattern noise. , , and These represent the preset weights corresponding to each precision characterization parameter, and The monitoring accuracy coefficient is compared with the preset monitoring accuracy threshold. If the monitoring accuracy coefficient is lower than the preset monitoring accuracy threshold, it is marked as not meeting the preset monitoring requirements; if the monitoring accuracy coefficient is not lower than the preset monitoring accuracy threshold, it is marked as meeting the preset monitoring requirements.

[0018] This invention uses pixel response rate, noise equivalent temperature difference, blind pixel rate, and response non-uniformity coefficient as core accuracy characterization parameters for thermal imaging sensors, assigning preset weights to each. A unified formula is used to quantify and calculate the monitoring accuracy coefficient. This allows for a comprehensive, quantitative, and standardized evaluation of the monitoring accuracy of thermal imaging sensors from multiple dimensions, including response sensitivity, temperature detection capability, pixel effectiveness, and response consistency. This avoids the bias and subjectivity of single-parameter evaluations, achieving a precise characterization of sensor monitoring performance. By normalizing each parameter and assigning weights based on their impact on monitoring accuracy, it ensures that parameters of different magnitudes and physical meanings can be reasonably weighted and integrated within the same evaluation system, thereby improving the scientific validity and comparability of the monitoring accuracy coefficient. By clarifying the physical meaning, measurement methods, and judgment criteria of noise equivalent temperature difference, blind element rate, and response non-uniformity coefficient, especially by introducing national standards to standardize the definition of blind elements and using the coefficient of variation to quantify response non-uniformity, the standardization, reproducibility, and authority of the monitoring accuracy assessment process can be guaranteed, effectively reducing measurement errors and judgment ambiguities. By comparing the calculated monitoring accuracy coefficient with the preset threshold, the system can automatically and objectively judge whether the sensor meets the monitoring requirements, thus providing a clear and reliable basis for subsequent accuracy optimization. This ensures the quality of thermal imaging data acquisition from the source, laying a stable and accurate data foundation for subsequent fire identification, area division, and early warning decisions, and fundamentally improving the reliability and stability of the entire intelligent fire early warning system.

[0019] The steps for adaptive matching of fire detection thresholds include: marking the difference between the minimum resolvable pixel count of the thermal imaging sensor and the preset baseline resolvable pixel count as the pixel resolution requirement; matching the preset deviation level intervals corresponding to each pixel resolution requirement range based on the pixel resolution requirement, determining the deviation level interval where the current pixel resolution requirement is located, and obtaining the area threshold adjustment coefficient corresponding to the deviation level interval; calculating the unit pixel spatial scale factor by multiplying the pixel size of the thermal imaging sensor and the current actual monitoring distance; calculating the over-limit pixel area threshold adjustment amount by multiplying the area threshold adjustment coefficient and the unit pixel spatial scale factor; and dynamically summing and correcting the preset over-limit pixel area base threshold based on the over-limit pixel area threshold adjustment amount to obtain the over-limit pixel area threshold used for fire detection.

[0020] This invention defines the difference between the minimum resolvable pixel count of the thermal imaging sensor and the preset benchmark resolvable pixel count as the pixel resolution requirement. This accurately quantifies the difference between the sensor's actual resolution capability and the preset benchmark requirement, providing an intuitive and hardware-performance-aligned quantitative basis for adaptive adjustment of the fire detection threshold. This avoids the problem of mismatched detection standards due to differences in sensor resolution performance. By matching the corresponding deviation level range based on the pixel resolution requirement and obtaining the area threshold adjustment coefficient, standardized quantification and adaptive adjustment of the resolution capability deviation can be achieved, allowing the threshold adjustment to accurately respond to the actual level of the sensor's resolution capability. By multiplying the pixel size of the thermal imaging sensor by the current actual monitoring distance to calculate the unit pixel spatial scale factor, the actual physical space coverage corresponding to a single pixel at different monitoring distances can be objectively reflected, thus solving the problem that a fixed threshold cannot adapt to different monitoring distances. This addresses the challenge of scene adaptation for different sensor pixel sizes. By multiplying the area threshold adjustment coefficient by the unit pixel spatial scale factor to obtain the over-limit pixel area threshold adjustment amount, it achieves the fusion and quantitative calculation of multiple key factors such as sensor resolution performance, hardware pixel parameters, and actual monitoring distance, thus providing a precise adjustment value for dynamic threshold correction. Finally, this adjustment amount is used to dynamically sum and correct the preset over-limit pixel area base threshold, obtaining an over-limit pixel area threshold adapted to the current hardware and monitoring scene. This makes the fire judgment threshold no longer a fixed value, but a dynamic parameter that can be adaptively adjusted in real time according to sensor performance and monitoring distance. This fundamentally improves the scene adaptability, accuracy, and rationality of the fire judgment standard, effectively reducing the probability of misjudgment and missed judgment of fires caused by different hardware parameters and monitoring conditions, and providing a more scientific and reliable judgment benchmark for subsequent regional status division, fire confirmation, and graded early warning.

[0021] The steps for optimizing the monitoring accuracy of the thermal imaging sensor include: if the blind pixel rate of the thermal imaging sensor is not lower than the preset blind pixel rate threshold, then the blind pixel compensation intensity and the response gain coefficient are dynamically adjusted; if the blind pixel rate of the thermal imaging sensor is lower than the preset blind pixel rate threshold, then the response gain coefficient is dynamically adjusted.

[0022] The steps for dynamically adjusting the blind pixel compensation intensity include: marking the difference between the blind pixel rate and the blind pixel rate threshold as the blind pixel compensation demand; recording the difference between the monitoring accuracy threshold and the monitoring accuracy coefficient of the thermal imaging sensor as the monitoring accuracy deviation; matching the monitoring accuracy deviation with the preset deviation level intervals corresponding to each monitoring accuracy deviation range, and obtaining the blind pixel compensation adjustment coefficient corresponding to the deviation level interval; calculating the blind pixel compensation adjustment ratio by multiplying the blind pixel compensation adjustment coefficient and the blind pixel compensation demand; and calculating the blind pixel compensation adjustment ratio by multiplying the preset blind pixel compensation unit. The intensity of blind pixel compensation is adjusted dynamically based on the intensity of blind pixel compensation. This is to suppress the interference of discrete dead pixels on temperature measurement and imaging accuracy. The intensity of blind pixel compensation refers to the strength of correction processing for blind pixels (including dead pixels with low response rate or overheated pixels with excessive noise) in thermal imaging sensors. Usually, the influence of blind pixels on image quality and temperature measurement accuracy is suppressed by using the response values ​​of adjacent normal pixels for interpolation replacement or algorithm compensation. Its function is to eliminate the interference of discrete dead pixels in the image and ensure imaging integrity and temperature measurement accuracy.

[0023] The steps for dynamically adjusting the response gain coefficient include: matching the monitoring accuracy deviation with a preset deviation level range corresponding to each monitoring accuracy deviation range; obtaining the response gain adjustment level corresponding to the deviation level range; and dynamically adjusting the pixel response gain of the thermal imaging array based on the response gain adjustment level. The response gain adjustment level refers to dividing the amplification factor of the pixel response signal into several preset discrete levels or continuous intervals according to the magnitude of the monitoring accuracy deviation. Each level corresponds to a specific gain adjustment range and amplification factor to achieve graded adaptive adjustment of the sensor sensitivity. The response gain adjustment levels include low gain, medium gain, high gain, and super gain. The low gain level... The first accuracy setting has a deviation range of <5% and a gain adjustment factor of 1.2x. This setting is designed for scenarios with minor accuracy deviations and is a fine-tuning setting. The adjustment range is limited to a weak amplitude gain compensation range, only performing a small linear amplification of the pixel response signal without changing the signal's floor noise level. It is suitable for situations where the monitoring accuracy is close to the threshold and only slight calibration of pixel response deviations is required. The 1.2x fixed gain factor achieves a small sensitivity enhancement, avoiding excessive amplification that could lead to noise increase, and accurately correcting minor accuracy gaps. The second accuracy setting has a deviation range of 5% ≤ deviation <15% and a gain adjustment factor of 1.5x. This setting is designed for scenarios with moderate accuracy deviations and is a fine-tuning setting. In the standard calibration setting, the adjustment range covers the standard gain compensation range, providing moderate linear amplification of the pixel response signal while balancing effective signal amplification and noise suppression. It is suitable for scenarios with uneven pixel response and moderate accuracy degradation caused by weak noise interference. A 1.5x gain multiplier steadily improves sensor sensitivity, quickly bringing monitoring accuracy back to the acceptable range. The high-gain setting has a monitoring accuracy deviation range of 15% ≤ deviation < 30%, with a gain adjustment multiplier of 2.0x. This setting is designed for scenarios with more severe accuracy deviations and is a powerful calibration setting. Its adjustment range covers a large gain compensation range, providing high-intensity linear amplification of the pixel response signal, focusing on amplifying weak effective thermal radiation signals and pressure... To mitigate moderate noise interference, this mode is suitable for situations where pixel response deviations are large and localized imaging distortions cause significant accuracy degradation. A 2.0x gain multiplier significantly enhances sensor sensitivity, quickly compensating for severe accuracy gaps. The super-gain mode monitors accuracy deviations ≥30%, with a 2.5x gain adjustment. This mode is designed for extreme accuracy deviation scenarios and represents an extreme enhancement level. Its adjustment range covers a very large gain compensation interval, providing full-scale linear amplification of the pixel response signal. It maximizes the amplification of weak effective signals and counteracts the sharp drop in accuracy caused by strong noise and large-area pixel response anomalies. This mode is suitable for extreme situations where monitoring accuracy is severely inadequate and extreme sensitivity improvements are required.A 5x gain multiplier maximizes sensor sensitivity, quickly reversing monitoring accuracy failures. Four levels of gain, through gradient amplification and precise adjustment ranges, enable graded adaptive adjustment of the thermal imaging sensor's pixel response gain. This balances the adjustment needs of different accuracy deviation scenarios with the core relationship between signal amplification and noise suppression, steadily improving thermal imaging monitoring accuracy. Pixel response gain refers to the amplification factor of the sensor pixel output signal. Dynamically adjusting the gain adjusts the pixel's response sensitivity, optimizing the sensor's dynamic range, enhancing the detection capability of subtle temperature changes, and preventing overload from strong radiation signals, thereby improving overall monitoring accuracy and adaptability.

[0024] This invention addresses the optimization steps for monitoring accuracy of thermal imaging sensors. It employs differentiated control based on whether the blind pixel rate reaches a preset threshold. When the blind pixel rate is not lower than the preset threshold, dynamic adjustment of the blind pixel compensation intensity and response gain coefficient is performed simultaneously. Dynamic adjustment of the response gain coefficient is only performed when the blind pixel rate meets the threshold. This achieves scenario-specific adaptation of the accuracy optimization strategy and efficient resource utilization, thereby avoiding over-compensation or under-control problems caused by indiscriminate optimization. During the dynamic adjustment of the blind pixel compensation intensity, the difference between the blind pixel rate and the blind pixel rate threshold is defined as the blind pixel compensation demand, and the difference between the monitoring accuracy threshold and the monitoring accuracy coefficient is recorded as the monitoring accuracy deviation. The blind pixel compensation adjustment coefficient is obtained by matching the corresponding deviation level range based on the monitoring accuracy deviation, and the blind pixel compensation adjustment ratio and blind pixel compensation adjustment intensity are obtained sequentially through product calculation. This dynamically adjusts the blind pixel compensation intensity of the thermal imaging sensor, correcting dead pixels and over-corrected pixels through interpolation replacement of adjacent normal pixels or algorithmic compensation. By eliminating blind pixels with abnormal responses, such as thermal pixels, interference from discrete dead pixels on temperature measurement accuracy and imaging quality is effectively suppressed, ensuring the integrity of thermal imaging images and the accuracy of temperature monitoring. During the dynamic adjustment of the response gain coefficient, four adjustment levels—low, medium, high, and ultra-high—are defined based on the monitoring accuracy deviation matching the corresponding deviation level range. These four levels precisely adapt to different degrees of monitoring accuracy attenuation while strictly balancing the core relationship between effective signal amplification and noise suppression. By dynamically adjusting the amplification factor of the pixel output signal, the sensor's dynamic range is optimized, the ability to detect weak temperature changes is enhanced, and overload of strong radiation signals is avoided. This steadily repairs sensor accuracy defects and continuously improves monitoring accuracy, providing a solid accuracy guarantee for thermal imaging data acquisition from a hardware performance calibration perspective. This lays a reliable hardware foundation for subsequent fire discrimination threshold matching, area status division, and graded fire early warning, comprehensively improving the monitoring stability and discrimination accuracy of the multimodal data fire intelligent early warning system.

[0025] Environmental gaseous parameters include water molecule concentration, aerosol concentration, and carbon dioxide concentration. The steps for infrared radiation intensity attenuation correction include: obtaining the current measured values ​​of each environmental gaseous parameter within the target area, and calculating the environmental meteorological interference coefficient of the target area by combining the preset weights corresponding to each environmental gaseous parameter. The specific formula is as follows: In the formula, The environmental meteorological interference coefficient for the target area. , and These represent the water molecule concentration, aerosol concentration, and carbon dioxide concentration after normalization. , and These represent the preset weights corresponding to each environmental gas phase parameter, and The process involves: determining whether the environmental meteorological interference coefficient exceeds a preset meteorological interference threshold; if so, marking the difference between the environmental meteorological interference coefficient and the preset meteorological interference threshold as the meteorological interference excess; matching the meteorological interference excess with a preset meteorological adjustment mapping relationship to obtain the meteorological adjustment benchmark coefficient, which is a preset correspondence between each meteorological interference excess range and each meteorological adjustment benchmark coefficient; matching the current monitoring distance corresponding to each pixel in the thermal imaging data with a preset distance attenuation mapping relationship to obtain the distance attenuation coefficient of each pixel, which is a preset correspondence between each pixel's monitoring distance and each distance attenuation coefficient; calculating the meteorological impact adjustment intensity of each pixel by multiplying the meteorological adjustment benchmark coefficient and the distance attenuation coefficient; and using the meteorological impact adjustment intensity to correct the original infrared radiation intensity of each pixel in the thermal imaging data to obtain the true infrared radiation intensity of each pixel; if the environmental meteorological interference level does not exceed the preset interference threshold, the original infrared radiation intensity is directly used as the true infrared radiation intensity of each pixel.

[0026] This invention selects water molecule concentration, aerosol concentration, and carbon dioxide concentration as core environmental gaseous parameters. Combining these parameters with preset weights, a unified formula is used to quantify the environmental meteorological interference coefficient of the target area. This allows for a comprehensive quantitative characterization of the interference from three key gaseous media—water vapor, aerosols, and carbon dioxide—on infrared radiation transmission. This enables accurate and quantitative assessment of the degree of interference from environmental meteorological factors, avoiding the biased corrections caused by single environmental parameter assessments and providing a scientifically reliable quantitative basis for infrared radiation intensity attenuation correction. By comparing the environmental meteorological interference coefficient with a preset meteorological interference threshold, it distinguishes between scenarios where meteorological interference exceeds or does not exceed the limit. When interference exceeds the limit, the difference between the interference coefficient and the threshold is marked as an excess of meteorological interference. A meteorological adjustment benchmark coefficient is obtained based on a preset meteorological adjustment mapping relationship, enabling graded adaptive adjustment of the degree of meteorological interference exceeding the limit. This ensures that the correction intensity accurately corresponds to the actual interference level, effectively avoiding over- or under-correction. Simultaneously, based on the actual monitoring data of each pixel within the thermal imaging data... The distance attenuation coefficient is obtained by matching the distance to a preset distance attenuation mapping relationship. The meteorological adjustment benchmark coefficient is multiplied by the distance attenuation coefficient to obtain the meteorological influence adjustment intensity of each pixel. This is used to refine the original infrared radiation intensity point by point to obtain the true infrared radiation intensity. When the meteorological interference does not exceed the limit, the original infrared radiation intensity is directly used as the true value. This fully integrates the dual effects of absorption attenuation by the environmental gaseous medium and spatial distance transmission attenuation, realizing personalized attenuation correction by pixel and scene. It also simplifies the correction process and ensures monitoring and processing efficiency in low-interference scenarios. This effectively eliminates the attenuation interference caused by environmental meteorology and transmission distance on the thermal imaging infrared signal, restores the true and accurate infrared radiation information of the target area, and greatly improves the authenticity and reliability of thermal imaging data. This provides accurate core data support for subsequent dynamic classification of area categories, fire status marking, and graded intelligent early warning. From the data preprocessing level, it reduces the risk of misjudgment and missed judgment of fire caused by environmental factors, and significantly improves the environmental adaptability and overall monitoring stability of the multimodal data fire intelligent early warning method.

[0027] The steps for dynamically classifying the target area based on the corrected thermal imaging data include: marking pixels whose actual infrared radiation intensity does not exceed a preset radiation intensity threshold as normal pixels, and marking pixels whose actual infrared radiation intensity exceeds the preset radiation intensity threshold as abnormal pixel regions; merging spatially adjacent pixels with the same state through connected component analysis to generate normal pixel regions and abnormal pixel regions; and dynamically marking the state of each abnormal pixel region to generate second-state regions and third-state regions, including: determining whether the pixel area of ​​each abnormal pixel region exceeds the threshold for the area of ​​over-limit pixels; if so, marking the corresponding abnormal pixel region as a second-state region; otherwise, marking the corresponding abnormal pixel region as a third-state region.

[0028] This invention accurately distinguishes between normal and abnormal pixels by comparing the actual infrared radiation intensity with a preset radiation intensity threshold. Pixels below the threshold are marked as normal, while those exceeding the threshold are marked as abnormal. This classification method directly relies on core data corrected for environmental and distance factors, enabling precise capture of abnormal temperature signals at the pixel level. This effectively avoids misjudgments of pixel states caused by distortion of the original data, laying a precise pixel-level foundation for subsequent region classification. Through connected component analysis, spatially adjacent pixels with the same state are connected and merged to generate complete normal and abnormal pixel regions. This effectively solves the problem of misjudging isolated abnormal pixels (potentially caused by noise or minor local interference) as fire hazards, achieving a transformation from pixel-level identification to region-level integration. This makes region classification more closely match the actual spatial distribution characteristics of the target area, improving the rationality and completeness of region classification. When dynamically marking abnormal region states, the threshold for the area of ​​pixels exceeding the limit obtained through adaptive matching is used as the judgment criterion. By judging whether the pixel area of ​​each abnormal pixel region exceeds the threshold... This threshold marks abnormal areas exceeding the limit as second-state areas (confirmed fire areas) and areas within the limit as third-state areas (suspected fire areas). This judgment method relies on dynamic thresholds adapted to the current hardware and monitoring scenario. It can accurately locate areas with a certain size and actual fire characteristics, avoiding misjudging small abnormal areas as confirmed fires, and effectively capture small, nascent suspected fire areas, without overlooking early fire hazards. The entire process of dynamic classification of areas and marking of abnormal areas is progressive and logically closed-loop, from pixel-level screening to area-level integration, and then to graded marking of abnormal areas. It achieves refined and differentiated identification of the fire status of target areas, ensuring the accuracy of confirmed fire area judgment and providing clear target guidance for secondary verification of suspected fire areas. It effectively reduces the probability of fire misjudgment and missed judgment, and provides accurate and reliable area classification basis for subsequent secondary marking based on smoke characteristics, dynamic adjustment of monitoring frequency, and graded fire early warning, further improving the pertinence and reliability of the entire intelligent fire early warning method.

[0029] The steps for performing secondary labeling of the third-state regions include: extracting smoke feature vectors for each third-state region; specifically, after preprocessing the visible light image frame sequence corresponding to the third-state region, extracting the moving foreground region through Gaussian mixture background modeling, filtering smoke candidate regions by combining the color feature space model of smoke, extracting the texture features and edge contour features of the smoke candidate regions using the local binary mode operator and the gradient direction histogram operator respectively, and fusing the texture features and edge contour features at the feature layer; obtaining standard smoke feature vectors corresponding to each preset smoke template, which are obtained in advance by feature extraction from various smoke samples; calculating the cosine similarity between the smoke feature vectors of each third-state region and the standard smoke feature vectors corresponding to each preset smoke template to obtain the smoke feature matching coefficient between each third-state region and each smoke template, and selecting the maximum value as the smoke feature similarity coefficient of the third-state region; if the smoke feature similarity coefficient of any third-state region reaches the preset feature similarity threshold, the third-state region is labeled as the second-state region; if the smoke feature similarity coefficient of any third-state region does not reach the preset feature similarity threshold, the third-state region is labeled as the first-state region.

[0030] This invention performs secondary labeling of suspected third-state fire areas identified by thermal imaging. It preprocesses the visible light image frame sequence corresponding to the third-state area and uses Gaussian mixture background modeling to accurately extract the moving foreground region. Combined with a smoke color feature space model, it filters out candidate smoke regions. Then, it uses a local binary mode operator and a gradient direction histogram operator to extract the texture and edge contour features of the candidate smoke regions and performs feature layer fusion, thereby constructing a multi-dimensional smoke feature vector that comprehensively represents the motion, color, texture, and edge contour characteristics of smoke. This avoids the one-sidedness and recognition errors caused by single feature recognition, achieving complete and refined extraction of smoke-related features in suspected areas. By calculating the cosine similarity between the smoke feature vectors of each third-state area and the standard smoke feature vectors corresponding to various preset smoke samples, and selecting the maximum matching value as the smoke feature similarity coefficient for that area, it can objectively and quantitatively measure the feature fit between the suspected area and the real smoke, thus providing... The smoke detection system provides a stable and reliable quantitative basis for determining the authenticity of smoke, eliminating the subjectivity and instability caused by manual judgment or single-rule judgment. By comparing the smoke feature similarity coefficient with a preset feature similarity threshold, the third-state area that reaches the threshold is marked as the second-state area confirming a fire, while the area that does not reach the threshold is marked as the first-state area without a fire. This completes the accurate verification and identification of suspected fire areas, effectively eliminating false anomalies caused by non-fire factors such as sudden changes in illumination, dust interference, and water vapor disturbance, and significantly reducing the false alarm probability of the fire early warning system. The entire secondary marking process relies on the multi-feature fusion and similarity matching mechanism of visible light images to achieve multimodal cross-validation of suspected abnormal areas in thermal imaging, thereby significantly improving the accuracy and reliability of fire status determination. This provides more accurate and reliable regional status support for subsequent monitoring frequency adjustment of fire-free areas and graded early warning of confirmed fire areas, further strengthening the discrimination accuracy and environmental anti-interference capability of the intelligent fire early warning method under multimodal data fusion.

[0031] The steps for dynamically adjusting the monitoring frequency based on the potential risk of the target area include: acquiring thermal imaging data of each normal pixel area for each time frame, extracting time-series thermal radiation characteristic parameters, including the average infrared radiation intensity of each pixel in each normal pixel area and the rate of change and acceleration of change of the average infrared radiation intensity for each time frame; and calculating the potential risk value of the target area based on the time-series thermal radiation characteristic parameters and the preset risk weights corresponding to each time-series thermal radiation characteristic parameter, using the following formula: In the formula, The potential risk value for the target area. To normalize the average infrared radiation intensity, The normalized rate of change For normalized change acceleration, , and These represent the preset risk weights corresponding to each time series thermal radiation characteristic parameter, and The system determines whether the potential risk value reaches the preset potential danger threshold. If so, the difference between the potential risk value and the potential danger threshold is marked as the potential risk excess. Based on the potential risk excess, a preset monitoring frequency adjustment mapping relationship is matched to obtain the monitoring frequency adjustment coefficient. The monitoring frequency adjustment mapping relationship is a preset correspondence between each risk excess range and each monitoring frequency adjustment coefficient. The monitoring frequency for normal pixel areas is dynamically adjusted according to the monitoring frequency adjustment coefficient. If the potential risk value does not reach the preset potential danger threshold, no additional processing is performed.

[0032] This invention acquires thermal imaging data from each time frame of normal pixel areas, extracting the average infrared radiation intensity, the rate of change of the average infrared radiation intensity in each time frame, and the acceleration of change as temporal thermal radiation characteristic parameters. This allows for a comprehensive capture of the temporal variation characteristics of thermal radiation in normal areas from three dimensions: static temperature level, dynamic rate of change, and trend. This avoids the limitation of single static parameters failing to capture potential temperature anomalies, thus accurately identifying potential fire hazards in normal areas and providing comprehensive and dynamic feature support for potential risk assessment. By combining preset risk weights corresponding to each temporal thermal radiation characteristic parameter, a unified formula is used to quantify and calculate the potential risk value of the target area. This incorporates temporal characteristic parameters with different physical meanings and magnitudes into the same risk assessment system, achieving quantitative and standardized assessment of potential risks. This effectively avoids biases caused by subjective experience judgments, making potential risk assessment more scientific and objective. By comparing the potential risk value with a preset potential hazard threshold, scenarios that meet or do not meet the threshold are distinguished. In cases where the risk value exceeds the threshold, the difference between the risk value and the threshold is marked as the potential risk exceeding the threshold. A corresponding monitoring frequency adjustment coefficient is obtained based on the preset monitoring frequency adjustment mapping relationship, thereby dynamically adjusting the monitoring frequency in normal areas. This achieves precise matching between the monitoring frequency and the potential risk level; the higher the potential risk, the higher the monitoring frequency, allowing for timely detection of further temperature anomalies and early prevention of fires. If the potential risk value does not reach the threshold, no additional processing is performed, avoiding the waste of system computing power and resource consumption caused by over-monitoring, and achieving efficient and reasonable allocation of monitoring resources. The entire dynamic monitoring frequency adjustment process not only achieves dynamic monitoring and accurate prediction of potential risks in fire-free areas, early detection of nascent fire hazards, and compensates for the shortcomings of traditional fixed-frequency monitoring that easily misses early weak anomalies, but also balances monitoring accuracy and system operating efficiency through differentiated monitoring strategies. This further improves the full-process control of intelligent fire early warning, enhancing the foresight and reliability of the entire early warning method from the perspective of hazard prediction, and providing more targeted monitoring support for early fire prevention and control.

[0033] The steps for implementing graded fire early warning based on fire spread trends include: acquiring thermal imaging data of each time frame for each abnormal pixel area, extracting fire feature parameters in real time, including the average temperature, temperature change rate, average area, and area expansion rate of each abnormal pixel area; acquiring synchronously collected environmental wind speed and direction data for each abnormal pixel area; and inputting the fire feature parameters, environmental wind speed data, and environmental wind direction data into the fire spread prediction model to predict the fire spread direction, fire spread trend, and predicted burning area within a preset period. Fire spread prediction models refer to computational models that combine real-time fire data (temperature, area), environmental meteorological parameters (wind speed, wind direction), and geographic information (vegetation, topography) to quantitatively predict the direction, speed, and scope of fire spread over a future period using mathematical algorithms or artificial intelligence techniques. According to existing research, the main fire spread prediction models can be divided into two categories: traditional models based on combustion physics processes, such as FARSITE (Fire Area SIMULATOR) and FlamMap; and data-driven machine learning models, such as ConvLSTM (Convolutional Long Short-Term Memory Network), FIRA (Fire Intensity and SpRead forecAst), and Diffusion Models. The former has clear physical meaning and reliable accuracy, but relies on detailed fuel data and involves a large amount of computation; the latter has better flexibility and computational efficiency, and can integrate multi-source data such as satellite remote sensing, but still faces challenges in cross-regional generalization ability; the predicted fire area is compared with the preset area emergency warning threshold: if the predicted fire area exceeds the area emergency warning threshold, an emergency warning is triggered; if the predicted fire area does not exceed the emergency warning threshold, a primary warning is triggered, and the current fire information and fire prediction results are synchronized to the fire intelligent monitoring platform. When an emergency warning is triggered, it indicates a high risk of rapid fire spread. The monitoring platform immediately pushes alarm information with the highest priority, automatically displaying the fire location, current fire area, abnormal area temperature, and predicted spread path in a pop-up window. At the same time, it activates on-site audible and visual alarm equipment, forcibly notifying the fire department to activate the emergency plan, organize personnel evacuation, and carry out firefighting. When a primary warning is triggered, it indicates that an initial open flame has been confirmed, but the fire has not yet spread over a large area. While marking the fire location and continuously tracking it, the monitoring platform automatically sends firefighting instructions to the nearest on-duty personnel or security terminals, requiring them to bring firefighting equipment to the scene for early response, and continuously provides feedback on the progress of firefighting until the fire is extinguished.

[0034] This invention acquires thermal imaging data from multiple time frames of abnormal pixel regions, extracts multi-dimensional fire characteristic parameters such as average temperature, temperature change rate, average area, and area expansion rate in real time, and simultaneously collects environmental wind speed and direction data. This allows for comprehensive collection of core information for fire assessment from both the fire's own development trend and the influence of external meteorological environment, providing complete, multi-source, and reliable data support for accurate prediction of fire spread trends. This effectively avoids prediction deviations caused by single fire parameters or neglecting environmental interference. By simultaneously inputting the aforementioned fire characteristic parameters and environmental wind field data into the fire spread prediction model, it can intelligently deduce and output the fire spread direction, spread trend, and predicted burning area within a preset period. This enables forward-looking and quantitative prediction of fire development trends, allowing for early understanding of fire spread risks and evolution patterns, and providing a scientific and accurate basis for graded early warning decisions. By closely comparing the predicted burning area with the preset area… The system compares the thresholds for emergency warnings and differentiates between them to trigger emergency and primary warnings. Emergency warnings are for high-risk scenarios where fires are spreading rapidly. They are pushed to the monitoring platform with the highest priority, displaying key information such as the fire location, fire area, temperature, and spread path. Simultaneously, on-site audible and visual alarms are activated, and the fire department is forcibly notified to activate emergency plans, organize personnel evacuation, and carry out firefighting. Primary warnings are for fires that are initially controllable. While the platform continuously tracks and marks the fire, it issues early firefighting instructions to the nearest on-duty personnel and requires real-time feedback on the progress of the response. This achieves precise tiered matching between fire warnings and emergency response, avoiding the waste of resources caused by treating minor fires as major ones and preventing the escalation of disasters caused by treating major fires as minor ones. This significantly improves the response efficiency, targeted response, and emergency reliability of intelligent fire warnings, effectively reducing the loss of life and property caused by fires, and comprehensively strengthening the practical application capabilities and full-process emergency management level of multimodal data-based intelligent fire warning methods.

[0035] like Figure 3The diagram shows the structure of a fire intelligent early warning system based on multimodal data provided in this application embodiment. It includes: a monitoring accuracy control module, a radiation intensity correction module, a regional status discrimination module, and a fire classification early warning module. The monitoring accuracy control module acquires the accuracy characterization parameters of the thermal imaging sensor and performs a monitoring accuracy assessment. It determines whether the monitoring accuracy assessment result meets the preset monitoring requirements. If it does, it performs adaptive matching of the fire discrimination threshold; if not, it optimizes the monitoring accuracy of the thermal imaging sensor and re-performs the monitoring accuracy assessment until the monitoring accuracy assessment result meets the preset monitoring requirements. The radiation intensity correction module corrects the infrared radiation intensity attenuation of the thermal imaging data collected by the thermal imaging sensor based on the environmental gas phase parameters collected in real time by the meteorological sensor in the target area. The system obtains corrected thermal imaging data. A region status discrimination module dynamically classifies the target area based on the corrected thermal imaging data, generating normal and abnormal pixel regions. Normal pixel regions are marked as first-state regions. Based on the dynamic region classification results and the adaptive matching results of the fire discrimination threshold, the status of each abnormal pixel region is dynamically marked, generating second and third-state regions. A fire classification and early warning module performs secondary labeling of the third-state region based on the smoke features extracted from the visible light image of the target area, dividing it into first-state and second-state regions. For the first-state region, the monitoring frequency is dynamically adjusted based on the potential risk of the target area. For the second-state region, a graded fire warning is issued based on the fire spread trend.

[0036] Through the above description of the implementation methods, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the above functions can be divided into different functional modules to complete all or part of the functions described above.

[0037] In the embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between devices or units, and may be electrical, mechanical, or other forms.

[0038] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A fire intelligent early warning method based on multimodal data, characterized in that, Includes the following steps: Acquire the accuracy characterization parameters of the thermal imaging sensor and perform monitoring accuracy evaluation. Determine whether the monitoring accuracy evaluation result meets the preset monitoring requirements. If it does, perform adaptive matching of the fire discrimination threshold. If it does not, perform monitoring accuracy optimization on the thermal imaging sensor and re-perform the monitoring accuracy evaluation until the monitoring accuracy evaluation result meets the preset monitoring requirements. Based on the environmental gas phase parameters collected in real time by the meteorological sensor in the target area, the thermal imaging data collected by the thermal imaging sensor is corrected by infrared radiation intensity attenuation to obtain the corrected thermal imaging data. Based on the corrected thermal imaging data, the target area is dynamically divided into regions, generating normal pixel regions and abnormal pixel regions. Normal pixel regions are marked as first state regions. Based on the dynamic division results of region categories and the adaptive matching results of fire discrimination threshold, the state of each abnormal pixel region is dynamically marked, generating second state regions and third state regions. Based on the smoke features extracted from the visible light image of the target area, the third state region is subjected to secondary labeling of the region state and divided into the first state region and the second state region. For the first state region, the monitoring frequency is dynamically adjusted based on the potential risks in the target region. For the second-state area, a graded fire warning is implemented based on the fire spread trend.

2. The fire intelligent early warning method based on multimodal data as described in claim 1, characterized in that: The accuracy characterization parameters include pixel responsivity, noise equivalent temperature difference, blind pixel rate, and response nonuniformity coefficient. The step of conducting a monitoring accuracy assessment and determining whether the monitoring accuracy assessment result meets the preset monitoring requirements includes: The current measured values ​​of each accuracy characterization parameter of the thermal imaging sensor are obtained, and the monitoring accuracy coefficient of the thermal imaging sensor is calculated according to the preset weights corresponding to each accuracy characterization parameter. The monitoring accuracy coefficient is compared with a preset monitoring accuracy threshold. If the monitoring accuracy coefficient is lower than the preset monitoring accuracy threshold, it is marked as not meeting the preset monitoring requirements. If the monitoring accuracy coefficient is not lower than the preset monitoring accuracy threshold, it is marked as meeting the preset monitoring requirements.

3. The intelligent fire early warning method based on multimodal data as described in claim 1, characterized in that: The step of performing adaptive matching of fire detection thresholds includes: The difference between the minimum resolvable pixel count of the thermal imaging sensor and the preset benchmark resolvable pixel count is marked as the pixel resolution requirement. Based on the pixel resolution requirement matching the preset deviation level range corresponding to each pixel resolution requirement range, determine the deviation level range where the current pixel resolution requirement is located, and obtain the area threshold adjustment coefficient corresponding to the deviation level range. The spatial scale factor per unit pixel is calculated based on the pixel size of the thermal imaging sensor and the current actual monitoring distance. Based on the area threshold adjustment coefficient and the unit pixel spatial scale factor, the area threshold adjustment amount of the out-of-limit pixel is calculated. Based on the over-limit pixel area threshold adjustment amount, the preset over-limit pixel area base threshold is dynamically corrected to obtain the over-limit pixel area threshold used for fire detection.

4. The intelligent fire early warning method based on multimodal data as described in claim 1, characterized in that: The step of optimizing the monitoring accuracy of the thermal imaging sensor includes: If the blind pixel rate of the thermal imaging sensor is not lower than the preset blind pixel rate threshold, then dynamic adjustment of the blind pixel compensation intensity and dynamic adjustment of the response gain coefficient will be performed. If the blind pixel rate of the thermal imaging sensor is lower than the preset blind pixel rate threshold, then the response gain coefficient is dynamically adjusted. The steps for dynamically adjusting the blind element compensation intensity include: The difference between the blind pixel rate and the blind pixel rate threshold is marked as the blind pixel compensation requirement, and the difference between the monitoring accuracy threshold and the monitoring accuracy coefficient of the thermal imaging sensor is recorded as the monitoring accuracy deviation. Based on the monitoring accuracy deviation, a preset deviation level interval is matched for each monitoring accuracy deviation range, and the blind pixel compensation adjustment coefficient corresponding to the deviation level interval is obtained. Based on the blind pixel compensation adjustment coefficient and the blind pixel compensation requirement, the blind pixel compensation adjustment ratio is calculated. Based on the blind pixel compensation adjustment ratio and the preset blind pixel compensation unit, the blind pixel compensation adjustment intensity is calculated. Based on the blind pixel compensation adjustment intensity, the blind pixel compensation intensity of the thermal imaging sensor is dynamically adjusted. The steps for dynamically adjusting the response gain coefficient include: Based on the deviation level range corresponding to each preset monitoring accuracy deviation range, the response gain adjustment level corresponding to the deviation level range is obtained, and the pixel response gain of the thermal imaging array is dynamically adjusted based on the response gain adjustment level.

5. The intelligent fire early warning method based on multimodal data as described in claim 1, characterized in that: The environmental gaseous parameters include water molecule concentration, aerosol concentration, and carbon dioxide concentration; The steps for correcting infrared radiation intensity attenuation include: The current measured values ​​of each environmental gas phase parameter within the target area are obtained, and the environmental meteorological interference coefficient of the target area is calculated by combining the preset weights corresponding to each environmental gas phase parameter. If the environmental meteorological interference coefficient exceeds the preset meteorological interference threshold, the difference between the environmental meteorological interference coefficient and the preset meteorological interference threshold is marked as the meteorological interference excess. Based on the meteorological interference excess, a preset meteorological regulation mapping relationship is matched to obtain the meteorological regulation benchmark coefficient. Based on the current monitoring distance corresponding to each pixel in the thermal imaging data and the preset distance attenuation mapping relationship, the distance attenuation coefficient of each pixel is obtained. Based on the meteorological adjustment benchmark coefficient and the distance attenuation coefficient, the meteorological influence adjustment intensity of each pixel is calculated. The original infrared radiation intensity of each pixel in the thermal imaging data is corrected using the meteorological influence adjustment intensity to obtain the true infrared radiation intensity of each pixel. If the degree of environmental meteorological interference does not exceed the preset interference threshold, the original infrared radiation intensity is directly used as the true infrared radiation intensity of each pixel.

6. The intelligent fire early warning method based on multimodal data as described in claim 5, characterized in that: The step of dynamically classifying the target area based on the corrected thermal imaging data includes: Pixels whose actual infrared radiation intensity does not exceed the preset radiation intensity threshold are marked as normal pixels, and pixels whose actual infrared radiation intensity exceeds the preset radiation intensity threshold are marked as abnormal pixel regions. By connecting the pixels that are spatially adjacent and have the same state, we can generate normal pixel regions and abnormal pixel regions. The step of dynamically marking the state of each abnormal pixel region and generating the second and third state regions includes: Determine whether the pixel area of ​​each abnormal pixel region exceeds the threshold of the excessive pixel area. If so, mark the corresponding abnormal pixel region as a second state region; otherwise, mark the corresponding abnormal pixel region as a third state region.

7. The intelligent fire early warning method based on multimodal data as described in claim 1, characterized in that: The step of performing secondary labeling of the region state in the third state region includes: Extract the smoke feature vectors of each third-state region; Obtain the standard smoke feature vector corresponding to each preset smoke template; The smoke feature vector of each third state region is compared with the standard smoke feature vector corresponding to each preset smoke template by calculating the cosine similarity. The smoke feature matching coefficient between each third state region and each smoke template is obtained, and the maximum value is selected as the smoke feature similarity coefficient of the third state region. If the smoke feature similarity coefficient of any third state region reaches the preset feature similarity threshold, then the third state region is marked as the second state region; if the smoke feature similarity coefficient of any third state region does not reach the preset feature similarity threshold, then the third state region is marked as the first state region.

8. The intelligent fire early warning method based on multimodal data as described in claim 1, characterized in that: The steps for dynamically adjusting the monitoring frequency based on potential risks in the target area include: Acquire thermal imaging data of each normal pixel region at each time frame, and extract time-series thermal radiation characteristic parameters, including the average infrared radiation intensity of each pixel in each normal pixel region and the rate of change and acceleration of change of the average infrared radiation intensity at each time frame. Based on the time-series thermal radiation characteristic parameters and the preset risk weights corresponding to each time-series thermal radiation characteristic parameter, the potential risk value of the target area is calculated. Determine whether the potential risk value reaches a preset potential danger threshold. If so, mark the difference between the potential risk value and the potential danger threshold as the potential risk excess. Based on the potential risk exceedance range matched with the preset monitoring frequency adjustment mapping relationship, the monitoring frequency adjustment coefficient is obtained. The monitoring frequency adjustment mapping relationship is a preset correspondence between each risk exceedance range and each monitoring frequency adjustment coefficient. The monitoring frequency for the normal pixel region is dynamically adjusted according to the monitoring frequency adjustment coefficient. If the potential risk value does not reach the preset potential danger threshold, no additional processing will be performed.

9. The intelligent fire early warning method based on multimodal data as described in claim 1, characterized in that: The steps for implementing graded fire early warning based on the fire spread trend include: Acquire thermal imaging data of each time frame of each abnormal pixel region, and extract fire feature parameters in real time. The fire feature parameters include the average temperature, temperature change rate, average area and area expansion rate of each abnormal pixel region. Acquire synchronously collected environmental wind speed and direction data within each abnormal pixel region; The fire characteristic parameters, environmental wind speed data, and environmental wind direction data are input into the fire spread prediction model to predict the fire spread direction, fire spread trend, and predicted fire area within a preset period. The predicted fire area is compared with a preset area emergency warning threshold: if the predicted fire area exceeds the area emergency warning threshold, an emergency warning is triggered; if the predicted fire area does not exceed the emergency warning threshold, a primary warning is triggered.

10. A fire intelligent early warning system based on multimodal data, employing the fire intelligent early warning method based on multimodal data as described in any one of claims 1-9, characterized in that, include: The system includes a monitoring accuracy control module, a radiation intensity correction module, a regional status discrimination module, and a fire classification and early warning module. The monitoring accuracy control module is used to acquire the accuracy characterization parameters of the thermal imaging sensor and perform monitoring accuracy evaluation, determine whether the monitoring accuracy evaluation result meets the preset monitoring requirements, if it meets the requirements, perform adaptive matching of the fire discrimination threshold, if it does not meet the requirements, perform monitoring accuracy optimization on the thermal imaging sensor, and re-perform the monitoring accuracy evaluation until the monitoring accuracy evaluation result meets the preset monitoring requirements. The radiation intensity correction module is used to correct the infrared radiation intensity attenuation of the thermal imaging data collected by the thermal imaging sensor based on the environmental gas phase parameters collected in real time by the meteorological sensor in the target area, so as to obtain the corrected thermal imaging data. The region state discrimination module is used to dynamically divide the target region into region categories based on the corrected thermal imaging data, generate normal pixel regions and abnormal pixel regions, mark the normal pixel regions as the first state region, and dynamically mark the state of each abnormal pixel region based on the dynamic region category division result and the adaptive matching result of the fire discrimination threshold, and generate the second state region and the third state region. The fire classification and early warning module is used to perform secondary labeling of the third state area based on the smoke features extracted from the visible light image of the target area, dividing it into a first state area and a second state area; for the first state area, the monitoring frequency is dynamically adjusted based on the potential risks of the target area; for the second state area, a graded fire warning is performed based on the fire spread trend.

Images