Intelligent fire monitoring method and platform based on computer vision

Abstract

The invention discloses an intelligent fire monitoring method and platform based on computer vision, and belongs to the technical field of computer vision. Through synchronous acquisition and fusion of visible light and thermal imaging video data, double-spectrum fire feature extraction and space-time alignment analysis are realized, and a preliminary fire event is generated by applying a multi-rule decision. And furthermore, multi-view space verification and external data association verification are carried out at the cloud to confirm a real fire behavior. And finally, accurately positioning the confirmed fire point, running a spreading model in combination with meteorological and geographic data to predict a fire trend, and automatically generating a structured early warning report containing a threat target and a disposal suggestion. According to the method, the accuracy, reliability and timeliness of fire identification are remarkably improved, full-process automation from early perception to intelligent auxiliary decision making is realized, and the method is suitable for fireproof safety monitoring of various complex environments.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a computer vision-based intelligent fire monitoring method and platform. Background Technology

[0002] Fire, as one of the most destructive public safety threats, poses a significant challenge to people's lives and property, the stable operation of society, and the ecological environment. Although traditional and modern fire monitoring technologies have been continuously developed, they still have significant limitations and are unable to meet the growing demand for precise security measures such as "early detection, early warning, and early response."

[0003] Initially, fire monitoring relied primarily on discrete point sensors, such as smoke detectors and heat detectors. While these devices were widely used in enclosed indoor environments, their inherent limitations were significant: limited detection range and numerous blind spots; response depended on the diffusion of smoke particles or hot air, resulting in significant alarm delays; and they were ineffective in detecting open or large spaces, as well as the smoldering stage of a fire. Furthermore, they were susceptible to interference from environmental factors such as dust, water vapor, and cooking fumes, leading to a high false alarm rate and making them unsuitable for complex open environments such as forests, warehouses, and industrial parks.

[0004] With the widespread adoption of video surveillance technology, fire visual monitoring methods based on visible light cameras have emerged. These methods identify fires by analyzing flame color features (such as red and yellow bright spots), dynamic textures (such as flickering and jumping), and smoke diffusion patterns in video images. Compared to traditional sensors, they achieve non-contact, wide-area visual coverage. However, relying on analysis of a single visible spectral band presents a fundamental bottleneck: High false alarm rate and environmental interference: The system is highly susceptible to misidentifying artificial light sources such as sunset, vehicle headlights, welding sparks, and high-intensity lighting, as well as natural phenomena such as clouds, fog, and dust storms, as fires. Complex lighting changes, shadows, and severe weather (such as rain, snow, and fog) can seriously interfere with image quality and the stability of the analysis algorithm.

[0005] The ability to detect early and concealed fires is weak: Visible light imaging relies on the light radiation of flames or dense smoke. For low-temperature smoldering (producing a small amount of light smoke) in the early stages of a fire, weak fire sources that are obscured (such as fires inside equipment), or at night when there is no obvious open flame, visible light cameras have difficulty effectively capturing features, resulting in missed reports or serious delays in alarms.

[0006] Limited intelligence and lack of situational awareness: Most existing solutions can only provide binary alarms for the presence or absence of fires, lacking the ability to dynamically analyze the precise location of the fire, the scale of the fire, and the direction and speed of its spread. Monitoring centers still need to manually screen a large number of alarm videos, which cannot provide critical auxiliary decision-making information for emergency command (such as optimal rescue routes and evacuation range estimates).

[0007] In recent years, some studies have attempted to introduce thermal imaging (infrared) technology to improve the reliability of fire detection. Thermal imaging generates temperature distribution images by sensing infrared radiation from the surface of objects. It is extremely sensitive to high-temperature targets, unaffected by visible light illumination conditions, and can penetrate a certain degree of smoke. However, standalone thermal imager applications or simple dual-spectrum systems still face challenges: on the one hand, using thermal imaging alone may misreport normal high-temperature objects such as industrial heat sources and high-temperature exhaust pipes as fires; on the other hand, existing solutions that simply combine visible light and thermal imaging often remain at the level of mechanical superposition of data or independent backend analysis, lacking an efficient front-end embedded real-time fusion and judgment mechanism, resulting in large system response delays. Simultaneously, the massive amounts of raw video data directly transmitted back to the cloud for processing place enormous pressure on network bandwidth and central server computing power, making large-scale, real-time deployment difficult.

[0008] Furthermore, existing system architectures are mostly siloed, with the perception, analysis, and decision-making processes separated. Monitoring points are only responsible for data collection, while intelligent analysis is entirely centralized in the cloud. This model will paralyze the system when faced with network instability or interruption, and it cannot meet the requirements for second-level real-time response in harsh environments such as forests and remote areas.

[0009] Therefore, how to build a fire monitoring platform that can achieve early and accurate identification, strong environmental robustness, real-time and efficient response, and intelligent situational assessment capabilities has become a key technical problem that urgently needs to be solved in this field. There is an urgent need for an innovative, integrated method and platform that tightly combines advanced multispectral sensing, edge intelligent computing, cloud-based deep analysis, and multidimensional data fusion. Summary of the Invention

[0010] This invention addresses the technical problems existing in the background art by proposing a computer vision-based intelligent fire monitoring method and platform.

[0011] To solve the technical problem, the technical solution of the present invention is as follows: A computer vision-based intelligent fire monitoring method, the method comprising: S1: Synchronously acquire visible light video data and thermal imaging video data of the monitored area; S2: Extract visible light fire features from the visible light video data and extract thermal imaging fire features from the thermal imaging video data; perform comprehensive analysis on the visible light fire features and the thermal imaging fire features based on a preset multi-rule decision method to generate preliminary fire event information; S3: Perform multi-source data fusion verification on the preliminary fire event information, and confirm the final fire information when the verification is successful; perform location and situation analysis on the confirmed final fire information, and generate and output a structured early warning report.

[0012] Furthermore, step S1 includes: By deploying multispectral sensing devices in the monitoring area, raw visible light video data and raw thermal imaging video data under the same monitoring field of view are collected simultaneously; the collected raw visible light video data and raw thermal imaging video data are preprocessed, including automatic exposure correction of visible light images and non-uniformity correction of thermal imaging images, to obtain preprocessed visible light video data and thermal imaging video data.

[0013] Furthermore, step S2 includes: The visible light video data is processed using a lightweight deep learning model to output visible light fire features that characterize flames or smoke. The thermal imaging video data is processed by dynamic temperature threshold segmentation and region growing algorithm to output thermal imaging fire features that characterize abnormally high temperature areas. A spatiotemporal mapping relationship is established, and the visible light fire features and the thermal imaging fire features are spatiotemporally aligned and logically fused to obtain a fused comprehensive feature criterion. The comprehensive feature criterion is analyzed according to a preset multi-rule decision method to generate preliminary fire event information including high confidence, medium confidence and low confidence.

[0014] Furthermore, the establishment of the spatiotemporal mapping relationship and the performance of spatiotemporal alignment and logical fusion specifically include: Based on the calibration parameters of the multispectral sensing device, the target position in the visible light fire feature is mapped to the thermal imaging coordinate system to obtain the mapped position; the Euclidean distance between the mapped position and the actual position of the abnormal high temperature area in the thermal imaging fire feature is calculated; when the Euclidean distance is less than a preset threshold and the visible light fire feature and the thermal imaging fire feature appear continuously in consecutive frames, it is determined that the feature matching is successful, and a corresponding matching success criterion is generated as the fused comprehensive feature criterion.

[0015] Furthermore, the preset multi-rule decision-making method includes: If the visible light fire feature identifies an open flame, and the temperature of the corresponding thermal imaging fire feature is higher than the first temperature threshold, then preliminary fire event information with a high confidence level is generated. If the visible light fire feature identifies smoke, and the temperature of the corresponding thermal imaging fire feature is higher than the ambient temperature and shows an upward trend, then preliminary fire event information with a medium confidence level is generated. If only the thermal imaging fire feature shows abnormally high temperature, but no corresponding visible light fire feature is found, then preliminary fire event information with a low confidence level is generated.

[0016] Furthermore, the process of using a lightweight deep learning model to process the visible light video data includes: Image frames from the visible light video data are input into the trained lightweight deep learning model; The lightweight deep learning model is a YOLOv5s object detection model improved based on the MobileNetV3 backbone network, and is trained using a mixed dataset including real flame images and synthetic smoke data; MobileNetV3 is a lightweight convolutional neural network architecture.

[0017] The lightweight deep learning model processes the input image frames and outputs: bounding boxes of flames or smoke targets, category labels, and visible light fire features with corresponding confidence levels.

[0018] Furthermore, the preliminary fire incident information is verified through multi-source data fusion, specifically including: Information parsing and verification: The received preliminary fire event information is parsed in the cloud, and multi-view spatial geometric verification is performed on related events from different monitoring points in the same geographical area; Multi-source correlation verification: Correlate at least one external monitoring data source with geographic information data to verify the physical and environmental correlation of the preliminary fire event information; Comprehensive judgment: Based on the results of the spatial geometric verification, physical and environmental correlation verification, a comprehensive judgment is made. When the preset confirmation conditions are met, the final fire information is output and confirmed.

[0019] Furthermore, the confirmed final fire information is used for location and situation analysis, specifically including: Precise positioning: Based on the final fire information, the fire location is marked on the digital map to obtain positioning results including geographic coordinates and surrounding environmental information; Situation simulation: Using the positioning results as input, combined with real-time meteorological data and geographical environment data, a fire spread model is run to generate fire spread trend prediction information; Early warning generation and push: Integrate the location results and the spread trend prediction information to generate a structured early warning report that includes disposal suggestions.

[0020] Furthermore, in the situation simulation, the positioning results, real-time meteorological data, and geographical environment data are input; the Rothermel surface fire spread model is used to calculate the fire spread rate in different directions; based on the spread rate, fire spread trend prediction information for a specified future time period is simulated and generated on a digital map. In the early warning generation and push, the location results and the spread trend prediction information are integrated, and risk analysis is performed based on the spread trend prediction information and a preset geographic information database to determine the key targets under threat and generate a structured early warning report including the key targets and corresponding handling suggestions. The structured early warning report includes: precise coordinates of the fire point, current estimated burned area, a map showing the predicted spread range for future time periods, a list of key threatened targets, and suggestions for priority rescue routes.

[0021] A computer vision-based intelligent fire monitoring platform, the platform being used to perform any of the methods described above, the platform comprising: The synchronous acquisition module is used to simultaneously acquire visible light video data and thermal imaging video data of the monitored area; An edge fusion analysis module, deployed at the monitoring site and communicatively connected to the synchronous acquisition module, is used to extract visible light fire features from the visible light video data and thermal imaging fire features from the thermal imaging video data; based on a preset multi-rule decision method, the visible light fire features and the thermal imaging fire features are comprehensively analyzed to generate preliminary fire event information; The cloud-based analysis module is communicatively connected to the edge fusion analysis module. It is used to receive the preliminary fire event information, perform multi-source data fusion verification on the preliminary fire event information, and confirm the final fire information. The early warning generation module is communicatively connected to the cloud-based analysis module. It is used to locate and analyze the confirmed final fire information, and generate and output the structured early warning report.

[0022] This application has the following advantages: This method utilizes dual-spectral fusion of visible light and thermal imaging to simultaneously acquire flame morphology and abnormal temperature characteristics, constructing a dual chain of evidence. This significantly improves the accuracy of fire identification and drastically reduces false alarms. Employing a collaborative architecture of real-time edge detection and deep cloud-based verification, it balances the timeliness of early warning with the rigor of analysis. The platform performs high-level verification through multi-view validation and correlation with external data, ensuring highly reliable output information. Beyond traditional alarms, it achieves precise fire location, physical model-based spread prediction, and automatically generates structured reports containing threat targets and response suggestions, forming a complete closed loop from perception and analysis to decision support. The entire solution has a clear technical path, strong engineering practicality, and is suitable for intelligent security control in various complex scenarios. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This application provides a technical roadmap for a computer vision-based intelligent fire monitoring method. Detailed Implementation

[0025] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0026] Example 1: Figure 1 A computer vision-based intelligent fire monitoring method, the method comprising: Step S1 includes: synchronously acquiring raw visible light video data and raw thermal imaging video data under the same monitoring field of view through multispectral sensing devices deployed in the monitoring area; preprocessing the acquired raw visible light video data and raw thermal imaging video data, including automatic exposure correction of visible light images and non-uniformity correction of thermal imaging images, to obtain preprocessed visible light video data and thermal imaging video data.

[0027] For example, the core objective of this step is to acquire high-quality, spatiotemporally aligned visible light and thermal imaging video data streams, laying a reliable data foundation for subsequent accurate feature extraction and fusion analysis. This step consists of two core components: hardware collaboration and data preprocessing.

[0028] 1. Hardware collaboration and synchronous acquisition of raw data; The platform's data acquisition end consists of a specially integrated multispectral sensing device. This device is not simply a collection of components, but rather an integrated design and precise calibration of the visible light camera and thermal imaging sensor in terms of physical structure and optical path, ensuring that both have the same monitoring field of view, that is, observing the exact same physical area.

[0029] The key to the acquisition process lies in synchronization. The device has a built-in synchronization triggering mechanism to ensure that every frame of visible light image and thermal imaging image is captured at the same millisecond-level timestamp. This eliminates dynamic target position deviations caused by time differences, which is a prerequisite for subsequent spatiotemporal alignment and logical fusion. The output of this stage is two raw, unoptimized video data streams: one is raw visible light video data reflecting the object's color and texture; the other is raw thermal imaging video data reflecting the object's surface temperature distribution.

[0030] 2. Data preprocessing; Using raw data directly for analysis may result in quality issues, therefore targeted preprocessing is necessary to improve the accuracy of feature extraction.

[0031] Automatic exposure correction for visible light images: Objective: To overcome the problem of overexposure or underexposure of images caused by drastic changes in ambient light, and to ensure that the visual features of targets such as flames and smoke can be clearly presented under any lighting conditions.

[0032] Method: The algorithm analyzes the overall brightness and contrast distribution of the image in real time. When the image is detected to be too bright overall, the exposure time or gain is automatically reduced to prevent loss of detail in bright areas; when the image is detected to be too dark overall, the exposure parameters are automatically increased to brighten details in dark areas. This is a dynamic and continuous adaptive optimization process designed to ensure that the output image always maintains the best visual effect and level of detail.

[0033] Non-uniformity correction of thermal imaging images: Objective: To eliminate the inherent noise of thermal imaging sensors caused by factors such as pixel response differences, lens radiation, and internal thermal disturbances. This noise manifests as fixed, unrealistic bright and dark stripes or spots on the image in uniform temperature scenes, which severely interferes with the measurement of the true temperature field and the identification of abnormally high temperature areas.

[0034] Method: The platform employs a real-time correction algorithm based on calibration parameters. This algorithm utilizes the correction coefficients of each pixel obtained before the device leaves the factory or periodically in a standard uniform temperature field. When processing each frame of raw thermal imaging data, the algorithm uses these coefficients to compensate and calibrate the response value of each pixel, thereby outputting a thermal imaging image with accurate temperature values ​​and uniform image quality, ensuring the reliability of the temperature data upon which subsequent dynamic temperature thresholding relies.

[0035] The output of step S1: After the hardware synchronous acquisition and software preprocessing described above, two optimized, spatiotemporally aligned visible light video data and thermal imaging video data are obtained. These two high-quality data streams will be fed in parallel into the next analysis step for feature extraction.

[0036] Step S2 includes: processing the visible light video data using a lightweight deep learning model to output visible light fire features representing flames or smoke; processing the thermal imaging video data using a dynamic temperature threshold segmentation and region growing algorithm to output thermal imaging fire features representing abnormally high temperature areas; establishing a spatiotemporal mapping relationship, aligning and logically fusing the visible light fire features and the thermal imaging fire features to obtain a fused comprehensive feature criterion; and analyzing the comprehensive feature criterion according to a preset multi-rule decision method to generate preliminary fire event information including high confidence, medium confidence, and low confidence levels.

[0037] The establishment of spatiotemporal mapping relationships, spatiotemporal alignment, and logical fusion specifically include: Based on the calibration parameters of the multispectral sensing device, the target position in the visible light fire feature is mapped to the thermal imaging coordinate system to obtain the mapped position; the Euclidean distance between the mapped position and the actual position of the abnormal high temperature area in the thermal imaging fire feature is calculated; when the Euclidean distance is less than a preset threshold and the visible light fire feature and the thermal imaging fire feature appear continuously in consecutive frames, it is determined that the feature matching is successful, and a corresponding matching success criterion is generated as the fused comprehensive feature criterion.

[0038] For example, this step is the core of dual-spectral feature fusion, aiming to solve a key problem: whether a suspected fire target identified from a visible light image and an abnormally high temperature area detected from a thermal imaging image are manifestations of the same physical entity in two different modalities; to achieve accurate judgment, the following three rigorous logical steps need to be executed: 1. Spatial coordinate mapping; Foundation: Relies on precise calibration parameters pre-computed by the multispectral sensing device. These precise calibration parameters define the geometric relationship between the visible light camera and the thermal imaging sensor, much like a precise conversion rule.

[0039] Action: When a fire feature is extracted from a frame of visible light image, the platform immediately converts the center point of the bounding box to the thermal imaging image coordinate system according to the aforementioned calibration parameters, obtaining a theoretical mapped position. This step ensures that positional information from two images with different viewpoints and resolutions can be compared under the same spatial reference.

[0040] 2. Feature matching degree calculation; Calculation: In the thermal imaging image, there are already abnormally high-temperature regions and their actual locations obtained through dynamic temperature threshold segmentation. The platform calculates the straight-line distance, i.e., the Euclidean distance, between the theoretically mapped location obtained in the previous step and the actual location of the high-temperature region.

[0041] Judgment: The distance is compared with a preset matching threshold. This threshold is set comprehensively based on actual conditions such as device installation height, viewing angle, and resolution. If the distance is less than the threshold, it indicates that the target identified by visible light and the high-temperature area detected by thermal imaging highly overlap in spatial location, thus passing the spatial consistency check.

[0042] 3. Time persistence verification and fusion criterion generation; Verification: Matching in a single frame may be accidental. The platform further introduces temporal verification. It requires that the state of successful spatial location matching must continuously appear in multiple consecutive video frames.

[0043] Logical Fusion and Output: The platform only determines a successful feature match when both spatial location matching and temporal persistence are simultaneously met. At this point, morphological evidence from visible light and temperature evidence from thermal imaging mutually corroborate each other in space and time. The platform will generate a successful match criterion, a comprehensive feature criterion integrating target spatial location, timestamp, visible light category confidence, and thermal imaging temperature characteristics, which will serve as direct input for subsequent multi-rule decision analysis.

[0044] In summary, this process rigorously achieves the spatiotemporal alignment and logical fusion of visible light features and thermal imaging features through a three-step process of spatial coordinate alignment, location distance judgment, and continuous time verification. This transforms two independent types of sensor information into a unified and reliable joint evidence, laying a solid foundation for the final fire confidence level decision.

[0045] The preset multi-rule decision-making method includes: If the visible light fire feature identifies an open flame, and the temperature of the corresponding thermal imaging fire feature is higher than the first temperature threshold, then preliminary fire event information with a high confidence level is generated. If the visible light fire feature identifies smoke, and the temperature of the corresponding thermal imaging fire feature is higher than the ambient temperature and shows an upward trend, then preliminary fire event information with a medium confidence level is generated. If only the thermal imaging fire feature shows abnormally high temperature, but no corresponding visible light fire feature is found, then preliminary fire event information with a low confidence level is generated.

[0046] For example, this stage marks the end of information fusion and preliminary assessment. Its core task is to conduct preliminary risk classification and characterization of potential fire events based on comprehensive feature criteria refined through spatiotemporal alignment and logical fusion, using a rule-based knowledge base based on the physical characteristics of fire. This decision-making method follows the principle that the more sufficient the evidence, the higher the level; the more typical the features, the clearer the judgment, dividing preliminary fire event information into three confidence levels: high, medium, and low. The specific rules are as follows: 1. High confidence level rules; Triggering condition: When the comprehensive feature criterion simultaneously satisfies the following two conditions: Morphological evidence: Visible light fire characteristics clearly identify open flames.

[0047] Temperature evidence: At the location corresponding to the open flame, thermal imaging fire characteristics show that the temperature in that area consistently exceeds the first temperature threshold.

[0048] Judgment Logic and Output: Open flame and sustained extremely high temperature are the two most direct and intense characteristics of the combustion process. Their simultaneous occurrence constitutes a strong chain of evidence for fire confirmation. Based on this, the platform generates preliminary fire event information with a high confidence level, indicating a very high probability of an ongoing open flame fire.

[0049] 2. Medium confidence level rules; Triggering condition: When the comprehensive feature criterion simultaneously satisfies the following two conditions: Morphological evidence: visible light fire characteristics identified smoke.

[0050] Temperature evidence: At the location corresponding to the smoke, thermal imaging fire features show that the temperature in the area is higher than the background temperature of the surrounding environment, and shows a clear upward trend over a period of time.

[0051] Judgment Logic and Output: Smoke is often a precursor or byproduct of fire. A single smoke image may originate from non-fire sources such as water vapor or dust. Therefore, this rule introduces dynamic temperature-assisted verification: the corresponding area must not only be relatively hot, but the temperature must also be rising. This aligns with the physical process of gradually increasing heat release in the early stages of a fire. This combination of smoke and temperature rise is considered a reliable early or potential fire indication. Based on this, the platform generates preliminary fire event information with a medium confidence level, indicating a need for high attention and further verification.

[0052] 3. Low confidence level rule; Triggering conditions: When the comprehensive feature criteria present the following conditions: Temperature evidence: Thermal imaging of the fire showed abnormally high temperatures in a certain area.

[0053] Morphological evidence: Within the same spatiotemporal range, no corresponding flame or smoke targets were detected by visible light fire features.

[0054] Judgment Logic and Output: This rule applies to single-modal alarm scenarios. A combustion pattern with only a high-temperature signal and no visible combustion could originate from various non-fire heat sources, such as high-intensity lighting, radiators, hot water pipes, or strong sunlight reflection. However, this does not completely rule out the possibility that the fire is obscured, is in a very early smoldering stage, or is occurring in a camera blind spot. Therefore, the platform classifies it as a low-confidence preliminary fire event. This level of output indicates the detection of abnormal thermal phenomena, but its nature is highly uncertain and requires further in-depth analysis through subsequent multi-source data fusion verification steps to rule out false alarms or confirm hidden fires.

[0055] In summary, this multi-rule decision-making method transforms the abstract features obtained from dual-spectrum fusion into preliminary event conclusions with clear pyrochemical significance and varying degrees of urgency. It constitutes a crucial conversion node from sensor features to actionable fire information, and its tiered output directly guides the allocation of subsequent validation resources and the initiation of response strategies.

[0056] The process of processing the visible light video data using a lightweight deep learning model includes: Image frames from the visible light video data are input into the trained lightweight deep learning model; The lightweight deep learning model is a YOLOv5s target detection model improved based on the MobileNetV3 backbone network, and is trained using a mixed dataset including real flame images and synthetic smoke data. The lightweight deep learning model processes the input image frames and outputs: bounding boxes of flame or smoke targets, category labels, and visible light fire features with corresponding confidence levels.

[0057] For example, this step aims to automatically and in real-time identify two key visual features of a fire—flames and smoke—from a visible light video stream using artificial intelligence technology. Its core process involves using a target detection model optimized for edge computing environments to analyze and judge the input image.

[0058] 1. Model selection and optimization: Balancing lightweight deployment with accurate detection; To meet the combined requirements of real-time performance, low power consumption, and high accuracy in monitoring scenarios, the platform employs an improved lightweight deep learning model. This model is based on the efficient single-stage object detection framework YOLOv5s, but its original backbone network has been replaced with the lighter and more computationally efficient MobileNetV3 network structure.

[0059] Lightweight design purpose: This improvement significantly reduces the computational complexity and number of parameters of the model, enabling it to run quickly on edge devices with limited computing power, achieving real-time frame analysis of video streams and avoiding missing early fire information due to processing delays.

[0060] Core functionality retained: While being lightweight, the model retains powerful object detection capabilities, capable of locating multiple targets simultaneously in a single image and identifying their categories, making it ideal for scenarios requiring rapid detection of flame or smoke areas.

[0061] 2. Model training and data preparation: Generalization capability for complex scenarios; The model's performance depends on training with high-quality, diverse, professional datasets.

[0062] Hybrid dataset strategy: The training data is not from a single source, but rather uses a hybrid dataset. This includes: Realistic Flame Image Set: Images collected from various real fire scenes, experimental fire pits, and public datasets enable the model to learn the real color, texture, shape, and dynamic texture of flames under different environments, scales, and lighting conditions.

[0063] Synthetic smoke data: Due to the varied morphology and difficulty in annotation of early smoke, synthetic smoke data generated through computer graphics technology was specifically introduced. This data can accurately overlay smoke onto various background scenes, greatly expanding the diversity and complexity of smoke samples, especially helping the model to identify semi-transparent and sparse early smoke.

[0064] Training objective: By training on this mixed dataset, the model learns to distinguish visual patterns related to flames and smoke from complex natural backgrounds and is robust to interference factors such as changes in lighting, occlusion, and changes in target scale.

[0065] 3. Model Inference and Feature Output: Extraction of Structured Fire Features; In actual operation, the model executes the following standardized inference process: Input: The preprocessed visible light video data is input frame by frame into the pre-trained lightweight deep learning model.

[0066] Processing: The model performs forward propagation calculations on each frame of the image and analyzes its visual features pixel by pixel.

[0067] Output: The model ultimately outputs the detection results for this frame of image, namely, structured visible light fire features. These features specifically contain information in three dimensions: Bounding boxes: These are rectangular boxes that mark all areas in an image that are identified as suspected flames or smoke. Each box is defined by its position coordinates and size in the image.

[0068] Category label: Clearly indicates the category of the target within each bounding box, whether it is fire or smoke.

[0069] Corresponding confidence level: The degree to which the model is confident in each judgment it makes, i.e., each bounding box and its category. It is a value between 0 and 1. The higher the value, the more confident the model is in the correctness of the detection result.

[0070] In summary, this step transforms the raw visible light video into a series of structured target information with precise spatial location, clear category, and reliability assessment. These visible light fire features serve as the source of morphological evidence for subsequent spatiotemporal alignment and logical fusion with thermal imaging fire features, forming the visual perception backbone of the dual-spectrum fire detection platform.

[0071] Step S3 includes: performing multi-source data fusion verification on the preliminary fire event information, specifically including: information parsing and verification: parsing the received preliminary fire event information in the cloud and performing multi-view spatial geometric verification on related events from different monitoring points in the same geographical area; multi-source correlation verification: associating at least one external monitoring data source with geographic information data to perform physical and environmental correlation verification on the preliminary fire event information; comprehensive judgment: making a comprehensive judgment based on the results of the spatial geometric verification and physical and environmental correlation verification, and outputting the confirmed final fire information when the preset confirmation conditions are met.

[0072] For example, this stage, the core decision-making layer for fire confirmation, is completed in the cloud. Its core task is to perform higher-dimensional aggregated verification and logical correlation analysis on the preliminary fire event information from the front end, which has a confidence level, in order to eliminate occasional false alarms to the greatest extent possible, confirm the real fire, and finally output verified information that can be used for command and decision-making. This process is a rigorous, multi-step verification chain.

[0073] 1. Information parsing and verification: Multi-view spatial geometric cross-validation; Cloud-based aggregation and analysis: The cloud platform receives preliminary fire event information reported by one or more front-end devices within the monitoring area, and performs standardized analysis on this information to extract key fields such as event location, time, confidence level, and feature description.

[0074] Spatial geometric verification: The platform automatically performs intelligent correlation analysis on related events reported from different monitoring points within the same geographical area. Its core logic is multi-view spatial geometric verification. Objective: To examine whether suspected fires reported by cameras at different physical locations point to the same actual location in space.

[0075] Method: The platform utilizes calibration data such as precise geographic coordinates, installation height, and viewing angle of each monitoring point to back-project the target locations reported in different video streams onto a real-world geographic coordinate system. If the intersecting areas of the back-projected targets observed from multiple independent perspectives highly overlap spatially, this constitutes strong spatial cross-validation evidence, indicating the existence of a real three-dimensional fire source. Conversely, if the reported locations are spatially discrete and cannot converge, it is likely a false alarm or an irrelevant event.

[0076] 2. Multi-source correlation verification: physical and environmental correlation analysis; External data was introduced: To further verify the physical plausibility of the event, the platform proactively associated with and invoked at least one external monitoring data source and static geographic information data, including: digital elevation models, land use type maps, and distribution maps of key protection targets.

[0077] Logical consistency check: Using this external data, a consistency analysis is performed on the initial fire incident. Correlation with meteorological conditions: Check whether the real-time wind direction, wind speed, air humidity, and other data around the fire location in the inspection report are consistent with the environmental conditions in which the fire may have occurred, or whether they can explain the direction of smoke dispersion.

[0078] Explanability of the geographical environment: Verify whether the geographical environment of the event site belongs to a fire-prone or high-risk area, which provides background support for the possibility of the event; the geographical environment includes woodland, warehouse area and gas station, etc.

[0079] Eliminate known sources of interference: Combine geographic information data to verify whether there are known, fixed high heat sources around the event point to help determine whether it is a routine thermal interference.

[0080] 3. Comprehensive assessment: Generate final fire information; Evidence synthesis: The platform performs a comprehensive weighted evaluation of the results of spatial geometric verification and multi-source correlation verification.

[0081] Preset confirmation conditions: Based on the reliability level requirements set by the platform, pre-set clear confirmation conditions. For example, a typical confirmation condition may require: simultaneously satisfying the reporting of a high-confidence preliminary event and passing multi-view spatial cross-validation, or for a medium-confidence event, simultaneously satisfying the support of multi-view evidence and the absence of contradictions in meteorological and environmental data.

[0082] Judgment and Output: The platform determines a fire event to be real and reliable only if the comprehensive assessment results meet the preset confirmation conditions. Subsequently, it generates and outputs verified final fire information. This information includes not only the features identified by the front end but also the conclusions verified by the cloud, serving as the authoritative basis for initiating subsequent precise location, situation analysis, and emergency response. If verification fails, the event will be marked as unconfirmed or a false alarm. The platform may log this information for optimization analysis but will not trigger a high-level alarm.

[0083] In summary, the multi-source data fusion verification process, by introducing spatial multi-perspective cross-verification and correlation with external physical environment data, constructs a more comprehensive and rigorous verification network in the cloud. Essentially, it represents a higher-level review of the front-end sensing results, minimizing the limitations of single sensors or single-point perspectives. This ensures that the fire information output by the platform possesses extremely high reliability and authority, laying a solid foundation for subsequent precise response.

[0084] The confirmed final fire information is used for location and situation analysis, specifically including: Precise positioning: Based on the final fire information, the fire location is marked on a digital map, and the positioning result including geographic coordinates and surrounding environmental information is obtained; Situation simulation: Using the positioning result as input, combined with real-time meteorological data and geographic environmental data, a fire spread model is run to generate fire spread trend prediction information; Early warning generation and push: Integrate the positioning result and the spread trend prediction information to generate a structured early warning report including disposal suggestions.

[0085] For example, this stage represents a crucial transition for the platform from fire confirmation to decision support. Its core objective is to precisely locate the confirmed fire information spatially, scientifically predict its development trend, and ultimately generate structured intelligence products that can guide emergency response. This process is interconnected, forming a complete analytical chain.

[0086] 1. Precise location: Identification of the fire location and environmental assessment; Spatial calibration: Based on the target location data contained in the final fire information, the platform calls upon high-precision digital map services. Through coordinate transformation and mapping algorithms, the pixel positions in the image are converted into real-world geographic coordinates, and then accurately tagged on the electronic map.

[0087] Environmental Information Acquisition: Using the geographic coordinates as the core, the platform automatically extracts and correlates detailed environmental information from an integrated geographic information database. This typically includes: topographic elevation, slope and aspect, vegetation type and distribution, land use, road network, water system distribution, and the location of nearby key protection targets. The output of this step is a location result containing the precise geographic coordinates of the fire point and a multi-level surrounding environmental situation map, answering the questions of where the fire is located and what is around it.

[0088] 2. Situation simulation: Dynamic prediction of fire development; Input integration: Based on the above positioning results, the platform accesses the latest refined meteorological data and geographical environment data of the region in real time.

[0089] Model Simulation: The platform inputs this data into its embedded fire spread model for calculation. Based on the principles of fire physics, this model comprehensively considers core driving factors such as wind force, terrain slope, and fuel flammability to simulate the spread speed and intensity of the fire in different directions.

[0090] Trend prediction generation: Based on dynamic calculations using a model, the platform generates visualized fire spread trend predictions for a specified future time period on a digital map. This is typically represented by a series of predicted range maps or isochrones centered on the fire point and expanding outwards over time, visually demonstrating the area and direction that the fire may threaten.

[0091] 3. Early Warning Generation and Push: Structured Report Synthesis; Information Integration and Risk Analysis: The platform automatically integrates precise location results with fire spread trend prediction information. Combined with a pre-set geographic information database, it automatically traverses and performs risk analysis on key targets within the predicted spread range, identifying the most threatened priority targets.

[0092] Structured Report Generation: Based on all the above analyses, the platform generates a complete structured alert report. This report is not simply an event notification, but a decision support document containing multiple key elements, typically covering: Key information about the incident: precise coordinates of the fire, time of discovery, and current estimated burned area.

[0093] Situation prediction visualization: a map showing the predicted range of fire spread over a future time period.

[0094] Impact assessment list: a list of key targets under threat, their attributes, and their distance from the front line.

[0095] Action recommendations: Based on the direction of the fire, road accessibility, and the location of key targets, generate recommendations for priority rescue routes, evacuation directions, or resource deployment areas.

[0096] Tiered push notification: Ultimately, this structured early warning report will be automatically pushed to the relevant command centers, rescue units or managers according to the preset emergency plan, providing them with timely, objective and comprehensive data support for fire situation assessment, force dispatch and emergency decision-making.

[0097] In summary, the location and situation analysis phase, through a three-step process of precise deployment, dynamic prediction, and comprehensive reporting, expands a single fire ignition point into a three-dimensional situational awareness product that includes spatial location, environmental context, future development trends, and specific action guidelines. This truly realizes a leap in fire monitoring capabilities from passive alarms to proactive intelligent decision support.

[0098] In the situation simulation, the location results, real-time meteorological data, and geographic environment data are input; the Rothermel surface fire spread model is used to calculate the fire spread rate in different directions; based on the spread rate, fire spread trend prediction information for a specified future time period is simulated and generated on a digital map; in the early warning generation and push, the location results and the spread trend prediction information are integrated, and risk analysis is performed based on the spread trend prediction information and a preset geographic information database to identify key threatened targets and generate a structured early warning report including the key targets and corresponding handling suggestions; wherein, the structured early warning report includes: precise coordinates of the fire point, current estimated burned area, a spread prediction range map for the future time period, a list of key threatened targets, and priority rescue route suggestions.

[0099] For example, this stage represents a high level of platform value, transforming confirmed static fire information into dynamic and forward-looking decision-making knowledge. Its core process consists of two closely linked stages: first, a scientific simulation of fire development; and second, the generation of actionable command recommendations based on the simulation results.

[0100] Phase 1: Situation Simulation – Fire Spread Simulation Based on Physical Model; The goal of this phase is to answer the crucial question of where and how quickly the fire will spread.

[0101] Multi-source data input integration: The platform uses the geographical coordinates of the fire point obtained through precise positioning as a spatial anchor point to automatically gather and integrate real-time dynamic data from three aspects: Fire location results: used as the initial ignition location and boundary for the simulation.

[0102] Real-time meteorological data: This focuses on acquiring data such as wind speed, wind direction, ambient temperature, and relative humidity in the fire area. Wind is the primary driving force behind the spread of fire, while temperature and humidity affect the flammability of combustibles.

[0103] Geographical environmental data: This includes access to a high-precision digital elevation model of the region, a map showing the distribution of combustible types, and data on the moisture content of combustibles. Topography affects the rate of fire ascent, while the type and moisture content of combustibles determine the intensity of combustion and the ease with which it spreads.

[0104] Physical Model Calculation and Spread Prediction: The platform inputs the above data into the Rothermel surface fire spread model for calculation. This model is an internationally recognized classic physical model, whose core principle is to comprehensively consider three major factors: wind force, terrain, and fuel characteristics, and quantitatively calculate the spread rate of the fire line in different directions.

[0105] Calculation process: The model will analyze, for example, that fires spread fastest in the downwind direction due to wind propulsion, and slowest in the upwind direction; fires that go uphill spread faster than fires that go downhill due to preheating and terrain uplift; and fire lines that pass through dry, dense shrubs spread faster than fire lines that pass through wet, sparse grasslands.

[0106] Trend visualization: Based on the calculated spread speed in each direction, the platform dynamically extrapolates on a digital map, simulating and generating fire spread trend predictions for a specified future time period. This is typically presented as isochrones of fire spread or heat maps of the predicted range, clearly showing the potential extent and timing of the fire.

[0107] Phase Two: Early Warning Generation and Dissemination – From Situation Forecasting to Action Recommendations; The goal of this phase is to transform forecast information into decision-making reports that can directly guide emergency response.

[0108] Risk Analysis and Key Target Identification: The platform automatically overlays the generated fire spread trend prediction map with a pre-set geographic information database for spatial analysis. The database contains detailed information on key protection targets, such as residential areas, schools, hospitals, factories, gas stations, power transmission lines, and historical and cultural sites.

[0109] Analysis process: The platform traverses all preset targets within or at the edge of the prediction range and performs automated risk level assessment based on their distance from the predicted fire line, the target's own attributes, and the estimated time for the fire to arrive.

[0110] Output: Generates a list of threatened priority targets sorted by threat urgency, identifying which targets have the highest priority and require immediate protection or evacuation measures.

[0111] Structured Early Warning Report Synthesis and Delivery: Integrating all analysis results, the platform automatically generates a comprehensive, richly illustrated structured early warning report. This report aims to provide commanders with one-stop decision-making intelligence and typically includes the following core components: Event summary: precise geographical coordinates of the fire location, alarm time, and current estimated burned area.

[0112] Situation prediction: A map showing the predicted range of fire spread over a specific future time period, presented visually as a map layer.

[0113] Impact Assessment: A list of key threatened targets, detailing the target name, type, estimated duration of threat, and recommended actions.

[0114] Response recommendations: Based on the dominant direction of fire spread, the accessibility of the road network, and the distribution of key targets, the platform generates priority rescue route suggestions or resource pre-deployment suggestions to provide reference solutions for force dispatch.

[0115] Supporting data: A summary of the core real-time meteorological data on which the report is based.

[0116] This structured early warning report will be automatically and instantly pushed to relevant command centers, rescue teams, and management personnel through channels such as the command system platform and mobile terminal applications, completing a complete closed loop from fire awareness to intelligent assisted decision-making.

[0117] Example 2: This embodiment provides a mathematical modeling and experimental implementation of a computer vision-based intelligent fire monitoring method. The specific process includes the following steps: I. Implementation Environment Description; This embodiment is based on the intelligent fire monitoring method described in Embodiment 1. Multispectral sensing devices are deployed within the monitoring area. These devices include a coaxially mounted visible light camera and a thermal imaging sensor, both of which are factory calibrated and have a unified time synchronization mechanism. The system is implemented through collaborative operation between front-end edge computing devices and a cloud server.

[0118] II. Implementation examples of step S1; In step S1, the system at time Synchronous acquisition of visible light video frames With thermal imaging video frames .

[0119] (a) Automatic exposure correction for visible light images; For the acquired visible light images After performing automatic exposure correction, the corrected image is represented as follows:

[0120] in: Indicates the first The original visible light image at the pixel level The grayscale or brightness value at that location; This represents the visible light image after exposure correction. Indicates the first Frame luminance gain coefficient; Indicates the first The frame's brightness offset. and The image is adaptively adjusted based on its overall brightness histogram to avoid overexposure or underexposure.

[0121] (ii) Correction of non-uniformity in thermal imaging images; thermal imaging images Non-uniformity correction is performed, and the correction model is as follows:

[0122] in: Indicates the first Frame of original thermal imaging image pixel values; This represents the corrected thermal imaging image; Indicates the pixel gain correction coefficient; This represents the pixel offset correction coefficient.

[0123] The and Obtained through equipment calibration, used to eliminate fixed pattern noise.

[0124] III. Implementation examples of step S2; (a) Extraction of visible light fire features; Corrected visible light image Input a lightweight deep learning model, and the model outputs the first... Visible light fire characteristics of each detected target :

[0125] in: Indicates the first The first goal in Bounding boxes in a frame; Indicates the target category (flame or smoke); This indicates the corresponding category confidence level.

[0126] (ii) Extraction of fire features from thermal imaging; Dynamic temperature thresholding is performed on thermal imaging images, with the temperature threshold... Defined as:

[0127] in: Indicates the first Frame ambient background temperature; This indicates the preset temperature offset.

[0128] The first [unclear] is obtained through segmentation and region growing algorithms. Abnormally high temperature areas :

[0129] in: Indicates the first The spatial range of a high-temperature zone; This indicates the highest temperature value in the area.

[0130] (III) Spatiotemporal mapping and spatial matching; Map the visible light feature center point to the thermal imaging coordinate system using calibration parameters:

[0131] in: This represents the center point of the target in the visible light coordinate system. Represents a coordinate mapping function; This indicates the position mapped to the thermal imaging coordinate system.

[0132] Calculate spatial Euclidean distance:

[0133] in: : Indicates the time. At that time, the first The visible fire characteristics and the first The Euclidean distance between two thermal imaging fire features. This value measures how close two fire features are in a spatial coordinate system.

[0134] : indicates the first The x-coordinate of a visible fire feature in the thermal imaging coordinate system, at time [time]. (Time point after coordinate mapping).

[0135] : indicates the first The y-coordinate of a visible fire feature in the thermal imaging coordinate system, at time [time value]. (Time point after coordinate mapping).

[0136] : indicates the first The x-coordinate of a thermal imaging fire feature in the thermal imaging coordinate system, at time [time]. .

[0137] : indicates the first The y-coordinate of a thermal imaging fire feature in the thermal imaging coordinate system, at time [time value missing]. .

[0138] When the following conditions are met:

[0139] : Indicates the preset distance threshold.

[0140] And in continuous If the match is valid within the same frame, then the feature matching is considered successful.

[0141] (iv) Mathematical implementation examples of multi-rule decision-making; Define the overall confidence function:

[0142] in: Indicates the first Frame-based fire confidence level; Indicates the confidence level of visible light recognition; Indicates the temperature of the high-temperature region; This represents the temperature normalization constant; Indicates the first Frame ambient background temperature; Indicates the duration of time in the score; Represents the weighting coefficient, and .according to The size generates preliminary fire event information with high, medium, and low confidence levels.

[0143] IV. Implementation examples of step S3; (a) Multi-perspective spatial consistency verification; If the locations of fire events from different monitoring points are back-projected, and multiple projection results form overlapping areas in geographic space, then spatial consistency is determined.

[0144] (II) Fire spread scenario simulation; Using the Rothermel Surface Fire Spread Model, the fire spread rate is defined as:

[0145] in: Indicates the speed at which the fire spreads; Indicates the intensity of the reaction; Indicates propagation efficiency; Indicates the wind correction factor; Indicates the slope correction factor; Indicates the bulk density of a combustible material; Indicates the effective heating coefficient; This indicates the amount of heat required for ignition.

[0146] This embodiment also provides an experimental scenario, specifically an experimental demonstration embodiment that compares with existing technologies.

[0147] To verify the technical effectiveness of the computer vision-based intelligent fire monitoring method of this application in terms of fire identification accuracy, false alarm suppression capability, and response reliability, a variety of existing technical solutions were selected as comparison objects, and comparative experiments were conducted under the same experimental conditions.

[0148] The experiment selected the following three typical existing technical solutions for comparison: Comparison with Scheme A: Flame / smoke recognition method based solely on visible light; employs a deep learning model to detect flames or smoke in visible light video; does not incorporate thermal imaging data; and does not perform cross-modal spatiotemporal consistency verification.

[0149] Comparison with Scheme B: Fire detection methods based solely on thermal imaging temperature thresholds; detection of abnormally high-temperature areas based on fixed or dynamic temperature thresholds; lack of consideration for visible light morphological characteristics; susceptible to interference from environmental heat sources.

[0150] Comparison Scheme C: A simple dual-spectral fusion method without spatiotemporal verification; it uses both visible light and thermal imaging data; it only performs feature correspondence at the single-frame level; and it does not introduce a continuous frame spatiotemporal consistency verification mechanism.

[0151] The method proposed in this application involves: simultaneously extracting visible light fire features and thermal imaging fire features; performing spatial mapping based on equipment calibration; introducing Euclidean distance criteria and continuous frame time persistence verification; and combining a multi-rule decision-making method to classify fire confidence.

[0152] Experimental environment and dataset; Experimental scenarios: warehouse scenario, woodland scenario, industrial park scenario; Total video duration: 30 hours; Actual number of fire incidents: 42; Number of non-fire-related disturbances (lighting, heat equipment, sunlight reflection, etc.): 137; Video resolution: 1920×1080; Thermal imaging temperature range: 20℃~550℃; All methods are run on the same video dataset.

[0153] Evaluation index definition: To ensure the objectivity of the experimental results, the following evaluation indicators were used, as shown in Table 1: Table 1

[0154] The results of the comparative experiment on the fire detection performance of different methods are shown in Table 2.

[0155] Table 2

[0156] Analysis of experimental results and explanation of technical effects; Compared to the visible light-only method, Comparative Scheme A is prone to misjudgment or missed detection under conditions such as changes in lighting and smoke obstruction. This application effectively compensates for the shortcomings of visible light under conditions such as low light and backlight by introducing thermal imaging temperature characteristics, thereby improving the accuracy by about 8% and reducing the false alarm rate by more than 10 percentage points.

[0157] Compared to thermal imaging-only methods, Scheme B is prone to misidentifying non-fire heat sources such as equipment heating and solar reflection as fires. This application utilizes visible light morphological characteristics to logically constrain the thermal imaging results, reducing the false alarm rate from 21.4% to 3.1%, significantly improving practical reliability.

[0158] Compared to the dual-spectral method without spatiotemporal verification, although scheme C integrates dual-modal information, it lacks continuous frame spatiotemporal consistency verification and still suffers from false alarms caused by transient interference. This application effectively filters out occasional interference by introducing a spatial Euclidean distance criterion and a temporal persistence verification mechanism, thereby further improving the overall detection stability.

[0159] The comprehensive performance improvement experiment results show that, while ensuring the response time still meets the real-time monitoring requirements, this application significantly improves the accuracy and reliability of fire identification, and has obvious technical advantages, especially in complex environments.

[0160] The experimental results in summary demonstrate that the fire intelligent monitoring method based on the fusion of visible light and thermal imaging described in this application, through spatiotemporal alignment, continuous frame verification, and multi-rule decision-making mechanisms, is significantly superior to existing technical solutions in terms of fire identification accuracy, false alarm suppression capability, and overall reliability, and possesses significant technological advancement and practical value.

[0161] Example 3: This embodiment provides a computer vision-based intelligent fire monitoring platform. The platform is used to execute a computer vision-based intelligent fire monitoring method. The platform includes: The synchronous acquisition module is used to simultaneously acquire visible light video data and thermal imaging video data of the monitored area; An edge fusion analysis module, deployed at the monitoring site and communicatively connected to the synchronous acquisition module, is used to extract visible light fire features from the visible light video data and thermal imaging fire features from the thermal imaging video data; based on a preset multi-rule decision method, the visible light fire features and the thermal imaging fire features are comprehensively analyzed to generate preliminary fire event information; The cloud-based analysis module is communicatively connected to the edge fusion analysis module. It is used to receive the preliminary fire event information, perform multi-source data fusion verification on the preliminary fire event information, and confirm the final fire information. The early warning generation module is communicatively connected to the cloud-based analysis module. It is used to locate and analyze the confirmed final fire information, and generate and output the structured early warning report.

[0162] Example 4: This embodiment provides a terminal device, which includes a processor and a memory. The memory stores a computer program, which includes program instructions. The processor executes the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions to achieve a corresponding method flow or corresponding function. The processor described in this embodiment can be used in the operation of a computer vision-based intelligent fire monitoring method, including the following steps: S1: Synchronously acquire visible light video data and thermal imaging video data of the monitored area; S2: Extract visible light fire features from the visible light video data and extract thermal imaging fire features from the thermal imaging video data; perform comprehensive analysis on the visible light fire features and the thermal imaging fire features based on a preset multi-rule decision method to generate preliminary fire event information; S3: Perform multi-source data fusion verification on the preliminary fire event information, and confirm the final fire information when the verification is successful; perform location and situation analysis on the confirmed final fire information, and generate and output a structured early warning report.

[0163] Example 5: This embodiment provides a storage medium, specifically a computer-readable storage medium (Memory), which is a memory device in a terminal device used to store programs and data. It is understood that the computer-readable storage medium here can include both the built-in storage medium in the terminal device and extended storage media supported by the terminal device. The computer-readable storage medium provides storage space that stores the terminal's operating system. Furthermore, this storage space also stores one or more instructions suitable for loading and execution by a processor. These instructions can be one or more computer programs (including program code). It should be noted that the computer-readable storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device.

[0164] One or more instructions stored in a computer-readable storage medium can be loaded and executed by a processor to implement the corresponding steps of the computer vision-based intelligent fire monitoring method in the above embodiments; one or more instructions in the computer-readable storage medium are loaded and executed by the processor in the following steps: S1: Synchronously acquire visible light video data and thermal imaging video data of the monitored area; S2: Extract visible light fire features from the visible light video data and extract thermal imaging fire features from the thermal imaging video data; perform comprehensive analysis on the visible light fire features and the thermal imaging fire features based on a preset multi-rule decision method to generate preliminary fire event information; S3: Perform multi-source data fusion verification on the preliminary fire event information, and confirm the final fire information when the verification is successful; perform location and situation analysis on the confirmed final fire information, and generate and output a structured early warning report.

[0165] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0166] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0167] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0168] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0169] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0170] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A computer vision-based intelligent fire monitoring method, characterized in that, The method includes: S1: Synchronously acquire visible light video data and thermal imaging video data of the monitored area; S2: Process the visible light video data using a lightweight deep learning model to output visible light fire features representing flames or smoke; process the thermal imaging video data using dynamic temperature threshold segmentation and region growing algorithms to output thermal imaging fire features representing abnormally high temperature areas; establish a spatiotemporal mapping relationship, and perform spatiotemporal alignment and logical fusion of the visible light fire features and the thermal imaging fire features to obtain a fused comprehensive feature criterion; analyze the comprehensive feature criterion according to a preset multi-rule decision method to generate preliminary fire event information including high confidence, medium confidence, and low confidence levels; S3: Perform multi-source data fusion verification on the preliminary fire event information, and confirm the final fire information when the verification is successful; perform location and situation analysis on the confirmed final fire information, and generate and output a structured early warning report.

2. The intelligent fire monitoring method based on computer vision according to claim 1, characterized in that, Step S1 includes: By deploying multispectral sensing devices in the monitoring area, raw visible light video data and raw thermal imaging video data under the same monitoring field of view are collected simultaneously; the collected raw visible light video data and raw thermal imaging video data are preprocessed, including automatic exposure correction of visible light images and non-uniformity correction of thermal imaging images, to obtain preprocessed visible light video data and thermal imaging video data.

3. The intelligent fire monitoring method based on computer vision according to claim 1, characterized in that, The establishment of spatiotemporal mapping relationships, spatiotemporal alignment, and logical fusion specifically include: Based on the calibration parameters of the multispectral sensing device, the target position in the visible light fire feature is mapped to the thermal imaging coordinate system to obtain the mapped position; the Euclidean distance between the mapped position and the actual position of the abnormal high temperature area in the thermal imaging fire feature is calculated; when the Euclidean distance is less than a preset threshold and the visible light fire feature and the thermal imaging fire feature appear continuously in consecutive frames, it is determined that the feature matching is successful, and a corresponding matching success criterion is generated as the fused comprehensive feature criterion.

4. The intelligent fire monitoring method based on computer vision according to claim 1, characterized in that, The preset multi-rule decision-making method includes: If the visible light fire feature identifies an open flame, and the temperature of the corresponding thermal imaging fire feature is higher than the first temperature threshold, then preliminary fire event information with a high confidence level is generated. If the visible light fire feature identifies smoke, and the temperature of the corresponding thermal imaging fire feature is higher than the ambient temperature and shows an upward trend, then preliminary fire event information with a medium confidence level is generated. If only the thermal imaging fire feature shows abnormally high temperature, but no corresponding visible light fire feature is found, then preliminary fire event information with a low confidence level is generated.

5. The intelligent fire monitoring method based on computer vision according to claim 1, characterized in that, The process of processing the visible light video data using a lightweight deep learning model includes: Image frames from the visible light video data are input into the trained lightweight deep learning model; The lightweight deep learning model is a YOLOv5s target detection model based on the MobileNetV3 backbone network, which is trained using a mixed dataset including real flame images and synthetic smoke data. The lightweight deep learning model processes the input image frames and outputs: bounding boxes of flames or smoke targets, category labels, and visible light fire features with corresponding confidence levels.

6. The intelligent fire monitoring method based on computer vision according to claim 1, characterized in that, The preliminary fire incident information is verified by multi-source data fusion, specifically including: Information parsing and verification: The received preliminary fire event information is parsed in the cloud, and multi-view spatial geometric verification is performed on related events from different monitoring points in the same geographical area; Multi-source correlation verification: Correlate at least one external monitoring data source with geographic information data to verify the physical and environmental correlation of the preliminary fire event information; Comprehensive judgment: Based on the results of the spatial geometric verification, physical and environmental correlation verification, a comprehensive judgment is made. When the preset confirmation conditions are met, the final fire information is output and confirmed.

7. The intelligent fire monitoring method based on computer vision according to claim 1, characterized in that, The confirmed final fire information is used for location and situation analysis, specifically including: Precise positioning: Based on the final fire information, the fire location is marked on the digital map to obtain positioning results including geographic coordinates and surrounding environmental information; Situation simulation: Using the positioning results as input, combined with real-time meteorological data and geographical environment data, a fire spread model is run to generate fire spread trend prediction information; Early warning generation and push: Integrate the location results and the spread trend prediction information to generate a structured early warning report that includes disposal suggestions.

8. The intelligent fire monitoring method based on computer vision according to claim 7, characterized in that, In the situation simulation, the positioning results, real-time meteorological data, and geographical environment data are input; the Rothermel surface fire spread model is used to calculate the fire spread rate in different directions; Based on the spread rate, fire spread trend prediction information for a specified future time period is generated on a digital map. In the early warning generation and push, the location results and the spread trend prediction information are integrated, and risk analysis is performed based on the spread trend prediction information and a preset geographic information database to determine the key targets under threat and generate a structured early warning report including the key targets and corresponding handling suggestions. The structured early warning report includes: precise coordinates of the fire point, current estimated burned area, a map showing the predicted spread range for future time periods, a list of key threatened targets, and suggestions for priority rescue routes.

9. A computer vision-based intelligent fire monitoring platform, characterized in that, The platform is used to perform the method according to any one of claims 1-8, and the platform comprises: The synchronous acquisition module is used to simultaneously acquire visible light video data and thermal imaging video data of the monitored area; An edge fusion analysis module, deployed at the monitoring site and communicatively connected to the synchronous acquisition module, is used to extract visible light fire features from the visible light video data and thermal imaging fire features from the thermal imaging video data; based on a preset multi-rule decision method, the visible light fire features and the thermal imaging fire features are comprehensively analyzed to generate preliminary fire event information; The cloud-based analysis module is communicatively connected to the edge fusion analysis module. It is used to receive the preliminary fire event information, perform multi-source data fusion verification on the preliminary fire event information, and confirm the final fire information. The early warning generation module is communicatively connected to the cloud-based analysis module. It is used to locate and analyze the confirmed final fire information, and generate and output the structured early warning report.

Images