Multi-scene intelligent fire video detection method based on deep learning and dynamic difference

Abstract

The invention relates to the technical field of artificial intelligence, in particular to a multi-scene intelligent fire video detection method based on deep learning and dynamic difference. According to the invention, through core technology innovations such as a dual verification mechanism of dynamic difference and deep learning fusion, multi-modal data collaborative decision, lightweight modular design and the like, all-round breakthrough is realized in detection precision, environmental adaptability, cost effectiveness and scene expansibility.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a multi-scenario intelligent fire video detection method based on deep learning and dynamic difference. Background Technology

[0002] Traditional fire detection methods often rely on manual judgment, fire sensor detection, or monitoring methods based on single image processing technologies. These methods suffer from significant shortcomings in response speed and detection accuracy, such as response delays, susceptibility to environmental interference (dust, chemical corrosion), fixed installation locations, and limited detection space, making them unsuitable for large or open spaces and failing to meet the practical needs of rapid early warning in the initial stages of a fire. This invention is a fire video detection technology based on a combination of deep learning and dynamic differential methods. It not only helps achieve rapid and accurate identification in the early stages of a fire but also significantly reduces false alarms and missed alarms in complex environments, buying valuable time for emergency rescue and thus protecting people's lives and property.

[0003] This system will be integrated into existing video surveillance systems in factories and warehouses. It captures dynamic changes in the scene through a built-in dynamic differential module and achieves rapid detection of flames and smoke using a deep learning module, eliminating the need for additional hardware. Simultaneously, it links with traditional fire sensors to form a dual verification mechanism, addressing the issues of high false alarm rates and delayed initial fire response in dusty and corrosive environments, thus meeting the need for real-time and accurate fire monitoring. Potential application areas include high-risk industrial scenarios (chemical plants, hazardous chemical warehouses) and public safety fields (large shopping malls, subway hubs).

[0004] Chinese patent CN110516609A discloses a fire video detection and early warning method based on multi-feature fusion of images. This technology preprocesses the acquired video image sequence, extracts the foreground region, and then obtains candidate regions for detection. Static and dynamic features are extracted from these candidate regions. When detecting flames, these features are used as input to an SVM classifier to determine the presence of flames. When detecting smoke, the feature discrimination results are logically combined, selected, and calculated to determine the presence of smoke. If flames or smoke are detected, fire is determined based on their growth trend. If a fire is determined, a fire alarm is issued at the monitored site; otherwise, a fire warning is issued only in the background. This technology, combined with existing monitoring systems, can be applied to places such as shopping malls and warehouses.

[0005] While existing technologies have implemented a series of measures to improve detection efficiency, delays still need further improvement while ensuring reliable detection, making it difficult to meet the demand for rapid early warning in the early stages of a fire. This technology primarily focuses on flame detection and has limited ability to detect smoldering fires, which are common in the early stages of a fire. Failure to detect smoldering fires in a timely manner can lead to the spread of the fire. Furthermore, this technology lacks intelligent identification capabilities for normal or controllable fire conditions, making it unable to provide accurate early warnings based on actual circumstances. The main problems are as follows: 1. High false alarm rate for smoldering flame / smoke images: Static flame images on billboards, posters, and screens are easily misjudged as real fires, leading to false alarms.

[0006] 2. The model is relatively complex: it directly processes high-definition video, requiring a large amount of storage space.

[0007] 3. Insufficient recognition accuracy: Models trained only on experimental datasets will show a significant drop in detection performance when applied to real-world scenarios (new scenes with significant differences in lighting conditions and background environment). Summary of the Invention

[0008] This invention discloses a multi-scenario intelligent fire video detection method based on deep learning and dynamic differential to solve the above-mentioned technical problems.

[0009] The present invention adopts the following technical solution: The first aspect of this invention provides a multi-scenario intelligent fire video detection method based on deep learning and dynamic differential spectroscopy. The method is deployed in a layered architecture system, which includes a perception layer, an edge computing layer, a cloud collaboration layer, and an application layer. The method includes the following steps: Step S1: Collect real-time video streams and environmental sensor data of the monitored area through the perception layer; Step S2: At the edge computing layer, perform dynamic differential processing on the real-time video stream to extract the moving target region; Step S3: Using an optimized lightweight deep learning target detection model, fire target identification is performed on the moving target area to obtain detection results; Step S4: Based on the multimodal fusion decision-making mechanism, the dynamic differential processing results, the fire target identification results, and the environmental sensor data are fused and analyzed to determine the fire event; Step S5: Based on the judgment result, execute hierarchical alarm and equipment linkage control through the application layer.

[0010] In some preferred embodiments, the perception layer is responsible for collecting fire video and environmental data, adapting to the data needs of multiple scenarios such as factories and warehouses.

[0011] In some preferred embodiments, the edge computing layer is deployed in a local monitoring room and undertakes core tasks such as real-time data preprocessing, dynamic differential calculation, and lightweight model inference, thereby reducing data transmission bandwidth consumption.

[0012] In some preferred embodiments, the cloud collaboration layer is deployed in a remote monitoring center and is responsible for dataset updates, model iterative training, multi-site data aggregation and analysis, and historical data storage.

[0013] In some preferred implementations, the application layer provides functions such as alarm display, video playback, data statistics, and device linkage control for security management personnel, and supports multi-terminal access via web, mobile devices, and monitoring center large screens.

[0014] In some preferred embodiments, the dynamic differential processing in step S2 specifically involves: using an improved ViBe background modeling algorithm to extract dynamic regions and adding a spatiotemporal consistency check to distinguish between fire spread movement and regular movement; simultaneously, using an adaptive frame sampling strategy based on scene activity to dynamically adjust the sampling frequency according to the differential energy between video frames.

[0015] In some preferred embodiments, the lightweight deep learning object detection model in step S3 is an optimized YOLOv8n model; the optimization includes: introducing a coordinate attention mechanism into the model backbone network, improving the feature pyramid structure to enhance the small object detection capability, and using fire dataset clustering to generate dedicated prior boxes.

[0016] In some preferred embodiments, the multimodal fusion decision mechanism in step S4 specifically involves fusing the motion features output by the dynamic difference module, the detection confidence level output by the target detection model, and the environmental sensor readings to calculate a unified fire confidence value and then classifying and determining the fire based on a preset threshold.

[0017] In some preferred embodiments, the graded alarm and equipment linkage control in step S5 specifically includes: If the warning is detected, the local audible and visual alarm will be triggered to issue a warning and the warning information will be sent to the management personnel. If a fire is confirmed, a fire alarm will be triggered immediately, and a linkage operation will be executed through a relay module. The linkage operation includes cutting off non-fire-fighting power, starting smoke exhaust equipment, and starting fire extinguishing equipment after a preset delay.

[0018] In some preferred embodiments, the method further includes a model iteration step: the edge computing layer periodically transmits video frames and sensor data related to false alarms, early warnings and confirmed fires back to the cloud collaboration layer; the cloud collaboration layer updates the training dataset, incrementally trains the deep learning object detection model, and distributes the iteratively optimized model to the edge computing layer for update and deployment.

[0019] In some preferred embodiments, the edge computing layer in the hierarchical architecture system is deployed in a local monitoring room to carry the core computing tasks of dynamic differential processing, fire target identification, and multimodal fusion decision-making.

[0020] In some preferred embodiments, the environmental sensor data includes smoke sensor data and temperature sensor data; the multimodal fusion decision mechanism uses the sensor data to assist in verifying the video detection results.

[0021] A second aspect of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the aforementioned multi-scenario intelligent fire video detection method based on deep learning and dynamic differential.

[0022] A third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the aforementioned multi-scenario intelligent fire video detection method based on deep learning and dynamic differential.

[0023] I. System Architecture and Hardware This system adopts a "layered deployment + multimodal fusion" architecture, which is divided into four layers: perception layer, edge computing layer, cloud collaboration layer, and application layer. Each layer achieves data interaction through industrial Ethernet or 5G wireless network.

[0024] In some preferred embodiments, the hardware components include video acquisition devices, edge computing units, environmental sensing devices, alarm and linkage devices, cloud servers, etc.

[0025] In some preferred embodiments, the video acquisition device, a high-definition camera (which may utilize existing equipment), is deployed in key areas of the factory or warehouse.

[0026] In some preferred embodiments, the edge computing unit, an industrial-grade edge server (which may utilize existing equipment), is deployed in a local monitoring room.

[0027] In some preferred embodiments, the alarm and linkage equipment includes one audible and visual alarm and one relay module. The audible and visual alarm receives instructions from the edge computing unit and supports tiered alarms (early warning: yellow light + intermittent buzzer; fire alarm: red light + continuous buzzer); linkage control: the relay module controls fire sprinkler valves, ventilation equipment, emergency lighting, etc., with a response delay ≤100ms. The audible and visual alarm is installed at the factory entrance and the monitoring room, and the relay module is integrated inside the edge computing unit.

[0028] In some preferred embodiments, the cloud server: a rack server is deployed at a remote monitoring center.

[0029] In some preferred embodiments, the core module functions and operates on the following principles: 1. Data Acquisition and Preprocessing Module The camera acquires real-time video streams, enabling real-time acquisition and standardized processing of video streams and sensor data, providing high-quality input data for subsequent detection modules; 2. Dynamic Differential Moving Target Extraction Module This module employs an improved ViBe background modeling algorithm to extract dynamic regions. To address interference in industrial scenarios, a spatiotemporal consistency check is added to distinguish between fire spread and regular motion. Simultaneously, the large volume of multi-channel signal data from factory video surveillance necessitates an adaptive frame sampling strategy based on scene activity to achieve efficient processing: the system analyzes the inter-frame differential energy in real time and dynamically adjusts the sampling frequency (e.g., reducing to 510fps for static scenes and restoring to 25fps when dynamic scenes are detected), significantly reducing computational consumption while maintaining accuracy. This significantly reduces computational resource consumption while ensuring detection accuracy, thereby improving algorithm efficiency.

[0030] 3. YOLO Fire Target Detection Module This module employs an optimized YOLOv8n model for fire target identification. Optimization measures include: introducing a coordinate attention mechanism into the backbone network to enhance focus on flame and smoke features; improving the feature pyramid structure to strengthen small target detection capabilities; and using fire dataset clustering to generate dedicated prior boxes. After training with fire data and undergoing lightweight processing, the model is deployed at the edge, receiving suspected regions from dynamic differential output and performing accurate classification and localization.

[0031] 4. Multimodal fusion decision module Fuse dynamic differential features, YOLO detection results, and sensor data to achieve accurate determination of fire events and reduce false alarms / missing alarms. During the detection process, use dynamic difference to divide and extract suspected fire areas, and then use YOLO to detect suspicious areas. By complementing the advantages of the two, reduce the false alarm rate, missing alarm rate, and computing power requirements. Embed the fusion algorithm into the overall detection system. After the fire alarm is confirmed, first (≤100 ms) cut off the non-fire-fighting power supply and start the smoke exhaust; if the fire continues or is manually confirmed, start fire extinguishing equipment such as sprinklers after a 25-second delay, forming accurate linkage by region and time sequence.

[0032] 5. Alarm and linkage control module Fuse intelligent fire video detection and traditional sensor data for joint decision-making, fuse video detection confidence, motion features, and sensor readings into a unified fire confidence, and trigger early warnings (manual review) or fire alarms (immediate response) accordingly. Optimize the alarm trigger logic and device linkage strategy. After the fire alarm is confirmed, first (≤100 ms) cut off the non-fire-fighting power supply and start the smoke exhaust; if the fire continues or is manually confirmed, start fire extinguishing equipment such as sprinklers after a 25-second delay, forming accurate linkage by region and time sequence, and improving the accuracy of fire judgment and response efficiency.

[0033] Early warning: Trigger the local audible and visual alarm of the edge computing unit, and push the early warning information to the mobile terminal of the safety management personnel; Confirm the fire: Trigger the audible and visual alarm, and at the same time start the fire sprinkler valve, turn off the unnecessary power supply, and start the exhaust equipment through the relay module, and push the alarm information to the monitoring center large screen, mobile terminal, and superior management department; Alarm information content: Include alarm time, alarm location, target type, real-time video link, sensor data curve, and support video playback; False alarm解除: Management personnel can manually解除 false alarms through the mobile terminal / Web terminal. After解除, the system records the reason for the false alarm (such as dust interference, light reflection), and adds the scene data to the training set for subsequent model optimization.

[0034] 6. Model management and iteration module Implement model version management, dataset update, and iterative training to improve the long-term adaptability of the system; The edge computing unit regularly uploads video frames and sensor data related to "suspected interference", "false alarm", and "confirmed fire" to the cloud server; the cloud updates the training set once a month, adds it to the dataset after manual review, and uses the incremental training method to avoid model forgetting; the iterated model is pushed to the edge computing unit.

[0035] The technical solution adopted by the present invention can achieve the following beneficial effects: Compared to existing fire detection technologies, this invention achieves comprehensive breakthroughs in detection accuracy, environmental adaptability, cost-effectiveness, and scenario scalability through core technological innovations such as a "dual verification mechanism integrating dynamic differential and deep learning," "multi-modal data collaborative decision-making," and "lightweight modular design." In early fire detection, addressing the high false negative rate of existing technologies for low-contrast targets such as small flames and weak smoke, this invention improves the extraction rate of small dynamic regions by enhancing background modeling. Combined with a multi-scale YOLO model trained on factory fire datasets, it improves the accuracy of early fire detection, reduces the false negative rate, and can capture nascent fires such as localized overheating and smoke from cables. In terms of balancing cost and real-time performance, the lightweight model reduces power consumption and adapts to existing monitoring networks, thus lowering hardware costs. It solves the problems of limited functionality and scenario limitations of existing technologies, providing an efficient and economical intelligent solution for fire prevention and control in multiple fields. This is mainly manifested in the following aspects: 1. Improve accuracy and reduce false alarm rate: To address the issue of small flame size and indistinct features in the early stages of a fire, a high-quality dataset for specific scenarios was established. The model was then precisely trained, and the target detection model structure and parameters were optimized. This enabled the efficient capture of small flames and weak smoke, improving the accuracy of early fire identification and reducing false alarms caused by factors such as scene depth, lighting changes, human movement, dust and smoke, and normal open flames. A dual verification mechanism was constructed for fire video detection, which fully utilizes the efficiency and accuracy of the YOLO algorithm in target detection, and combines the real-time performance and robustness of traditional dynamic difference technology in moving target extraction.

[0036] By integrating video detection technology with traditional fire alarm systems in a multimodal manner, and leveraging the complementary advantages of deep learning and dynamic differential video detection technology with heat and smoke sensors, a full-chain monitoring system covering "early feature capture (video) - dynamic scene confirmation (video visualization) - threshold trigger verification (traditional sensors)" is constructed. This system overcomes the detection blind spots of single technologies in complex environments, forming an intelligent fire protection system with "multi-source data fusion and dual-verification alarm," significantly improving the reliability and real-time performance of early fire warnings in high-risk scenarios.

[0037] 2. Low hardware cost and lightweight deployment: In the moving target extraction stage, the computational power requirement is reduced by selecting an appropriate sampling interval and an adaptive threshold setting algorithm, enabling real-time fire detection even when the data volume of multi-channel monitoring video streams is large; the system is deployed in conjunction with existing video surveillance systems to reduce hardware costs.

[0038] 3. Enhance functionality: Automatic emergency call dialing function; integrated IoT device control network (such as automatic sprinkler system). Attached Figure Description

[0039] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below, forming part of the present invention. The illustrative embodiments of the present invention and their descriptions explain the present invention and do not constitute an improper limitation of the present invention. In the accompanying drawings: Figure 1 This is a schematic diagram of the process of the present invention. Detailed Implementation

[0040] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. In the description of this invention, it should be noted that the term "or" is generally used to include the meaning of "and / or," unless otherwise expressly indicated.

[0041] Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0042] Example 1 This embodiment further illustrates how to construct a fire dataset suitable for this method and how to train and validate the model. The dataset construction process includes data collection, annotation, and augmentation: publicly available fire datasets and monitoring videos of actual industrial scenes are collected, covering various positive samples such as flames, smoke, and smoldering fires, as well as interference samples such as light reflections, welding sparks, steam, and dust. Rectangular bounding boxes are annotated using the LabelImg tool to generate YOLO format annotation files. To improve the model's generalization ability, data augmentation operations such as random brightness and contrast adjustments, Gaussian noise addition, rotation, and scaling are performed on the training images. The final dataset contains 12,500 images, including 10,000 images in the training set and 2,500 images in the validation set, covering categories such as flames, smoke, and interference samples.

[0043] Model training is based on the PyTorch framework and is carried out on an NVIDIA RTX 3090 GPU. A pre-trained YOLOv8n model is adopted, and a coordinate attention mechanism is introduced into its backbone network to enhance the feature perception ability. The AdamW optimizer is used, with an initial learning rate set to 0.001 and adjusted using the cosine annealing strategy, and the number of training epochs is 200. By performing K-means clustering on the fire dataset, the prior box sizes suitable for flame and smoke targets are generated. After training is completed, it is evaluated on the validation set. The model's mAP@0.5 reaches 94.7%, the flame detection accuracy is 95.2%, the recall rate is 93.8%, the smoke detection accuracy is 94.1%, the recall rate is 92.5%, the model size is 6.8 MB, and the average inference speed on the Jetson Xavier NX edge device is 45 ms / frame.

[0044] To verify the overall performance of the system, actual deployment tests were carried out in a chemical plant building for 30 consecutive days. During this period, the system successfully detected 3 real fire incidents (cable overheating and smoking, electric welding sparks igniting sundries, equipment short-circuit sparks), and all early warnings were completed within 10 seconds without false alarms. The system triggered a total of 12 early warnings, and after manual review, they were all normal operation disturbances, and the false alarm rate dropped to 2.1%. The average time from the occurrence of the fire to the edge early warning is 3.2 seconds, and the average time from the early warning to the linkage control is 1.5 seconds. The average CPU occupancy rate of the edge computing unit is 38%, and the average network bandwidth occupancy is 8 Mbps. The above results show that this method meets the requirements of multi-scenario intelligent fire detection in terms of detection accuracy, response speed, and resource efficiency.

[0045] Example 2 This example illustrates the deployment and optimization of the method in the public area of a large shopping mall. In view of the characteristics of the shopping mall such as dense personnel, complex light changes, and diverse interference sources, the following adaptive adjustments are made to the system: Optical flow-assisted verification is introduced into the dynamic difference module to distinguish the diffusion of flames and smoke from the regular movement of pedestrians; in the adaptive frame sampling strategy, the dynamic threshold is associated with the regional population density, and a higher sampling rate (30 fps) is adopted in high-density areas. In multi-modal fusion decision-making, the weight of the smoke sensor is reduced to 0.2, and the confidence weight of video detection is increased to 0.6. The system uploads 500 - 1000 suspected interference samples from the edge to the cloud every month for incremental training to maintain the adaptability of the model.

[0046] After three months of deployment in a large shopping mall, the system successfully issued warnings for two real fire incidents (one caused by a cigarette butt igniting a trash can, and the other by an overheated electrical cabinet), with response times not exceeding 8 seconds in both cases. The average number of false alarms per month decreased from 9 initially to 3, reducing the false alarm rate to 1.8%. The system supports simultaneous processing of 32 camera video streams, with edge device resource utilization remaining stable below 42%. This embodiment demonstrates that through scene adaptive optimization and continuous model iteration, this method can maintain high detection performance and a low false alarm rate even in complex public environments, exhibiting good scalability and practicality.

[0047] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of the present invention.

Claims

1. A multi-scenario intelligent fire video detection method based on deep learning and dynamic difference, characterized in that, The method is deployed in a layered architecture system, which includes a perception layer, an edge computing layer, a cloud collaboration layer, and an application layer; the method includes the following steps: Step S1: Collect real-time video streams and environmental sensor data of the monitored area through the perception layer; Step S2: At the edge computing layer, perform dynamic differential processing on the real-time video stream to extract the moving target region; Step S3: Using an optimized lightweight deep learning target detection model, fire target identification is performed on the moving target area to obtain detection results; Step S4: Based on the multimodal fusion decision-making mechanism, the dynamic differential processing results, the fire target identification results, and the environmental sensor data are fused and analyzed to determine the fire event; Step S5: Based on the judgment result, execute hierarchical alarm and equipment linkage control through the application layer.

2. The method according to claim 1, characterized in that, The dynamic differential processing in step S2 specifically involves: using an improved ViBe background modeling algorithm to extract dynamic regions and adding a spatiotemporal consistency check to distinguish between fire spread movement and regular movement; simultaneously, using an adaptive frame sampling strategy based on scene activity to dynamically adjust the sampling frequency according to the differential energy between video frames.

3. The method according to claim 1, characterized in that, The lightweight deep learning object detection model in step S3 is the optimized YOLOv8n model; the optimization includes: introducing a coordinate attention mechanism into the backbone network of the model, improving the feature pyramid structure to enhance the small object detection capability, and using fire dataset clustering to generate dedicated prior boxes.

4. The method according to claim 1, characterized in that, The multimodal fusion decision mechanism in step S4 specifically involves fusing the motion features output by the dynamic difference module, the detection confidence level output by the target detection model, and the environmental sensor readings to calculate a unified fire confidence value, and then classifying and judging the fire based on a preset threshold.

5. The method according to claim 1, characterized in that, The hierarchical alarm and equipment linkage control in step S5 specifically includes: If the warning is detected, the local audible and visual alarm will be triggered to issue a warning and the warning information will be sent to the management personnel. If a fire is confirmed, a fire alarm will be triggered immediately, and a linkage operation will be executed through a relay module. The linkage operation includes cutting off non-fire-fighting power, starting smoke exhaust equipment, and starting fire extinguishing equipment after a preset delay.

6. The method according to claim 1, characterized in that, The method also includes a model iteration step: the edge computing layer periodically transmits video frames and sensor data related to false alarms, early warnings and confirmed fires back to the cloud collaboration layer; The cloud-based collaborative layer updates the training dataset, incrementally trains the deep learning object detection model, and distributes the iteratively optimized model to the edge computing layer for update and deployment.

7. The method according to claim 1, characterized in that, The edge computing layer in the hierarchical architecture system is deployed in the local monitoring room to carry the core computing tasks of dynamic differential processing, fire target identification, and multimodal fusion decision-making.

8. The method according to claim 1, characterized in that, The environmental sensor data includes smoke sensor data and temperature sensor data; the multimodal fusion decision mechanism uses sensor data to assist in verifying the video detection results.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the multi-scenario intelligent fire video detection method based on deep learning and dynamic difference as described in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the multi-scenario intelligent fire video detection method based on deep learning and dynamic difference as described in any one of claims 1 to 8.

Images