A real-time fire detection method and system for firefighting drones

Abstract

The present invention discloses a real-time fire detection method and system for firefighting drones, wherein the method comprises: acquiring a target fire scene image in real time, and preprocessing the target fire scene image; discriminating the fire situation according to the pixel color space characteristics of the target fire scene image, and obtaining a first discrimination result; extracting the flame depth characteristics of the target fire scene image by using a YOLOV4 image recognition model, and discriminating the fire situation again based on the flame depth characteristics, and obtaining a second discrimination result; combining the first discrimination result and the second discrimination result according to a preset weight coefficient through a fusion discrimination function, and performing fusion discrimination to obtain a final fire incidence rate; the method can effectively reduce the false alarm rate and the missed alarm rate, thereby improving the efficiency of fire rescue.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision and pattern recognition, and in particular to a real-time fire detection method and system for firefighting drones. Background Art

[0002] Fire, one of the most common disasters in society, poses a significant threat to people's lives. Furthermore, with the continuous development of society and the rise in industrialization, the probability of fire continues to increase, and the harm it causes is increasing day by day. The harm caused by fire varies significantly depending on how early it is discovered. Therefore, while strengthening fire prevention and eliminating fire hazards, it is also crucial to quickly and effectively identify fires when they occur.

[0003] At most fire monitoring sites, fire prevention and control efforts currently focus on sensor placement, including smoke sensors and the introduction of infrared and laser sensor technology. However, these sensors only serve a limited purpose, primarily determining the presence or absence of a fire. They cannot accurately reflect indicators critical to the success of fire rescue operations, such as the fire's location, intensity, and current status. The on-site fire investigation falls heavily on rescue personnel. However, the harsh and rapidly changing fire environment makes manual investigation extremely dangerous.

[0004] In harsh fire environments, using drones for fire detection is a common approach. Pilots operate drones to capture images of the fire scene, which are then transmitted back to a command center for analysis. Consequently, current drone rescue efforts often require manual analysis, which is inefficient and subject to increased uncertainty. Therefore, a fire identification module embedded within the drone becomes even more crucial.

[0005] However, the existing fire situation judgment method based on pixel features has certain limitations and a high false alarm rate, which undoubtedly brings a lot of redundant workload to fire rescue. Therefore, how to improve the fire situation judgment function, reduce the false alarm rate and missed alarm rate, and thus improve the efficiency of fire rescue has become a key issue in current research. Summary of the Invention

[0006] In view of the above problems, the present invention provides a real-time fire detection method and system for firefighting drones that solves at least some of the above technical problems, which can effectively reduce the false alarm rate and missed alarm rate, thereby improving the efficiency of fire rescue.

[0007] On the one hand, an embodiment of the present invention provides a real-time fire detection method for a firefighting drone, comprising:

[0008] S1. Acquire a target fire scene image in real time and preprocess the target fire scene image;

[0009] S2. Distinguishing the fire situation based on the pixel color space features of the target fire scene image to obtain a first discrimination result;

[0010] S3. Using the YOLOV4 image recognition model to extract the flame depth features of the target fire scene image, and discriminate the fire situation again based on the flame depth features to obtain a second discrimination result;

[0011] S4. Combining the first discrimination result and the second discrimination result according to a preset weight coefficient through a fusion discrimination function, performing fusion discrimination, and obtaining a final fire incidence rate.

[0012] Furthermore, it also includes:

[0013] S5. The final recognition result is uploaded to the host computer, and the host computer gives feedback. According to the feedback result of the host computer, the weight coefficient of the fusion discriminant function is adjusted, and the fusion discriminant function is corrected.

[0014] Furthermore, the preprocessing includes removing blurred images.

[0015] Furthermore, the S2 specifically includes:

[0016] Taking the pixel information of the target fire scene image as the unit, the color space features in the target fire scene image are extracted;

[0017] A threshold is set in a specific color space, and based on the characteristics of the RGB color space and the HSV color space, the characteristics of each color space are compared with the set threshold, and the offset between each color space characteristic and the threshold is weighted to calculate the first fire occurrence rate;

[0018] A first threshold is set, and if the first fire occurrence rate is greater than the first threshold, the first fire occurrence rate is used as a first discrimination result.

[0019] Furthermore, in S3, the YOLOV4 image recognition model includes an input layer, a backbone network, a neck network, and a prediction output layer;

[0020] The input layer is used to obtain the target fire scene image;

[0021] The backbone network is used to aggregate and form multiple image features at different image granularities;

[0022] The neck network is used to mix and combine the multiple image features to form a flame depth feature;

[0023] The prediction output layer is used to use the YOLOV4 intelligent recognition algorithm to predict the fire situation again according to the flame depth feature, generate a bounding box of the fire confidence, and use the bounding box of the fire confidence as the second discrimination result.

[0024] Furthermore, the YOLOV4 intelligent recognition algorithm uses the backbone feature extraction network of CSP Darknet53, uses SPP and PANet structures in the enhanced feature extraction network, introduces Mosaic for data enhancement, uses CIOU as regression LOSS, and Mish as the activation function.

[0025] Furthermore, in S4, the fusion discriminant function is expressed as:

[0026] γ=ω red ×f red (γ red )+ω y o l o×f y o l o(γ y o l o)

[0027] γ≤γ Th

[0028] Among them, γ represents the final fire occurrence rate; γ red Represents the first discrimination result; γ yolo Indicates the second discrimination result; f red (·) represents the calculation function of the first discrimination result, ω red is the corresponding weight; f yolo (·) represents the calculation function of the second discrimination result, ω yolo is the corresponding weight; γ Th Indicates the fire occurrence judgment threshold.

[0029] On the other hand, an embodiment of the present invention provides a real-time fire detection system for firefighting drones, applying the above method. The system includes: a data acquisition module, a preprocessing module, a pixel feature fire discrimination module, an intelligent algorithm fire discrimination module, and a fusion discrimination module; wherein:

[0030] The data acquisition module is used to acquire the target fire scene image in real time;

[0031] The preprocessing module is used to preprocess the target fire scene image;

[0032] The pixel feature fire condition discrimination module is used to discriminate the fire condition based on the pixel color space features of the target fire scene image to obtain a first discrimination result;

[0033] The intelligent algorithm fire condition discrimination module is used to extract the flame depth features of the target fire scene image using the YOLOV4 image recognition model, and discriminate the fire condition again based on the flame depth features to obtain a second discrimination result;

[0034] The fusion discrimination module is used to fuse and discriminate the first discrimination result and the second discrimination result through a fusion discrimination function to obtain a final fire occurrence rate.

[0035] Furthermore, it also includes a host computer;

[0036] The host computer is used to obtain the final fire incidence rate and perform real-time correction on the fusion discriminant function based on the final fire incidence rate.

[0037] Furthermore, the YOLOV4 image recognition model includes an input layer, a backbone network, a neck network and a prediction output layer;

[0038] The input layer is used to obtain the target fire scene image;

[0039] The backbone network is used to aggregate and form multiple image features at different image granularities;

[0040] The neck network is used to mix and combine the multiple image features to form a flame depth feature;

[0041] The prediction output layer is used to use the YOLOV4 intelligent recognition algorithm to discriminate the fire situation again according to the flame depth feature to obtain a second discrimination result.

[0042] Compared with the existing technology, the real-time fire detection method and system for firefighting drones described in the present invention has the following beneficial effects:

[0043] In this invention, the YOLOV4 intelligent algorithm recognition and flame pixel feature recognition are combined by fusing the discriminant function, and the results of the two recognitions are fused and discriminated to achieve the purpose of accurately identifying fire conditions and eliminating interference items as much as possible. This improves the accuracy of fire identification and reduces the false alarm rate and missed alarm rate. Traditional recognition methods that only use pixel feature recognition or intelligent algorithm recognition cannot simultaneously guarantee the missed alarm rate and false alarm rate of fire conditions.

[0044] The method used in this paper is universal and has good self-adjustment capabilities for different fire scene environments. It adjusts the fusion discriminant function in different environments based on manual feedback or existing recognition results. Traditional fire identification methods, on the other hand, mostly have fixed parameters and cannot be adaptively adjusted in different environments.

[0045] The method used in the present invention can be combined with the cloud platform sending module and fire location module, which is conducive to the development of fire rescue work. However, the traditional fire identification module has poor correlation with other modules.

[0046] Other features and advantages of the present invention will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present invention. The purposes and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings.

[0047] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0049] Figure 1 A flow chart of a real-time fire detection method for a firefighting drone provided in an embodiment of the present invention.

[0050] Figure 2 A schematic diagram of the fire situation identification process based on pixel color space features provided in an embodiment of the present invention.

[0051] Figure 3 A schematic diagram of the YOLOV4 image recognition module framework provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0052] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0053] See also Figure 1 As shown, the embodiment of the present invention provides a real-time fire detection method for firefighting drones, which specifically includes the following steps:

[0054] S1. Acquire a target fire scene image in real time and preprocess the target fire scene image;

[0055] S2. Distinguishing the fire situation based on the pixel color space features of the target fire scene image to obtain a first discrimination result;

[0056] S3. Using the YOLOV4 image recognition model to extract the flame depth features of the target fire scene image, and discriminate the fire situation again based on the flame depth features to obtain a second discrimination result;

[0057] S4. Combining the first discrimination result and the second discrimination result according to a preset weight coefficient through a fusion discrimination function, performing fusion discrimination, and obtaining a final fire incidence rate;

[0058] S5. The final recognition result is uploaded to the host computer, and the host computer gives feedback. According to the feedback result of the host computer, the weight coefficient of the fusion discriminant function is adjusted, and the fusion discriminant function is corrected.

[0059] The following is a detailed description of each of the above steps.

[0060] In the above step S1, an intelligent drone-mounted camera is used to collect frame data of the target fire scene image, and then the image is pre-screened to remove blurred images, and the clear images are used as valid images for subsequent steps.

[0061] In the above step S2, if Figure 2 As shown, the color space features of the target fire scene image under different color spaces are extracted with the pixel information of the target fire scene image as a unit; a threshold is set in a specific color space, and each color space feature is compared with the set threshold according to the features of the RGB color space and the HSV color space, and the offset between each color space feature and the threshold is weighted to calculate the first fire incidence rate; a first threshold is set, and if the first fire incidence rate is greater than the first threshold, the first fire incidence rate is used as the first discrimination result; if the first fire incidence rate is less than the first threshold, the weighting coefficients between the offsets are adjusted, and after recalculating the first threshold, the color space features of the target fire scene image are re-extracted; by setting the first threshold, the low missed alarm rate in the fire discrimination process can be reduced. In an embodiment of the present invention, the first threshold is set on multiple color feature spaces, including but not limited to RGB color space, HSV color space, etc., and the corresponding threshold discrimination function can be expressed by formulas (1)-(3):

[0062] R>RThre (1)

[0063] R≥G≥B (2)

[0064]

[0065] The above equations (1)-(3) are used to perform a preliminary fire identification based on pixel features. Pixel features are one of the most obvious characteristics of flames. Using equations (1)-(3) to identify fire pixel by pixel can effectively ensure a low false alarm rate for flame detection. At the same time, the module provides feedback based on the identification results and adjusts the identification threshold and threshold to ensure the accuracy and versatility of fire identification.

[0066] In the above step S3, if Figure 3 As shown in the figure, the YOLOV4 image recognition model includes an input layer, a backbone network, a neck network and a prediction output layer; the input layer is used to obtain the target fire scene image captured by the drone-mounted camera; the target fire scene image is aggregated at different image granularities through the backbone network and forms multiple image features; the neck network is used to mix and combine these multiple image features to form flame depth features, and transmit the flame depth features to the prediction output layer (Prediction); the prediction output layer is used to use the YOLOV4 intelligent recognition algorithm to predict the fire situation again according to the flame depth features, generate a fire confidence bounding box, and use the fire confidence bounding box as the second discrimination result.

[0067] The YOLO (You Only Look Once) network is a regression-based target detection algorithm with a relatively fast detection speed. The YOLOV4 intelligent recognition algorithm in the embodiment of the present invention uses the backbone feature extraction network of CSP Darknet53, and uses the SPP and PANet structures in the enhanced feature extraction network. Mosaic is introduced for data enhancement, CIOU is used as the regression loss, and Mish is used as the activation function.

[0068] The Darknet53 network is the backbone of YoloV3, used to extract features for 8x, 16x, and 32x downsampling, respectively. YOLOv4 improves the feature extraction network, drawing on the cross-stage partial network (CSPNet). CSPNet integrates gradient changes into the feature graph, resolving the gradient duplication problem encountered in other network optimization processes. This reduces the model's parameter count, size, and FLOPS, while ensuring inference speed and accuracy.

[0069] The SPP-Net architecture (Spatial Pyramid Pooling Networks) allows feature maps of varying sizes to be fed into fully connected layers, allowing them to be pooled at a fixed size to produce a fixed number of features. The PANet (Path Aggregation Network) performs parameter aggregation, enabling it to be applied to object detection of varying standards. The Mosaic data augmentation method combines four training images into a single one for continued training. This enhances the detection of objects outside of the normal background, enriching the context of the detected object.

[0070] CIOU (Complete IOU) adds the detection box scale, that is, the length and width loss, to DIoU (Distance IOU), making the predicted box more consistent with the real box.

[0071] Mish is a smoother activation function that does not completely truncate negative values, allowing better information and smaller negative gradients to penetrate the neural network, resulting in better accuracy and generalization.

[0072] The YOLOv4 intelligent recognition algorithm in the embodiments of the present invention builds on the existing YOLO target detection architecture and draws on the best optimization strategies of CNN. It optimizes the YOLO algorithm to varying degrees in data processing, backbone networks, network training, activation functions, and loss functions. As mentioned above, the intelligent algorithm fire identification module uses the efficient and powerful YOLOv4 target detection model to identify fire conditions. Because this algorithm incorporates a large number of previous research techniques, it can comprehensively consider multiple flame characteristics and separate flames from other targets, achieving a perfect balance between speed and accuracy. This effectively ensures a low false alarm rate for flame detection.

[0073] In the above step S4,

[0074] The fusion discriminant function is expressed as:

[0075] γ=ω red ×f red (γ red )+ω y o l o×f y o l o(γ y o l o)

[0076] γ≤γ Th

[0077] Among them, γ represents the final fire occurrence rate; γ red Represents the first discrimination result; γ yoloIndicates the second discrimination result; f red (·) represents the calculation function of the first discrimination result, ω red is the corresponding weight; f yolo (·) represents the calculation function of the second discrimination result, ω yolo is the corresponding weight; γ Th Indicates the fire occurrence judgment threshold.

[0078] In the above step S5, the fusion discrimination module feeds back the comprehensive discrimination results to the fire scene command center for final decision. According to the feedback results from the command center, the fusion discrimination function can be modified twice to adjust the weight ω of the fusion discrimination function. red With weight ω yolo , so that the fire situation identification results are as consistent as possible with the feedback results of the command center, and the impact of the complex fire environment is minimized as much as possible.

[0079] Fire occurrence judgment threshold γ Th It is an important basis for judging whether a fire has occurred. The threshold is obtained by fusing the output of the discriminant function with the feedback of personnel based on the actual situation. When the fire incidence rate in a certain area is greater than the judgment threshold, it can be determined that a fire has occurred in the area. The fire incidence rate γ is combined with the fire occurrence judgment threshold γ Th By comparison, it can determine whether a fire has occurred. At the same time, the module can provide feedback based on the judgment results and adjust the weight ratio and processing function structure to ensure the module's adaptability to complex environments.

[0080] The present invention proposes a real-time fire detection method for firefighting drones, which specifically includes: collecting image frame data from a camera mounted on the drone side and performing preprocessing; extracting pixel features of images taken by the drone camera; using a trained neural network model to extract deep features of the image; designing a weighted judgment function, and completing the design of a fire detection module based on the judgment module; in actual application, simultaneously ensuring low missed detection rate and low false alarm rate to detect the fire, and then passing it to subsequent modules. The real-time fire detection method for firefighting drones proposed by the present invention combines the advantages of traditional pixel feature fire recognition and machine learning intelligent recognition, and can efficiently and accurately identify fire by designing a fusion discrimination module. The present invention is not limited to a single fire recognition method such as traditional pixel features or neural network calculations, but integrates the advantages of various recognition methods for fusion recognition, which can ensure a low missed alarm rate and false alarm rate, and is suitable for the task requirements of identifying fire conditions on firefighting drones.

[0081] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A real-time fire detection method for firefighting drones, characterized in that: include: S1. Acquire a target fire scene image in real time and preprocess the target fire scene image; S2. Distinguishing the fire situation based on the pixel color space features of the target fire scene image to obtain a first discrimination result; S3. Using the YOLOV4 image recognition model to extract the flame depth features of the target fire scene image, and discriminate the fire situation again based on the flame depth features to obtain a second discrimination result; S4. Combining the first discrimination result and the second discrimination result according to a preset weight coefficient through a fusion discrimination function, performing fusion discrimination, and obtaining a final fire incidence rate; Said S2 specifically includes: Taking the pixel information of the target fire scene image as the unit, the color space features in the target fire scene image are extracted; A threshold is set in a specific color space, and based on the characteristics of the RGB color space and the HSV color space, the characteristics of each color space are compared with the set threshold, and the offset between each color space characteristic and the threshold is weighted to calculate the first fire occurrence rate; A first threshold is set, and if the first fire occurrence rate is greater than the first threshold, the first fire occurrence rate is used as a first discrimination result.

2. A real-time fire detection method for a firefighting drone according to claim 1, characterized in that: Also includes: S5. The final fire incidence rate is uploaded to the host computer, and the host computer gives feedback. According to the feedback result of the host computer, the weight coefficient of the fusion discriminant function is adjusted, and the fusion discriminant function is corrected.

3. The real-time fire detection method for a firefighting drone according to claim 1, characterized in that: The pre-processing includes removing blurred images.

4. The real-time fire detection method for a firefighting drone according to claim 1, characterized in that: In S3, the YOLOV4 image recognition model includes an input layer, a backbone network, a neck network and a prediction output layer; The input layer is used to obtain the target fire scene image; The backbone network is used to aggregate and form multiple image features at different image granularities; The neck network is used to mix and combine the multiple image features to form a flame depth feature; The prediction output layer is used to use the YOLOV4 intelligent recognition algorithm to predict the fire situation again according to the flame depth feature, generate a bounding box of the fire confidence, and use the bounding box of the fire confidence as the second discrimination result.

5. A real-time fire detection method for a firefighting drone as claimed in claim 4, characterized in that: The YOLOV4 intelligent recognition algorithm uses the CSPDarknet53 backbone feature extraction network, uses the SPP and PANet structures in the enhanced feature extraction network, introduces Mosaic for data enhancement, uses CIOU as the regression LOSS, and uses Mish as the activation function.

6. The real-time fire detection method for a firefighting drone according to claim 1, characterized in that: In S4, the fusion discriminant function is expressed as: c = ω red ×f red (c red )+ω yolo ×f yolo (c yolo ) γ≤γ Th Among them, γ represents the final fire occurrence rate; γ red Represents the first discrimination result; γ yolo Indicates the second discrimination result; f red (·) represents the calculation function of the first discrimination result, ω red is the corresponding weight; f yolo (·) represents the calculation function of the second discrimination result, ω yolo is the corresponding weight; γ Th Indicates the fire occurrence judgment threshold.

7. A real-time fire detection system for firefighting drones, characterized by: Applying the method described in claims 1 to 6 above, the system comprises: a data acquisition module, a preprocessing module, a pixel feature fire condition discrimination module, an intelligent algorithm fire condition discrimination module, and a fusion discrimination module; wherein: The data acquisition module is used to acquire the target fire scene image in real time; The preprocessing module is used to preprocess the target fire scene image; The pixel feature fire condition discrimination module is used to discriminate the fire condition based on the pixel color space features of the target fire scene image to obtain a first discrimination result; The intelligent algorithm fire condition discrimination module is used to extract the flame depth features of the target fire scene image using the YOLOV4 image recognition model, and discriminate the fire condition again based on the flame depth features to obtain a second discrimination result; The fusion discrimination module is used to fuse and discriminate the first discrimination result and the second discrimination result through a fusion discrimination function to obtain a final fire occurrence rate.

8. The real-time fire detection system for firefighting drones according to claim 7, characterized in that: Also includes the host computer; The host computer is used to obtain the final fire incidence rate and perform real-time correction on the fusion discriminant function based on the final fire incidence rate.

9. The real-time fire detection system for firefighting drones according to claim 7, characterized in that: The YOLOV4 image recognition model includes an input layer, a backbone network, a neck network, and a prediction output layer; The input layer is used to obtain the target fire scene image; The backbone network is used to aggregate and form multiple image features at different image granularities; The neck network is used to mix and combine the multiple image features to form a flame depth feature; The prediction output layer is used to use the YOLOV4 intelligent recognition algorithm to discriminate the fire situation again according to the flame depth feature to obtain a second discrimination result.

Images