A high-precision forest fire detection method suitable for any scene

Abstract

The application discloses a high-precision forest firework detection method suitable for any scene, relates to the field of firework detection and recognition, and comprises the following steps: acquiring real visible light video data, sampling the video data by setting a time interval, and obtaining corresponding multiple frames of visible light images; constructing and training a weather recognition network model to recognize the weather type of a current scene; selecting a single-frame firework detection model corresponding to the weather scene to detect firework in each frame of the acquired multiple frames of visible light images; and performing post-processing on the firework detection result and collecting votes of multiple frame prediction results to obtain a final result. The application has strong generalization ability, high detection precision and strong robustness, and effectively reduces the false detection caused by clouds and fog.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of smoke detection and identification, and particularly relates to a high-precision forest smoke detection method suitable for any scene. BACKGROUND

[0002] Forest fire is the first of the three major natural disasters in forest, which seriously affects the development of forestry, and threatens the safety of life and property of the state and the people. The detection technology based on visible light video has the characteristics of wide monitoring range, strong real-time performance and high reliability, which can timely alarm and take protective measures, and is an effective means to prevent and control forest fires.

[0003] The existing smoke detection method based on visible light video generally uses deep neural network to realize the judgment of smoke target through sample labeling and model training. The diffusion degree of smoke is affected by wind direction, wind speed, air flow, temperature distribution and atmospheric stability, etc., so the shape features of smoke in different seasons and different weather conditions are different. On the other hand, the interference conditions for smoke detection are different in different weather scenes. In the scenes with low visibility such as dust and haze, the contrast between smoke and background, and the brightness of smoke are lower than those in sunny scenes, which is easy to cause the missed detection of smoke. In the fog scene, the visibility of smoke is very low, and the brightness features of cloud and fog are similar to those of smoke, which is easy to cause the false detection of smoke. In the snow scene, the features of wind-blown snow are similar to those of smoke, which is easy to cause the false detection of smoke. Therefore, it is an urgent technical problem for those skilled in the art to provide a smoke detection method suitable for multiple scenes.

[0004] The Chinese patent with publication number CN115294520A "A smoke detection method based on convolutional neural network" proposes a smoke detection method based on deep learning. The method first constructs a single-frame smoke detection data set and trains an improved yolov5 model, then uses the trained model to detect smoke in single-frame images, and finally compares the confidence in the detection result with a pre-set threshold value. If it is greater than the set threshold value, it is judged as a smoke target. This method only detects single-frame images and does not use the related information of video similar frames, resulting in more false detections.

[0005] Chinese patent CN114821414A, entitled "A Fireworks Detection Method Based on Improved YOLOv5," proposes a fireworks detection algorithm based on an improved YOLOv5. The algorithm first performs fireworks detection on a single frame of an image. After detecting a fireworks target, it detects adjacent frames of the same frame. Based on the detection results, it calculates the area change frequency of the flame target and the area change value of the fireworks target across multiple frames, and then compares these values ​​with a pre-set threshold. If the value exceeds the threshold, the image is identified as a fireworks target. However, this method fails to consider the differences in the rate of change of flames and fireworks under different weather conditions. Inappropriately set thresholds can easily lead to missed detections in different weather scenarios.

[0006] In summary, existing smoke detection methods have the following drawbacks:

[0007] (1) Weak generalization ability: The characteristics of fireworks vary greatly in different scenarios, and the interference factors for fireworks detection are different. The same model cannot adapt to any scenario, which can easily lead to missed detection.

[0008] (2) High false detection rate of clouds and fog: Clouds and fog have similar characteristics to fireworks, and the neural network has insufficient ability to distinguish them, resulting in more false detections and a decrease in detection accuracy.

[0009] (3) The use of multi-frame image information is unreasonable and insufficient, resulting in false detection. Summary of the Invention

[0010] The purpose of this invention is to provide a high-precision forest fire detection method applicable to any scenario, in order to solve the problems of weak generalization ability, high false detection rate of clouds and fog, and low detection accuracy of existing fire detection methods.

[0011] The technical solution adopted by this invention to solve the technical problem is as follows:

[0012] The present invention provides a high-precision forest fire detection method applicable to any scenario, comprising the following steps:

[0013] Step S1: Acquire real visible light video data, and obtain corresponding multi-frame visible light images by setting time intervals;

[0014] Step S2: Build and train a weather recognition network model, and use the weather recognition network model to identify the weather type of the current scene;

[0015] Step S3: Construct and train single-frame smoke detection models for different weather scenarios, and use the single-frame smoke detection models to detect smoke in each frame of the acquired multi-frame visible light images;

[0016] Step S4: Post-processing of smoke detection results;

[0017] Step S5: Summarize and analyze the prediction results of multiple frames of images.

[0018] Furthermore, the specific operation process of step S2 is as follows:

[0019] S2.1 Construct a weather recognition dataset, dividing it into a training set, a validation set, and a test set;

[0020] S2.2 The weather recognition network model is trained on the constructed weather recognition dataset using model weights pre-trained on the ImageNet dataset;

[0021] S2.3 Uses a weather recognition network model to identify the weather type of the current scene.

[0022] Furthermore, in step S2.1, the ratio of the training set, validation set, and test set is 8:1:1.

[0023] Furthermore, in step S2.2, the weather recognition network model is a VGG network model, a ResNet50 network model, or a ViT network model.

[0024] Furthermore, the specific operation procedure for step S3 is as follows:

[0025] S3.1 Construct a fireworks detection dataset under different weather scenarios, label the location information of fireworks, and label the interference targets under different weather scenarios. Divide the fireworks detection dataset into training set, validation set and test set. During the training phase, use Mosic data augmentation to combine multiple images after cropping, rotating and flipping transformations to participate in the training.

[0026] S3.2 Construct a single-frame fireworks detection model. This single-frame fireworks detection model is a YOLOv5 network model. The structure of the YOLOv5 network model includes a backbone network, a Neck, and a detection head. The backbone network is responsible for feature extraction, and coordinate attention is added at its end to enhance the ability to extract fireworks features.

[0027] S3.3 Based on the fireworks detection dataset under different weather scenarios, train the corresponding single-frame fireworks detection model, select SGD optimizer and cosine annealing learning rate algorithm, and continuously adjust the training hyperparameters according to the results to obtain the optimal single-frame fireworks detection model.

[0028] S3.4 Based on the weather type identification results of step S2, select the optimal single-frame smoke detection model for the corresponding weather scene to perform smoke detection on each frame of the multi-frame images. The detection results include the location information and confidence level of the smoke target and other interfering targets.

[0029] Furthermore, in step S3.1, the ratio of the training set, validation set, and test set is 8:1:1.

[0030] Furthermore, the specific operation procedure for step S3.2 is as follows:

[0031] S3.2.1 After feature extraction by the backbone network, a feature map with shape H×W×C is obtained, where H represents the height of the feature map, W represents the width of the feature map, and C represents the number of channels in the feature map. Average pooling is performed on each channel of the feature map along the vertical and horizontal directions to aggregate the features along the two directions in space, producing two intermediate feature vectors with direction awareness in the H and W directions.

[0032] S3.2.2 The two intermediate feature vectors obtained from the above transformation are concatenated along the channel dimension. The channels are compressed using a convolution operation with a kernel size of 1×1. Then, they are activated by the BN activation function and the ReLU activation function. Finally, they are activated by a non-linear activation function to obtain an intermediate vector containing information in the vertical and horizontal directions.

[0033] S3.2.3 The intermediate vector containing vertical and horizontal information is divided into two independent feature tensors along the H and W directions. A convolution operation with a kernel size of 1×1 is used to restore the number of channels of these two feature tensors to the number of channels at the input. The Sigmoid function is used for activation. Finally, attention weight matrices in the H and W directions are obtained by expansion.

[0034] Furthermore, the specific operation procedure for step S4 is as follows:

[0035] The Canny edge detection method was used to extract the skyline in the current image. The position information of the fireworks target was compared with the position information of the skyline. Targets located above the skyline were identified as false targets caused by clouds, while targets located below the skyline were identified as real fireworks targets.

[0036] Furthermore, the specific operation procedure for step S5 is as follows:

[0037] Obtain the location information and confidence level of the fireworks targets that appeared in the prediction results of multiple frames of images; count the predicted fireworks targets in the multiple frames of images according to the set IOU threshold, and judge each fireworks target one by one; compare the confidence level of the same fireworks target in each frame of images with the set confidence level threshold. If the confidence level of the same fireworks target in each frame of images is greater than the set confidence level threshold, it is judged as a pre-selected fireworks target. Count the number of times the pre-selected fireworks target appears in the multiple frames of images. If it exceeds half of the number of multiple frames of images, it is a real fireworks target.

[0038] The beneficial effects of this invention are:

[0039] This invention provides a high-precision forest fire detection method applicable to any scenario. First, real visible light video data is acquired, and multiple visible light images are obtained by setting time intervals. Then, a weather recognition network model is constructed and trained to identify the weather type of the current scenario. Next, a single-frame fire detection model for the corresponding weather scenario is selected to detect fire in each frame of the acquired multiple visible light images. Finally, the fire detection results are post-processed, and the prediction results of the multiple frames are aggregated and voted to obtain the final detection result.

[0040] This invention boasts strong generalization ability, selecting a single-frame smoke detection model for the corresponding weather scenario to perform smoke detection on each frame of the acquired multi-frame visible light images, effectively mitigating the interference of complex weather on smoke detection. It also exhibits high detection accuracy by adding coordinate attention to the single-frame image smoke detection network model, enhancing the network's ability to express and extract smoke features, thus improving detection accuracy with almost no increase in computation. Furthermore, post-processing the model's smoke detection results effectively reduces false detections caused by clouds and fog. Finally, this invention demonstrates strong robustness, acquiring multiple frames of visible light images from the video for smoke detection, and summarizing and analyzing the prediction results of these multiple frames to obtain the final detection result. Finally, this invention is applicable to various weather scenarios, effectively mitigating the interference of complex weather on smoke detection. Attached Figure Description

[0041] Figure 1 This is a flowchart of a high-precision forest fire detection method applicable to any scenario according to the present invention. Detailed Implementation

[0042] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0043] This invention provides a high-precision forest fire detection method applicable to any scenario, mainly comprising the following steps: acquiring real visible light video data, obtaining corresponding multi-frame visible light images by setting time intervals; constructing and training a weather recognition network model, using the weather recognition network model to identify the weather type of the current scene; constructing and training single-frame fire detection models corresponding to different weather scenarios, using the single-frame fire detection models to perform fire detection on each frame of the acquired multi-frame visible light images; post-processing the fire detection results to filter out cloud and fog targets; and summarizing and analyzing the prediction results of the multi-frame images.

[0044] like Figure 1As shown, the present invention provides a high-precision forest fire detection method applicable to any scenario, which specifically includes the following steps:

[0045] Step S1: Acquire real visible light video data and obtain corresponding multi-frame visible light images by setting time intervals.

[0046] Step S2: Build and train a weather recognition network model, and use the weather recognition network model to identify the weather type of the current scene. The weather recognition network model is pre-trained.

[0047] The specific operating steps are as follows:

[0048] S2.1 Construct a weather recognition dataset, including categories such as sunny, hazy, dusty, rainy, and snowy days. Divide the weather recognition dataset into training set, validation set, and test set in a ratio of 8:1:1.

[0049] S2.2 Construct a weather recognition network model, which includes, but is not limited to, VGG network model (Visual Geometry Group, super-resolution test sequence), ResNet50 network model (residual neural network) and ViT network model (Vision Transformer, visual task backbone network). The weather recognition network model is trained on the constructed weather recognition dataset using model weights pre-trained on the ImageNet dataset.

[0050] S2.3 Uses a weather recognition network model to identify the weather type of the current scene.

[0051] Step S3: Construct and train single-frame smoke detection models for different weather scenarios. Use the single-frame smoke detection model to detect smoke in each frame of the acquired multi-frame visible light images. The single-frame smoke detection model is pre-trained.

[0052] The specific operating steps are as follows:

[0053] S3.1 Constructs a fireworks detection dataset for different weather scenarios, labeling the location information of the fireworks and also labeling interference targets under different weather scenarios. These interference targets include, but are not limited to, water droplets on the lens in the rainy scene dataset, flying snow in the snowy scene dataset, and reflective river surfaces in the sunny scene dataset. The fireworks detection dataset is divided into training, validation, and test sets in an 8:1:1 ratio. During the training phase, Mosic data augmentation is used, combining multiple images after cropping, rotating, and flipping transformations for training.

[0054] S3.2 Construct a single-frame smoke detection model, which includes, but is not limited to, a YOLOv5 network model. This YOLOv5 network model (baseline model) consists of three parts: a backbone network, a neck, and a detection head. The backbone network is responsible for feature extraction, and coordinate attention is added at its end to enhance the ability to extract smoke features. Specifically:

[0055] S3.2.1 After feature extraction by the backbone network, a feature map of shape H×W×C is obtained, where H represents the height of the feature map, W represents the width of the feature map, and C represents the number of channels in the feature map. Average pooling is performed on each channel of the feature map along both the vertical and horizontal directions, aggregating the features along these two spatial directions to generate two direction-aware intermediate feature vectors in the H and W directions. and

[0056] The calculation formula is as follows:

[0057]

[0058] In the formula, x c (h,i) represents the value in the h-th row and i-th column of the c-th channel of the feature map, where x c (j,w) represents the value of the j-th row and w-th column in the c-th channel of the feature map, where h and w represent the position information of the point in the feature map in the vertical and horizontal directions, respectively; i represents the horizontal position information of the point in the h-th row, and j represents the vertical position information of the point in the w-th column.

[0059] S3.2.2 The two intermediate feature vectors obtained from the above transformation and First, the channels are concatenated along the channel dimension. Then, a 1×1 convolution operation is used to compress the channels. Next, the channels are activated using BN and ReLU activation functions, and finally activated using a non-linear activation function. The resulting intermediate vector contains information in both the vertical and horizontal directions. The calculation formula is as follows: Where f is an intermediate vector containing information about the vertical and horizontal directions, f∈R C / r×(H+W) R represents the feature space, r represents the channel downsampling ratio, F1 represents a convolution operation with a kernel size of 1×1, and δ represents a non-linear activation function.

[0060] S3.2.3 For the aforementioned intermediate vector containing vertical and horizontal information, firstly, it is split into two independent feature tensors along the H and W directions. Secondly, a 1×1 convolution operation is used to restore the number of channels in these two feature tensors to the number of channels at the input. Then, the sigmoid function is used for activation. Finally, the attention weight matrices in the H and W directions are obtained through expansion. The calculation formula is as follows: Among them, g h Let g represent the attention weight matrix in the H direction. w Let F represent the attention weight matrix in the W direction. h This represents a convolution operation with a kernel size of 1×1 in the H direction, F w This represents a convolution operation with a kernel size of 1×1 in the W direction, f h Let f represent the characteristic tensor in the direction of H. w Let σ represent the feature tensor along the W direction, and let σ represent the sigmoid function. Multiplying the original feature map by the corresponding attention weight matrix yields the final output value y. c (i,j), the calculation formula is: Where i represents the horizontal position information of the point in the h-th row of the c-th channel of the feature map, j represents the vertical position information of the point in the w-th row of the c-th channel of the feature map, and x... c (i,j) represents the value of the point in the i-th row and j-th column of the c-th channel of the feature map. This represents the value of the i-th row in the attention weight matrix along the H direction corresponding to the c-th channel. This represents the value of the j-th column in the attention weight matrix along the W direction corresponding to the c-th channel;

[0061] S3.3 Based on the fireworks detection dataset under different weather scenarios, train the corresponding single-frame fireworks detection model, select SGD optimizer and cosine annealing learning rate algorithm, and continuously adjust the training hyperparameters according to the results to obtain the optimal single-frame fireworks detection model.

[0062] S3.4 Based on the weather type identification results of step S2, select the optimal single-frame smoke detection model for the corresponding weather scene to perform smoke detection on each frame of the multi-frame images. The detection results include the location information and confidence level of the smoke target and other interfering targets.

[0063] Step S4: Post-processing of smoke detection results;

[0064] The Canny edge detection method is used to extract the skyline in the current image. Then, the position information of the fireworks target is compared with the position information of the skyline. Targets located above the skyline are identified as false targets caused by clouds, while targets located below the skyline are identified as real fireworks targets.

[0065] Step S5: Summarize and analyze the prediction results of multiple frames;

[0066] Obtain the location information and confidence level of the fireworks targets that appeared in the prediction results of multiple frames of images; count the predicted fireworks targets in the multiple frames of images according to the set IOU threshold, and judge each fireworks target one by one; compare the confidence level of the same fireworks target in each frame of images with the set confidence level threshold. If the confidence level of the same fireworks target in each frame of images is greater than the set confidence level threshold, it is judged as a pre-selected fireworks target. Count the number of times the pre-selected fireworks target appears in the multiple frames of images. If it exceeds half of the number of multiple frames of images, it is a real fireworks target.

[0067] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A high-precision forest fire detection method applicable to any scenario, characterized in that, Includes the following steps: Step S1: Acquire real visible light video data, and obtain corresponding multi-frame visible light images by setting time intervals; Step S2: Build and train a weather recognition network model, and use the weather recognition network model to identify the weather type of the current scene; Step S3: Construct and train single-frame smoke detection models for different weather scenarios, and use the single-frame smoke detection models to detect smoke in each frame of the acquired multi-frame visible light images; S3.1 Construct a fireworks detection dataset under different weather scenarios, label the location information of fireworks, and label the interference targets under different weather scenarios. Divide the fireworks detection dataset into training set, validation set and test set. During the training phase, use Mosic data augmentation to combine multiple images after cropping, rotating and flipping transformations to participate in the training. S3.2 Construct a single-frame fireworks detection model. This single-frame fireworks detection model is a YOLOv5 network model. The structure of the YOLOv5 network model includes a backbone network, a Neck, and a detection head. The backbone network is responsible for feature extraction, and coordinate attention is added at its end to enhance the ability to extract fireworks features. S3.2.1 After feature extraction by the backbone network, a feature map with shape H×W×C is obtained, where H represents the height of the feature map, W represents the width of the feature map, and C represents the number of channels in the feature map. Average pooling is performed on each channel of the feature map along the vertical and horizontal directions to aggregate the features along the two directions in space, producing two intermediate feature vectors with direction awareness in the H and W directions. S3.2.2 The two intermediate feature vectors obtained are concatenated along the channel dimension. The channels are compressed using a convolution operation with a kernel size of 1×1. Then, the channels are activated by the BN activation function and the ReLU activation function. Finally, the channels are activated by a non-linear activation function to obtain an intermediate vector containing information in the vertical and horizontal directions. S3.2.3 The intermediate vector containing vertical and horizontal information is divided into two independent feature tensors along the H and W directions. A convolution operation with a kernel size of 1×1 is used to restore the number of channels of these two feature tensors to the number of channels at the input. The Sigmoid function is used for activation. Finally, the attention weight matrices in the H and W directions are obtained by expansion. S3.3 Based on the fireworks detection dataset under different weather scenarios, train the corresponding single-frame fireworks detection model, select SGD optimizer and cosine annealing learning rate algorithm, and continuously adjust the training hyperparameters according to the results to obtain the optimal single-frame fireworks detection model. S3.4 Based on the weather type identification results in step S2, select the optimal single-frame smoke detection model for the corresponding weather scene and perform smoke detection on each frame of the multi-frame image. The detection results include the location information and confidence level of the smoke target and other interfering targets. Step S4: Post-processing of smoke detection results; Step S5: Summarize and analyze the prediction results of multiple frames of images.

2. The high-precision forest fire detection method applicable to any scenario according to claim 1, characterized in that, The specific operation procedure for step S2 is as follows: S2.1 Construct a weather recognition dataset, dividing it into a training set, a validation set, and a test set; S2.2 The weather recognition network model is trained on the constructed weather recognition dataset using model weights pre-trained on the ImageNet dataset; S2.3 Uses a weather recognition network model to identify the weather type of the current scene.

3. The high-precision forest fire detection method applicable to any scenario according to claim 2, characterized in that, In step S2.1, the ratio of the training set, validation set, and test set is 8:1:

1.

4. The high-precision forest fire detection method applicable to any scenario according to claim 2, characterized in that, In step S2.2, the weather recognition network model is a VGG network model, a Resnet50 network model, or a ViT network model.

5. The high-precision forest fire detection method applicable to any scenario according to claim 1, characterized in that, In step S3.1, the ratio of the training set, validation set, and test set is 8:1:

1.

6. The high-precision forest fire detection method applicable to any scenario according to claim 1, characterized in that, The specific operation procedure for step S4 is as follows: The Canny edge detection method was used to extract the skyline in the current image. The position information of the fireworks target was compared with the position information of the skyline. Targets located above the skyline were identified as false targets caused by clouds, while targets located below the skyline were identified as real fireworks targets.

7. A high-precision forest fire detection method applicable to any scenario as described in claim 6, characterized in that, The specific operation procedure for step S5 is as follows: Obtain the location information and confidence level of the fireworks targets that appeared in the prediction results of multiple frames of images; count the predicted fireworks targets in multiple frames of images according to the set IOU threshold, and judge each fireworks target one by one; The confidence level of the same firework target in each frame image is compared with the set confidence threshold. If the confidence level of the same firework target in each frame image is greater than the set confidence threshold, it is judged as a pre-selected firework target. The number of times the pre-selected firework target appears in multiple frames image is counted. If it exceeds half of the number of multiple frames image, it is a real firework target.

Images