A Deep Learning-Based Video Wildfire Detection Method for Power Transmission Lines

Abstract

This invention discloses a deep learning-based method for detecting wildfires in power transmission line videos, belonging to the field of target detection. Addressing the difficulty in extracting wildfire and smoke features from power transmission lines, this invention designs a multi-level feature map fusion method. This allows the extracted features to better reflect the characteristics of wildfires and smoke, ultimately detecting wildfires and smoke and outputting corresponding alarm information, showing strong application prospects. Specifically, to improve the feature extraction capability of wildfires and smoke from power transmission lines, multi-level feature map fusion is performed. Different sized prediction feature maps are designed for targets of different sizes, thereby more accurately predicting wildfire and smoke targets of different scales. To better learn wildfire and smoke features and small target features, an optimized self-attention mechanism is introduced to further improve the accuracy of wildfire detection on power transmission lines. The final model can be embedded in video surveillance, and the output results are visualized, making them readable and highly applicable.

Description

Technical Field

[0001] This invention belongs to the field of target detection, specifically a method for detecting wildfires in video image surveillance. Background Technology

[0002] Wildfires have consistently been a major cause of power line tripping in Yunnan's power grid. When wildfires occur within transmission line corridors, the success rate of reclosing is very low, thus impacting transmission line safety. With the gradual implementation of smart grids, power grid departments have accumulated a large amount of video monitoring data on transmission lines and historical wildfire data. However, current wildfire detection processes on transmission lines are easily affected by factors such as cloud cover and sunlight, leading to numerous false detections and false alarms. Traditional methods for wildfire detection on transmission lines involve extracting the shape and color features of smoke and wildfires. Deep learning-based methods also exist, using object detection networks to automatically learn the characteristics of smoke and wildfires. However, due to the complexity of application scenarios, these studies cannot effectively reduce false detections and false alarms. Summary of the Invention

[0003] The purpose of this invention is to provide a deep learning-based method for detecting wildfires in power transmission line videos, addressing the issue of low accuracy in existing methods. By incorporating a self-attention mechanism, the method further improves the feature extraction of wildfires and smoke from the target detection network. This invention solves the problems of difficulty in extracting wildfire and smoke features from power transmission line videos and addresses the issue of numerous false detections.

[0004] To achieve the above objectives, the technical solution of the present invention is: a deep learning-based method for detecting wildfires in power transmission line videos, the method comprising:

[0005] Step 1: Obtain video surveillance images of wildfires along power transmission lines;

[0006] Step 2: Label the wildfires and smoke in the collected video surveillance images of the transmission lines, and label them as smoke and fire to obtain the training dataset;

[0007] Step 3: Expand the dataset of existing power transmission line video surveillance wildfire images by using Poisson fusion, random cropping, random flipping, and random noise addition.

[0008] Step 4: Establish a wildfire detection network;

[0009] Step 4.1: After the training image is input, it first passes through a feature extraction network to obtain three feature maps at different scales. Figure 1 The dimensions are 20×20×1024, and the features are... Figure 2 The dimensions are 40×40×512, and the features are... Figure 3The dimensions are 80×80×256;

[0010] Step 4.2: Perform feature fusion on the obtained feature maps, combining the features... Figure 1 Perform a 1×1 convolution to reduce the dimensionality to 512, and obtain the fused features. Figure 1 The size is 20×20×512; then the features Figure 1 Upsampling is performed, followed by 1×1 convolution to reduce the dimensionality to 256 dimensions, and then combined with the features. Figure 2 Add to obtain fusion features Figure 2 The size is 40×40×256; then the features Figure 2 Upsampling is performed, followed by a 1×1 convolution to reduce the dimensionality to 128 dimensions. This fusion feature is then added to the feature map to obtain the fused feature. Figure 3 The size is 80×80×128; further, regarding the fusion features Figure 3 A branch fusion operation is performed: the upper branch consists of max pooling followed by a convolutional operation, and the lower branch consists of two convolutional operations. After merging the upper and lower branches, the results are then fused with the fused features. Figure 2 Add them together to obtain the output of the second branch; for the fused features Figure 2 A branch fusion operation is performed: the upper branch consists of max pooling followed by a convolutional operation, and the lower branch consists of two convolutional operations. After merging the upper and lower branches, the results are then fused with the fused features. Figure 1 Add them together to get the first branch output;

[0011] Step 4.3: Process the first branch output, the second branch output, and the fused features. Figure 3 The predicted features are obtained by passing the data through a 3×3 convolutional layer 1 and a batch normalization layer 1, followed by a 3×3 convolutional layer 2, a batch normalization layer 2, and an activation layer. Figure 1 The size is 20×20×21, and the predicted features are... Figure 2 The size is 40×40×21, and the predicted features are... Figure 3 The size is 80×80×21; and finally, regression and classification operations are performed on the three predicted feature maps to determine whether a fire has occurred and the location of the fire.

[0012] Step 5: Use the dataset obtained in Step 3 as input to train a wildfire monitoring network obtained in Step 4;

[0013] Step 6: When performing wildfire detection, specifically input the power line monitoring image and scale the image to 1024×1024 to obtain the wildfire smoke detection frame and trigger a wildfire smoke alarm.

[0014] Furthermore, in step 4.1, the feature extraction network is as follows: the training image first passes through four feature extraction network first modules, then two second modules are connected in series, then a first module structure with a convolution kernel size of 3 is connected in series, then two second modules are connected in parallel, and then the attention mechanism module is passed through the attention mechanism module, which is composed of a channel self-attention module and a spatial attention module connected in series; finally, a third module is connected in series, and the output of the third module is the final output of the feature extraction network.

[0015] The first module consists of convolutional layers, batch normalization layers, and SiLU activation function layers;

[0016] The second module consists of two branches. The upper branch structure, after passing through a first module structure with a convolution kernel size of 1, serves as the upper limb output. The lower branch consists of a first module structure with a convolution kernel size of 1, which is then connected in series with four first module structures with a convolution kernel size of 3 to serve as the output. In the lower branch, the third first module structure with a convolution kernel size of 3 is branched and merged with the upper limb output, and then merged with the output of the lower branch. After merging,

[0017] The third module consists of two branches: the upper branch is max pooling and pooling downsampling with a kernel size of 1, and the lower branch is cascaded convolutional pooling downsampling with kernel sizes of 1 and 3 and a stride of 2. Finally, the two results are concatenated in terms of dimensions to serve as the output of the third module.

[0018] Furthermore, in step 5, the training epochs are 200, a stochastic gradient descent optimizer is used, the initial momentum is 0.9, the momentum decay rate is 0.002, the initial learning rate is 0.01, the learning rate decays to 0.00002 / epoch, and the batch size is 8.

[0019] Furthermore, in step 4.1, the channel self-attention module mainly consists of pooling layers and convolutional layers, divided into two branches: max pooling and average pooling. These are then input into a shared convolutional layer, which consists of two fully connected layers connected in series. The first fully connected layer has 64 neurons, and the second has 512 neurons. The outputs of the two branches are then added together, and finally multiplied by the input features using a sigmoid function to obtain the input features of the spatial attention mechanism. The spatial attention mechanism also mainly consists of pooling layers and convolutional layers, also divided into two branches: max pooling and average pooling. The two results are then merged, and then passed through a 7×7 convolutional layer to reduce the dimension to 1 dimension. Finally, it is multiplied by the input features using a sigmoid function to obtain the final feature map.

[0020] This invention employs a deep learning-based method for detecting wildfires in power transmission line video images, offering advantages such as high accuracy, precise localization, and high speed. The invention utilizes a wildfire detection network to extract features of wildfires and their smoke from power transmission line video surveillance images. These extracted features are easily distinguishable from natural clouds and fog, resulting in higher target detection accuracy for wildfires and their smoke. To further improve the detection accuracy for small-target wildfires in video surveillance images, multiple feature map fusion methods are designed into the network structure to enhance the ability of feature maps to extract wildfire and smoke features. Furthermore, considering the characteristics of wildfires and their smoke in Yunnan Province, a self-attention module is designed to reduce false detections of wildfires and their smoke in video surveillance images. Simultaneously, due to the lightweight nature of the model, the number of model parameters is low, allowing for seamless integration into the video surveillance process. To facilitate application and visualization, result alarms and wildfire and smoke detection localization are introduced, providing comprehensive wildfire and smoke detection capabilities for power transmission lines. This invention proposes a deep learning-based method for detecting wildfires in power transmission line video images. This method starts from the direction of multi-level fusion feature maps, performs high-precision learning on wildfires and their smoke in video surveillance images of power transmission lines, and optimizes the model parameters multiple times. It has developed a set of wildfire detection procedures for power transmission lines and constructed a high-precision wildfire detection model for power transmission lines, which can be used for disaster prevention and mitigation of power grid transmission lines. Attached Figure Description

[0021] Figure 1 This is a flowchart of the deep learning-based method for detecting wildfires in video images of power transmission lines, as described in this invention.

[0022] Figure 2 This is a diagram of the wildfire detection network structure of the present invention;

[0023] Figure 3 This is a structural diagram of the self-attention module of the present invention;

[0024] Figure 4 This is a video image of a power transmission line and the result of a wildfire detection. Detailed Implementation

[0025] The present invention will now be described in detail with reference to specific embodiments.

[0026] The technical solution of the present invention is as follows Figure 1 Taking a power transmission line in Kunming, Yunnan Province as an example, the following steps are included:

[0027] Step 1: Data Processing and Dataset Creation

[0028] First, the obtained transmission line monitoring data was cleaned to extract 376 images containing wildfire data. Then, wildfire targets in the monitoring images were labeled as "wildfire." Data augmentation was performed on the labeled monitoring images, primarily using Poisson fusion, random cropping, and random flipping, resulting in 1443 augmented images.

[0029] Step 2: Building a Wildfire Detection Model

[0030] Wildfire monitoring model construction, such as Figure 2 First, a feature extraction module is constructed, which mainly consists of a deep residual network to obtain the original features. Figure 1 ,feature Figure 2 and characteristics Figure 3 The sizes are 20×20×1024, 40×40×512, and 80×80×256 respectively; secondly, in terms of features... Figure 1 Previously, a self-attention module was introduced. First, there's the channel self-attention mechanism, mainly consisting of two branches: max pooling and average pooling, resulting in two 1×1×512 feature maps. These are then fed into cascaded fully connected layers with 64 and 512 neurons respectively. The outputs of the two branches are then summed, and finally, after passing through a sigmoid function, multiplied by the input features to obtain the input features for the spatial attention mechanism. Next is the spatial self-attention mechanism, also mainly consisting of two branches: max pooling and average pooling, resulting in two 20×20×1 feature maps. These two feature maps are then concatenated channel-wise, passed through a 7×7 convolutional layer, and then multiplied by a sigmoid function before being multiplied by the output features of the channel self-attention mechanism to obtain the feature output of the self-attention mechanism. Finally, the feature output of the self-attention mechanism is added to the input features to obtain the final feature. Secondly, a feature fusion module is constructed, mainly to fuse the features... Figure 1 Upsampling is performed, followed by dimensionality reduction using 1×1 convolution to obtain fused features. Figure 1 The size is 20×20×512; then the features Figure 1 Upsampling is performed, followed by 1×1 convolution to reduce the dimensionality to 256 dimensions, and then combined with the features. Figure 2 Add to obtain fusion features Figure 2 The size is 40×40×256; then the features Figure 2 Upsampling is performed, followed by a 1×1 convolution to reduce the dimensionality to 128 dimensions. This fusion feature is then added to the feature map to obtain the fused feature. Figure 3 The size is 80×80×128; then a second fusion operation is performed, mainly to fuse the features. Figure 3 Perform a branch fusion operation: the upper branch is a max pooling operation followed by a convolutional operation, and the lower branch is two convolutional operations. The upper and lower branches are merged and then combined with the fused features. Figure 2 Addition; fusion features Figure 2Perform a branch fusion operation: the upper branch is a max pooling operation followed by a convolutional operation, and the lower branch is two convolutional operations. The upper and lower branches are merged and then combined with the fused features. Figure 1 Addition; fusion features after multi-level fusion Figure 1 Fusion characteristics Figure 2 and fusion features Figure 3 Then, perform 3×3 convolutions and batch normalization layers, followed by another 3×3 convolution, batch normalization, and activation layer to obtain the final prediction features used to predict targets of different sizes. Figure 1 Predictive features Figure 2 and predictive features Figure 3 Finally, the categories are classified and the location of the detected targets are regressed on the three feature maps to determine whether a fire has occurred and the location of the fire.

[0031] Step 3: Wildfire Detection Network Training

[0032] For the training data obtained in step 1, design the corresponding VOC2012 training data format and input it into the transmission line wildfire detection model built in step 2. The training parameters involved in the network model mainly include optimizer, learning rate, learning rate decay, batch size, initial momentum value, momentum decay, and training epochs. Among them, optimizer and learning rate are relatively important parameters, and the specific parameters are shown in Table 1. After training is completed, save the optimal training model.

[0033] Table 1. Some hyperparameters involved in the wildfire detection model.

[0034]

[0035] Step 4: Wildfire Detection Model Testing and Application

[0036] Reacquire transmission line monitoring data different from that in step 1, input it into the optimal model trained in step 3, compare the final test results with the true values, calculate the accuracy index, and output the corresponding alarm information if a wildfire occurs. The test results are shown in Table 2.

[0037] Table 2 Results of Wildfire Detection Accuracy

[0038]

[0039] Step 5: Result Verification and Explanation

[0040] In this example, the characteristics of wildfires along power transmission lines are difficult to extract and are often confused with natural clouds and fog. Features such as wildfires and smoke can be accurately located and detected in the wildfire detection network of this invention, with a detection accuracy exceeding 80% and a detection speed of over 20 FPS, essentially achieving real-time detection. Figure 4It can be seen that wildfires and smoke, whether large or small targets, can be detected very well, and the number of false detections is relatively small, which can effectively support wildfire prevention and mitigation for power grid transmission lines.

[0041] The beneficial effects of this invention are its ability to quickly and accurately detect wildfires and smoke targets in transmission line monitoring data and output corresponding alarm information. The wildfire detection model trained by this invention can achieve 80% accuracy in detecting wildfires and smoke on transmission lines, and the results have high reliability. It can be used for wildfire prevention and control of transmission lines during power grid operation and maintenance.

Claims

1. A deep learning-based method for detecting wildfires in power transmission line videos, the method comprising: Step 1: Obtain video surveillance images of wildfires along power transmission lines; Step 2: Label the wildfires and smoke in the collected video surveillance images of the transmission lines, and label them as smoke and fire to obtain the training dataset; Step 3: Expand the dataset of existing power transmission line video surveillance wildfire images by using Poisson fusion, random cropping, random flipping, and random noise addition. Step 4: Establish a wildfire detection network; Step 4.1: After the training image is input, it first passes through the feature extraction network to obtain three feature maps of different scales. Feature map 1 has a size of 20×20×1024, feature map 2 has a size of 40×40×512, and feature map 3 has a size of 80×80×256. Step 4.2: Perform feature fusion on the obtained feature maps. Perform a 1×1 convolution on feature map 1 to reduce the dimension to 512, resulting in fused feature map 1 with a size of 20×20×512. Then upsample feature map 1 and perform a 1×1 convolution to reduce the dimension to 256. Add this to feature map 2 to obtain fused feature map 2 with a size of 40×40×256. Then upsample feature map 2 and perform a 1×1 convolution to reduce the dimension to 128. Add this to the feature map 1 to obtain fused feature map 2. The fused feature map 3 has a size of 80×80×128. Further, a branch fusion operation is performed on fused feature map 3: the upper branch is max pooling followed by a convolution operation, and the lower branch is two convolution operations. After merging the upper and lower branches, it is added to fused feature map 2 to obtain the second branch output. Similarly, a branch fusion operation is performed on fused feature map 2: the upper branch is max pooling followed by a convolution operation, and the lower branch is two convolution operations. After merging the upper and lower branches, it is added to fused feature map 1 to obtain the first branch output. Step 4.3: The first branch output, the second branch output, and the fused feature map 3 are passed through a 3×3 convolutional layer 1 and a batch normalization layer 1, respectively; then through a 3×3 convolutional layer 2, a batch normalization layer 2, and an activation layer, respectively, to obtain prediction feature map 1 (size 20×20×21), prediction feature map 2 (size 40×40×21), and prediction feature map 3 (size 80×80×21); finally, regression and classification operations are performed on the three prediction feature maps to determine whether a fire has occurred and the location of the fire. Step 5: Use the dataset obtained in Step 3 as input to train a wildfire detection network obtained in Step 4; Step 6: When performing wildfire detection, specifically input the power line monitoring image and scale the image to 1024×1024 to obtain the wildfire smoke detection frame and trigger a wildfire smoke alarm.

2. The method for detecting wildfires in transmission line videos based on deep learning as described in claim 1, characterized in that, In step 4.1, the feature extraction network is as follows: the training image first passes through four feature extraction network first modules, then two second modules are connected in series, then a first module structure with a convolution kernel size of 3 is connected in series, then two second modules are connected in parallel, and then the attention mechanism module is passed through the attention mechanism module, which is composed of a channel self-attention module and a spatial attention module connected in series; finally, a third module is connected in series, and the output of the third module is the final output of the feature extraction network. The first module consists of convolutional layers, batch normalization layers, and SiLU activation function layers; The second module consists of two branches. The upper branch structure is passed through the first module structure with a convolution kernel size of 1 and then serves as the upper limb output. The lower branch is the first module structure with a convolution kernel size of 1, which is then connected in series with four first module structures with a convolution kernel size of 3 and then serves as the output. In the lower branch, the third first module structure with a convolution kernel size of 3 is branched and merged with the upper limb, and then merged with the output of the lower branch to serve as the output of the second module. The third module consists of two branches: the upper branch is max pooling and pooling downsampling with a kernel size of 1, and the lower branch is cascaded convolutional pooling downsampling with kernel sizes of 1 and 3 and a stride of 2. Finally, the two results are concatenated in terms of dimensions to serve as the output of the third module.

3. The method for detecting wildfires in transmission line videos based on deep learning as described in claim 1, characterized in that, In step 5, the training rounds are 200, a stochastic gradient descent optimizer is used, the initial momentum is 0.9, the momentum decay rate is 0.002, the initial learning rate is 0.01, the learning rate decays to 0.00002 / round, and the batch size is 8.

4. The method for detecting wildfires in transmission line videos based on deep learning as described in claim 2, characterized in that, In step 4.1, the channel self-attention module consists of pooling layers and convolutional layers, divided into two branches: max pooling and average pooling. These are then fed into a shared convolutional layer, which consists of two fully connected layers in series. The first fully connected layer has 64 neurons, and the second has 512 neurons. The outputs of the two branches are then added together, and finally multiplied by the input features using a sigmoid function to obtain the input features for the spatial attention mechanism. The spatial attention mechanism also consists of pooling layers and convolutional layers, also divided into two branches: max pooling and average pooling. The two results are then merged, passed through a 7×7 convolutional layer to reduce the dimension to 1 dimension, and finally multiplied by the input features using a sigmoid function to obtain the final feature map.

Images