A method for detecting smoke and fire on power transmission lines based on YOLOv7-tiny
By introducing convolutional attention and progressive pyramid network modules in the YOLOv7-tiny model, the feature extraction and fusion are optimized, and the problems of limited computing power and low recognition accuracy in the firework detection of power grid transmission lines are solved, achieving higher accuracy and Economical fireworks detection effect.
Patent Information
- Application Number
- CN202311615213.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-11-28
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2043-11-28
AI Technical Summary
In the prior art, there are problems in the detection of fireworks in power grid transmission lines, which are limited computing power and complex environments that ignite fireworks are prone to high false alarm rates and high false alarm rates. The existing model based on convolutional neural networks is generalized and There are shortcomings in computing power.
Using the YOLOv7-tiny-based transmission line firework detection method, the convolution attention module and the progressive pyramid network module are introduced to integrate channel and spatial attention, and the model is optimized to improve the semantic information of the feature map, and solve the inconsistency of the identification results between different levels. and reduce the parameter quantity and computing power requirements.
It achieves higher average accuracy and economic benefits, can effectively identify transmission line fireworks in complex environments, reduce false alarms and missed alarms, and is suitable for large-scale deployment of power grids.
Smart Images

Figure CN117911937B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of power grid transmission line smoke and fire detection, and in particular relates to a transmission line smoke and fire detection method based on YOLOv7-tiny. Background Art
[0002] With the continuous development and expansion of the scale of power grid, the transmission distance of transmission lines is getting longer and longer, and it is inevitable that they will cross dense forest areas. Coupled with the high temperature in summer or the high frequency of electricity consumption in winter, the probability of fire risk in transmission lines is greatly increased. The chain reaction caused by mountain fire tripping will cause large-scale power outages in the power grid, which will in turn affect the safety and stability of electricity use.
[0003] Flame detectors based on physical signals are widely used in ordinary buildings and high-speed railways, such as smoke sensors, light sensors, and thermal sensors. However, these traditional physical sensors need to be close to the source of smoke and flames, have a small scope of application, and are single in scenarios. They are very expensive to use in power grid wildfire detection, and cannot provide detailed information about smoke and fire, such as the location, scale, and degree of burning of the flames. In order to overcome these limitations, vision-based smoke and flame detection systems have been proposed. The traditional vision-based smoke and flame detection task starts from the sample feature color, texture, and shape, manually extracts features, and inputs them into the classifier for smoke and flame detection. However, due to the high complexity of fire risk scenarios in power grids, the manually designed features are highly redundant, and have problems of low efficiency and poor generalization, and cannot achieve the purpose of early warning.
[0004] In recent years, models based on deep learning have provided visual interpretation and applications. Fireworks detection technology represented by convolutional neural networks has received widespread attention from academia and industry. This type of method has greatly improved the detection accuracy and speed. Detecting smoke can achieve early warning to a certain extent. However, it is difficult to collect samples of fireworks datasets in the wild and there are few of them. Although the model using the video dataset can achieve good accuracy on its dataset, it has problems such as poor generalization, high missed detection rate of small target fireworks and high false alarm rate. In addition, the existing fireworks detection model based on convolutional neural networks has limited computing power and cannot effectively fuse features to obtain higher-dimensional semantic information and suppress interference from complex backgrounds. Therefore, a new high-efficiency detection method with strong generalization for transmission line fireworks is needed. Summary of the invention
[0005] In view of the above-mentioned deficiencies in the prior art, the present invention provides a transmission line smoke and fire detection method based on YOLOv7-tiny, which can effectively solve the technical problems that smoke and fire identification in complex environments easily leads to high false alarm rate and high missed alarm rate due to limited computing power.
[0006] In order to achieve the above-mentioned invention object, the technical solution adopted by the present invention is: a transmission line smoke and fire detection method based on YOLOv7-tiny, comprising the following steps:
[0007] Step S1: Collecting fireworks images of the power transmission line and the surrounding environment, as well as collecting fire-like and smoke-like interference images, and preprocessing them to obtain preprocessed images;
[0008] Step S2: annotate the preprocessed images, and divide the annotated images into a training set and a validation set according to the ratio;
[0009] Step S3: Optimizing the convolutional neural network model based on YOLOv7-tiny to obtain an optimized convolutional neural network model;
[0010] Step S4: using the training set and the validation set to train and validate the optimized convolutional neural network model to obtain the optimal convolutional neural network model;
[0011] Step S5: deploy the optimal convolutional neural network model on the visualization edge device to perform smoke and fire detection on the transmission line.
[0012] The beneficial effects of the present invention are as follows: the present invention provides a transmission line smoke and fire detection method based on YOLOv7-tiny. The present invention improves the YOLOv7-tiny basic model by using a convolutional attention module and a progressive pyramid network module, and integrates channel and spatial attention to enrich the semantic information of the feature map. At the same time, the three different feature levels of the pyramid module are fused and filtered, so as to obtain higher-dimensional semantic information, effectively solve the problem of inconsistent recognition results between different levels, and has lower parameter quantity and computing power requirements, but has a higher average accuracy rate and higher economic benefits.
[0013] Further: the specific steps of step S4 are as follows:
[0014] Step S401: Processing the images of the training set using a backbone network that incorporates channel and space perception mechanisms to obtain a third feature map, a fourth feature map, and a fifth feature map;
[0015] Step S402: processing the third feature map, the fourth feature map and the fifth feature map using a spatially aware asymptotic feature pyramid network module CAFPN to obtain three final feature maps;
[0016] Step S403: performing detection using a detection head according to the three final feature maps to obtain a transmission line smoke and fire detection result;
[0017] Step S404: Calculate the loss function of the optimized convolutional neural network model based on the power transmission line smoke and fire detection results and the validation set, and calculate the AP of the optimized convolutional neural network model on the validation set. 0.5:0.95 index;
[0018] Step S405: Optimizing the parameters of the optimized convolutional neural network model according to the loss function of the optimized convolutional neural network model to obtain a new convolutional neural network model;
[0019] Step S406: Repeat steps S401 to S405 to train the new convolutional neural network model, and determine whether the number of training times of the new convolutional neural network model reaches the preset number of times. If so, the training is completed and AP is selected. 0.5:0.95 The convolutional neural network model with the highest index is taken as the optimal convolutional neural network model and output, otherwise, return to step S401.
[0020] The beneficial effect of the above further scheme is: the present invention is optimized based on the YOLOv7-tiny network model, so that the network model can obtain high-dimensional semantic information, solve the problem of inconsistent recognition results between different levels, have lower parameter quantity and computing power requirements, and higher average accuracy.
[0021] Further: the specific steps of step S401 are as follows:
[0022] Step S4011: extracting features from the image of the training set using the base layer to obtain a first feature map;
[0023] Step S4012: extracting features from the first feature map using the first feature extraction layer to obtain a second feature map;
[0024] Step S4013: extracting features from the second feature map in the second feature extraction layer, the third feature extraction layer, and the fourth feature extraction layer in sequence to obtain a third feature map, a fourth feature map, and a fifth feature map, respectively.
[0025] The beneficial effect of the above further scheme is: the convolutional block attention CBAM is integrated with the first feature extraction layer, the second feature extraction layer, the third feature extraction layer and the fourth feature extraction layer in the backbone network of the YOLOv7-tiny network model, so that the backbone network obtains channel and spatial perception capabilities.
[0026] Further: the expression of the first feature map is as follows:
[0027] x 0 =CBL(CBL(x))
[0028] CBL(x)=LeakyRelu(BatchNorm(Conv2d(x)))
[0029] Among them, x 0 is the first feature map, CBL() is the basic convolution operation, CBL(CBL(x)) performs two basic convolution operations on the training set image, x is the image of the training set, LeakyRelu is the activation function, BatchNorm is the batch normalization operation, and Conv2d is the two-dimensional convolution operation.
[0030] The beneficial effect of the above further solution is: using the base layer to process the images of the training set makes it easier for the subsequent feature extraction layer to extract features.
[0031] Further: the expressions of the second characteristic graph, the third characteristic graph, the fourth characteristic graph and the fifth characteristic graph are respectively as follows:
[0032] x 1 =CBAM(EELANt(x 0 ))
[0033] x 2 =CBAM(EELANt(MaxPooling(x 1 )))
[0034] x 3 =CBAM(EELANt(MaxPooling(x 2 )))
[0035] x 4 =CBAM(EELANt(MaxPooling(x 3 )))
[0036] Among them, x 0 is the first feature map, x 1 is the second feature map, x 2 is the third characteristic graph, x 3 is the fourth characteristic graph, x 4 is the fifth feature map, CBAM is the convolutional attention operation, EELANt is the efficient layer aggregation module, and MaxPooling is the maximum pooling operation.
[0037] The beneficial effect of the above further scheme is: by using the feature extraction layer and the convolutional attention module CBAM, it is possible to extract features from the image and obtain a feature map, while obtaining the channel and spatial perception of the feature map, so that the model can preliminarily distinguish target objects and non-target objects from the color and contour, which is convenient for subsequent attention analysis.
[0038] Further: the specific steps of step S402 are as follows:
[0039] Step S4021: according to the third feature map, the fourth feature map and the fifth feature map, feature extraction is performed using a CBS convolution module and a CA attention layer respectively, and a sixth feature map with coordinate perception, a seventh feature map with coordinate perception and an eighth feature map with coordinate perception are obtained in one-to-one correspondence;
[0040] Step S4022: According to the sixth feature map with coordinate perception and the seventh feature map with coordinate perception, a three-layer BasicBlock residual basic block and an ASFF2 secondary feature fusion module are used for fusion processing respectively, so as to obtain a ninth feature map incorporating features of the fourth feature map and a tenth feature map incorporating features of the third feature map in a one-to-one correspondence;
[0041] Step S4023: inputting the ninth feature map incorporating the features of the fourth feature map and the tenth feature map incorporating the features of the third feature map into the three-layer CBS convolution module for activation, and inputting the eighth feature map with coordinate perception into the single-layer CBS convolution module for activation, to obtain three activated feature maps;
[0042] Step S4024: input the three activated feature maps into the ASFF3 three-level feature fusion module for semantic fusion, and obtain three feature maps with mutually integrated features;
[0043] Step S4025: According to the three feature maps that integrate features with each other, they are respectively input into the three-layer BasicBlock residual basic block and the single-layer CBS convolution module for processing to obtain three final feature maps, which are: a first final feature map, a second final feature map and a third final feature map.
[0044] The beneficial effect of the above further scheme is: using the spatially aware asymptotic feature pyramid network module CAFPN to process the feature map can enable the model to fuse and filter the features of the three feature maps, solve the problem of inconsistent recognition results between different levels, further reduce the number of parameters, and improve the average accuracy.
[0045] Further: the expressions of the first final feature map, the second final feature map and the third final feature map in step S4025 are respectively as follows:
[0046] x 2,4 =CBS(B 3 (x 2,3 ))
[0047] x 3,4 =CBS(B 3 (x 3,3 ))
[0048] x 4,4 =CBS(B 3 (x 4,3 ))
[0049] x 2,3 =ASFF3(x 2,2 ,upsample(B 3 (x 3,2 )),upsample(CBS(x 4,1 )))
[0050] x 3,3 =ASFF3(x 3,2 ,downsample(B 3 (x 2,2 )),upsample(CBS(x 4,1 )))
[0051] x 4,3 =ASFF3(x 4,1 ,downsample(B 3 (x 2,2 )),downsample(B 3 (x 3,2 )))
[0052] x 2,2 =B 3 (ASFF2(x 2,1 ,upsample(cBS(x 3,1 ))))
[0053] x 3,2 =B 3 (ASFF2(x 3,1 ,downsample(cBS(x 2,1 ))))
[0054] B 3 (·)=BasciBlock(BasciBlock(BasciBlock(·)))
[0055] CBS(·)=Silu(BatchNorm(Conv2d(·)))
[0056] x 2,1 =CA(Silu(BatchNorm(Conv2d(x 2 ))))
[0057] x 3,1 =CA(Silu(BatchNorm(conv2d(x 3 ))))
[0058] x 4,1 =CA(Silu(BatchNorm(conv2d(x 4 ))))
[0059] Among them, x 2,4 is the first final feature map, x 3,4 is the second final feature map, x 4,4 is the third final feature map, CBS is the CBS convolution operation, x 3,2 is the tenth feature map that incorporates the features of the third feature map, x 2,2 The ninth feature map that incorporates the features of the fourth feature map, B 3 is a three-layer BasicBlock residual basic block, BasicBlock is a BasicBlock residual basic operation, x 4,3 is the thirteenth feature map that integrates features, x 3,3 is the twelfth feature map that integrates features, x 2,3 is the eleventh feature map that integrates features. ASFF3 is a three-level feature fusion module. Downsample is downsampling, upsample is upsampling, and x 4,1 is the eighth feature map with spatial perception ability, x 3,1 is the seventh feature map with spatial perception ability, x 2,1 is the sixth feature map with spatial perception ability, CA is the coordinate attention operation, Silu is the adaptive activation function operation, BatchNorm is the batch normalization operation, Conv2d is the two-dimensional convolution operation, x 4 is the fifth characteristic graph, x 3 is the fourth characteristic graph, x 2 It is the third feature map.
[0060] The beneficial effect of the above further scheme is that the spatially aware asymptotic feature pyramid network module CAFPN processes the feature map through the above formula, has lower parameter quantity and computing power requirements, and has a higher average accuracy.
[0061] Further: in step S403, the detection head includes an implicit addition function ImplicitA, a two-dimensional convolution Conv2d and an implicit multiplication function ImplicitM;
[0062] The expressions of the implicit addition function ImplicitA and the implicit multiplication function ImplicitM are as follows:
[0063] ImplicitA(X) = X + θ;
[0064] ImplicitM(X)=X*θ;
[0065] Among them, θ is the learnable vector, X is the three final feature maps, ImplicitA is the implicit addition function, and ImplicitM is the implicit multiplication function.
[0066] The beneficial effect of the above further solution is that by analyzing and detecting the three final characteristic images through the detection head, the fireworks situation of the transmission line can be accurately and quickly obtained. BRIEF DESCRIPTION OF THE DRAWINGS
[0067] Figure 1 is a flow chart of the present invention;
[0068] Figure 2 It is a framework structure diagram of the neural network model of the present invention;
[0069] Figure 3 It is a specific structural diagram of the asymptotic feature pyramid network module CAFPN with spatial perception of the present invention. DETAILED DESCRIPTION
[0070] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.
[0071] like Figure 1 As shown, the present invention provides a transmission line smoke and fire detection method based on YOLOv7-tiny, comprising the following steps:
[0072] Step S1: Collecting fireworks images of the power transmission line and the surrounding environment, as well as collecting fire-like and smoke-like interference images, and preprocessing them to obtain preprocessed images;
[0073] Step S2: annotate the preprocessed images, and divide the annotated images into a training set and a validation set according to the ratio;
[0074] Step S3: Optimizing the convolutional neural network model based on YOLOv7-tiny to obtain an optimized convolutional neural network model;
[0075] Step S4: using the training set and the validation set to train and validate the optimized convolutional neural network model to obtain the optimal convolutional neural network model;
[0076] Step S5: deploy the optimal convolutional neural network model on the visualization edge device to perform smoke and fire detection on the transmission line.
[0077] In one embodiment of the present invention, fireworks images of power transmission lines and surrounding environments are collected, including 3357 large-scale fireworks images, 4722 medium-scale fireworks images, and 349 small-scale fireworks images, and 1034 fire-like and smoke-like interference images are collected, wherein the interference images include fire-like images such as strong light and orange-red clothing, and smoke-like images such as clouds and fog. When collecting, avoid using a large number of similar frames in the video, and collect images of different sizes under more backgrounds as much as possible; Mosaic enhancement method Mosaic, mixed enhancement method Mixup, hue saturation and brightness enhancement method HSV, channel random enhancement and random flipping methods are used to pre-process the images and mark them, and the categories are recorded as flame, smoke and others, and they are divided into training set and verification set in a ratio of 7:3.
[0078] like Figure 2 As shown, it is a neural network model framework structure diagram of the present invention. In one embodiment of the present invention, the backbone network Backbone of the YOLOv7-tiny network model includes a base layer layer0, a first feature extraction layer layer1, a second feature extraction layer layer2, a third feature extraction layer layer3 and a fourth feature extraction layer layer 4. The transmission line image is input from the input layer Input, and the format is 640×640×3. First, the first feature map is obtained through the base layer layer0, and then the second feature map, the third feature map, the fourth feature map and the fifth feature map are obtained after passing through four feature extraction layers (the first feature extraction layer to the fourth feature extraction layer) in sequence. The third feature map, the fourth feature map and the fifth feature map are used as the input of the asymptotic feature pyramid network module CAFPN with spatial perception, and the first final feature map, the second final feature map and the third final feature map are obtained after the convolution coordinate attention CBSCA, that is, the convolution module CBS and the coordinate attention module CA, and the detection head Head is used for detection, and the loss function loss is obtained;
[0079] Among them, the base layer layer0 includes two basic convolution modules CBL, and the details Details show the structure of each module. The basic convolution module CBL includes two-dimensional convolution Conv2d, batch normalization BatchNorm and activation function LeakyRelu. Among the four feature extraction layers layer1, layer2, layer3 and layer4, the first feature extraction layer layer1 includes an efficient layer aggregation module EELANt and a convolution attention module CBAM, and the second feature extraction layer layer2, the third feature extraction layer layer3 and the fourth feature extraction layer layer4 all include a maximum pooling layer MaxPool2d, an efficient layer aggregation module EELANt and a convolution attention module CBAM; among them, the efficient layer aggregation module EELANt is as shown in Figure 2 As shown in the E-ELAN-t module in the Details, the first basic convolution module CBL is connected to the link layer Concat, the second basic convolution module CBL, the third basic convolution module CBL and the fourth basic convolution module CBL are connected in sequence and are all connected to the link layer Concat, and the link layer Concat is connected to the fifth basic convolution module CBL; the convolution attention module CBAM is shown Figure 2 As shown in the details in Details, it includes two steps: channel attention and spatial attention: the process of extracting channel attention is composed of the upper half of the convolutional attention module CBAM, where GAP is global average pooling, GMP is global maximum pooling, Conv is a convolution module, Relu and Sigmod are both activation functions; the process of extracting spatial attention is as follows Figure 2 The lower part of the module is composed of GAP, GMP, Concat, Conv, BN, and Sigmod. The detection head IDetect is as follows: Figure 2 As shown in the details, it includes the implicit addition function ImplicitA, the two-dimensional convolution Conv2d, and the implicit multiplication function ImplicitM.
[0080] Figure 2 The following is a diagram of the neural network model framework of the present invention. The specific steps for training and verifying the optimized convolutional neural network model are as follows:
[0081] Step S401: Processing the images of the training set using a backbone network that incorporates channel and space perception mechanisms to obtain a third feature map, a fourth feature map, and a fifth feature map;
[0082] Step S402: processing the third feature map, the fourth feature map and the fifth feature map using a spatially aware asymptotic feature pyramid network module CAFPN to obtain three final feature maps;
[0083] Step S403: performing detection using a detection head according to the three final feature maps to obtain a transmission line smoke and fire detection result;
[0084] Step S404: Calculate the loss function of the optimized convolutional neural network model based on the power transmission line smoke and fire detection results and the validation set, and calculate the AP of the optimized convolutional neural network model on the validation set. 0.5:0.95 index;
[0085] Step S405: Optimizing the parameters of the optimized convolutional neural network model according to the loss function of the optimized convolutional neural network model to obtain a new convolutional neural network model;
[0086] Step S406: Repeat steps S401 to S405 to train the new convolutional neural network model, and determine whether the number of training times of the new convolutional neural network model reaches the preset number of times. If so, the training is completed and AP is selected. 0.5:0.95 The convolutional neural network model with the highest index is taken as the optimal convolutional neural network model and output, otherwise, return to step S401.
[0087] The specific steps of step S401 are as follows:
[0088] Step S4011: extract features from the image of the training set using the base layer to obtain a first feature map, where the expression of the first feature map is as follows:
[0089] x 0 =CBL(CBL(x))
[0090] CBL(x)=LeakyRelu(BatchNorm(Conv2d(x)))
[0091] Among them, x 0 is the first feature map, CBL() is the basic convolution operation, CBL(CBL(x)) performs two basic convolution operations on the training set image, x is the image of the training set, LeakyRelu is the activation function, BatchNorm is the batch normalization operation, and Conv2d is the two-dimensional convolution operation.
[0092] Step S4012: extract features from the first feature map using the first feature extraction layer to obtain a second feature map, wherein the expression of the second feature map is as follows:
[0093] x 1=CBAM(EELANt(x 0 ))
[0094] Among them, x 1 is the second feature map, CBAM is the convolutional attention operation, EELANt is the efficient layer aggregation module, x 0 is the first feature map;
[0095] Step S4013: extracting features from the second feature map in the second feature extraction layer, the third feature extraction layer, and the fourth feature extraction layer in sequence to obtain a third feature map, a fourth feature map, and a fifth feature map, respectively. The expressions of the third feature map, the fourth feature map, and the fifth feature map are as follows:
[0096] x 2 =CBAM(EELANt(MaxPooling(x 1 )))
[0097] x 3 =CBAM(EELANt(MaxPooling(x 2 )))
[0098] x 4 =CBAM(EELANt(MaxPooling(x 3 )))
[0099] Among them, x 2 is the third characteristic graph, x 3 is the fourth characteristic graph, x 4 is the fifth feature map, and MaxPooling is the maximum pooling operation.
[0100] The detection head Head is IDetect, which includes an implicit addition function ImplicitA, a two-dimensional convolution Conv2d, and an implicit multiplication function ImplicitM, wherein the expressions of the implicit addition function ImplicitA and the implicit multiplication function ImplicitM are as follows:
[0101] ImplicitA(X) = X + θ;
[0102] ImplicitM(X)=X*θ;
[0103] Among them, θ is the learnable vector, X is the three final feature maps, ImplicitA is the implicit addition function, and ImplicitM is the implicit multiplication function.
[0104] like Figure 3As shown, it is a specific structural diagram of the asymptotic feature pyramid network module CAFPN with spatial perception. The specific steps of the asymptotic feature pyramid network module CAFPN with spatial perception are as follows:
[0105] Step S4022: According to the sixth feature map with coordinate perception and the seventh feature map with coordinate perception, three layers of BasicBlock residual basic blocks are used respectively, namely Figure 3 The B3 module in the ASFF2 is fused with the secondary feature fusion module to obtain a ninth feature map incorporating the features of the fourth feature map and a tenth feature map incorporating the features of the third feature map in a one-to-one correspondence;
[0106] Step S4023: inputting the ninth feature map incorporating the features of the fourth feature map and the tenth feature map incorporating the features of the third feature map into the three-layer CBS convolution module for activation, and inputting the eighth feature map with coordinate perception into the single-layer CBS convolution module for activation, to obtain three activated feature maps;
[0107] Step S4024: input the three activated feature maps into the ASFF3 three-level feature fusion module for semantic fusion, and obtain three feature maps with mutually integrated features;
[0108] Step S4025: According to the three feature maps that integrate features with each other, they are respectively input into the three-layer BasicBlock residual basic block and the single-layer CBS convolution module for processing to obtain three final feature maps, which are: a first final feature map, a second final feature map and a third final feature map.
[0109] Depend on Figure 3 From the details, we can see that the residual basic block BasicBlock includes two-dimensional convolution Conv2d, batch normalization BatchNorm and adaptive activation function Silu; the convolution module CBS includes two-dimensional convolution Conv2d, batch normalization BatchNorm and adaptive activation function Silu; the convolution coordinate attention module CBSCA module includes the convolution module CBS and the coordinate attention module CA, the convolution module CBS includes two-dimensional convolution Conv2d, batch normalization BatchNorm and adaptive activation function Silu; the coordinate attention module CA includes average pooling layers XAvgpool and YAvgpool, link layer Concat, two-dimensional convolution Conv2d, batch normalization BatchNorm, non-linear transformation Non-linear and activation function Sigmoid, ASFF3 is a three-level feature fusion module, including upsample upsample and downsample downsample, and the details Details show the thirteenth feature map x that integrates features with each other. 4,3The calculation process is described as follows: the expression of the thirteenth feature map that integrates the features in step S4024;
[0110] The expressions of the sixth feature map with spatial perception capability, the seventh feature map with spatial perception capability, and the eighth feature map with spatial perception capability in step S4021 are as follows:
[0111] x 2,1 =CA(Silu(BatchNorm(Conv2d(x 2 ))))
[0112] x 3,1 =CA(Silu(BatchNorm(Conv2d(x 3 ))))
[0113] x 4,1 =CA(Silu(BatchNorm(conv2d(x 4 ))))
[0114] Among them, x 2,1 is the sixth characteristic map with spatial perception ability, x 3,1 is the seventh feature map with spatial perception ability, x 4,1 is the eighth feature map with spatial perception ability, x 2 is the third characteristic graph, x 3 is the fourth characteristic graph, x 4 is the fifth feature map, CA is the coordinate attention operation, Silu is the adaptive activation function operation, BatchNorm is the batch normalization operation, and Conv2d is the two-dimensional convolution operation;
[0115] In step S4022, the expressions of the ninth feature map incorporating the features of the fourth feature map and the tenth feature map incorporating the features of the third feature map are respectively as follows:
[0116] x 2,2 =B 3 (ASFF2(x 2,1 ,upsample(CBS(x 3,1 ))))
[0117] x 3,2 =B 3 (ASFF2(x 3,1 ,downsample(CBS(x 2,1 ))))
[0118] B 3 (·)=BasciBlock(BasciBlock(BasciBlock(·)))
[0119] CBS(·)=Silu(BatchNorm(Conv2d(·)))
[0120] Among them, x 2,2 is the ninth feature map that incorporates the features of the fourth feature map, ASFF2 is the three-level feature fusion, upsample is upsampling, CBS is the CBS convolution operation, and x 3,2 The tenth feature map is integrated into the third feature map, downsample is downsampling, B 3 It is a three-layer BasicBlock residual basic block, and BasicciBlock is a BasicBlock residual basic operation;
[0121] The three feature graphs integrated with the features in step S4024 include an eleventh feature graph integrated with the features, a twelfth feature graph integrated with the features, and a thirteenth feature graph integrated with the features, and their expressions are as follows:
[0122] x 2,3 =ASFF3(x 2,2 ,upsample(B 3 (x 3,2 )), upsample(CBS(x 4,1 )))
[0123] x 3,3 =ASFF3(x 3,2 ,downsample(B 3 (x 2,2 )), upsample(CBS(x 4,1 )))
[0124] x 4,3 =ASFF3(x 4,1 ,downsample(B 3 (x 2,2 )), downsample(B 3 (x 3,2 )))
[0125] Among them, ASFF3 is a three-level feature fusion module, x 2,3 is the eleventh feature map that integrates features, x 3,3 is the twelfth feature map that integrates features, x 4,3 The thirteenth characteristic diagram is a mutually integrated characteristic;
[0126] The expressions of the first final feature map, the second final feature map and the third final feature map in step S4025 are respectively as follows:
[0127] x2,4 =CBS(B 3 (x 2,3 ))
[0128] x 3,4 =CBS(B 3 (x 3,3 ))
[0129] x 4,4 =CBS(B 3 (x 4,3 ))
[0130] Among them, x 2,4 is the first final feature map, x 3,4 is the second final feature map, x 4,4 is the third final feature map.
[0131] In one embodiment of the present invention, the loss function of the optimized convolutional neural network model includes three parts: classification loss, confidence loss and bounding box regression loss. The classification loss and confidence loss both use cross entropy loss, and the bounding box regression loss uses C-IOU loss. The final loss function is the sum of the three loss functions. The expression of the loss function of the optimized convolutional neural network model is as follows:
[0132] loss=loss cls +loss obj +loss bbox
[0133] loss cls =loss obj =H(p,q)=-∑ x (p(·)log q(·)+(1-p(·))log(1-q(·)))
[0134]
[0135]
[0136] Among them, loss is the loss function of the optimized convolutional neural network model, loss cls is the classification loss, loss obj is the confidence loss, loss bbox is the bounding box regression loss, loss CIoU is the CIOU intersection-over-union loss, H is the cross entropy, p is the true probability distribution, q is the expected probability distribution, log is the logarithmic function, q(·) is the expected output, p(·) is the actual situation, c is the diagonal distance of the minimum closure area that can contain both the predicted box and the true box, b is the center point of the predicted box, and b gtis the center point of the real box, ρ is the Euclidean distance between the two centers, α is the weight function, and v is used to measure the similarity of the aspect ratio.
[0137] According to the AP indicator, the intersection-over-union (IOU) of the predicted box A and the real box B is defined as follows:
[0138]
[0139] Among them, TP means that the target object is correctly predicted (the IOU between the predicted box and the true box is greater than the preset threshold), FP means that an object that does not belong to the target is incorrectly predicted, that is, a negative sample is predicted as a positive sample, and FN means that the target object is not predicted, that is, a positive sample is regarded as a negative sample.
[0140] The formulas for precision P and recall R are as follows:
[0141]
[0142] In order to measure the precision and recall rate, the AP indicator is used. The definition of the AP indicator is as follows:
[0143]
[0144]
[0145] When the IOU thresholds are 0.5, 0.75, and 0.5:0.95 (step size 0.05), the AP indices obtained are AP 0.5 , A.P. 0.75 and AP 0.5:0.95 , the larger the evaluation index value is, the better the performance of the model of the present invention is.
[0146] During the training process, the optimal model will be updated in real time according to the model's indicators on the validation set until the training round ends. The optimal model is saved and deployed on the visual edge device, and the situation next to the monitored power transmission line is fed back in real time according to the detection results. In order to verify that the smoke and fire detection method provided by the embodiment of the present invention is superior to the existing methods, the performance of the present invention and similar cutting-edge methods in recent years are evaluated and compared on the DFS data set using evaluation indicators, as shown in Table 1:
[0147] Table 1
[0148] Model yolov7-tiny yolov5_m yolov7-l The present invention #Param 6.026M 21.191M 36.514M 4.557M FLOPS 6.568G 24.529G 51.758G 5.97G <![CDATA[AP 0.5:0.95 ]]> 0.16 0.164 0.15 0.203 <![CDATA[AP 0.5 ]]> 0.377 0.34 0.338 0.437 <![CDATA[AP 0.75 ]]> 0.107 0.137 0.104 0.158 <![CDATA[f_mAP 50 ]]> 0.577 0.527 0.604 0.622 <![CDATA[s_mAP 50 ]]> 0.36 0.298 0.189 0.426 <![CDATA[o_mAP 50 ]]> 0.195 0.195 0.223 0.262
[0149] Evaluation and comparison are performed on the UMA dataset, as shown in Table 2:
[0150] Table 2
[0151]
[0152]
[0153] It can be seen from Tables 2 and 3 that the method of the present invention has lower parameter quantity and model complexity and higher average accuracy than other methods, can meet the requirements of real-time flame detection, has lower hardware requirements when deployed on a large scale, and therefore has higher economic benefits.
[0154] The beneficial effects of the present invention are as follows: the present invention provides a transmission line smoke and fire detection method based on YOLOv7-tiny. The present invention improves the YOLOv7-tiny basic model by using a convolutional attention module and a progressive pyramid network module, and integrates channel and spatial attention to enrich the semantic information of the feature map. At the same time, the three different feature levels of the pyramid module are fused and filtered, so as to obtain higher-dimensional semantic information, effectively solve the problem of inconsistent recognition results between different levels, and has lower parameter quantity and computing power requirements, but has a higher average accuracy rate and higher economic benefits.
Claims
1. A method for detecting smoke and fire on a power transmission line based on YOLOv7-tiny, comprising the following steps: Step S1: Collecting fireworks images of the power transmission line and the surrounding environment, as well as collecting fire-like and smoke-like interference images, and preprocessing them to obtain preprocessed images; Step S2: annotate the preprocessed images, and divide the annotated images into a training set and a validation set according to the ratio; Step S3: Optimizing the convolutional neural network model based on YOLOv7-tiny to obtain an optimized convolutional neural network model; Step S4: using the training set and the validation set to train and validate the optimized convolutional neural network model to obtain the optimal convolutional neural network model; Step S5: deploying the optimal convolutional neural network model on a visualization edge device to perform smoke and fire detection on the transmission line; The optimized convolutional neural network model includes a YOLOv7-tiny network model, and the backbone network Backbone of the YOLOv7-tiny network model includes a base layer layer 0, a first feature extraction layer layer 1, a second feature extraction layer layer 2, a third feature extraction layer layer 3 and a fourth feature extraction layer layer 4; The transmission line image is input from the input layer Input, with a format of 640×640×3. It first passes through the base layer layer 0 to obtain the first feature map, and then passes through four feature extraction layers in sequence to obtain the second feature map, the third feature map, the fourth feature map and the fifth feature map respectively. The third feature map, the fourth feature map and the fifth feature map are used as the input of the spatially aware asymptotic feature pyramid network module CAFPN, and after the convolution coordinate attention CBSCA, that is, the convolution module CBS and the coordinate attention module CA, the first final feature map, the second final feature map and the third final feature map are obtained, and input to the detection head Head, and the detection head IDetect is used for detection, and the loss function loss is obtained; Among them, the base layer layer 0 includes two basic convolution modules CBL. Details show the structure of each module. The basic convolution module CBL includes two-dimensional convolution Conv2d, batch normalization BatchNorm and activation function LeakyRelu. Among the four feature extraction layers layer 1, layer 2, layer 3 and layer 4, the first feature extraction layer layer 1 includes an efficient layer aggregation module EELANt and a convolution attention module CBAM. The second feature extraction layer layer 2, the third feature extraction layer layer 3 and the fourth feature extraction layer layer 4 all include a maximum pooling layer MaxPool2d, an efficient layer aggregation module EELANt and a convolution attention module CBAM. Among them, the efficient layer aggregation module EELANt includes a first basic convolution module CBL connected to a link layer Concat, a second basic convolution module CBL, a third basic convolution module CBL and a fourth basic convolution module CBL connected in sequence and all connected to the link layer Concat, and the link layer Concat is connected to a fifth basic convolution module CBL; The convolutional attention module CBAM includes two steps: channel attention and spatial attention. The process of extracting channel attention consists of the upper part of the convolutional attention module CBAM, where GAP is global average pooling, GMP is global maximum pooling, Conv is a convolution module, and Relu and Sigmod are both activation functions. The process of extracting spatial attention consists of the lower part of the middle module, where GAP is global average pooling, GMP is global maximum pooling, Concat is a link layer, Conv is a convolution module, BN is a normalization module, and Sigmod is an activation function. The detection head IDetect includes an implicit addition function ImplicitA, a two-dimensional convolution Conv2d and an implicit multiplication function ImplicitM.
2. The method for detecting smoke and fire on a power transmission line based on YOLOv7-tiny according to claim 1, characterized in that: The specific steps of step S4 are as follows: Step S401: Processing the images of the training set using a backbone network that incorporates channel and space perception mechanisms to obtain a third feature map, a fourth feature map, and a fifth feature map; Step S402: processing the third feature map, the fourth feature map and the fifth feature map using a spatially aware asymptotic feature pyramid network module CAFPN to obtain three final feature maps; Step S403: performing detection using a detection head according to the three final feature maps to obtain a transmission line smoke and fire detection result; Step S404: Calculate the loss function of the optimized convolutional neural network model based on the power transmission line smoke and fire detection results and the validation set, and calculate the AP of the optimized convolutional neural network model on the validation set. 0.5:0.95 index; Step S405: Optimizing the parameters of the optimized convolutional neural network model according to the loss function of the optimized convolutional neural network model to obtain a new convolutional neural network model; Step S406: Repeat steps S401 to S405 to train the new convolutional neural network model, and determine whether the number of training times of the new convolutional neural network model reaches the preset number of times. If so, the training is completed and AP is selected. 0.5:0.95 The convolutional neural network model with the highest index is taken as the optimal convolutional neural network model and output, otherwise, return to step S401.
3. The method for detecting smoke and fire on power transmission lines based on YOLOv7-tiny according to claim 2, characterized in that: The specific steps of step S401 are as follows: Step S4011: extracting features from the image of the training set using the base layer to obtain a first feature map; Step S4012: extracting features from the first feature map using the first feature extraction layer to obtain a second feature map; Step S4013: extracting features from the second feature map in the second feature extraction layer, the third feature extraction layer, and the fourth feature extraction layer in sequence to obtain a third feature map, a fourth feature map, and a fifth feature map, respectively.
4. The method for detecting smoke and fire on a power transmission line based on YOLOv7-tiny according to claim 3, characterized in that: The expression of the first feature map is as follows: x 0 =CBL(CBL(x)) CBL(x)=LeakyRelu(BatchNrom(Conv2d(x))) Among them, x 0 is the first feature map, CBL() is the basic convolution operation, CBL(CBL(x)) performs two basic convolution operations on the training set image, x is the image of the training set, LeakyRelu is the activation function, BatchNorm is the batch normalization operation, and Conv2d is the two-dimensional convolution operation.
5. The method for detecting smoke and fire on a power transmission line based on YOLOv7-tiny according to claim 3, characterized in that: The expressions of the second characteristic graph, the third characteristic graph, the fourth characteristic graph and the fifth characteristic graph are respectively as follows: x 1 =CBAM(EELANt(x 0 )) x 2 =CBAM(EELANt(MaxPooling(x 1 ))) x 3 =CBAM(EELANt(MaxPooling(x 2 ))) x 4 =CBAM(EELANt(MaxPooling(x 3 ))) Among them, x 0 is the first feature map, x 1 is the second feature map, x 2 is the third characteristic graph, x 3 is the fourth characteristic graph, x 4 is the fifth feature map, CBAM is the convolutional attention operation, EELANt is the efficient layer aggregation module, and MaxPooling is the maximum pooling operation.
6. The method for detecting smoke and fire on a power transmission line based on YOLOv7-tiny according to claim 3, characterized in that: The specific steps of step S402 are as follows: Step S4021: according to the third feature map, the fourth feature map and the fifth feature map, feature extraction is performed using a CBS convolution module and a CA attention layer respectively, and a sixth feature map with coordinate perception, a seventh feature map with coordinate perception and an eighth feature map with coordinate perception are obtained in one-to-one correspondence; Step S4022: According to the sixth feature map with coordinate perception and the seventh feature map with coordinate perception, a three-layer BasicBlock residual basic block and an ASFF2 secondary feature fusion module are used for fusion processing respectively, so as to obtain a ninth feature map incorporating features of the fourth feature map and a tenth feature map incorporating features of the third feature map in a one-to-one correspondence; Step S4023: inputting the ninth feature map incorporating the features of the fourth feature map and the tenth feature map incorporating the features of the third feature map into the three-layer CBS convolution module for activation, and inputting the eighth feature map with coordinate perception into the single-layer CBS convolution module for activation, to obtain three activated feature maps; Step S4024: input the three activated feature maps into the ASFF3 three-level feature fusion module for semantic fusion, and obtain three feature maps with mutually integrated features; Step S4025: According to the three feature maps that integrate features with each other, they are respectively input into the three-layer BasicBlock residual basic block and the single-layer CBS convolution module for processing to obtain three final feature maps, which are: a first final feature map, a second final feature map and a third final feature map.
7. The method for detecting smoke and fire in power transmission lines based on YOLOv7-tiny according to claim 6, characterized in that: The expressions of the first final feature map, the second final feature map and the third final feature map in step S4025 are respectively as follows: x 2,4 =CBS(B 3 (x 2,3 )) x 3,4 =CBS(B 3 (x 3,3 )) x 4,4 =CBS(B 3 (x 4,3 )) x 2,3 =ASFF3(x 2,2 ,upasmple(B 3 (x 3,2 )),upsample(CBS(x 4,1 ))) x 3,3 =ASFF3(x 3,2 ,downsample(B 3 (x 2,2 )),upsample(CBS(x 4,1 ))) x 4,3 =ASFF3(x 4,1 ,downsample(B 3 (x 2,2 )),downsample(B 3 (x 3,2 ))) x 2,2 =B 3 (ASFF2(x 2,1 ,upsample(CBS(x 3,1 )))) x 3,2 =B 3 (ASFF2(x 3,1 ,downsample(CBS(x 2,1 )))) B 3 (·)=BasciBlock(BasciBlock(BasciBlock(·))) CBS(·)=Silu(BatchNorm(Conv2d(·))) x 2,1 =CA(Silu(BatchNorm(Conv2d(x 2 )))) x 3,1 =CA(Silu(BatchNorm(Conv2d(x 3 )))) x 4,1 =CA(Silu(BatchNorm(Conv2d(x 4 )))) Among them, x 2,4 is the first final feature map, x 3,4 is the second final feature map, x 4,4 is the third final feature map, CBS is the CBS convolution operation, x 3,2 is the tenth feature map that incorporates the features of the third feature map, x 2,2 The ninth feature map that incorporates the features of the fourth feature map, B 3 is a three-layer BasicBlock residual basic block, BasicBlock is a BasicBlock residual basic operation, x 4,3 is the thirteenth feature map that integrates features, x 3,3 is the twelfth feature map that integrates features, x 2,3 is the eleventh feature map that integrates features. ASFF3 is a three-level feature fusion module. Downsample is downsampling, upsample is upsampling, and x 4,1 is the eighth feature map with spatial perception ability, x 3,1 is the seventh feature map with spatial perception ability, x 2,1 is the sixth feature map with spatial perception ability, CA is the coordinate attention operation, Silu is the adaptive activation function operation, BatchNorm is the batch normalization operation, Conv2d is the two-dimensional convolution operation, x 4 is the fifth characteristic graph, x 3 is the fourth characteristic graph, x 2 It is the third feature map.
8. The method for detecting smoke and fire on a power transmission line based on YOLOv7-tiny according to claim 2, characterized in that: In step S403, the detection head includes an implicit addition function ImplicitA, a two-dimensional convolution Conv2d and an implicit multiplication function ImplicitM; The expressions of the implicit addition function ImplicitA and the implicit multiplication function ImplicitM are as follows: ImplicitA(X) = X + θ; ImplicitM(X)=X*θ; Among them, θ is the learnable vector, X is the three final feature maps, ImplicitA is the implicit addition function, and ImplicitM is the implicit multiplication function.
Citation Information
Patent Citations
Small target detection method based on attention mechanism
CN114202672A
Method for detecting smoke and fire around power transmission line based on improved YOLOv4
CN114626439A
Lightweight small target detection method based on improved YOLOv7
CN116206185A
Steel surface defect detection method based on YOLOv7-ECD
CN116468716A
Power grid transmission line smoke detection method and system, electronic equipment and medium
CN117036361A
Cited By
Outdoor fire detection method based on multi-scale feature map
CN120976868A
