Device and method for smoke detection in power transmission lines
By introducing a combined structure of feature pyramid, pixel aggregation network and pooling network into the smoke detection model, the problem of poor smoke detection effect on transmission lines is solved, high-precision recognition and rapid detection of weak targets are achieved, and the detection effect is improved .
Patent Information
- Application Number
- CN202111016343.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-08-31
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2041-08-31
AI Technical Summary
In the prior art, the smoke detection effect of transmission lines is poor, especially the feature extraction ability of small targets is gradually weakened, resulting in unsatisfactory detection effect.
A smoke detection model is adopted, including the input end, feature extraction network, feature reprocessing network and prediction end. The feature reprocessing network combines feature pyramid (FPN), pixel aggregation network (PAN) and pooling network (SPP), through The combination of depth-direction splicing and pooling network improves feature extraction and prediction capabilities.
The effect of smoke detection in transmission lines is improved, the accuracy and speed of detection of weak targets is improved, the average average accuracy reaches 81.6%, and real-time performance is ensured, which can effectively identify smoke images of high-voltage transmission lines.
Smart Images

Figure CN114119454B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of electric power safety monitoring, and in particular to a device and method for smoke detection of power transmission lines. Background Art
[0002] As the national electricity demand continues to grow, the scale of high-voltage transmission line construction has gradually expanded. Due to the frequent occurrence of extreme climate or man-made wildfires in recent years, the reliability of transmission line operation and the safe operation of the power grid have been seriously affected. In the initial stage of a wildfire, the flames are relatively small and easily obscured, but the smoke characteristics that accompany a wildfire are easy to spread and have a large range, so smoke detection has become one of the important methods for fire detection in high-voltage transmission lines.
[0003] However, smoke morphology is easily affected by air and its own diffusion, and the change process is very complicated. In recent years, deep neural networks have achieved great success in the field of machine vision.
[0004] The application publication number is CN 110222559 A, and its name is "Smoke Image Detection Method and Device Based on Convolutional Neural Network". A deep separable convolutional neural network with a branch network structure is designed, and a deep separable convolutional structure is used to replace the conventional convolutional structure to achieve smoke detection.
[0005] Liu Lijuan, Chen Songnan, et al. wrote a paper titled A real-time smoke detection model based on improved SSD, Journal of Xinyang Normal University (Natural Science Edition), 2020(2). Based on the single-stage detection model SSD, multi-feature fusion and progressive pooling technology are used to realize real-time smoke detection.
[0006] However, as the network depth increases, the feature extraction capabilities of these two models for small targets gradually weaken, resulting in unsatisfactory detection results.
[0007] Existing technical problems and considerations:
[0008] How to solve the technical problem of poor smoke detection effect on transmission lines. Summary of the invention
[0009] The technical problem to be solved by the present invention is to provide a device and method for smoke detection in power transmission lines, so as to solve the technical problem that the smoke detection effect of power transmission lines is poor.
[0010] In order to solve the above technical problems, the technical solution adopted by the present invention is: a device for smoke detection in a transmission line is a smoke detection model, and the smoke detection model includes an input end Input, a feature extraction network Backbone, a feature reprocessing network Neck and a prediction end Prediction connected in sequence, and the feature reprocessing network Neck includes a feature pyramid FPN, a pixel aggregation network PAN and a pooling network SPP connected in sequence.
[0011] A further technical solution is that: the pixel aggregation network PAN includes third to fifth cross-stage local networks CSP2_3-3 to CSP2_3-5 with the same structure and a first convolution structure and a second convolution structure with the same structure, the third cross-stage local network CSP2_3-3 is connected to the first convolution structure, the first convolution structure is connected to the fourth cross-stage local network CSP2_3-4 after splicing in the depth direction, the fourth cross-stage local network CSP2_3-4 is connected to the second convolution structure, and the second convolution structure is connected to the fifth cross-stage local network CSP2_3-5 after splicing in the depth direction; the prediction end Prediction includes the same structure The first to third convolutional layers and the first to third output feature maps y1~y3 are the same, the first convolutional layer is connected to the first output feature map y1, the second convolutional layer is connected to the second output feature map y2, and the third convolutional layer is connected to the third output feature map y3; the pooling network SPP includes the first to third pooling networks with the same structure, the first pooling network is connected between the fifth cross-stage local network CSP2_3-5 and the first convolutional layer, the second pooling network is connected between the fourth cross-stage local network CSP2_3-4 and the second convolutional layer, and the third pooling network is connected between the third cross-stage local network CSP2_3-3 and the third convolutional layer.
[0012] A device for smoke detection of a power transmission line includes a smoke detection model module, wherein the smoke detection model module is a program module, and a processor is used to construct the above-mentioned smoke detection model.
[0013] A further technical solution is that it also includes three program modules, namely, a sample set acquisition module, a training module and a detection module. The sample set acquisition module is used for a processor to obtain a sample set, wherein the sample set includes a test sample set and a training sample set; the training module is used for a processor to input the training sample set into a smoke detection model until the number of training cycles is greater than 1000 and a trained smoke detection model is obtained; and the detection module is used for a processor to input the test sample set into a trained smoke detection model to obtain a category label, a smoke prediction box Bounding box and a confidence level in a smoke image.
[0014] A further technical solution is that in a sample set acquisition module, the sample set also includes a verification sample set, a camera acquires a smoke image near a power transmission line, a plurality of smoke images form a smoke image set, a processor manually annotates the smoke area in the smoke image with an annotation box and obtains an annotated image with an annotation box, a plurality of annotated images form an annotated image set, and the processor extracts and obtains a test sample set, a training sample set and a verification sample set from the annotated image set; in a training module, the processor inputs the training sample set into a smoke detection model, the smoke detection model outputs corresponding parameters of the predicted target box and the real target box, returns the loss function loss to the trained smoke detection model, uses a gradient descent algorithm to transfer the gradient loss to each convolution kernel in the smoke detection model, and updates the training parameters.
[0015] A further technical solution is: it also includes a camera, and the camera is connected to and communicates with the processor through a communication device.
[0016] A method for smoke detection of power transmission lines is based on an interconnected camera and a processor, and comprises the following steps: a step of constructing a smoke detection model, wherein the processor constructs the smoke detection model; a step of obtaining a sample set, wherein the processor obtains a sample set, wherein the sample set comprises a test sample set and a training sample set; a step of training, wherein the processor inputs the training sample set into the smoke detection model until the number of training cycles is greater than 1000 and obtains the trained smoke detection model; and a step of detecting, wherein the processor inputs the test sample set into the trained smoke detection model to obtain a category label, a smoke prediction box Bounding box in a smoke image, and a confidence level.
[0017] A further technical solution is that: in the step of obtaining a sample set, the sample set also includes a verification sample set, the camera obtains a smoke image near the transmission line, multiple smoke images form a smoke image set, the smoke area in the smoke image is manually annotated by a processor using a labeling box and an annotated image with an annotated box is obtained, multiple annotated images form an annotated image set, and the processor extracts and obtains a test sample set, a training sample set and a verification sample set from the annotated image set; in the training step, the processor inputs the training sample set into a smoke detection model, the smoke detection model outputs corresponding parameters of the predicted target box and the real target box, returns the loss function loss to the trained smoke detection model, uses the gradient descent algorithm to transfer the gradient loss to each convolution kernel in the smoke detection model, and updates the training parameters.
[0018] A device for smoke detection in a power transmission line comprises a memory, a processor and a computer program stored in the memory and executable on the processor. The computer program comprises a smoke detection model module, a sample set acquisition module, a training module and a detection module. The processor implements the above corresponding steps when executing the computer program.
[0019] A device for smoke detection in a power transmission line is a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes a smoke detection model module, a sample set acquisition module, a training module, and a detection module, and wherein the computer program implements the above corresponding steps when executed by a processor.
[0020] The beneficial effects of adopting the above technical solution are:
[0021] A device for smoke detection in power transmission lines is a smoke detection model, which includes an input terminal Input, a feature extraction network Backbone, a feature reprocessing network Neck and a prediction terminal Prediction connected in sequence, and the feature reprocessing network Neck includes a feature pyramid FPN, a pixel aggregation network PAN and a pooling network SPP connected in sequence. The smoke detection effect of the power transmission line is improved through the smoke detection model and the like.
[0022] A device for smoke detection of power transmission lines includes a smoke detection model module, which is a program module and a processor for constructing the smoke detection model. By constructing the smoke detection model module, etc., the smoke detection effect of the power transmission line is improved.
[0023] A method for smoke detection in power transmission lines is based on interconnected cameras and processors, and includes the following steps: a step of constructing a smoke detection model, in which the processor constructs the smoke detection model; a step of obtaining a sample set, in which the processor obtains a sample set, and the sample set includes a test sample set and a training sample set; a step of training, in which the processor inputs the training sample set into the smoke detection model until the number of training cycles is greater than 1000 and obtains a trained smoke detection model; a step of detecting, in which the processor inputs the test sample set into the trained smoke detection model to obtain a category label, a smoke prediction box Bounding box in a smoke image, and a confidence level. The method improves the smoke detection effect of power transmission lines by constructing a smoke detection model, obtaining a sample set, and the steps of training and detecting.
[0024] A device for smoke detection of power transmission lines includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program includes a smoke detection model module, a sample set acquisition module, a training module, and a detection module, and the processor implements the above corresponding steps when executing the computer program. The smoke detection effect of power transmission lines is improved by constructing a smoke detection model module, a sample set acquisition module, a training module, and a detection module.
[0025] A device for smoke detection of power transmission lines is a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes a smoke detection model module, a sample set acquisition module, a training module, and a detection module, and when the computer program is executed by a processor, the above corresponding steps are implemented. By constructing a smoke detection model module, a sample set acquisition module, a training module, and a detection module, the smoke detection effect of the power transmission line is improved.
[0026] Please refer to the detailed description of the specific implementation method. BRIEF DESCRIPTION OF THE DRAWINGS
[0027] Figure 1 is a structural diagram of Embodiment 1 of the present invention;
[0028] Figure 2 is a principle block diagram of embodiment 2 of the present invention;
[0029] Figure 3 is a flow chart of smoke detection in the present invention;
[0030] Figure 4 It is a schematic diagram of the Mosaic enhancement principle in the present invention;
[0031] Figure 5 It is a schematic diagram of the structure of the Focus module in the present invention;
[0032] Figure 6 It is a schematic diagram of the CSP1_3 module in the present invention;
[0033] Figure 7 It is a schematic diagram of the structure of the CSP2_3 module in the present invention;
[0034] Figure 8 It is a schematic diagram of the SPP module in the present invention;
[0035] Fig. 9 It is a schematic diagram of the GIOU principle in the present invention;
[0036] Fig.10 is the smoke image in the present invention;
[0037] Fig.11 It is the labeled image in the present invention. DETAILED DESCRIPTION
[0038] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. The following description of at least one exemplary embodiment is actually only illustrative and is by no means intended to limit the present application and its application or use. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0039] In the following description, many specific details are set forth to facilitate a full understanding of the present application, but the present application may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present application. Therefore, the present application is not limited to the specific embodiments disclosed below.
[0040] Embodiment 1:
[0041] like Figure 1 As shown, the present invention discloses a device for smoke detection in a power transmission line, which is a smoke detection model, namely, a YOLOv5l-SPP model.
[0042] The smoke detection model includes an input terminal Input, a feature extraction network Backbone, a feature reprocessing network Neck and a prediction terminal Prediction which are connected in sequence. The feature reprocessing network Neck includes a feature pyramid FPN, a pixel aggregation network PAN and a pooling network SPP which are connected in sequence.
[0043] The pixel aggregation network PAN includes third to fifth cross-stage local networks CSP2_3-3 to CSP2_3-5 with the same structure and a first convolution structure and a second convolution structure with the same structure. The third cross-stage local network CSP2_3-3 is connected to the first convolution structure, and the first convolution structure is connected to the fourth cross-stage local network CSP2_3-4 after splicing in the depth direction. The fourth cross-stage local network CSP2_3-4 is connected to the second convolution structure, and the second convolution structure is connected to the fifth cross-stage local network CSP2_3-5 after splicing in the depth direction.
[0044] The prediction end Prediction includes first to third convolutional layers and first to third output feature maps y1~y3 with the same structure, the first convolutional layer is connected to the first output feature map y1, the second convolutional layer is connected to the second output feature map y2, and the third convolutional layer is connected to the third output feature map y3.
[0045] The pooling network SPP includes first to third pooling networks with the same structure, the first pooling network is connected between the fifth cross-stage local network CSP2_3-5 and the first convolutional layer, the second pooling network is connected between the fourth cross-stage local network CSP2_3-4 and the second convolutional layer, and the third pooling network is connected between the third cross-stage local network CSP2_3-3 and the third convolutional layer.
[0046] Embodiment 2:
[0047] like Figure 2 As shown, the present invention discloses a device for smoke detection of power transmission lines, including a drone and a computer, and four program modules, namely, a smoke detection model module, a sample set acquisition module, a training module and a detection module. The drone is equipped with a camera, the computer includes a processor and a memory, and the camera is connected to and communicates with the computer via a wireless communication device.
[0048] A smoke detection model module is constructed, and a smoke detection model is constructed for the processor.
[0049] The sample set acquisition module is used for the processor to acquire a sample set, wherein the sample set includes a test sample set, a training sample set and a verification sample set.
[0050] In the sample set acquisition module, the camera acquires smoke images near the transmission lines, and multiple smoke images form a smoke image set. The smoke areas in the smoke images are manually annotated using a processor with an annotation box and annotated images with annotation boxes are obtained. Multiple annotated images form an annotated image set. The processor extracts and obtains a test sample set, a training sample set, and a verification sample set from the annotated image set.
[0051] The training module is used for the processor to input the training sample set into the smoke detection model until the number of training cycles is greater than 1000 and obtain the trained smoke detection model.
[0052] In the training module, the processor inputs the training sample set into the smoke detection model, and the smoke detection model outputs the corresponding parameters of the predicted target box and the real target box. It returns the loss function loss to the trained smoke detection model, and uses the gradient descent algorithm to pass the gradient loss to each convolution kernel in the smoke detection model to update the training parameters.
[0053] The detection module is used for the processor to input the test sample set into the trained smoke detection model to obtain the category label, the smoke prediction box Bounding box in the smoke image and the confidence.
[0054] Among them, the drone, camera, wireless communication device, computer itself and corresponding communication connection technology are existing technologies and will not be described in detail here.
[0055] Embodiment 3:
[0056] The present invention discloses a method for smoke detection in a power transmission line, based on the device of embodiment 2, comprising the following steps:
[0057] In the step of building a smoke detection model, the processor builds a smoke detection model.
[0058] In the step of obtaining a sample set, the processor obtains a sample set, and the sample set includes a test sample set, a training sample set, and a verification sample set.
[0059] In the step of obtaining a sample set, a camera obtains smoke images near a transmission line, and multiple smoke images form a smoke image set. A processor manually annotates smoke areas in the smoke images with annotation boxes and obtains annotated images with annotation boxes. Multiple annotated images form an annotated image set. The processor extracts and obtains a test sample set, a training sample set, and a verification sample set from the annotated image set.
[0060] In the training step, the processor inputs the training sample set into the smoke detection model until the number of training cycles is greater than 1000 and obtains a trained smoke detection model.
[0061] In the training step, the processor inputs the training sample set into the smoke detection model, and the smoke detection model outputs the corresponding parameters of the predicted target box and the real target box, returns the loss function loss to the trained smoke detection model, and uses the gradient descent algorithm to pass the gradient loss to each convolution kernel in the smoke detection model to update the training parameters.
[0062] In the detection step, the processor inputs the test sample set into the trained smoke detection model to obtain the category label, the smoke prediction box Bounding box in the smoke image, and the confidence level.
[0063] Embodiment 4:
[0064] The present invention discloses a device for smoke detection of power transmission lines, including a memory, a processor, and a computer program stored in the memory and executable on the processor. The computer program includes a smoke detection model module, a sample set acquisition module, a training module, and a detection module. When the processor executes the computer program, the steps of Example 3 are implemented.
[0065] Embodiment 5:
[0066] The present invention discloses a computer-readable storage medium, which stores a computer program. The computer program includes a smoke detection model module, a sample set acquisition module, a training module and a detection module. When the computer program is executed by a processor, the steps in Example 3 are implemented.
[0067] The concept of this application:
[0068] The original YOLOv5l model did not reach 80% of the detection MAP for smoke images. In this field, every increase in the MAP value is a big challenge. The smoke in the image has irregular shapes, lighter colors, and uncertain regions, which makes the neural network model lose its features during the convolution process as the network continues to deepen. The pooling network SPP module can fuse low-level features. Therefore, the present invention considers applying the SPP module at the end of the model feature reprocessing part to improve the detection effect. After a large number of experiments, the final number and position of the SPP module were confirmed.
[0069] In view of the shortcomings of the prior art, the technical problem to be solved by the present invention is to provide a method for smoke image detection of power transmission lines based on the YOLOv5l-SPP model. The detection method is based on the YOLOv5l framework of model transfer training, and an SPP network structure is additionally added before the output detection head. The method is used for smoke image detection of power transmission lines, and has higher accuracy, adaptability and real-time performance. The mean average accuracy of defect recognition reaches 81.6%, which solves the problem that the detection accuracy and speed of smoke images of high-voltage transmission lines cannot achieve the detection effect at the same time.
[0070] Technical contributions of this application:
[0071] Based on the original YOLOv5l model, the SPP module is additionally integrated in front of the last convolutional layer conv before each output scale in the YOLOv5l model, that is, between the convolutional layer conv and the CSP2-3 modules.
[0072] The existing technology only has the YOLOv5l model, and no other models are added.
[0073] Technical solution description:
[0074] like Figure 3 As shown, smoke detection is divided into the following steps:
[0075] The first step is image preprocessing:
[0076] 1-1 Image acquisition: Use drone inspection to collect smoke images (in jpg format) near high-voltage lines and use them as the original data set.
[0077] 1-2 Preparation of test sample set: Use LabelImg software to manually select and annotate the images in step 1-1, and add defect type labels. Modify all smoke images with existing annotated information, including adjusting the annotation box size, changing Chinese labels to English labels, removing other defect labels and deleting images obtained by abnormal shooting, and obtain an xml file (this file is automatically generated when the LabelImg software annotates each image and saves it). Then, convert the xml annotation files of the 3332 smoke defect images that have been completely annotated and modified to a unified standard into txt files and json files to obtain the data set. Finally, randomly (random method is not limited) extract 16% of the images from the smoke images as the test sample set.
[0078] 1-3 Preparation of training sample set and verification sample set: The remaining 84% of smoke images from step 1-2 are used as training sample set and verification sample set.
[0079] The second step is to build the YOLOv5l-SPP model:
[0080] like Figure 1 As shown in the figure, the YOLOv5l-SPP model includes the input terminal Input, the feature extraction network Backbone, the feature reprocessing network Neck and the prediction terminal Prediction.
[0081] 2-1 Input design: The original YOLOv5l model uses Mosaic data enhancement for the input image to enrich the image content and improve the detection effect for shallow smoke or small smoke areas.
[0082] Mosaic data enhancement has the following characteristics:
[0083] like Figure 4 As shown, first, a batch (batch refers to Batch, batch refers to Batch size, which is a hyperparameter of the model, and the batch is equal to 64 in this embodiment) of images is taken from the training sample set; secondly, 4 images are randomly selected from the batch of images, and the four images are randomly operated by color gamut change, reduction, inversion and / or cropping, and each image is subjected to at least one operation; then the 4 images are arranged in four directions of upper left, lower left, upper right, and lower right, and then spliced to obtain a new image, and the size of the new image is the same as the size of the original image that has not been operated, which is 608×608×3; repeat the above operation, and set the number of cycles equal to the Batch size.
[0084] 2-2 Feature extraction network design: The image obtained in step 2-1 is sent to the Focus module. The Focus module connects three groups of CBL (convolution layer-batch normalization-activation function Leaky Relu, CBL) convolution structures and a cross-stage local network CSP1_X module, and then connects a pooling network SPP module to form a feature extraction network; the image is sliced by using the Focus module, and then sent to the structure composed of CBL convolution structure and CSP1_X module after passing through the Focus module, and then the low-level features and high-level features are fused through the SPP module.
[0085] The Focus module has the following features:
[0086] like Figure 5 As shown, first, the original image of size 608×608×3 is divided into four numbers 1, 2, 3, and 4 according to Figure 4 Mark them in this way, and then mark the pixels with the same number according to Figure 4 The four parts are then concatenated into a feature map of 304×304×12 in the depth direction according to the digital size, and then connected to a CBL convolution structure.
[0087] The characteristics of the CBL convolution structure contained in the Focus module are: the number of convolution kernels in the convolution layer (convolution, conv) is 64, the size is 3×3, and the stride is 1.
[0088] The CSP1_X module has the following features:
[0089] X represents the number of residual structures, and other structures are the same except for the number of residual structures.
[0090] like Figure 6 As shown in the figure, taking the CSP1_3 module as an example, the CSP1_3 module first performs a CBL convolution operation on the input feature map; then it is sent to three residual structures, and then the feature map after the residual structure is convolved, and the new feature map obtained by direct convolution with the input feature map is concat in the depth direction; finally, it is input into the next module through batch normalization, activation function Leaky relu and a layer of CBL convolution structure.
[0091] The convolution layer conv of the CBL convolution structure that directly convolves the input feature map in the CSP1_X module is the same as the convolution layer conv in the last CBL convolution structure in the CSP1_X module. The relevant dimensions of the convolution layer conv are: the convolution kernel size is 1×1, and the stride is 1.
[0092] 2-3 Feature reprocessing network design: The feature pyramid FPN and pixel aggregation network PAN structures are adopted, where the FPN structure is composed of two groups of CSP2_3 modules, CBL convolution structure and up sample structure in series; PAN includes two CBL convolution structures to downsample the data.
[0093] like Figure 1 As shown in the figure, the output of each upsampling structure in FPN and the output feature map of the CSP1_9-1 and CSP1_9-2 modules in the feature extraction network are concatenated in the depth direction. At the same time, the output of each CBL convolution structure in FPN and the feature map of the corresponding size of the CBL convolution structure in PAN are concatenated in the depth direction. A CSP2_3 module and an SPP module are added to the PAN structure each time after the CBL convolution structure, and a CSP2_3 module and an SPP module are also added before the first CBL convolution structure of the PAN structure; Neither the CSP2_3 nor the SPP modules change the size of the feature map. The output of the CSP2_3-3 module in Neck is 76×76, and then connected to the CBL convolution structure to change the image size to 38×38. The output of the CSP2_3-4 module is connected to the input of the CBL convolution structure to change the image to 19×19.
[0094] FPN includes two CBL convolution structures, but the CBL here are all convolutions with a step size of 1, which has no effect on the size of the feature map. It is the upsampling in FPN that changes the size of the feature map and learns multi-scale target information.
[0095] The cross-stage partial network CSP2_3 module has the following characteristics:
[0096] like Figure 7 As shown in the figure, the module structure is the same as the structure after the add fusion process is deleted in each residual structure of the CSP1_3 module. The CSP2_3 module includes an initial CBL convolution structure and a final CBL convolution structure, multiple repeating units, a 1×1 convolution layer is connected to a 3×3 convolution layer after batch normalization and activation function Leaky relu, and the 3×3 convolution layer is connected to batch normalization and activation function Leaky relu to form a repeating unit. There are three repeating units. The input of the first repeating unit is connected to the output of the initial CBL convolution structure. The outputs of the three repeating units are connected in series to a convolution layer. The output of the convolution layer is concatenated with the original input after passing through a convolution layer, and then connected to the final CBL convolution structure after batch normalization and activation function Leaky relu.
[0097] The CBL convolution structure in FPN has the following characteristics: the convolution kernel size is 1×1 and the step size is 1.
[0098] The CBL convolution structure in PAN has the following characteristics: the convolution kernel size is 3×3 and the stride is 2.
[0099] The SPP module used in step 2-1 and step 2-2 has the following characteristics:
[0100] like Figure 8 As shown in Figure 1, the SPP module consists of four parallel pooling layers with maximum pooling kernel sizes of 1×1, 5×5, 9×9, and 13×13, respectively. The SPP module itself contains two layers of CBL convolutional structures at the beginning and end.
[0101] like Figure 1 As shown, its position can be known, and the SPP module is additionally integrated in front of the last convolution layer conv before each output scale in the YOLOv5l model, that is, between the convolution layer conv and the CSP2-3 module.
[0102] 2-4 Prediction end design: The output of the YOLOv5l model inherits the idea of YOLOv3 and uses three scale feature maps for detection, with feature map sizes of 19×19, 38×38, and 76×76. A corresponding number of anchor boxes are assigned to each scale after each SPP module added in step 2-2. Nine anchor boxes are generated for each pixel in the feature map. The optimal box is selected through weighted non-maximum suppression, and GIOU is returned to the network as the loss function to train the parameters.
[0103] The loss function has the following characteristics:
[0104] like Fig. 9 As shown in the figure, the outermost square represents the image, the gray filled ellipse represents the detected target smoke, the solid rectangular box G in the lower left corner represents the ground truth box; the solid rectangular box B in the upper right corner represents the predicted target box Bounding box output by the model; the dotted line box Ac represents the minimum circumscribed rectangular box of G and B; the part filled with the dotted line “\” represents the union of G and B, represented by U; the part I filled with the dotted line “\” and “ / ” at the same time represents the intersection of G and B. IOU is equal to the area of the intersection part I of G and B divided by the area of the union part U.
[0105] Let S represent the area, and the loss function formula is as follows:
[0106]
[0107] The third step is image feature extraction and defect detection:
[0108] 3-1 Parameter initialization: Using the model migration method, the YOLOv5l model parameters trained on the COCO dataset are used as the initial parameters for training the smoke dataset. The initial parameters of the SPP module added by the present invention are 0, and the settings of the hyperparameters are shown in Table 1:
[0109] Table 1: Hyperparameter settings of the model
[0110]
[0111] 3-2 Start training: Input the training sample set into the neural network model after the parameters are initialized in step 3-1.
[0112] 3-3 Update parameters: Compare the parameters of the predicted target box and the real target box output in step 3-2, return the loss function loss to the neural network model trained in step 3-2, and use the gradient descent algorithm to pass the gradient loss to the parameters of each convolution kernel to update the trainable parameters.
[0113] 3-4 Stop training: Repeat training step 3-3. When the number of training cycles is greater than 1000, stop training to obtain the final YOLOv5l-SPP neural network model.
[0114] 3-5 Test the smoke detection effect: Input the test sample set data into the final neural network model to obtain the category label, i.e. smoke, the smoke prediction box Bounding box in the image, and the confidence level.
[0115] like Fig.10 The image shown is the input model.
[0116] like Fig.11 The image shown is the output from the model.
[0117] 3-6 End the high-voltage line smoke image detection.
[0118] In the method of the present invention, when starting training, the weights are first initialized, and the weights are continuously optimized during the training process. The YOLOv5l-SPP model uses the GIOU loss between the output prediction box and the true target box to reversely control the change of network weights, and uses stochastic gradient descent SGD to solve the optimal value of back propagation, and updates the trainable parameters through this algorithm.
[0119] This example was completed on the Centos7.9.2 platform and implemented using Python programming. The computer performance for testing and training the network model was: Tesla v100, Intel Xeon(R) Gold 6271c CPU@2.6GHz; the framework used was the pytorch deep learning framework. The learning rate of the YOLOv5l-SPP model was selected as λ=0.01, and the number of training steps was 450.
[0120] The present invention utilizes MAP and detection speed to measure network performance.
[0121] Define precision, recall, mean average precision and detection speed as:
[0122]
[0123]
[0124]
[0125]
[0126] In formula (2), TP represents true positive, that is, the image marked as defective is correctly detected; FP represents false positive, that is, the image marked as good is wrongly detected as defective.
[0127] In formula (3), FN represents false negative, that is, an image marked as defective is mistakenly detected as non-defective.
[0128] In formula (4), k is the defect type, which is equal to 1 in the present invention; f i (p,r) represents the function of the relationship between precision and recall of the i-th class.
[0129] In formula (5), n is the number of images; t is the detection time.
[0130] The specific indicators of different models are shown in Table 2:
[0131] Table 2: Comparison of MAP values and detection speed of the models
[0132]
[0133] As shown in Table 2, the model with the best MAP among other detection models is Transformer, which reaches 80.4%, but it is still 1.2 percentage points lower than the model of the present invention, and the detection speed lags behind by nearly 170fps; on the other hand, the detection speed of the model proposed by the present invention is only 2.5fps lower than the YOLOv3 model with the fastest detection speed among other models, but the MAP value is 3 percentage points higher. The YOLOv5s model has the fastest detection speed, which is more than twice that of the model of the present invention, but the MAP is the lowest, which is 8 percentage points lower than that of the model of the present invention; the YOLOv5l model has the highest MAP value among the first four models, and the detection speed also reaches 200fps, but the MAP of the model of the present invention after adding the SPP module increases by 1.7 percentage points at the expense of a part of the speed.
[0134] The method of the present invention has a high detection speed of about 170fps while ensuring that the MAP index reaches more than 80%, can meet the requirements of accuracy and real-time speed at the same time, and can be widely used in smoke detection of high-voltage power lines.
[0135] After this application was run confidentially for a period of time, the on-site technicians reported that the benefits were:
[0136] The present invention uses the YOLOv5l model as the framework, which significantly improves the detection effect of faint targets, while ensuring a faster detection speed and realizing accurate positioning of faint targets.
[0137] The YOLOv5l model integrates data enhancement, Focus structure, cross-stage local structure CSP and pooling network SPP on the basis of YOLOv3, strengthens the feature extraction effect, and uses the loss function GIOU_Loss and weighted non-maximum suppression at the output to improve the screening effect of multi-target frames. At the same time, by using the technical method of model migration to train the parameters of the YOLOv5l-SPP model, it can avoid the problem that the detection effect cannot reach the expected level due to the small number of samples and low versatility of model parameters when directly using a self-made data set to train the neural network. Model migration uses the weights trained by the model on a large data set as the initial weights for training the target data, which can improve the generalization ability of the model. The YOLOv5l-SPP model trained by the COCO data set in the present invention detects smoke images, has a stronger feature extraction ability, and can effectively improve the detection effect of weak targets such as smoke.
[0138] For the smoke image of transmission lines, since the smoke is diffuse and has uncertain position, this application adds an SPP module to the detection end of YOLOv5l, which can realize the fusion of features of different scales and has higher recognition accuracy for smoke image detection.
[0139] The mean average precision MAP of smoke image detection exceeds 80%, and the detection speed is also higher than 150fps.
Claims
1. A device for smoke detection in power transmission lines, Features: The device is a smoke detection model, which includes an input terminal Input, a feature extraction network Backbone, a feature reprocessing network Neck and a prediction terminal Prediction connected in sequence, the feature reprocessing network Neck includes a feature pyramid FPN, a pixel aggregation network PAN and a pooling network SPP connected in sequence; the pixel aggregation network PAN includes third to fifth cross-stage local networks CSP2_3-3 to CSP2_3-5 with the same structure and a first convolution structure and a second convolution structure with the same structure, the third cross-stage local network CSP2_3-3 is connected to the first convolution structure, the first convolution structure is connected to the fourth cross-stage local network CSP2_3-4 after splicing in the depth direction, the fourth cross-stage local network CSP2_3-4 is connected to the second convolution structure, the second convolution structure is connected to the fifth cross-stage local network CSP2_3-5 after splicing in the depth direction; the prediction terminal Prediction tion includes first to third convolutional layers with the same structure and first to third output feature maps y1~y3, the first convolutional layer is connected to the first output feature map y1, the second convolutional layer is connected to the second output feature map y2, and the third convolutional layer is connected to the third output feature map y3; the pooling network SPP includes first to third pooling networks with the same structure, the first pooling network is connected between the fifth cross-stage local network CSP2_3-5 and the first convolutional layer, the second pooling network is connected between the fourth cross-stage local network CSP2_3-4 and the second convolutional layer, and the third pooling network is connected between the third cross-stage local network CSP2_3-3 and the third convolutional layer; a smoke image near the transmission line is obtained, multiple smoke images form a smoke image set, the smoke area in the smoke image is annotated with a labeling box and an annotated image with an annotated box is obtained, multiple annotated images form an annotated image set, and a sample set is extracted from the annotated image set and obtained.
2. A device for smoke detection in power transmission lines, Features: The invention comprises a smoke detection model module, wherein the smoke detection model module is a program module, and is used for a processor to construct the smoke detection model in claim 1.
3. The device for smoke detection in power transmission lines according to claim 2, Features: The module also includes three program modules, namely, a sample set acquisition module, a training module and a detection module. The sample set acquisition module is used for a processor to acquire a sample set, wherein the sample set includes a test sample set and a training sample set. A training module, used for the processor to input the training sample set into the smoke detection model until the number of training cycles is greater than 1000 and obtain a trained smoke detection model; The detection module is used for the processor to input the test sample set into the trained smoke detection model to obtain the category label, the smoke prediction box Bounding box in the smoke image and the confidence.
4. The device for smoke detection in power transmission lines according to claim 3, Features: In the sample set acquisition module, the sample set also includes a verification sample set. The camera acquires a smoke image near the power transmission line. The smoke area in the smoke image is manually annotated with a labeling frame through a processor to obtain an annotated image with an annotated frame. The processor extracts and obtains a test sample set, a training sample set, and a verification sample set from the annotated image set. In the training module, the processor inputs the training sample set into the smoke detection model, and the smoke detection model outputs the corresponding parameters of the predicted target box and the real target box. It returns the loss function loss to the trained smoke detection model, and uses the gradient descent algorithm to pass the gradient loss to each convolution kernel in the smoke detection model to update the training parameters.
5. The device for smoke detection in power transmission lines according to claim 2, Features: A camera is also included, and the camera is connected and communicates with the processor through a communication device.
6. A method for smoke detection in power transmission lines, Features: Based on interconnected cameras and processors, the following steps are included: A step of constructing a smoke detection model, wherein the processor constructs the smoke detection model described in claim 1; In the step of obtaining a sample set, the processor obtains a sample set, wherein the sample set includes a test sample set and a training sample set; In a training step, the processor inputs the training sample set into the smoke detection model until the number of training cycles is greater than 1000 and obtains a trained smoke detection model; In the detection step, the processor inputs the test sample set into the trained smoke detection model to obtain the category label, the smoke prediction box Bounding box in the smoke image, and the confidence level.
7. The method for smoke detection in power transmission lines according to claim 6, Features: In the step of obtaining a sample set, the sample set also includes a verification sample set. The camera obtains a smoke image near the transmission line, and multiple smoke images form a smoke image set. The smoke area in the smoke image is manually annotated with a labeling box through a processor and a labeled image with a labeling box is obtained. The multiple labeled images form a labeled image set. The processor extracts and obtains a test sample set, a training sample set and a verification sample set from the labeled image set; in the training step, the processor inputs the training sample set into a smoke detection model, and the smoke detection model outputs corresponding parameters of the predicted target box and the real target box, returns the loss function loss to the trained smoke detection model, uses the gradient descent algorithm to pass the gradient loss to each convolution kernel in the smoke detection model, and updates the training parameters.
8. A device for smoke detection in power transmission lines, Features: The invention comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the computer program comprises a smoke detection model module, a sample set acquisition module, a training module and a detection module, and the processor implements the steps in claim 6 when executing the computer program.
9. A device for smoke detection in power transmission lines, Features: The device is a computer-readable storage medium, which stores a computer program. The computer program includes a smoke detection model module, a sample set acquisition module, a training module and a detection module. When the computer program is executed by a processor, the steps in claim 6 are implemented.
