Power transmission line external damage detection method based on integrated target detection and incremental learning

Abstract

The invention discloses a power transmission line external damage detection method based on integrated target detection and incremental learning, and the method comprises the following steps: obtaining abnormal object picture data of a power transmission line, carrying out the data enhancement of the abnormal object picture data, and recording the data as enhanced data; establishing a hybrid model, and recording the hybrid model as an integrated model; training the enhanced data by adopting an integrated model, and recording the data as integrated data; deploying the integrated data to an edge server for detecting an abnormal object in a real-time image on the power transmission line; picture data with abnormal objects in the edge server is uploaded to a cloud server to serve as a new training sample for incremental learning, and the integrated model is updated; repeating the steps to realize continuous optimization of the system; if no abnormal object picture data exists, ending. According to the invention, through combination of incremental learning and an integrated target detection technology, accurate and timely detection of abnormal objects near the power transmission line is realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of automated power transmission line inspection, and relates to a power transmission line external damage detection method based on integrated target detection and incremental learning. Background Art

[0002] The safety and reliability of transmission lines are crucial to the stability of the power grid. Transmission lines generally have long routes, complex and changeable terrain along the lines, large regional spans, large environmental and climatic influences, and long inspection times. In addition, a variety of factors threaten the safety of power grid infrastructure, such as abnormal events such as wildfires and smoke, as well as human damage caused by working engineering machinery. With the continuous development of my country's power system, the continuous extension of the installation area, and the continuous growth of transmission lines, the requirements for the safety and stability of the power system are becoming higher and higher.

[0003] When high-voltage transmission lines are long and dispersed, passing through complex terrain conditions such as mountains and rivers, it becomes more difficult to continuously monitor the transmission lines to detect abnormal targets through traditional methods such as manpower, mobile robots or helicopters, because it involves a lot of manpower investment and high startup and maintenance costs. Recently, vision-based automated inspection systems have been proposed for monitoring transmission lines, which involve a series of processing techniques such as image acquisition and enhancement, segmentation, feature extraction, target localization and classification. However, the computational workload of model training is large, and the testing is often transferred to cloud servers far away from the equipment installed on the transmission line, and the bandwidth of the equipment is also limited, so there will be a large delay in uploading the collected images.

[0004] Therefore, how to accurately and timely detect whether there are abnormal objects near the transmission lines so that administrators can be alerted immediately and some measures can be taken to ensure the safety of the power grid remains a challenging problem. Summary of the invention

[0005] The purpose of the present invention is to provide a power transmission line external damage detection method based on integrated target detection and incremental learning, which solves the problems of inaccurate abnormal object recognition, poor real-time performance and large data transmission delay in transmission line monitoring in the prior art.

[0006] The technical solution adopted by the present invention is a power transmission line external damage detection method based on integrated target detection and incremental learning, comprising the following steps: Step 1, obtaining abnormal object image data of the transmission line on the cloud server, and performing data enhancement on the abnormal object image data, which is recorded as enhanced data; Step 2: Establish a hybrid model of Swin Transformer and Mask R-CNN, recorded as an integrated model; Step 3: Use the integrated model to train the enhanced data, recorded as integrated data; Step 4: deploy the integrated data to the edge server to detect abnormal objects in the real-time image on the transmission line; Step 5, determine whether there is new abnormal object image data; Step 6: If yes, upload the image data with abnormal objects in the edge server to the cloud server as new training samples for incremental learning, update the integrated model, and repeat steps 1-6 to achieve continuous optimization of the system; Step 7: If no, end.

[0007] The present invention is also characterized in that: Step 1 specifically includes the following steps: Step 101, constructing a power line anomaly data set with 7 categories and a total of 926 samples, recorded as abnormal object image data; Step 102, using OpenCV-based data enhancement technology to enhance the data of the abnormal object image data, without changing the data features of the original abnormal object image data, by simulating image changes under various shooting conditions, to enrich the training data set, recorded as enhanced data; Step 103, data enhancement technology mainly includes image compression, image cropping, image rotation and color transformation.

[0008] Abnormal object image data includes abnormal shooting angles, images with a large amount of interference information, and abnormal data sets with severe target occlusion or overlap.

[0009] The seven categories are construction machinery, cranes, tower cranes, fire, smoke, shelters and hanging objects.

[0010] Data augmentation is used to increase the number of samples and improve the generalization ability of the integrated model.

[0011] The Swin Transformer is responsible for extracting hierarchical feature representations from the input image, which are then fed into the Mask R-CNN part for object detection and segmentation mask generation.

[0012] Step 3 uses an integrated model to train the enhanced data, which specifically includes the following steps: Step 301, the integrated model receives enhanced data; Step 302, using the Swin Transformer module to extract features from the enhanced data. Swin Transformer captures global and local features in the enhanced data through its hierarchical structure and self-attention mechanism, flattens the feature map, divides the feature map into several small blocks, and then performs a self-attention mechanism inside the small blocks. The small blocks are reorganized by moving the spatial position to form a new feature map, which is recorded as a reorganized feature; Step 303, mapping the extracted recombined features into a unified feature space, converting the features at different levels into a unified representation form, recorded as mapping features, so as to ensure that the feature information at different levels can be effectively integrated; Step 304: Input the extracted recombined features and mapped features into the Mask R-CNN model, perform mask processing in the self-attention mechanism, and form new modules in three different ways. The flattened feature map forms a Mask matrix. Mask R-CNN uses the features in the Mask matrix to identify objects in the image and generate a bounding box of the object. Step 305: output the category and bounding box position of the object.

[0013] Step 6: Incremental learning specifically includes: using existing data to build an initial model, and after receiving new data, updating the model to adapt to the new data features, thereby improving the recognition accuracy of abnormal objects.

[0014] The beneficial effects of the present invention are: The present invention proposes a method for detecting external damage of transmission lines based on integrated target detection and incremental learning for photos of transmission lines taken by cameras. A hybrid model of Swin Transformer and Mask R-CNN, i.e. an integrated model, is adopted to make full use of the advantages of Swin Transformer in feature extraction and the precise performance of Mask R-CNN in target positioning to achieve more efficient and accurate image recognition and processing. At the same time, the cloud-edge fusion architecture is adopted to distribute computing power between the cloud and the edge, thereby achieving rapid data processing and decision-making. In order to improve the accuracy and generalization ability of detection, the incremental learning method is adopted to enable the model to continuously learn and improve in the process of continuously receiving new data, and quickly adapt to new changes to reduce training time and computing costs. The present invention realizes accurate and timely detection of abnormal objects near transmission lines by combining incremental learning and integrated target detection technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a flow chart of a method for detecting external failure of a power transmission line based on integrated target detection and incremental learning of the present invention; Figure 2It is a network framework diagram of the SwinTransformer method for detecting external damage of power transmission lines based on integrated target detection and incremental learning of the present invention; Figure 3 This is a network framework diagram of the Mask R-CNN method for detecting external damage to power transmission lines based on integrated target detection and incremental learning in the present invention.

[0016] Figure 4 This is a detection effect diagram of the Mask R-CNN model of the power transmission line external damage detection method based on integrated target detection and incremental learning of the present invention; Figure 5 This is a diagram showing the detection effect of using enhanced data training for the Mask R-CNN model of the power transmission line external damage detection method based on integrated target detection and incremental learning in the present invention; Figure 6 This is a detection effect diagram of the integrated model of the power transmission line external damage detection method based on integrated target detection and incremental learning, which is a mixture of SwinTransformer and Mask R-CNN, using enhanced data training. DETAILED DESCRIPTION

[0017] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0018] like Figure 1 As shown, the power transmission line external damage detection method based on integrated target detection and incremental learning provided by the present invention comprises the following steps: Step 1, obtain abnormal object image data of the transmission line on the cloud server, and perform data enhancement on the abnormal object image data, which is recorded as enhanced data; the abnormal object image data includes abnormal shooting angles, images with a large amount of interference information, and abnormal data sets with severe target occlusion or overlap; Step 101, constructing a power line anomaly data set with 7 categories and a total of 926 samples, recorded as abnormal object image data; the 7 categories are specifically construction machinery, cranes, tower cranes, fire, smoke, obstructions and hanging objects; Step 102, using OpenCV-based data enhancement technology to enhance the data of the abnormal object image data, without changing the data features of the original abnormal object image data, by simulating image changes under various shooting conditions, to enrich the training data set, recorded as enhanced data; Step 103, data enhancement technology mainly includes image compression, image cropping, image rotation and color transformation; Step 2: Build a hybrid model of Swin Transformer and Mask R-CNN, referred to as an integrated model. Data augmentation is used to increase the number of samples and improve the generalization ability of the integrated model. Swin Transformer is responsible for extracting hierarchical feature representations from the input image, which are then fed into the Mask R-CNN part for object detection and segmentation mask generation. Step 201, the Swin Transformer model network framework diagram is as follows Figure 2 As shown in the figure, the model goes through four stages in the feature extraction process. The first stage includes 1 linear encoding module and 2 consecutive Swin Transformer modules; the second stage includes 1 region merging module and 2 consecutive Swin Transformer modules. The region merging module interacts by fusing adjacent regions to avoid information loss; the third stage includes 1 region merging module and 6 Swin Transformer modules; the last stage includes 1 region merging module and 2 consecutive Swin Transformer modules, which are used to expand the perception range and extract deep features; Step 202, the network framework structure diagram of the Mask R-CNN model is as follows Figure 3 As shown in the figure, first input the picture, generate the image feature map after processing by the deep residual network and feature pyramid network, then use the region proposal network RPN to generate candidate boxes on the feature map, and then align the feature map pixels with the original image through ROI Align, and finally use the FCN full convolutional network to realize target detection and semantic segmentation functions; Step 3: Use the integrated model to train the enhanced data, recorded as integrated data; Step 301, the integrated model receives enhanced data; Step 302, using the Swin Transformer module to extract features from the enhanced data. Swin Transformer captures global and local features in the enhanced data through its hierarchical structure and self-attention mechanism, flattens the feature map, divides the feature map into several small blocks, and then performs a self-attention mechanism inside the small blocks. The small blocks are reorganized by moving the spatial position to form a new feature map, which is recorded as a reorganized feature; Step 303, mapping the extracted recombined features into a unified feature space, converting the features at different levels into a unified representation form, recorded as mapping features, so as to ensure that the feature information at different levels can be effectively integrated; Step 304: Input the extracted recombined features and mapped features into the Mask R-CNN model, perform mask processing in the self-attention mechanism, and form new modules in three different ways. The flattened feature map forms a Mask matrix. Mask R-CNN uses the features in the Mask matrix to identify objects in the image and generate a bounding box of the object. Step 305, output the category and bounding box position of the object; Step 4: deploy the integrated data to the edge server to detect abnormal objects in the real-time image on the transmission line; Step 5, determine whether there is new abnormal object image data; Step 6: If yes, upload the image data with abnormal objects in the edge server to the cloud server as new training samples for incremental learning, update the integrated model, and repeat steps 1-6 to achieve continuous optimization of the system; incremental learning specifically includes: using existing data to build an initial model, and updating the model after receiving new data to adapt to the new data features, thereby improving the recognition accuracy of abnormal objects; Step 7: If no, end.

[0019] Example 1 like Figure 1 As shown, the power transmission line external damage detection method based on integrated target detection and incremental learning provided in this embodiment includes the following steps: Step 1, obtain abnormal object image data of the transmission line on the cloud server, and perform data enhancement on the abnormal object image data, which is recorded as enhanced data; the abnormal object image data includes abnormal shooting angles, images with a large amount of interference information, and abnormal data sets with severe target occlusion or overlap; Step 2: Build a hybrid model of Swin Transformer and Mask R-CNN, referred to as an integrated model. Data augmentation is used to increase the number of samples and improve the generalization ability of the integrated model. Swin Transformer is responsible for extracting hierarchical feature representations from the input image, which are then fed into the Mask R-CNN part for object detection and segmentation mask generation. Step 3: Use the integrated model to train the enhanced data, recorded as integrated data; Step 305, output the category and bounding box position of the object; Step 4: deploy the integrated data to the edge server to detect abnormal objects in the real-time image on the transmission line; Step 5, determine whether there is new abnormal object image data; Step 6: If yes, upload the image data with abnormal objects in the edge server to the cloud server as new training samples for incremental learning, update the integrated model, and repeat steps 1-6 to achieve continuous optimization of the system; Step 7: If no, end.

[0020] Example 2 Based on Example 1, this embodiment provides a method for detecting power line failure based on integrated target detection and incremental learning, such as Figure 1 As shown, the following steps are included: Step 1, obtain abnormal object image data of the transmission line on the cloud server, and perform data enhancement on the abnormal object image data, which is recorded as enhanced data; the abnormal object image data includes abnormal shooting angles, images with a large amount of interference information, and abnormal data sets with severe target occlusion or overlap; Step 101, constructing a power line anomaly data set with 7 categories and a total of 926 samples, recorded as abnormal object image data; the 7 categories are specifically construction machinery, cranes, tower cranes, fire, smoke, obstructions and hanging objects; Step 102, using OpenCV-based data enhancement technology to enhance the data of the abnormal object image data, without changing the data features of the original abnormal object image data, by simulating image changes under various shooting conditions, to enrich the training data set, recorded as enhanced data; Step 103, data enhancement technology mainly includes image compression, image cropping, image rotation and color transformation; Step 2: Build a hybrid model of Swin Transformer and Mask R-CNN, referred to as an integrated model. Data augmentation is used to increase the number of samples and improve the generalization ability of the integrated model. Swin Transformer is responsible for extracting hierarchical feature representations from the input image, which are then fed into the Mask R-CNN part for object detection and segmentation mask generation. Step 3: Use the integrated model to train the enhanced data, recorded as integrated data; Step 301, the integrated model receives enhanced data; Step 302, using the Swin Transformer module to extract features from the enhanced data. Swin Transformer captures global and local features in the enhanced data through its hierarchical structure and self-attention mechanism, flattens the feature map, divides the feature map into several small blocks, and then performs a self-attention mechanism inside the small blocks. The small blocks are reorganized by moving the spatial position to form a new feature map, which is recorded as a reorganized feature; Step 303, mapping the extracted recombined features into a unified feature space, converting the features at different levels into a unified representation form, recorded as mapping features, so as to ensure that the feature information at different levels can be effectively integrated; Step 304: Input the extracted recombined features and mapped features into the Mask R-CNN model, perform mask processing in the self-attention mechanism, and form new modules in three different ways. The flattened feature map forms a Mask matrix. Mask R-CNN uses the features in the Mask matrix to identify objects in the image and generate a bounding box of the object. Step 305, output the category and bounding box position of the object; Step 4: deploy the integrated data to the edge server to detect abnormal objects in the real-time image on the transmission line; Step 5, determine whether there is new abnormal object image data; Step 6: If yes, upload the image data with abnormal objects in the edge server to the cloud server as new training samples for incremental learning, update the integrated model, and repeat steps 1-6 to achieve continuous optimization of the system; incremental learning specifically includes: using existing data to build an initial model, and updating the model after receiving new data to adapt to the new data features, thereby improving the recognition accuracy of abnormal objects; Step 7: If no, end.

[0021] Example 3 On the basis of Example 2, the power transmission line external damage detection method based on integrated target detection and incremental learning provided in this embodiment includes the following steps: Step 1, obtain abnormal object image data of the transmission line on the cloud server, and perform data enhancement on the abnormal object image data, which is recorded as enhanced data; the abnormal object image data includes abnormal shooting angles, images with a large amount of interference information, and abnormal data sets with severe target occlusion or overlap; Step 101, constructing a power line anomaly data set with 7 categories and a total of 926 samples, recorded as abnormal object image data; the 7 categories are specifically construction machinery, cranes, tower cranes, fire, smoke, obstructions and hanging objects; Step 102, using OpenCV-based data enhancement technology to enhance the data of the abnormal object image data, without changing the data features of the original abnormal object image data, by simulating image changes under various shooting conditions, to enrich the training data set, recorded as enhanced data; Step 103, data enhancement technology mainly includes image compression, image cropping, image rotation and color transformation; Step 2: Build a hybrid model of Swin Transformer and Mask R-CNN, referred to as an integrated model. Data augmentation is used to increase the number of samples and improve the generalization ability of the integrated model. Swin Transformer is responsible for extracting hierarchical feature representations from the input image, which are then fed into the Mask R-CNN part for object detection and segmentation mask generation. Step 201, the Swin Transformer model network framework diagram is as follows Figure 2 As shown in the figure, the model goes through four stages in the feature extraction process. The first stage includes 1 linear encoding module and 2 consecutive Swin Transformer modules; the second stage includes 1 region merging module and 2 consecutive Swin Transformer modules. The region merging module interacts by fusing adjacent regions to avoid information loss; the third stage includes 1 region merging module and 6 Swin Transformer modules; the last stage includes 1 region merging module and 2 consecutive Swin Transformer modules, which are used to expand the perception range and extract deep features; Step 202, the network framework structure diagram of the Mask R-CNN model is as follows Figure 3 As shown in the figure, first input the picture, generate the image feature map after processing by the deep residual network and feature pyramid network, then use the region proposal network RPN to generate candidate boxes on the feature map, and then align the feature map pixels with the original image through ROI Align, and finally use the FCN full convolutional network to realize target detection and semantic segmentation functions; Step 3: Use the integrated model to train the enhanced data, recorded as integrated data; Step 301, the integrated model receives enhanced data; Step 302, using the Swin Transformer module to extract features from the enhanced data. Swin Transformer captures global and local features in the enhanced data through its hierarchical structure and self-attention mechanism, flattens the feature map, divides the feature map into several small blocks, and then performs a self-attention mechanism inside the small blocks. The small blocks are reorganized by moving the spatial position to form a new feature map, which is recorded as a reorganized feature; Step 303, mapping the extracted recombined features into a unified feature space, converting the features at different levels into a unified representation form, recorded as mapping features, so as to ensure that the feature information at different levels can be effectively integrated; Step 304: Input the extracted recombined features and mapped features into the Mask R-CNN model, perform mask processing in the self-attention mechanism, and form new modules in three different ways. The flattened feature map forms a Mask matrix. Mask R-CNN uses the features in the Mask matrix to identify objects in the image and generate a bounding box of the object. Step 305, output the category and bounding box position of the object; Step 4: deploy the integrated data to the edge server to detect abnormal objects in the real-time image on the transmission line; Step 5, determine whether there is new abnormal object image data; Step 6: If yes, upload the image data with abnormal objects in the edge server to the cloud server as new training samples for incremental learning, update the integrated model, and repeat steps 1-6 to achieve continuous optimization of the system; incremental learning specifically includes: using existing data to build an initial model, and updating the model after receiving new data to adapt to the new data features, thereby improving the recognition accuracy of abnormal objects; Step 7: If no, end.

[0022] Example 4 On the basis of Example 3, the power transmission line external damage detection method based on integrated target detection and incremental learning provided in this embodiment obtains abnormal object image data of the power transmission line on the cloud server, and performs image detection through the power transmission line external damage detection method based on integrated target detection and incremental learning of the present invention, wherein the Mask R-CNN model is used to train the data, and after training, it is deployed on the edge server to detect abnormal objects in real-time images on the transmission line. The average accuracy of abnormal object recognition in the actual transmission line scenario is as high as 82%, and the detection results are shown in the figure. Figure 4 shown.

[0023] Example 5 On the basis of the above-mentioned embodiment 4, data enhancement is performed on the abnormal object image data, recorded as enhanced data, and image detection is performed by the power transmission line external damage detection method based on integrated target detection and incremental learning of the present invention, wherein the Mask R-CNN model is used to train the enhanced data, and after training, it is deployed on the edge server to detect abnormal objects in real-time images on the transmission line. The average accuracy of abnormal object recognition in the actual transmission line scenario is as high as 86%, and the detection results are shown in the figure. Figure 5 shown.

[0024] Example 6 By obtaining abnormal object image data of the transmission line on the cloud server and performing data enhancement on the abnormal object image data, recorded as enhanced data, image detection is performed through the transmission line external damage detection method based on integrated target detection and incremental learning of the present invention, and a hybrid model of Swin Transformer and Mask R-CNN is established, recorded as an integrated model, wherein the enhanced data is trained using the integrated model, and after training, it is deployed on the edge server to detect abnormal objects in real-time images on the transmission line. The average accuracy of abnormal object recognition in the actual transmission line scenario is as high as 89%, and the detection results are shown in the figure. Figure 6 shown.

[0025] It can be seen from the above-mentioned embodiments 1-6 that the image detection performed by the power transmission line external damage detection method based on integrated target detection and incremental learning of the present invention has a highly efficient detection effect.

[0026] Comparative Example 1 To verify the effectiveness of the present invention, a comparative experiment was conducted. The experimental results show that the integrated model of SwinTransformer and Mask R-CNN can well display the category label of the target object, and can well display the confidence score of the target object. Moreover, the confidence of the results measured by the integrated model is higher than the detection results when the Mask R-CNN model is used alone, reaching 89.45% in mAP.

Claims

1. A power transmission line external damage detection method based on integrated target detection and incremental learning, characterized in that: The following steps are involved: Step 1, obtaining abnormal object image data of the transmission line on the cloud server, and performing data enhancement on the abnormal object image data, which is recorded as enhanced data; Step 2: Establish a hybrid model of Swin Transformer and Mask R-CNN, recorded as an integrated model; Step 3, using the integrated model to train the enhanced data, recorded as integrated data; Step 4, deploying the integrated data to an edge server for detecting abnormal objects in real-time images on the transmission line; Step 5, determine whether there is new abnormal object image data; Step 6: If yes, upload the image data with abnormal objects in the edge server to the cloud server as new training samples for incremental learning, update the integrated model, and repeat steps 1-6 to achieve continuous optimization of the system; Step 7: If no, end.

2. The power transmission line external failure detection method based on integrated target detection and incremental learning according to claim 1 is characterized in that: Step 1 specifically includes the following steps: Step 101, constructing a power line anomaly data set with 7 categories and a total of 926 samples, recorded as abnormal object image data; Step 102, using OpenCV-based data enhancement technology to enhance the data of the abnormal object image data, without changing the data features of the original abnormal object image data, by simulating image changes under various shooting conditions, to enrich the training data set, recorded as enhanced data; Step 103, the data enhancement technology mainly includes image compression, image cropping, image rotation and color transformation.

3. The power transmission line external failure detection method based on integrated target detection and incremental learning according to claim 2 is characterized in that: The abnormal object image data includes abnormal shooting angles, images with a large amount of interference information, and abnormal data sets with severe target occlusion or overlap.

4. The power transmission line external failure detection method based on integrated target detection and incremental learning according to claim 2 is characterized in that: The seven categories are specifically construction machinery, cranes, tower cranes, fire, smoke, shelters and hanging objects.

5. The power transmission line external damage detection method based on integrated target detection and incremental learning according to claim 2 is characterized in that: The data enhancement is used to increase the number of samples and improve the generalization ability of the integrated model.

6. The power transmission line external damage detection method based on integrated target detection and incremental learning according to claim 1 is characterized in that: The Swin Transformer is responsible for extracting hierarchical feature representations from the input image, which are then fed into the Mask R-CNN part for object detection and segmentation mask generation.

7. The power transmission line external damage detection method based on integrated target detection and incremental learning according to claim 6 is characterized in that: Step 3 uses the integrated model to train the enhanced data, specifically including the following steps: Step 301, the integrated model receives enhanced data; Step 302, using the Swin Transformer module to extract features from the enhanced data. The Swin Transformer captures global and local features in the enhanced data through its hierarchical structure and self-attention mechanism, flattens the feature map, divides the feature map into several small blocks, and then performs a self-attention mechanism inside the small blocks, recombines the small blocks through a spatial position movement operation to form a new feature map, which is recorded as a recombined feature; Step 303, mapping the extracted recombined features into a unified feature space, converting the features at different levels into a unified representation form, recorded as mapping features, so as to ensure that the feature information at different levels can be effectively integrated; Step 304: Input the extracted recombined features and mapped features into the Mask R-CNN model, perform mask processing in the self-attention mechanism, and form new modules in three different ways. The flattened feature map forms a Mask matrix. Mask R-CNN uses the features in the Mask matrix to identify objects in the image and generate a bounding box of the object. Step 305: output the category of the object and the position of the bounding box.

8. The power transmission line external failure detection method based on integrated target detection and incremental learning according to claim 1, characterized in that: The incremental learning described in step 6 specifically includes: using existing data to build an initial model, and after receiving new data, updating the model to adapt to the new data features, thereby improving the recognition accuracy of abnormal objects.

Images