A sample knowledge driven power transmission line suspended matter special detection method and system based on improved YOLOv7

Abstract

The application discloses a sample knowledge driven power transmission line suspended matter special detection method and system based on improved YOLOv7, relates to the power line detection field, and comprises the following steps: collecting power transmission line images with suspended matters and labeling the suspended matters in the images to obtain an initial data set; performing data enhancement on the images in the initial data set to obtain a training data set; training a target detection model by using the training data set to obtain a final power transmission line suspended matter detection model, and detecting whether there is a suspended matter on the power transmission line; and the target detection model is obtained by integrating a SimAM module in a YOLOv7 target detection model. The application combines data enhancement and a YOLOv7 attention mechanism, thereby improving the accuracy of unmanned aerial vehicle suspended matter identification in a complex power transmission line environment and obtaining relatively higher identification precision.

Description

Technical Field

[0001] The present invention relates to the field of power line detection technology, and more particularly to a method and system for detecting suspended debris on power transmission lines based on sample knowledge driven by an improved YOLOv7. Background Art

[0002] With the development of the economy, electricity consumption in industry, agriculture and residents' lives has continued to increase, and the investment and scale of power grids have continued to expand. The stable and reliable operation of power grid lines has a direct impact on the safe production of power companies and people's normal production and life.

[0003] The inspection of transmission lines is an indispensable part of ensuring the safe operation of the power grid. To ensure the uninterrupted distribution of electricity, transmission lines need to be regularly and effectively monitored and maintained. Manual inspection and patrol work is difficult and inefficient. As the scale of the power grid expands, it is becoming increasingly difficult to meet the requirements of transmission line inspection. Suspended debris on transmission lines is a major cause of short circuits and grounding faults. Detecting suspended debris on transmission lines using drones and image detection methods can significantly reduce the workload of manual inspection. However, due to the slender physical structure of transmission lines, images captured by drones contain a variety of different content in the environment, including the transmission lines. Due to interference from background content and the influence of drone vibration, the accuracy of suspended debris detection on transmission lines is insufficient.

[0004] Therefore, how to improve the accuracy of UAV suspended object identification in complex transmission line environments is an urgent problem that technicians in this field need to solve. Summary of the Invention

[0005] In view of this, the present invention provides a special detection method and system for suspended debris on transmission lines based on sample knowledge driven by improved YOLOv7. The method improves YOLOv7 and improves the accuracy of suspended debris detection by enhancing training data and improving the attention mechanism.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] The present invention discloses a method for detecting suspended debris on power transmission lines based on improved YOLOv7 sample knowledge-driven method. The specific steps are as follows:

[0008] Collect images of power transmission lines with suspended debris and annotate the suspended debris in the images to obtain an initial dataset.

[0009] Performing data augmentation on the images in the initial data set to obtain a training data set;

[0010] The training data set is used to train a target detection model to obtain a final transmission line suspended object detection model to detect whether there are suspended objects on the transmission line; the target detection model inserts the SimAM module into the convolutional layer of feature extraction in the YOLOv7 target detection model, thereby integrating the SimAM module into the YOLOv7 backbone network.

[0011] Furthermore, the data enhancement includes: jitter simulation enhancement and environment enhancement. The jitter simulation enhancement is used to simulate random jitter during unmanned aerial photography; the environment enhancement is used to simulate image noise and brightness changes caused by environmental and weather changes.

[0012] Furthermore, the jitter simulation enhancement includes the following steps:

[0013] Convert the images in the initial data set into RGB images and perform HSV transformation;

[0014] Perform random flipping and random translation on each image after HSV transformation to obtain several flipped images and several translated images of each image;

[0015] The flipped image and the translated image of the same image are superimposed and synthesized to obtain a simulated shaking image, and the simulated shaking image is cut into a fixed size.

[0016] Furthermore, the environment enhancement includes the following steps:

[0017] Set the standard deviation and mean parameters of Gaussian noise to generate a Gaussian random number;

[0018] Generate a plurality of Gaussian random numbers, and form a noise matrix and add it to the image pixel matrix;

[0019] Re-limit or scale the pixel values ​​of the image to between [0 and 255] to obtain a Gaussian noise image;

[0020] Create a blank image and generate random weights;

[0021] The blank image and the Gaussian noise image are superimposed according to the random weight to obtain an environment enhanced image.

[0022] Furthermore, the target detection model includes a backbone network and a head network; the backbone network extracts features from the input image, and the head network fuses the extracted features and outputs a recognition result;

[0023] The SimAM module is inserted into the feature extraction convolution layer of the backbone network to improve the feature representation of the backbone network.

[0024] Furthermore, the backbone network includes: four layers of Conv modules, three layers of MPConv modules, and SPPCSPC modules connected in sequence; an ELAN1 module is further provided after the fourth layer of Conv modules and all three layers of MPConv modules;

[0025] The SimAM module is also arranged between the three layers of MPConv modules; the two layers of SimAM modules and the SPPCSPC modules respectively process the features of different sizes extracted by the three layers of MPConv modules and transmit them to the head network.

[0026] Furthermore, the head network includes an uplink branch, a downlink branch, and an identification branch;

[0027] The upstream branch includes two upstream feature fusion layers, which include a Conv module, an Upsample module, a Concat module, and an ELAN2 module connected in sequence; the upstream feature fusion layer performs convolution and upsampling on the input features through the Conv module and the Upsample module, and performs feature fusion on the features of corresponding sizes output by the SimAM module through the Concat module, and then uses the output of the ELAN2 module;

[0028] The downlink branch includes two downlink feature fusion layers, each of which includes an MPConv module, a Concat module, and an ELAN2 module connected in sequence; the downlink feature fusion layer convolves the input features through the MPConv module, and fuses the features of corresponding sizes output by the uplink branch through the Concat module, and then outputs them using the ELAN2 module;

[0029] The recognition branch includes a RepConv module, an ImpConv module, and a Detect module connected in sequence; the recognition branch performs convolution and recognition on the features of three different scales output by the downstream branch, and outputs a recognition result.

[0030] The present invention also discloses a special detection system for suspended debris on power transmission lines based on improved YOLOv7 sample knowledge drive, which is characterized by comprising:

[0031] Data acquisition module: collects images of transmission lines with suspended debris and labels the suspended debris in the images to obtain the initial data set;

[0032] Data enhancement module: performing data enhancement on the images in the initial data set to obtain a training data set;

[0033] Recognition model module: Use the training data set to train the target detection model to obtain the final transmission line suspended object detection model to detect whether there is suspended object on the transmission line; the target detection model is improved by integrating the SimAM module into the YOLOv7 target detection model.

[0034] Through the above technical solutions, it can be seen that compared with the prior art, the present invention discloses a special detection method and system for suspended objects on power transmission lines driven by sample knowledge based on an improved YOLOv7. By integrating the SimAM module into the YOLOv7 target detection model, the attention mechanism is used to increase the expressiveness, focus on important features and suppress unnecessary features, enhance the effect of feature extraction, and thus significantly improve the detection accuracy of suspended objects. Through data enhancement technology, it can effectively simulate the unstable factors and different environmental conditions in the unmanned aerial photography process, avoiding the overfitting of the model to specific training data; the SimAM module improves the feature representation ability of the backbone network, so that the robustness of the model in complex environments is enhanced; by introducing the SPPCSPC module, the ELAN module and the multi-level feature fusion strategy, the model can use features with multi-scale and rich semantic information; the design of the head network optimizes the feature flow through effective feature fusion, so that the multi-scale features can ensure the integrity and accuracy of the information during the final recognition. The present invention combines data enhancement and the YOLOv7 attention mechanism to improve the accuracy of drone suspended object recognition in complex power transmission line environments, and can obtain relatively higher recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0036] Figure 1 Schematic diagram of the overall process of an embodiment of the present invention.

[0037] Figure 2 FIG. 4 is a schematic diagram of jitter simulation enhancement according to an embodiment of the present invention.

[0038] Figure 3 Schematic diagram of environmental enhancement according to an embodiment of the present invention.

[0039] Figure 4 This is a schematic diagram of the overall structure of the improved YOLOv7 target detection model according to an embodiment of the present invention. DETAILED DESCRIPTION

[0040] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0041] The embodiment of the present invention discloses a method for detecting suspended debris on power transmission lines based on sample knowledge driven by improved YOLOv7. Figure 1 The specific steps are as follows:

[0042] Collect images of power transmission lines with suspended debris and annotate the suspended debris in the images to obtain an initial dataset.

[0043] Perform data augmentation on the images in the initial dataset to obtain a training dataset;

[0044] The training dataset is used to train the target detection model to obtain the final transmission line suspended debris detection model to detect whether there are suspended debris on the transmission line; the target detection model inserts the SimAM module into the convolutional layer of feature extraction in the YOLOv7 target detection model, thereby integrating the SimAM module into the YOLOv7 backbone network.

[0045] In a specific embodiment, data enhancement includes: jitter simulation enhancement and environment enhancement. Jitter simulation enhancement simulates random jitter during unmanned aerial photography; environment enhancement simulates image noise and brightness changes caused by environmental and weather changes.

[0046] In a specific embodiment, Figure 2 As shown, the jitter simulation enhancement includes the following steps:

[0047] The images in the initial dataset are converted to RGB images and then subjected to HSV transformation. The parameters in the HSV transformation are set as follows: hue = 0.1, sat = 0.7, val = 0.4, where hue represents hue, sat represents saturation, and val represents brightness.

[0048] Perform random flipping and random translation on each image after HSV transformation to obtain several flipped images and several translated images of each image;

[0049] The flipped image and the translated image of the same image are superimposed and synthesized to obtain a simulated shaking image, and the simulated shaking image is cropped to a fixed size.

[0050] In a specific embodiment, Figure 3 As shown, the environment enhancement includes the following steps:

[0051] Set the standard deviation and mean parameters of Gaussian noise to generate a Gaussian random number;

[0052] Generate multiple Gaussian random numbers and form a noise matrix and add it to the image pixel matrix;

[0053] Re-limit or scale the pixel values ​​of the image to between [0 and 255] to obtain a Gaussian noise image;

[0054] Create a blank image and generate random weights;

[0055] The environment-enhanced image is generated by superimposing a blank image and a Gaussian noise image using random weights. Using a weighted average method, different weights are assigned to the two images to control their influence. A larger weight for the blank image has a greater impact on the final result; conversely, a smaller weight for the Gaussian noise image has a greater impact on the final result.

[0056] In a specific embodiment, the object detection model includes a backbone network and a head network; the backbone network extracts features from the input image, and the head network fuses the extracted features and outputs a recognition result;

[0057] The SimAM module is inserted into the feature extraction convolution layer of the backbone network to improve the feature representation of the backbone network.

[0058] Furthermore, the backbone network includes: a four-layer Conv module, a three-layer MPConv module, and an SPPCSPC module connected in sequence; an ELAN1 module is also provided after the fourth-layer Conv module and all three-layer MPConv modules;

[0059] A SimAM module is also set between the three-layer MPConv modules; the two-layer SimAM module and SPPCSPC module respectively process the features of different sizes extracted by the three-layer MPConv modules and transmit them to the head network.

[0060] Furthermore, the head network includes an ascending branch, a descending branch, and a recognition branch;

[0061] The upstream branch includes two upstream feature fusion layers, which include the Conv module, Upsample module, Concat module, and ELAN2 module connected in sequence. The upstream feature fusion layer convolves and upsamples the input features through the Conv module and the Upsample module, and fuses the features of corresponding sizes output by the SimAM module through the Concat module, and then outputs them through the ELAN2 module.

[0062] The downlink branch includes two downlink feature fusion layers, which include the MPConv module, the Concat module, and the ELAN2 module connected in sequence. The downlink feature fusion layer convolves the input features through the MPConv module, and fuses the features of corresponding sizes output by the uplink branch through the Concat module, and then outputs them through the ELAN2 module.

[0063] The recognition branch includes the RepConv module, ImpConv module and Detect module connected in sequence; the recognition branch convolves and recognizes the features of three different scales output by the downstream branch and outputs the recognition result.

[0064] Among them, Figure 4 As shown in the figure, the SimAM module is a self-attention mechanism module. Its core is to simulate the attention mechanism by minimizing the attention weight of the input feature map, thereby enhancing the feature extraction capability of YOLOv7. SimAM is mainly used in tasks such as image classification, target detection, and semantic segmentation. Compared with other traditional attention mechanisms (such as the self-attention mechanism), the calculation is simpler, avoiding complex matrix multiplication and reducing computational complexity. In addition, SimAM does not rely on the traditional query-key-value structure, but directly uses similarity measurement for feature weighting, which makes it easier to implement and understand. Since only a small number of parameters are introduced in SimAM, its computational complexity is limited, so it does not significantly increase the computational overhead during training. According to experimental needs, the hyperparameters in SimAM are adjusted to ensure that the attention mechanism can work effectively in different layers.

[0065] The Conv module is a convolution module, where k represents the convolution sum size and s represents the stride; the MPConv module is a multi-path convolution module, which enables the model to capture richer features, covering different scales and levels of abstraction; the ELAN1 module and the ELAN2 module are efficient aggregation networks, which can better capture feature information of different scales and levels by aggregating features between different layers; the SPPCSPC module is a network structure that combines spatial pyramid pooling (SPP) and convolution operations, aiming to improve the feature extraction capabilities in target detection and image classification tasks; the Upsample module is an upsampling module; the Concat module is a splicing operation module for multi-scale feature map feature fusion; the RepConv module is a repeated convolution module, which mainly reduces the computational complexity of the model by optimizing the convolution operation while maintaining or improving its accuracy; the ImpConv module is a depth-separable convolution module, which decomposes the standard convolution into depth-wise convolution and point-by-point convolution to reduce the computational complexity; the Detect module is a prediction module used to output the final recognition result.

[0066] In a specific embodiment, the model training process is:

[0067] First, different data augmentation methods were used on the initial dataset of suspended debris on transmission lines. The original YOLOv7 was used for training and the results were compared with those on the original data. Then, the improved YOLOv7 was trained on the improved dataset and the results were compared with those of the unimproved YOLOv7 network trained on the improved dataset.

[0068] The mAP metric was used as the evaluation standard. mAP (mean Average Precision) is a widely used evaluation metric in object detection. It is the average of the mean precision (AP) across all categories. The AP metric is used to evaluate a model's detection performance for a single category of objects. It is calculated based on the area under the area calculated based on the IoU values ​​between the prediction results at different confidence levels and the true annotations. The mAP metric represents the model's detection accuracy for multiple categories of objects and is the average of the AP values ​​across all categories. mAP_0.5 refers to the model's mean average precision (mAP) when the IoU threshold is 0.5. In object detection tasks, AP values ​​are typically calculated based on different IoU thresholds, and the mAP metric is obtained by averaging the AP values ​​at different IoU thresholds. mAP_0.5 is a common calculation method, representing the model's mean average precision when the IoU threshold is 0.5. It is an important metric for measuring model detection accuracy. The final training results are shown in Table 1.

[0069] Table 1 Training results

[0070] Data processing mAP_0.5 Improvement ratio No treatment 0.911 / Random jitter 0.917 0.659％ Noise and brightness 0.920 0.989％ overall 0.927 1.75％ Model mAP_0.5 Improvement ratio YOLOv7 0.927 / YOLOv7+SimAM 0.934 0.755％

[0071] The embodiment of the present invention further discloses a dedicated detection system for suspended debris on power transmission lines based on improved YOLOv7 sample knowledge drive, which is characterized by comprising:

[0072] Data acquisition module: collects images of transmission lines with suspended debris and labels the suspended debris in the images to obtain the initial data set;

[0073] Data enhancement module: performs data enhancement on the images in the initial data set to obtain the training data set;

[0074] Recognition model module: Use the training dataset to train the target detection model to obtain the final transmission line suspended debris detection model to detect whether there is suspended debris on the transmission line; the target detection model is improved by integrating the SimAM module into the YOLOv7 target detection model.

[0075] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0076] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A special detection method for suspended debris on power transmission lines based on improved YOLOv7 sample knowledge driven, characterized by: The specific steps are as follows: Collect images of power transmission lines with suspended debris and annotate the suspended debris in the images to obtain an initial dataset. Performing data augmentation on the images in the initial data set to obtain a training data set; Using the training data set to train the target detection model, a final transmission line suspended object detection model is obtained to detect whether there is suspended object on the transmission line; The target detection model inserts the SimAM module into the convolutional layer of feature extraction in the YOLOv7 target detection model, thereby integrating the SimAM module into the YOLOv7 backbone network; The target detection model includes a backbone network and a head network; the backbone network extracts features from the input image, and the head network fuses the extracted features and outputs the recognition result; Inserting the SimAM module into the feature extraction convolution layer of the backbone network to improve the feature representation of the backbone network; The backbone network includes: four layers of Conv modules, three layers of MPConv modules and SPPCSPC modules connected in sequence; an ELAN1 module is further provided after the fourth layer of Conv modules and all three layers of MPConv modules; The SimAM module is further provided between the three layers of MPConv modules; the two layers of SimAM modules and the SPPCSPC modules respectively process the features of different sizes extracted by the three layers of MPConv modules and transmit them to the head network; The head network includes an uplink branch, a downlink branch and an identification branch; The upstream branch includes two upstream feature fusion layers, which include a Conv module, an Upsample module, a Concat module, and an ELAN2 module connected in sequence; the upstream feature fusion layer performs convolution and upsampling on the input features through the Conv module and the Upsample module, and performs feature fusion on the features of corresponding sizes output by the SimAM module through the Concat module, and then uses the output of the ELAN2 module; The downlink branch includes two downlink feature fusion layers, each of which includes an MPConv module, a Concat module, and an ELAN2 module connected in sequence; the downlink feature fusion layer convolves the input features through the MPConv module, and fuses the features of corresponding sizes output by the uplink branch through the Concat module, and then outputs them using the ELAN2 module; The recognition branch includes a RepConv module, an ImpConv module, and a Detect module connected in sequence; the recognition branch performs convolution and recognition on the features of three different scales output by the downstream branch, and outputs a recognition result.

2. The method for detecting suspended debris on power transmission lines based on sample knowledge driven by improved YOLOv7 according to claim 1 is characterized in that: The data enhancement includes: jitter simulation enhancement and environment enhancement. The jitter simulation enhancement is used to simulate random jitter in the unmanned aerial photography process; the environment enhancement is used to simulate image noise and brightness changes caused by environmental and weather changes.

3. The method for detecting suspended debris on power transmission lines based on sample knowledge driven by improved YOLOv7 according to claim 2 is characterized in that: The jitter simulation enhancement comprises the following steps: Convert the images in the initial data set into RGB images and perform HSV transformation; Perform random flipping and random translation on each image after HSV transformation to obtain several flipped images and several translated images of each image; The flipped image and the translated image of the same image are superimposed and synthesized to obtain a simulated shaking image, and the simulated shaking image is cut into a fixed size.

4. The method for detecting suspended debris on power transmission lines based on sample knowledge driven by improved YOLOv7 according to claim 2 is characterized in that: The environmental enhancement comprises the following steps: Set the standard deviation and mean parameters of Gaussian noise to generate a Gaussian random number; Generate a plurality of Gaussian random numbers, and form a noise matrix and add it to the image pixel matrix; Re-limit or scale the pixel values ​​of the image to [0~255] to obtain a Gaussian noise image; Create a blank image and generate random weights; The blank image and the Gaussian noise image are superimposed according to the random weight to obtain an environment enhanced image.

5. A transmission line suspended debris detection system based on improved YOLOv7 sample knowledge driven, applying the transmission line suspended debris detection method based on improved YOLOv7 sample knowledge driven according to any one of claims 1 to 4, characterized in that: include: Data acquisition module: collects images of transmission lines with suspended debris and labels the suspended debris in the images to obtain the initial data set; Data enhancement module: performing data enhancement on the images in the initial data set to obtain a training data set; Identification model module: uses the training data set to train the target detection model to obtain the final transmission line suspended object detection model to detect whether there is suspended object on the transmission line; The target detection model is improved by integrating the SimAM module into the YOLOv7 target detection model.

Images