Special detection method and system for power transmission line suspended matters driven by sample knowledge based on improved YOLOv7

Abstract

The invention discloses an improved YOLOv7 sample knowledge driven special detection method and system for power transmission line suspended solids, and relates to the field of power line detection.The method comprises the steps that a power transmission line image with the suspended solids is collected, the suspended solids in the image are marked, and an initial data set is obtained; performing data enhancement on images in the initial data set to obtain a training data set; training a target detection model by using the training data set to obtain a final power transmission line suspended matter detection model, and detecting whether suspended matter exists on the power transmission line; the target detection model is obtained by integrating a SimAM module in a YOLOv7 target detection model. According to the method, data enhancement and a YOLOv7 attention mechanism are combined, so that the accuracy of unmanned aerial vehicle suspended matter recognition in a complex power transmission line environment is improved, and relatively higher recognition precision can be obtained.

Description

Technical Field

[0001] The present invention relates to the technical field of power line detection, and more specifically to a method and system for detecting suspended objects in power transmission lines driven by sample knowledge based on improved YOLOv7. Background Art

[0002] With the development of economy, the electricity consumption in industry, agriculture and residents' life is increasing, the investment and scale of power grid are expanding, and the stable and reliable operation of power grid lines directly affects the safe production of power enterprises and the normal production and life of people.

[0003] The detection of transmission lines is an indispensable part of ensuring the safe operation of the power grid. In order to ensure the uninterrupted distribution of electricity, the transmission lines need to be regularly and effectively monitored and maintained. Manual detection and inspection are difficult and inefficient. As the scale of the power grid expands, it is increasingly difficult to meet the requirements for transmission line detection. Suspended matter on transmission lines is an important cause of short circuits and grounding faults in transmission lines. Detecting suspended matter on transmission lines through drones and image detection can significantly reduce the workload of manual detection. However, due to the slender physical structure of the transmission line, the images taken by the drone contain a variety of different contents in the environment including the transmission line. Due to the interference of background content and the vibration of the drone, the accuracy of suspended matter detection on the transmission line is insufficient.

[0004] Therefore, how to improve the accuracy of UAV suspended object identification in a complex power transmission line environment is a problem that technical personnel in this field need to solve urgently. Summary of the invention

[0005] In view of this, the present invention provides a special detection method and system for suspended objects in power transmission lines driven by sample knowledge based on improved YOLOv7, which is improved on the basis of YOLOv7 and improves the accuracy of suspended object detection by enhancing training data and improving the attention mechanism.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] The present invention discloses a special detection method for suspended objects in power transmission lines based on sample knowledge driven by improved YOLOv7, and the specific steps are as follows:

[0008] Collecting transmission line images with suspended objects and marking the suspended objects in the images to obtain an initial data set;

[0009] Performing data enhancement on the images in the initial data set to obtain a training data set;

[0010] The training data set is used to train the target detection model to obtain the final transmission line suspended object detection model to detect whether there are suspended objects on the transmission line; the target detection model inserts the SimAM module into the convolutional layer of feature extraction in the YOLOv7 target detection model, thereby integrating the SimAM module into the YOLOv7 backbone network.

[0011] Furthermore, the data enhancement includes: jitter simulation enhancement and environment enhancement. Through the jitter simulation enhancement, random jitter in the unmanned aerial photography process is simulated; through the environment enhancement, image noise and brightness changes caused by environmental and weather changes are simulated.

[0012] Furthermore, the jitter simulation enhancement comprises the following steps:

[0013] Convert the images in the initial data set into RGB images and perform HSV transformation;

[0014] Randomly flip and randomly translate each image after HSV transformation to obtain several flipped images and several translated images of each image;

[0015] The flipped image and the translated image of the same image are superimposed and synthesized to obtain a simulated shaking image, and the simulated shaking image is cut into a fixed size.

[0016] Furthermore, the environment enhancement comprises the following steps:

[0017] Set the standard deviation and mean parameters of Gaussian noise to generate a Gaussian random number;

[0018] Generate a plurality of Gaussian random numbers and form a noise matrix to be added to the image pixel matrix;

[0019] Re-limit or scale the pixel values ​​of the image to between [0 and 255] to obtain a Gaussian noise image;

[0020] Create a blank image and generate random weights;

[0021] The blank image and the Gaussian noise image are superimposed according to the random weight to obtain an environment enhanced image.

[0022] Furthermore, the target detection model includes a backbone network and a head network; the backbone network extracts features from the input image, and the head network fuses the extracted features and outputs a recognition result;

[0023] The SimAM module is inserted into the feature extraction convolution layer of the backbone network to improve the feature representation of the backbone network.

[0024] Furthermore, the backbone network includes: four layers of Conv modules, three layers of MPConv modules and SPPCSPC modules connected in sequence; an ELAN1 module is also arranged after the fourth layer of Conv modules and all three layers of MPConv modules;

[0025] The SimAM module is also arranged between the three layers of MPConv modules; the two layers of SimAM modules and the SPPCSPC modules respectively process the features of different sizes extracted by the three layers of MPConv modules and transmit them to the head network.

[0026] Further, the head network includes an uplink branch, a downlink branch and an identification branch;

[0027] The upstream branch includes two layers of upstream feature fusion layers, and the upstream feature fusion layer includes a Conv module, an Upsample module, a Concat module, and an ELAN2 module connected in sequence; the upstream feature fusion layer performs convolution and upsampling on the input features through the Conv module and the Upsample module, and performs feature fusion on the features of corresponding sizes output by the SimAM module through the Concat module, and then uses the ELAN2 module output;

[0028] The downlink branch includes two downlink feature fusion layers, and the downlink feature fusion layer includes an MPConv module, a Concat module, and an ELAN2 module connected in sequence; the downlink feature fusion layer convolves the input features through the MPConv module, and performs feature fusion on the features of the corresponding size output by the uplink branch through the Concat module, and then outputs them using the ELAN2 module;

[0029] The recognition branch includes a RepConv module, an ImpConv module and a Detect module connected in sequence; the recognition branch performs convolution and recognition on the features of three different scales output by the downstream branch, and outputs a recognition result.

[0030] The present invention also discloses a transmission line suspended object detection system driven by sample knowledge based on improved YOLOv7, which is characterized by comprising:

[0031] Data acquisition module: collects images of power transmission lines with suspended objects and marks the suspended objects in the images to obtain an initial data set;

[0032] Data enhancement module: performing data enhancement on the images in the initial data set to obtain a training data set;

[0033] Identification model module: Use the training data set to train the target detection model to obtain the final transmission line suspended object detection model to detect whether there are suspended objects on the transmission line; the target detection model is improved by integrating the SimAM module in the YOLOv7 target detection model.

[0034] Through the above technical solutions, it can be known that compared with the prior art, the present invention discloses a special detection method and system for suspended objects in power transmission lines driven by sample knowledge based on improved YOLOv7. By integrating the SimAM module into the YOLOv7 target detection model, the attention mechanism is used to increase the expressiveness, focus on important features and suppress unnecessary features, and enhance the effect of feature extraction, thereby significantly improving the detection accuracy of suspended objects. Through data enhancement technology, the unstable factors and different environmental conditions in the process of unmanned aerial photography can be effectively simulated, avoiding the overfitting of the model to specific training data; the SimAM module improves the feature representation ability of the backbone network, so that the robustness of the model in complex environments can be enhanced; by introducing the SPPCSPC module, the ELAN module and the multi-level feature fusion strategy, the model can continue to use the features of multi-scale and rich semantic information; the design of the head network optimizes the feature flow through effective feature fusion, so that the multi-scale features can ensure the integrity and accuracy of the information in the final recognition. The present invention combines data enhancement and the YOLOv7 attention mechanism to improve the accuracy of the identification of suspended objects in drones in complex power transmission line environments, and can obtain relatively higher recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0036] Figure 1 It is a schematic diagram of the overall process of an embodiment of the present invention.

[0037] Figure 2 It is a schematic diagram of jitter simulation enhancement according to an embodiment of the present invention.

[0038] Figure 3 Schematic diagram of environment enhancement according to an embodiment of the present invention.

[0039] Figure 4 The figure is a schematic diagram of the overall structure of the improved YOLOv7 target detection model according to an embodiment of the present invention. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] The embodiment of the present invention discloses a special detection method for suspended objects in power transmission lines based on sample knowledge driven by improved YOLOv7, such as Figure 1 As shown, the specific steps are as follows:

[0042] Collecting transmission line images with suspended objects and marking the suspended objects in the images to obtain an initial data set;

[0043] Perform data augmentation on the images in the initial data set to obtain a training data set;

[0044] The training data set is used to train the target detection model to obtain the final transmission line suspended object detection model to detect whether there are suspended objects on the transmission line; the target detection model inserts the SimAM module into the convolutional layer of feature extraction in the YOLOv7 target detection model, thereby integrating the SimAM module into the YOLOv7 backbone network.

[0045] In a specific embodiment, data enhancement includes: jitter simulation enhancement and environment enhancement. Through jitter simulation enhancement, random jitter in the unmanned aerial photography process is simulated; through environment enhancement, image noise and brightness changes caused by environmental and weather changes are simulated.

[0046] In a specific embodiment, Figure 2 As shown, the jitter simulation enhancement includes the following steps:

[0047] The images in the initial data set are converted into RGB images and subjected to HSV transformation. The parameters in the HSV transformation are set as follows: hue = 0.1, sat = 0.7, val = 0.4, where hue represents hue, sat represents saturation, and val represents brightness.

[0048] Randomly flip and randomly translate each image after HSV transformation to obtain several flipped images and several translated images of each image;

[0049] The flipped image and the translated image of the same image are superimposed and synthesized to obtain a simulated shaking image, and the simulated shaking image is cut into a fixed size.

[0050] In a specific embodiment, Figure 3 As shown, the environment enhancement includes the following steps:

[0051] Set the standard deviation and mean parameters of Gaussian noise to generate a Gaussian random number;

[0052] Generate multiple Gaussian random numbers and form a noise matrix and add it to the image pixel matrix;

[0053] Re-limit or scale the pixel values ​​of the image to between [0 and 255] to obtain a Gaussian noise image;

[0054] Create a blank image and generate random weights;

[0055] The environment enhanced image is obtained by superimposing the blank image and the Gaussian noise image according to random weights. Different weights are assigned to the two images to control their influence using the weighted average method. If the blank image weight value is larger, it will have a greater impact on the final result; conversely, the Gaussian noise image will have a greater impact on the final result.

[0056] In a specific embodiment, the target detection model includes a backbone network and a head network; the backbone network extracts features from an input image, and the head network fuses the extracted features and outputs a recognition result;

[0057] The SimAM module is inserted into the feature extraction convolution layer of the backbone network to improve the feature representation of the backbone network.

[0058] Further, the backbone network includes: a four-layer Conv module, a three-layer MPConv module, and an SPPCSPC module connected in sequence; an ELAN1 module is also arranged after the fourth-layer Conv module and all three-layer MPConv modules;

[0059] A SimAM module is also set between the three-layer MPConv modules; the two-layer SimAM module and the SPPCSPC module respectively process the features of different sizes extracted by the three-layer MPConv modules and transmit them to the head network.

[0060] Furthermore, the head network includes an ascending branch, a descending branch, and a recognition branch;

[0061] The upstream branch includes two upstream feature fusion layers, which include a Conv module, an Upsample module, a Concat module, and an ELAN2 module connected in sequence. The upstream feature fusion layer performs convolution and upsampling on the input features through the Conv module and the Upsample module, and performs feature fusion on the features of the corresponding size output by the SimAM module through the Concat module, and then outputs them using the ELAN2 module.

[0062] The downlink branch includes two downlink feature fusion layers, which include MPConv module, Concat module, and ELAN2 module connected in sequence. The downlink feature fusion layer convolves the input features through the MPConv module, and fuses the features of the corresponding size output by the uplink branch through the Concat module, and then outputs them using the ELAN2 module.

[0063] The recognition branch includes a RepConv module, an ImpConv module, and a Detect module connected in sequence; the recognition branch performs convolution and recognition on the features of three different scales output by the downstream branch, and outputs the recognition result.

[0064] Among them, Figure 4 As shown in the figure, the SimAM module is a self-attention mechanism module. Its core is to simulate the attention mechanism by minimizing the attention weight of the input feature map to enhance the feature extraction capability of YOLOv7. SimAM is mainly used in tasks such as image classification, target detection, and semantic segmentation. Compared with other traditional attention mechanisms (such as self-attention mechanisms), the calculation is more concise, avoiding complex matrix multiplication and reducing the computational complexity. In addition, SimAM does not rely on the traditional query-key-value structure, but directly uses similarity metrics for feature weighting, which makes it easier to implement and understand. Since only a small number of parameters are introduced in SimAM, its computational complexity increases only slightly, so it does not significantly increase the computational overhead during training. According to experimental needs, adjust the hyperparameters in SimAM to ensure that the attention mechanism can work effectively in different layers.

[0065] The Conv module is a convolution module, in which k represents the convolution and size, and s represents the step size; the MPConv module is a multi-path convolution module, which enables the model to capture richer features, covering different scales and abstract levels; the ELAN1 module and the ELAN2 module are efficient aggregation networks, which can better capture feature information of different scales and levels by aggregating features between different layers; the SPPCSPC module is a network structure that combines spatial pyramid pooling (SPP) and convolution operations, aiming to improve the feature extraction capabilities in target detection and image classification tasks; the Upsample module is an upsampling module; the Concat module is a splicing operation module for multi-scale feature map feature fusion; the RepConv module is a repeated convolution module, which mainly reduces the computational complexity of the model by optimizing the convolution operation while maintaining or improving its accuracy; the ImpConv module is a depth-separable convolution module, which decomposes the standard convolution into depth-wise convolution and point-by-point convolution to reduce the computational complexity; the Detect module is a prediction module, which is used to output the final recognition result.

[0066] In a specific embodiment, the model training process is:

[0067] Firstly, different data augmentation methods were used on the initial dataset of suspended matter on transmission lines. The original YOLOv7 was used for training and the results were compared with those on the original data. Then, the improved YOLOv7 was used for training on the improved dataset and the results were compared with those of the unimproved YOLOv7 network trained on the improved dataset.

[0068] The mAP indicator is used as the evaluation standard. mAP (meanAverage Precision) is an evaluation indicator widely used in target detection. It is the average of the average precision (AP) of all categories. The AP indicator is used to evaluate the detection effect of the model on a single category of targets. It is the area under the area calculated based on the IOU value between the prediction results of different confidence levels and the true annotation. The mAP indicator represents the detection accuracy of the model for multiple categories of targets and is the average of the AP values ​​of all categories. Among them, mAP_0.5 refers to the average precision (mAP) indicator of the model when the IOU threshold is equal to 0.5. In the target detection task, the AP value is usually calculated according to different IOU thresholds, and the AP values ​​under different IOU thresholds are averaged to obtain the mAP indicator. mAP_0.5 is one of the common calculation methods, which represents the average precision of the model when the IOU threshold is equal to 0.5. It is one of the important indicators for measuring the detection accuracy of the model. The final training results are shown in Table 1.

[0069] Table 1 Training results

[0070] Data processing mAP_0.5 Improvement ratio No treatment 0.911 / Random Jitter 0.917 0.659％ Noise and brightness 0.920 0.989％ overall 0.927 1.75％ Model mAP_0.5 Improvement ratio YOLOv7 0.927 / YOLOv7+SimAM 0.934 0.755％

[0071] The embodiment of the present invention further discloses a transmission line suspended object detection system driven by sample knowledge based on improved YOLOv7, which is characterized by comprising:

[0072] Data acquisition module: collects images of power transmission lines with suspended objects and marks the suspended objects in the images to obtain an initial data set;

[0073] Data enhancement module: perform data enhancement on the images in the initial data set to obtain the training data set;

[0074] Recognition model module: The target detection model is trained using the training data set to obtain the final transmission line suspended object detection model to detect whether there are suspended objects on the transmission line; the target detection model is improved by integrating the SimAM module into the YOLOv7 target detection model.

[0075] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0076] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A special detection method for suspended objects in power transmission lines based on improved YOLOv7 sample knowledge driven, characterized in that: The specific steps are as follows: Collecting transmission line images with suspended objects and marking the suspended objects in the images to obtain an initial data set; Performing data enhancement on the images in the initial data set to obtain a training data set; Using the training data set to train the target detection model, a final transmission line suspended object detection model is obtained to detect whether there is suspended object on the transmission line; The target detection model inserts the SimAM module into the convolutional layer of feature extraction in the YOLOv7 target detection model, thereby integrating the SimAM module into the YOLOv7 backbone network.

2. According to claim 1, a method for detecting suspended matter in power transmission lines based on sample knowledge driven by improved YOLOv7, characterized in that: The data enhancement includes: jitter simulation enhancement and environment enhancement. The jitter simulation enhancement is used to simulate random jitter in the unmanned aerial photography process; the environment enhancement is used to simulate image noise and brightness changes caused by environmental and weather changes.

3. According to claim 2, a method for detecting suspended matter in power transmission lines based on sample knowledge driven by improved YOLOv7 is characterized in that: The jitter simulation enhancement comprises the following steps: Convert the images in the initial data set into RGB images and perform HSV transformation; Randomly flip and randomly translate each image after HSV transformation to obtain several flipped images and several translated images of each image; The flipped image and the translated image of the same image are superimposed and synthesized to obtain a simulated shaking image, and the simulated shaking image is cut into a fixed size.

4. According to claim 2, a method for detecting suspended matter in power transmission lines based on sample knowledge driven by improved YOLOv7 is characterized in that: The environment enhancement comprises the following steps: Set the standard deviation and mean parameters of Gaussian noise to generate a Gaussian random number; Generate a plurality of Gaussian random numbers and form a noise matrix to be added to the image pixel matrix; Re-limit or scale the pixel values ​​of the image to between [0 and 255] to obtain a Gaussian noise image; Create a blank image and generate random weights; The blank image and the Gaussian noise image are superimposed according to the random weight to obtain an environment enhanced image.

5. According to claim 1, a method for detecting suspended matter in power transmission lines driven by sample knowledge based on improved YOLOv7, characterized in that: The target detection model includes a backbone network and a head network; the backbone network extracts features from the input image, and the head network fuses the extracted features and outputs a recognition result; The SimAM module is inserted into the feature extraction convolution layer of the backbone network to improve the feature representation of the backbone network.

6. According to claim 5, a method for detecting suspended matter in power transmission lines based on sample knowledge driven by improved YOLOv7 is characterized in that: The backbone network includes: four layers of Conv modules, three layers of MPConv modules and SPPCSPC modules connected in sequence; an ELAN1 module is also arranged after the fourth layer of Conv modules and all three layers of MPConv modules; The SimAM module is also arranged between the three layers of MPConv modules; the two layers of SimAM modules and the SPPCSPC modules respectively process the features of different sizes extracted by the three layers of MPConv modules and transmit them to the head network.

7. According to claim 6, a method for detecting suspended matter in power transmission lines based on sample knowledge driven by improved YOLOv7, characterized in that: The head network includes an uplink branch, a downlink branch and an identification branch; The upstream branch includes two layers of upstream feature fusion layers, and the upstream feature fusion layer includes a Conv module, an Upsample module, a Concat module, and an ELAN2 module connected in sequence; the upstream feature fusion layer performs convolution and upsampling on the input features through the Conv module and the Upsample module, and performs feature fusion on the features of corresponding sizes output by the SimAM module through the Concat module, and then uses the ELAN2 module output; The downlink branch includes two downlink feature fusion layers, and the downlink feature fusion layer includes an MPConv module, a Concat module, and an ELAN2 module connected in sequence; the downlink feature fusion layer convolves the input features through the MPConv module, and performs feature fusion on the features of the corresponding size output by the uplink branch through the Concat module, and then outputs them using the ELAN2 module; The recognition branch includes a RepConv module, an ImpConv module and a Detect module connected in sequence; the recognition branch performs convolution and recognition on the features of three different scales output by the downstream branch, and outputs a recognition result.

8. A transmission line suspended object detection system based on improved YOLOv7 sample knowledge driven, using a transmission line suspended object detection method based on improved YOLOv7 sample knowledge driven according to any one of claims 1 to 7, characterized in that: include: Data acquisition module: collects images of power transmission lines with suspended objects and marks the suspended objects in the images to obtain the initial data set; Data enhancement module: performing data enhancement on the images in the initial data set to obtain a training data set; Identification model module: using the training data set to train the target detection model, to obtain the final transmission line suspended object detection model, to detect whether there is suspended object on the transmission line; The target detection model is improved by integrating the SimAM module into the YOLOv7 target detection model.

Images