Small-scale target detection method and device for power line inspection, medium and terminal

Abstract

The invention provides a small-scale target detection method and device for power line inspection, a medium and a terminal. The method comprises the steps of obtaining a power transmission line data set and performing small-scale target labeling; the reference Faster-RCNN algorithm model is improved, and an improved Faster-RCNN algorithm model is obtained; and training and evaluating the improved Faster-RCNN algorithm model by using the marked data set, so as to perform small-scale target detection on the power transmission line image of the inspection area acquired in real time by using the unmanned aerial vehicle end. The improved Faster-RCNN algorithm model adopts an AdvResNet network combined with a feature pyramid structure to improve the feature extraction efficiency, a guide feature refining module improves the detection capability of a small-scale target, and an ROI alignment network improves the positioning precision of a bounding box, so that the reliability and safety of power transmission line inspection are improved.

Description

Technical Field

[0001] The present application relates to the technical field of power transmission line inspection, and in particular to a small-scale target detection method, device, medium and terminal for power line inspection. Background Art

[0002] Transmission lines are an important part of the power system. With the development of the economy, a large number of transmission lines have been laid to transmit electric energy. The safety of transmission lines plays an important role in the stable operation of the power grid. At present, the detection method of transmission lines mainly adopts the method of manual inspection. However, this method requires a large number of transmission line operation and maintenance personnel, and the inspection efficiency is low. Transmission lines, especially high-voltage overhead transmission lines, are high above the ground, and maintenance personnel need to climb the power tower to complete the inspection of key positions of the transmission lines.

[0003] With the development of communication networks, artificial intelligence and other technologies, the method of power inspection based on drones has been widely used. Drones can inspect transmission lines in complex environments and obtain high-resolution images of key components such as transmission lines and towers through high-definition cameras. Since transmission lines have the characteristics of many points, wide areas and long lines, using drones instead of manual inspections can effectively improve the efficiency of inspections and reduce the error rate of inspections. Transmission line drone inspections include transmission line channels, towers, hardware and foreign objects in transmission lines. In intelligent inspections, the identification of small-scale targets on transmission lines, such as nut shedding, missing pins, uneven gaskets, and small hardware rust, is of great significance. However, since small-scale targets account for a small proportion of the image, the pixel size they occupy is small, the resolution is low, and the information in the corresponding field is small, when using existing target detection algorithms for detection, they are easily disturbed by the surrounding environment and miss detection and misdetection occur, resulting in the problem of low accuracy of small-scale target detection in current drone inspections. Summary of the invention

[0004] In view of the shortcomings of the prior art described above, the purpose of the present application is to provide a small-scale target detection method, device, medium and terminal for power line inspection, which is used to solve the problem of low accuracy of small-scale target detection in drone inspection in the prior art.

[0005] To achieve the above-mentioned purpose and other related purposes, the first aspect of the present application provides a small-scale target detection method for power line inspection, including: obtaining a transmission line data set and annotating small-scale targets; taking the benchmark Faster-RCNN algorithm model as the benchmark model, replacing the VGG16 network in the benchmark model with the AdvResNet network combined with the feature pyramid structure, introducing a guided feature refinement module in the region proposal network, and replacing the region of interest pooling network in the benchmark model with the ROI alignment network to obtain an improved Faster-RCNN algorithm model; using the annotated data set, training and evaluating the improved Faster-RCNN algorithm model for small-scale target detection of transmission line images of the inspection area collected in real time by a drone.

[0006] In some embodiments of the first aspect of the present application, the method for obtaining a transmission line data set includes: collecting transmission line images in a variety of real-life scenarios; performing color dithering and random horizontal flipping on the collected transmission line images to form a preliminary data set; and segmenting and copying and pasting the preliminary data set to obtain a transmission line data set.

[0007] In some embodiments of the first aspect of the present application, the AdvResNet network combined with the feature pyramid structure is used to extract features of the transmission line images in the labeled data set to obtain multi-scale feature images; wherein, ResNet50 is used as the benchmark network, and the last four stages in the benchmark network are improved respectively to obtain the AdvResNet network; each improved stage consists of an initial module, multiple intermediate modules and a final module; each module adopts a newly constructed bottleneck structure.

[0008] In some embodiments of the first aspect of the present application, the structure of the initial module includes: a first 1×1 convolutional layer, a first batch normalization layer, a first ReLU layer, a first 3×3 convolutional layer, a second batch normalization layer, a second ReLU layer, a second 1×1 convolutional layer, and a third batch normalization layer connected in sequence; the structure of the intermediate module includes: a fourth batch normalization layer, a third ReLU layer, a third 1×1 convolutional layer, a fifth batch normalization layer, a fourth ReLU layer, a second 3×3 convolutional layer, a sixth batch normalization layer connected in sequence The structure of the final module includes: a seventh batch normalization layer, a sixth ReLU layer, a fifth 1×1 convolutional layer, an eighth batch normalization layer, a sixth ReLU layer, a third 3×3 convolutional layer, a ninth batch normalization layer, a seventh ReLU layer, and a sixth 1×1 convolutional layer; the main path of the final module is also sequentially provided with a tenth batch normalization layer and an eighth ReLU layer.

[0009] In some embodiments of the first aspect of the present application, the guided feature refinement module runs in parallel with the RPN detection head in the region proposal network to process the multi-scale feature image to obtain a candidate region; wherein the specific process of processing the multi-scale feature image includes: inputting the P3 feature image, P4 feature image, P5 feature image and P6 feature image in the multi-scale feature image into the four RPN detection heads of the region proposal network respectively, and inputting the P3 feature image and P2 feature image in the multi-scale feature image into the guided feature refinement module to obtain a candidate region.

[0010] In some embodiments of the first aspect of the present application, the processing process of the P3 feature image and the P2 feature image by the guided feature refinement module includes: inputting the P3 feature image into a double convolution layer for processing to obtain a probability feature image, and identifying grid cells containing small-scale targets from the probability feature image; mapping the identified grid cells to the P2 feature image using a central coordinate mapping method to obtain a sparse feature map; and inputting the obtained sparse feature map into a sparse convolutional RPN detection head to obtain a candidate area.

[0011] In some embodiments of the first aspect of the present application, the region proposal network in the improved Faster-RCNN algorithm model adopts a total loss function for model training; wherein the total loss function is composed of the loss function of the region proposal network in the baseline Faster-RCNN algorithm model and the loss function of the guided feature refinement module.

[0012] To achieve the above-mentioned purpose and other related purposes, the second aspect of the present application provides a small-scale target detection device for power line inspection, including: a data set acquisition module, used to acquire a transmission line data set and perform small-scale target annotation; a model construction module, used to use the benchmark Faster-RCNN algorithm model as the benchmark model, and use the AdvResNet network combined with the feature pyramid structure to replace the VGG16 network in the benchmark model, introduce a guided feature refinement module in the region proposal network, and use the ROI alignment network to replace the region of interest pooling network in the benchmark model to obtain an improved Faster-RCNN algorithm model; a model training module, used to use the annotated data set to train and evaluate the improved Faster-RCNN algorithm model for small-scale target detection of transmission line images of the inspection area collected in real time by a drone.

[0013] To achieve the above-mentioned purpose and other related purposes, the third aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the small-scale target detection method for power line inspection is implemented.

[0014] To achieve the above-mentioned purpose and other related purposes, the fourth aspect of the present application provides an electronic terminal, including a memory, a processor and a computer program stored in the memory; the processor executes the computer program to implement the small-scale target detection method for power line inspection.

[0015] As described above, the small-scale target detection method, device, medium and terminal for power line inspection of the present application have the following beneficial effects:

[0016] The improved Faster-RCNN algorithm model of this application uses the AdvResNet network combined with the feature pyramid structure to improve the efficiency of feature extraction, guides the feature refinement module to improve the detection capability of small-scale targets, and the ROI alignment network to improve the positioning accuracy of the bounding box, thereby improving the reliability and safety of transmission line inspection. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 Shown is a flow chart of a small-scale target detection method for power line inspection in one embodiment of the present application.

[0018] Figure 2 Shown is a schematic diagram of the structure of the improved Faster-RCNN in one embodiment of the present application.

[0019] Figure 3 Shown is a schematic diagram of a bottleneck structure in a specific embodiment of the present application.

[0020] Figure 4 Shown is a schematic diagram of a newly constructed bottleneck structure in a specific embodiment of the present application.

[0021] Figure 5 It shows a schematic diagram of the structure of the initial module, the intermediate module and the final module in one embodiment of the present application.

[0022] Figure 6 Shown is a structural schematic diagram of an SE module in one embodiment of the present application.

[0023] Figure 7 Shown is a structural schematic diagram of a guided feature refinement module in one embodiment of the present application.

[0024] Figure 8 Shown is a schematic structural diagram of a small-scale target detection device for power line inspection in one embodiment of the present application.

[0025] Fig. 9 Shown is a schematic diagram of the structure of an electronic terminal in one embodiment of the present application. DETAILED DESCRIPTION

[0026] The following describes the embodiments of the present application through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0027] In the embodiments of the present application, words such as "first" and "second" are used to distinguish the same or similar items with substantially the same functions and effects. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit the difference.

[0028] It should be noted that in the embodiments of the present application, words such as "exemplary" or "for example" represent examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0029] In the embodiments of the present application, "at least one" refers to one or more, and "plurality" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can represent: a, b, c, ab, ac, bc or abc, where a, b, c can be single or multiple.

[0030] The present application provides a small-scale target detection method, device, medium and terminal for power line inspection, the method comprising: obtaining a transmission line data set and annotating small-scale targets; improving the baseline Faster-RCNN algorithm model to obtain an improved Faster-RCNN algorithm model; using the annotated data set, training and evaluating the improved Faster-RCNN algorithm model for small-scale target detection of transmission line images of the inspection area collected in real time by a drone. The AdvResNet network combined with the feature pyramid structure used in the improved Faster-RCNN algorithm model of the present application improves the efficiency of feature extraction, guides the feature refinement module to improve the detection capability of small-scale targets, and the ROI alignment network improves the positioning accuracy of the bounding box, thereby improving the reliability and safety of transmission line inspection.

[0031] Before further describing the present invention in detail, the application environment of the present invention is explained:

[0032] The small-scale target detection method for power line inspection of the present invention is applied to a power line inspection system based on a 6G network. A drone end equipped with a high-resolution camera and a 6G network module is used in the system. The drone end can autonomously navigate in a complex environment, collect ultra-high-resolution transmission line images of the inspection area in real time, and send the collected images to the edge server through the 6G network module. The high-bandwidth characteristics of the 6G network enable the ultra-high-resolution transmission line images collected by the drone end to achieve real-time high-speed data transmission and low-latency image transmission. The edge server is used to remotely control the drone end, process and store the collected images, and run the small-scale target detection method for power line inspection of the present invention to perform target detection on small-scale components in the transmission line.

[0033] To facilitate understanding of the embodiments of the present application, first Figure 1 Detailed description. Figure 1 The flowchart of a small-scale target detection method for power line inspection in an embodiment of the present invention is shown. The small-scale target detection method for power line inspection in this embodiment mainly includes the following steps:

[0034] Step S11: Acquire the transmission line data set and perform small-scale target annotation.

[0035] In one embodiment, a method for obtaining a transmission line data set includes: collecting transmission line images in a variety of real-life scenarios; performing color dithering and random horizontal flipping on the collected transmission line images to form a preliminary data set; and segmenting and copying and pasting the preliminary data set to obtain a transmission line data set.

[0036] In one embodiment, a small-scale target generally refers to an object whose absolute pixel value is less than 32×32 pixels, or occupies less than 1% of the image area; types of small-scale targets include but are not limited to: insulators, vibrating hammers, screws, etc.

[0037] The following is an explanation of how to obtain the transmission line dataset:

[0038] (1) Image acquisition: Collecting images of transmission lines in various real-world scenarios; the types of real-world scenarios include but are not limited to mountains, rivers, grasslands, etc.;

[0039] (2) Data enhancement processing: First, the transmission line images collected in each real-life scenario in (1) are subjected to color dithering and random horizontal flipping to form a preliminary data set.

[0040] Random horizontal flipping is a commonly used image preprocessing technique, especially in the fields of machine learning and computer vision. This technique flips the image along the vertical axis so that the image content appears mirrored in the horizontal direction. The purpose of random horizontal flipping is to increase the diversity of the dataset and help the model learn more robust features.

[0041] Color dithering can be used to simulate a variety of lighting conditions that may occur at different times of day or in different weather conditions. Color dithering involves adjusting key attributes of an image, such as brightness, saturation, contrast, etc.

[0042] The following formula 1 can be used for color dithering:

[0043] g(x)=a×f(x)+b; (Formula 1)

[0044] Among them, f(x) represents the original pixel value of the transmission line image, g(x) represents the adjusted pixel value, a is the gain factor, which affects the contrast of the image, and b is the bias factor, which affects the brightness of the image.

[0045] In a specific embodiment, the gain factor a is limited to a standard ratio range of 0.8 to 1.25. Compared with a large ratio jitter with a gain factor a between 0.1 and 2.0, the jitter within the standard ratio range has a higher detection accuracy in subsequent target detection.

[0046] Furthermore, small-scale targets are segmented from the transmission line images in the preliminary data set; wherein, image segmentation models such as convolutional neural network (CNN) and Mask R-CNN can be used for image segmentation, which is not limited in the present invention.

[0047] Furthermore, the small-scale objects obtained after image segmentation are copied and pasted to different scene images to generate a large number of composite images containing small-scale components, thereby obtaining a transmission line dataset. The copy-paste process enhances the robustness of the model.

[0048] Step S12: Taking the benchmark Faster-RCNN algorithm model as the benchmark model, the AdvResNet network combined with the feature pyramid structure is used to replace the VGG16 network in the benchmark model, a guided feature refinement module is introduced into the region proposal network, and the ROI alignment network is used to replace the region of interest pooling network in the benchmark model to obtain the improved Faster-RCNN algorithm model.

[0049] In one embodiment, the benchmark Faster-RCNN algorithm model includes: a VGG16 network, a region proposal network (RPN), a region of interest (ROI) pooling network, and a classification regression network connected in sequence; wherein the process of using the benchmark Faster-RCNN algorithm model for target detection includes:

[0050] The VGG16 network is used for feature processing to generate feature maps. These feature maps are then input into the region proposal network (RPN). First, an anchor representing the potential target location is generated, and then a 3×3 sliding window processes the anchor to generate a vector. The generated vectors are input into the box regression layer and the box classification layer respectively. The regression layer is used to predict the bounding box, and the classification layer is used to evaluate whether the anchor is the foreground or the background. The RPN generates candidate boxes based on these predictions and maps the candidate boxes to the feature map to obtain the corresponding feature matrix. Finally, each matrix is ​​normalized by ROI pooling. The classification regression network then produces the final prediction result, completing the entire process.

[0051] It should be noted that VGG (Visual Geometry Group) is a convolutional neural network (CNN) architecture. The most commonly used VGG architectures are VGG16 and VGG19. The main difference between VGG16 and VGG19 is the number of convolutional layers. VGG16 has 13 convolutional layers and 3 fully connected layers, while VGG19 has 16 convolutional layers and 3 fully connected layers.

[0052] In one embodiment, ResNet50 is used as a baseline network, and the last four stages in the baseline network are improved respectively to obtain an AdvResNet network; each improved stage consists of an initial module, multiple intermediate modules and a final module; each module adopts a newly constructed bottleneck structure.

[0053] Specifically, the structure of ResNet50 includes: five stages connected in sequence; after the fifth stage of ResNet50, an average pooling layer and a fully connected layer are connected in sequence; the first stage of ResNet50 includes: a convolutional layer, a normalization layer, and an activation function. Each stage from the second to the fifth stage of ResNet50 is composed of multiple residual blocks. The AdvResNet network is obtained by improving the structures of the second to the fifth stages of ResNet50 respectively. Each improved stage consists of an initial module, multiple intermediate modules, and a final module; each module adopts a newly constructed bottleneck structure.

[0054] The AdvResNet network improves the bottleneck structure. The baseline bottleneck structure first uses a 1×1 convolution layer to reduce the number of channels of the input feature map, then uses a 3×3 convolution layer for feature extraction, and finally uses a 1×1 convolution layer again to increase the number of channels of the feature map to match the number of channels of the input feature map.

[0055] For example, if Figure 3 As shown in the bottleneck structure, after the feature map with 256 channels is input into the 1×1 convolution layer, the number of channels becomes 64, and the number of channels of the feature map is reduced. After being processed by the 3×3 convolution layer and the 1×1 convolution layer in sequence, the number of channels of the final feature map is 256, which is the same as the number of channels of the input feature map.

[0056] The first convolution layer (1×1 convolution layer) of the newly constructed bottleneck structure used by the AdvResNet network is used to expand the dimension of the feature map (increase the number of channels of the input feature map), the second convolution layer (3×3 convolution layer) performs feature extraction, and the third convolution layer (1×1 convolution layer) is used to return the original dimension of the feature map to match the number of channels of the input feature map. The newly constructed bottleneck structure allows the second convolution layer to process more channels, which helps to extract a wider range of information.

[0057] For example, if Figure 4 As shown in the bottleneck structure, after the feature map with 256 channels is input into the 1×1 convolution layer, the number of channels becomes 512, and the number of channels of the feature map is expanded. After being processed by the 3×3 convolution layer and the 1×1 convolution layer in sequence, the number of channels of the final feature map is 256, which is the same as the number of channels of the input feature map.

[0058] Each of the second to fifth stages of the AdvResNet network consists of an initial module, multiple intermediate modules, and a final module connected in sequence. It should be noted that the number of intermediate modules included in each stage may be different. The initial module, intermediate module, and final module all use the newly constructed bottleneck structure.

[0059] like Figure 5As shown in Figure 1, the initial module omits the ReLU layer on the main path to allow negative signals to propagate. The main path structure of the initial module includes: the first 1×1 convolution layer, the first batch normalization (Batch Norm) layer, the first ReLU layer, the first 3×3 convolution layer, the second batch normalization layer, the second ReLU layer, the second 1×1 convolution layer, and the third batch normalization layer connected in sequence. The output of the third batch normalization layer is added to the input of the initial module to obtain the output of the initial module.

[0060] The intermediate module relocates the ReLU layer to the beginning of the main path, changing the order of components without increasing the parameter complexity of the network. The main path structure of the intermediate module includes: the fourth batch normalization layer, the third ReLU layer, the third 1×1 convolution layer, the fifth batch normalization layer, the fourth ReLU layer, the second 3×3 convolution layer, the sixth batch normalization layer, the fifth ReLU layer, the fourth 1×1 convolution layer and the scale layer connected in sequence. Among them, the SE (Squeeze-and-Excitation) module is also connected between the fourth 1×1 convolution layer and the scale layer. The SE module is also called the squeeze excitation module. The output of the scale layer is added to the input of the intermediate module to obtain the output of the intermediate module.

[0061] like Figure 6 As shown in Figure 1, the SE module consists of a global pooling layer, a first fully connected layer (FC), a ReLU layer, a second fully connected layer (FC), and a sigmiod layer connected in sequence. The SE module uses a channel attention mechanism to enhance the model's sensitivity to channel features, which helps to extract key information.

[0062] like Figure 5 As shown in the figure, the main path structure of the final module is similar to that of the intermediate module, including: the seventh batch normalization layer, the sixth ReLU layer, the fifth 1×1 convolution layer, the eighth batch normalization layer, the sixth ReLU layer, the third 3×3 convolution layer, the ninth batch normalization layer, the seventh ReLU layer, and the sixth 1×1 convolution layer connected in sequence. The output of the sixth 1×1 convolution layer is added to the input of the final module, and then the result is input to the tenth batch normalization layer and the eighth ReLU layer in sequence to obtain the output of the final module. This structure ensures the normalization of stage information and enhances the network's ability to perform nonlinear learning.

[0063] It should be understood that the first 1×1 convolutional layer, the first 3×3 convolutional layer, and the second 1×1 convolutional layer of the initial module adopt the above-mentioned newly constructed bottleneck structure; the third 1×1 convolutional layer, the second 3×3 convolutional layer, and the fourth 1×1 convolutional layer of the intermediate module adopt the above-mentioned newly constructed bottleneck structure, and the fifth 1×1 convolutional layer, the third 3×3 convolutional layer, and the sixth 1×1 convolutional layer of the final module adopt the above-mentioned newly constructed bottleneck structure. ReLU (Rectified Linear Unit) is an activation function widely used in deep learning. Its function is to perform nonlinear transformation on the input signal to increase the expressive power of the neural network.

[0064] In one embodiment, if Figure 2 As shown, the AdvResNet network combined with the feature pyramid structure is used to extract features of the power transmission line images in the labeled data set to obtain multi-scale feature images; wherein, the AdvResNet network is used to extract features of the power transmission line images in the labeled data set to obtain feature images C1~C5 (one feature image is output in each stage), and then the feature pyramid structure (FPN) performs 1×1 convolution, upsampling, feature image fusion, and 3×3 convolution processing on the feature images C1~C5 in sequence to obtain multi-scale feature images; the multi-scale feature images include: P2 feature images, P3 feature images, P4 feature images, P5 feature images and P6 feature images.

[0065] It should be understood that the main components of the FPN structure include: a basic network: usually a deep convolutional network, which is an AdvResNet network in the embodiment of the present invention, which is used to extract the initial features of the image; a bottom-up path: the output of the basic network is used as the input of the bottom-up path, and multi-scale feature representations are gradually extracted through multiple convolutional layers; lateral connections: at each level of the bottom-up path, high-resolution feature maps are combined with low-level feature maps from the top-down path through lateral connections; a top-down path: a top-down path that gradually reduces the resolution of high-level feature maps through upsampling and is laterally connected to the feature maps of the bottom-up path; feature fusion: through lateral connections and top-down paths, feature maps of different levels are fused to form multi-scale feature representations.

[0066] It should be noted that the feature extraction system formed by the AdvResNet network combined with the feature pyramid structure (FPN) can effectively cope with various complex scenes and lighting changes.

[0067] In a specific embodiment, a comparison experiment was conducted using Faster RCNN models configured with different feature extraction networks. The feature extraction networks are VGG16, ResNet50, ResNet101, and AdvResNet. The dataset consisting of images collected by the drone is divided into training, validation, and test sets in a ratio of 7:2:1. After 400 iterations of training, the detection model parameters converged, indicating that the model has been fully trained for evaluation. Table 1 shows the results of the comparison test.

[0068] Table 1. Performance comparison of different feature extraction modules

[0069] Feature extraction network AP(I) AP(S) AP(SH) mAP VGG16 0.905 0.647 0.793 0.782 ResNet50 0.942 0.761 0.852 0.852 ResNet101 0.936 0.764 0.876 0.857 AdvResNet 0.949 0.744 0.897 0.873

[0070] As can be seen from Table 1, AdvResNet50 has strong information extraction capabilities, especially for small-scale targets. Compared with ResNet50, the detection accuracy is improved by 4.5% and 2.1% compared with ResNet101. AdvResNet, as the feature extraction network of Faster-RCNN, improves the accuracy of target detection and makes the overall model more stable and reliable.

[0071] In one embodiment, if Figure 2 As shown in FIG. 1 , a guided feature refinement (GFR) module is introduced into the benchmark Faster-RCNN algorithm model. The guided feature refinement module runs in parallel with the four RPN detection heads (RPNhead) in the region proposal network to process the P2 to P6 feature images to obtain candidate regions (region-proposal). The specific process of processing the P2 to P6 feature images includes:

[0072] The P3 feature image, P4 feature image, P5 feature image and P6 feature image in the multi-scale feature image are respectively input into the four RPN detection heads of the region proposal network (each feature image corresponds to an RPN detection head). At the same time, the P3 feature image and the P2 feature network are input into the guided feature refinement module for processing to obtain the candidate region.

[0073] In one embodiment, the RPN detection head is generally composed of a 3×3 convolutional layer, a 1×1 convolutional layer for classification, and a 1×1 convolutional layer for regression. Specifically, the 3×3 convolutional layer is used to process the input feature map, and the output feature map maintains the original size. The 1×1 convolutional layer for classification and the 1×1 convolutional layer for regression run in parallel. The 1×1 convolutional layer for classification is used to perform binary classification of the anchor at each position as foreground (object) or background, and the 1×1 convolutional layer for regression is used to perform bounding box regression on the anchor at each position to generate a proposed region. Non-maximum suppression (NMS) is then used to select the final proposed region.

[0074] In one embodiment, the processing process of the P3 feature image and the P2 feature image by the guided feature refinement module includes: inputting the P3 feature image into a double convolution layer for processing to obtain a probability feature image, and identifying grid cells containing small-scale targets from the probability feature image; using a central coordinate mapping method, mapping the identified grid cells to the P2 feature image to obtain a sparse feature map; and inputting the obtained sparse feature map into the sparse convolution RPN detection head to obtain a candidate area.

[0075] Specifically, Figure 2 As shown in Figure 7, the guided feature refinement module first predicts the location of small and difficult-to-detect small-scale targets on the P3 feature image. This step is crucial for finding the region of interest of small-scale targets. By determining the potential area of ​​small-scale targets, the guided feature refinement module effectively narrows the scope of attention during the calculation process, focusing on the area with a high probability of containing small-scale targets. The specific process of this step is to input the P3 feature image into a 3×3 double convolution layer, and the 3×3 double convolution layer outputs a probability feature image T. The probability feature image T effectively measures the probability of the existence of a small-scale target in each grid cell (i, j) of the P3 feature image. These small-scale targets are defined as objects that are smaller than the minimum anchor point of the P3 feature image. For each detected small-scale target, its center coordinates (x, y) are mapped to the corresponding grid cell of the P2 feature image. Then the area corresponding to the small-scale target in the P2 feature image is retained, and other areas are set to 0 to form a sparse feature map. The sparse feature map is input into the sparse convolution RPN detection head (RPNhead with Spconv) to obtain the candidate region.

[0076] It should be understood that the sparse convolution RPN detection head is obtained by replacing the 3×3 convolution layer in the RPN detection head with a sparse convolution. Sparse images generally refer to images in which only a few pixels have non-zero intensity values, while most pixels have intensity values ​​of zero or very close to zero. Such images may appear visually as having many black areas, with image content only in certain specific areas. Grid cells generally refer to single pixels or pixel blocks in a feature map, which are the basic units used by the network to identify and understand specific features in an image. In the field of object detection, anchor points generally refer to the concept of anchor boxes. Anchor boxes are predefined boxes used to assist object detection. They are evenly distributed on the feature map and are used to quickly locate and identify target objects in images.

[0077] In one embodiment, if Figure 2 As shown, the ROI alignment network (ROI Align) is used to align the candidate region with the P2~P6 feature images to output a feature map of the same size. The processing process of the ROI alignment network includes: determining the position of the candidate region on the feature map and dividing it into grid cells of the same size as the output feature map. Each grid cell corresponds to a pixel in the output feature map. The four neighboring points of each grid cell are mapped back to the original feature map. For any pixel point of each grid cell, the following formula 2 is used for bilinear interpolation to calculate the relative distance between the sampling point and the four neighboring points:

[0078] f(x,y)=f(Q 11 )(x2-x)(y2-y)+f(Q 21 )(x-x1)(y2-y)+f(Q 12 )(x2-x)(y-y1)+f(Q 22 )(x-x1)(y-y1); (Formula 2)

[0079] Among them, Q 11 , Q 21 , Q 12 , Q 22 It is represented by the coordinates of four adjacent points, (x, y) is represented by the coordinates of the pixel point, and f(x, y) is represented by the pixel value.

[0080] It should be noted that the ROI pooling network has a pixel misalignment problem. This problem stems from the quantization error caused by rounding floating point numbers to integer pixel coordinates during the process of converting image coordinates to feature map coordinates and then to ROI feature coordinates. The ROI Alignment Network (ROI Align) retains the original floating point pixel coordinates, ensuring high precision and alignment when extracting features from the region of interest, and improving the accuracy of the target detection model in complex imaging scenes.

[0081] In one embodiment, if Figure 2 As shown in the figure, the classification regression network structure of the baseline Faster-RCNN algorithm model and the improved Faster-RCNN algorithm model is the same. The structure of the classification regression network specifically includes: a fully connected layer, a ReLU layer, a fully connected layer, and a ReLU layer connected in sequence; the output of the second ReLU layer is input into two convolutional layers respectively, the output of one convolutional layer is input into the bounding box prediction, and the output of the other convolutional layer is input into the bounding box classification through softmax.

[0082] Step S13: Using the labeled data set, train and evaluate the improved Faster-RCNN algorithm model to detect small-scale targets in the transmission line images of the inspection area collected in real time by the drone.

[0083] In one embodiment, the specific process of performing small-scale target detection on the transmission line images of the inspection area collected in real time by the drone end includes: planning the flight path and inspection route of the drone end according to the geographic information of the inspection area; during the inspection process, the camera carried by the drone end collects the transmission line images of the inspection area in real time, and transmits them to the edge server in real time through the 6G communication module; after the edge server receives the transmission line image, it uses the improved Faster-RCNN algorithm model to detect small-scale targets; classifies and annotates the detected small-scale targets with bounding boxes, generates an inspection report and feeds it back to the technicians.

[0084] In one embodiment, the improved Faster-RCNN algorithm model uses the generalized intersection-over-union (GIOU) to replace the traditional intersection-over-union (IOU). The generalized intersection-over-union (GIOU) is calculated using the following formula 3:

[0085]

[0086] Among them, A represents the predicted box, B represents the real box, and C is the smallest closed area containing A and B.

[0087] It should be noted that the traditional intersection-over-union (IOU) is used to predict the overlap ratio between the bounding box and the actual target, but the size and shape of the target and the bounding box are not taken into account. And when there is no overlap between the predicted bounding box and the actual target, its value is zero, resulting in failure to optimize. Even if the loss value of the bounding box is the same but the IOU score is different, IOU cannot distinguish the different alignments of the bounding box relative to the true value. When the two boxes do not overlap, the GIOU method can reflect the distance between the predicted bounding box and the actual target through the C area. In the bounding box regression process, GIOU as a loss function can more effectively guide the predicted box to converge to the real box, especially when the two boxes are far apart. The GIOU method improves the accuracy of regression, so that the predicted box can be aligned with the real box, and can better handle situations where the bounding boxes do not overlap or partially overlap, thereby improving the robustness of target detection.

[0088] In one embodiment, the following formula 4 is used to calculate the GIOU (generalized intersection-over-union) loss:

[0089]

[0090] Among them, L GIOU is the GIOU loss function.

[0091] Specifically, the calculation steps include: calculating the intersection-over-union ratio of the predicted box A and the true box B; determining the minimum bounding box C, and calculating the area of ​​the minimum bounding box C that contains the predicted box A and the true box B; calculating the area of ​​the area in the minimum bounding box C that is not covered by A∪B; calculating GIOU; calculating the GIOU loss L GIOU .

[0092] In one embodiment, the region proposal network in the improved Faster-RCNN algorithm model adopts a total loss function for model training; wherein the total loss function is composed of the loss function of the region proposal network in the baseline Faster-RCNN algorithm model and the loss function of the guided feature refinement module.

[0093] Specifically, the total loss function L total The calculation method of is shown in Formula 5:

[0094] L total =L FR +λLG FR ; (Formula 5)

[0095] Among them, L FR Denotes the loss function of the region proposal network of the baseline Faster-RCNN algorithm model, LG FR It is expressed as the loss function of the guided feature refinement module, and λ is the adjustment coefficient.

[0096] LGFR The following formula 6 is used for calculation:

[0097]

[0098] Where N is the total number of grid cells, y i is a real value (0 or 1), is the predicted probability.

[0099] L FR The following formula 7 is used for calculation:

[0100]

[0101] Among them, L cls Expressed as classification loss, calculated using binary cross entropy, L reg It is expressed as regression loss, using Smooth L1 loss. i It is expressed as the predicted probability of the anchor point as the target, is the true label (positive sample is 1, negative sample is 0), t i , is the ground-truth bounding box parameter vector, N cls and N reg is the normalization term, λ FR A balance parameter for adjusting the relative weights of classification loss and regression loss.

[0102] The classification loss L of the region proposal network cls The following formula 8 is used for calculation:

[0103]

[0104] Regression loss L reg The following formula 9 is used for calculation:

[0105]

[0106] in,

[0107] It should be noted that Smooth L1 loss is less sensitive to outliers than L2 loss.

[0108] In one embodiment, the classification regression network is trained using the loss function represented by the following formula 10:

[0109] L=L cls2 (P,u)+λ2L reg (t u ,v); (Formula 10)

[0110] Among them, L cls2(P,u) is the classification loss of the classification regression network, which calculates the cross entropy loss of the softmax probability P of the correct category label u; L reg (t u ,v) is the regression loss of the classification regression network, and Smooth L1 loss is used for loss calculation, t u represents the predicted bounding box adjustment of category u, v represents the actual bounding box parameters, and λ2 is the weight for balancing the classification loss and regression loss of the classification regression network.

[0111] It should be noted that the above λ, λ FR As well as the λ2 weight, those skilled in the art can adjust them according to actual conditions during model training, and the present invention is not limited to this.

[0112] In one embodiment, three indicators, True Positives (TP), False Positives (FP), and False Negatives (FN), are used to evaluate the performance of the model. TP counts the number of correctly detected bounding boxes. If the intersection over union (GIOU) of the bounding box with the ground truth (GT) annotation is greater than 0.5, and each GT only matches one detection, then it is considered a TP. If the GIOU of the bounding box with the GT is less than or equal to 0.5, or it is a redundant detection of the same GT object, then it is considered a FP. FN represents the number of GT objects that the model failed to detect.

[0113] In one embodiment, the model is evaluated using two indicators, precision and recall. The precision and recall are calculated using the following formulas 11 and 12:

[0114]

[0115] In one embodiment, the average precision (AP) indicator is also introduced. AP represents the area under the precision-recall curve of each object category. The mean average precision (mAP) is obtained by averaging the AP values ​​of all categories. mAP is a value from 0 to 1, which is a comprehensive indicator of the overall performance of the model. The higher the value, the better the detection ability.

[0116] In one embodiment, the improved Faster-RCNN algorithm model is compared with the benchmark Faster R-CNN (ResNet50), SSD, YOLOv3 and YOLOv4. These models are trained on a dataset designed for power line target detection. The goal is to perform detection on a unified test set after the model has been trained and learned for 400 iterations to ensure that their parameters have fully converged. The results of the test are shown in Table 2.

[0117] Table 2. Comparative test results

[0118] Model mAP@0.5 mAP@0.75 FPS Faster RCNN 0.852 0.447 5 YOLO v3 0.793 0.367 14 YOLO v4 0.815 0.419 18 SSD 0.759 0.385 15 Ours 0.893 0.624 8

[0119] As can be seen from Table 2, the improved Faster-RCNN algorithm model (denoted as Ours in Table 2) shows excellent performance in terms of mAP (mean average precision). When the IOU threshold is 50%, the improved Faster-RCNN algorithm model is better than the single-stage model (SSD, YOLOv3 and YOLOv4) and the baseline Faster-RCNN algorithm model in detection accuracy. When the IOU threshold is 75%, the improvement in mAP is more significant. The baseline Faster-RCNN algorithm model runs at 5 frames per second (FPS), while the improved Faster-RCNN algorithm model is increased to 8 frames per second, and the detection efficiency of the model is improved.

[0120] In order to better illustrate the small-scale target detection method for power line inspection of the present invention, a specific embodiment is now provided.

[0121] Embodiment 1: A test environment for a small-scale target detection method for power line inspection.

[0122] The small-scale target detection method for power line inspection in this embodiment is tested on an edge server. The edge server runs the Windows 11 operating system and uses Python 3.8 as the programming language. The hardware configuration includes a Xeon Gold5218R CPU and 32GB of memory, and the graphics card is an RTX2080Ti, which is a GPU with 16GB of video memory. The software environment is based on CUDA (Compute Unified Device Architecture) version 11.8. In addition, the model also uses the open source machine learning library PyTorch1.11.2 version.

[0123] Figure 8 is a schematic block diagram of a small-scale target detection device for power line inspection provided by an embodiment of the present application. Figure 8 As shown, the small-scale target detection device for power line inspection includes:

[0124] A data set acquisition module 81 is used to acquire a data set of power transmission lines and perform small-scale target annotation;

[0125] A model building module 82 is used to use the benchmark Faster-RCNN algorithm model as the benchmark model, replace the VGG16 network in the benchmark model with the AdvResNet network combined with the feature pyramid structure, introduce a guided feature refinement module in the region proposal network, and replace the region of interest pooling network in the benchmark model with the ROI alignment network to obtain an improved Faster-RCNN algorithm model;

[0126] The model training module 83 is used to train and evaluate the improved Faster-RCNN algorithm model using the labeled data set for small-scale target detection of power transmission line images in the inspection area collected in real time by the drone end.

[0127] It should be understood that the specific process of each module executing the above corresponding steps has been described in detail in the above method embodiment, and for the sake of brevity, it will not be repeated here.

[0128] It should also be understood that the division of modules in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional module in each embodiment of the present application may be integrated into a processor, or may exist physically separately, or two or more modules may be integrated into one module. The above-mentioned integrated modules may be implemented in the form of hardware or in the form of software functional modules.

[0129] In one embodiment, a method for obtaining a transmission line data set includes: collecting transmission line images in a variety of real-life scenarios; performing color dithering and random horizontal flipping on the collected transmission line images to form a preliminary data set; and segmenting and copying and pasting the preliminary data set to obtain a transmission line data set.

[0130] In one embodiment, the AdvResNet network combined with the feature pyramid structure is used to extract features from the power transmission line images in the labeled data set to obtain a multi-scale feature image; wherein, ResNet50 is used as the benchmark network, and the last four stages in the benchmark network are improved respectively to obtain the AdvResNet network; each improved stage consists of an initial module, multiple intermediate modules and a final module; each module adopts a newly constructed bottleneck structure.

[0131] In one embodiment, the structure of the initial module includes: a first 1×1 convolutional layer, a first batch normalization layer, a first ReLU layer, a first 3×3 convolutional layer, a second batch normalization layer, a second ReLU layer, a second 1×1 convolutional layer and a third batch normalization layer connected in sequence; the structure of the intermediate module includes: a fourth batch normalization layer, a third ReLU layer, a third 1×1 convolutional layer, a fifth batch normalization layer, a fourth ReLU layer, a second 3×3 convolutional layer, a sixth batch normalization layer, a fifth ReLU layer, a fourth 1×1 convolutional layer and a scale layer connected in sequence; wherein an SE module is further connected between the fourth 1×1 convolution and the scale layer; the structure of the final module includes: a seventh batch normalization layer, a sixth ReLU layer, a fifth 1×1 convolutional layer, an eighth batch normalization layer, a sixth ReLU layer, a third 3×3 convolutional layer, a ninth batch normalization layer, a seventh ReLU layer and a sixth 1×1 convolutional layer; the main path of the final module is further provided with a tenth batch normalization layer and an eighth ReLU layer in sequence.

[0132] In one embodiment, the guided feature refinement module runs in parallel with the RPN detection head in the region proposal network to process the multi-scale feature image to obtain a candidate region; wherein the specific process of processing the multi-scale feature image includes: inputting the P3 feature image, P4 feature image, P5 feature image and P6 feature image in the multi-scale feature image into the four RPN detection heads of the region proposal network respectively, and inputting the P3 feature image and P2 feature image in the multi-scale feature image into the guided feature refinement module to obtain a candidate region.

[0133] In one embodiment, the processing process of the P3 feature image and the P2 feature image by the guided feature refinement module includes: inputting the P3 feature image into a double convolution layer for processing to obtain a probability feature image, and identifying grid cells containing small-scale targets from the probability feature image; using a central coordinate mapping method, mapping the identified grid cells to the P2 feature image to obtain a sparse feature map; and inputting the obtained sparse feature map into the sparse convolution RPN detection head to obtain a candidate area.

[0134] In some embodiments of the first aspect of the present application, the region proposal network in the improved Faster-RCNN algorithm model adopts a total loss function for model training; wherein the total loss function is composed of the loss function of the region proposal network in the baseline Faster-RCNN algorithm model and the loss function of the guided feature refinement module.

[0135] Fig. 9 is a schematic block diagram of an electronic terminal provided in an embodiment of the present application. Fig. 9As shown, the electronic terminal includes: at least one processor 901, a memory 902, at least one network interface 903 and a user interface 905. The various components in the device are coupled together through a bus system 904. It can be understood that the bus system 904 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 904 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, Fig. 9 In the specification, various buses are labeled as bus systems.

[0136] The user interface 905 may include a display, a keyboard, a mouse, a trackball, a click gun, keys, buttons, a touch pad or a touch screen.

[0137] It is understood that the memory 902 can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM). The memory described in the embodiments of the present invention is intended to include but is not limited to these and any other suitable categories of memory.

[0138] The memory 902 in the embodiment of the present invention is used to store various categories of data to support the operation of the electronic terminal 900. Examples of these data include: any executable program for operating on the electronic terminal 900, such as an operating system 9021 and an application 9022; the operating system 9021 includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks. The application 9022 can include various applications, such as a media player (Media Player), a browser (Browser), etc., for implementing various application services. The small-scale target detection method for power line inspection provided in the embodiment of the present invention can be included in the application 9022.

[0139] The method disclosed in the above embodiment of the present invention can be applied to the processor 901, or implemented by the processor 901. The processor 901 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit in the processor 901 or the instruction in the form of software. The above processor 901 may be a general processor, a digital signal processor (DSP, Digital Signal Processor), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor 901 can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiment of the present invention. The general processor 901 may be a microprocessor or any conventional processor, etc. In combination with the steps of the accessory optimization method provided in the embodiment of the present invention, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0140] In an exemplary embodiment, the electronic terminal 900 may be implemented by one or more application specific integrated circuits (ASIC), DSP, programmable logic device (PLD), complex programmable logic device (CPLD) to execute the aforementioned method.

[0141] According to the method provided in the embodiment of the present application, the present application also provides a computer program product, which includes: a computer program code, when the computer program code is run on a computer, the computer executes Figure 1 The illustrated embodiment is a small-scale target detection method for power line inspection.

[0142] According to the method provided in the embodiment of the present application, the present application also provides a computer-readable storage medium, which stores a program code, and when the program code is run on a computer, the computer executes Figure 1 The illustrated embodiment is a small-scale target detection method for power line inspection.

[0143] The terms "component", "module", "system", etc. used in this specification are used to represent computer-related entities, hardware, firmware, a combination of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program and / or a computer. By way of illustration, both applications running on a computing device and a computing device can be components. One or more components may reside in a process and / or an execution thread, and a component may be located on a computer and / or distributed between two or more computers. In addition, these components may be executed from various computer-readable media having various data structures stored thereon. Components may, for example, communicate through local and / or remote processes according to signals having one or more data packets (e.g., data from two components interacting with another component between a local system, a distributed system and / or a network, such as the Internet interacting with other systems through signals).

[0144] Those of ordinary skill in the art will appreciate that the various illustrative logical blocks and steps described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or in a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0145] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0146] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0147] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0148] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0149] In the above embodiments, the functions of each functional unit can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions (programs). When loading and executing computer program instructions (programs) on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. Computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media integrations. Available media may be magnetic media (e.g., floppy disks, hard disks, tapes), optical media (e.g., high-density digital video discs (DVDs), or semiconductor media (e.g., solid state disks (SSDs), etc.).

[0150] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0151] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0152] In summary, the present application provides a small-scale target detection method, device, medium and terminal for power line inspection, the method comprising: obtaining a transmission line data set and annotating small-scale targets; improving the benchmark Faster-RCNN algorithm model to obtain an improved Faster-RCNN algorithm model; using the annotated data set, training and evaluating the improved Faster-RCNN algorithm model for small-scale target detection of transmission line images of the inspection area collected in real time by the drone end. The AdvResNet network combined with the feature pyramid structure adopted by the improved Faster-RCNN algorithm model of the present application improves the efficiency of feature extraction, guides the feature refinement module to improve the detection capability of small-scale targets, and the ROI alignment network improves the positioning accuracy of the bounding box, thereby improving the reliability and safety of the transmission line inspection. Therefore, the present application effectively overcomes the various shortcomings in the prior art and has a high industrial utilization value.

[0153] The above embodiments are merely illustrative of the principles and effects of the present application and are not intended to limit the present application. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed in the present application shall still be covered by the claims of the present application.

Claims

1. A small-scale target detection method for power line inspection, characterized in that: include: Obtain the transmission line dataset and perform small-scale target annotation; Taking the benchmark Faster-RCNN algorithm model as the benchmark model, the AdvResNet network combined with the feature pyramid structure is used to replace the VGG16 network in the benchmark model, the guided feature refinement module is introduced into the region proposal network, and the ROI alignment network is used to replace the region of interest pooling network in the benchmark model to obtain the improved Faster-RCNN algorithm model; The annotated dataset is used to train and evaluate the improved Faster-RCNN algorithm model for small-scale target detection in the transmission line images of the inspection area collected in real time by drones.

2. The small-scale target detection method for power line inspection according to claim 1 is characterized in that: The method of obtaining a transmission line data set includes: collecting transmission line images in various real-life scenarios; performing color dithering and random horizontal flipping on the collected transmission line images to form a preliminary data set; and performing segmentation and copy-paste processing on the preliminary data set to obtain a transmission line data set.

3. The small-scale target detection method for power line inspection according to claim 1 is characterized in that: The AdvResNet network combined with the feature pyramid structure is used to extract features of the transmission line images in the labeled data set to obtain multi-scale feature images; wherein, ResNet50 is used as the benchmark network, and the last four stages in the benchmark network are improved respectively to obtain the AdvResNet network; each improved stage consists of an initial module, multiple intermediate modules and a final module; each module adopts a newly constructed bottleneck structure.

4. The small-scale target detection method for power line inspection according to claim 3 is characterized in that: The structure of the initial module includes: a first 1×1 convolutional layer, a first batch normalization layer, a first ReLU layer, a first 3×3 convolutional layer, a second batch normalization layer, a second ReLU layer, a second 1×1 convolutional layer and a third batch normalization layer connected in sequence; the structure of the intermediate module includes: a fourth batch normalization layer, a third ReLU layer, a third 1×1 convolutional layer, a fifth batch normalization layer, a fourth ReLU layer, a second 3×3 convolutional layer, a sixth batch normalization layer, a fifth ReLU layer, a fourth 1×1 convolutional layer and a scale layer connected in sequence; wherein an SE module is further connected between the fourth 1×1 convolutional layer and the scale layer; the structure of the final module includes: a seventh batch normalization layer, a sixth ReLU layer, a fifth 1×1 convolutional layer, an eighth batch normalization layer, a sixth ReLU layer, a third 3×3 convolutional layer, a ninth batch normalization layer, a seventh ReLU layer and a sixth 1×1 convolutional layer; the main path of the final module is further provided with a tenth batch normalization layer and an eighth ReLU layer in sequence.

5. The small-scale target detection method for power line inspection according to claim 3 is characterized in that: The guided feature refinement module runs in parallel with the RPN detection head in the region proposal network to process the multi-scale feature image to obtain a candidate region; wherein the specific process of processing the multi-scale feature image includes: The P3 feature image, P4 feature image, P5 feature image and P6 feature image in the multi-scale feature image are respectively input into the four RPN detection heads of the region proposal network, and the P3 feature image and P2 feature image in the multi-scale feature image are input into the guided feature refinement module to obtain the candidate region.

6. The small-scale target detection method for power line inspection according to claim 5, characterized in that: The process of the guided feature refinement module processing the P3 feature image and the P2 feature image includes: The P3 feature image is input into a double convolutional layer for processing to obtain a probability feature image, and the grid cells containing small-scale targets are identified from the probability feature image; The identified grid cells are mapped to the P2 feature image by means of center coordinate mapping to obtain a sparse feature map; The obtained sparse feature map is input into the sparse convolutional RPN detection head to obtain the candidate region.

7. The small-scale target detection method for power line inspection according to claim 1, characterized in that: The region proposal network in the improved Faster-RCNN algorithm model adopts a total loss function for model training; wherein the total loss function is composed of the loss function of the region proposal network in the benchmark Faster-RCNN algorithm model and the loss function of the guided feature refinement module.

8. A small-scale target detection device for power line inspection, characterized in that: include: The data set acquisition module is used to acquire the data set of power transmission lines and perform small-scale target annotation; The model building module is used to replace the VGG16 network in the benchmark model with the AdvResNet network combined with the feature pyramid structure, introduce the guided feature refinement module in the region proposal network, and replace the region of interest pooling network in the benchmark model with the ROI alignment network to obtain the improved Faster-RCNN algorithm model. The model training module is used to train and evaluate the improved Faster-RCNN algorithm model using the labeled data set for small-scale target detection of power transmission line images in the inspection area collected in real time by drones.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. An electronic terminal comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the method according to any one of claims 1 to 7.

Images