Improved YOLOv8 tower foundation target detection method and device
By introducing DAttention and ShapeIoU loss function in the YOLOv8 tower base object detection method, combined with the occlusion perception attention mechanism, the problem of poor robustness of tower base object detection in the prior art is solved, and higher accuracy and faster detection is achieved Effect.
Patent Information
- Application Number
- CN202411093396.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-09
- Publication Date
- 2024-11-29
AI Technical Summary
The existing target detection methods have limitations and poor robustness in the case of many feature information and complex backgrounds, making it difficult to accurately detect and locate the base targets of the transmission tower.
Using the improved YOLOv8 tower base object detection method, the model is optimized to improve feature extraction and detection accuracy by adding a DAttention attention mechanism to the backbone network, using the ShapeIoU loss function to replace the original coordinate loss function, and combining the occlusion perception attention mechanism.
The accuracy and speed of tower base target detection were improved. The detection result mAP@50-95 reached 82.3%, which was 1.6% higher than the original YOLOv8, and GFLOPs and FPS were reduced by 9.9% and 10.3% respectively, which significantly improved the detection accuracy and efficiency.
Smart Images

Figure CN119048736A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of computer vision target detection, and in particular relates to a method and device for detecting a target based on an improved YOLOv8 tower base. Background Art
[0002] The energy industry is developing rapidly. With the continuous increase in energy demand, transmission towers have been widely used as important facilities for power transmission. Traditional transmission tower inspection methods usually rely on manual inspections, which are inefficient and have certain safety risks. The demand for automation and intelligence has promoted the research and development of transmission tower inspection technology. With the rapid development of computer vision and deep learning technology, target detection algorithms have made significant progress in various fields. These algorithms can realize automatic target recognition and position detection by analyzing image or video data. Transmission towers have specific shapes and structural characteristics. The bottom is the tower base. If the tower base is abnormal or damaged, it may affect the stability and safety of the transmission tower. In order to ensure the stable operation of the power system, higher requirements are placed on the safety and reliability of the transmission tower. Therefore, it is of great significance to accurately and quickly detect and locate the bottom target of the tower base of the transmission tower.
[0003] In this context, research on tower base target detection algorithms can provide an automated and efficient way to detect and identify the bottom of transmission towers. By using computer vision and deep learning technology, combined with corresponding data collection and processing methods, it is possible to quickly and accurately detect and locate targets at the bottom of transmission towers, provide support for transmission tower inspections, and improve the safety and reliability of power grid systems.
[0004] Object detection is a task in the field of computer vision, which aims to identify the position and size of the target object in an image or video. That is, each pixel in the input image or video is classified as a target object or background, and the position and size of the target object in the image are determined. Object detection is widely used in many fields, such as intelligent transportation, security monitoring, medical imaging, etc. Through object detection, the computer can identify the objects in the image, and then perform operations such as classification, positioning and tracking.
[0005] Traditional target detection methods process images using a sliding window, first generating candidate boxes, then extracting features from each candidate box, and using classifiers such as Adaboost and decision trees to determine whether it is a target. Finally, the classification and recognition results are post-processed to remove redundant candidate boxes to obtain the best target detection position. These methods are simple to operate and require less computation. They have certain advantages for targets with fewer features, but they also have obvious limitations and poor robustness when there is a lot of feature information and complex backgrounds. Summary of the invention
[0006] The purpose of the present invention is to overcome the shortcomings of the above-mentioned prior art and provide a method and device for target detection based on an improved YOLOv8 tower base, so as to solve the defects that the existing target detection methods in the prior art have obvious limitations and poor robustness when there is a lot of feature information and complex background.
[0007] In order to achieve the above object, the present invention adopts the following technical solutions:
[0008] The improved YOLOv8 tower base target detection method includes the following steps:
[0009] S1, obtain the power grid tower base image collected by the drone;
[0010] S2, detect the collected image through the improved YOLOv8 tower base target detection network model, obtain and mark the tower base in the image;
[0011] The network model of tower base target detection of YOLOv8 includes an input end, a backbone network, a neck network and a detection head; the collected image is input into the input end, the input end pre-processes the input image and then passes it into the backbone network, the backbone network extracts features from the image to obtain a feature map, the feature maps of different levels are fused through the neck network, and the fused feature maps are output to the detection head for detection and classification, and finally the detection head outputs the tower base target detection result;
[0012] A DAttention mechanism is added after the SPPF of the backbone network, and the output feature map after DAttention feature extraction is used as the input of Concat and UpSample in the neck network;
[0013] An occlusion-aware attention mechanism is added between the Conv and Conv2d modules in the detection head, and the loss function in the detection head is the ShapeIoU loss function;
[0014] S3, outputs an image labeled with the tower base.
[0015] A further improvement of the present invention is:
[0016] Preferably, in S2, Mosaic data enhancement is provided in the input end.
[0017] Preferably, in S2, the C3 positions in the backbone network are all C2f structures.
[0018] Preferably, in S2, the neck network is a dual-stream FPN architecture.
[0019] Preferably, in S2, the detection head is provided with two detection heads, and the two detection heads include separate regression branches and prediction branches.
[0020] Preferably, in S2, M attention heads are provided in the Dattention, each attention head is provided with K initial sampling points, the coordinates of the K sampling points are updated by the offset generated by the linear operation, the attention weight of each sampling point is obtained by the linear operation, the product of the K sampling points and the corresponding attention weight is summed, and the summed results of the M attention heads are superimposed to obtain the feature information.
[0021] Preferably, in S2, the occlusion-aware attention is provided with a plurality of depth-wise separable convolutions, and in each depth-wise separable convolution module, a point-by-point convolution operation is first performed on the feature map using a depth convolution with a residual network structure, and then the output feature map of each point-by-point convolution module is fused with the initial input of the occlusion-aware attention module, and attention weights are generated for the fused feature map through global average pooling and full connection, and the attention weights are processed through an exponential function to obtain an attention map of the occluded area; after multiplying the attention map of the occluded area and the feature map output by Conv, a feature map that pays more attention to the occluded area is obtained.
[0022] Preferably, in S2, the network model of the tower base target detection of YOLOv8 is processed by a basic data set, and the process of obtaining the basic data set is:
[0023] (1) Collect and compress grid tower base images;
[0024] (2) Label the tower bases in some images using Labelme, and pre-input the labeled images into the YOLOv8 network model as data sets. The remaining power grid tower base images are screened using the pre-processed pre-trained weight files to obtain the screened power grid tower base images.
[0025] (3) The screened power grid tower base images are corrected using the Labelme annotation software to obtain the basic data set.
[0026] Preferably, the network model of tower base target detection of YOLOv8 is obtained by training with ShapeIoU loss function.
[0027] Based on the improved YOLOv8 tower base target detection device, including:
[0028] A collection module is used to obtain the power grid tower base images collected by the drone;
[0029] A detection module is used to detect the collected image through an improved YOLOv8 tower base target detection network model, and obtain and mark the tower base in the image;
[0030] The network model of the tower base target detection of YOLOv8 includes an input end, a backbone network, a neck network and a detection head; the collected image is input into the input end, the output end of the input end is connected to the input end of the backbone network, the output end of the backbone network is connected to the input end of the neck network, the output end of the neck network is connected to the output end of the detection head, and the output end of the detection head outputs an image marked with the tower base;
[0031] C2f is set in both the backbone network and the neck network, Dattention is added after the SPPF in the backbone network, and the output end of Dattention is connected with Concat and Upsample in the neck network;
[0032] The detection head is provided with an occlusion-aware attention; each C2f output terminal in the neck network is connected to an occlusion-aware attention, and the attention maps output by all occlusion-aware attention connections are multiplied and added with the feature maps output by the C2f, and an image marked with the tower base is output;
[0033] Output module, used to output images marked with tower bases.
[0034] Compared with the prior art, the present invention has the following beneficial effects:
[0035] The present invention discloses a tower base target detection method and device based on improved YOLOv8. The high-resolution image of the tower base area collected by the drone is used as the basic data, and is finally used as the data set of this experiment through manual screening, tower base annotation, and image enhancement. Since the pictures collected by the drone have problems such as complex background, incomplete tower base, occlusion, and tower top interference, the present invention proposes a tower base target detection algorithm based on improved YOLOv8. Adding a deformable attention mechanism DAttention in the backbone network allows the model to focus on the most critical area of the current task, improve the feature extraction capability of the tower base, and improve the expression and recognition capabilities of the model; using ShapeIoU to replace the original coordinate loss function CIoU, more accurately measure the difference between the prediction result of the target detection model and the actual situation, thereby helping to optimize the training process of the model; using the occlusion-aware attention in YOLO-Face V2 to improve the Head, while ensuring the detection accuracy, the model is lightweighted so that it can effectively handle occlusion scenes. The experimental results show that the model detection result mAP@50-95 of this study reaches 82.3%, which is 1.6% higher than the original YOLOv8, and GFLOPs and FPS are reduced by 9.9% and 10.3% respectively, indicating that the improved algorithm has higher detection accuracy and faster detection speed for tower base target detection.
[0036] Furthermore, by training on a large amount of data, the algorithm continuously optimizes model parameters and learns to acquire the ability to identify targets from images to improve the accuracy and robustness of target detection. This deep learning method enables the target detection system to better understand and identify complex visual scenes, laying the foundation for more accurate and efficient target detection tasks. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] Figure 1 It is a tower base target detection flow chart of YOLOv8 of the present invention;
[0038] Figure 2 It is a network model diagram acquisition route map of tower base target detection of YOLOv8 of the present invention;
[0039] Figure 3 It is a network model diagram of tower base target detection of YOLOv8 of the present invention;
[0040] Figure 4 It is the structural diagram of the DAttention mechanism;
[0041] Figure 5 It is a schematic diagram of the ShapeIoU loss function;
[0042] Figure 6 It is the structural diagram of the occlusion-aware attention network module;
[0043] Figure 7 This is an example of the tower base dataset obtained after preprocessing;
[0044] Figure 8 This is the tower base target detection result image using the unimproved YOLOv8d;
[0045] Fig. 9 This is a tower base target detection result diagram using the improved model of the present invention. DETAILED DESCRIPTION
[0046] See also Figure 1 The first aspect of the present invention discloses a tower base target detection method based on an improved YOLOv8, and the detection method specifically comprises the following steps:
[0047] S1, obtain high-resolution images collected by drone.
[0048] S2, the compressed image is detected by the improved YOLOv8 tower base target detection network model, and the tower base is obtained and marked from the image; the improved YOLOv8 tower base target detection network model is as follows Figure 2 shown.
[0049] Among them, the tower base target detection network model based on Yolov8 is divided into input end, backbone network, neck network and detection head: first, the collected picture is passed into the input end, and the input end performs image preprocessing on the input picture and then passes it into the backbone network. Then, the feature map after feature extraction in the backbone network is used as the input of the neck network. The neck network performs feature fusion on feature maps at different levels, and outputs the fused feature map to the detection head for detection and classification. Finally, the detection head outputs the tower base target detection result.
[0050] Add the DAttention mechanism after the SPPF of the backbone network, and use the output feature map after DAttention feature extraction as the input of Concat and UpSample in the neck network;
[0051] An occlusion-aware attention mechanism is added between the Conv and Conv2d modules in the detection head, and the ShapeIoU loss function is used to replace the original coordinate loss function to obtain the improved detection head SEAM-Detect; the output feature map of C2f in the backbone network layer is input into SEAM-Detect for classification and recognition, and finally an image result with tower base detection box and detection accuracy is obtained.
[0052] S3, outputting the image marked with the tower base.
[0053] This method realizes the target detection of tower base based on the improved YOLOv8 to improve the accuracy and speed of tower base detection.
[0054] In S2, Figure 2 As shown, the acquisition process of the improved YOLOv8 tower base target detection network model mainly includes the following steps:
[0055] S21, preprocessing the images of the power grid tower base area collected by the UAV to obtain a data set;
[0056] S211, in order to facilitate the subsequent preprocessing of the data set and simplify the training of the network model, the high-resolution images collected by the drone are compressed and the image resolution is reduced to 640*640.
[0057] S212, in order to facilitate the screening of the images collected by the drone to eliminate the images without tower bases, the images are preprocessed first. 5% to 15% of the collected images are extracted, and the tower bases in some images are manually annotated by the Labelme annotation software, and are input into the original YOLOv8 network model as the basic data set for pre-training to obtain the pre-trained YOLOv8 network model. The remaining images collected by the drone are screened by the pre-trained YOLOv8 network model, and finally the images containing the transmission tower bases are retained as the basic images for subsequent operations.
[0058] S213, use Lableme annotation software to correct the above-screened pictures containing the tower base to improve the accuracy of the data set, so as to ensure the accuracy of the subsequent network model training and improve the accuracy of tower base target detection, and save the annotation file in XML file format as the basic data set of this study.
[0059] S22, see Figure 3 , build a network model for tower base target detection based on improved YOLOv8;
[0060] The overall structure of the YOLOv8 target detection algorithm consists of four parts: input, backbone network, neck network, and detection head.
[0061] In some implementation schemes of the present invention, the input end performs a preprocessing operation on the input image, and the specific preprocessing operation is Mosaic data enhancement to expand the data set to improve the generalization ability of the model.
[0062] As a preferred solution, Mosaic data enhancement includes the following steps:
[0063] (1) Randomly read four images from the dataset each time;
[0064] (2) Perform random flipping, random scaling, and color gamut changes (changing the brightness, saturation, and hue of the original images) on the four images respectively.
[0065] (3) Finally, the four images are randomly cropped and then spliced into one image as training data.
[0066] Mosaic data enhancement is used to preprocess the data set, which greatly enriches the background of the detected objects. In addition, the data of four images are calculated during the standardized BN calculation, which can effectively improve the performance and robustness of the model.
[0067] Combination Figure 3It can be seen that the backbone network includes multiple convolutional layers. In the backbone network part, the C3 structure of YOLOv5 is replaced with the C2f structure with richer gradient flow. Figure 3 In the figure, C2f structures are set between convolutional layer 1 and convolutional layer 3, between convolutional layer 3 and convolutional layer 5, between convolutional layer 5 and convolutional layer 7, and between convolutional layer 7 and SPPF. The setting of C2f reduces the amount of calculation of the model, improves the feature expression, and improves the convergence speed and convergence effect of the model.
[0068] Figure 2 The specific structure diagram of the C2f module is also given. The C2f module includes a convolutional layer, a split layer, a stack of multiple bottleneck layers, a series layer, and a final convolutional layer, which realizes feature extraction, parallel processing, feature fusion, and result output of the input data.
[0069] In some embodiments of the present invention, a deformable attention mechanism called DAttention is added to the backbone network, and DAttention is added after SPPF in the backbone network, and the output of SPPF is used as the input of DAttention. DAttention dynamically adjusts the shape and size of the attention model and the key areas in the focused image, thereby reducing the amount of computation while making the model focus on more critical information, improving the feature extraction capability of the tower base, and improving the expression and recognition capabilities of the model.
[0070] For details, see Figure 4 The following is a diagram of the DAttention network structure. This diagram shows the DAttention model structure. For the input feature map, point q is taken. Each attention head has K initial sampling points for q. The offset Δp is generated by linear operation. mqk , to update the coordinates of K sampling points. The attention weight A at each sampling point is obtained through linear operation mqk , combined with the attention weight A mqk The features of the K sampling points are weighted and summed, and finally M attention heads are superimposed to obtain more critical feature information output through linear operations.
[0071] In some embodiments of the present invention, the PAN-FPN architecture, called dual-stream FPN, is used in the Neck layer. This structure fuses feature maps of different resolutions together and converts them into feature maps of the same size through convolution and upsampling operations. That is, by capturing image information of different scales, the feature expression ability of the model is improved to enhance the performance of the model.
[0072] See also Figure 2, the Neck layer is equipped with multiple connection layers (Concat), upsampling (Upsampe), C2f structure and convolution layer (Conv). The PAN-FPN structure is adopted, which is connected from bottom to top and then from top to bottom. The output of the backbone network is used as the input of the neck network to extract higher-level semantic information from the input features for subsequent target detection tasks. Among them, the Concat connection layer is used to connect feature maps from multiple sources according to the channel dimension, which helps to fuse feature information at different levels and improve the feature expression ability of the model; upsampling increases the spatial resolution of the feature map and restores the low-resolution feature map to the specified image size, which is convenient for fusion with high-resolution images; the C2f module and the convolution layer perform convolution operations on the feature map to extract and process the features. In summary, the design of the Neck layer aims to process and extract higher-level representations of input features through connection layers, upsampling and convolution layers. The output feature maps after DAttention feature extraction are used as the input of the connection layer 21 and upsampling layer 11 in the neck network; upsampling layer 11 restores the low-resolution feature map to the specified image size, and splices the features output from C2f6 of the backbone network layer with the high-resolution feature map through the connection layer, and the spliced features are processed by C2f13, and the output results of C2f13 are processed by upsampling layer 14, connection layer 15 and C2f16 in turn; the output structure of C2f16 is passed to SEAM-Detect and convolution layer 17 at the same time, and the connection layer 18 connects the structure of convolution layer 17 and C2f13 and processes them through C2f19, and the output results of C2f19 are passed to SEAM-Detect and convolution layer 20 respectively, and the connection layer 21 splices the output feature map after DAttention feature extraction and the output result of convolution layer 20, and processes them through C2f22, and C2f22 outputs the extracted and processed features to SEAM-Detect.
[0073] In some embodiments of the present invention, a Decoupled Head is used in the Head layer to separate the regression branch and the prediction branch, that is, they are implemented by two convolutional layers respectively. This structure makes the classification and detection tasks more independent. At the same time, the loss function of the Head layer during the training process uses classification loss and bounding box regression loss. By reasonably allocating weights and considering multiple factors, the performance and robustness of the model are improved.
[0074] Specifically, the process of using Decoupled Head in the Head layer to separate the regression branch and the prediction branch is as follows: the Head layer of YOLOv8 uses convolution kernels of different sizes to capture features of different scales. The goal of these convolution layers is to map the information in the feature map to the output space. After that, the feature map is fully connected and convolved, and the activation function is used to predict the category probability of each bounding box (the confidence of each category), the location coordinates of the bounding box (such as the center coordinates, width and height), and the confidence of the target's existence, ultimately achieving accurate recognition and positioning of the target.
[0075] As a preferred solution, during the training phase, the loss function of Head compares the model's prediction results and the true label to calculate the difference between the bounding box position and category predicted by the model and the actual target, and then uses the back-propagation algorithm to update the model's weights, so that the model can gradually optimize the accuracy of the prediction.
[0076] In some embodiments of the present invention, an occlusion-aware attention mechanism is added between the Conv and Conv2d modules in the detection head, and the ShapeIoU loss function is used to replace the original coordinate loss function to obtain an improved detection head SEAM-Detect; the output feature map of C2f in the backbone network layer is input into SEAM-Detect for classification and recognition, and finally an image result with a tower base detection frame and detection accuracy is obtained.
[0077] The ShapeIoU loss function is used to replace the original coordinate loss function CIoU as the training function of the entire model to more accurately measure the difference between the prediction results of the target detection model and the actual situation, thereby helping to optimize the model training process.
[0078] Specifically, Figure 5 The figure below shows the calculation formula of ShapeIoU. ShapeIoU measures the overlap between the model-predicted target and the actual target, and pays attention to the shape and size of the bounding box itself to calculate the loss, thereby achieving accurate matching of the target shape and improving the recognition ability of non-rectangular and irregular shaped target detection tasks. Therefore, it can more accurately measure the difference between the prediction results of the target detection model and the actual situation, thereby helping to optimize the model training process.
[0079] Specifically, the calculation formula of ShapeIoU is:
[0080]
[0081] Among them, scale is the scaling factor. The calculation method of the loss function is adjusted according to the size of the tower base target in the data set. The appropriate scaling factor value helps to better adapt to the tower base area scene and data set photographed by drones and improve the generalization ability of the algorithm. This paper sets the scale to 1.3 by comparing the experimental results.
[0082] Where B is the prediction box, B gt is the real target box, C is the sum of B and B gt The minimum circumscribed matrix of C, where c is the diagonal length of C. w, h, (x, y) and w gt 、h gt 、(x gt ,y gt ) are the width, height and center coordinates of the prior box and the real box, respectively. Scale is the scale factor, which is related to the scale of the target in the data set. W W With H h are the weight coefficients in the horizontal and vertical directions, respectively, and their values are related to the shape of the GT box. θ controls the degree of attention paid to shape loss. In order to avoid paying too much attention to the loss and ignoring the movement of the predicted box, the value of θ is determined to be 4 through the genetic algorithm. The corresponding frame regression loss is as follows:
[0083] L Shape-IoU =1-IoU+D shape +0.5×S shape (6)
[0084] The Shape-IoU loss function is composed of the IoU loss function and the distance loss function D shape And the shape loss function S shape constitute.
[0085] In some embodiments of the present invention, the occlusion-aware attention in YOLO-Face V2 is used to improve Head, and the occlusion-aware attention (SEAM) in YOLO-Face V2 is used to improve the Detect in Yolov8, and the occlusion-aware attention, i.e., SEAM, is added between Conv and Conv2d in the Detect module. Under the premise of ensuring the detection accuracy, the model is lightweight so that it can effectively handle the occlusion scene. The first part of the occlusion-aware attention is to set a plurality of depth-separable convolutions with different pixel sizes. In each depth-separable convolution module, the feature map is first subjected to a depth convolution with a residual network structure and then a point-by-point convolution operation is performed. Then, the output feature map of each point-by-point convolution module is fused with the initial input of the occlusion-aware attention module, and the fused feature map is subjected to global average pooling and full connection to generate attention weights. After the attention weights are processed by an exponential function, an attention map of the occluded area is obtained, and after multiplying the attention feature map of the occluded area with the feature map output by Conv, a feature map that pays more attention to the occluded area is output.
[0086] Specifically, we use the occlusion-aware attention (SEAM) in YOLO-Face V2 to improve Head, such as Figure 6 The figure shows the structure of the occlusion-aware attention network module. The SEAM network module is a lightweight convolutional network. It generates an attention map of the occluded area by calculating the correlation between the feature map and the occluded area to enhance the responsiveness of the occluded tower base in the output of the Neck layer. The CSMM uses patches of different pixel sizes to learn multi-scale features from local to global. The first part is the depth-wise separable convolution with residual connection. The depth-wise separable convolution realizes channel-by-channel separation convolution through depth-by-depth operations. Although it can learn the importance of different channels and reduce the number of parameters, it ignores the information relationship between channels. To compensate for this loss, the feature outputs of convolutions of different depths are combined through point-by-point (1x1) convolutions. Then, for the two-dimensional tensor output after global average pooling, a two-layer fully connected network (Fc) is used to realize cross-channel global information fusion, strengthen the connection between all channels, learn complex inter-channel relationships, and generate attention weights from global features. The attention weight vector output by the fully connected layer is processed by an exponential function (Channel exp) to expand the value range from [0, 1] to [1, e]. This exponential normalization provides a monotonic mapping relationship, making the result more tolerant to position errors. Finally, the attention map of the occluded area generated by the SEAM module is multiplied with the original feature map, so that the model can effectively handle the tower base occlusion.
[0087] S23, dividing the data set according to a certain ratio and inputting it into the network model for training and prediction;
[0088] S231: Use the python file to convert the annotation file in XML format into a TXT text file as a whole, and divide the data set into a training set, a validation set and a test set according to the ratio of 8:1:1 for the above original image and the corresponding annotation file.
[0089] S232: The divided data set is then input into the target detection network model for training. After the training, the best training weight file is obtained for subsequent predictions.
[0090] S24, through experimental data and test results, the effectiveness of the network model is compared, and the improved effective model is used to predict and output the tower base.
[0091] The second aspect of the present invention discloses a tower base target detection device based on an improved YOLOv8, comprising:
[0092] Based on the improved YOLOv8 tower base target detection device, including:
[0093] A collection module is used to obtain the power grid tower base images collected by the drone;
[0094] A detection module is used to detect the collected image through the tower base target detection network model of YOLOv8, and obtain and mark the tower base in the image;
[0095] The network model of the tower base target detection of YOLOv8 includes an input end, a backbone network, a neck network and a detection head; the collected image is input into the input end, the output end of the input end is connected to the input end of the backbone network, the output end of the backbone network is connected to the input end of the neck network, the output end of the neck network is connected to the output end of the detection head, and the output end of the detection head outputs an image marked with the tower base;
[0096] C2f is set in both the backbone network and the neck network, Dattention is added after the SPPF in the backbone network, and the output end of Dattention is connected with Concat and Upsample in the neck network;
[0097] The detection head is provided with an occlusion-aware attention; each C2f output terminal in the neck network is connected to an occlusion-aware attention, and the attention maps output by all occlusion-aware attention connections are multiplied and added with the feature maps output by the C2f, and an image marked with the tower base is output;
[0098] Output module, used to output images marked with tower bases.
[0099] The following is further described in conjunction with specific embodiments:
[0100] like Figures 2 to 8 As shown, the invention is based on the improved YOLOv8 target detection algorithm to identify the tower base in the image, and the specific steps are as follows:
[0101] Step 1: Preprocess the tower base area images of the northern Shaanxi power grid collected by the drone to obtain the basic data set. First, compress the high-resolution images collected by the drone and reduce the image resolution to 640*640. Then, use the Lableme annotation software to manually annotate the tower bases in some pictures and input them into the original YOLOv8 network model as the basic data set for pre-training. Use the trained weight file to filter the pictures collected by the drone, and finally retain the images containing the transmission tower bases as the basic pictures. Then use the Lableme annotation software to correct the label files containing the tower bases obtained by the above screening, so that the annotation box can more appropriately mark the tower base targets, and save the annotation files in XML file format. The 3495 well-annotated pictures are used as the basic experimental data of this article.
[0102] Step 2: Build a network model for tower base target detection based on the improved YOLOv8. The overall structure of the YOLOv8 target detection algorithm includes four parts: input, backbone network, neck network, and detection head.
[0103] Step 3: Divide the data set according to a certain ratio and input it into the network model for training and prediction. Use the python file to convert the XML-formatted annotation file into a TXT text file. The 3495 images after the operation are randomly divided into 2830 training sets, 315 validation sets, and 350 test sets according to the file structure. The original images and the divided annotation files are stored in the dataset folder of the project. Then the divided data set is input into the target detection network model for training. After the training, the best training weight file is obtained for subsequent predictions. Figure 6 The following are some examples of tower base datasets.
[0104] Step 4, compare the effectiveness of the network model through experimental data and test results, and use the improved effective model to predict and output the tower base. Using the control variable method, based on the homemade experimental data set in this study, comparative experiments were carried out under the same experimental environment. The experiment selected mAP, FPS, GFLOPs and Parameters as evaluation indicators of the experimental results. By comparing the experimental results, mAP@50-95 reached 82.3%, which was 1.6% higher than the original YOLOv8 model; GFLOPs and FPS were reduced by 9.9% and 10.3% respectively compared with the original network experimental results, indicating that the improved YOLOv8 algorithm has better and faster detection effect on the tower base.
[0105] S41: Using the control variable method, a comparative experiment was conducted under the same experimental environment based on the experimental data set made in this study. The experimental environment and model parameter settings are shown in Table 1:
[0106] Tab.1 Training environment and parameter settings
[0107]
[0108] S42: The experiment selected mAP, FPS and GFLOPs as the evaluation indicators of the experimental results. FPS is the number of frames processed per second, which is used to evaluate the speed of the model processing images; GFLOPs is the number of floating-point operations per second, which is used to evaluate the model complexity and computational efficiency. The definition of mean average precision mAP is as follows:
[0109]
[0110] Precision (P) refers to the probability that a certain category is correctly predicted in the prediction results; Recall (R) refers to the probability that a certain category is correctly predicted in the actual situation. TP is a true positive example, FP is a false positive example, and FN is a false negative example.
[0111]
[0112] The Precision-Recall (PR) curve of each category is plotted with precision and recall as the horizontal and vertical coordinates. The average precision (AP) of a category can be obtained by integrating the PR curve. The mAP is obtained by averaging the AP values of all N categories, which reflects the comprehensive performance of the model on N categories.
[0113] By comparing the experimental results, mAP@50-95 reached 82.3%, which is 1.6% higher than the original YOLOv8 model; GFLOPs and FPS were reduced by 9.9% and 10.3% respectively compared with the original network experimental results, indicating that the improved YOLOv8 algorithm has better and faster detection effect on the tower base. This study adds the DAttention mechanism to the Backbone layer, then uses ShapeIoU to replace the original coordinate loss function CIoU, and uses the occlusion-aware attention in YOLO-Face V2 to improve the Head to optimize and improve the original YOLOv8 model. In order to verify the contribution of these three improvements to the detection of tower base targets, ablation experiments were performed for comparison. The experimental comparison results are shown in Table 2.
[0114] Tab.2Ablation experiment
[0115]
[0116] S43: Use the improved YOLOv8 target detection network model to identify and output the tower base from the tower base area images and videos collected by the drone to be detected.
[0117] In order to compare the prediction results more intuitively, the present invention selects images with complex backgrounds for prediction, see Figure 7 and Figure 8 , which is a comparison chart of the detection performance of the original YOLOv8 model and the algorithm in this paper on some selected images. Figure 7 The detection result image of the original YOLOv8 model is shown. Figure 8 The detection result image of the algorithm in this paper is shown. It is observed that when the tower base is incomplete (the third figure), the original YOLOv8 model misses detection, while the improved model can avoid missed detection and false detection in the environment of incomplete tower base, noisy background and disturbed tower top, and can accurately detect tower base targets. In addition, as shown in the figure, the confidence score predicted by the algorithm of the present invention is usually higher than that of the original YOLOv8, which shows that the improved network model has better overall performance.
[0118] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. Based on the improved YOLOv8 tower base target detection method, it is characterized by: The following steps are involved: S1, obtain the power grid tower base image collected by the drone; S2, detect the collected image through the improved YOLOv8 tower base target detection network model, obtain and mark the tower base in the image; The network model of tower base target detection of YOLOv8 includes an input end, a backbone network, a neck network and a detection head; the collected image is input into the input end, the input end pre-processes the input image and then passes it into the backbone network, the backbone network extracts features from the image to obtain a feature map, the feature maps of different levels are fused through the neck network, and the fused feature maps are output to the detection head for detection and classification, and finally the detection head outputs the tower base target detection result; A DAttention mechanism is added after the SPPF of the backbone network, and the output feature map after DAttention feature extraction is used as the input of Concat and UpSample in the neck network; An occlusion-aware attention mechanism is added between the Conv and Conv2d modules in the detection head, and the loss function in the detection head is the ShapeIoU loss function; S3, outputs an image labeled with the tower base.
2. The improved YOLOv8 tower base target detection method according to claim 1, characterized in that: In S2, Mosaic data enhancement is provided in the input end.
3. The improved YOLOv8 tower base target detection method according to claim 1, characterized in that: In S2, the C3 positions in the backbone network are all C2f structures.
4. The improved YOLOv8 tower base target detection method according to claim 1, characterized in that: In S2, the neck network is a dual-stream FPN architecture.
5. The improved YOLOv8 tower base target detection method according to claim 1, characterized in that: In S2, the detection head is provided with two detection heads, and the two detection heads include separate regression branches and prediction branches.
6. The improved YOLOv8 tower base target detection method according to claim 1, characterized in that: In S2, M attention heads are set in the Dattention, each of which is set with K initial sampling points. The coordinates of the K sampling points are updated by the offset generated by the linear operation, and the attention weight of each sampling point is obtained by the linear operation. The products of the K sampling points and the corresponding attention weights are summed, and the summed results of the M attention heads are superimposed to obtain the feature information.
7. The improved YOLOv8 tower base target detection method according to claim 1, characterized in that: In S2, the occlusion-aware attention is provided with a plurality of depth-wise separable convolutions. In each depth-wise separable convolution module, a point-by-point convolution operation is first performed on the feature map using a depth-wise convolution with a residual network structure. Subsequently, the output feature map of each point-by-point convolution module is fused with the initial input of the occlusion-aware attention module. Attention weights are generated for the fused feature map through global average pooling and full connection. After the attention weights are processed through an exponential function, an attention map of the occluded area is obtained. After multiplying the attention map of the occluded area with the feature map output by Conv, a feature map that pays more attention to the occluded area is obtained.
8. The improved YOLOv8 tower base target detection method according to claim 1, characterized in that: In S2, the network model of the tower base target detection of YOLOv8 is processed by the basic data set, and the process of obtaining the basic data set is: (1) Collect and compress grid tower base images; (2) Label the tower bases in some images using Labelme, input the labeled images as data sets into the YOLOv8 network model, and filter the remaining power grid tower base images using the pre-processed pre-trained weight files to obtain the filtered power grid tower base images; (3) The screened power grid tower base images are corrected using the Labelme annotation software to obtain the basic data set.
9. The tower base target detection method based on improved YOLOv8 according to claim 1, characterized in that: The network model of tower base target detection of YOLOv8 is obtained by training with ShapeIoU loss function.
10. The tower base target detection device based on the improved YOLOv8 is characterized in that: include: A collection module is used to obtain the power grid tower base images collected by the drone; A detection module is used to detect the collected image through an improved YOLOv8 tower base target detection network model, and obtain and mark the tower base in the image; The network model of the tower base target detection of YOLOv8 includes an input end, a backbone network, a neck network and a detection head; the collected image is input into the input end, the output end of the input end is connected to the input end of the backbone network, the output end of the backbone network is connected to the input end of the neck network, the output end of the neck network is connected to the output end of the detection head, and the output end of the detection head outputs an image marked with the tower base; C2f is set in both the backbone network and the neck network, Dattention is added after the SPPF in the backbone network, and the output end of Dattention is connected with Concat and Upsample in the neck network; The detection head is provided with occlusion perception attention; Each C2f output terminal in the neck network is connected to an occlusion-aware attention connection, and the attention maps output by all occlusion-aware attention connections are multiplied and added with the feature maps output by C2f to output an image marked with the tower base; Output module, used to output images marked with tower bases.
Citation Information
Patent Citations
Lightweight target detection network design method for indoor semantic SLAM system
CN118097365A
Underwater target detection robot based on deep learning
CN118411593A
Cited By
Target detection method under view angle of unmanned aerial vehicle
CN119919639A
A target detection method from the perspective of an unmanned aerial vehicle (UAV)
CN119919639B
Method and system for pitting corrosion detection and residual strength calculation of logging data of multi-arm caliper
CN119942197A
Tower foreign matter detection method and system
CN120236204A
A method and system for detecting foreign matter in a tower
CN120236204B
