Construction method of lightweight and robust unmanned aerial vehicle power transmission line inspection target detection model
The LR-YOLOv8 model addresses UAV inspection challenges by using a lightweight network architecture and enhanced loss function to improve detection accuracy and resource efficiency in power transmission line inspections.
Patent Information
- Application Number
- CN202411269773.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-11
- Publication Date
- 2024-12-13
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing UAV-based power transmission line inspection systems face challenges due to limited computing resources, complex environments, and diverse inspection targets, leading to inefficient and inaccurate target detection.
A lightweight and robust UAV transmission line inspection model (LR-YOLOv8) is constructed using a Ghost network backbone, hardware-friendly attention module, soft threshold strategy, and enhanced loss function, along with HSV and atomization to simulate complex environments, reducing resource consumption and improving detection accuracy.
The model effectively reduces computing resource requirements and enhances detection accuracy in complex environments, minimizing false and missed detections, and adapts to the diverse targets and environmental conditions of power transmission lines.
Smart Images

Figure CN119131365A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of power transmission line inspection, and specifically relates to a method for constructing a lightweight and robust unmanned aerial vehicle power transmission line inspection target detection model. Background Art
[0002] Transmission line systems are very important for modern industry and urban society, transmitting electricity from power stations to residential and industrial load centers. Traditional manual inspections require hiring a large number of personnel to patrol along the lines, resulting in time-consuming and costly inspections.
[0003] With the development of smart grids, unmanned aerial vehicles (UAVs) are widely used in power transmission line inspections. UAV-based inspections have become a major research direction, significantly improving the inspection efficiency. In many cases, transmission lines can span hundreds of kilometers and cross challenging terrains such as deserts, mountains, forests, and lakes. These transmission lines are subject to different weather and climate conditions, including fog, cloudy days, and dusk. In addition, due to the interference of factors such as the electric field in the transmission lines, UAV inspections need to maintain a safe distance. Existing UAV-based inspection systems have difficulty in object recognition in complex environments such as fog and strong winds, and the quality of the collected data is often compromised. Therefore, the inspection environment of transmission lines is often complex, which poses a severe challenge to UAV inspections. On the other hand, the computing resources of most UAVs are very limited. It is a big challenge to process the massive visual image data collected by UAVs with limited computing resources, which hinders the deployment of advanced computer vision algorithms. Therefore, it is crucial to deploy lightweight deep learning models into UAV inspection systems to better improve the efficiency and accuracy of inspections.
[0004] However, there are several knowledge gaps in the literature that hinder the further development of drone inspection technology: (1) Most existing methods focus only on improving the detection accuracy of the model and ignore the computing resources of mobile devices. Complex deep learning models such as YOLO consist of millions of parameters and require a lot of computing resources, while edge devices such as drones have limited computing power due to severe energy constraints. (2) Considering the complex environment of transmission lines, the data collected by drones inevitably contain large amounts of noise. (3) The diversity of inspection targets in inspection tasks; drones must be able to identify transmission line components and external objects such as bird nests, insulators, etc. Summary of the invention
[0005] The technical problem to be solved by the present invention is to provide a method for constructing a lightweight and robust UAV power transmission line inspection target detection model LR-YOLOv8 (Light and Robustness You Only LookOnce v8, LR-YOLOv8) to detect power transmission line targets and foreign objects in view of the deficiencies of the above-mentioned prior art. Specifically, a lightweight network architecture that can effectively reduce model parameters is proposed; a soft threshold strategy is designed to effectively resist sample noise interference; an automatic regulation network (Automatic regulation network, ARN) is designed to avoid manually setting thresholds; augmentation Hue, Saturation, and Value (Augmentation Hue, Saturation, and Value, HSV) and atomization are used to simulate power transmission lines in complex environments; a loss function based on multi-scale domain adaptation is proposed to effectively improve the mean average precision (mAP). Through a lightweight and robust network backbone, the model weights can be effectively reduced and the robustness of the model can be improved. Compared with other algorithms, the present invention can effectively reduce the computing resource consumption of drones and lower the hardware threshold. At the same time, it can effectively reduce the frequency of false detection and missed detection in model recognition during inspections in complex environments, solve the problem of poor robustness of traditional models, and better adapt to the needs of drones in power transmission line inspections.
[0006] In order to achieve the above technical objectives, the technical solution adopted by the present invention is:
[0007] A method for constructing a lightweight and robust UAV transmission line inspection target detection model comprises the following steps:
[0008] Step 1: Using Yolov8 as the network basis of the target detection model, reconstruct the backbone network of Yolov8 for feature extraction: referring to the Ghost network, introduce the cheap convolution in Ghost to obtain the feature extraction network Ghost module of the lightweight target detection model to reduce the model edge computing resource consumption;
[0009] Step 2: Based on the Ghost module, a hardware-friendly attention module DFC is introduced to build the spatial information capture network Ghostv2 of the target detection model to reduce the loss of edge features caused by cheap convolution; and a soft threshold strategy and threshold automatic adjustment network ARN are designed and combined with Ghostv2 to obtain the LR-C2f module to optimize the C2f module in Yolov8;
[0010] Step 3: Introduce power exponential transformation and SIoU loss function, establish an enhanced loss function for the model, improve the recognition accuracy of multi-scale object detection, and obtain a higher mAP;
[0011] Step 4: Introduce the HSV module and the atomization module, simulate the transmission lines in complex environments through HSV and atomization, realize the enhanced processing of the data set of the model input, and finally obtain a lightweight and robust UAV transmission line inspection target detection model based on data enhancement that conforms to the real UAV transmission line inspection environment.
[0012] To optimize the above technical solutions, the specific measures taken also include:
[0013] The above step 1 generates multiple intrinsic feature maps by using filters, further applies the cheap convolution conversion operation to generate feature maps, realizes Yolov8 backbone network reconstruction, and obtains the lightweight target detection model network Ghostmodule;
[0014] The theoretical speedup ratio of the feature extraction network Ghost module of the lightweight target detection model and the backbone network used for feature extraction by Yolov8 is:
[0015]
[0016]
[0017] Where h*w*c is the size of the input feature map of the feature extraction network of the lightweight target detection model, h′*w′*c′ is the size of the output feature map of the feature extraction network of the lightweight target detection model, k*k is the size of the network convolution kernel, d*d is the kernel size of the linear operation, and n is the number of output channels;
[0018] s represents the number of conversion operations of cheap convolution, where s>>c.
[0019] In the spatial information capture network Ghostv2 of the target detection model constructed in step 2 above, the input V is fed to the Gaussian branch and the DFC attention branch. In the Gaussian branch, the feature extraction network Ghost module of the lightweight target detection model obtained in step 1 is convolved to generate the output feature Conv1*1(V). In the DFC attention branch, the attention map A is generated by the attention module DFC, and the scaling function Sigmoid is used to obtain Sigmoid(A).
[0020] The final output O is the result of element-by-element multiplication of the two branch outputs, as shown in the formula:
[0021] O=Sigmoid(A)⊙Conv1*1(V)
[0022] The soft threshold strategy of step 2 above is as follows:
[0023]
[0024] Where X represents the input feature, Y represents the output feature, and the threshold τ is a positive number.
[0025] The above threshold automatic adjustment network ARN includes a GAP layer and two FC networks. The threshold automatic adjustment method is as follows:
[0026] The absolute value of the input feature X is used to ensure that the subsequent feature mapping will not be affected by the positive or negative sign of the feature value. The GAP layer averages all pixel values in each channel of the input feature X to reduce the spatial dimension while retaining the statistical information of each channel. The two are combined for feature mapping and simplified into a one-dimensional vector, which is then propagated to the two-layer FC network;
[0027] The second layer of the FC network has multiple neurons, and the number of neurons is equal to the number of channels of the input feature map;
[0028] The output of the FC network is scaled in the range (0, 1) as follows:
[0029]
[0030] Where z c is the feature of the c-th layer neuron, α c is the c-th layer scaling parameter;
[0031] The threshold automatic adjustment formula is:
[0032]
[0033] where τ c is the threshold of the c-th layer channel of the feature map, i, j, c are the width, height and channel index of the feature map respectively.
[0034] The enhanced loss function established in step 3 above is:
[0035]
[0036] Where IoU represents the intersection over union ratio between the real box and the predicted box, α represents the power parameter, 6 is the distance cost, and Ω is the shape cost.
[0037] The HSV module in step 4 above simulates the transmission line in a complex environment through HSV, including:
[0038] Step 4-1-1: Normalize the values of R, G, and B to obtain R', G', and B', as shown below:
[0039] R′, G′, B′=R, G, B / 255
[0040] Step 4-1-2: Calculate the difference between the maximum and minimum values of R′, G′ and B′, recorded as As shown below:
[0041]
[0042] Step 4-1-3: Calculate the color gamut values H, S, V corresponding to the enhanced hue, saturation and value, and obtain the corresponding HSV image:
[0043]
[0044]
[0045] V=max(R', G', B').
[0046] The atomization module in step 4 above simulates the power transmission line in a complex environment through atomization, including:
[0047] Step 4-2-1: Set the fog size S and thickness fog coefficient D, and determine the transmittance t(x), as shown below:
[0048]
[0049] Where -0.04 is the selected fixed parameter, (w' c , h' c ) is the image center, and the image center is the fog center;
[0050] Step 4-2-2: According to the simulated fogging effect, the original fog-free image is processed to obtain a fogged image, as shown in the formula:
[0051] I(x)=J(x)t(x)+L(1-t(x))
[0052] Where J(x) is the original fog-free image, L is the brightness of the atmospheric light component, and L ranges from 0 to 1, indicating the color of the added fog, that is, the grayscale value. When L is set to 0 and 1, pure white fog and black fog are added, respectively.
[0053] The present invention has the following beneficial effects:
[0054] Compared with the traditional deep learning target detection model for UAV inspection of transmission lines, this paper proposes a lightweight and robust transmission line detection model LR-YOLOv8. To this end, a new lightweight module LR-C2f module is proposed, which can effectively reduce the model weight and reduce the consumption of edge computing resources: a robust network backbone Ghost module is constructed. Considering that the transmission lines are in a complex field environment for a long time, the hardware-friendly attention module DFC and soft threshold strategy and ARN network are used to effectively reduce the impact of weather, lighting, and image scale changes.
[0055] The present invention also constructs a new loss function to reduce the impact of the cross-scale domain of the image and improve the target recognition accuracy, thereby realizing the inspection of the UAV in the power transmission line.
[0056] The present invention takes into account the complex environment and the large span of the scale domain of the target, enhances the loss function and adjusts the corresponding parameter weights to obtain a higher mAP.
[0057] The lightweight network for LR-YOLOv8 power transmission line detection proposed in the present invention reduces the parameters and weights required by the model, thereby reducing the resource requirements of the mobile terminal.
[0058] The soft threshold strategy designed in the present invention is used to reduce noise interference caused by complex environments, and an automatic adjustment network for automatically adjusting the threshold is designed.
[0059] When the present invention enhances the data set input by the model, it addresses the problem that traditional atomization can only add fog of a single color. It takes into account the color of the fog caused by the influence of light when facing real scenes, and enhances the hue, saturation and value and atomization processing to simulate the complex environment in the natural environment. It can verify the robustness of the model and evaluate the results from various complex environmental perspectives. BRIEF DESCRIPTION OF THE DRAWINGS
[0060] Figure 1 The lightweight and robust UAV power transmission line inspection target detection model of the present invention;
[0061] Figure 2 This is a schematic diagram of the UAV inspecting the power transmission line of the present invention;
[0062] Figure 3 This is a schematic diagram of the spatial information capture network of the present invention;
[0063] Figure 4 A schematic diagram of the structure of the network designed for the present invention;
[0064] in Figure 4 Part (a)(b)(c) of Figure 4The disassembled part of (d), (d) is the core module LR-C2f module proposed by the present invention. DETAILED DESCRIPTION
[0065] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. This will enable the application to fully understand and implement the implementation process of how to use technical means to solve technical problems and achieve technical effects. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.
[0066] like Figure 1-3 As shown, a method for constructing a lightweight and robust UAV power transmission line inspection target detection model of the present invention comprises the following steps:
[0067] Step 1: Using Yolov8 as the network basis of the target detection model, reconstruct the backbone network of Yolov8 for feature extraction: referring to the Ghost network, introduce the cheap convolution in Ghost to obtain the feature extraction network Ghost module of the lightweight target detection model to reduce the model edge computing resource consumption;
[0068] In the embodiment, a lightweight target detection network backbone is constructed, and the constructed model is used to reduce the parameters and weights required for the model and reduce the consumption of edge computing resources; by modifying the backbone network, the model weight can be effectively reduced while ensuring the robustness of the model;
[0069] Step 2: Based on the Ghost module, a hardware-friendly attention module DFC is introduced to build the spatial information capture network Ghostv2 of the target detection model to reduce the loss of edge features caused by cheap convolution; and a soft threshold strategy and threshold automatic adjustment network ARN are designed and combined with Ghostv2 to obtain the LR-C2f module to optimize the C2f module in Yolov8;
[0070] In the embodiment, a spatial information capture network of a target detection model is constructed. On the basis of cheap convolution, a hardware-friendly attention module DFC is introduced to effectively reduce the loss of edge features caused by cheap convolution, and a soft threshold strategy and a threshold automatic adjustment network ARN are designed to improve the robustness of the model.
[0071] Build a robust target detection network backbone. The constructed model is used to detect components such as insulators and foreign objects such as bird nests in complex environments of power transmission lines.
[0072] Step 3: Introduce power exponential transformation and SIoU loss function, establish an enhanced loss function for the model, improve the recognition accuracy of multi-scale object detection, and obtain a higher mAP;
[0073] In the embodiment, according to the complex environment and cross-scale domain detection of the target, the model loss function is enhanced to obtain a higher mAP;
[0074] Step 4: Introduce the HSV module and the atomization module. Use HSV and atomization to simulate the transmission lines in complex environments to achieve enhanced processing of the data set of the pattern input. The model detection results can be evaluated from the perspective of various complex environments. Finally, a lightweight and robust UAV transmission line inspection target detection model based on data enhancement that conforms to the actual UAV transmission line inspection environment is obtained.
[0075] In the embodiment, in order to verify the robustness of the model, an evaluation scheme is established that can evaluate the results from various complex environmental perspectives;
[0076] The specific embodiments are as follows:
[0077] The step 1 constructs a lightweight model for UAV inspection of power transmission lines. The model is used to describe the mobile terminal and edge computing resource constraints that meet the power transmission line inspection. The construction of the lightweight model specifically includes the following steps:
[0078] Step 1-1: Based on the limitations of mobile terminals and edge computing resources in the UAV transmission line inspection, reconstruct the backbone of the target detection model network and use lightweight modules;
[0079] Step 1-2: Design the backbone network module of the lightweight model. Use fewer filters to generate multiple intrinsic feature maps, and further apply cheap convolution transformation operations to efficiently generate feature maps. Compare the calculation parameters of traditional convolution and lightweight modules, as shown in the formula:
[0080]
[0081] In the formula, r s The theoretical speedup ratio of ordinary convolution is upgraded by using lightweight modules.
[0082] The size of the input feature map of the feature extraction network of the lightweight target detection model is h*w*c, the size of the output feature map of the feature extraction network of the lightweight target detection model is h′*w′*c′, the size of the convolution kernel is k*k, and the kernel size of the linear operation is d*d, which is similar to the size of k*k, and n is the number of output channels.
[0083] s represents the number of conversion operations of cheap convolution, where s>>c.
[0084] When the same size of d*d and k*k is used, since s>>c, the computational complexity and parameter size of the lightweight module can be approximated to that of the traditional module.
[0085] In this embodiment, the present invention lists lightweight model formulas based on the limited computing resources and other problems encountered in the overall UAV inspection of power transmission lines. By reconstructing the network model backbone, the efficiency of UAV inspection is improved while ensuring mAP, and the threshold of UAV computing resources is lowered, which can better adapt to the needs of the real environment of UAV inspection of power transmission lines.
[0086] The step 2 constructs a robustness model for UAV inspection of power transmission lines. The model is used to describe the complex environmental constraints in power transmission line inspection. The construction of the robustness model specifically includes the following steps:
[0087] Step 2-1: Design a network that improves the ability to capture spatial information and adds a decoupled fully connected (DFC) attention mechanism.
[0088] like Figure 4 As shown in (a), the input V is fed to the Gaussian branch and the DFC attention branch. In the Gaussian branch, the feature extraction network Ghost module of the lightweight target detection model obtained in step 1 is convolved to generate the output feature Conv1*1(V). In the DFC attention branch, the output feature Conv1*1(V) is generated. Figure 4 The attention map A generated by the attention module DFC shown in (b) is obtained by scaling the function Sigmoid to obtain Sigmoid(A);
[0089] The final output O is the result of element-by-element multiplication of the two branch outputs, as shown in the formula:
[0090] O=Sigmoid(A)⊙Conv1*1(V)
[0091] Where is element-by-element multiplication, Sigmoid is the scaling function that normalizes the attention map A to the range of (0, 1);
[0092] Step 2-2: Considering the necessity of noise interference in complex environments, a soft threshold strategy is adopted to set the features whose absolute values are lower than a certain threshold τ to zero, and other features are also adjusted to zero, as shown in the formula:
[0093]
[0094] Where X represents the input feature, Y represents the output feature, and the setting of τ must meet two conditions: τ is a positive number, and τ cannot be greater than the maximum value of the input, otherwise, the output will be zero.
[0095] Since τ is different for different data sets, such as Figure 4 As shown in (d), the design Figure 4The ARN network shown in (c) avoids manually setting τ and prevents the output from being all zero. Using the absolute value of the input feature X (to ensure that the subsequent feature map will not be affected by the sign of the feature value), the global average pooling (GAP) layer averages all pixel values in each channel of the input feature X to reduce the spatial dimension while retaining the statistical information of each channel. The two are combined for feature mapping and simplified into a one-dimensional vector, which is then propagated to a two-layer fully connected (FC) network. The second layer of the FC network has multiple neurons, and the number of neurons is equal to the number of channels of the input feature map. The output of the FC network is scaled in the range of (0,1), as shown in the formula:
[0096]
[0097] Where z c is the feature of the c-th layer neuron, α c is the c-th layer scaling parameter. The threshold calculation formula is shown as follows:
[0098]
[0099] where τ c is the threshold of the c-th layer channel of the feature map, i, j, c are the width, height and channel index of the feature map respectively.
[0100] In this embodiment, the present invention lists the spatial information capture network model formula based on the environmental noise and other problems encountered in the overall drone inspection of power transmission lines. By using a soft threshold strategy and designing ARN to avoid manually setting thresholds, the efficiency of drone inspections is improved, the robustness of the model of drone inspection of power transmission lines is enhanced, and it can better adapt to the needs of the real environment of drone inspection of power transmission lines.
[0101] In summary, if Figure 4 As shown in (a) and (b), through steps 1-2, a Ghostv2 network based on Ghost module convolution and attention module DFC is obtained; Figure 4 As shown in (d), the Ghostv2 network (corresponding to Figure 4 (d) GhostBottlenneck) with soft threshold strategy and Figure 4 The network ARN shown in (c) is combined with to obtain the LR-C2f module, which is used to optimize the C2f module in Yolov8.
[0102] In step 3, the model loss function is enhanced to obtain a higher mAP, which specifically includes the following steps:
[0103] Step 3-1: Construct a new loss function to reduce the impact of the cross-scale domain of the image and improve the accuracy of target recognition;
[0104] Step 3-2: Introduce power exponential transformation and SIoU loss function to achieve higher mAP and robustness, as shown in the formula:
[0105]
[0106] In the formula, IoU represents the intersection-over-union ratio of the real box and the predicted box, α represents the power parameter, δ is the distance cost, and Ω is the shape cost. α is the power exponential change parameter, and the remaining part is SloU.
[0107] In this embodiment, the present invention lists an enhanced loss function formula based on the complex environment and cross-scale domain changes encountered in the overall UAV inspection of power transmission lines. By introducing power exponential transformation and SloU loss function, the accuracy of UAV inspection is improved, the robustness of the model of UAV inspection of power transmission lines is enhanced, and it can better adapt to the needs of the real environment of UAV inspection of power transmission lines.
[0108] In step 4, the HSV module and the atomization module are introduced to simulate the transmission lines in a complex environment through HSV and atomization to achieve the following Figure 2 The data set enhancement processing shown can evaluate the evaluation scheme of the results from various complex environmental perspectives, and obtain the robust model LR-Yolov8 based on data enhancement that conforms to the real inspection environment of UAV power transmission lines. Specifically, it includes the following steps:
[0109] Step 4-1: Considering the light changes during field inspection of power transmission lines, such as early morning, noon, and dusk, an HSV spatial channel that directly expresses the brightness, hue, and vividness of colors is constructed, and the data set is preprocessed. Specifically, the steps include:
[0110] Step 4-1-1: Normalize the values of R, G, and B to obtain R', G', and B', as shown in the formula:
[0111] R′, G′, B′=R, G, B / 255
[0112] Step 4-1-2: Calculate the difference between the maximum and minimum values of R′, G′ and B′, recorded as As shown in the formula:
[0113]
[0114] Step 4-1-3: Calculate the color gamut values corresponding to H, S, and V, and obtain the corresponding HSV image, that is, the power transmission line diagram in the complex environment simulated by HSV:
[0115]
[0116]
[0117] V = max(R′, G′, B′)
[0118] The HSV model is described by color, depth, and brightness. Compared with the RGB model, HSV is more consistent with the real environment of the transmission line, such as dusk, cloudy, foggy and other weather.
[0119] Step 4-2: Considering the weather changes during field inspection of power transmission lines, such as foggy and cloudy days, an algorithm module is designed to simulate the formation principle of fog, process and synthesize images, and preprocess the data set, which includes the following steps:
[0120] Step 4-2-1: Set the atomization size S and thickness fog coefficient D, and determine the transmittance t(x), as shown in the formula:
[0121]
[0122] Where -0.04 is the selected fixed parameter. (w' c , h' c ) is the image center, and the image center is selected as the fog center.
[0123] Step 4-2-2: Process the image according to the simulated fogging effect, as shown in the formula:
[0124] I(x)=J(x)t(x)+L(1-t(x))
[0125] Where J(x) is the original fog-free image, and L is the brightness of the atmospheric light component. The brightness L ranges from 0 to 1, indicating the color of the added fog, that is, the grayscale value. When L is set to 0 and 1, pure white fog and black fog are added respectively.
[0126] In the actual inspection environment of power transmission lines, foggy days will reduce visibility and affect drone aerial photography, thus reducing inspection efficiency. Therefore, data set enhancement processing based on fog simulation is necessary.
[0127] This embodiment proposes a lightweight network architecture that can effectively reduce model parameters; designs a soft threshold strategy that can effectively resist sample noise interference; and designs an automatic adjustment network to avoid manually setting thresholds; HSV and atomization are used to simulate transmission lines in complex environments; and a loss function based on multi-scale domain adaptation is proposed to effectively improve the average precision. This method has been verified on multiple datasets. The datasets and verification results are shown in Tables 1 and 2. Compared with Yolov8n, the weight is reduced by 25.4%, the FPS is increased by 21.4%, and the mAP is increased by 96.7%.
[0128] Table 1 TL dataset captured by drone aerial photography
[0129] category Image size quantity Insulator 1 1152*864 600 Insulator 2 1920*1080 474 Shock-proof hammer 1920*1080 1441 Pressure equalizing ring 1920*1080 634 Bird's Nest 1920*1080 1085
[0130] Table 2 Ablation test results
[0131]
[0132] Those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a program, so the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.
[0133] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.
Claims
1. A method for constructing a lightweight and robust UAV transmission line inspection target detection model, characterized in that: The steps include: Step 1: Taking Yolov8 as the network basis of the target detection model, reconstruct the backbone network of Yolov8 for feature extraction: referring to the Ghost network, introduce the cheap convolution in Ghost to obtain the feature extraction network Ghostmodule of the lightweight target detection model to reduce the model edge computing resource consumption; Step 2: Based on the Ghost module, a hardware-friendly attention module DFC is introduced to build the spatial information capture network Ghostv2 of the target detection model to reduce the loss of edge features caused by cheap convolution; And design a soft threshold strategy and threshold automatic adjustment network ARN, combined with Ghostv2, to obtain the LR-C2f module to optimize the C2f module in Yolov8; Step 3: Introduce power exponential transformation and SIoU loss function, establish an enhanced loss function for the model, improve the recognition accuracy of multi-scale object detection, and obtain a higher mAP; Step 4: Introduce the HSV module and the atomization module, simulate the transmission lines in complex environments through HSV and atomization, realize the enhanced processing of the data set of the model input, and finally obtain a lightweight and robust UAV transmission line inspection target detection model based on data enhancement that conforms to the real UAV transmission line inspection environment.
2. According to claim 1, a method for constructing a lightweight and robust UAV power transmission line inspection target detection model is characterized in that: The step 1 generates multiple intrinsic feature maps by using filters, further applies the cheap convolution conversion operation, generates feature maps, realizes Yolov8 backbone network reconstruction, and obtains a lightweight target detection model network Ghost module; The theoretical speedup ratio of the feature extraction network Ghost module of the lightweight target detection model and the backbone network used for feature extraction by Yolov8 is: Where h*w*c is the size of the input feature map of the feature extraction network of the lightweight target detection model, h′*w′*c′ is the size of the output feature map of the feature extraction network of the lightweight target detection model, k*k is the size of the network convolution kernel, d*d is the kernel size of the linear operation, n is the number of output channels; s represents the number of conversion operations of the cheap convolution, where s>>c.
3. The method for constructing a lightweight and robust UAV power transmission line inspection target detection model according to claim 1 is characterized in that: In the spatial information capture network Ghostv2 of the target detection model constructed in step 2, the input V is fed to the Gaussian branch and the DFC attention branch. In the Gaussian branch, the feature extraction network Ghost module of the lightweight target detection model obtained in step 1 is convolved to generate the output feature Conv 1*1 (V), the attention map A generated by the attention module DFC in the DFC attention branch, and the scaling function Sigmoid is used to obtain Sigmoid(A); The final output O is the result of element-by-element multiplication of the two branch outputs, as shown in the formula: O=Sigmoid(A)⊙Conv 1*1 (V) 4. The method for constructing a lightweight and robust UAV power transmission line inspection target detection model according to claim 1 is characterized in that: The soft threshold strategy of step 2 is as follows: Where X represents the input feature, Y represents the output feature, and the threshold τ is a positive number.
5. The method for constructing a lightweight and robust UAV power transmission line inspection target detection model according to claim 1 is characterized in that: The threshold automatic adjustment network ARN includes a GAP layer and two layers of FC networks, and the threshold automatic adjustment method is as follows: The absolute value of the input feature X is used to ensure that the subsequent feature mapping will not be affected by the positive or negative sign of the feature value. The GAP layer averages all pixel values in each channel of the input feature X to reduce the spatial dimension while retaining the statistical information of each channel. The two are combined for feature mapping and simplified into a one-dimensional vector, which is then propagated to the two-layer FC network. The second layer of the FC network has multiple neurons, and the number of neurons is equal to the number of channels of the input feature map; The output of the FC network is scaled in the range (0,1) as follows: Where z c is the feature of the c-th layer neuron, α c is the c-th layer scaling parameter; The threshold automatic adjustment formula is: where τ c is the threshold of the c-th layer channel of the feature map, i, j, c are the width, height and channel index of the feature map respectively.
6. The method for constructing a lightweight and robust UAV power transmission line inspection target detection model according to claim 1, characterized in that: The enhanced loss function established in step 3 is: In the formula, IoU represents the intersection-over-union ratio of the real box and the predicted box; α represents the power parameter, i.e., the power exponential variation parameter; δ is the distance cost and Ω is the shape cost.
7. The method for constructing a lightweight and robust UAV power transmission line inspection target detection model according to claim 1 is characterized in that: The HSV module in step 4 simulates the transmission line in a complex environment through HSV, and the process includes: Step 4-1-1: Normalize the values of R, G, and B to obtain R′, G′, and B′ as shown below: R′,G′,B′=R,G,B / 255 Step 4-1-2: Calculate the difference between the maximum and minimum values of R′, G′ and B′, recorded as As shown below: Step 4-1-3: Calculate the color gamut values H, S, V corresponding to the enhanced hue, saturation and value, and obtain the corresponding HSV image, that is, the power transmission line diagram in the complex environment simulated by HSV: V=max(R′,G′,B′) 8. The method for constructing a lightweight and robust UAV power transmission line inspection target detection model according to claim 1, characterized in that: The atomization module in step 4 simulates the power transmission line in a complex environment by atomization, and the process includes: Step 4-2-1: Set the fog size S and thickness fog coefficient D, and determine the transmittance t(x), as shown below: Where -0.04 is the selected fixed parameter, (w' c , h' c ) is the image center, and the image center is the fog center; Step 4-2-2: According to the simulated fogging effect, the original fog-free image is processed to obtain a fogged image, that is, a power transmission line diagram in a complex environment of fogging simulation, as shown in the formula: I(x)=J(x)t(x)+L(1-t(x)) Where J(x) is the original fog-free image, L is the brightness of the atmospheric light component, and L ranges from 0 to 1, indicating the color of the added fog, that is, the grayscale value. When L is set to 0 and 1, pure white fog and black fog are added, respectively.
Citation Information
Patent Citations
Remote sensing image de-fogging method based on dark channel prior model
CN106530257A
Color gamut conversion method, color gamut converter and displaying device
CN108810507A
Multi-modal emotion recognition method based on deep residual shrinkage network
CN114758676A
Lightweight texture surface defect detection method and system based on deep learning
CN116777842A
Fish abnormal state detection method and system based on improved Yolov5s
CN117197553A
