Improved yolov5-based auxiliary photographing method and device for distribution network line unmanned aerial vehicle inspection

Abstract

This invention provides a method for assisting UAV inspection and photography of power distribution lines based on an improved YOLOv5 network. First, the UAV is controlled to perform a preliminary inspection of the power distribution lines along a desired flight path and records the preliminary inspection images. Then, based on the preliminary inspection images and the improved YOLOv5 network, the actual flight path of the UAV is determined. The improved YOLOv5 network includes at least two detection heads and a feature comparison layer. The feature comparison layer compares the feature differences identified by each detection head; these differences represent image recognition errors caused by flight path deviations. Next, based on the desired and actual flight paths, the UAV's gimbal adjustment scheme and positioning deviation information are determined. Finally, based on the gimbal adjustment scheme and positioning deviation information, the UAV is controlled to inspect the power distribution lines along the desired flight path. This invention improves the YOLOv5 network by using different detection heads to detect different targets and by utilizing a feature comparison layer to quantify recognition errors, ensuring accurate visual positioning to assist UAV inspection.

Description

Technical Field

[0001] This invention belongs to the field of drone inspection technology, and in particular relates to a method and device for auxiliary photography of distribution network line inspection by drone based on an improved YOLOV5. Background Technology

[0002] The development of the power industry has led to an expansion in the scale and complexity of power transmission lines. Traditional manual inspections are inefficient, limited by environmental constraints, and have limited accuracy. However, the rise of drone technology offers a solution. Various types of drones, including multi-rotor, fixed-wing, and compound-wing drones, each possess unique flight advantages and can be equipped with advanced detection equipment such as high-definition cameras, infrared thermal imagers, and lidar. Simultaneously, supporting technologies such as communication, positioning, and data analysis are constantly evolving. For example, high-speed wireless communication ensures data transmission, precise positioning aids in flight route planning, and intelligent data analysis improves defect identification efficiency. All these factors combined make inspection drones a crucial means of ensuring the safe operation of power transmission lines.

[0003] Currently, during drone inspections, the insufficient positioning accuracy of automated inspections results in non-standard inspection images, making it difficult to complete detailed inspections. Therefore, the technical means of drone inspections in the existing technology still need to be improved. Summary of the Invention

[0004] In view of this, the present invention provides a method and device for auxiliary photography of distribution network line inspection by drone based on improved YOLOV5, which aims to solve the problem that the technical means of drone inspection in the prior art still need to be improved.

[0005] The first aspect of this invention provides a method for assisting in the photographic inspection of power distribution lines using unmanned aerial vehicles (UAVs) based on an improved YOLOv5, comprising:

[0006] Control the drone to conduct a preliminary inspection of the power distribution network lines along the desired route and record the preliminary inspection images;

[0007] Based on the preliminary inspection images and the improved YOLOv5 network, the actual flight path of the UAV is determined. The improved YOLOv5 network is equipped with at least two detection heads and a feature comparison layer. The feature comparison layer is used to compare the feature differences identified by each detection head, and the feature differences represent the image recognition error caused by the flight path deviation.

[0008] Based on the expected and actual flight paths, determine the gimbal adjustment scheme and positioning deviation information for the UAV;

[0009] Based on the gimbal adjustment plan and positioning deviation information, the drone is controlled to inspect the power distribution lines along the expected route.

[0010] In one possible implementation, the improved YOLOv5 network includes a backbone network, a neck network, and a head network; based on the initial inspection images and the improved YOLOv5 network, the actual flight path of the UAV is determined, including:

[0011] The initial inspection images are input into the backbone network to obtain feature maps at multiple scales;

[0012] Based on the first detection head, small target recognition is performed on feature maps of multiple scales to obtain the first target recognition result;

[0013] Based on the feature fusion module in the neck network, a fused feature map is obtained;

[0014] The second detection head performs target recognition on the feature fusion map to obtain the second target recognition result;

[0015] The image recognition error caused by the flight path deviation is determined by comparing the first target recognition result and the second target recognition result with the feature comparison layer in the head network.

[0016] The actual flight path of the UAV is obtained by performing positioning analysis based on image recognition error, first target recognition result, and second target recognition result.

[0017] In one possible implementation, a positioning analysis is performed based on the image recognition error, the first target recognition result, and the second target recognition result to obtain the actual flight path of the UAV, including:

[0018] The targets identified in the first target identification result and the second target identification result are used as reference points;

[0019] The first positioning information of the UAV is calculated based on the coordinates of each reference point and the distance between each reference point and the UAV.

[0020] Based on the image recognition error and the first positioning information, the second positioning information of the UAV is determined.

[0021] In one possible implementation, the first positioning information of the UAV is calculated based on the coordinates of each reference point and the distance between each reference point and the UAV, including:

[0022] Based on the coordinates of each reference point and the distance between each reference point and the UAV, establish a set of distance equations;

[0023] Solving the distance equations yields the coordinates of the UAV, which serve as the primary positioning information.

[0024] In one possible implementation, the second positioning information of the UAV is determined based on the image recognition error and the first positioning information, including:

[0025] The drone's coordinates are updated based on the image recognition error, serving as secondary positioning information.

[0026] In one possible implementation, based on the desired flight path and the actual flight path, the gimbal adjustment scheme and positioning deviation information of the UAV are determined, including:

[0027] Based on the visual recognition model, the deviation distance between each positioning point on the actual route and each target point on the actual route is identified from the preliminary inspection images.

[0028] Based on the deviation distance, determine the gimbal adjustment plan and positioning deviation information.

[0029] In one possible implementation, based on the gimbal adjustment plan and positioning deviation information, the drone is controlled to inspect the power distribution lines along a desired route, including:

[0030] Based on the gimbal adjustment plan and positioning deviation information, determine the adjustment information for the drone;

[0031] The drone is controlled according to the adjustment information, so that it can inspect the power distribution lines along the desired route.

[0032] In one possible implementation, the method further includes:

[0033] Based on the drone's positioning information and preliminary inspection images, the accuracy of feature selection is determined;

[0034] Based on the feature selection accuracy, the target attention mechanism is selected from the preset attention mechanisms.

[0035] In one possible implementation, a target attention mechanism is selected from a set of attention mechanisms based on the feature selection accuracy, including:

[0036] If the feature selection accuracy is greater than the first preset threshold, then the convolutional block attention module will be used as the target attention mechanism.

[0037] If the feature selection accuracy is not greater than the first preset threshold, then the efficient channel attention module will be used as the target attention mechanism.

[0038] A second aspect of this invention provides a drone-assisted photography device for power distribution line inspection based on an improved YOLOv5, comprising:

[0039] The initial inspection module is used to control the drone to conduct a preliminary inspection of the power distribution network lines according to the desired route and record the preliminary inspection images;

[0040] The flight path calculation module is used to determine the actual flight path of the UAV based on the preliminary inspection images and the improved YOLOv5 network. The improved YOLOv5 network is equipped with at least two detection heads and a feature comparison layer. The feature comparison layer is used to compare the feature differences recognized by each detection head. The feature differences represent the image recognition error caused by the flight path deviation.

[0041] The deviation determination module is used to determine the gimbal adjustment scheme and positioning deviation information of the UAV based on the expected flight path and the actual flight path.

[0042] The inspection control module is used to control the drone to inspect the power distribution lines according to the desired route based on the gimbal adjustment plan and positioning deviation information.

[0043] This invention provides a method for assisting UAV inspection and photography of power distribution lines based on an improved YOLOv5 network. First, the UAV is controlled to perform a preliminary inspection of the power distribution lines along a desired flight path and records the preliminary inspection images. Then, based on the preliminary inspection images and the improved YOLOv5 network, the actual flight path of the UAV is determined. The improved YOLOv5 network includes at least two detection heads and a feature comparison layer. The feature comparison layer compares the feature differences identified by each detection head; these differences represent image recognition errors caused by flight path deviations. Next, based on the desired and actual flight paths, a gimbal adjustment scheme and positioning deviation information for the UAV are determined. Finally, based on the gimbal adjustment scheme and positioning deviation information, the UAV is controlled to inspect the power distribution lines along the desired flight path. This invention improves the YOLOv5 network by using different detection heads to detect different targets and by utilizing a feature comparison layer to quantify recognition errors, ensuring accurate visual positioning to assist UAV inspection. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 This is a flowchart illustrating the implementation of the UAV-assisted photography method for power distribution line inspection based on the improved YOLOV5, as provided in this embodiment of the invention.

[0046] Figure 2 This is a schematic diagram of the structure of the UAV inspection auxiliary photography device for power distribution lines based on the improved YOLOV5 provided in the embodiment of the present invention. Detailed Implementation

[0047] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of the invention. However, those skilled in the art will understand that the invention can be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods are omitted so as not to obscure the description of the invention with unnecessary detail.

[0048] Figure 1 This is a flowchart illustrating the implementation of an improved YOLOv5-based drone-assisted photography method for power distribution line inspection, as provided in this embodiment of the invention. Figure 1 As shown, the method for assisting in the inspection and photography of power distribution lines using drones based on the improved YOLOv5 includes:

[0049] S110 controls the drone to conduct a preliminary inspection of the power distribution network lines along the desired route and records the preliminary inspection images.

[0050] In this embodiment of the invention, a suitable drone model is selected based on the inspection requirements of the power distribution network lines. For example, for inspections of power distribution network lines over long distances in complex environments, an industrial-grade drone with strong endurance, high flight stability, and a certain degree of anti-interference capability may be required. Simultaneously, it is essential to ensure that the drone is equipped with high-definition cameras or professional aerial cameras to meet the requirement of clearly recording inspection images.

[0051] A comprehensive inspection of the drone was conducted, including checking the battery level and health, the tightness of the connections of all aircraft components, the integrity and balance of the propellers, and the calibration of the flight control system, to ensure that the drone was in optimal working condition.

[0052] In this embodiment of the invention, specialized UAV flight path planning software is used to formulate a desired inspection route, taking into account the distribution network line's direction, geographical location, surrounding environment, and key inspection areas (such as poles, insulators, and conductor connection points). During planning, the safety of the flight path must be considered to avoid collisions with surrounding obstacles (such as buildings, trees, and mountains). Simultaneously, the flight path's altitude, speed, and the spacing between inspection points must be appropriately set to ensure comprehensive and detailed coverage of the distribution network line and the acquisition of clear images.

[0053] The planned desired flight path data is imported into the drone's flight control system so that the drone can fly automatically along the set route.

[0054] In this embodiment of the invention, the drone automatically flies along the desired route during flight, relying on its built-in flight control and positioning systems. The flight control system adjusts the drone's attitude, speed, and direction in real time to ensure that it accurately follows the set route.

[0055] Operators continuously monitor the drone's flight status, including altitude, speed, battery level, and signal strength, via remote control or ground station software. If any anomalies are detected, such as deviation from the flight path, strong electromagnetic interference, or insufficient battery power, appropriate measures are taken promptly, such as manually adjusting the drone's flight attitude or issuing a return-to-home command.

[0056] Before the drone takes off, the image acquisition equipment on the drone is properly configured based on the ambient lighting conditions, the characteristics of the power grid, and the required image resolution. For example, the camera's focal length, aperture, and ISO are adjusted to ensure that clear images with accurate color reproduction can be captured.

[0057] Typically, the image acquisition frequency is set, that is, an image is taken at regular intervals or flight distances. This ensures that enough image data is acquired evenly throughout the entire inspection route, fully covering all parts of the distribution network.

[0058] Images captured by the drone are stored in real time on its own memory card. To facilitate subsequent viewing, analysis, and processing, it is essential to ensure that the memory card has sufficient storage space and that the storage format is a common, easy-to-read, and easy-to-process format, such as JPEG.

[0059] Meanwhile, some advanced drone systems also support real-time wireless transmission of captured images to ground station software, allowing ground personnel to view the image quality and content immediately, so as to promptly identify problems and make corresponding decisions.

[0060] S120 determines the actual flight path of the UAV based on the preliminary inspection images and the improved YOLOV5 network. The improved YOLOV5 network is equipped with at least two detection heads and a feature comparison layer. The feature comparison layer is used to compare the feature differences identified by each detection head, and the feature differences represent the image recognition error caused by the flight path deviation.

[0061] During inspections, drones need to process large amounts of image data rapidly to detect problems in real time. YOLOv5 employs advanced deep learning algorithms and network structures, enabling accurate detection and classification of various targets. Whether it's a minor defect in power lines, a crack in a building, or an anomaly in the natural environment, YOLOv5 can identify them with high accuracy, reducing false positives and false negatives. Furthermore, YOLOv5 is optimized to maintain a high detection speed even with limited hardware resources, meeting the real-time requirements of drone inspections. This allows drones to quickly analyze images during flight, promptly identifying potential faults, defects, or anomalies, thus improving inspection efficiency.

[0062] Currently, visual positioning technology is often used to assist in the inspection of routes using drones. However, when the drone's positioning is inaccurate, the inspection images taken will also be inaccurate and non-compliant with standards. Traditional YOLOv5 networks have a problem with low precision when analyzing these images. Therefore, this invention proposes an improved YOLOv5 network with multiple detection heads and feature comparison layers to assist in the analysis of flight routes.

[0063] In some embodiments, the improved YOLOv5 network includes a backbone network, a neck network, and a head network. Determining the actual flight path of the UAV based on the preliminary inspection image and the improved YOLOv5 network includes: inputting the preliminary inspection image into the backbone network to obtain feature maps at multiple scales; performing small target recognition on the feature maps at multiple scales using a first detection head to obtain a first target recognition result; obtaining a fused feature map using a feature fusion module in the neck network; performing target recognition on the fused feature map using a second detection head to obtain a second target recognition result; comparing the first target recognition result and the second target recognition result using a feature comparison layer in the head network to determine the image recognition error caused by the flight path deviation; and performing a positioning analysis based on the image recognition error, the first target recognition result, and the second target recognition result to obtain the actual flight path of the UAV.

[0064] In this embodiment of the invention, the backbone network is the basic architecture of the entire object detection network, and its main function is to extract deep features from the input image. In this scenario, the preliminary inspection image is input into the backbone network, which performs feature extraction through a series of complex convolutional layers, pooling layers, and possibly residual connections. Convolutional layers perform convolution operations by sliding convolution kernels across the image, capturing local features. Different sized convolution kernels can acquire local feature information at different scales. Pooling layers mainly serve a downsampling function, reducing the amount of data while preserving the main features of the image, enabling the network to process larger images and speeding up computation. Residual connections help solve the gradient vanishing problem during deep network training, allowing the network to train and converge better.

[0065] As these layers in the backbone network process the image sequentially, the original pixel information is gradually transformed into feature maps with different levels of semantic information. For example, assuming the initial inspection image has a large resolution (e.g., 640×640 pixels), after multiple layers of processing by the backbone network, multiple feature maps of different scales may be generated, such as 80×80, 40×40, and 20×20. These feature maps of different scales contain different semantic and detail information. Larger-scale feature maps (e.g., 80×80) are closer to the original image and retain more detail information, making them suitable for detecting relatively small targets; while smaller-scale feature maps (e.g., 20×20) have undergone more downsampling and feature extraction, containing richer semantic information, and are more suitable for detecting relatively large targets.

[0066] In this embodiment of the invention, the first detection head is a module specifically designed for identifying small targets. Because small targets occupy a small area in an image, their features are relatively subtle and easily overlooked. Therefore, the design of the first detection head needs to focus more on capturing and processing detailed information.

[0067] It receives feature maps of multiple scales generated from the backbone network, and usually focuses on selecting those relatively large-scale feature maps (such as 80×80 scale feature maps) because these feature maps retain more detailed information and are more advantageous for recognizing small targets.

[0068] Inside the first detection head, there is typically a series of convolutional layers and other related processing units. After receiving the feature map, it is first processed through these convolutional layers for further feature extraction. The kernel size, stride, and other parameters of these convolutional layers may be optimized according to the characteristics of small targets.

[0069] The first detection head can identify relevant features of small targets from the feature map. Then, based on pre-set classification criteria and bounding box prediction algorithms, it classifies the identified small targets, determines their category, and predicts the bounding box position of the small targets in the image (such as center point coordinates, width, and height), ultimately obtaining the first target recognition result. This result includes the category information of the detected small targets and their specific location information in the image.

[0070] In this embodiment of the invention, the neck network plays a crucial role in the overall target detection network architecture, situated between the backbone network and the detection head. The feature fusion module within it is particularly critical, its main function being to fuse feature maps of different scales generated by the backbone network to better utilize multi-scale information for target detection.

[0071] Feature maps of different scales each have their advantages and disadvantages. Larger-scale feature maps retain more detailed information but relatively less semantic information, while smaller-scale feature maps are the opposite. By fusing them together through a feature fusion module, the fused feature map can contain both rich semantic information and sufficient detailed information, thus providing higher-quality feature input for subsequent object detection.

[0072] Feature fusion modules typically employ specific fusion strategies. For example, they may use upsampling and downsampling techniques to enlarge smaller-scale feature maps through upsampling and reduce the size of larger-scale feature maps through downsampling. Then, the processed feature maps of different scales are concatenated or added together to achieve the fusion of feature maps of different scales.

[0073] Taking the feature maps of different scales such as 80×80, 40×40, and 20×20 generated by the backbone network mentioned earlier as an example, through the processing of the feature fusion module, a feature map that fuses features of different scales may be obtained. This fused feature map contains both the detailed information extracted from the 80×80 feature map and the rich semantic information obtained from the 20×20 feature map, providing a more comprehensive feature basis for subsequent object detection.

[0074] In this embodiment of the invention, the second detection head is a module used for target recognition of the fused feature map processed by the feature fusion module. Since the fused feature map has incorporated multi-scale information and possesses more comprehensive feature data, the second detection head is designed to utilize these fused features for comprehensive target recognition. It receives the fused feature map generated by the feature fusion module and further processes and analyzes it through a series of internal convolutional layers, other processing units, and possibly special mechanisms (such as attention mechanisms).

[0075] Upon receiving the fused feature map, the convolutional layer inside the second detection head first refines it further, extracting features more conducive to target recognition. Then, based on pre-defined classification criteria and bounding box prediction algorithms, the targets in the fused feature map are classified to determine their category, and the bounding box positions of the targets in the image (such as center point coordinates, width, and height) are predicted, ultimately yielding the second target recognition result. This result includes the category information of the detected targets and their specific location information in the image, similar to the first target recognition result. However, because it is based on the fused feature map for recognition, its accuracy and comprehensiveness may differ.

[0076] In some embodiments, the image recognition error caused by the flight path deviation is determined by comparing the first target recognition result and the second target recognition result with the feature comparison layer in the head network, including: determining the small target in the second target recognition result; and determining the image recognition error caused by the flight path deviation based on the proportion of the small target in the second target recognition result to the first target recognition result.

[0077] In power line inspections, large targets generally refer to relatively easy-to-detect, large objects or components, such as transmission towers, large insulator strings, and transformers. These targets occupy a significant number of pixels in the image, and their shapes and outlines are usually quite distinct, making them easy to observe. Small targets, on the other hand, typically refer to relatively small objects or components, such as small screws on the power line. The size of these small targets can range from a few pixels to tens of pixels, depending on factors such as image resolution and shooting distance.

[0078] Traditional YOLOv5 networks perform multiple downsampling operations to acquire multi-scale features and broader semantic information from images. While downsampling allows the network to better handle targets of different sizes, for small targets, multiple downsampling operations continuously reduce their feature size, resulting in the loss of a significant amount of detailed information and making accurate identification difficult. For example, a small target that was originally only a few pixels in size may become almost unrecognizable in the feature map after multiple downsampling operations, making it impossible for the network to accurately determine its location and category. This invention addresses this problem by setting up a small target detection head; however, when there is a flight path deviation, the initial inspection image itself has certain problems, which also affect target recognition. Before flight path correction, it is usually difficult to quantify this impact.

[0079] Traditional YOLOv5 network detectors (i.e., the second detector) lose small targets due to the aforementioned reasons. This is especially problematic when flight path deviations cause image quality issues; further reducing resolution makes the details of small targets even more blurred, exacerbating the loss problem. This invention leverages this characteristic by extracting the small targets identified by the second detector and querying the corresponding preset image recognition error based on their proportion. If the proportion of small targets identified by the second detector in the first target recognition result is large, it indicates that the flight path deviation has little impact, and the corresponding preset image recognition error is small. Conversely, if the proportion is small, it indicates that the flight path deviation has a significant impact, and the corresponding preset image recognition error is large.

[0080] In some embodiments, the actual flight path of the UAV is obtained by performing positioning analysis based on image recognition error, first target recognition result, and second target recognition result, including: using the targets identified in the first target recognition result and second target recognition result as reference points; calculating the first positioning information of the UAV based on the coordinates of each reference point and the distance between each reference point and the UAV; and determining the second positioning information of the UAV based on the image recognition error and the first positioning information.

[0081] In this embodiment of the invention, the coordinates of the UAV are calculated using a four-point positioning method based on the coordinates of each reference point and the distance between each reference point and the UAV. This coordinates constitute the UAV's first positioning information. When more than four targets are identified, different combinations of reference points are used for four-point positioning. The average of the calculated UAV coordinates is the UAV's second positioning information. However, when errors exist, the identified small targets may be inaccurate, affecting the UAV's positioning. The larger the image recognition error, the greater the impact. Therefore, this invention uses a weighted average of the reference points to recalculate the UAV's coordinates. The weight of each set of reference points depends on the number of small targets within the set and the image recognition error.

[0082] For example, if all reference points in a group are large targets and their initial weights are all 1, then the total weight of this group is 1 * 4 / 4 = 1. If there are 3 small targets among the reference points in a group and the image recognition error is 0.5, then the total weight of this group is (1 + 3 * 0.5) / 4 = 0.625.

[0083] In some embodiments, the first positioning information of the UAV is calculated based on the coordinates of each reference point and the distance between each reference point and the UAV, including: establishing a set of distance equations based on the coordinates of each reference point and the distance between each reference point and the UAV; solving the set of distance equations to obtain the coordinates of the UAV, which are used as the first positioning information.

[0084] In some embodiments, determining the second positioning information of the UAV based on the image recognition error and the first positioning information includes: updating the coordinates of the UAV based on the image recognition error as the second positioning information.

[0085] S130 determines the gimbal adjustment scheme and positioning deviation information for the UAV based on the desired and actual flight paths.

[0086] In some embodiments, based on a visual recognition model, the deviation distance between each positioning point on the actual route and each target point on the actual route is identified from the preliminary inspection image; based on the deviation distance, the gimbal adjustment scheme and positioning deviation information are determined.

[0087] In this embodiment of the invention, the visual recognition model can be a deep convolutional neural network. The visual recognition model plays a crucial role in this method. Built on deep learning technology and trained with a large amount of power distribution network image data, it possesses the ability to accurately identify various targets (such as towers, insulators, conductor connection points, etc.) and location point features within the power distribution network. When processing initial inspection images, the model uses feature extraction and pattern recognition algorithms to accurately determine the relative positional relationship between the location point and the target point on the actual route, calculating the deviation distance between them. This deviation distance covers horizontal, vertical, and depth deviations, providing crucial data for subsequent pan-tilt unit adjustments and positioning deviation determination. For example, the model extracts target feature maps from the image using a convolutional neural network, processes them through pooling and fully connected layers, and matches them with predefined target feature templates to obtain the coordinate difference between the location point and the target point, i.e., the deviation distance. This overcomes the limitations of traditional methods that rely on manual labeling or simple image processing algorithms, and adapts to the challenges of complex environments and target diversity. Power distribution network image data covering different environments, seasons, time periods, and line conditions are collected. The location and category information of the location points and target points are accurately labeled using annotation tools to construct training, validation, and test sets, ensuring data diversity and annotation accuracy. During training, the backpropagation algorithm is used to optimize model parameters. The prediction bias is evaluated and network weights are adjusted according to the loss function (such as a weighted combination of cross-entropy loss and position regression loss). Data augmentation techniques (such as random cropping, rotation, flipping, and brightness and contrast adjustment) are introduced to improve the model's generalization ability. After multiple rounds of iterative training, the model achieves high-precision and stable performance indicators on the test set, laying the foundation for accurate recognition and deviation measurement of actual inspection images.

[0088] The gimbal adjustment scheme is determined based on the identified deviation distance. The gimbal controls the camera attitude, and its adjustment directly affects the image acquisition angle and target imaging position. If the deviation distance indicates that the target point deviates from the desired position in the horizontal direction of the image, the gimbal adjusts the yaw angle to rotate the camera horizontally to compensate for the deviation; for vertical deviation, the pitch angle is adjusted for compensation; for depth deviation (such as due to inaccurate drone altitude), the drone altitude is finely adjusted in conjunction with flight control system commands, and the gimbal angle is adjusted in coordination to maintain clear target imaging. The adjustment scheme is generated according to the magnitude and direction of the deviation, based on predefined rules or adaptive algorithms, to ensure that the camera quickly aligns with the target and optimizes image acquisition quality and positioning accuracy. For example, if a 5° yaw angle adjustment is set for every 1 meter of horizontal deviation, the required adjustment amount in each direction is accurately calculated accordingly. The adjustment strategy is optimized in real time, taking into account drone flight speed, stability, and environmental interference factors, to avoid image jitter and positioning fluctuations caused by excessive or insufficient adjustment.

[0089] A gimbal adjustment algorithm module is embedded in the flight control system. It receives deviation distance data from the visual recognition model in real time and calculates the gimbal adjustment angle or displacement in each direction according to a predefined mapping relationship. Considering the dynamic response characteristics and stability of the UAV, the algorithm includes adjustment thresholds and filtering mechanisms to prevent frequent over-adjustment due to minor deviations, which could cause gimbal jitter. Furthermore, it adaptively adjusts the scaling factor based on the UAV's flight speed, acceleration, and environmental interference factors to ensure stable and accurate compensation of positioning deviations under different operating conditions, maintaining stable and clear image acquisition.

[0090] The gimbal adjustment scheme includes yaw, pitch, and roll adjustment commands, as well as dynamic parameters such as adjustment speed and acceleration. The scheme analyzes the absolute value and direction (clockwise or counterclockwise) of each angle adjustment, along with the values ​​of the dynamic parameters. For example, the yaw adjustment is +10° (right rotation), the adjustment speed is 5° / s, and the acceleration is 1° / s. 2 These parameters reflect the gimbal's motion trajectory and characteristics, providing a key basis for determining positioning deviation information. Since changes in gimbal attitude are directly related to changes in camera viewpoint and target imaging position shift, the adjustment speed and acceleration affect the timeliness and stability of positioning compensation.

[0091] The deviation distance is decomposed into dx, dy, and dz along the horizontal (x-axis), vertical (y-axis), and depth (z-axis). If the deviation distance is a spatial vector D = (3, -2, 1) meters, then dx = 3 meters, dy = -2 meters, and dz = 1 meter. Coordinate transformation is performed based on the transformation relationship between the UAV coordinate system and the world coordinate system (including rotation and translation matrices, determined by positioning information, inertial measurement unit data, and initial calibration parameters). For example, the deviation distance in the camera coordinate system is transformed to the world coordinate system using an extrinsic parameter matrix, accurately determining the position deviation of the target point relative to the UAV within the global positioning framework. This lays the foundation for accurate navigation and positioning correction. Furthermore, the depth deviation dz is significant for UAV altitude adjustment and 3D spatial path planning, while the horizontal and vertical deviations guide planar trajectory correction.

[0092] Considering the dynamic process of gimbal adjustment, the positioning deviation compensation is calculated based on the adjustment speed, acceleration, and current attitude. Assuming an initial yaw angle deviation causes a horizontal shift in the target point, the compensation effect at different times is calculated based on the adjustment speed and time. For example, if the gimbal adjusts the yaw angle at 5° / s, the theoretical compensation for the horizontal target imaging position after 2 seconds is a 10° horizontal displacement compensation. The actual pixel or spatial distance compensation value is calculated using trigonometric functions, camera field of view, and focal length. This compensation reflects the degree of positioning deviation correction by the gimbal adjustment. Real-time comparison with the deviation distance evaluates the adjustment effect, providing a quantitative indicator for dynamically optimizing the adjustment strategy. This ensures that positioning accuracy gradually becomes more precise and stable with gimbal movement, which is indispensable for maintaining positioning accuracy, especially under conditions of variable-speed UAV inspections or complex environmental airflow interference.

[0093] This system integrates data from multiple sensors, including positioning information, inertial measurement unit (IMU), and lidar, to calibrate positioning deviations. Positioning information provides the UAV's absolute position, but its accuracy fluctuates due to environmental interference. The IMU outputs high-frequency attitude change data, but cumulative errors exist. LiDAR provides high-precision measurements of the distances between obstacles and targets to construct a local environmental map to aid positioning. Using positioning information as a baseline, IMU attitude data compensates for the impact of instantaneous attitude deviations on positioning. LiDAR detects terrain features around the route to correct visual positioning deviations caused by terrain undulations or obstacle obstructions. Through Kalman filtering, data fusion algorithms, weighted fusion of multi-source data, error correction, and state estimation, high-precision calibrated positioning deviation information is generated, improving positioning reliability and robustness. This provides a solid data foundation for reliable decision-making and precise operation of UAV intelligent inspections, ensuring the safe and efficient execution of inspection tasks in complex power grid environments.

[0094] After determining the deviation distance, the system first checks whether the drone's position needs adjustment. If not, a gimbal adjustment plan is calculated based on the deviation distance, while the positioning deviation information is left blank. If yes, both the positioning deviation information and the gimbal adjustment plan are calculated based on the deviation distance. In other words, if the inspection can be completed by adjusting the drone's gimbal, the drone's position is not adjusted. If the drone's position needs adjustment, only the drone's position can be adjusted, or both the drone's position and the gimbal can be adjusted simultaneously; there are no restrictions on this.

[0095] S140, based on the gimbal adjustment plan and positioning deviation information, controls the drone to inspect the power distribution lines along the desired route.

[0096] In some embodiments, the method of controlling a drone to inspect power distribution lines along a desired route based on a gimbal adjustment scheme and positioning deviation information includes: determining adjustment information for the drone based on the gimbal adjustment scheme and positioning deviation information; and controlling the drone based on the adjustment information to enable the drone to inspect the power distribution lines along the desired route.

[0097] In this embodiment of the invention, the positioning deviation information includes the deviation distance between each positioning point on the actual flight path and the corresponding ideal position point on the desired flight path. By analyzing these deviation distances, the degree and direction of the UAV's deviation from the desired flight path at different positions can be understood. For example, if the deviation distance of a certain positioning point is large, it indicates that the UAV has deviated far from the desired flight path at that position; the direction of the deviation distance can be determined by comparing the coordinate difference between the positioning point and the corresponding ideal position point. For example, in a two-dimensional plane, if the x-coordinate of the positioning point is greater than the x-coordinate of the ideal position point, it indicates that the UAV has deviated from the desired flight path to the right in the horizontal direction.

[0098] Determine the position adjustment strategy: Based on the analysis of positioning deviation information, formulate corresponding position adjustment strategies to determine the position adjustment information. Common adjustment strategies are as follows:

[0099] Direction Adjustment: Determine the required flight direction for the drone based on the direction of the deviation distance. If the drone deviates to the right from the desired flight path, the position adjustment information should include a command to adjust the flight direction to the left. The specific adjustment angle can be determined based on factors such as the degree of deviation. For example, a smaller deviation can be set with a smaller adjustment angle, while a larger deviation may require a larger adjustment angle. The specific adjustment angle value is usually determined through a certain proportional relationship or preset adjustment rules.

[0100] Speed ​​Adjustment: In some situations, in addition to directional adjustments, it may also be necessary to adjust the drone's flight speed. If the drone deviates significantly from the intended flight path, the flight speed can be increased appropriately to return to the intended path as quickly as possible; conversely, if the deviation is small, the original speed may be maintained or the speed may be appropriately reduced for fine-tuning. The specific speed adjustment value can also be determined comprehensively based on factors such as the deviation distance and the urgency of the mission.

[0101] Altitude Adjustment: For some power distribution line inspection tasks that require flight at specific altitudes, altitude adjustment is necessary when the positioning deviation information shows that the drone is also deviating in the altitude direction. For example, if the drone is below the expected flight altitude, the position adjustment information should include an instruction to ascend to the expected altitude, with the specific ascent value determined based on the actual deviation.

[0102] In some embodiments, the method further includes: determining the feature selection accuracy based on the UAV's positioning information and preliminary inspection images; and selecting a target attention mechanism from preset attention mechanisms based on the feature selection accuracy.

[0103] In some embodiments, a target attention mechanism is selected from a preset attention mechanism based on the feature selection accuracy, including: if the feature selection accuracy is greater than a first preset threshold, then the convolutional block attention module is selected as the target attention mechanism; if the feature selection accuracy is not greater than the first preset threshold, then the efficient channel attention module is selected as the target attention mechanism.

[0104] In this embodiment of the invention, the Convolutional Block Attention Module (CBAM) is a mechanism that combines channel attention and spatial attention. First, the channel attention module assigns different weights to different channels of the feature map, highlighting important channel information. Then, the spatial attention module assigns different levels of attention to different spatial locations in the feature map, emphasizing important spatial regions. It can filter and weight features from both channel and spatial dimensions, better focusing on the key features of the target and reducing attention to irrelevant information such as the background, effectively improving detection accuracy. However, UAV inspection has high requirements for detection speed, and the additional computation it introduces may have a certain impact.

[0105] The Efficient Channel Attention (ECA) module primarily focuses on channel-level information. It uses fast one-dimensional convolutional operations to obtain the weights for each channel, thus adaptively adjusting channel importance. This approach effectively captures the differences in importance between different channels without significantly increasing computational cost. Its high computational efficiency enhances the model's ability to extract important features while maintaining high detection speed. Giving greater weight to channels with significant information in the feature map helps the model better focus on key information. However, it pays less attention to spatial information, and in some complex contexts, it may not fully utilize spatial location information.

[0106] Based on the drone's location information and preliminary inspection images, the approximate location of the drone and the number of small targets at that location can be determined. The feature selection accuracy is determined based on the number of small targets; the higher the number of small targets, the higher the feature selection accuracy. Utilizing the feature selection accuracy, attention mechanisms with varying computational costs can be employed. For example, if the feature selection accuracy exceeds a first preset threshold, a convolutional block attention module can be used as the target attention mechanism; if the feature selection accuracy is not greater than the first preset threshold, an efficient channel attention module can be used as the target attention mechanism.

[0107] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0108] Figure 2 This is a schematic diagram of the structure of the UAV inspection auxiliary photography device for power distribution lines based on the improved YOLOV5 provided in an embodiment of the present invention. Figure 2 As shown, in some embodiments, the UAV inspection auxiliary photography device 2 for power distribution lines based on the improved YOLOv5 includes:

[0109] The initial inspection module 210 is used to control the UAV to conduct a preliminary inspection of the power distribution network lines according to the desired route and record the preliminary inspection images.

[0110] The flight path calculation module 220 is used to determine the actual flight path of the UAV based on the preliminary inspection images and the improved YOLOv5 network. The improved YOLOv5 network is equipped with at least two detection heads and a feature comparison layer. The feature comparison layer is used to compare the feature differences identified by each detection head. The feature differences represent the image recognition error caused by the flight path deviation.

[0111] The deviation determination module 230 is used to determine the gimbal adjustment scheme and positioning deviation information of the UAV based on the expected flight path and the actual flight path.

[0112] The inspection control module 240 is used to control the drone to inspect the power distribution line according to the desired route based on the gimbal adjustment plan and positioning deviation information.

[0113] Optionally, the improved YOLOv5 network includes a backbone network, a neck network, and a head network; the flight path calculation module 220 is used for: inputting the preliminary inspection image into the backbone network to obtain feature maps at multiple scales; performing small target recognition on the feature maps at multiple scales based on the first detection head to obtain a first target recognition result; obtaining a fused feature map based on the feature fusion module in the neck network; performing target recognition on the fused feature map based on the second detection head to obtain a second target recognition result; comparing the first target recognition result and the second target recognition result based on the feature comparison layer in the head network to determine the image recognition error caused by the flight path deviation; and performing positioning analysis based on the image recognition error, the first target recognition result, and the second target recognition result to obtain the actual flight path of the UAV.

[0114] Optionally, the flight path calculation module 220 is used to: use the targets identified in the first target recognition result and the second target recognition result as reference points; calculate the first positioning information of the UAV based on the coordinates of each reference point and the distance between each reference point and the UAV; and determine the second positioning information of the UAV based on the image recognition error and the first positioning information.

[0115] Optionally, the flight path calculation module 220 is used to: establish a set of distance equations based on the coordinates of each reference point and the distance between each reference point and the UAV; solve the set of distance equations to obtain the coordinates of the UAV as the first positioning information.

[0116] Optionally, the flight path calculation module 220 is used to update the coordinates of the UAV based on the image recognition error, as a second positioning information.

[0117] Optionally, the deviation determination module 230 is used to: identify the deviation distance between each positioning point and each target point on the actual route from the preliminary inspection image based on the visual recognition model; and determine the gimbal adjustment scheme and positioning deviation information based on the deviation distance.

[0118] Optionally, the inspection control module 240 is used to: determine the adjustment information of the UAV based on the gimbal adjustment plan and positioning deviation information; and control the UAV according to the adjustment information so that the UAV can inspect the power distribution line according to the expected route.

[0119] Optionally, the device further includes: a filtering module, used to determine the feature filtering accuracy based on the UAV's positioning information and preliminary inspection images; and to select a target attention mechanism from preset attention mechanisms based on the feature filtering accuracy.

[0120] Optionally, a filtering module is used to: if the feature filtering accuracy is greater than a first preset threshold, then use the convolutional block attention module as the target attention mechanism; if the feature filtering accuracy is not greater than the first preset threshold, then use the efficient channel attention module as the target attention mechanism.

[0121] The UAV inspection auxiliary photography device for distribution network lines based on the improved YOLOV5 provided in this embodiment can be used to execute the above method embodiment. Its implementation principle and technical effect are similar, and will not be described again in this embodiment.

[0122] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0123] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0124] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for assisting in the photographic inspection of power distribution lines using unmanned aerial vehicles (UAVs) based on an improved YOLOv5, characterized in that, include: Control the drone to conduct a preliminary inspection of the power distribution network lines along the desired route and record the preliminary inspection images; Based on the preliminary inspection images and the improved YOLOv5 network, the actual flight path of the UAV is determined; wherein, the improved YOLOv5 network is configured with at least two detection heads and a feature comparison layer; the feature comparison layer is used to compare the feature differences identified by each detection head, and the feature differences represent the image recognition error caused by the flight path deviation; Based on the desired flight path and the actual flight path, determine the gimbal adjustment scheme and positioning deviation information for the UAV; Based on the gimbal adjustment scheme and the positioning deviation information, the drone is controlled to inspect the power distribution line along the desired route.

2. The method for assisting in the inspection and photography of power distribution lines using unmanned aerial vehicles (UAVs) based on improved YOLOv5 according to claim 1, characterized in that, The improved YOLOv5 network includes a backbone network, a neck network, and a head network. Based on the preliminary inspection images and the improved YOLOv5 network, the actual flight path of the UAV is determined, including: The preliminary inspection images are input into the backbone network to obtain feature maps at multiple scales; Based on the first detection head, small target recognition is performed on feature maps of multiple scales to obtain the first target recognition result; Based on the feature fusion module in the neck network, a fused feature map is obtained; The second detection head performs target recognition on the fused feature map to obtain the second target recognition result; The image recognition error caused by the flight path deviation is determined by comparing the first target recognition result and the second target recognition result with the feature comparison layer in the head network. The actual flight path of the UAV is obtained by performing positioning analysis based on image recognition error, first target recognition result, and second target recognition result.

3. The method for assisting in the inspection and photography of power distribution lines using unmanned aerial vehicles (UAVs) based on improved YOLOv5 according to claim 2, characterized in that, Based on image recognition error, first target recognition result, and second target recognition result, a positioning analysis is performed to obtain the actual flight path of the UAV, including: The targets identified in the first target identification result and the second target identification result are used as reference points; The first positioning information of the UAV is calculated based on the coordinates of each reference point and the distance between each reference point and the UAV. Based on the image recognition error and the first positioning information, the second positioning information of the UAV is determined.

4. The method for assisting in the inspection and photography of power distribution lines using unmanned aerial vehicles (UAVs) based on improved YOLOv5 according to claim 3, characterized in that, Based on the coordinates of each reference point and the distance between each reference point and the UAV, the first positioning information of the UAV is calculated, including: Based on the coordinates of each reference point and the distance between each reference point and the UAV, establish a set of distance equations; Solving the distance equations yields the coordinates of the UAV, which serve as the first positioning information.

5. The method for assisting in the inspection and photography of power distribution lines using unmanned aerial vehicles (UAVs) based on improved YOLOv5 according to claim 4, characterized in that, Based on the image recognition error and the first positioning information, the second positioning information of the UAV is determined, including: The coordinates of the UAV are updated based on the image recognition error, serving as the second positioning information.

6. The method for assisting in the inspection and photography of power distribution lines using unmanned aerial vehicles (UAVs) based on improved YOLOv5 according to claim 1, characterized in that, Based on the desired flight path and the actual flight path, determine the UAV gimbal adjustment scheme and positioning deviation information, including: Based on the visual recognition model, the deviation distance between each positioning point and each target point on the actual route is identified from the preliminary inspection image; Based on the deviation distance, the gimbal adjustment scheme and positioning deviation information are determined.

7. The method for assisting in the inspection and photography of power distribution lines using unmanned aerial vehicles (UAVs) based on improved YOLOv5 according to claim 1, characterized in that, Based on the gimbal adjustment scheme and the positioning deviation information, the drone is controlled to inspect the power distribution line along the desired route, including: Based on the gimbal adjustment scheme and the positioning deviation information, the adjustment information of the UAV is determined; The drone is controlled according to the adjustment information, so that it can inspect the power distribution line along the desired route.

8. The method for assisting in the inspection and photography of power distribution lines using unmanned aerial vehicles (UAVs) based on improved YOLOv5 according to claim 1, characterized in that, The method further includes: Based on the drone's location information and the preliminary inspection images, the accuracy of feature selection is determined; Based on the feature selection accuracy, a target attention mechanism is selected from the preset attention mechanisms.

9. The method for assisting in the unmanned aerial vehicle (UAV) inspection of power distribution lines based on improved YOLOv5 according to claim 8, characterized in that, Based on the feature selection accuracy, a target attention mechanism is selected from a preset set of attention mechanisms, including: If the feature selection accuracy is greater than the first preset threshold, then the convolutional block attention module is used as the target attention mechanism. If the feature selection accuracy is not greater than the first preset threshold, then the efficient channel attention module will be used as the target attention mechanism.

10. A drone-assisted photography device for power distribution line inspection based on an improved YOLOv5, characterized in that, include: The initial inspection module is used to control the drone to conduct a preliminary inspection of the power distribution network lines according to the desired route and record the preliminary inspection images; The flight path calculation module is used to determine the actual flight path of the UAV based on the preliminary inspection image and the improved YOLOv5 network. The improved YOLOv5 network is equipped with at least two detection heads and a feature comparison layer. The feature comparison layer is used to compare the feature differences identified by each detection head, and the feature differences represent the image recognition error caused by the flight path deviation. The deviation determination module is used to determine the gimbal adjustment scheme and positioning deviation information of the UAV based on the expected flight path and the actual flight path. The inspection control module is used to control the UAV to inspect the power distribution line according to the desired route based on the gimbal adjustment scheme and the positioning deviation information.

Images