Improved YOLOV5-based distribution network line unmanned aerial vehicle inspection auxiliary photographing method and device

Abstract

The invention provides a distribution network line unmanned aerial vehicle inspection auxiliary photographing method based on improved YOLOV5, and the method comprises the steps: firstly controlling an unmanned aerial vehicle to carry out the preliminary inspection of a distribution network line according to an expected route, and recording a preliminary inspection image; according to the preliminary inspection image and the improved YOLOV5 network, determining an actual route of the unmanned aerial vehicle; wherein the improved YOLOV5 network is provided with at least two detection heads and a feature comparison layer; the feature comparison layer is used for comparing feature differences recognized by the detection heads, and the feature differences represent image recognition errors caused by route deviation; determining a holder adjustment scheme and positioning deviation information of the unmanned aerial vehicle according to the expected route and the actual route; and finally, according to the holder adjustment scheme and the positioning deviation information, controlling the unmanned aerial vehicle to inspect the distribution network line according to the expected route. According to the invention, different targets are detected by setting different detection heads, and the feature comparison layer is utilized to quantify the identification error to improve the YOLOV5 network, so that visual positioning can accurately assist the inspection of the unmanned aerial vehicle.

Description

Technical Field

[0001] The present invention belongs to the technical field of drone inspection, and in particular relates to a distribution network line drone inspection auxiliary photography method and device based on improved YOLOV5. Background Art

[0002] The development of the power industry has led to the expansion of the scale and complexity of the distribution of power transmission lines. Traditional manual inspections are inefficient, restricted by the environment, and have limited detection accuracy. However, drone technology has emerged, and its multi-rotor, fixed-wing, and composite-wing types each have their own flight advantages, and can be equipped with advanced detection equipment such as high-definition cameras, infrared thermal imagers, and lidar. At the same time, supporting technologies such as communications, positioning, and data analysis and processing are constantly developing. For example, high-speed wireless communications ensure data transmission, precise positioning helps flight route planning, and intelligent data analysis improves defect identification efficiency. Combining these factors, inspection drones have become a key means to ensure the safe operation of power transmission lines.

[0003] At present, during drone inspections, due to insufficient positioning accuracy of automated inspections, the inspection images taken are not standardized, making it difficult to complete detailed inspections. Therefore, the technical means of drone inspections in existing technologies still need to be improved. Summary of the invention

[0004] In view of this, the present invention provides a distribution network line drone inspection auxiliary photography method and device based on improved YOLOV5, aiming to solve the problem that the technical means of drone inspection in the prior art still need to be improved.

[0005] A first aspect of an embodiment of the present invention provides a distribution network line drone inspection auxiliary photography method based on improved YOLOV5, comprising:

[0006] Control the drone to conduct preliminary inspection of the distribution network line according to the expected route and record the preliminary inspection images;

[0007] Determine the actual route of the drone based on the preliminary inspection image and the improved YOLOV5 network; wherein the improved YOLOV5 network is provided with at least two detection heads and a feature comparison layer; the feature comparison layer is used to compare the feature differences recognized by each detection head, and the feature differences represent the image recognition errors caused by the route deviation;

[0008] Determine the gimbal adjustment plan and positioning deviation information of the drone based on the expected route and actual route;

[0009] According to the gimbal adjustment plan and positioning deviation information, the drone is controlled to inspect the distribution network lines along the expected route.

[0010] In a possible implementation, the improved YOLOV5 network includes a backbone network, a neck network, and a head network; according to the preliminary inspection image and the improved YOLOV5 network, the actual route of the drone is determined, including:

[0011] Input the preliminary inspection image into the backbone network to obtain feature maps at multiple scales;

[0012] Performing small target recognition on feature maps of multiple scales according to the first detection head to obtain a first target recognition result;

[0013] According to the feature fusion module in the neck network, a fused feature map is obtained;

[0014] Performing target recognition on the feature fusion image according to the second detection head to obtain a second target recognition result;

[0015] Comparing the first target recognition result with the second target recognition result according to the feature comparison layer in the head network to determine the image recognition error caused by the route deviation;

[0016] Positioning analysis is performed based on the image recognition error, the first target recognition result, and the second target recognition result to obtain the actual route of the UAV.

[0017] In a possible implementation, positioning analysis is performed based on the image recognition error, the first target recognition result, and the second target recognition result to obtain the actual route of the UAV, including:

[0018] Using the targets identified in the first target recognition result and the second target recognition result as reference points;

[0019] Calculate the first positioning information of the UAV according to the coordinates of each reference point and the distance between each reference point and the UAV;

[0020] The second positioning information of the UAV is determined according to the image recognition error and the first positioning information.

[0021] In a possible implementation, calculating the first positioning information of the drone according to the coordinates of each reference point and the distance between each reference point and the drone includes:

[0022] According to the coordinates of each reference point and the distance between each reference point and the UAV, a distance equation group is established;

[0023] The distance equations are solved to obtain the coordinates of the UAV as the first positioning information.

[0024] In a possible implementation, determining the second positioning information of the drone according to the image recognition error and the first positioning information includes:

[0025] The coordinates of the drone are updated according to the image recognition error as the second positioning information.

[0026] In a possible implementation, the gimbal adjustment scheme and positioning deviation information of the drone are determined according to the expected route and the actual route, including:

[0027] According to the visual recognition model, the deviation distance between each positioning point on the actual route and each target point on the actual route is identified from the preliminary inspection image;

[0028] According to the deviation distance, determine the gimbal adjustment plan and positioning deviation information.

[0029] In a possible implementation, according to the gimbal adjustment plan and positioning deviation information, the drone is controlled to inspect the distribution network line according to the expected route, including:

[0030] Determine the adjustment information of the drone based on the gimbal adjustment plan and positioning deviation information;

[0031] The drone is controlled according to the adjustment information so that it can inspect the distribution network lines according to the expected route.

[0032] In a possible implementation, the method further includes:

[0033] Determine the feature screening accuracy based on the positioning information of the drone and the preliminary inspection images;

[0034] According to the feature screening accuracy, the target attention mechanism is selected from the preset attention mechanisms.

[0035] In a possible implementation, a target attention mechanism is selected from preset attention mechanisms according to feature screening accuracy, including:

[0036] If the feature screening accuracy is greater than the first preset threshold, the convolutional block attention module is used as the target attention mechanism;

[0037] If the feature screening accuracy is not greater than the first preset threshold, the efficient channel attention module is used as the target attention mechanism.

[0038] A second aspect of an embodiment of the present invention provides a distribution network line drone inspection auxiliary photography device based on improved YOLOV5, comprising:

[0039] The initial inspection module is used to control the UAV to conduct a preliminary inspection of the distribution network line according to the expected route and record the preliminary inspection images;

[0040] A route calculation module is used to determine the actual route of the drone based on the preliminary inspection image and the improved YOLOV5 network; wherein the improved YOLOV5 network is provided with at least two detection heads and a feature comparison layer; the feature comparison layer is used to compare the feature differences recognized by each detection head, and the feature differences represent the image recognition errors caused by the route deviation;

[0041] The deviation determination module is used to determine the gimbal adjustment plan and positioning deviation information of the drone according to the expected route and the actual route;

[0042] The inspection control module is used to control the UAV to inspect the distribution network lines along the expected route according to the gimbal adjustment plan and positioning deviation information.

[0043] The embodiment of the present invention provides a distribution network line drone inspection auxiliary photography method based on improved YOLOV5, firstly controlling the drone to perform preliminary inspection on the distribution network line according to the expected route, and recording the preliminary inspection image; then determining the actual route of the drone according to the preliminary inspection image and the improved YOLOV5 network; wherein the improved YOLOV5 network is provided with at least two detection heads and a feature comparison layer; the feature comparison layer is used to compare the feature differences recognized by each detection head, and the feature difference represents the image recognition error caused by the route deviation; then determining the pan-tilt adjustment scheme and positioning deviation information of the drone according to the expected route and the actual route; finally, according to the pan-tilt adjustment scheme and the positioning deviation information, controlling the drone to inspect the distribution network line according to the expected route. The present invention detects different targets by setting different detection heads, and at the same time uses the feature comparison layer to quantify the recognition error to improve the YOLOV5 network, so as to ensure that visual positioning can accurately assist drone inspection. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0045] Figure 1 It is a flow chart of the implementation of the distribution network line drone inspection auxiliary photography method based on improved YOLOV5 provided in an embodiment of the present invention;

[0046] Figure 2 It is a structural schematic diagram of a distribution network line drone inspection auxiliary photography device based on improved YOLOV5 provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0047] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present invention. However, it should be clear to those skilled in the art that the present invention may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present invention.

[0048] Figure 1 1 is a flowchart of the implementation of the method for assisting photography of distribution network line drone inspection based on improved YOLOV5 provided by an embodiment of the present invention. Figure 1 As shown in the figure, the distribution network line drone inspection auxiliary photography method based on improved YOLOV5 includes:

[0049] S110, controlling the UAV to perform a preliminary inspection on the distribution network line according to the expected route, and recording the preliminary inspection image.

[0050] In the embodiment of the present invention, a suitable drone model is selected according to the inspection requirements of the distribution network line. For example, for the inspection of distribution network lines over long distances in some complex environments, an industrial-grade drone with long endurance, high flight stability and certain anti-interference capabilities may be required. At the same time, it is necessary to ensure that the drone is equipped with image acquisition equipment such as a high-definition camera or a professional aerial camera to meet the requirements of clearly recording the inspection images.

[0051] Conduct a comprehensive inspection of the drone, including the battery charge and health status, the connection firmness of the aircraft components, the integrity and balance of the propellers, the calibration of the flight control system, etc., to ensure that the drone is in the best working condition.

[0052] In the embodiment of the present invention, a professional drone route planning software is used to formulate a desired inspection route in combination with the direction, geographical location, surrounding environment, and areas that need to be inspected (such as poles, insulators, and wire connection points) of the distribution network line. When planning, the safety of the route should be considered to avoid collisions with surrounding obstacles (such as buildings, trees, and mountains); at the same time, the height, speed, and spacing between inspection points of the route should be reasonably set to ensure that the distribution network line can be fully and carefully covered and clear images can be obtained.

[0053] Import the planned desired route data into the UAV's flight control system so that the UAV can fly automatically according to the set route.

[0054] In the embodiment of the present invention, the drone automatically flies along the desired route by relying on its built-in flight control system and positioning system during flight. The flight control system adjusts the attitude, speed and direction of the drone in real time to ensure that it accurately moves along the set route.

[0055] The operator continuously monitors the flight status of the drone through the remote control or ground station software, including flight altitude, speed, battery power, signal strength, etc. Once any abnormal situation is found, such as deviation from the route, strong electromagnetic interference, insufficient battery power, etc., appropriate measures will be taken in time to deal with it, such as manually intervening to adjust the flight attitude of the drone, issuing a return command, etc.

[0056] Before the drone takes off, the image acquisition equipment on the drone should be properly set according to the on-site ambient light conditions, the characteristics of the distribution network line, and the required image resolution. For example, the camera's focal length, aperture, sensitivity (ISO) and other parameters should be adjusted to ensure that clear images with accurate color reproduction can be captured.

[0057] Usually the frequency of image acquisition is set, that is, an image is taken at a certain interval or flight distance, so as to ensure that enough image data is evenly obtained throughout the inspection route to fully cover all parts of the distribution network line.

[0058] The images taken by the drone are stored in real time in the memory card equipped with it. In order to facilitate subsequent viewing, analysis and processing, it is necessary to ensure that the memory card has sufficient storage space and the storage format uses a common, easy-to-read and process format, such as JPEG.

[0059] At the same time, some advanced drone systems also support real-time transmission of captured images to ground station software via wireless transmission, so that ground personnel can check the quality and content of the images at the first time to detect problems in time and make corresponding decisions.

[0060] S120, determining the actual route of the UAV based on the preliminary inspection image and the improved YOLOV5 network; wherein the improved YOLOV5 network is provided with at least two detection heads and a feature comparison layer; the feature comparison layer is used to compare the feature differences recognized by each detection head, and the feature differences represent the image recognition errors caused by the route deviation.

[0061] During the inspection process, drones need to quickly process a large amount of image data in order to detect problems in real time. YOLOv5 uses advanced deep learning algorithms and network structures to accurately detect and classify various targets. Whether it is a tiny defect on a power line, a crack in a building, or an abnormal object in the natural environment, YOLOv5 can identify it with high accuracy, reducing misjudgments and missed judgments. At the same time, YOLOv5 has been optimized to maintain a high detection speed even with limited hardware resources, meeting the real-time requirements of drone inspections. This enables drones to quickly analyze images during flight, promptly detect potential faults, defects or abnormalities, and improve inspection efficiency.

[0062] At present, when drones are used for line inspection, visual positioning technology is often used to assist. However, when the positioning of the drone is deviated, the inspection images taken will also be deviated and not meet the specifications. When the traditional YOLOv5 network analyzes them, there will be a problem of low refinement. Therefore, the present invention proposes an improved YOLOV5 network with multiple detection heads and feature comparison layers to perform auxiliary analysis of the route.

[0063] In some embodiments, the improved YOLOV5 network includes a backbone network, a neck network and a head network; the actual route of the UAV is determined according to the preliminary inspection image and the improved YOLOV5 network, including: inputting the preliminary inspection image into the backbone network to obtain feature maps of multiple scales; performing small target recognition on the feature maps of multiple scales according to the first detection head to obtain a first target recognition result; obtaining a fused feature map according to the feature fusion module in the neck network; performing target recognition on the feature fusion map according to the second detection head to obtain a second target recognition result; comparing the first target recognition result and the second target recognition result according to the feature comparison layer in the head network to determine the image recognition error caused by the route deviation; performing positioning analysis according to the image recognition error, the first target recognition result, and the second target recognition result to obtain the actual route of the UAV.

[0064] In an embodiment of the present invention, the backbone network is the infrastructure of the entire target detection network, and its main function is to perform deep feature extraction on the input image. In this scenario, the preliminary inspection image is input into the backbone network, which realizes feature extraction through a series of complex convolutional layers, pooling layers, and possible residual connections. The convolution layer performs convolution operations by sliding the convolution kernel on the image, which can capture local features in the image. Convolution kernels of different sizes can obtain local feature information of different scales. The pooling layer mainly plays the role of downsampling, which can reduce the amount of data while retaining the main features of the image, so that the network can process larger images and speed up the calculation. Residual connections help solve the problem of gradient disappearance in the deep network training process, so that the network can be better trained and converged.

[0065] As these layers in the backbone network are processed sequentially, the original pixel information of the image is gradually converted into feature maps with different levels of semantic information. For example, assuming that the input preliminary inspection image size is a larger resolution (such as 640×640 pixels), after being processed by the backbone network, multiple feature maps of different scales may be generated, such as 80×80, 40×40, 20×20, etc. The semantic information and detail information contained in these feature maps of different scales are different. The larger scale feature map (such as 80×80) is relatively closer to the original image, retains more detail information, and is suitable for detecting relatively small targets; while the smaller scale feature map (such as 20×20) has undergone more downsampling and feature refinement, contains richer semantic information, and is more conducive to detecting relatively large targets.

[0066] In the embodiment of the present invention, the first detection head is a module specially used for identifying small targets. Since small targets occupy a small area in the image, their features are relatively subtle and easy to be ignored, so the design of the first detection head needs to pay more attention to the capture and processing of detail information.

[0067] It receives feature maps of multiple scales generated by the backbone network, and usually focuses more on selecting feature maps of relatively larger scales (such as 80×80 scale feature maps), because these feature maps retain more detail information and are more beneficial for identifying small targets.

[0068] Inside the first detection head, there are usually a series of convolutional layers and other related processing units. After receiving the feature map, it is first further refined through these convolutional layers. The parameters such as the convolution kernel size and step size of these convolutional layers may be optimized according to the characteristics of the small target.

[0069] The first detection head can identify the relevant features of the small target from the feature map, and then classify and judge the identified small target according to the pre-set classification standard and bounding box prediction algorithm, determine the category to which it belongs, and predict the bounding box position of the small target in the image (such as the center point coordinates, width and height) and other information, and finally obtain the first target recognition result. This result includes the category information of the detected small targets and their specific location information in the image.

[0070] In the embodiment of the present invention, the neck network plays a connecting role in the entire target detection network architecture, and is located between the backbone network and the detection head. The feature fusion module is the key, and its main function is to fuse the feature maps of different scales generated by the backbone network, so as to better utilize multi-scale information for target detection.

[0071] Feature maps of different scales have their own advantages and disadvantages. Feature maps of larger scales retain more detail information but relatively less semantic information, while feature maps of smaller scales do the opposite. By fusing them together through the feature fusion module, the fused feature map can contain both rich semantic information and sufficient detail information, thus providing better feature input for subsequent target detection.

[0072] The feature fusion module usually adopts some specific fusion strategies. For example, upsampling and downsampling techniques may be used to enlarge the size of the smaller-scale feature map through upsampling operations, and reduce the size of the larger-scale feature map through downsampling operations, and then splicing or adding the processed feature maps of different scales to achieve the fusion of feature maps of different scales.

[0073] Taking the feature maps of different scales such as 80×80, 40×40, and 20×20 generated by the backbone network mentioned above as an example, through the processing of the feature fusion module, a feature map that combines features of different scales may be obtained. This fused feature map contains both the detail information extracted from the 80×80 feature map and the rich semantic information obtained from the 20×20 feature map, etc., providing a more comprehensive feature basis for subsequent target detection.

[0074] In an embodiment of the present invention, the second detection head is a module for target recognition of the fused feature map processed by the feature fusion module. Since the fused feature map has integrated multi-scale information and has a more comprehensive feature basis, the second detection head is designed to focus more on using these fused features for comprehensive target recognition. It receives the fused feature map generated by the feature fusion module, and further processes and analyzes the fused feature map through a series of internal convolutional layers, other processing units, and special mechanisms that may be used (such as attention mechanisms, etc.).

[0075] After receiving the fused feature map, the convolution layer inside the second detection head will first further refine it and extract features that are more conducive to target recognition. Then, according to the pre-set classification criteria and bounding box prediction algorithm, the target in the fused feature map is classified and judged to determine its category, and the bounding box position of the target in the image (such as center point coordinates, width and height) and other information are predicted, and finally the second target recognition result is obtained. This result contains the category information of the detected targets and their specific location information in the image, etc., which is similar to the first target recognition result, but because it is based on the fused feature map for recognition, it may be different in accuracy and comprehensiveness.

[0076] In some embodiments, the first target recognition result and the second target recognition result are compared according to the feature comparison layer in the head network to determine the image recognition error caused by the route deviation, including: determining the small target in the second target recognition result; determining the image recognition error caused by the route deviation according to the proportion of the small target in the second target recognition result in the first target recognition result.

[0077] Among them, in power line inspection, large targets generally refer to objects or components that are relatively easy to be found and larger in size. Such as the transmission tower itself, large insulator strings, transformers, etc. These targets occupy more pixels in the image, usually have obvious shapes and outlines, and can be easily observed. Small targets usually refer to objects or components that are relatively small in size. For example, small screws on the line. The size of these small targets may range from a few pixels to dozens of pixels, depending on factors such as image resolution and shooting distance.

[0078] In order to obtain the multi-scale features and broader semantic information of the image, the traditional YOLOv5 network will perform multiple downsampling operations. Although downsampling allows the network to better handle targets of different sizes, for small targets, multiple downsampling will cause their feature size to continue to decrease, and a large amount of detail information will be lost, making it difficult for the network to accurately identify them. For example, a small target that is only a few pixels in size may hardly reflect effective features on the feature map after multiple downsampling, making it impossible for the network to accurately determine its position and category. The present invention solves this problem by setting a small target detection head, but when the route deviates, the initial inspection image itself has certain problems, which will also have a certain impact on target recognition. Before the route is corrected, it is usually difficult to quantify this impact.

[0079] The detection head (i.e., the second detection head) of the traditional YOLOv5 network loses small targets due to the above reasons. Especially when the route deviation causes image quality problems, further reducing the resolution will make the details of the small targets more blurred, and this loss problem is more obvious. The present invention takes advantage of this feature, that is, extracting the small targets identified by the second detection head and querying the corresponding preset image recognition error according to the proportion. If the small targets identified by the second detection head account for a large proportion of the first target recognition results, it means that the influence of the route deviation is not great, and the corresponding preset image recognition error is small. When the proportion is small, it means that the influence of the route deviation is not small, and the corresponding preset image recognition error is large.

[0080] In some embodiments, positioning analysis is performed based on image recognition errors, first target recognition results, and second target recognition results to obtain the actual route of the drone, including: using the targets identified in the first target recognition results and the second target recognition results as reference points; calculating the first positioning information of the drone based on the coordinates of each reference point and the distance between each reference point and the drone; and determining the second positioning information of the drone based on the image recognition error and the first positioning information.

[0081] In an embodiment of the present invention, the coordinate value of the drone is calculated by using the four-point positioning method according to the coordinates of each reference point and the distance between each reference point and the drone, which is the first positioning information of the drone. When the identified targets are greater than four, different reference point combinations are used for four-point positioning, and the average of the calculated coordinate values ​​of the drone is the second positioning information of the drone. However, when there are errors, the identified small targets will be inaccurate, affecting the positioning of the drone. The greater the image recognition error, the greater the impact. Therefore, the present invention adopts the method of weighted averaging of each group of reference points to recalculate the coordinates of the drone. The weight of each group of reference points depends on the number of small targets in the group and the image recognition error.

[0082] For example, if all reference points in a group are large targets and their initial weights are all 1, then the weight of the group is 1*4 / 4 = 1. If there are 3 small targets in each reference point in a group and the image recognition error is 0.5, then the weight of the group is (1+3*0.5) / 4 = 0.625.

[0083] In some embodiments, the first positioning information of the drone is calculated based on the coordinates of each reference point and the distance between each reference point and the drone, including: establishing a group of distance equations based on the coordinates of each reference point and the distance between each reference point and the drone; solving the group of distance equations to obtain the coordinates of the drone as the first positioning information.

[0084] In some embodiments, determining the second positioning information of the drone based on the image recognition error and the first positioning information includes: updating the coordinates of the drone based on the image recognition error as the second positioning information.

[0085] S130, determining a gimbal adjustment plan and positioning deviation information of the UAV according to the expected route and the actual route.

[0086] In some embodiments, based on a visual recognition model, the deviation distance between each positioning point on the actual route and each target point on the actual route is identified from a preliminary inspection image; based on the deviation distance, a gimbal adjustment plan and positioning deviation information are determined.

[0087] In an embodiment of the present invention, the visual recognition model can be a deep convolutional neural network. The visual recognition model plays a key role in this method. It is built based on deep learning technology, and after training with a large amount of distribution network line image data, it has the ability to accurately identify various targets (such as poles, insulators, wire connection points, etc.) and positioning point features in the line. When processing the preliminary inspection image, the model accurately determines the relative position relationship between the positioning point and the target point on the actual route based on the feature extraction and pattern recognition algorithm, and calculates the deviation distance between the two. This deviation distance covers horizontal, vertical and depth deviations, and provides key data for subsequent pan-tilt adjustment and positioning deviation determination. For example, the model extracts the target feature map in the image through a convolutional neural network, and after pooling, fully connected layer processing and matching and comparison with predefined target feature templates, the coordinate difference between the positioning point and the target point, that is, the deviation distance, overcomes the limitations of traditional methods that rely on manual marking or simple image processing algorithms, and adapts to complex environments and target diversity challenges. Collect distribution network line image data covering different environments, seasons, time periods and line states, and accurately mark the position and category information of the positioning point and the target point through annotation tools to construct training sets, verification sets and test sets to ensure data diversity and annotation accuracy. During training, the back-propagation algorithm is used to optimize the model parameters. The network weights are adjusted by evaluating the prediction deviation based on the loss function (such as a weighted combination of cross entropy loss and position regression loss). Data enhancement techniques (such as random cropping, rotation, flipping, and brightness contrast adjustment) are introduced to improve the generalization ability of the model. After multiple rounds of iterative training, the model achieves high-precision and stable performance indicators in the test set, laying the foundation for accurate recognition and deviation measurement of actual inspection images.

[0088] The gimbal adjustment plan is determined based on the identified deviation distance. The gimbal controls the camera posture, and its adjustment directly affects the image acquisition angle of view and the target imaging position. If the deviation distance shows that the target point deviates from the expected position in the horizontal direction of the image, the gimbal adjusts the yaw angle to make the camera rotate horizontally to offset the deviation; the vertical deviation is compensated by adjusting the pitch angle; the depth deviation (such as due to inaccurate drone height) is combined with the flight control system command to fine-tune the drone height and coordinately adjust the gimbal angle to maintain clear imaging of the target. The adjustment plan is generated according to the size and direction of the deviation according to predefined rules or adaptive algorithms to ensure that the camera quickly aligns with the target and optimizes the image acquisition quality and positioning accuracy. For example, if the horizontal deviation is set to 5° for every 1 meter of yaw angle adjustment, the required adjustment amount in each direction is accurately calculated, and the adjustment strategy is optimized in real time considering the drone's flight speed, stability and environmental interference factors to avoid excessive or insufficient adjustment causing image jitter and positioning fluctuations.

[0089] The gimbal adjustment algorithm module is embedded in the flight control system. The deviation distance data output by the visual recognition model is received in real time, and the gimbal adjustment angle or displacement in each direction is calculated according to the predefined mapping relationship. Considering the dynamic response characteristics and stability of the drone, the algorithm sets the adjustment threshold and filtering mechanism to prevent small deviations from frequently triggering excessive adjustments and causing gimbal jitter, and adaptively adjusts the proportional coefficient according to the drone's flight speed, acceleration and environmental interference factors to ensure that the gimbal can steadily and accurately compensate for positioning deviations under different working conditions, and maintain stable and clear image acquisition.

[0090] The gimbal adjustment plan includes yaw, pitch and roll adjustment instructions and dynamic parameters such as adjustment speed and acceleration. The plan analyzes the absolute value and direction (clockwise or counterclockwise) of each angle adjustment and the dynamic parameter value. For example, the yaw angle adjustment is +10° (rotate right), the adjustment speed is 5° / s, and the acceleration is 1° / s. 2 ,These parameters reflect the motion trajectory and characteristics of the gimbal and provide the key basis for ,determining the positioning deviation information, because the gimbal posture change is directly related to the ,change of the camera viewing angle and the target imaging position offset, and the ,adjustment of speed and acceleration affects the timeliness and stability of ,positioning compensation.

[0091] Decompose the deviation distance into dx, dy, and dz according to the horizontal (x-axis), vertical (y-axis), and depth (z-axis). If the deviation distance is a spatial vector D = (3,-2,1) meters, then dx = 3 meters, dy = -2 meters, and dz = 1 meter. Perform coordinate transformation based on the conversion relationship between the drone coordinate system and the world coordinate system (including rotation and translation matrices, determined by positioning information, inertial measurement unit data, and initial calibration parameters). For example, the deviation distance in the camera coordinate system is transformed to the world coordinate system through the external parameter matrix to accurately determine the position deviation of the target point relative to the drone in the global positioning framework, laying the foundation for precise navigation and positioning correction. The depth deviation dz is of great significance to the height adjustment of the drone and the 3D space path planning. The horizontal and vertical deviations guide the plane trajectory correction.

[0092] Considering the dynamic process of gimbal adjustment, the positioning deviation compensation is calculated according to the adjustment speed, acceleration and current attitude. Assuming that the initial yaw angle deviation causes the target point to shift horizontally, the compensation effect at different times is calculated according to the adjustment speed and time. If the gimbal adjusts the yaw angle at 5° / s, the theoretical compensation amount of the horizontal target imaging position after 2s is the horizontal displacement compensation corresponding to 10°. The actual pixel or spatial distance compensation value is calculated according to the trigonometric function and the camera field of view angle and focal length. This compensation amount reflects the degree of correction of the positioning deviation by the gimbal adjustment. The adjustment effect is evaluated by comparing with the deviation distance in real time, providing quantitative indicators for the dynamic optimization adjustment strategy, ensuring that the positioning accuracy becomes more accurate and stable with the movement of the gimbal, especially in the case of variable speed inspection of drones or airflow interference in complex environments. It is indispensable to maintain positioning accuracy.

[0093] The positioning information, inertial measurement unit, lidar and other multi-source sensor data are integrated to calibrate the positioning deviation. The positioning information provides the absolute position of the drone, but the accuracy is affected by environmental interference and fluctuations. The IMU outputs high-frequency attitude change data but there are cumulative errors. The lidar measures the distance between obstacles and targets with high precision to build a local environment map to assist in positioning. For example, based on the positioning information, the IMU attitude data compensates for the impact of the drone's instantaneous attitude deviation on positioning, and the lidar detects the terrain and objects around the line to correct the visual positioning deviation caused by terrain undulations or obstacle obstruction. Through Kalman filtering and data fusion algorithms, multi-source data are weightedly fused, errors are corrected and state estimated to generate high-precision calibrated positioning deviation information, improve positioning reliability and robustness, and provide a solid data foundation for reliable decision-making and precise operation of drone intelligent inspections, ensuring the safe and efficient execution of inspection tasks in complex power grid environments.

[0094] Among them, after determining the deviation distance, first determine whether the position of the drone needs to be adjusted. If not, calculate the gimbal adjustment plan based on the deviation distance, and the positioning deviation information is empty. If yes, calculate the positioning deviation information and gimbal adjustment plan based on the deviation distance. That is, if the inspection can be completed by adjusting the gimbal of the drone, the position of the drone will not be adjusted. If the position of the drone needs to be adjusted, only the position of the drone can be adjusted, or the position of the drone and the gimbal can be adjusted at the same time, which is not limited here.

[0095] S140, according to the gimbal adjustment plan and the positioning deviation information, the UAV is controlled to inspect the distribution network line along the expected route.

[0096] In some embodiments, based on the gimbal adjustment plan and positioning deviation information, the drone is controlled to inspect the distribution network line along the expected route, including: determining the adjustment information of the drone based on the gimbal adjustment plan and positioning deviation information; controlling the drone based on the adjustment information so that the drone inspects the distribution network line along the expected route.

[0097] In an embodiment of the present invention, the positioning deviation information includes the deviation distance between each positioning point on the actual route and the corresponding ideal position point on the expected route. By analyzing these deviation distances, the degree and direction of the deviation of the drone from the expected route at different positions can be understood. For example, if the deviation distance of a certain positioning point is large, it means that the drone deviates far from the expected route at this position; the direction of the deviation distance can be determined by comparing the coordinate difference between the positioning point and the corresponding ideal position point. For example, in a two-dimensional plane, if the horizontal coordinate of the positioning point is greater than the horizontal coordinate of the ideal position point, it means that the drone has deviated from the expected route to the right in the horizontal direction.

[0098] Determine the position adjustment strategy: Based on the analysis of the positioning deviation information, formulate the corresponding position adjustment strategy to determine the position adjustment information. Common adjustment strategies are as follows:

[0099] Direction adjustment: Determine the flight direction that the drone needs to adjust based on the direction of the deviation distance. If the drone deviates to the right from the expected route, the position adjustment information should include instructions to adjust the flight direction to the left. The specific adjustment angle can be determined based on factors such as the degree of deviation. For example, a smaller deviation can be set to a smaller adjustment angle, while a larger deviation may require a larger adjustment angle. The specific adjustment angle value can usually be determined through a certain proportional relationship or preset adjustment rules.

[0100] Speed ​​adjustment: In some cases, in addition to adjusting the direction, the flight speed of the drone may also need to be adjusted. If the drone deviates far from the desired route, in order to return to the desired route as soon as possible, the flight speed can be appropriately increased; conversely, if the deviation is small, the original speed may be maintained or the speed may be appropriately reduced for fine adjustment. The specific value of the speed adjustment can also be determined based on factors such as the deviation distance and the urgency of the mission.

[0101] Altitude adjustment: For some distribution network line inspection tasks that require flying at a specific altitude, altitude adjustment is required when the positioning deviation information shows that the drone has also deviated in the altitude direction. For example, if the drone is below the expected flight altitude, the position adjustment information should include instructions to increase to the expected altitude, and the specific increase value is determined based on the actual deviation.

[0102] In some embodiments, the method further includes: determining a feature screening accuracy based on the positioning information of the drone and a preliminary inspection image; and selecting a target attention mechanism from preset attention mechanisms based on the feature screening accuracy.

[0103] In some embodiments, a target attention mechanism is selected from preset attention mechanisms based on the feature screening accuracy, including: if the feature screening accuracy is greater than a first preset threshold, the convolution block attention module is used as the target attention mechanism; if the feature screening accuracy is not greater than the first preset threshold, the efficient channel attention module is used as the target attention mechanism.

[0104] In an embodiment of the present invention, the Convolutional Block Attention Module (CBAM) is a mechanism that combines channel attention and spatial attention. First, different weights are assigned to different channels of the feature map through the channel attention module to highlight important channel information; then the spatial attention module is used to give different degrees of attention to different spatial positions of the feature map to emphasize important spatial areas. It can screen and weight features from two dimensions, channel and space, to better focus on the key features of the target, reduce attention to irrelevant information such as background, and effectively improve detection accuracy. However, drone inspections have high requirements for detection speed, and the additional calculations they bring may have a certain impact.

[0105] The Efficient Channel Attention (ECA) module focuses on information in the channel dimension. The weight of each channel is obtained through a fast one-dimensional convolution operation, so as to adaptively adjust the importance of the channel. This method can effectively capture the importance differences between different channels without increasing the amount of calculation. It has high computational efficiency and can enhance the model's ability to extract important features while maintaining a high detection speed. Giving greater weight to channels with important information in the feature map helps the model pay better attention to key information. However, it pays less attention to spatial information, and in some complex backgrounds, it may not fully utilize spatial location information.

[0106] According to the positioning information and preliminary inspection images of the drone, the approximate location of the current drone and the number of small targets at the current location can be determined. The feature screening accuracy is determined according to the number of small targets. The larger the number of small targets, the higher the feature screening accuracy. Using the feature screening accuracy, attention mechanisms with different computational amounts can be used to participate in the calculation. For example, when the feature screening accuracy is greater than the first preset threshold, the convolution block attention module is used as the target attention mechanism; if the feature screening accuracy is not greater than the first preset threshold, the efficient channel attention module is used as the target attention mechanism.

[0107] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0108] Figure 2 1 is a schematic diagram of the structure of the distribution network line drone inspection auxiliary photography device based on the improved YOLOV5 provided by an embodiment of the present invention. Figure 2 As shown, in some embodiments, a distribution network line drone inspection auxiliary photography device 2 based on improved YOLOV5 includes:

[0109] The initial inspection module 210 is used to control the UAV to perform a preliminary inspection on the distribution network line according to the expected route and record the preliminary inspection image;

[0110] The route calculation module 220 is used to determine the actual route of the drone based on the preliminary inspection image and the improved YOLOV5 network; wherein the improved YOLOV5 network is provided with at least two detection heads and a feature comparison layer; the feature comparison layer is used to compare the feature differences recognized by each detection head, and the feature differences represent the image recognition errors caused by the route deviation;

[0111] The deviation determination module 230 is used to determine the gimbal adjustment plan and positioning deviation information of the UAV according to the expected route and the actual route;

[0112] The inspection control module 240 is used to control the UAV to inspect the distribution network line according to the expected route according to the gimbal adjustment plan and positioning deviation information.

[0113] Optionally, the improved YOLOV5 network includes a backbone network, a neck network and a head network; the route calculation module 220 is used to: input the preliminary inspection image into the backbone network to obtain feature maps of multiple scales; perform small target recognition on the feature maps of multiple scales according to the first detection head to obtain a first target recognition result; obtain a fused feature map according to the feature fusion module in the neck network; perform target recognition on the feature fusion map according to the second detection head to obtain a second target recognition result; compare the first target recognition result and the second target recognition result according to the feature comparison layer in the head network to determine the image recognition error caused by the route deviation; perform positioning analysis based on the image recognition error, the first target recognition result and the second target recognition result to obtain the actual route of the drone.

[0114] Optionally, the route calculation module 220 is used to: use the targets identified in the first target recognition result and the second target recognition result as reference points; calculate the first positioning information of the drone based on the coordinates of each reference point and the distance between each reference point and the drone; and determine the second positioning information of the drone based on the image recognition error and the first positioning information.

[0115] Optionally, the route calculation module 220 is used to: establish a distance equation group according to the coordinates of each reference point and the distance between each reference point and the drone; solve the distance equation group to obtain the coordinates of the drone as the first positioning information.

[0116] Optionally, the route calculation module 220 is used to update the coordinates of the drone according to the image recognition error as the second positioning information.

[0117] Optionally, the deviation determination module 230 is used to: identify the deviation distance between each positioning point on the actual route and each target point on the actual route from the preliminary inspection image according to the visual recognition model; and determine the gimbal adjustment plan and positioning deviation information according to the deviation distance.

[0118] Optionally, the inspection control module 240 is used to: determine the adjustment information of the drone according to the gimbal adjustment plan and the positioning deviation information; and control the drone according to the adjustment information so that the drone inspects the distribution network line according to the expected route.

[0119] Optionally, the device also includes: a screening module, used to determine the feature screening accuracy based on the positioning information of the drone and the preliminary inspection image; and select a target attention mechanism from preset attention mechanisms based on the feature screening accuracy.

[0120] Optionally, a screening module is used to: if the feature screening accuracy is greater than a first preset threshold, use the convolution block attention module as the target attention mechanism; if the feature screening accuracy is not greater than the first preset threshold, use the efficient channel attention module as the target attention mechanism.

[0121] The distribution network line drone inspection auxiliary photography device based on improved YOLOV5 provided in this embodiment can be used to execute the above method embodiments. Its implementation principle and technical effects are similar, and this embodiment will not be repeated here.

[0122] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0123] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0124] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention.

Claims

1. A distribution network line drone inspection auxiliary photography method based on improved YOLOV5, characterized in that: include: Control the drone to conduct preliminary inspection of the distribution network line according to the expected route and record the preliminary inspection images; Determine the actual route of the drone based on the preliminary inspection image and the improved YOLOV5 network; wherein the improved YOLOV5 network is provided with at least two detection heads and a feature comparison layer; the feature comparison layer is used to compare the feature differences recognized by each detection head, and the feature differences represent the image recognition errors caused by the route deviation; Determining a gimbal adjustment plan and positioning deviation information of the UAV according to the expected route and the actual route; According to the gimbal adjustment plan and the positioning deviation information, the drone is controlled to inspect the distribution network line along the expected route.

2. The distribution network line drone inspection auxiliary photography method based on improved YOLOV5 according to claim 1 is characterized in that: The improved YOLOV5 network includes a backbone network, a neck network and a head network; Based on the preliminary inspection images and the improved YOLOV5 network, the actual route of the drone is determined, including: Inputting the preliminary inspection image into the backbone network to obtain feature maps of multiple scales; Performing small target recognition on feature maps of multiple scales according to the first detection head to obtain a first target recognition result; According to the feature fusion module in the neck network, a fused feature map is obtained; Performing target recognition on the feature fusion image according to the second detection head to obtain a second target recognition result; Comparing the first target recognition result with the second target recognition result according to the feature comparison layer in the head network to determine the image recognition error caused by the route deviation; Positioning analysis is performed based on the image recognition error, the first target recognition result, and the second target recognition result to obtain the actual route of the UAV.

3. The distribution network line drone inspection auxiliary photography method based on improved YOLOV5 according to claim 2 is characterized in that: Based on the image recognition error, the first target recognition result, and the second target recognition result, positioning analysis is performed to obtain the actual route of the drone, including: Using the targets identified in the first target recognition result and the second target recognition result as reference points; Calculate the first positioning information of the UAV according to the coordinates of each reference point and the distance between each reference point and the UAV; The second positioning information of the UAV is determined according to the image recognition error and the first positioning information.

4. The distribution network line drone inspection auxiliary photography method based on improved YOLOV5 according to claim 3 is characterized in that: According to the coordinates of each reference point and the distance between each reference point and the drone, the first positioning information of the drone is calculated, including: According to the coordinates of each reference point and the distance between each reference point and the UAV, a distance equation group is established; The distance equation group is solved to obtain the coordinates of the UAV as the first positioning information.

5. The distribution network line drone inspection auxiliary photography method based on improved YOLOV5 according to claim 4 is characterized in that: Determining second positioning information of the drone according to the image recognition error and the first positioning information includes: The coordinates of the drone are updated according to the image recognition error as second positioning information.

6. The distribution network line drone inspection auxiliary photography method based on improved YOLOV5 according to claim 1 is characterized in that: According to the expected route and the actual route, a gimbal adjustment scheme and positioning deviation information of the UAV are determined, including: According to the visual recognition model, identifying the deviation distance between each positioning point on the actual route and each target point on the actual route from the preliminary inspection image; According to the deviation distance, a gimbal adjustment plan and positioning deviation information are determined.

7. The distribution network line drone inspection auxiliary photography method based on improved YOLOV5 according to claim 1 is characterized in that: According to the gimbal adjustment scheme and the positioning deviation information, the drone is controlled to inspect the distribution network line according to the expected route, including: Determining adjustment information of the drone according to the gimbal adjustment scheme and the positioning deviation information; The UAV is controlled according to the adjustment information so that the UAV inspects the distribution network line according to the expected route.

8. The distribution network line drone inspection auxiliary photography method based on improved YOLOV5 according to claim 1 is characterized in that: The method further comprises: Determining feature screening accuracy based on the positioning information of the drone and the preliminary inspection image; According to the feature screening accuracy, a target attention mechanism is selected from preset attention mechanisms.

9. The distribution network line drone inspection auxiliary photography method based on improved YOLOV5 according to claim 8 is characterized in that: According to the feature screening accuracy, a target attention mechanism is selected from preset attention mechanisms, including: If the feature screening accuracy is greater than a first preset threshold, the convolutional block attention module is used as a target attention mechanism; If the feature screening accuracy is not greater than the first preset threshold, the efficient channel attention module is used as the target attention mechanism.

10. A distribution network line drone inspection auxiliary photography device based on improved YOLOV5, characterized in that: include: The initial inspection module is used to control the UAV to conduct a preliminary inspection of the distribution network line according to the expected route and record the preliminary inspection images; A route calculation module, used to determine the actual route of the drone based on the preliminary inspection image and the improved YOLOV5 network; wherein the improved YOLOV5 network is provided with at least two detection heads and a feature comparison layer; the feature comparison layer is used to compare the feature differences recognized by each detection head, and the feature differences represent the image recognition errors caused by the route deviation; A deviation determination module, used to determine a gimbal adjustment plan and positioning deviation information of the UAV according to the expected route and the actual route; The inspection control module is used to control the UAV to inspect the distribution network line along the expected route according to the gimbal adjustment plan and the positioning deviation information.

Images