Inspection correction method and system of electric power inspection unmanned aerial vehicle, terminal equipment and storage medium

Abstract

The invention discloses an inspection correction method and system for an electric power inspection unmanned aerial vehicle, terminal equipment and a storage medium, and belongs to the field of inspection correction. Waypoint omission or redundancy caused by manual route setting is avoided, and the path feasibility is guaranteed; the substation monitoring camera is used for collecting inspection images, contour recognition is carried out through the target detection algorithm model, an unmanned aerial vehicle does not need to carry a high-load visual module, and the problem of low image quality caused by large load of airborne equipment is solved; the two-dimensional coordinates of the key structure points of the unmanned aerial vehicle are obtained through the key point detection model, the pose is solved by combining the three-dimensional coordinates and the parameters of the monitoring camera, the pose of the unmanned aerial vehicle is adjusted according to the error between the current pose and the historical optimal pose, the accurate shooting pose is ensured, and the reliability of the inspection result of the unmanned aerial vehicle is improved.

Description

Technical Field

[0001] This invention relates to the field of inspection and calibration, and in particular to an inspection and calibration method, system, terminal equipment and storage medium for a power line inspection drone. Background Art

[0002] With the increasing demand for intelligent operation and maintenance of power systems, drone inspection technology has been widely used in substation equipment inspection. Compared with traditional manual inspection, drones have advantages such as non-contact operation, wide coverage, and high inspection efficiency, which can significantly reduce labor costs and environmental interference, and are especially suitable for equipment status monitoring in high-voltage and complex terrain scenarios.

[0003] Currently, existing drone inspection technologies mainly rely on manually setting flight paths for inspections, and onboard vision modules for target recognition and attitude adjustment to enable autonomous adjustments during inspections. However, existing drone inspection technologies suffer from several problems: manually setting flight paths can easily lead to missed or redundant waypoints, and the onboard vision modules can result in low-quality inspection images due to the heavy load on the drone's onboard equipment and flight vibrations, thus affecting the inspection results. Summary of the Invention

[0004] This invention provides a method, system, terminal equipment, and storage medium for power line inspection drones. It can solve the problems of existing drone inspection technologies, such as the easy omission or redundancy of waypoints due to manual route setting, and the low quality of inspection images under flight vibration caused by the heavy load of onboard equipment due to the visual module mounted on the drone, which in turn affects the inspection results.

[0005] To address the aforementioned technical problems, an embodiment of the present invention provides an inspection and calibration method for a power line inspection drone, comprising:

[0006] Acquire historical inspection data of substations by drones and construct a set of substation equipment inspection routes; the historical inspection data includes drone flight path data, inspection point data, flight data and inspection shooting point data;

[0007] Based on the set of substation equipment inspection routes, a path planning algorithm is used, combined with a pre-constructed objective function and constraints that minimize flight resource consumption, to generate the latest UAV cruise route.

[0008] The inspection images of the drone are obtained by the substation monitoring camera at the inspection point when the drone is inspecting according to the latest drone cruise route. The inspection images of the drone are then input into the target detection algorithm model so that the target detection algorithm model can perform drone contour image recognition based on the drone inspection images to obtain the current drone contour image.

[0009] The current drone's contour image is input into the key point detection model, so that the key point detection model can identify the key structural points of the drone based on the current drone's contour image and obtain the two-dimensional coordinates of the key structural points of the current drone.

[0010] Based on the two-dimensional coordinates of the key structural points of the current UAV, determine the three-dimensional coordinates of the key structural points of the current UAV, and solve the pose of the current UAV based on the two-dimensional coordinates, three-dimensional coordinates and monitoring camera parameters.

[0011] Based on the current pose of the drone and the best pose of the drone in history, the pose error is calculated, and the pose of the current drone is adjusted according to the pose error to obtain the best pose of the current drone.

[0012] Furthermore, the constraints include range constraints, path constraints, and task coverage constraints;

[0013] The expression for the objective function that minimizes flight resource consumption is as follows:

[0014]

[0015] Where, min J is the objective function value that minimizes flight resource consumption; α, β, and λ represent the weighting coefficients for flight time, energy consumption, and mission completion priority, respectively; T total The total flight time required for the drone to complete the entire path. k is the number of nodes in the path when the drone executes the complete path, t i，i+1 E represents the time it takes for the drone to fly between the two inspection points. total This represents the total energy consumed by the drone during its path. c i，i+1 The energy consumed by the drone flying between the two inspection points; ω i Let be the task priority weight for the i-th inspection point; n is the number of inspection points that need to be covered; c i c represents the coverage status of the i-th inspection point in the inspection route. i =1 indicates that it has been covered, otherwise it is 0;

[0016] The expression for the range constraint is:

[0017] D total ≤D max ,T total ≤T max ;

[0018] Among them, D total T represents the actual total flight distance of the UAV's planned execution path; total D represents the actual total flight time of the UAV executing the planned path;max T represents the maximum flight distance of the drone. max This refers to the maximum flight time of the drone;

[0019] The expression for the path constraint is:

[0020]

[0021] Where L is the set of route paths; G is the set of substation equipment inspection routes; V is the set of all feasible inspection points; E is the set of all feasible route paths; P is the set of inspection points, p n This is the nth inspection point;

[0022] The expression for the task coverage constraint is:

[0023]

[0024] Wherein, τ is a high-priority threshold; Let be the task priority weight for the i-th inspection point.

[0025] Furthermore, the step of inputting the current UAV inspection image into the target detection algorithm model, so that the target detection algorithm model performs UAV contour image recognition based on the current UAV inspection image to obtain the current UAV contour image, includes:

[0026] The inspection images of the current UAV captured by the substation monitoring camera at each inspection point are input into the target detection algorithm model based on the YOLOv5 algorithm. The Backbone module, Neck module and Head module of the target detection algorithm model based on the YOLOv5 algorithm perform feature extraction, feature fusion and contour localization on the inspection images of the UAV, respectively, to obtain the contour image of the UAV captured at each inspection point.

[0027] Furthermore, the step of inputting the current UAV contour image into the key point detection model, so that the key point detection model can identify the key structural points of the UAV based on the current UAV contour image, and obtain the two-dimensional coordinates of the key structural points of the current UAV, includes:

[0028] By using a key point detection model based on the HRNet algorithm to sequentially perform deep feature extraction, feature fusion, and key structural point identification on the contour image of the UAV, the two-dimensional coordinates of the key structural points of the UAV are obtained.

[0029] Furthermore, the step of solving the pose of the current UAV based on the two-dimensional coordinates and three-dimensional coordinates of the key structural points and the monitoring camera parameters includes:

[0030] Based on the two-dimensional coordinates and three-dimensional coordinates of the key structural points of the current UAV and the parameters of the monitoring camera, the EPnP algorithm is used to initially solve the pose of the current UAV, and the initial pose of the current UAV is obtained.

[0031] Based on the two-dimensional coordinates and three-dimensional coordinates of the key structural points of the current UAV, the monitoring camera parameters, and the initial pose of the current UAV, the RANSAC algorithm is used to optimize the pose of the current UAV to obtain the final pose of the current UAV.

[0032] Furthermore, the calculation of pose error based on the current UAV pose and the historical best UAV pose includes:

[0033] The translation vector error of the current UAV's pose is obtained by calculating the difference between the translation vector of the current UAV's final pose and the translation vector of the historical UAV's best pose.

[0034] The rotation vector of the current UAV's final pose and the rotation vector of the historical UAV's best pose are transformed into unit quaternions to obtain the rotation vector quaternions of the current UAV and the historical UAV. Based on the rotation vector quaternions of the current UAV and the historical UAV, the rotation vector quaternion error of the current UAV is calculated.

[0035] Furthermore, adjusting the current UAV pose based on the pose error to obtain the optimal pose of the current UAV includes:

[0036] Based on the translation vector error of the current UAV pose, the expected acceleration of the current UAV is calculated using the PID algorithm, and the current UAV is controlled to make translation adjustments based on the expected acceleration to obtain the current UAV pose after translation adjustment.

[0037] Based on the current rotation vector quaternion error of the UAV, calculate the expected angular velocity of the current UAV, and adjust the roll, pitch and yaw of the current UAV according to the expected angular velocity to obtain the current UAV attitude after rotation adjustment.

[0038] The optimal pose of the current UAV is obtained based on the current UAV pose after translation adjustment and the current UAV pose after rotation adjustment.

[0039] Based on the above method embodiments, the present invention provides corresponding system embodiments;

[0040] One embodiment of the present invention provides an inspection and correction system for a power line inspection drone, comprising: a data acquisition module, a cruise route generation module, a contour recognition module, a key structural point recognition module, a pose solving module, and a pose adjustment module;

[0041] The data acquisition module is used to acquire historical inspection data of the substation by drone and construct a set of substation equipment inspection routes; wherein, the historical inspection data includes drone flight path data, inspection point data, flight data and inspection shooting point data;

[0042] The cruise route generation module is used to generate the latest UAV cruise route based on the set of substation equipment inspection routes, using a path planning algorithm, and combining a pre-built objective function and constraints that minimize flight resource consumption.

[0043] The contour recognition module is used to acquire inspection images of the UAV when it is conducting inspections according to the latest UAV cruise route through the substation monitoring camera at the inspection shooting point, and input the UAV inspection images into the target detection algorithm model so that the target detection algorithm model can perform UAV contour image recognition based on the UAV inspection images to obtain the current UAV contour image.

[0044] The key structural point recognition module is used to input the contour image of the current UAV into the key point detection model, so that the key point detection model can identify the key structural points of the UAV based on the contour image of the current UAV and obtain the two-dimensional coordinates of the key structural points of the current UAV.

[0045] The pose solving module is used to determine the three-dimensional coordinates of the key structural points of the current UAV based on the two-dimensional coordinates of the key structural points of the current UAV, and to solve the pose of the current UAV based on the two-dimensional coordinates of the key structural points, the three-dimensional coordinates of the key structural points of the current UAV, and the monitoring camera parameters.

[0046] The pose adjustment module is used to calculate the pose error based on the current pose of the UAV and the best pose of the UAV in history, and adjust the pose of the current UAV based on the pose error to obtain the best pose of the current UAV.

[0047] Based on the above-described method embodiments, another embodiment of the present invention provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements an inspection and correction method for a power line inspection drone as described in the above embodiments.

[0048] Based on the above-described method embodiments, another embodiment of the present invention provides a computer-readable storage medium, the computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to execute the inspection and calibration method of a power line inspection drone described in the above embodiments.

[0049] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0050] This invention acquires historical inspection data of substations using drones and constructs a route set. It then utilizes a path planning algorithm combined with a function and constraints aimed at minimizing flight resource consumption to generate the latest cruise route. This optimizes flight energy consumption while ensuring route feasibility and avoids the problem of missed or redundant waypoints caused by manually setting flight paths. Inspection images are captured by substation monitoring cameras at the shooting points, and contour recognition is performed using a target detection algorithm model. This eliminates the need for a high-load vision module on the drone, solving the problem of low image quality due to flight vibration caused by heavy loads on airborne equipment. A key point detection model obtains the two-dimensional coordinates of key structural points on the drone, thereby determining their three-dimensional coordinates. The drone's pose is then calculated based on the three-dimensional coordinates and monitoring camera parameters, and the drone's pose is adjusted according to the error between the current pose and the historical best pose, ensuring the accuracy of the shooting pose. This improves the quality of inspection images and the reliability of inspection results, solving the problems of missed or redundant waypoints due to manual route setting in existing drone inspection technologies, and low image quality due to flight vibration caused by heavy loads on airborne equipment caused by the vision module on the drone, thus affecting the inspection results. Attached Figure Description

[0051] Figure 1 A flowchart illustrating the steps of a power line inspection drone's inspection and calibration method, as provided in this embodiment of the invention.

[0052] Figure 2 This is a block diagram of an inspection and calibration system for a power line inspection drone provided in an embodiment of the present invention. Detailed Implementation

[0053] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0054] Example 1:

[0055] Reference Figure 1 This is a flowchart illustrating the steps of a power line inspection drone calibration method according to an embodiment of the present invention. To address the problems of existing drone inspection technologies where manual route setting can easily lead to missed or redundant waypoints, and the low image quality under flight vibration due to the heavy load of onboard equipment caused by the onboard vision module, thus affecting inspection results, this method includes at least the following steps:

[0056] Step S1: Obtain historical inspection data of substations by drone and construct a set of substation equipment inspection routes; the historical inspection data includes drone flight path data, inspection point data, flight data and inspection shooting point data;

[0057] In this embodiment, historical drone inspection mission route data, inspection point data, and flight data are obtained from the drone flight log; and inspection shooting point data are obtained through the substation monitoring and shooting system.

[0058] Step S2: Based on the set of substation equipment inspection routes, use the path planning algorithm and combine it with the pre-constructed objective function and constraints that minimize flight resource consumption to generate the latest UAV cruise route;

[0059] In this embodiment, the constraints include endurance constraints, path constraints, and task coverage constraints;

[0060] The expression for the objective function that minimizes flight resource consumption is as follows:

[0061]

[0062] Where, min J is the objective function value that minimizes flight resource consumption; α, β, and λ represent the weighting coefficients for flight time, energy consumption, and mission completion priority, respectively; T total The total flight time required for the drone to complete the entire path. k is the number of nodes in the path when the drone executes the complete path, t i，j+1 E represents the time it takes for the drone to fly between the two inspection points. total This represents the total energy consumed by the drone during its path. c i，j+1 The energy consumed by the drone flying between the two inspection points; ω i Let be the task priority weight for the i-th inspection point; n is the number of inspection points that need to be covered; c i c represents the coverage status of the i-th inspection point in the inspection route. i =1 indicates that it has been covered, otherwise it is 0;

[0063] The expression for the range constraint is:

[0064] D total ≤D max ,T total ≤T max ;

[0065] Among them, D total T represents the actual total flight distance of the UAV's planned execution path; total D represents the actual total flight time of the UAV executing the planned path; maxT represents the maximum flight distance of the drone. max This refers to the maximum flight time of the drone;

[0066] The expression for the path constraint is:

[0067]

[0068] Where L is the set of route paths; G is the set of substation equipment inspection routes; V is the set of all feasible inspection points; E is the set of all feasible route paths; P is the set of inspection points, p n This is the nth inspection point;

[0069] In this embodiment, the path constraint is that all flight segments must be selected from the substation equipment inspection route set constructed in step S1, i.e., the path set. The set of substation equipment inspection routes is G = (V, E); each edge e ij The attribute ∈E is distance d ij Expected time t ij And the expected energy consumption c ij Inspection point assembly Each point p i Additional priority ω i .

[0070] The expression for the task coverage constraint is:

[0071]

[0072] Wherein, τ is a high-priority threshold; Let be the task priority weight for the i-th inspection point.

[0073] In this embodiment, the path planning algorithm adopts the A* search algorithm; the flight data includes the flight time, energy consumption, flight distance of the UAV to each inspection point, and the task completion priority of the inspection point; the step of generating the latest UAV cruise route based on the substation equipment inspection route set, using the path planning algorithm, and combining a pre-constructed objective function and constraints that minimize flight resource consumption, is as follows:

[0074] Based on the inspection points, flight time, energy consumption, flight distance, and task completion priority of each historical route in the substation equipment inspection route set, convert it into a graph structure of the substation equipment inspection route set.

[0075] Based on flight data and the objective function expression, the actual cost function and the heuristic cost function of the A* search algorithm are constructed respectively; wherein, the actual cost function is used to calculate the cumulative flight resource consumption from the starting point to the current node; the heuristic cost function is used to calculate the remaining cost from the current node to the destination, that is, to calculate the remaining flight resource consumption from the current node to the destination.

[0076] In this embodiment, the expression for the actual cost function is: Where g(n) is the actual cost from the origin to node n, T(n) is the sum of flight times from the origin to node n across all historical routes, E(n) is the sum of energy consumption from the origin to node n across all historical routes, S(n) is the set of high-priority checkpoints already covered by the current path, and n is the number of checkpoints that need to be covered; the expression for the heuristic cost function is: Where h(n) is the remaining cost from the current node n to the destination, and T 剩余 (n) represents the sum of the remaining flight times from the current node n to the destination across all historical flight paths, E 剩余 (n) represents the sum of remaining energy consumption from the current node n to the destination in all historical routes, and τ is the high-priority threshold.

[0077] Based on the actual cost function and heuristic cost function of the A* search algorithm, and combined with the inspection constraints, the inspection point search is performed from the graph structure of the substation equipment inspection route set to generate the target UAV cruise route that minimizes flight resource consumption under the constraints. The target UAV cruise route is then used as the latest UAV cruise route.

[0078] Step S3: Obtain inspection images of the drone during its latest patrol route from the substation monitoring camera at the inspection shooting point, and input the drone's inspection images into the target detection algorithm model so that the target detection algorithm model can perform drone contour image recognition based on the drone's inspection images to obtain the current drone contour image.

[0079] In this embodiment, the step of inputting the current UAV inspection image into the target detection algorithm model, so that the target detection algorithm model performs UAV contour image recognition based on the current UAV inspection image to obtain the current UAV contour image, includes:

[0080] The inspection images of the current UAV captured by the substation monitoring camera at each inspection point are input into the target detection algorithm model based on the YOLOv5 algorithm. The Backbone module, Neck module and Head module of the target detection algorithm model based on the YOLOv5 algorithm perform feature extraction, feature fusion and contour localization on the inspection images of the UAV, respectively, to obtain the contour image of the UAV captured at each inspection point.

[0081] In this embodiment, the Backbone module is a deep multi-scale feature extraction based on C2f+Res-TFM, supporting spatial and channel attention fusion; the Neck module is an improved FPN+BiFPN structure, introducing depthwise separable convolution to reduce computation; the Head module is an Anchor-Free dynamic center region prediction, outputting bounding boxes + class confidence + multi-label state attributes (such as whether it is damaged, pose tilt, etc.).

[0082] In this embodiment, the model training of the target detection algorithm model based on the YOLOv5 algorithm includes:

[0083] Acquire drone inspection images with drone outline markings;

[0084] The drone inspection image with drone outline markings is input into the target detection algorithm model based on YOLOv5. The Backbone module, Neck module and Head module of the target detection algorithm model based on YOLOv5 are used to extract features, fuse features and locate outlines of the drone inspection image with drone outline markings, respectively, to obtain the drone outline recognition image.

[0085] The contour recognition image is compared with the UAV contour marker to calculate the loss value. Based on the loss value, the parameters of the target detection algorithm model based on YOLOv5 are optimized until the loss value converges, thus obtaining the trained target detection algorithm model based on YOLOv5.

[0086] Step S4: Input the contour image of the current UAV into the key point detection model so that the key point detection model can identify the key structural points of the UAV based on the contour image of the current UAV and obtain the two-dimensional coordinates of the key structural points of the current UAV.

[0087] In this embodiment, the step of inputting the contour image of the current UAV into the key point detection model, so that the key point detection model can identify the key structural points of the UAV based on the contour image of the current UAV and obtain the two-dimensional coordinates of the key structural points of the current UAV, includes:

[0088] By using a key point detection model based on the HRNet algorithm to sequentially perform deep feature extraction, feature fusion, and key structural point identification on the contour image of the UAV, the two-dimensional coordinates of the key structural points of the UAV are obtained.

[0089] In this embodiment, the model training of the keypoint detection model based on the HRNet algorithm includes:

[0090] The UAV inspection image with key structural point coordinates marked is input into the key point detection model based on the HRNet algorithm. The key point detection model based on the HRNet algorithm performs deep feature extraction, feature fusion and key structural point recognition on the UAV inspection image in sequence to obtain the two-dimensional coordinates of the key structural points in the UAV inspection image.

[0091] The loss value is calculated by comparing the two-dimensional coordinates of key structural points in the UAV inspection image with the coordinate markers of the key structural points. Based on the loss value, the parameters of the key point detection model based on the HRNet algorithm are optimized until the loss value converges, thus obtaining the trained key point detection model based on the HRNet algorithm.

[0092] In this embodiment, the key structural points of the drone include, but are not limited to, the arms, the propeller center, and the light markings.

[0093] Step S5: Determine the three-dimensional coordinates of the key structural points of the current UAV based on the two-dimensional coordinates of the key structural points of the current UAV, and solve the pose of the current UAV based on the two-dimensional coordinates of the key structural points, the three-dimensional coordinates of the key structural points of the current UAV, and the monitoring camera parameters.

[0094] In this embodiment, based on the two-dimensional coordinates of the key structural points of the current UAV, the CAD model or sparse point cloud data of the UAV model is retrieved from the database to obtain the precise three-dimensional coordinates of the two-dimensional coordinates of the key structural points of the current UAV in the UAV body coordinate system, thus obtaining the three-dimensional coordinates of the key structural points of the current UAV.

[0095] In this embodiment, the method of solving the pose of the current UAV based on the two-dimensional coordinates and three-dimensional coordinates of the key structural points and the monitoring camera parameters includes:

[0096] Based on the two-dimensional coordinates and three-dimensional coordinates of the key structural points of the current UAV and the parameters of the monitoring camera, the EPnP algorithm is used to initially solve the pose of the current UAV, and the initial pose of the current UAV is obtained.

[0097] Based on the two-dimensional coordinates and three-dimensional coordinates of the key structural points of the current UAV, the monitoring camera parameters, and the initial pose of the current UAV, the RANSAC algorithm is used to optimize the pose of the current UAV to obtain the final pose of the current UAV.

[0098] In this embodiment, the specific process of solving the final pose of the current UAV using the EPnP algorithm and the RANSAC algorithm is as follows:

[0099] ① Input data

[0100] Two-dimensional coordinates of key points in image space: (u i ,vi ); The three-dimensional coordinate set of the corresponding key structural points in the spatial reference: (X i ,Y i Z i ); Monitoring camera intrinsic parameter matrix K;

[0101] ② PnP model construction:

[0102] Using the camera projection formula:

[0103]

[0104] Where [R|T] is the rotation vector and translation vector relative to the world coordinate system;

[0105] ③EPnP solution:

[0106] 3D points are represented using control point parameterization; the problem is converted into a linear system to solve the least squares problem; finally, the rotation R and translation T are solved, which are the pose of the UAV in the camera coordinate system, and the pose includes the rotation vector R and the translation vector T.

[0107] ④ RANSAC optimization:

[0108] At key points where mismatches may exist, iteratively random sample subsets are used; pose is estimated and projection error is calculated to eliminate outliers and improve robustness.

[0109] Step S6: Calculate the pose error based on the current UAV pose and the best pose of the historical UAVs, and adjust the current UAV pose according to the pose error to obtain the best pose of the current UAV.

[0110] In this embodiment, calculating the pose error based on the current UAV pose and the best pose of historical UAVs includes:

[0111] The translation vector error of the current UAV's pose is obtained by calculating the difference between the translation vector of the current UAV's final pose and the translation vector of the historical UAV's best pose.

[0112] In this embodiment, the formula for calculating the translation vector error is ep = Tc - Th; where ep is the translation vector error, Tc is the translation vector of the current UAV's final pose, and Th is the translation vector of the historical UAV's best pose.

[0113] The rotation vector of the current UAV's final pose and the rotation vector of the historical UAV's best pose are converted into unit quaternions to obtain the rotation vector quaternions of the current UAV and the historical UAV. Based on the rotation vector quaternions of the current UAV and the historical UAV, the rotation vector quaternion error of the current UAV is calculated.

[0114] In this embodiment, the formula for calculating the quaternion error of the rotation vector is: That is, multiplying the inverse quaternion of the current attitude by the quaternion of the desired attitude to obtain q. err This represents the rotation from the current position to the target; where q c Let q be the quaternion of the current UAV's rotation vector; h The rotation vector quaternion of the historical drone; This represents quaternion multiplication, indicating rotational superposition.

[0115] In this embodiment, adjusting the current UAV pose based on the pose error to obtain the optimal pose of the current UAV includes:

[0116] Based on the translation vector error of the current UAV pose, the expected acceleration of the current UAV is calculated using the PID algorithm, and the current UAV is controlled to make translation adjustments based on the expected acceleration to obtain the current UAV pose after translation adjustment.

[0117] In this embodiment, the formula for calculating the desired acceleration is:

[0118]

[0119] Among them, a sp e is the desired acceleration; p For translation vector error, K is the derivative of the translation vector error. p K is the proportional term. d K is the differential term. i It is an integral term;

[0120] Based on the current rotation vector quaternion error of the UAV, calculate the expected angular velocity of the current UAV, and adjust the roll, pitch and yaw of the current UAV according to the expected angular velocity to obtain the current UAV attitude after rotation adjustment.

[0121] In this embodiment, the formula for calculating the desired angular velocity is:

[0122] ω sp =2q err,xyz .sign(q err,w );

[0123] Where, ω sp q represents the desired angular velocity. err，xyz Let q be the imaginary part of the quaternion error of the rotation vector (a three-dimensional vector). err，w It is the real part (scalar part) of the quaternion error of the rotation vector. sign() is the sign function, which indicates the sign of the real part and ensures the consistency of the direction of the desired angular velocity.

[0124] The optimal pose of the current UAV is obtained based on the current UAV pose after translation adjustment and the current UAV pose after rotation adjustment.

[0125] In this embodiment, after calculating the pose error based on the current UAV pose and the best pose of historical UAVs, and adjusting the current UAV pose based on the pose error to obtain the best pose of the current UAV, the method further includes:

[0126] Acquire images of the substation taken by the drone in its current best pose and in its historical best pose;

[0127] The image of the substation taken by the UAV at the current best pose is compared with the image of the substation taken at the historical best pose. Based on the comparison results, the pose corresponding to the best pixel quality of the image is taken as the target best pose.

[0128] Based on the target's optimal pose and the latest UAV cruise route, update the historical UAV pose and cruise route, and save them to the database.

[0129] Example 2:

[0130] Reference Figure 2 This is a block diagram of an inspection and correction system for a power line inspection drone provided in an embodiment of the present invention. To address the problems of existing drone inspection technologies, such as the easy omission or redundancy of waypoints due to manual route setting and the low quality of inspection images under flight vibration caused by the heavy load of onboard equipment due to the visual module mounted on the drone, thus affecting the inspection results, the system includes at least the following modules: a data acquisition module, a cruise route generation module, a contour recognition module, a key structural point recognition module, a pose solving module, and a pose adjustment module.

[0131] The data acquisition module is used to acquire historical inspection data of the substation by drone and construct a set of substation equipment inspection routes; wherein, the historical inspection data includes drone flight path data, inspection point data, flight data and inspection shooting point data;

[0132] The cruise route generation module is used to generate the latest UAV cruise route based on the set of substation equipment inspection routes, using a path planning algorithm, and combining a pre-built objective function and constraints that minimize flight resource consumption.

[0133] The contour recognition module is used to acquire inspection images of the UAV when it is conducting inspections according to the latest UAV cruise route through the substation monitoring camera at the inspection shooting point, and input the UAV inspection images into the target detection algorithm model so that the target detection algorithm model can perform UAV contour image recognition based on the UAV inspection images to obtain the current UAV contour image.

[0134] The key structural point recognition module is used to input the contour image of the current UAV into the key point detection model, so that the key point detection model can identify the key structural points of the UAV based on the contour image of the current UAV and obtain the two-dimensional coordinates of the key structural points of the current UAV.

[0135] The pose solving module is used to determine the three-dimensional coordinates of the key structural points of the current UAV based on the two-dimensional coordinates of the key structural points of the current UAV, and to solve the pose of the current UAV based on the two-dimensional coordinates of the key structural points, the three-dimensional coordinates of the key structural points of the current UAV, and the monitoring camera parameters.

[0136] The pose adjustment module is used to calculate the pose error based on the current pose of the UAV and the best pose of the UAV in history, and adjust the pose of the current UAV based on the pose error to obtain the best pose of the current UAV.

[0137] Another embodiment of the present invention provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the inspection and calibration method for a power line inspection drone as described in the above embodiments. The terminal device may be a desktop computer, laptop, handheld computer, or cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory.

[0138] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the terminal device, connecting various parts of the terminal device via various interfaces and lines.

[0139] The memory can be used to store the computer program. The processor implements various functions of the terminal device by running or executing the computer program stored in the memory and calling data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function, etc.; the data storage area may store data created based on the use of the mobile phone, etc. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, RAM, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0140] Another embodiment of the present invention provides a computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the inspection and calibration method of a power line inspection drone described in the above embodiment.

[0141] The storage medium is a computer-readable storage medium, and the computer program is stored in the computer-readable storage medium. When executed by a processor, the computer program can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0142] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. In particular, it should be noted that any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention for those skilled in the art.

Claims

1. A method for inspecting and calibrating a power line inspection drone, characterized in that, include: Acquire historical inspection data of substations by drones and construct a set of substation equipment inspection routes; the historical inspection data includes drone flight path data, inspection point data, flight data and inspection shooting point data; Based on the set of substation equipment inspection routes, a path planning algorithm is used, combined with a pre-constructed objective function and constraints that minimize flight resource consumption, to generate the latest UAV cruise route. The inspection images of the drone are obtained by the substation monitoring camera at the inspection point when the drone is inspecting according to the latest drone cruise route. The inspection images of the drone are then input into the target detection algorithm model so that the target detection algorithm model can perform drone contour image recognition based on the drone inspection images to obtain the current drone contour image. The current drone's contour image is input into the key point detection model, so that the key point detection model can identify the key structural points of the drone based on the current drone's contour image and obtain the two-dimensional coordinates of the key structural points of the current drone. Based on the two-dimensional coordinates of the key structural points of the current UAV, determine the three-dimensional coordinates of the key structural points of the current UAV, and solve the pose of the current UAV based on the two-dimensional coordinates, three-dimensional coordinates and monitoring camera parameters. Based on the current pose of the drone and the best pose of the drone in history, the pose error is calculated, and the pose of the current drone is adjusted according to the pose error to obtain the best pose of the current drone.

2. The inspection and calibration method for a power line inspection drone according to claim 1, characterized in that, The constraints include endurance constraints, path constraints, and task coverage constraints. The expression for the objective function that minimizes flight resource consumption is as follows: Where, minJ is the objective function value that minimizes flight resource consumption; α, β, and λ represent the weighting coefficients for flight time, energy consumption, and mission completion priority, respectively; T total The total flight time required for the drone to complete the entire path. k is the number of nodes in the path when the drone executes the complete path, t i，i+1 E represents the time it takes for the drone to fly between the two inspection points. total This represents the total energy consumed by the drone during its path. c i，i+1 The energy consumed by the drone flying between the two inspection points; ω i Let be the task priority weight for the i-th inspection point; n is the number of inspection points that need to be covered; c i c represents the coverage status of the i-th inspection point in the inspection route. i =1 indicates that it has been covered, otherwise it is 0; The expression for the range constraint is: D total ≤D max ,T total ≤T max ; Among them, D total T represents the actual total flight distance of the UAV's planned execution path; total D represents the actual total flight time of the UAV executing the planned path; max T represents the maximum flight distance of the drone. max This refers to the maximum flight time of the drone; The expression for the path constraint is: Where L is the set of route paths; G is the set of substation equipment inspection routes; V is the set of all feasible inspection points; E is the set of all feasible route paths; P is the set of inspection points, p n This is the nth inspection point; The expression for the task coverage constraint is: Wherein, τ is a high-priority threshold; Let be the task priority weight for the i-th inspection point.

3. The inspection and calibration method for a power line inspection drone according to claim 2, characterized in that, The step of inputting the current UAV inspection image into the target detection algorithm model, so that the target detection algorithm model can perform UAV contour image recognition based on the current UAV inspection image to obtain the current UAV contour image, includes: The inspection images of the current UAV captured by the substation monitoring camera at each inspection point are input into the target detection algorithm model based on the YOLOv5 algorithm. The Backbone module, Neck module and Head module of the target detection algorithm model based on the YOLOv5 algorithm perform feature extraction, feature fusion and contour localization on the inspection images of the UAV, respectively, to obtain the contour image of the UAV captured at each inspection point.

4. The inspection and calibration method for a power line inspection drone according to claim 3, characterized in that, The step of inputting the current UAV contour image into the key point detection model, so that the key point detection model can identify the key structural points of the UAV based on the current UAV contour image and obtain the two-dimensional coordinates of the key structural points of the current UAV, includes: By using a key point detection model based on the HRNet algorithm to sequentially perform deep feature extraction, feature fusion, and key structural point identification on the contour image of the UAV, the two-dimensional coordinates of the key structural points of the UAV are obtained.

5. The inspection and calibration method for a power line inspection drone according to claim 4, characterized in that, The process of determining the pose of the current UAV based on the two-dimensional coordinates and three-dimensional coordinates of the key structural points and the monitoring camera parameters includes: Based on the two-dimensional coordinates and three-dimensional coordinates of the key structural points of the current UAV and the parameters of the monitoring camera, the EPnP algorithm is used to initially solve the pose of the current UAV, and the initial pose of the current UAV is obtained. Based on the two-dimensional coordinates and three-dimensional coordinates of the key structural points of the current UAV, the monitoring camera parameters, and the initial pose of the current UAV, the RANSAC algorithm is used to optimize the pose of the current UAV to obtain the final pose of the current UAV.

6. The inspection and calibration method for a power line inspection drone according to claim 5, characterized in that, The step of calculating the pose error based on the current UAV pose and the historical best UAV poses includes: The translation vector error of the current UAV's pose is obtained by calculating the difference between the translation vector of the current UAV's final pose and the translation vector of the historical UAV's best pose. The rotation vector of the current UAV's final pose and the rotation vector of the historical UAV's best pose are transformed into unit quaternions to obtain the rotation vector quaternions of the current UAV and the historical UAV. Based on the rotation vector quaternions of the current UAV and the historical UAV, the rotation vector quaternion error of the current UAV is calculated.

7. The inspection and calibration method for a power line inspection drone according to claim 6, characterized in that, The step of adjusting the current UAV pose based on the pose error to obtain the optimal pose of the current UAV includes: Based on the translation vector error of the current UAV pose, the expected acceleration of the current UAV is calculated using the PID algorithm, and the current UAV is controlled to make translation adjustments based on the expected acceleration to obtain the current UAV pose after translation adjustment. Based on the current rotation vector quaternion error of the UAV, calculate the expected angular velocity of the current UAV, and adjust the roll, pitch and yaw of the current UAV according to the expected angular velocity to obtain the current UAV attitude after rotation adjustment. The optimal pose of the current UAV is obtained based on the current UAV pose after translation adjustment and the current UAV pose after rotation adjustment.

8. A power line inspection drone inspection and calibration system, characterized in that, include: The system includes a data acquisition module, a cruise route generation module, a contour recognition module, a key structural point recognition module, a pose solving module, and a pose adjustment module. The data acquisition module is used to acquire historical inspection data of the substation by drone and construct a set of substation equipment inspection routes; wherein, the historical inspection data includes drone flight path data, inspection point data, flight data and inspection shooting point data; The cruise route generation module is used to generate the latest UAV cruise route based on the set of substation equipment inspection routes, using a path planning algorithm, and combining a pre-built objective function and constraints that minimize flight resource consumption. The contour recognition module is used to acquire inspection images of the UAV when it is conducting inspections according to the latest UAV cruise route through the substation monitoring camera at the inspection shooting point, and input the UAV inspection images into the target detection algorithm model so that the target detection algorithm model can perform UAV contour image recognition based on the UAV inspection images to obtain the current UAV contour image. The key structural point recognition module is used to input the contour image of the current UAV into the key point detection model, so that the key point detection model can identify the key structural points of the UAV based on the contour image of the current UAV and obtain the two-dimensional coordinates of the key structural points of the current UAV. The pose solving module is used to determine the three-dimensional coordinates of the key structural points of the current UAV based on the two-dimensional coordinates of the key structural points of the current UAV, and to solve the pose of the current UAV based on the two-dimensional coordinates of the key structural points, the three-dimensional coordinates of the key structural points of the current UAV, and the monitoring camera parameters. The pose adjustment module is used to calculate the pose error based on the current pose of the UAV and the best pose of the UAV in history, and adjust the pose of the current UAV based on the pose error to obtain the best pose of the current UAV.

9. A terminal device comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements an inspection and calibration method for a power line inspection drone as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device where the storage medium is located to perform an inspection and calibration method for a power line inspection drone as described in any one of claims 1 to 7.

Images