Method and system for controlling and processing cross-gear line-imitating flight based on visual identification of unmanned aerial vehicle

Abstract

The invention relates to the technical field of unmanned aerial vehicle inspection, and discloses an inter-gear line-imitating flight control processing method and system based on unmanned aerial vehicle visual identification, and when an unmanned aerial vehicle is used for inspection operation, the change of the relative position of the unmanned aerial vehicle and a wire can be quickly responded by obtaining the target horizontal distance between the unmanned aerial vehicle and the wire in real time. Meanwhile, in combination with a PID algorithm, an accurate flight direction instruction is generated according to the deviation between the target horizontal distance and the preset expected flight distance of the unmanned aerial vehicle, so that fine adjustment of the flight path of the unmanned aerial vehicle is realized, and accumulation of flight errors is effectively avoided. The problem that an existing unmanned aerial vehicle flight control system is poor in flight precision in a dynamic environment is solved, and the accuracy and reliability of inspection operation are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of unmanned aerial vehicle (UAV) inspection technology, and in particular to a method and system for controlling inter-gear line-flight based on UAV visual recognition. Background Art

[0002] Transmission line conductors are prone to sag, a phenomenon particularly pronounced in lines crossing rivers, mountains, and with large spans. Traditionally, manually operated drones, which fly along the conductors to detect sag, are dangerous and difficult to operate, requiring extremely high technical skills from operators and severely impacting inspection efficiency.

[0003] Existing drone flight control systems have poor flight accuracy in dynamic environments. Especially during inspections, if the control algorithm fails to effectively adjust the relative position of the drone and the conductors in real time, it can easily lead to unstable flight and low inspection accuracy. Summary of the Invention

[0004] The present invention provides a flight control processing method and system for inter-gear line-mimicking based on UAV visual recognition, which solves the technical problem of poor flight accuracy of existing UAV flight control systems in dynamic environments.

[0005] The first aspect of the present invention provides a method for controlling and processing inter-gear line-flight based on UAV visual recognition, comprising:

[0006] Obtaining the tower coordinates and the flight speed threshold of the drone, and determining the target tower according to the tower coordinates;

[0007] When the drone inspects the conductor at the midpoint of the line connecting the adjacent target towers based on the flight speed threshold, the target horizontal distance is obtained in real time;

[0008] The target horizontal distance and the preset expected flight distance of the UAV are calculated and processed by the PID algorithm to generate a flight direction instruction for the UAV.

[0009] Optionally, the real-time acquisition of the target horizontal distance includes:

[0010] Acquiring environmental image data in real time using a visual sensor pre-installed on the UAV;

[0011] Preprocessing the environmental image data to obtain environmental image data to be processed;

[0012] Obtaining the drone image position and the wire image position in the environmental image data to be processed;

[0013] Performing a monocular depth estimation operation on the environment image data to be processed to obtain a target depth distance;

[0014] The target horizontal distance is determined according to the camera intrinsic parameters of the visual sensor, the target depth distance, the drone image position and the wire image position.

[0015] Optionally, performing a monocular depth estimation operation on the environment image data to be processed to obtain a target depth distance includes:

[0016] Obtaining pixel displacement and center point coordinates of the wire in the to-be-processed environment image data of consecutive frames;

[0017] averaging the pixel displacements of the wires to obtain parallax data;

[0018] Performing Euclidean distance calculation on the center point coordinates to obtain a baseline length;

[0019] Performing a multiplication operation on the baseline length in a camera of the visual sensor to obtain a target multiplication value;

[0020] A ratio operation is performed on the target multiplication value and the disparity data to obtain a target depth distance.

[0021] Optionally, determining the target horizontal distance according to the camera intrinsic parameters of the visual sensor, the target depth distance, the drone image position, and the wire image position includes:

[0022] Performing a difference operation between the drone image position and the wire image position to obtain a target difference;

[0023] Performing a ratio operation on the target depth distance and the camera intrinsic parameter to obtain a target ratio;

[0024] The target difference value and the target ratio value are multiplied to obtain the target horizontal distance.

[0025] Optionally, the generating of the flight direction instruction of the UAV by performing calculation processing based on the target horizontal distance and a preset expected flight distance of the UAV through a PID algorithm includes:

[0026] Performing difference calculations on the target horizontal distance corresponding to each moment and the preset expected flight distance of the UAV to obtain multiple flight errors;

[0027] The flight error corresponding to each moment is input into a preset PID controller to output multiple UAV control variables;

[0028] Performing a difference calculation on the control amount of the UAV at two adjacent moments to obtain a plurality of control increments;

[0029] A flight direction instruction of the UAV is generated according to the control increment corresponding to each moment.

[0030] Optionally, the preset PID controller is a position PID controller.

[0031] A second aspect of the present invention provides a flight control and processing system for inter-gear line-following based on UAV visual recognition, comprising:

[0032] An acquisition module is used to obtain the tower coordinates and the flight speed threshold of the UAV, and determine the target tower according to the tower coordinates;

[0033] An inspection module, configured to obtain a target horizontal distance in real time when the drone inspects a conductor at a midpoint of a line connecting adjacent target towers based on the flight speed threshold;

[0034] The calculation module is used to perform calculation processing based on the target horizontal distance and the preset expected flight distance of the drone through a PID algorithm to generate a flight direction instruction for the drone.

[0035] The third aspect of the present invention provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory. When the computer program is executed by the processor, the processor executes the steps of the inter-gear linear flight control processing method based on drone visual recognition as described in any one of the above items.

[0036] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed, implements the inter-gear linear flight control processing method based on drone visual recognition as described in any one of the above items.

[0037] A fifth aspect of the present invention provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions, wherein when the program instructions are executed by a computer, the computer executes the inter-gear linear flight control processing method based on drone visual recognition as described in any one of the above items.

[0038] It can be seen from the above technical solutions that the present invention has the following advantages:

[0039] When using drones for inspections, this invention rapidly responds to changes in their relative positions by acquiring the target horizontal distance between the drone and the conductor in real time. Furthermore, incorporating a PID algorithm, it generates precise flight direction instructions based on the deviation between the target horizontal distance and the drone's preset desired flight distance, enabling fine-tuning of the drone's flight trajectory and effectively preventing the accumulation of flight errors. This solves the problem of poor flight accuracy in dynamic environments faced by existing drone flight control systems, significantly improving the accuracy and reliability of inspections. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0041] Figure 1 A flowchart of a method for controlling inter-gear line-flight based on UAV visual recognition provided in the first embodiment of the present invention;

[0042] Figure 2 A flowchart of a method for controlling inter-gear line-flight based on UAV visual recognition provided by the second embodiment of the present invention;

[0043] Figure 3 This is a typical PID controller structure diagram provided in Example 2 of the present invention;

[0044] Figure 4 This is a structural block diagram of a flight control and processing system for inter-gear line-following based on UAV visual recognition provided by the third embodiment of the present invention;

[0045] Figure 5 This is a structural block diagram of a computer device provided in Example 4 of the present invention. DETAILED DESCRIPTION

[0046] The embodiments of the present invention provide a method and system for inter-gear line-flight control processing based on UAV visual recognition, which are used to solve the technical problem of poor flight accuracy of existing UAV flight control systems in dynamic environments.

[0047] In order to make the purpose, features, and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described below are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0048] Existing drone flight control systems have poor flight accuracy in dynamic environments. This is especially true during inspections, where the control algorithm fails to effectively adjust the relative position of the drone and the conductors in real time, leading to unstable flight and low inspection accuracy.

[0049] To overcome the drawback of traditional control algorithms that fail to effectively adjust the relative position of the drone and the wire, the technical solution of the present invention automatically adjusts the flight path based on the error between the current flight state and the target state to ensure the drone's flight accuracy. When the distance between the drone and the target wire is too close or too far, the PID controller will adjust based on the deviation to avoid the flight from the target.

[0050] See also Figure 1 , Figure 1 This is a flowchart of the steps of a method for controlling inter-gear line-flight based on UAV visual recognition provided in Example 1 of the present invention.

[0051] The present invention provides a method for controlling and processing inter-gear line-flight based on UAV visual recognition, comprising:

[0052] Step 101: Obtain the tower coordinates and the flight speed threshold of the UAV, and determine the target tower according to the tower coordinates.

[0053] Tower coordinates refer to the tower coordinates manually set by the operator. During the power line inspection process, they are used to identify the coordinate data of the precise location of the transmission line tower in geographic space.

[0054] In an embodiment of the present invention, during the initial inspection, when the UAV follows the line for autonomous flight, due to the deviation of the tower coordinate information, the direction of the UAV may deviate during the following flight, resulting in the distance from the power line being too close or too far. If the distance is too close, it is easy to be dangerous, and if it is too far, the shooting effect will be poor. Through the tower coordinates manually set by the operator and the flight speed threshold of the UAV, the conductors between the tower coordinates (specifically, the conductors that determine the associated target towers based on the tower coordinates and further determine the midpoints of the lines connecting adjacent target towers) are the targets of the UAV flight inspection. By manually setting the tower coordinates to limit the UAV's flight inspection work, the UAV's flight inspection work is reduced, and the waste of computing power caused by the UAV searching for inspection targets during the inspection is reduced; by setting the flight speed threshold, inaccurate inspection results caused by the UAV flying too fast or waste of resources caused by the UAV flying too slowly are avoided.

[0055] Step 102: When the UAV inspects the conductor at the midpoint of the line connecting adjacent target towers based on the flight speed threshold, the target horizontal distance is obtained in real time.

[0056] In an embodiment of the present invention, when a drone inspects a conductor in the middle of a tower coordinate based on a flight speed threshold, the drone obtains a target horizontal distance from the drone to the conductor in real time by scanning with a visual sensor pre-installed on the drone.

[0057] It should be noted that a visual sensor is mounted on the drone, which scans the wires that the drone is inspecting, and thus performs distance measurement based on the visual sensor (distance measurement refers to measuring the horizontal distance from the drone to the wires).

[0058] Step 103: Based on the target horizontal distance and the preset expected flight distance of the UAV, a PID algorithm is used to perform calculations and generate a flight direction instruction for the UAV.

[0059] In an embodiment of the present invention, a PID algorithm is used to perform calculations based on the horizontal distance and the preset expected flight distance of the drone, and the flight direction of the drone is output.

[0060] In the specific implementation, during the initial inspection phase, the operator manually sets the coordinates of the pole tower as a reference point for the drone's inspection path. By setting the pole tower coordinates, the control system can clearly identify the drone's inspection target, namely the conductors between the pole towers. The flight speed threshold is set to limit the drone's flight speed. By setting the threshold, the drone can be prevented from flying too fast (resulting in inaccurate inspections or failure to complete the task) or too slow (wasting time and battery resources). Furthermore, the drone is equipped with a visual sensor, which scans the surrounding environment in real time and obtains the target horizontal distance between the drone and the conductor. By measuring the distance, the drone is ensured to maintain a safe distance from the power line. Finally, the drone's flight direction is calculated using a PID algorithm based on the target horizontal distance and the pre-set expected flight distance. The flight path is automatically adjusted based on the error between the current flight state and the target state to ensure the drone's flight accuracy. When the distance between the drone and the target conductor is too close or too far, adjustments are made based on the deviation to avoid flight deviation from the target.

[0061] When using drones for inspections, this invention rapidly responds to changes in their relative positions by acquiring the target horizontal distance between the drone and the conductor in real time. Furthermore, incorporating a PID algorithm, it generates precise flight direction instructions based on the deviation between the target horizontal distance and the drone's preset desired flight distance, enabling fine-tuning of the drone's flight trajectory and effectively preventing the accumulation of flight errors. This solves the problem of poor flight accuracy in dynamic environments faced by existing drone flight control systems, significantly improving the accuracy and reliability of inspections.

[0062] See also Figure 2 , Figure 2 This is a flowchart of the steps of a method for controlling inter-gear line-flight based on UAV visual recognition provided in the second embodiment of the present invention.

[0063] The present invention provides a method for controlling and processing inter-gear line-flight based on UAV visual recognition, comprising:

[0064] Step 201: Obtain the tower coordinates and the flight speed threshold of the UAV, and determine the target tower according to the tower coordinates.

[0065] In the embodiment of the present invention, the specific implementation process of step 201 is similar to that of step 101 and will not be repeated here.

[0066] Step 202: When the UAV inspects the conductor at the midpoint of the line connecting adjacent target towers based on the flight speed threshold, the visual sensor pre-installed on the UAV is used to obtain environmental image data in real time.

[0067] Environmental image data refers to the visual information of the surrounding environment captured by the drone through visual sensors (such as cameras, infrared cameras, etc.) during flight.

[0068] In an embodiment of the present invention, when the drone inspects the conductor in the middle of the tower coordinates based on the flight speed threshold, the drone obtains environmental image data by real-time scanning using a visual sensor pre-installed on the drone.

[0069] Step 203: Preprocess the environmental image data to obtain environmental image data to be processed.

[0070] Preprocessing refers to a series of operations used to improve data quality.

[0071] As a preferred embodiment, the present invention provides a pretreatment method, which is as follows:

[0072] T1. Perform image distortion correction on environmental image data through camera calibration technology;

[0073] T2, performing cropping and scaling operations on the environmental image data after the image distortion correction operation;

[0074] T3, performing denoising operations on the environmental image data after the cropping and scaling operations;

[0075] T4, using a contrast-limited adaptive histogram equalization algorithm to perform a contrast enhancement operation on the environmental image data after the denoising operation;

[0076] T5, performing a color conversion operation on the environmental image data after the contrast enhancement operation;

[0077] T6. performing a segmentation operation on the environmental image data after the color conversion operation;

[0078] T7. Normalize the environmental image data after the segmentation operation to obtain the environmental image data to be processed.

[0079] In an embodiment of the present invention, first, by prioritizing the correction of image distortion and dynamically cropping key areas, more accurate target position information is provided for the position PID controller. Then, adaptive denoising and contrast enhancement algorithms are used to ensure the clarity and stability of the input data and reduce controller errors. Then, the image is converted into grayscale or HSV color space through HSV color space conversion to extract brightness and saturation information, and deep learning segmentation is used to dynamically extract inspection target areas to adapt to complex lighting and dynamic environments and improve the robustness of the controller. Finally, normalization processing reduces the amount of data, reduces computational complexity, and supports real-time control requirements.

[0080] The present invention can significantly improve the quality of environmental image data collected by the visual sensor through the above-mentioned preprocessing method, making it more suitable for the needs of the position PID controller, thereby improving the accuracy and efficiency of drone inspections.

[0081] Step 204: Obtain the drone image position and the wire image position in the environment image data to be processed.

[0082] The drone image position refers to the position of the drone in the environment image data to be processed (usually the center point of the image).

[0083] The wire image position refers to the position of the wire in the environment image data to be processed.

[0084] In an embodiment of the present invention, target detection is performed on the environmental image data to be processed based on edge detection to obtain the image position of the drone and the image position of the wire in the environmental image data to be processed.

[0085] Step 205: Perform a monocular depth estimation operation on the environment image data to be processed to obtain a target depth distance.

[0086] Target depth distance refers to the depth distance from the drone to the guide wire.

[0087] Monocular depth estimation refers to the use of computer vision and deep learning technology to estimate the distance (depth information) between objects in the scene and the camera using image data obtained from a monocular camera (i.e., a single camera).

[0088] Furthermore, step 205 may include the following sub-steps:

[0089] S11 , obtaining pixel displacements and center point coordinates of the wires in the to-be-processed environment image data of consecutive frames.

[0090] Conductor pixel displacement refers to the pixel displacement of the conductor in the image data to be processed in consecutive frames;

[0091] The center point coordinates refer to the center point coordinates in the image data to be processed of the continuous frames. Usually, the camera corresponds to the center point coordinates of the image.

[0092] In the embodiment of the present invention, firstly, the pixel displacement and center point coordinates of the wire in the to-be-processed environment image data of consecutive frames are obtained.

[0093] S12: performing an averaging process on the pixel displacements of the wires to obtain parallax data.

[0094] In the embodiment of the present invention, disparity data is calculated based on the displacement of pixels of the wires in the image data to be processed in the consecutive frames (the disparity data refers to the offset of the pixels of the wires in the image data to be processed in the consecutive frames).

[0095] In a specific implementation, first, the pixel coordinates of the wires are extracted from two consecutive frames of image data to be processed. Then, for each pair of matching wire pixels, the pixel displacement is calculated. Finally, the displacements of all matching wire pixels are averaged to obtain disparity data.

[0096] S13. Perform Euclidean distance calculation on the center point coordinates to obtain the baseline length.

[0097] In an embodiment of the present invention, a baseline length B is calculated based on the center point coordinates in the image data to be processed of consecutive frames (the baseline length B refers to the movement or change distance of the camera in the image. Usually, the camera corresponds to the center point coordinates of the image. Therefore, the baseline length in this embodiment refers to the moving length of the center point coordinates of the image data to be processed of consecutive frames).

[0098] In a specific implementation, first, the center point coordinates of the images are extracted from two consecutive frames of image data to be processed, and then the distance between the center point coordinates of the two frames is calculated using the Euclidean distance formula. This distance is the baseline length.

[0099] S14, performing a multiplication operation on the baseline length in the camera of the visual sensor to obtain a target multiplication value.

[0100] S15. Perform a ratio operation on the target multiplication value and the parallax data to obtain the target depth distance.

[0101] In a specific implementation, to facilitate the implementation of the method, the S14-S15 process can be converted into a formula encapsulation form, wherein the target depth distance can be as follows:

[0102]

[0103] Where, Indicates the target depth distance, represents the camera intrinsic parameters of the visual sensor, Indicates the baseline length, Represents disparity data.

[0104] In an embodiment of the present invention, first, a series of continuous frames of unprocessed image data are obtained as the basis for subsequent depth estimation, and the position change of the object in the camera's field of view is calculated, thereby obtaining disparity information; further, by comparing the position change of the wire in the image between consecutive frames, the pixel displacement of the wire is calculated, and the size of the displacement is the disparity, which provides an offset for each pixel (pixel on the wire), thereby providing key data support for subsequent depth estimation. By accurately calculating the disparity, the system can more accurately infer the actual distance between the wire and the drone; and, using the center point of the camera as a motion reference, its offset in the image is calculated as a baseline. The baseline length is very critical for depth estimation because it and the disparity data jointly affect the depth calculation result. The larger the baseline length, the more accurate the depth estimation; finally, the actual depth distance of the object is calculated by utilizing the camera's intrinsic parameters, disparity data and baseline length in combination with the perspective geometry model. .

[0105] Step 206: Determine the target horizontal distance based on the camera intrinsic parameters of the visual sensor, the target depth distance, the drone image position, and the wire image position.

[0106] The target horizontal distance refers to the horizontal distance from the drone to the wire obtained by real-time scanning using a visual sensor pre-installed on the drone.

[0107] Furthermore, step 206 may include the following sub-steps:

[0108] S21. Perform a difference operation between the drone image position and the wire image position to obtain a target difference.

[0109] S22. Perform a ratio operation using the target depth distance and the camera internal parameter to obtain a target ratio.

[0110] S23. Perform multiplication operation on the target difference and the target ratio to obtain the target horizontal distance.

[0111] In a specific implementation, to facilitate the implementation of the method, the above process can be converted into a formula encapsulation form, where the target horizontal distance can be as follows:

[0112]

[0113] Where, Indicates the horizontal distance to the target, Indicates the wire image position, specifically the wire position in the environment image data to be processed. Indicates the position of the drone image, specifically the position of the drone in the environment image data to be processed (usually the center point of the image). Indicates the target depth distance, Represents the camera intrinsic parameters of the vision sensor.

[0114] It should be noted that during the inspection process, drones use visual sensors to capture environmental image data in real time as the basis for subsequent analysis and processing. By collecting images in real time, they can obtain target objects in the environment (such as wires and drones) and their positions in the image. These images are the input for subsequent depth estimation, position calculation and other operations.

[0115] In an embodiment of the present invention, first, a monocular depth estimation method is used to calculate the target depth distance from the drone to the wire through the environmental image data to be processed, and then, based on the camera intrinsic parameters of the visual sensor, the target depth distance, the drone image position and the wire image position, the target horizontal distance from the drone to the wire is calculated.

[0116] Step 207: Based on the target horizontal distance and the preset expected flight distance of the UAV, a PID algorithm is used to perform calculations and generate a flight direction instruction for the UAV. The PID algorithm is an incremental PID algorithm.

[0117] It should be noted that the PID algorithm in the embodiment is a classic control algorithm with a history of nearly 70 years. It is one of the earliest automatic control strategies used in industrial process control. Its principle is simple and it has certain robustness and reliability. It has been widely used in industrial process control, especially in control systems where accurate mathematical models can be established. For systems with uncertain mathematical models, relatively ideal control effects can still be achieved by adjusting the algorithm parameters. Most mainstream drones on the market adopt this classic control strategy; the typical PID controller structure is as follows: Figure 3 As shown, given a desired input, the error between the system output and the desired input is detected, and the system output is adjusted to achieve the desired effect through PID regulation; in the embodiment of the present application, the system input is the desired position of the drone and the output is the actual position of the drone, and the PID controller directly controls the flight direction and speed of the drone.

[0118] The mathematical expression of PID controller is as follows:

[0119]

[0120] Where, Represents the output of the PID controller, represents the proportionality coefficient, represents the integral coefficient, represents the differential coefficient, Indicates the deviation between the expected value and the actual value.

[0121] From this formula, we can see that the proportional regulation directly acts on the deviation of the system. As long as there is a deviation in the system, the proportional regulation will take effect to reduce the deviation of the system.

[0122] Among them, the proportional adjustment The value of reflects the dynamic response speed of the system. The larger the value, the faster the system response speed, which can eliminate the system deviation in the shortest possible time, but it is easy to cause a large overshoot; if the value is too small, the system adjustment speed will slow down; a system with only proportional adjustment will eventually form a stable deviation; the integral coefficient acts on the deviation accumulation term of the system flow to eliminate the steady-state error of the system. Due to the existence of the integral term, as long as there is a deviation in the system, the controller will play a regulatory role and eventually eliminate the system deviation.

[0123] Integral adjustment The value directly determines the system's ability to eliminate steady-state errors. The larger the value, the faster the system eliminates steady-state errors, but it is easy to cause integral saturation in the early stages of regulation. If the value is too small, it will be difficult to eliminate the steady-state error of the system.

[0124] Differential adjustment The role of is to predict the changing trend of the system and suppress the system oscillation caused by the proportional term during the adjustment process. However, if the role of differential adjustment is too strong, it will cause the system to brake in advance during the adjustment process and prolong the system adjustment time.

[0125] To facilitate programming implementation, the above formula is usually written in discrete form as follows:

[0126]

[0127] Where, represents the output of the controller, express The deviation between the expected value and the actual value of the system at any moment, the parameter adjustment of the PID algorithm is to adjust, 、 、 The values ​​of the three parameters make the controller performance meet the expected requirements.

[0128] The output of the PID controller represented by the above formula directly acts on the system, so the PID controller represented by the above formula is also called a position PID controller; the defect of the position PID controller is that the output of the controller directly acts on the system, and the incorrect control output has a greater impact on the system; in addition, the position PID controller requires the accumulated value of past deviations, which is prone to produce large cumulative errors; therefore, in practical applications, an incremental PID controller has been developed, and the specific application is shown in S31-S34 below.

[0129] Furthermore, in step 207, a PID algorithm is used to calculate and process the horizontal distance and the preset expected flight distance of the drone, and the flight direction of the drone is output. Step 207 may include the following sub-steps:

[0130] S31. Perform difference calculations on the target horizontal distance corresponding to each moment and the preset expected flight distance of the UAV to obtain multiple flight errors.

[0131] In a specific implementation, in order to facilitate the implementation of the method, the above process can be converted into a formula encapsulation form, where the flight error can be as follows:

[0132]

[0133] Where, Indicates the flight error, Indicates the horizontal distance to the target, Indicates the expected flight distance of the preset drone.

[0134] S32: Using the flight error corresponding to each moment as input to a preset PID controller, outputting multiple UAV control variables. The preset PID controller is a position PID controller.

[0135] S33. Perform difference calculation on the drone control quantities at two adjacent moments to obtain multiple control increments.

[0136] In a specific implementation, to facilitate the implementation of the method, the S32-S33 process can be converted into a formula encapsulation form, where the control increment can be as follows:

[0137]

[0138] Where, represents the control increment, Indicates the flight error at the current moment, Indicates the flight error at the last moment, Indicates the flight error at the previous moment, Indicates the control quantity at the current moment, Indicates the control quantity at the previous moment.

[0139] S34. Generate a flight direction instruction for the UAV based on the control increment corresponding to each moment.

[0140] In the embodiment of the present invention, the flight direction of the UAV at each moment is generated according to the control increment corresponding to each moment.

[0141] It should be noted that the control increment Based on the above flight error To determine, specifically, when When the UAV is too close to the power line, it is dangerous and needs to fly backwards; when When the drone is too far away from the power line, the shooting effect is not good and it needs to fly forward; when When the drone is at a safe distance from the power line, the camera can still take pictures; that is, the flight error Determine the general direction of the drone's flight, and control the increment To determine the specific value of the drone's flight direction;

[0142] The output of the incremental PID controller is the increment of the controlled quantity Even if the controller fails, the system can maintain its original state operation; due to the obvious integral truncation effect of the incremental PID, this embodiment uses a position PID controller in actual application to make the drone move according to the expected trajectory. The horizontal difference between the tracking target position and the real-time position of the drone is calculated by the monocular ranging method as the input of the PID algorithm. The proportion, integral and differential of this difference are linearly combined to form a control variable to update and adjust the spatial position of the drone.

[0143] When using drones for inspections, this invention rapidly responds to changes in their relative positions by acquiring the target horizontal distance between the drone and the conductor in real time. Furthermore, incorporating a PID algorithm, it generates precise flight direction instructions based on the deviation between the target horizontal distance and the drone's preset desired flight distance, enabling fine-tuning of the drone's flight trajectory and effectively preventing the accumulation of flight errors. This solves the problem of poor flight accuracy in dynamic environments faced by existing drone flight control systems, significantly improving the accuracy and reliability of inspections.

[0144] See also Figure 4 , Figure 4 This is a structural block diagram of an inter-gear line-flight control and processing system based on UAV visual recognition provided in Example 3 of the present invention.

[0145] The present invention provides a flight control and processing system for inter-gear line-following based on UAV visual recognition, comprising:

[0146] The acquisition module 301 is used to obtain the tower coordinates and the flight speed threshold of the UAV, and determine the target tower according to the tower coordinates;

[0147] Inspection module 302, configured to obtain the target horizontal distance in real time when the UAV inspects the conductor at the midpoint of the line connecting adjacent target towers based on the flight speed threshold;

[0148] The calculation module 303 is used to perform calculation processing based on the target horizontal distance and the preset expected flight distance of the drone through the PID algorithm to generate a flight direction instruction for the drone.

[0149] Furthermore, the inspection module 302 includes:

[0150] The environmental image data submodule is used to obtain environmental image data in real time using the visual sensor pre-installed on the UAV;

[0151] The submodule of the environmental image data to be processed is used to pre-process the environmental image data to obtain the environmental image data to be processed;

[0152] The image position acquisition submodule is used to obtain the drone image position and the wire image position in the environmental image data to be processed;

[0153] The monocular depth estimation operation submodule is used to perform a monocular depth estimation operation on the processed environment image data to obtain the target depth distance;

[0154] The target horizontal distance submodule is used to determine the target horizontal distance based on the camera intrinsic parameters of the visual sensor, the target depth distance, the drone image position and the wire image position.

[0155] Furthermore, the monocular depth estimation operation submodule includes:

[0156] A parsing unit, configured to obtain pixel displacements and center point coordinates of the wires in the to-be-processed environment image data of successive frames;

[0157] A disparity data unit, used to average the pixel displacements of the wires to obtain disparity data;

[0158] Baseline length unit, used to calculate the Euclidean distance of the center point coordinates to obtain the baseline length;

[0159] A target multiplication unit is used to perform a multiplication operation using the baseline length within the camera of the visual sensor to obtain a target multiplication value;

[0160] The target depth distance unit is used to perform a ratio operation using the target multiplication value and the disparity data to obtain the target depth distance.

[0161] Furthermore, the target horizontal distance submodule includes:

[0162] A target difference unit is used to perform a difference operation between the drone image position and the conductor image position to obtain a target difference;

[0163] A target ratio unit is used to perform a ratio operation using a target depth distance and a camera internal parameter to obtain a target ratio;

[0164] The target horizontal distance unit is used to perform a multiplication operation on the target difference and the target ratio to obtain the target horizontal distance.

[0165] Furthermore, the operation module 303 includes:

[0166] The flight error submodule is used to perform difference calculations between the target horizontal distance corresponding to each moment and the preset expected flight distance of the UAV to obtain multiple flight errors;

[0167] The drone control quantum module is used to input the preset PID controller using the flight error corresponding to each moment and output multiple drone control quantities;

[0168] The control increment submodule is used to perform difference calculation on the UAV control quantity at two adjacent moments to obtain multiple control increments;

[0169] The flight direction instruction submodule is used to generate the flight direction instruction of the UAV according to the control increment corresponding to each moment.

[0170] Furthermore, the preset PID controller is a position PID controller.

[0171] When using drones for inspections, this invention rapidly responds to changes in their relative positions by acquiring the target horizontal distance between the drone and the conductor in real time. Furthermore, incorporating a PID algorithm, it generates precise flight direction instructions based on the deviation between the target horizontal distance and the drone's preset desired flight distance, enabling fine-tuning of the drone's flight trajectory and effectively preventing the accumulation of flight errors. This solves the problem of poor flight accuracy in dynamic environments faced by existing drone flight control systems, significantly improving the accuracy and reliability of inspections.

[0172] See also Figure 5 , Figure 5 This is a structural block diagram of a computer device provided in Example 4 of the present invention.

[0173] An electronic device according to an embodiment of the present invention includes: a memory 401 and a processor 402, wherein the memory 401 stores a computer program; when the computer program is executed by the processor 402, the processor 402 executes the inter-gear line-simulating flight control processing method based on drone visual recognition as described in any of the above embodiments.

[0174] Memory 401 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM. Memory 401 has storage space 403 for program code 413 for executing any of the method steps described above. For example, storage space 403 for program code may include individual program codes 413 for implementing various steps in the method described above. These program codes may be read from or written to one or more computer program products. These computer program products include program code carriers such as hard disks, compact disks (CDs), memory cards, or floppy disks. The program codes may be compressed, for example, in a suitable format. When executed by a processing device, these codes cause the processing device to execute the various steps in the method described above. These program codes may be read from or written to one or more computer program products. These computer program products include program code carriers such as hard disks, compact disks (CDs), memory cards, or floppy disks. The program codes may be compressed, for example, in a suitable format. When these codes are executed by a computing and processing device, they cause the computing and processing device to execute the various steps of the above-described inter-gear linear flight control processing method based on UAV visual recognition.

[0175] The fifth embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the inter-gear line-mimicking flight control processing method based on drone visual recognition as described in any of the above embodiments is implemented.

[0176] Embodiment 6 of the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer executes the inter-gear linear flight control processing method based on drone visual recognition as described in any of the above embodiments.

[0177] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0178] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0179] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0180] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0181] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the method of the present invention. The aforementioned storage medium includes various media that can store program code, such as USB flash drives, mobile hard drives, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0182] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A flight control method for inter-gear line-following based on UAV visual recognition, characterized in that: include: Obtaining the tower coordinates and the flight speed threshold of the drone, and determining the target tower according to the tower coordinates; When the drone inspects the conductor at the midpoint of the line connecting the adjacent target towers based on the flight speed threshold, the target horizontal distance is obtained in real time; The target horizontal distance and the preset expected flight distance of the UAV are calculated and processed by the PID algorithm to generate a flight direction instruction for the UAV.

2. The method for controlling inter-gear line-flight based on UAV visual recognition according to claim 1, characterized in that: The real-time acquisition of the target horizontal distance includes: Acquiring environmental image data in real time using a visual sensor pre-installed on the UAV; Preprocessing the environmental image data to obtain environmental image data to be processed; Obtaining the drone image position and the wire image position in the environmental image data to be processed; Performing a monocular depth estimation operation on the environment image data to be processed to obtain a target depth distance; The target horizontal distance is determined according to the camera intrinsic parameters of the visual sensor, the target depth distance, the drone image position and the wire image position.

3. The inter-gear line-flight control processing method based on UAV visual recognition according to claim 2 is characterized in that: The performing a monocular depth estimation operation on the environment image data to be processed to obtain a target depth distance includes: Obtaining pixel displacement and center point coordinates of the wire in the to-be-processed environment image data of consecutive frames; averaging the pixel displacements of the wires to obtain parallax data; Performing Euclidean distance calculation on the center point coordinates to obtain a baseline length; Performing a multiplication operation on the baseline length in a camera of the visual sensor to obtain a target multiplication value; A ratio operation is performed on the target multiplication value and the disparity data to obtain a target depth distance.

4. The method for controlling inter-gear line-flight based on UAV visual recognition according to claim 2, characterized in that: The determining of the target horizontal distance according to the camera intrinsic parameter of the visual sensor, the target depth distance, the drone image position, and the wire image position includes: Performing a difference operation between the drone image position and the wire image position to obtain a target difference; Performing a ratio operation on the target depth distance and the camera intrinsic parameter to obtain a target ratio; The target difference value and the target ratio value are multiplied to obtain the target horizontal distance.

5. The inter-gear line-flight control processing method based on UAV visual recognition according to any one of claims 1 to 4, characterized in that: The method of performing calculations based on the target horizontal distance and the preset expected flight distance of the UAV through a PID algorithm to generate a flight direction instruction for the UAV includes: Performing difference calculations on the target horizontal distance corresponding to each moment and the preset expected flight distance of the UAV to obtain multiple flight errors; The flight error corresponding to each moment is input into a preset PID controller to output multiple UAV control variables; Performing a difference calculation on the control amount of the UAV at two adjacent moments to obtain a plurality of control increments; A flight direction instruction of the UAV is generated according to the control increment corresponding to each moment.

6. The inter-gear line-flight control processing method based on UAV visual recognition according to claim 5 is characterized in that: The preset PID controller is a position PID controller.

7. A flight control processing system for inter-gear line-following based on UAV visual recognition, based on the flight control processing method for inter-gear line-following based on UAV visual recognition according to any one of claims 1 to 6, characterized in that: include: An acquisition module is used to obtain the tower coordinates and the flight speed threshold of the UAV, and determine the target tower according to the tower coordinates; An inspection module, configured to obtain a target horizontal distance in real time when the drone inspects a conductor at a midpoint of a line connecting adjacent target towers based on the flight speed threshold; The calculation module is used to perform calculation processing based on the target horizontal distance and the preset expected flight distance of the drone through a PID algorithm to generate a flight direction instruction for the drone.

8. An electronic device, characterized in that: It includes a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the steps of the inter-gear line-simulating flight control processing method based on drone visual recognition as described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed, the inter-gear line-simulating flight control processing method based on UAV visual recognition is implemented as described in any one of claims 1 to 6.

10. A computer program product, characterized in that The computer program product includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions, wherein when the program instructions are executed by a computer, the computer is caused to execute the inter-gear line-simulating flight control processing method based on drone visual recognition as described in any one of claims 1 to 6.

Images