Inspection unmanned aerial vehicle autonomous vision docking inhabitation method based on online learning active disturbance rejection

Abstract

The invention provides an inspection unmanned aerial vehicle autonomous vision docking inhabitation method based on online learning active disturbance rejection. The method comprises the steps that a video stream of a power transmission line inspection scene is shot based on an unmanned aerial vehicle, an image used for power transmission line detection is acquired to serve as a first image, and the first image carries depth information corresponding to pixel points; based on the trained image segmentation model, identifying the first image; when the identification result shows that the power transmission line exists in the first image, determining an inhabitation point corresponding to the unmanned aerial vehicle and a relative position of the inhabitation point and the unmanned aerial vehicle from a power transmission line graph area in the first image; generating a reference trajectory indicating that the unmanned aerial vehicle flies to the inhabitation point based on the relative position; determining control parameters of the unmanned aerial vehicle based on the relative position, a preset iterative learning linear active-disturbance-rejection control strategy and the reference trajectory; and controlling the unmanned aerial vehicle to fly towards the inhabitation point based on the control parameters. Therefore, the problem that the unmanned aerial vehicle cannot autonomously and accurately fly to the inhabitation point of the power transmission line can be improved.

Description

Technical Field

[0001] The present invention relates to the field of unmanned aerial vehicle (UAV) control technology, and in particular to an autonomous visual docking and perching method for an inspection UAV based on online learning and anti-interference control. Background Art

[0002] Regular inspections of power transmission and distribution lines are crucial for ensuring the safe and stable operation of power transmission systems. However, traditional manual inspections face numerous challenges, including long inspection distances, high workload, low efficiency, and high safety risks. In contrast, drones, thanks to their excellent environmental adaptability, are increasingly being used for power line inspections. However, drone inspections also have significant shortcomings, with limited endurance being a major constraint. Typically, a single drone flight lasts no more than 40 minutes, with an operating radius of no more than 5 kilometers. Long-distance inspections require frequent round trips, impacting operational efficiency. Currently, there are solutions for charging drones by hanging them from power lines. However, these solutions typically require remote control by the operator throughout the entire process, and the drone cannot autonomously fly to its perch. Furthermore, environmental disturbances (such as wind) can affect the control accuracy of the drone as it flies to its perch on the power lines, making it difficult for the drone to accurately land at the perch. Summary of the Invention

[0003] In view of this, the purpose of the embodiments of the present application is to provide an autonomous visual docking and perching method for inspection drones based on online learning anti-interference, which can improve the problem that drones cannot autonomously and accurately fly to the perching points of power lines.

[0004] To achieve the above technical objectives, the technical solutions adopted in this application are as follows:

[0005] The present application provides a method for autonomous visual docking and perching of an inspection drone based on online learning anti-interference control, the method comprising:

[0006] Based on a video stream of a transmission line inspection scene shot by a drone, an image for transmission line inspection is obtained as a first image, where the first image carries depth information corresponding to a pixel point;

[0007] Recognize the first image based on the trained image segmentation model to obtain a recognition result;

[0008] When the recognition result indicates that a power line exists in the first image, determining a perch corresponding to the UAV and a relative position between the perch and the UAV from a power line map region in the first image;

[0009] Based on the relative position, generating a reference trajectory representing the flight of the drone to the habitat;

[0010] Determining control parameters of the UAV based on the relative position, a preset iterative learning linear active disturbance rejection control strategy, and the reference trajectory;

[0011] Based on the control parameters, the UAV is controlled to fly toward the habitat so that the UAV stops at the habitat.

[0012] In some optional implementations, recognizing the first image based on a trained image segmentation model to obtain a recognition result includes:

[0013] Based on the trained U-Net model, the first image is recognized to obtain the recognition result, wherein the loss function D loss of the U-Net model is:

[0014] D loss=WCE loss+Focal loss

[0015]

[0016] Where WCE loss represents the cross entropy loss function; w is the foreground loss adjustment weight in the image; y is the sample label; is the probability of the foreground predicted by the U-Net model, wherein the foreground includes the power line;

[0017] Focal loss represents the focal loss function; γ is the parameter for adjusting the weight factor, p t It represents the probability that the sample belongs to the true label category when the sample is a true label; p represents the probability that the sample predicted by the U-Net model belongs to the positive class.

[0018] In some optional embodiments, determining a perch corresponding to the drone and a relative position between the perch and the drone from a power line map region in the first image includes:

[0019] Based on the bilateral filtering algorithm, the depth information in the first image is filtered to obtain the filtered first image as the second image, which is expressed as:

[0020]

[0021] Where BF represents the depth information of the corresponding pixel in the first image after filtering, I p represents the depth information of the corresponding pixel point p in the first image; ω p is the weighted sum of each pixel value in the filtering window of the bilateral filtering algorithm; ω s (i, j) represents the pixel weight of the bilateral filtering algorithm at the pixel point (i, j); ω r(i, j) represents the spatial distance weight of the bilateral filtering algorithm at the pixel point (i, j); Ω represents the pixel area of ​​the image; S(i, j) represents the pixel value of the pixel point (i, j) in the first image, and S(x, y) represents the pixel value of the central pixel in the neighborhood of the pixel point (i, j); σ s and σ r and ω s (i,j),ω r The standard deviation of the Gaussian function corresponding to (i, j);

[0022] dilating the power transmission line diagram region in the second image using an image morphological algorithm to obtain a third image;

[0023] From the third image, a target pixel point located on the power line that meets preset conditions is selected as the habitat point of the UAV, and based on the mode of the depth values ​​of a preset number of pixel points around the target pixel point, the coordinates of the habitat point in the camera coordinate system of the UAV are obtained as the relative position.

[0024] In some optional embodiments, between the step of determining a perch corresponding to the UAV and a relative position between the perch and the UAV in the power line map region in the first image, and the step of generating a reference trajectory representing a flight of the UAV to the perch based on the relative positions, the method further includes:

[0025] During the flight of the UAV, the coordinates of the habitat are updated using the sparse optical flow method, the RANSAC algorithm, and the PnP solution strategy to obtain the updated coordinates, which are expressed as:

[0026] P t:1 =R n P t +t n

[0027] Where, P t:1 represents the coordinates of the habitat at time t+1; P t represents the coordinates of the habitat at time t; R n represents the relative rotation matrix between the two first images at t+1 and t; t n Represents the relative displacement matrix between the two frames of the first image at time t+1 and t.

[0028] In some optional embodiments, determining the control parameters of the UAV based on the relative position, a preset iterative learning linear active disturbance rejection control strategy, and the reference trajectory includes:

[0029] Using a pre-created iterative learning controller, iteratively learning the first input data to obtain input data after iterative learning as second input data, wherein the first input data includes the relative position;

[0030] Using a pre-created tracking differentiator, smoothing the first input data to obtain smoothed first input data;

[0031] Based on the second input data and the current output data, using a pre-created extended state observer, determining an estimated value and an extended state variable corresponding to the smoothed first input data;

[0032] Based on the smoothed first input data, the estimated value and the expanded state variable, a control parameter of the UAV is determined using a pre-created linear feedback controller.

[0033] In some optional implementations, a pre-created iterative learning controller is used to iteratively learn the first input data to obtain input data after iterative learning as the second input data, wherein the first input data includes the relative position, including:

[0034] The second input data is obtained by the iterative learning controller using the following learning law, where:

[0035]

[0036] Where u k:1 (t) represents the second input data; u k (t) represents the first input data; Γ p and Γ d Represents proportional learning gain and differential learning gain respectively; e k (t) represents the tracking error obtained based on the reference trajectory and the current position of the UAV; Represents the tracking error e k The derivative of (t).

[0037] In some optional implementations, using a pre-created tracking differentiator to smooth the first input data to obtain smoothed first input data includes:

[0038] The tracking differentiator is used to smooth the first input data using the following formula to obtain the smoothed first input data, which is expressed as:

[0039]

[0040] Wherein, v1 refers to the tracking signal obtained by tracking the relative position in the first input data; v2 refers to the expected speed in the first input data; Refers to the data after smoothing the tracking signal v1; Refers to the data after smoothing the expected speed v2; v represents the actual input signal of the UAV; h represents the filter factor; f han () represents the preset secondary discrete function; r is the speed factor.

[0041] In some optional embodiments, based on the second input data and the current output data, using a pre-created extended state observer, determining an estimated value and an extended state variable corresponding to the smoothed first input data includes:

[0042] The estimated value of the first input data and the expanded state variable are determined by the expanded state observer using the following first expansion formula, which is expressed as:

[0043]

[0044] Where, e represents the tracking error; z1 and z2 represent the estimated values ​​corresponding to the tracking signal v1 and the expected velocity v2, respectively; β1, β2, and β3 represent the first state error feedback gain, the second state error feedback gain, and the third state error feedback gain, respectively; z3 represents the expanded state variable; fal() represents the preset continuous power function; α1 and α2 are parameters with values ​​between (0, 1); δ is the interval length of the linear segment; b represents; and u represents.

[0045] In some optional embodiments, determining the control parameters of the UAV based on the smoothed first input data, the estimated value, and the expanded state variable using a pre-created linear feedback controller includes:

[0046] Based on the linear feedback controller, the first expansion formula is converted into a second expansion formula to replace the nonlinear function in the extended state observer with a linear function. The second expansion formula is:

[0047]

[0048] Where b refers to the proportional factor; u refers to the control input data;

[0049] Based on the estimated values ​​obtained by the two expansion formulas, the expansion state variables, the smoothed first input data and the dynamic model of the UAV, the control parameters of the UAV are determined. The control parameters include the lift u corresponding to the x-axis, y-axis and z-axis in the UAV's body coordinate system. x 、u y 、u z , expressed as:

[0050]

[0051] Wherein, U1 represents the total lift of the UAV determined based on the dynamic model, θ represents the pitch angle, ψ represents the yaw angle, and φ represents the roll angle.

[0052] In some optional embodiments, the method further comprises:

[0053] When the UAV flies to the perch, controlling the clamping mechanism on the UAV to clamp the power line so that the UAV stops at the perch;

[0054] When the drone receives a charging instruction allowing charging, the inductive charging module on the drone is controlled to charge the battery module of the drone.

[0055] The invention adopting the above technical solution has the following advantages:

[0056] In the technical solution provided in the present application, an image (first image) carrying depth information is collected by a drone, and combined with a trained image segmentation model, the power transmission line area is accurately segmented. Depth information makes up for the shortcomings of traditional two-dimensional images in spatial positioning, making the positioning of the habitat more accurate. Then, based on the relative position of the habitat, a reference trajectory is automatically generated, and the flight parameters of the drone are dynamically adjusted based on the iterative learning linear auto-disturbance rejection control (ILC-LADRC) strategy. This control strategy combines iterative learning (adapting to periodic disturbances) with linear auto-disturbance rejection (compensating for unknown disturbances), which is conducive to improving the ability to resist wind interference and ensuring that the drone can stably fly to the habitat in a dynamic environment. In this way, the problem that the drone cannot fly autonomously and accurately to the habitat of the power transmission line can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The present application may be further illustrated by the non-limiting embodiments provided in the accompanying drawings. It should be understood that the following drawings illustrate only certain embodiments of the present application and are therefore not to be construed as limiting the scope of the present application. It is understood that a person skilled in the art can derive other relevant drawings from these drawings without inventive effort.

[0058] Figure 1 A flow chart of the autonomous visual docking and perching method for an inspection drone based on online learning anti-interference control provided in an embodiment of the present application.

[0059] Figure 2 This is a control block diagram of the iterative learning linear active disturbance rejection control strategy provided in an embodiment of the present application.

[0060] Figure 3A schematic diagram of the power line segmentation effect provided in an embodiment of the present application.

[0061] Figure 4 A schematic diagram of the effect of determining a perch point on a power line provided in an embodiment of the present application.

[0062] Figure 5 A schematic diagram of the simulation of the three-axis position of the fixed-point control of the drone provided in the embodiment of the present application.

[0063] Figure 6 A schematic diagram of a simulation of the spiral upward trajectory tracking effect of a drone provided in an embodiment of the present application.

[0064] Figure 7 A schematic diagram of a simulation of the ∞-shaped trajectory tracking effect of a drone provided in an embodiment of the present application. DETAILED DESCRIPTION

[0065] The present application will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that similar or identical parts in the drawings or descriptions are numbered the same. Implementations not shown or described in the drawings are known to those of ordinary skill in the art. In the description of this application, the terms "first," "second," etc. are used solely to distinguish descriptions and are not to be construed as indicating or implying relative importance.

[0066] Please refer to Figure 1 and Figure 2 The present application provides an autonomous visual docking and perching method for inspection drones based on online learning and anti-interference control (hereinafter referred to as the "method"), which can be applied to drones, and each step of the method can be performed or implemented by the drone. The drone has a clamping mechanism and an inductive charging module. The clamping mechanism is used to clamp the power line so that the drone can stop on the power line. The inductive charging module is used to use the current on the power line to charge the battery module of the drone through electromagnetic induction. The clamping mechanism and the inductive charging module are conventional structural modules.

[0067] Please refer to Figure 1 , this method may include the following steps:

[0068] Step 110: Based on the video stream of the transmission line inspection scene captured by the drone, an image for transmission line inspection is obtained as a first image, where the first image carries depth information corresponding to the pixel points;

[0069] Step 120: Recognize the first image based on the trained image segmentation model to obtain a recognition result;

[0070] Step 130: When the recognition result indicates that a power line exists in the first image, determining a perch corresponding to the UAV and a relative position between the perch and the UAV from the power line map region in the first image;

[0071] Step 140 , generating a reference trajectory representing the UAV flying to the habitat based on the relative position;

[0072] Step 150: determining control parameters of the UAV based on the relative position, a preset iterative learning linear active disturbance rejection control strategy, and the reference trajectory;

[0073] Step 160: Based on the control parameters, control the UAV to fly toward the habitat, so that the UAV stops at the habitat.

[0074] In the above-mentioned embodiment, an image (first image) carrying depth information is collected by a drone, and combined with a trained image segmentation model, the power line area is accurately segmented. Depth information makes up for the shortcomings of traditional two-dimensional images in spatial positioning, making the positioning of the perch more accurate, especially in complex terrain or occlusion scenarios, which can effectively reduce misjudgment. Then, based on the relative position of the perch, a reference trajectory is automatically generated, and the flight parameters of the drone are dynamically adjusted based on the iterative learning linear auto-disturbance rejection control (ILC-LADRC) strategy. This control strategy combines iterative learning (adapting to periodic disturbances) with linear auto-disturbance rejection (compensating for unknown disturbances), which is conducive to improving the ability to resist wind interference and ensuring that the drone can stably fly to the perch in a dynamic environment. The full process automation (identification → planning → control) reduces manual intervention and reduces the operational error rate. In this way, the problem that the drone cannot autonomously and accurately fly to the perch of the power line can be improved.

[0075] The following is a detailed description of each step of this method:

[0076] In step 110, the drone's onboard camera, a depth camera, captures a transmission line inspection scene, generating a video stream. A transmission line inspection scene can be understood as any scene requiring transmission line inspection, such as an outdoor overhead transmission line. During each control cycle, the drone captures a high-definition, most recent image from the real-time video stream as the first image. The method for selecting the first image is conventional, and the control cycle can be flexibly determined based on actual circumstances, which will not be detailed here.

[0077] In the first image, each pixel carries corresponding depth information. If the object represented by the pixel is too far away, the depth information is usually empty; if the object distance is within the detection range of the depth camera, the depth information can be understood as the distance between the object and the depth camera.

[0078] In step 120, the recognition result may include whether a power line exists in the first image or not. If no power line exists in the first image, the drone switches the viewing angle and retakes the image until the presence of a power line is determined from the captured image. If a power line exists in the first image, the process proceeds to step 130.

[0079] In step 120, the first image is recognized based on the trained image segmentation model to obtain a recognition result, including:

[0080] Based on the trained U-Net model, the first image is recognized to obtain the recognition result, wherein the loss function D loss of the U-Net model is:

[0081] D loss=WCE loss+Focal loss (1)

[0082]

[0083] Where WCE loss represents the cross entropy loss function; w is the foreground loss adjustment weight in the image; y is the sample label; is the probability of the foreground predicted by the U-Net model, wherein the foreground includes the power line;

[0084] Focal loss represents the focal loss function; γ is the parameter for adjusting the weight factor, p t It represents the probability that the sample belongs to the true label category when the sample is a true label; p represents the probability that the sample predicted by the U-Net model belongs to the positive class.

[0085] Understandably, given that the deep network has low losses due to the ease of classification of background pixels and the inaccurate learning of minority class pixel classification targets, this application proposes a lightweight power line segmentation algorithm based on a convolutional neural network of the U-Net model. Based on the U-Net model, on the one hand, by adjusting the loss function design, a higher weight is given to the foreground (i.e., the power line) class, and multiple loss functions are integrated to alleviate the class imbalance problem; on the other hand, the basic network structure is compressed to achieve lightweight under limited computing resources, thereby improving the recognition and segmentation speed and accuracy. The U-Net model originated from medical image segmentation and has a U-shaped structure. It includes two parts: feature extraction and upsampling. It has the following functional characteristics: simple image semantics and fixed structure, which makes high-level semantics and low-level features crucial; and the sample size is small, requiring the model not to be too large to prevent overfitting.

[0086] The existing U-Net uses the cross-entropy loss function, and the average pixel loss is used as the overall image loss during training. This application designs the weight coefficient of the cross-entropy loss function, so that more foreground loss is used for network training.

[0087] In addition, simply adjusting the foreground loss weight can alleviate the problem to a certain extent, but the foreground boundary position and background interference noise cannot be directly trained by weight adjustment. Therefore, this application uses the Focal loss function to measure the overlap between samples of different categories. It can adaptively adjust the loss ratio of each pixel, making the network pay more attention to the power transmission line area in the foreground. On the basis of the standard cross entropy loss, a new coefficient factor is added to strengthen the learning of difficult samples and improve the model's classification ability, which is more suitable for binary image segmentation scenarios.

[0088] This application introduces a coefficient factor γ to enable the U-Net model to adjust based on the difference between the predicted result and the actual label. A smaller deviation indicates a higher prediction accuracy, while a smaller deviation increases the loss weight. Increasing γ will increase the adjustment strength.

[0089] As an optional implementation method, after completing the loss function design, in order to achieve a lighter network structure and identify segmentation under limited computing resources, the image processing of the U-Net model needs to be adjusted. The original U-Net model expands the input image from 3 dimensions to 64 dimensions, and the maximum channel design is 1024. Because the transmission line structure is simple, there is no need for excessive channel design. Therefore, this application can expand the input image (i.e., the first image) from 3 dimensions to only 32 dimensions, and the subsequent number of channels is halved. At this time, the maximum number of channels of the entire model is 512 to reduce the amount of calculation.

[0090] It should be noted that the training process of the image segmentation model is conventional and will not be described in detail here. The trained image segmentation model has the function of identifying power transmission lines from images.

[0091] In step 130, determining a perch corresponding to the UAV and a relative position between the perch and the UAV from the power line map region in the first image includes:

[0092] Based on the bilateral filtering algorithm, the depth information in the first image is filtered to obtain the filtered first image as the second image, which is expressed as:

[0093]

[0094] Where BF represents the depth information of the corresponding pixel in the first image after filtering, I p represents the depth information of the corresponding pixel in the first image; ω p is the weighted sum of each pixel value in the filtering window of the bilateral filtering algorithm; ω s (i, j) represents the pixel weight of the bilateral filtering algorithm at the pixel point (i, j); ω r(i, j) represents the spatial distance weight of the bilateral filtering algorithm at the pixel point (i, j); Ω represents the pixel area of ​​the image; S(i, j) represents the pixel value of the pixel point (i, j) in the first image, and S(x, y) represents the pixel value of the central pixel in the neighborhood of the pixel point (i, j); σ s and σ r and ω s (i,j),ω r The standard deviation of the Gaussian function corresponding to (i, j);

[0095] dilating the power transmission line diagram region in the second image using an image morphological algorithm to obtain a third image;

[0096] From the third image, a target pixel point located on the power line that meets preset conditions is selected as the habitat point of the UAV, and based on the mode of the depth values ​​of a preset number of pixel points around the target pixel point (the preset number can be flexibly selected according to actual conditions, such as 10), the coordinates of the habitat point in the camera coordinate system of the UAV are obtained as the relative position.

[0097] In this embodiment, various preset thresholds (e.g., preset numbers) and preset conditions can be flexibly set based on actual circumstances. For example, the preset condition can be understood as eliminating pixels with abnormal depth values ​​that are located in the power line area. The method for determining and eliminating abnormal depth values ​​is conventional.

[0098] In this embodiment, the drone needs to obtain the coordinates of the habitat in the drone's camera coordinate system, and the depth information in the first image can be used for distance calculation. During the recognition process, the drone itself needs to maintain a certain distance from the power transmission line, but at long distances, the depth of the power line measured by the camera often has incomplete or depth "black hole" problems. To address this problem, the present application uses a bilateral filtering algorithm to filter the first image and remove depth information (outliers) that is too large or too small. Subsequently, the power transmission line segmentation mask is morphologically processed, and the range of the power transmission line area in the mask is expanded by an image morphological algorithm (expansion method), and the depth values ​​of each point in the corresponding depth map in the range are counted. At the same time, the camera intrinsic parameter matrix is ​​calibrated to solve the three-dimensional coordinates of the habitat in the camera coordinate system.

[0099] As an example, the implementation of depth information filtering can be: the depth information within the expansion range usually contains noise and some unreasonable depth values. This application proposes to use a bilateral filter to suppress noise, smooth the depth image, and optimize the image depth information. The mathematical expression of image filtering is as follows:

[0100]

[0101] Where I(x,y) represents the filtered depth image, Ω represents the pixel area of ​​the first image, ω(i,j) is the weight of the filter (bilateral filtering algorithm) at the pixel point (i,j), n(i,j) is the current input depth image, ω p is the weighted sum of each pixel value within the filter window.

[0102] In the bilateral filtering algorithm, the weight ω(i,j) contains the spatial distance weight ω r (i,j) and pixel weight ω s (i,j).

[0103]

[0104] Where S(i,j) represents the pixel value of the pixel point (i,j) in the first image, and S(x,y) represents the pixel value of the center pixel of the neighborhood of the pixel point (i,j). s and σ r is the standard deviation of the Gaussian function, σ s The larger the value is, the smoother the image will be. r The larger the value, the blurrier the edge.

[0105] The final filtering formula of the entire bilateral filter (BF) is the aforementioned formula (5).

[0106] By using this filtering formula, the depth information of the first image can be filtered to obtain the second image.

[0107] In this embodiment, the implementation process of the image morphology algorithm is as follows:

[0108] After completing depth information filtering, the identified and extracted power lines are too small relative to their surroundings. To address the depth incompleteness issue, morphological processing is used to expand the area where the power lines are located to improve depth acquisition. Perch selection is performed based on the expansion map. The expansion operation is performed by convolving image A with kernel B, finding the local maximum of image A covered by kernel B, and then assigning this local maximum to the center of the area covered by kernel B. The morphological mathematical expression is:

[0109]

[0110] Where A(u, v) is the pixel value at coordinate (u, v) in image A. (u, v) represent the horizontal and vertical coordinates of the currently processed pixel in image A, i.e., the position of the anchor point in the image; i and j represent the offset of kernel B relative to the anchor point; and sizeB represents the size of kernel B.

[0111] After completing the depth information filtering and performing morphological processing to remove outliers, we can ensure that the transmission line image information used is the most complete and the selected habitat points are not excessively offset. The midpoint of this point set is selected as the habitat point P. t For the selected habitat point in the expanded image, the mode of the depth values ​​of the ten pixels around it is counted as the depth value s of the high-voltage line target point. Mode is the function for obtaining the mode value.

[0112]

[0113] Get the selected habitat point P t And its depth value s, so we can get the (three-dimensional space) coordinates of the habitat in the drone camera coordinate system. t The pixel coordinates of the perch on the first image and the depth value s are used to obtain the three-dimensional spatial coordinates of the perch in the drone camera coordinate system. This is a conventional method and will not be described in detail here.

[0114] Between step 130 and step 140, the method further includes:

[0115] During the flight of the UAV, the coordinates of the habitat are updated using the sparse optical flow method, the RANSAC (Random Sample Consensus) algorithm, and the PnP (Perspective-n-Point) solution strategy to obtain the updated coordinates, which are expressed as:

[0116] P t:1 =R n P t +t n (11)

[0117] Where, P t:1 represents the coordinates of the habitat at time t+1; P t represents the coordinates of the habitat at time t; R n represents the relative rotation matrix between the two first images at t+1 and t; t n Represents the relative displacement matrix between the two frames of the first image at time t+1 and t.

[0118] In this embodiment, when the UAV is flying to the habitat, the most recently updated coordinates of the habitat are used as the flight destination of the UAV in each control cycle. The latest coordinates are also the control targets of the UAV.

[0119] Understandably, the proportion of transmission lines in the camera's field of view is very small, which makes it difficult to track the transmission line feature points. Therefore, this application does not directly track the transmission line feature points, but achieves position maintenance through indirect means. Compared with feature points, it is easier to track Harris corner points based on color image acquisition, and the sparse optical flow method is used to track the corner points extracted from the continuous image. Then, the RANSAC algorithm is used to eliminate false matches, and the PnP (Perspective-n-Point) strategy is used to solve the camera relative motion for the matched feature points. The coordinate update of the camera coordinate system of the transmission line perch is completed according to the relative motion of the camera. The coordinate update is divided into three stages. The first stage is sparse optical flow tracking; the second stage is RANSAC algorithm and PnP solution.

[0120] The first stage, sparse optical flow tracking:

[0121] If the roosting point is continuously positioned during motion, the drone's own positioning steady-state error may cause the selected roosting point to be inconsistent. To address this roosting point motion issue, a sparse optical flow method is used to characterize the velocity and direction of the corresponding Harris corner points in two adjacent image frames. The displacement of the corner points is calculated by pixel displacement between adjacent images, thereby maintaining the corresponding position of the selected roosting point and ensuring that the selected roosting point is always optimized under the current evaluation criteria.

[0122] Assume that at time t, the selected habitat pixel is I(x, y, t). After dt time, the habitat moves to (x+dx, y+dy). Under the grayscale value invariance principle of the sparse optical flow method:

[0123] I(x,y,t)=I(x+dx,y+dy,t+dt) (12)

[0124] Performing Taylor series expansion on the above formula, we have:

[0125]

[0126] The parameter ε is infinitesimal and can be ignored.

[0127] Eliminating I(x,y,t) on both sides of equation (13) above, we obtain:

[0128]

[0129] The above equation (14) is the optical flow constraint equation, which reflects the correspondence between the temporal and spatial gradients in the moving image. Dividing both sides by dt, we get:

[0130]

[0131] Where u and v are the velocity vectors of the optical flow along the X and Y axes respectively, I x ,I y is the image gradient, I t is the time gradient

[0132] u=dx / dt,v=dy / dt (16)

[0133]

[0134] Get the final optical flow equation:

[0135] I x u+I y v+I t =0 (8)

[0136] For two unknown quantities u and v, a single equation cannot determine their unique solution. We can use the optical flow tracking algorithm to impose additional constraints. This application uses a sparse optical flow algorithm and the Tomasi operator to detect the motion trajectory of feature corner points, thereby indirectly maintaining the position of the habitat.

[0137] The second stage, RANSAC algorithm and PnP solution:

[0138] The optimal parameter model is found by continuous iteration. Points that do not conform to the optimal model are defined as "outliers". For the feature points matched between two consecutive frames of images, the RANSAC algorithm searches for an optimal homography matrix H so that the number of feature points that meet the matrix is ​​the largest. Points that do not meet the matrix are considered "outliers" and removed. Suppose a pair of well-matched feature points between two frames of images I1 and I2 are p1(uv) and p2(u ′ v ′ ), let the homography matrix be H, which satisfies:

[0139] p2=Hp1 (19)

[0140] Let p1 and p2 be homogeneous coordinates:

[0141]

[0142] Where h1 to h9 refer to the corresponding parameters in the matrix. Expanding it reveals that a set of matching feature points can construct two constraint equations. It is easy to prove that point p1 is transformed into p2 after H mapping. The degree of freedom of the homography matrix H is 8. The required homography matrix can be calculated through four sets of non-collinear matching feature point data. In the experiment, these four sets of non-overlapping matching feature points are randomly selected, and the homography matrix is ​​constructed based on this, and all feature points are evaluated based on this. At the same time, the number of internal points that meet the matrix conditions (that is, feature points that meet specific requirements) and the projection error are calculated. The latter is defined as the cost function, that is:

[0143]

[0144] Iterate k times to find a set with the largest number of inliers whose projection error is less than the set threshold. This inlier set is defined as a set that can effectively remove the mismatched feature point groups. It should be noted that the number of iterations k is not fixed and is expressed as:

[0145]

[0146] As an example, the confidence level p is generally set to 0.995; w is the proportion of “inliers”; and m is the minimum number of samples, which may be set to 4, for example.

[0147] After correcting for false matches, the matched feature points are subjected to a PnP motion solution. By combining a set of feature points that have been carefully removed to eliminate false matches with the depth information provided by the depth camera, a set of these 3D points and their projections can be constructed. This set of points can be used to solve the relative position of the camera coordinate system within the time interval between two consecutive frames, including rotation and translation operations. The mathematical expression is:

[0148]

[0149] where s i is depth information; is the coordinate of the three-dimensional point in the camera coordinate system of two consecutive frames; R, t are the relative rotation and displacement between the two frames respectively; K is the camera intrinsic parameter matrix; p i for projection.

[0150] Calculations are performed for all three-dimensional points in space to obtain four control points. These control points can accurately describe the spatial distribution of these three-dimensional points and record their coordinates in the camera coordinate system of the previous frame. Then, by analyzing the two-dimensional point data captured by the camera, the specific position of the control points in the camera coordinate system of the current frame can be derived. Based on the coordinate information of the four control points known in the two frames of images, any three-dimensional to three-dimensional matching algorithm is further used to achieve a more accurate registration operation, and the optimized R n and t n , so the camera coordinate system transmission line habitat coordinate P t Update, that is, use the above formula (11) to achieve coordinate update.

[0151] In step 140, the drone can automatically generate an obstacle avoidance path from the drone's current position to the perch based on the three-dimensional coordinates of the perch in the camera coordinate system and the first image captured by the depth camera, as a reference trajectory (or desired trajectory). The generation method of this obstacle avoidance path is conventional and will not be repeated here.

[0152] Please refer to Figure 2 In step 150, an iterative learning linear active disturbance rejection control strategy is preset, which can also be called an ILC-LADRC (Iterative Learning Control-Linear Active Disturbance Rejection Control) algorithm, or an ILC-LADRC control algorithm. The software module that implements the ILC-LADRC control algorithm is the ILC-LADRC controller. The ILC-LADRC controller includes an iterative learning controller (ILC) and a linear active disturbance rejection controller (LADRC). The linear active disturbance rejection controller includes a tracking differentiator, an extended state observer, an iterative learning controller, and a linear feedback controller.

[0153] In step 150, based on the relative position, a preset iterative learning linear active disturbance rejection control strategy and the reference trajectory, control parameters of the UAV are determined, including:

[0154] Step 151, using a pre-created iterative learning controller, iteratively learning the first input data to obtain input data after iterative learning as second input data, wherein the first input data includes the relative position;

[0155] Step 152: Smoothing the first input data using a pre-created tracking differentiator to obtain smoothed first input data;

[0156] Step 153: Based on the second input data and the current output data, using a pre-created extended state observer, determine an estimated value and an extended state variable corresponding to the smoothed first input data;

[0157] Step 154 ​​: Based on the smoothed first input data, the estimated value, and the expanded state variable, a control parameter of the UAV is determined using a pre-created linear feedback controller.

[0158] In step 151, a pre-created iterative learning controller is used to iteratively learn the first input data to obtain input data after iterative learning as the second input data, wherein the first input data includes the relative position, including:

[0159] The second input data is obtained by the iterative learning controller using the following learning law, where:

[0160]

[0161] Where u k:1 (t) represents the second input data; u k (t) represents the first input data; Γ p and Γ d Represents proportional learning gain and differential learning gain respectively; e k (t) represents the tracking error obtained based on the reference trajectory and the current position of the UAV; Represents the tracking error e k The derivative of (t).

[0162] In this embodiment, during the creation of the iterative learning controller, the state change process of the drone is assumed to be:

[0163]

[0164] Among them, t∈[0,T] is a finite time, x(t)∈R n , y(t)∈R m , u∈R r are the system state (including the current position of the UAV), the system output (including the control parameters of the UAV, such as torque), and the system input (the position of the habitat). n , g()∈R m is a smooth vector-valued function.

[0165] Under known desired control u d (t), by generating a series of appropriate control inputs u k (t) makes the system output y k (t) is close to the given reference trajectory r(t), that is, by repeating the control operation cycle according to a certain learning control algorithm, the control input u k (t) Wireless approach to desired control quantity u d (t), the kth repeated control operation cycle is called the kth learning iteration. r(t) is constant in the iteration domain. At the same time, when running the kth iteration, the above formula can be expressed as:

[0166]

[0167] Define tracking error e k (t) is:

[0168] e k (t) = r k (t)-yk (t) (27)

[0169] Where r k (t) can be understood as the kth iteration, the expected position of the UAV in the reference trajectory at time t; y k (t) is the actual position of the UAV at time t in the kth iteration. According to the error update method, it is divided into open-loop iteration and closed-loop iteration. The open-loop iterative control method is to let the kth control quantity plus the correction value of the output error of the kth control be the k+1th control quantity:

[0170] u k:1 (t) = Γ(u k (t),e k (t)) (28)

[0171] The closed-loop iterative control scheme takes the k+1th operation error as the adjustment value, that is:

[0172] u k:1 (t) = Γ(u k (t),e k:1 (t)) (29)

[0173] Γ is a linear or nonlinear operator.

[0174] Different learning functions correspond to different control quantity outputs. This application designs a PD-type learning law for iterative learning control, and the learning law is the above formula (24).

[0175] In step 152, the first input data is smoothed using a pre-created tracking differentiator to obtain smoothed first input data, including:

[0176] The tracking differentiator is used to smooth the first input data using the following formula to obtain the smoothed first input data, which is expressed as:

[0177]

[0178] Wherein, v1 refers to the tracking signal (such as the expected position) obtained by tracking the relative position in the first input data; v2 refers to the expected speed in the first input data; Refers to the data after smoothing the tracking signal v1; Refers to the data after smoothing the expected speed v2; v represents the actual input signal of the drone (such as the current position of the drone); h represents the filter factor; f han () represents the preset secondary discrete function; r is the speed factor.

[0179] In this embodiment, the position in the reference trajectory r(t) is the desired position of the drone. The tracking differentiator uses the desired position in the reference trajectory r(t) to perform filtering and differentiation to obtain v1 and v2. v1 can be understood as the smoothed signal of the tracking r(t), and v2 is the desired velocity of the drone.

[0180] The function of the tracking differentiator is to design a smooth transition process for the system input, thereby obtaining a smooth input signal to reduce the initial error of the system. It is used to resolve the contradiction between rapidity and overshoot in PID control, and at the same time handle the contradiction between the following speed and overshoot of the system input signal, thereby enhancing the stability and robustness of the system.

[0181] The transition process of the tracking differentiator is in the form of the above formula:

[0182] Transform the above formula into discrete form:

[0183]

[0184] In order to solve the problem that the system cannot reach the expected input value in a limited time after entering the steady state and high-frequency vibration occurs, based on the existing second-order discrete function f han To improve, the expression is as follows:

[0185]

[0186] Where r is the speed factor, which determines how quickly the system tracks the input signal. h0 is different from the sampling period h in discrete systems. As an example, h is set to 0.001 in this application. It is also possible to simplify h0 and h to the same value, but in this case, when tracking step commands, the differentiator will overshoot and cause unnecessary amplification of command noise. Therefore, it is often taken to be some integer multiple of h. As an example, h0 is set to twice h in this application.

[0187] In step 153, based on the second input data and the current output data, a pre-created extended state observer is used to determine an estimated value and an extended state variable corresponding to the smoothed first input data, including:

[0188] The estimated value of the first input data and the expanded state variable are determined by the expanded state observer using the following first expansion formula, which is expressed as:

[0189]

[0190] Where e represents the tracking error; z1 and z2 represent the estimated values ​​corresponding to the tracking signal v1 and the expected velocity v2, respectively; β1, β2, and β3 represent the first state error feedback gain, the second state error feedback gain, and the third state error feedback gain, respectively; z3 represents the expanded state variable; fal() represents the preset continuous power function; α1 and α2 are parameters with values ​​between (0, 1); δ is the interval length of the linear segment; b refers to the proportional factor; and u refers to the control input data (such as the torque and flight direction of the UAV).

[0191] Understandably, during the flight of a drone, the high-speed rotation of the propeller will affect most of the sensors on the aircraft, including causing parameter changes, generating noise, and also being affected by external wind disturbances and environmental interference. All these internal and external disturbances are unknown disturbances. In order to further estimate the internal information of the system and perform better calculations on the control quantity, this application adopts an extended state observer. Its basic principle is to reconstruct the system through input and output, and expand the total disturbance into a new system variable for estimating disturbances from inside and outside. For the second-order system of the drone, there are:

[0192]

[0193] Where f(x1, x2, w(t), t) is the total unknown disturbance experienced by the UAV, w(t) is the external disturbance, u is the control input, and b is the proportional factor of the control variable.

[0194] make That is, the expansion state makes the original system become a linear system, then the converted formula can be obtained:

[0195]

[0196] Based on the above formula (36), an extended state observer model is established, which is the above formula (33).

[0197] The rationality of the Extended State Observer (ESO) parameters directly impacts the controller's effectiveness. Where β1, β2, and β3 are the corresponding state error feedback gains, all greater than 0. This directly impacts the ESO's convergence speed and is subsequently self-tuned through an iterative learning algorithm. α1 < α1, α2 < 1, and empirically, α5 is generally a good value.

[0198] In the aforementioned formula (33), by changing the various parameters (such as β1, β2, β3, α1, α2), the expanded state variables can be well estimated. Together with the model uncertainty and external disturbances, they are tracked and compensated, playing a role in anti-disturbance.

[0199] In step 154, based on the smoothed first input data, the estimated value, and the expanded state variable, a pre-created linear feedback controller is used to determine the control parameters of the UAV, including:

[0200] Based on the linear feedback controller, the first expansion formula is converted into a second expansion formula to replace the nonlinear function in the extended state observer with a linear function. The second expansion formula is:

[0201]

[0202] Where b refers to the proportional factor; u refers to the control input data;

[0203] Based on the estimated values ​​obtained by the two expansion formulas, the expansion state variables, the smoothed first input data and the dynamic model of the UAV, the control parameters of the UAV are determined. The control parameters include the lift u corresponding to the x-axis, y-axis and z-axis in the UAV's body coordinate system. x 、u y 、u z , expressed as:

[0204]

[0205] Where U1 represents the total lift of the UAV determined based on the dynamic model; θ represents the pitch angle, which is the rotation angle of the UAV around the horizontal axis (y-axis) of the body coordinate system, reflecting the longitudinal pitch motion of the fuselage (similar to the head-up or head-down posture of an airplane); ψ represents the yaw angle, which is the rotation angle of the UAV around the vertical axis (z-axis) of the body coordinate system; φ represents the roll angle, which is the rotation angle of the UAV around the longitudinal axis (x-axis) of the body coordinate system.

[0206] Traditional PID control algorithms typically use a linear combination of errors to calculate a signal output. However, this linear calculation method struggles to meet the high-performance, stability, and precision requirements of control systems. The nonlinear error feedback control law uses a nonlinear combination of the system variable errors to generate an output variable. This allows for more flexible adjustment of the relationship between proportional and differential, reducing steady-state error by orders of magnitude. To avoid high-frequency steady-state oscillation, the fal() function is used to calculate u0. The control variable u is formed based on u0 and the expanded state variable z3, improving controller performance.

[0207]

[0208] Where u0 is the intermediate control quantity generated by weighted combination of errors e1 and e2 based on the nonlinear function fal; k p , k d are the proportional gain parameter and differential gain parameter of the controller.

[0209] By decomposing and analyzing the ADRC, it is found that the desired signal is processed by a tracking differentiator to obtain a smooth tracking signal and a differential signal; the errors between the obtained tracking signal, the differential signal and the output signal of the extended state observer are used as the input signals of the controller, and then the basic control quantity is obtained through nonlinear state error feedback. The basic control quantity is combined with the expanded state observed by the extended state observer to obtain the final control quantity, thereby achieving a specific controller with anti-disturbance capability.

[0210] A controller that is suitable for the system and can improve the control performance, a key link is whether the parameter adjustment is appropriate. For universal controllers, the difficulty lies more in the parameter setting, and the active disturbance rejection controller is no exception. Its fast and effective tuning method and tuning rules have always required a lot of practice to achieve satisfactory control effects. There are many parameters that need to be adjusted in the traditional ADRC algorithm. For its parameter setting, the bandwidth method, the empirical method and the artificial intelligence-based method are usually used. Since the nonlinear ADRC control algorithm has many parameters and there is always a lack of theoretical proof of its closed-loop stability, this application adopts a control algorithm based on the improvement of the classic ADRC, namely the linear active disturbance rejection controller (LADRC), which mainly replaces the nonlinear function in the extended state observer with a linear function, that is, converts the first expansion formula into the second expansion formula.

[0211] At the same time, the nonlinear state error feedback control law is transformed into a linear one, which is the following formula. Based on the basic anti-disturbance performance of the existing active disturbance rejection controller, the parameter tuning process is simplified, and the control law adopts the PD control combination, as shown below:

[0212]

[0213] In the formula, u0 represents the intermediate control variable. The following is a control strategy design based on the mathematical model of the quadrotor drone. There are two subsystems in the entire drone system: the position subsystem and the attitude subsystem. Among them, the drone dynamic model is:

[0214]

[0215] Among them, x, y, z are the coordinates of the drone’s position in three-dimensional space. is the acceleration in the corresponding direction. U1 is the total lift of the drone; U2, U3, and U4 are related to the control torque around the axis, which can be understood as the control torque corresponding to the x, y, and z axes; m is the mass of the drone, g is the acceleration due to gravity, and I x ,I y ,I zis the moment of inertia about the body axis. θ is the pitch angle, ψ is the yaw angle, and φ is the roll angle. p, q, and r represent the angular velocity components of the drone's roll, pitch, and yaw in the body coordinate system. J represents the rotor's moment of inertia (related to the gyroscopic effect). Ω1 to Ω4 represent the rotational speeds of the drone's four rotors.

[0216] By the expected position (x d ,y d , z d ) and the desired yaw angle ψ d , combined with the ILC-ADRC controller, the virtual control quantity u of the position loop is obtained x ,u y ,u z , which can represent the lift of the drone in the x, y, and z axes respectively. The calculation method refers to the above formula (38). The desired pitch angle θ of the drone can be calculated by solving the above formula d , roll angle φ d , total lift U1, get the torque output τ b =[U2 U3 U4] T .

[0217] In step 160, the drone can automatically adjust the flight direction and motor torque based on the control parameters, so that the drone flies toward the habitat along the reference trajectory, so that the drone arrives at the habitat or arrives within a preset distance range of the habitat. The preset distance range can be flexibly set according to actual conditions, for example, 5 centimeters.

[0218] After step 160, the method may further include:

[0219] When the UAV flies to the perch, controlling the clamping mechanism on the UAV to clamp the power line so that the UAV stops at the perch;

[0220] When the drone receives a charging instruction allowing charging, the inductive charging module on the drone is controlled to charge the battery module of the drone.

[0221] In this embodiment, the power line is rigidly / flexibly locked by a clamping mechanism, so that the drone can be fixedly docked on the power line, eliminating the displacement of the drone caused by airflow or mechanical vibration.

[0222] The inductive charging module can achieve contactless charging through existing magnetic resonance coupling technology.

[0223] The charging instruction to allow charging can be issued remotely by the operator or automatically generated after the drone docks on the power line. There is no specific limitation on the triggering method of the charging instruction.

[0224] The inventors have conducted experiments to perform image segmentation and habitat location on the power lines in the first image, such as Figure 3 、 Figure 4 As shown. Among them, Figure 3 (a) shows the original image (first image) obtained by photographing the power line. Figure 3 (b) is the transmission line segmentation mask obtained based on the image segmentation model. Figure 3 (c) is the wire mask image after bilateral filtering based on power line extraction. Figure 4 Schematic diagram of the effect of the habitat point determined from the first image.

[0225] In order to verify the feasibility of the algorithm, Python is used to test the simulation model and control performance of the quadcopter. First, under the condition of random interference, based on the set parameters, the initial position of the quadcopter (0, 0, 0) is moved to (1, 2, 3) in the simulation system. The simulation results of the ILC-LADRC control algorithm are as follows: Figure 5 As shown in Figure 3, the tracking errors of the x, y, and z axes eventually tend to 0, achieving final bounded consistency.

[0226] The above results show that the control algorithm has a good effect in dealing with fixed point control, with good convergence speed and anti-interference ability. The tracking performance of the control algorithm is verified below. The position trajectory of the UAV is tracked as a spiral ascending trajectory and an ∞-shaped trajectory respectively. Figure 6 and Figure 7 ,The simulation results show that the trajectory tracking effect is good.

[0227] During the experimental phase, the drone hovered in front of the power line it was planning to perch on, ensuring it was within the camera's recognizable range. The recognition node then extracted the power line image information and sent it to the positioning and tracking node. The positioning node first selected the image feature points most suitable for perching, determined the three-dimensional position of the perch relative to the drone using depth information and ranging principles, and then used a tracking algorithm to track the perch's three-dimensional position in real time. Once the perch's three-dimensional position was acquired, it was sent to the control node for autonomous flight. Once the perch's three-dimensional position was determined, the drone was controlled using the designed position-based parameter self-tuning linear active disturbance rejection control algorithm, following the previously designed perch reference trajectory, until it reached the virtual control point, ensuring the gripper was positioned beneath the power line and completing the approach phase.

[0228] Based on the above design, the method provided in this application realizes transmission line identification and habitat location through a lightweight network. The linear active disturbance rejection controller (LADRC) based on iterative learning control (ILC) addresses the difficulty in parameter design and tuning in the active disturbance rejection controller. The active disturbance rejection algorithm is first linearized, and then the iterative learning control algorithm is used to perform real-time self-tuning of the parameters in the linear active disturbance rejection control. By designing a linear extended state observer, the internal uncertainty and external disturbance of the system are observed and compensated in real time. The controller observer bandwidth is adjusted in real time through iterative learning control to achieve improvement and enhancement of system control performance.

[0229] Through the description of the above implementation methods, those skilled in the art can clearly understand that the present application can be implemented through hardware or by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each implementation scenario of the present application.

[0230] In the embodiments provided in the present application, it should be understood that the disclosed systems and methods can also be implemented in other ways. The system and method embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to the multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of code, and a part of the module, program segment or code includes one or more executable instructions for implementing the specified logical function. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart can be implemented by a dedicated hardware-based system that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions. In addition, the functional modules in the various embodiments of the present application can be integrated together to form an independent part, or each module can exist alone, or two or more modules can be integrated to form an independent part.

[0231] The above description is merely an embodiment of the present application and is not intended to limit the scope of protection of the present application. For those skilled in the art, various modifications and variations of the present application are possible. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A method for autonomous visual docking and perching of inspection drones based on online learning and anti-interference control, characterized in that: The method comprises: Based on a video stream of a transmission line inspection scene shot by a drone, an image for transmission line inspection is obtained as a first image, where the first image carries depth information corresponding to a pixel point; Recognize the first image based on the trained image segmentation model to obtain a recognition result; When the recognition result indicates that a power line exists in the first image, determining a perch corresponding to the UAV and a relative position between the perch and the UAV from a power line map region in the first image; Based on the relative position, generating a reference trajectory representing the flight of the drone to the habitat; Determining control parameters of the UAV based on the relative position, a preset iterative learning linear active disturbance rejection control strategy, and the reference trajectory; Based on the control parameters, the UAV is controlled to fly toward the habitat so that the UAV stops at the habitat.

2. The method according to claim 1, characterized in that Recognizing the first image based on the trained image segmentation model to obtain a recognition result includes: Based on the trained U-Net model, the first image is recognized to obtain the recognition result, wherein the loss function D loss of the U-Net model is: D loss=WCE loss+Focal loss Where WCE loss represents the cross entropy loss function; w is the foreground loss adjustment weight in the image; y is the sample label; is the probability of the foreground predicted by the U-Net model, wherein the foreground includes the power line; Focal loss represents the focal loss function; γ is the parameter for adjusting the weight factor, p t It represents the probability that the sample belongs to the true label category when the sample is a true label; p represents the probability that the sample predicted by the U-Net model belongs to the positive class.

3. The method according to claim 1, characterized in that Determining a perch corresponding to the UAV and a relative position between the perch and the UAV from a power line map region in the first image includes: Based on the bilateral filtering algorithm, the depth information in the first image is filtered to obtain the filtered first image as the second image, which is expressed as: Where BF represents the depth information of the corresponding pixel in the first image after filtering, I p represents the depth information of the corresponding pixel point p in the first image; ω p is the weighted sum of each pixel value in the filtering window of the bilateral filtering algorithm; ω s (i, j) represents the pixel weight of the bilateral filtering algorithm at the pixel point (i, j); ω r (i, j) represents the spatial distance weight of the bilateral filtering algorithm at the pixel point (i, j); Ω represents the pixel area of ​​the image; S(i, j) represents the pixel value of the pixel point (i, j) in the first image, and S(x, y) represents the pixel value of the central pixel in the neighborhood of the pixel point (i, j); σ s and σ r and ω s (i,j),ω r The standard deviation of the Gaussian function corresponding to (i, j); dilating the power transmission line diagram region in the second image using an image morphological algorithm to obtain a third image; From the third image, a target pixel point located on the power line that meets preset conditions is selected as the habitat point of the UAV, and based on the mode of the depth values ​​of a preset number of pixel points around the target pixel point, the coordinates of the habitat point in the camera coordinate system of the UAV are obtained as the relative position.

4. The method according to claim 3, characterized in that Between the step of determining a perch corresponding to the UAV and a relative position between the perch and the UAV in the power line map region in the first image, and the step of generating a reference trajectory representing a flight of the UAV to the perch based on the relative position, the method further includes: During the flight of the UAV, the coordinates of the habitat are updated using the sparse optical flow method, the RANSAC algorithm, and the PnP solution strategy to obtain the updated coordinates, which are expressed as: P t+1 =R n P t +t n Where, P t+1 represents the coordinates of the habitat at time t+1; P t represents the coordinates of the habitat at time t; R n represents the relative rotation matrix between the two first images at t+1 and t; t n Represents the relative displacement matrix between the two frames of the first image at time t+1 and t.

5. The method according to claim 1, wherein Determining control parameters of the UAV based on the relative position, a preset iterative learning linear active disturbance rejection control strategy, and the reference trajectory includes: Using a pre-created iterative learning controller, iteratively learning the first input data to obtain input data after iterative learning as second input data, wherein the first input data includes the relative position; Using a pre-created tracking differentiator, smoothing the first input data to obtain smoothed first input data; Based on the second input data and the current output data, using a pre-created extended state observer, determining an estimated value and an extended state variable corresponding to the smoothed first input data; Based on the smoothed first input data, the estimated value and the expanded state variable, a control parameter of the UAV is determined using a pre-created linear feedback controller.

6. The method according to claim 5, characterized in that Using a pre-created iterative learning controller, iteratively learning is performed on the first input data to obtain input data after iterative learning as the second input data, wherein the first input data includes the relative position, including: The second input data is obtained by the iterative learning controller using the following learning law, where: Where u k+1 (t) represents the second input data; u k (t) represents the first input data; Γ p and Γ d Represents proportional learning gain and differential learning gain respectively; e k (t) represents the tracking error obtained based on the reference trajectory and the current position of the UAV; Represents the tracking error e k The derivative of (t).

7. The method according to claim 5, characterized in that Using a pre-created tracking differentiator, smoothing the first input data to obtain smoothed first input data includes: The tracking differentiator is used to smooth the first input data using the following formula to obtain the smoothed first input data, which is expressed as: Wherein, v1 refers to the tracking signal obtained by tracking the relative position in the first input data; v2 refers to the expected speed in the first input data; Refers to the data after smoothing the tracking signal v1; Refers to the data after smoothing the expected speed v2; v represents the actual input signal of the UAV; h represents the filter factor; f han () represents the preset secondary discrete function; r is the speed factor.

8. The method according to claim 7, characterized in that Based on the second input data and the current output data, using a pre-created extended state observer, determining an estimated value and an extended state variable corresponding to the smoothed first input data, including: The estimated value of the first input data and the expanded state variable are determined by the expanded state observer using the following first expansion formula, which is expressed as: Where e represents the tracking error; z1 and z2 represent the estimated values ​​corresponding to the tracking signal v1 and the expected velocity v2, respectively; β1, β2, and β3 represent the first state error feedback gain, the second state error feedback gain, and the third state error feedback gain, respectively; z3 represents the expanded state variable; fal() represents the preset continuous power function; α1 and α2 are parameters with values ​​between (0, 1); δ is the interval length of the linear segment; b refers to the proportional factor; and u refers to the control input data.

9. The method according to claim 8, characterized in that Determining control parameters of the UAV using a pre-created linear feedback controller based on the smoothed first input data, the estimated value, and the expanded state variable, including: Based on the linear feedback controller, the first expansion formula is converted into a second expansion formula to replace the nonlinear function in the extended state observer with a linear function. The second expansion formula is: Based on the estimated values ​​obtained by the two expansion formulas, the expansion state variables, the smoothed first input data and the dynamic model of the UAV, the control parameters of the UAV are determined. The control parameters include the lift u corresponding to the x-axis, y-axis and z-axis in the UAV's body coordinate system. x 、u y 、u z , expressed as: Wherein, U1 represents the total lift of the UAV determined based on the dynamic model, θ represents the pitch angle, ψ represents the yaw angle, and φ represents the roll angle.

10. The method according to claim 1, characterized in that The method further comprises: When the UAV flies to the perch, controlling the clamping mechanism on the UAV to clamp the power line so that the UAV stops at the perch; When the drone receives a charging instruction allowing charging, the inductive charging module on the drone is controlled to charge the battery module of the drone.

Images