A unmanned aerial vehicle visual servo control method for power transmission tower cross beam

Abstract

The application discloses a kind of unmanned aerial vehicle vision servo control methods for power transmission tower crossbeam, for coordinating position, distance and heading adjustment in the approach process of crossbeam.Utilize airborne image and attitude information to construct virtual image plane, four-dimensional features and interactive matrix are constructed by crossbeam double line endpoints;Translate dynamics is incorporated into second-order image equation, and force command and heading angle acceleration command are directly generated according to image error and its rate of change;Total thrust and roll, pitch reference are determined by force command, and heading reference is integrated to generate, and three-axis torque is calculated by linear sliding mode attitude controller.The method connects crossbeam geometric characteristics, image dynamics and flight control input, forms two-layer structure without translational velocity tracking loop.Under the condition that the given working domain and the collective disturbance are bounded, the image and attitude error are consistent and ultimately bounded.

Description

Technical Field

[0001] This invention relates to the field of UAV visual servo and flight control technology, specifically to a control method that uses the double-line feature of a power transmission tower beam to guide a quadcopter UAV to complete approach and alignment. Background Technology

[0002] When conducting close-range inspections and pre-contact positioning near the crossbeams of power transmission towers, drones need to adjust their lateral position, altitude, and heading as they approach the target to ultimately achieve a relative position and orientation that meets operational requirements. Simply keeping the crossbeam centered in the image is insufficient to determine the approach distance and relative heading. The upper and lower edges of the crossbeam have a clear geometric relationship; by utilizing its center position, edge spacing, and left-right imaging differences to construct visual feedback, the aforementioned approach requirements can be unified into the image space.

[0003] Quadrotor UAVs rely on attitude changes to adjust thrust direction and generate translational motion, causing the onboard camera to tilt accordingly. Therefore, changes in the spool image reflect not only the relative motion between the UAV and the target but also the effects of roll and pitch. If these effects are not coordinated in feature modeling and control design, changes in position, scale, and orientation in the image cannot directly correspond to the required flight adjustments. This necessitates that spool approach control simultaneously address imaging coupling and the relationship between thrust direction and attitude.

[0004] Visual servoing utilizes image feature errors to generate motion feedback, providing a fundamental method for robot localization relative to targets. Related research has applied image feedback to the translational motion control of quadcopters and further combined it with virtual cameras and aerial operating platform dynamics to establish a connection between image dynamics and platform motion. These studies demonstrate that visual control not only requires the appropriate selection of features but also the matching of feature changes with the aircraft's dynamic input. For beam approach tasks, lateral, vertical, depth, and relative heading errors should be incorporated into a unified control design, based on bilinear geometry.

[0005] First-order image relationships describe the connection between feature change rate and spatial velocity, while UAVs actually change their motion state through thrust and torque. If vision control outputs velocity commands, a velocity tracking loop needs to be configured and its connection with attitude control needs to be handled. Therefore, for beam approach and alignment, it is necessary to establish a direct connection from bilinear features and second-order image dynamics to thrust and attitude inputs, simplifying the control hierarchy while compensating for motion coupling. Summary of the Invention

[0006] This invention aims to address the problems of indirect connection between visual features and flight dynamics inputs during beam approach, and the need for coordination between translation and attitude movements. It provides a visual servo control method for UAV beams on power transmission towers. This method utilizes bilinear features on a virtual image plane to describe approach and alignment requirements, incorporates translational dynamics into second-order image equations, directly generates force commands and yaw angular acceleration commands, and then completes control through total thrust, attitude reference, and three-axis torque, forming a two-layer structure of an outer visual loop and an inner attitude loop.

[0007] The method uses image error as the control target and total thrust and triaxial torque as the execution outputs, ensuring that the beam geometry, visual feedback, and flight control inputs correspond to each other. Specifically, it includes the following five steps.

[0008] Step S1: Information Acquisition and Dynamic Modeling

[0009] The system acquires images and depth information of the crossbeam from the airborne camera, as well as the actual speed, attitude, and angular velocity information of the UAV; it also sets camera calibration parameters, crossbeam geometric parameters, and desired approach distance. Images and depth are used to determine the relative geometric relationship of the target, while flight status is used to calculate motion coupling terms and attitude feedback.

[0010] Assume the third axis of the world coordinate system points vertically upwards. The representation of the drone's speed in the world system. For quality, This represents the total thrust. A rotating matrix is ​​used. ,in These are roll angle, pitch angle, and yaw angle, respectively. This is the basic rotation matrix.

[0011] Assuming all axes of the machine system coincide with the principal axes of inertia, and the moment of inertia is... Within the small roll and pitch angle range, using Euler angular velocity to approximate the body angular velocity, the attitude dynamics can be written as:

[0012]

[0013] in To control the torque. The subscripted parts in each dynamic equation. The unified representation of lumped disturbances in the corresponding equations includes external disturbances, model uncertainties, and other unmodeled effects. Actual velocity, attitude, and angular velocity are provided by the airborne measurement system.

[0014] Step S2 Virtual Imaging and Image Feature Modeling

[0015] The camera is rigidly connected to the drone body, and the translational offset between the camera's optical center and the UAV's center of mass is ignored. To ensure that the beam features primarily reflect approach distance, relative position, and heading, a virtual camera that retains heading motion is constructed using attitude information, eliminating the direct impact of roll and pitch on imaging. The rotation matrix from its coordinate system to the world coordinate system is:

[0016]

[0017] make , The virtual second axis points downwards, with an angular velocity of... ,in , Let the rotation matrix from the machine body to the virtual frame be... The translational dynamics are:

[0018]

[0019] in It is the acceleration due to gravity. This represents the actual thrust in the virtual frame. The yaw rotation is... The phases cancel each other out, and the direction of the thrust is:

[0020]

[0021] set up For the camera intrinsic parameter matrix, This is the camera-to-body mounting rotation matrix. It sets the pixel homogeneous coordinates... Transform into Divide by its third component to obtain the virtual image coordinates. .

[0022] The upper and lower edges of the crossbeams are parallel, and the actual spacing is [missing information]. Take two endpoints at each of the left and right reference sections, and assign subscripts... These represent the top left, bottom left, top right, and bottom right, respectively. Define the image center and edge spacing:

[0023]

[0024]

[0025] Take the expected distance ,make Constructing image features :

[0026]

[0027] The four components reflect lateral offset, vertical offset, depth scale, and relative heading, respectively. The image center describes the lateral and vertical position, the average edge spacing reflects the depth scale, and the difference in left and right edge spacing reflects the relative heading. Thus, the endpoints of the same set of beams can provide the four-dimensional visual feedback needed for approach and alignment.

[0028] Let the coordinates of the center of the beam in the virtual camera system be... The direction angle of the beam is The distance between the left and right reference sections is The equation of motion for the stationary beam is:

[0029]

[0030] The depth of the left and right endpoints is , .Depend on , When the depth difference between the two sides is small, we have:

[0031]

[0032] Generalized speed Differentiating the above characteristic relationship and substituting it into the equation of relative motion, we obtain the nominal characteristic equation:

[0033]

[0034]

[0035] when , At this time, depth is positive and the interaction matrix is ​​invertible. Geometric quantities are obtained from the calibration image and depth information, and the derivative of the interaction matrix is ​​calculated based on relative motion:

[0036]

[0037] Step S3: Design and Stability Analysis of the Vision Control Law

[0038] The following design is carried out within the working domain of the UAV, where the desired attitude is maintained with small roll and pitch angles. It is assumed that the beam features remain continuously visible, and the depth difference between the two sides satisfies the aforementioned approximate conditions. Bounded.

[0039] Take fixed expected features Define image error ,have To directly apply image error feedback to the flight input, the first-order characteristic equation is differentiated, and translational dynamics are incorporated into the second-order image model, using force commands. and heading angle acceleration command For input, write as:

[0040]

[0041] To simultaneously adjust image error and its rate of change, a combined error is defined. ,in , Obtained from continuous images, This is the actual measured speed. (Selected) Construct positive definite Lyapunov functions:

[0042]

[0043] Depend on Differentiating along the error system, we get:

[0044]

[0045] To eliminate overlapping terms and make Generate negative feedback, select .Depend on Let the nominal effect of the control input satisfy:

[0046]

[0047] To uniformly represent translational and heading control requirements, a generalized acceleration command is defined:

[0048]

[0049] By mapping the generalized acceleration command to the translational dynamics of the virtual system, compensating for the rotation and gravity terms of the coordinate system, and directly inverse solving, the force command and yaw angular acceleration command are obtained:

[0050]

[0051] The aforementioned control law compensates for image motion coupling through the interaction matrix and its derivative, and then dynamically converts the required error into force and yaw angular acceleration inputs, eliminating the need for a separate translational velocity tracking loop. Substituting this into the image model yields the closed-loop equation:

[0052]

[0053] This closed-loop structure causes the error cross terms in the derivative of the Lyapunov function to cancel each other out, thus yielding:

[0054]

[0055] Assume there exist finite constants ,make Applying Young's inequality:

[0056]

[0057] Therefore:

[0058]

[0059] in , By the principle of comparison:

[0060]

[0061] therefore, and Consistent and ultimately bounded; by The rate of change of image error also eventually becomes bounded. At that time, the image error and its rate of change exponent converge to zero.

[0062] Step S4 Command Mapping and Attitude Reference

[0063] remember ,make .Depend on Based on the magnitude and direction of the thrust, calculate the total thrust and the desired attitude:

[0064]

[0065] The expected roll and pitch are constructed within the current virtual frame. Find within small-angle branches Taking the derivative of the above mapping yields The heading reference is obtained by integrating the angular acceleration commands sequentially:

[0066]

[0067] therefore Attitude loop tracking by Reference for composition.

[0068] The above mapping requirements , Maintain a positive margin. Assume... It is second-order continuously differentiable, and its first and second derivatives are uniformly bounded. Furthermore, the first and second derivatives required for attitude reference can be obtained and are uniformly bounded.

[0069] Step S5: Attitude control law design and stability analysis

[0070] The attitude loop is used to ensure the aircraft follows the thrust direction and heading required by the visual outer loop. All three attitude channels use the same control structure. Taking the roll channel as an example, it is defined as follows: Select a linear sliding surface:

[0071]

[0072] Its rate of change .

[0073] Choose the Lyapunov function:

[0074]

[0075] Differentiating along the sliding mode variable and substituting into the rolling dynamics, we get:

[0076]

[0077] To decay the sliding mode variable, linear feedback is used to provide convergence, and signed feedback is added to suppress the effects of disturbances. Let the part within the parentheses that does not contain disturbances satisfy:

[0078]

[0079] The inertial coupling term in the roll dynamics is compensated, and the desired roll angular acceleration is added as a feedforward. The control torque is obtained by inverse solution:

[0080]

[0081] in , Let be the sign function. Substituting, we get:

[0082]

[0083] Assume there exist finite constants ,make Discard non-positive terms. Applying Young's inequality, we obtain:

[0084]

[0085] By the principle of comparison, we have:

[0086]

[0087] therefore, Uniformity is eventually bounded, and:

[0088]

[0089] By the definition of sliding mode variables, the roll angle error satisfies:

[0090]

[0091] The equation is as follows: For a stable first-order system as input, therefore Uniformity is ultimately bounded, and has the following:

[0092]

[0093] Combination The roll angle error and its rate of change are both consistent and eventually bounded. The above conclusions do not require sign feedback gain. It is greater than the upper bound of the disturbance.

[0094] Pitch and yaw channels are taken separately , The control torque is solved in reverse using the same process, compensating for the respective inertial coupling terms and using the second derivative of the corresponding attitude reference. Under the condition that their respective lumped disturbances are bounded, it can be similarly proven that the corresponding attitude angle errors and their rates of change are ultimately bounded.

[0095] Combining the conclusions of the two loops, under the above working domain and bounded conditions, the image error and its rate of change, as well as the attitude angle error and its rate of change, are all ultimately bounded.

[0096] The system includes a quadcopter UAV, an onboard camera, a depth and flight status measurement unit, and an onboard processor. The camera and measurement unit provide crossbeam and flight status information. The processor executes steps S1 to S5, converting visual errors into total thrust and three-axis torque, and sends them to the rotor drive unit. The updated image and flight status are used in the next control cycle, forming a closed-loop feedback. The program is stored on a computer-readable storage medium.

[0097] The present invention has the following beneficial effects:

[0098] (1) Virtual imaging is used to eliminate the direct influence of roll and pitch on the image, and four-dimensional features are constructed based on the center, spacing and left-right differences of the double beam lines, so that the lateral, vertical, depth and relative heading have clear geometric meanings. This feature structure unifies the approach distance and position and heading alignment requirements into the image space, which is convenient for coordinated adjustment in the same control law.

[0099] (2) Translational dynamics are directly incorporated into the second-order image equations. The interaction matrix changes, coordinate system rotations, and gravity terms are compensated in the control law, establishing a direct mapping from image errors to force commands, thus eliminating the need for independent translational velocity command generation and tracking. Under the nominal model, each image error channel forms an independent second-order damped dynamic, compensating for the coupling effect between channels, and the error convergence characteristics can be set by controlling the gain.

[0100] (3) The total thrust and roll and pitch references are determined by the magnitude and direction of the force command, and then driven by the linear sliding mode attitude controller together with the heading reference, so as to realize the motion requirements of the visual outer loop as thrust and three-axis torque. Under the given working domain and bounded conditions of lumped disturbance, the image and attitude error are consistent and eventually bounded, providing a stability basis for visual feedback and attitude adjustment during beam approach. Attached Figure Description

[0101] Figure 1 This is a flowchart of the two-layer visual servo control method of the present invention.

[0102] Figure 2 This is a schematic diagram of a scene where a drone approaches a crossbeam, the real image plane, and the virtual image plane.

[0103] Figure 3 The virtual system velocity, the first three terms of generalized acceleration, attitude reference, and total thrust command curves are presented in the four-point projection continuous feedback test.

[0104] Figure 4 This is a schematic diagram of the imaging process under camera spatial trajectory, virtual image trajectory, and commanded posture. Detailed Implementation

[0105] The following is combined Figure 1 and Figure 2 The implementation process of this invention is illustrated using an example of a quadcopter drone equipped with a forward-looking RGB-D camera approaching a stationary crossbeam. The upper and lower edges of the crossbeam are parallel, and the control objective is to achieve lateral, vertical, and directional alignment while reaching the desired distance. Figure 1 The outer visual loop is responsible for converting beam imaging deviations into force and yaw angular acceleration commands, while the inner attitude loop is responsible for tracking the corresponding attitude reference. The onboard processor executes the following steps in a loop.

[0106] Step S1 Information Collection and Initialization

[0107] Read camera calibration parameters, input beam geometry parameters and desired distance, and acquire and synchronize images, depth, and flight status. Settings Set the initial heading reference values ​​to the current heading angle and heading angular velocity.

[0108] Step S2 Virtual Imaging and Feature Calculation

[0109] like Figure 2 As shown, the endpoints of the crossbeam are projected onto a virtual image plane using the current attitude, and then four-dimensional features are calculated from the image center, average edge spacing, and the difference between the left and right edge spacing. Relative geometric quantities are obtained by combining depth information, and the interaction matrix and its derivative are calculated, enabling subsequent visual control to update with changes in relative position and heading. The control process satisfies the aforementioned working domain conditions.

[0110] Step S3: Computer Vision Control Instructions

[0111] Feature errors and their rates of change are calculated from continuous images to construct combined errors and obtain generalized acceleration commands. The first three components are converted into force commands through virtual frame dynamics compensation to adjust the translational motion of the UAV; the fourth component serves as a heading angular acceleration command to adjust the relative orientation of the UAV to the crossbeam.

[0112] Step S4: Generate total thrust and attitude reference

[0113] The magnitude of the force command determines the total thrust, the direction of the force determines the desired roll and pitch angles, and the derivatives required for attitude control are calculated using the mapping relationship. The yaw acceleration command is integrated sequentially to update the desired yaw angular velocity and yaw angle, thereby converting the four-dimensional control requirements of the visual outer loop into total thrust and three-axis attitude references.

[0114] Step S5 Attitude Control and Rotor Execution

[0115] The three-axis attitude reference is compared with the current attitude, and sliding mode variables and control torques are calculated based on the attitude error and its rate of change. Total thrust and three-axis torque are executed by the rotor drive unit to adjust the thrust magnitude and direction. In the next cycle, images and flight status are reacquired, visual commands and attitude control outputs are updated, and relative deviations during the beam approach process are continuously corrected.

[0116] Visual outer ring simulation

[0117] This embodiment examines the approach process of the visual outer loop, control commands, and camera imaging changes through numerical simulation. The simulation employs an ideal execution model without external disturbances or measurement noise, directly executing force commands and yaw angular acceleration commands, and integrating the translational dynamics and relative motion equations of the virtual system. The image change rate is analytically calculated by the imaging model, and the dynamics of the attitude inner loop are not included in this simulation.

[0118] The simulation duration was 24 seconds. The UAV mass was set to 1.2 kg, the initial world frame position was (-1.1, -0.1, 0.9) m, the initial heading angle was 18°, and both the initial velocity and heading angular velocity were zero. The center of the crossbeam was located at (0, 6, 1.8) m, the distance between the left and right reference sections was 1.8 m, the distance between the top and bottom edges was 0.45 m, the desired approach distance was 1.5 m, corresponding to the desired position of (0, 4.5, 1.8) m. The camera resolution was 1280×720, and the focal length in both directions was 680 pixels. Control parameters were set. , , .

[0119] Three sets of tests were set up: a nominal approximate image model was used to check the design error dynamics; a four-point precise projection continuous feedback was used to check the applicability of the approximate image model under precise projection feedback; and a 50 Hz command update under four-point precise projection was used to examine the approaching process during discrete updates. All three sets of tests used the same control law, approximate interaction matrix, gain, and initial values; the 50 Hz test maintained the force and heading angular acceleration commands at zero order.

[0120] The errors of the four image features in all three test sets approached zero, and the error curves were similar. Using a position error of less than 5 cm and a heading error of less than 2°, maintained within the remaining simulation time, as criteria, the three test sets reached these ranges at 11.74 s, 11.76 s, and 11.76 s, respectively. This indicates that, under the given initial values ​​and parameters, using precise projection and a 50 Hz command update can still maintain a convergence process close to the nominal model, achieving synchronous adjustment of position and heading.

[0121] Figure 3 The motion and command changes of the four-point projection continuous feedback test are presented. The peak velocity modulus is approximately 1.153 m / s², the initial depth acceleration is approximately 2.263 m / s², and the peak total thrust command is approximately 12.508 N. The peak values ​​of the roll and pitch reference absolute values ​​are approximately 12.54° and 2.00°, respectively. The force and attitude commands in the initial stage are used to change the approach distance and correct position deviations. As the image error decreases, the velocity and acceleration gradually approach zero, and the total thrust approaches the gravity equilibrium value, reflecting the transition from approach motion to equilibrium at the target.

[0122] like Figure 4 As shown, the camera approaches the target position from its initial position, and the four endpoints in the virtual image move towards the desired position, remaining within the image boundaries during this simulation. The spatial trajectory and the image trajectory together illustrate the adjustment process of the approach distance, position, and heading. The view cone in the figure represents the orientation of the virtual camera at different times; the lower right figure shows the image position of the crossbeam in the camera under the desired attitude calculated according to the force command.

Claims

1. A visual servo control method for unmanned aerial vehicles (UAVs) oriented towards the crossbeams of power transmission towers, characterized in that, A two-layer control structure consisting of a visual outer loop and an attitude inner loop is adopted, including the following steps: Step S1: Acquire images of the crossbeam, depth, and actual speed, attitude, and angular velocity of the UAV; set camera calibration parameters, crossbeam geometric parameters, and desired approach distance. Step S2: Project the image feature points onto a virtual image plane that preserves the heading motion, construct lateral, vertical, depth and relative heading features using the four endpoints of the double beam, and establish an interaction matrix; Step S3: Substitute the translational dynamics of the virtual system into the second-order image equation, construct the combined error based on the image error and its rate of change, use the interaction matrix and its derivative to obtain the generalized acceleration command, compensate for the coordinate system rotation and gravity terms, and directly obtain the force and heading angular acceleration commands. Step S4: Obtain the total thrust and desired roll and pitch angles from the force command, and obtain the desired angular velocity and azimuth angle by integrating the azimuth acceleration command; Step S5: Within the small roll and pitch angle working domain, construct linear sliding mode variables from attitude errors and their rate of change, calculate the three-axis torque through inertial coupling compensation, reference acceleration feedforward, linear feedback and symbolic feedback, and drive the UAV together with the total thrust command.

2. The method according to claim 1, characterized in that, The virtual feature points in step S2 are calculated using the following formula: in, These are the original pixel homogeneous coordinates. For the camera intrinsic parameter matrix, For the camera-to-body mounting rotation matrix, For the rotation matrix from the machine body to the world system, For the rotation matrix from the virtual system to the world system, The heading angle is given; the camera is rigidly connected to the aircraft body, and the translational offset between the camera's optical center and mass center is ignored.

3. The method according to claim 2, characterized in that, Select two endpoints, upper and lower, at the reference sections on the left and right sides of the beam, and assign subscripts... These represent the top left, bottom left, top right, and bottom right, respectively. definition: in, This is the actual distance between the top and bottom edges. To approximate the desired distance, let the coordinates of the beam center in the virtual system be... The distance between the left and right reference sections is The direction angle of the beam is Under the condition that the depth difference between the two sides is small, the interaction matrix is: The matrix is ​​used for , And the feature remains visible in the working domain, and makes , and Keep it bounded.

4. The method according to claim 3, characterized in that, Step S3: Obtain the fixed expected features ,definition , Calculate the control command using the following formula: in, , Obtained from continuous images; actual generalized velocity , For world-class speed, , , ; For mass and gravitational acceleration, subscript , This represents the first three and the fourth component.

5. The method according to claim 4, characterized in that, Step S4: Calculate the total thrust and attitude reference using the following formula: in, For force command components, For the desired attitude angles along the three axes, .Pick , Maintain small-angle branches with positive margin; force commands are second-order continuously differentiable, and attitude references can be obtained and uniformly bounded by first and second derivatives.

6. The method according to claim 5, characterized in that, Roll channel definition in step S5 , The control torque is: in, Moment of inertia of the principal axis of inertia , The sign function is used. The pitch and yaw channels employ the same sliding mode structure, and the control torque is calculated using the corresponding attitude error, inertial coupling term, and reference second derivative, respectively.

7. A visual servo control method for a UAV oriented towards the crossbeam of a power transmission tower, characterized in that, It includes a quadcopter drone, an airborne camera, a depth and flight status measurement unit, and an airborne processor; the processor receives measurement data, executes the method described in any one of claims 1 to 6, and outputs commands to the rotor drive unit.

8. A computer-readable storage medium, characterized in that, The device contains a computer program that, when executed by a processor, implements the method described in any one of claims 1 to 6.

Images