Unmanned aerial vehicle mobile platform landing control method based on three-axis holder visual servo

Abstract

The invention discloses an unmanned aerial vehicle mobile platform landing control method based on three-axis pan-tilt visual servo, and the method comprises the steps: dynamically adjusting the visual angle of a camera through a three-axis pan-tilt, enlarging the visual observation range, and guaranteeing that a target feature point is always in a visual field; in combination with camera pixel coordinate constraints and a preset performance function, converting the feature point pixel errors into conversion errors containing constraints so as to realize transient and steady-state performance constraints of the errors; aiming at the problem that the speed of a mobile platform is unknown, a speed estimator based on a filter is designed, the platform translation speed is obtained in real time and serves as a feedforward compensation input control loop, and the tracking precision is improved; by constructing a visual servo controller and analyzing the speed of the unmanned aerial vehicle and the angular speed of the holder, coordinated movement of the unmanned aerial vehicle and the holder is achieved, and finally dynamic tracking and precise landing of the mobile platform are completed. The problems that a traditional fixed camera is limited in view field and poor in dynamic environment adaptability are solved, and the landing precision, stability and safety in a complex scene are remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of visual servo control of unmanned aerial vehicles (UAVs), and in particular relates to a landing control method for a UAV mobile platform based on three-axis gimbal visual servoing. Background Art

[0002] Rotary-wing drones (UAVs) are particularly well-suited for use in complex environments due to their hovering capabilities, low takeoff restrictions, and operational flexibility. UAV landing platforms are often connected to high-payload, long-endurance mobile vehicles (such as unmanned vehicles or unmanned ships), enabling the drone to receive continuous refueling during its mission. This technological advancement opens up new possibilities for extending the operational range and improving the endurance of UAVs in a variety of applications.

[0003] However, controlling the landing of a drone on a mobile platform presents a number of challenges. First, the drone must obtain accurate, real-time location information of the landing platform. Second, to maintain relative stability relative to the mobile platform in a dynamic environment, the drone must adapt to varying flight speeds. Finally, the landing process must be gradual and stable to reduce the risk of accidents. Therefore, this invention aims to develop a stable and effective autonomous landing technology for drones on mobile platforms.

[0004] During the landing tracking phase, while global navigation satellite systems can provide information such as relative position, angle, and velocity, more precise positioning methods are often required during the landing phase to obtain the relative altitude between the platform and the desired landing point. This requires onboard precision guidance sensors such as lidar, ultra-wideband technology, and visual servoing systems. Visual servoing landing has become a popular solution due to its convenience and cost-effectiveness, attracting extensive research and application.

[0005] Existing drone landing control methods primarily rely on visual servoing using fixed cameras. However, the limited field of view of fixed cameras can easily cause image features to exceed the field of view constraints, thus affecting landing accuracy and stability. This limitation is particularly pronounced for large and small drones, making precise landing control difficult. Furthermore, fixed cameras have a narrow application range and cannot meet the diverse requirements of drone landing in complex environments. Therefore, a new control method is urgently needed that can improve drone landing accuracy and stability while maintaining field of view constraints.

[0006] This paper focuses on autonomous landing control for drones based on image visual servoing. By employing a three-axis gimbal camera, the motion of the gimbal creates a wider viewing angle, enhancing the drone's field of view during landing. A landing target velocity estimator is designed for mobile landing platforms with unknown speeds. The estimated value is introduced into the velocity control loop as a feedforward compensation to ensure tracking and optimize the controller design. Furthermore, a preset performance function is applied to control the error of visual features, ensuring that the error converges to a controllable range at a specified speed. Image feature error constraints are introduced to further improve the stability and accuracy of the system. Summary of the Invention

[0007] The primary purpose of this invention is to overcome the shortcomings and deficiencies of existing technologies by providing a landing control method for a UAV mobile platform based on three-axis gimbal visual servoing. This method introduces camera pixel field of view constraints to ensure that the image features of the landing target remain within the field of view. The three-axis gimbal is also used to expand the camera's visual range, enhancing the UAV's target tracking capabilities. To address the unknown speed of the mobile platform, a landing target velocity estimator is designed. Based on this estimated value, a visual servo controller is constructed to ensure that the servo tracking error converges to a small range near zero within the constraints of a preset performance function, thereby achieving high-precision and high-stability landing control.

[0008] In order to achieve the above object, the present invention adopts the following technical solutions:

[0009] In a first aspect, the present invention provides a landing control method for a UAV mobile platform based on three-axis gimbal visual servoing, the method comprising the following steps:

[0010] (1) Define the coordinate relationship, the speed and attitude physical quantities of the rotorcraft, and the coordinate transformation relationship of the three-axis gimbal;

[0011] (2) Design image features based on the preset camera imaging model and derive the image feature error dynamic equation;

[0012] (3) Construct a visual servo feature error system to define the speed difference between the mobile platform and the UAV;

[0013] (4) Introducing a camera pixel coordinate restriction range to constrain the feature point pixel error to be within a preset field of view;

[0014] (5) According to the field of view constraints, the performance function is designed and fused with the image feature error to form a constrained conversion error;

[0015] (6) Design a landing target velocity estimator to estimate the speed of the mobile platform in real time;

[0016] (7) Design a UAV image visual servo landing controller to analyze the UAV speed and gimbal angular velocity, and control the UAV to track the mobile platform and land.

[0017] As a preferred technical solution, the step (1) is specifically as follows:

[0018] First, define the coordinate system relationship: the global coordinate system pointing to the north and east The drone body coordinate system with its origin attached to the center of the drone pointing in the north-east direction Define the three-axis gimbal coordinate system The origin and initial state are the same as Coincidence; Camera Coordinate System Defined as the gimbal coordinate system coincide with each other, and the origin and initial state are Coincident; the north-east direction means that the X axis points straight ahead, the positive direction of the Y axis points to the right, and the positive direction of the Z axis is perpendicular to the XOY plane and downward;

[0019] Then, the landing platform The unknown moving speed is defined as Drones in The speed under is defined as The UAV attitude vector described by Euler angles is η=[φ,θ,ψ] T , where φ, θ, ψ represent the roll, pitch, and yaw angles of the aircraft, respectively. Based on the premise that the changes in each angle are small, the change rate of the attitude angle is defined as The rotation matrix from the drone body coordinate system to the global coordinate system is expressed as:

[0020]

[0021] Then, define the angle vector of the three-axis gimbal as η g =[φ g ,θ g ,ψ g ] T , where each element represents the roll, pitch, and yaw angles of the gimbal respectively; the three-axis gimbal is divided into an outer frame, a middle frame, and an inner frame. The outer frame is the yaw frame, the middle frame is the roll frame, and the inner frame is the pitch frame. The outer frame motor shaft coincides with the Z axis, that is, the yaw angle rotates around the Z axis. The rotation matrix around the Z axis is:

[0022]

[0023] The middle frame motor axis coincides with the X-axis, that is, the roll angle rotates around the X-axis. The rotation matrix around the X-axis is:

[0024]

[0025] The inner frame motor shaft coincides with the Y axis, that is, the pitch angle rotates around the Y axis. The rotation matrix around the Y axis is:

[0026]

[0027] As a preferred technical solution, the step (2) is specifically as follows:

[0028] Let P ci =[x i ,y i ,z i ] T Indicates The coordinate vector of the i-th feature point in a set of n feature points, 3≤i≤n, according to the camera imaging principle, P ci The corresponding camera imaging pixel coordinates satisfy the following relationship:

[0029]

[0030] Among them, A is the camera intrinsic parameter matrix, which is in the form of:

[0031]

[0032] Where λ is the focal length of the camera, which is described in pixels in the camera coordinate system, and (u0, v0) represents the pixel coordinates of the center point of the image.

[0033] For a fixed visual target in three-dimensional space, the velocity of the feature point has the following relationship with the spatial velocity of the camera:

[0034]

[0035] where v c =[v cx ,v cy ,v cz ] T and ω c =[ω cx ,ω cy ,ω cz ] T are the translation and rotation speeds of the camera respectively. Expanding the above formula, we have:

[0036]

[0037] Using pixel s i As image features, it is defined as:

[0038]

[0039] The image feature s iBy taking the derivative of time and combining the relationship between the velocity of the image feature points and the spatial velocity of the camera, we can obtain:

[0040]

[0041] Arranged into a matrix form, the derivative of the image feature with respect to time, that is, the dynamic information, is:

[0042]

[0043] in Represented in the camera coordinate system The camera velocity vector under , including translation velocity and rotation velocity, It is called the image Jacobian matrix and is specifically expressed as:

[0044]

[0045] Considering that there will be n feature points in visual servoing, define an overall image feature vector and the overall image Jacobian matrix The image feature dynamic information is expressed as follows:

[0046]

[0047] in For depth estimation or approximation There is an estimate or approximation of the Jacobian matrix of the entire image

[0048] As a preferred technical solution, the step (3) is specifically as follows:

[0049] The landing platform moves at a translation speed v in the camera coordinate system. m Movement, the translation speed of the feature point on it is expressed as v m , the roll, pitch and yaw angles of the mobile platform are all 0, that is, ω m =[0,0,0] T , then the speed of the landing platform in the camera coordinate system is expressed as The speed difference of the mobile landing platform (V c -V m ), reorganize the dynamic information of the fixed target image features to obtain the dynamic information of the image features in the case of the mobile landing platform:

[0050]

[0051] Therefore, the image feature error is defined as:

[0052] e s =ss d

[0053] in is the pixel coordinate error of the feature point, s d is the desired feature point pixel coordinate, In the design of the visual servo landing task, the expected pixel coordinates of each feature point are artificially preset constants, that is, Therefore, the derivative of the feature point pixel coordinate error with respect to time is expressed as:

[0054]

[0055] As a preferred technical solution, the step (4) is specifically as follows:

[0056] The upper and lower limits of the feature point pixel coordinates are defined as:

[0057] u min ≤u i ≤u max

[0058] v min ≤v i ≤v max

[0059] Among them, u min ,u max ,v min ,v max is the preset pixel coordinate range;

[0060] Definition ui,0 , e vi,0 , are the initial upper and lower bounds of the feature point pixel coordinate error, that is,

[0061] e u i ,0 =u d iu m i n

[0062]

[0063] e vi,0 =v di -v min

[0064]

[0065] When the image features at the initial moment are all within the preset field of view, the initial error will remain within the upper and lower limits of the field of view constraint.

[0066] As a preferred technical solution, the step (5) is specifically as follows:

[0067] Image feature error e si The following constraints need to be met:

[0068]

[0069] in, e ui (t), e vi (t), is the preset performance function of the feature point pixel coordinate error, defined as:

[0070] e ui (t)=( e ui,0 - e ui,∞ )exp(- κ ui t)+ e ui,∞

[0071]

[0072] e vi (t)=( e vi,0 - e vi,∞ )exp(- κ vi t)+ e vi,∞

[0073]

[0074] Among them, the constant e hi,∞ , are the steady-state upper and lower bounds of the feature point pixel coordinate error, κ hi , is the convergence rate of the feature point pixel coordinate error; set e k i,∞ <e k i,0 and So that the error can be effectively converged at a preset rate;

[0075] Then, define the normalized error as:

[0076]

[0077] Introduce conversion error, defined as:

[0078]

[0079]

[0080] Where ln(·) is the natural logarithm function, and the overall feature point pixel coordinate conversion error vector is:

[0081] ε(s,t)=[ε u1 ,ε v1 ,…,ε un ,ε vn ] T .

[0082] As a preferred technical solution, the step (6) is specifically as follows:

[0083] First, two first-order filters are designed to estimate e s and Derivative information with respect to time:

[0084] e f (0)=0

[0085] h(0)=0

[0086] Then, let the unknown variable The filter state variables are Its dynamic equation is expressed as:

[0087] S(0)=0

[0088] therefore, Through S, we get

[0089]

[0090] Therefore, the landing visual target speed V is defined as m Estimated value of

[0091]

[0092] in yes The pseudo-inverse matrix is ​​defined as:

[0093]

[0094] As a preferred technical solution, the step (7) is specifically as follows:

[0095] First, the controller is designed as follows:

[0096]

[0097] Where k>0 is a design control gain parameter;

[0098] Then, for the designed landing controller Decomposed into camera coordinate system Represents the camera translation speed v c and the rotational speed ω c ;

[0099] According to the camera translation speed v c To the drone in the global coordinate system The conversion relationship of the moving speed v under:

[0100] v=R·R out (ψ g )R mid (φ g )R in (θ g )·v c

[0101] According to the three-dimensional geometric relationship between the drone and the gimbal, the angular velocity ω of the camera is obtained. c The relationship between:

[0102]

[0103] Define the gimbal angular velocity as The angular velocity control input of the three-axis gimbal is obtained as follows:

[0104]

[0105] in:

[0106]

[0107] Therefore, when When , M will have singularity, so it is necessary to change φ in the implementation g Restricted to Inside.

[0108] In a second aspect, the present invention provides a drone equipped with a controller, which executes the drone mobile platform landing control method based on three-axis gimbal visual servoing.

[0109] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0110] 1. This invention uses a three-axis gimbal to dynamically adjust the camera's posture, integrating gimbal motion with visual servoing depth, significantly expanding the camera's effective observation range. During landing, the gimbal actively adjusts the viewing angle to ensure that target features remain within its field of view, avoiding the tracking loss problem often associated with traditional fixed cameras due to their limited field of view. This design enhances the drone's ability to track dynamic targets, maintaining a stable visual lock, particularly in complex environments, significantly improving landing safety.

[0111] 2. To address the challenge of unknown mobile platform velocity, this paper designs a filter-based velocity estimator that can acquire the platform's translational velocity information in real time without relying on external communication. By introducing the estimated velocity as a feedforward compensation into the control loop, the impact of platform motion on tracking accuracy is effectively offset, enabling the drone to rapidly respond to changes in target velocity. This mechanism significantly improves the robustness of dynamic tracking, enabling stable and high-precision trajectory tracking, especially in communication-limited or highly dynamic scenarios.

[0112] 3. This invention innovatively introduces pixel-level field-of-view constraints, combined with a preset performance function for feature point errors, to achieve dual control of the landing control system's transient and steady-state errors. By dynamically constraining the error's convergence boundaries and rates, the feature points are ensured to remain within a preset safety range, thus avoiding the risk of loss of control due to sudden error fluctuations while ensuring high-precision landing. This design enables drones to achieve a gradual and controllable landing process in complex environments, balancing efficiency and safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0113] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0114] Figure 1 This is a flow chart of the invention design of a method for controlling the landing of a UAV mobile platform based on three-axis gimbal visual servoing in an embodiment of the present invention;

[0115] Figure 2 is the coordinate relationship between the UAV and the mobile platform in the embodiment of the present invention;

[0116] Figure 3 The structure and coordinate system of the three-axis gimbal mounted on the drone in the embodiment of the present invention;

[0117] Figure 4 This is a structural block diagram of a method for controlling the landing of a UAV mobile platform based on three-axis gimbal visual servoing according to an embodiment of the present invention;

[0118] Figure 5 A simulation diagram of the motion trajectory of the image feature points designed in an embodiment of the present invention within the camera field of view;

[0119] Figure 6 is the image feature pixel value u designed in the embodiment of the present invention i The error variable e ui Simulation diagram of

[0120] Figure 7 is the image feature pixel value v designed in the embodiment of the present invention i The error variable e vi Simulation diagram of

[0121] Figure 8 A simulation diagram of the speed observation effect of the mobile platform landing target speedometer designed in an embodiment of the present invention;

[0122] Figure 9 Schematic diagram of the simulated motion trajectory of a drone tracking and landing on a mobile platform using the drone mobile platform landing control method based on three-axis gimbal visual servoing in an embodiment of the present invention. DETAILED DESCRIPTION

[0123] In order to enable those skilled in the art to better understand the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0124] References to "embodiments" in this application mean that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0125] like Figures 1-4 As shown, this embodiment provides a landing control method for a UAV mobile platform based on three-axis gimbal visual servoing, comprising the following steps:

[0126] S1. Define the coordinate relationship, define the physical quantities of the speed and attitude of the rotorcraft, and define the coordinate transformation relationship of the three-axis gimbal. Specifically:

[0127] First, define the coordinate system relationship in the control method design. The defined coordinate systems are all right-handed systems. The drone body coordinate system with its origin attached to the center of the drone pointing in the north-east direction Define the three-axis gimbal coordinate system The origin and initial state are the same as Coincident. Camera coordinate system Defined as the gimbal coordinate system coincide with each other, and the origin and initial state are The north-east direction is when the X-axis points straight ahead, the positive Y-axis points to the right, and the positive Z-axis is perpendicular to the XOY plane and points downward.

[0128] Then, the landing platform The unknown moving speed is defined as Drones in The speed under is defined as The UAV attitude vector described by Euler angles is η=[φ,θ,ψ] T , where φ, θ, ψ represent the roll, pitch, and yaw angles of the aircraft, respectively. Based on the premise that the changes in each angle are small, the change rate of the attitude angle is defined as The rotation matrix from the body coordinate system to the global coordinate system is expressed as:

[0129]

[0130] Then, define the angle vector of the three-axis gimbal as η g =[φ g ,θ g ,ψ g ] T , where each element represents the roll, pitch, and yaw angles of the gimbal. The three-axis gimbal is divided into an outer frame (yaw frame), a middle frame (roll frame), and an inner frame (pitch frame). The outer frame motor axis coincides with the Z axis, meaning the yaw angle rotates around the Z axis. The rotation matrix around the Z axis is:

[0131]

[0132] The middle frame motor axis coincides with the X-axis, that is, the roll angle rotates around the X-axis. The rotation matrix around the X-axis is:

[0133]

[0134] The inner frame motor shaft coincides with the Y axis, that is, the pitch angle rotates around the Y axis. The rotation matrix around the Y axis is:

[0135]

[0136] In a specific embodiment, the drone The initial position and velocity are: p0 = [-1.5, 0.1, -1.5]T and v0 = [0,0,0] T The initial angle of the three-axis gimbal is Mobile platforms The moving trajectory according to the time t is: m =1.0t+0.2sin(t),y m =1.2t+0.1cos(0.6t) and z m = 0.15sin(t), the characteristic points of the mobile platform are set as four fixed points of a square with a side length of 0.3 meters. The operation relationship and coordinate system relationship between the drone and the mobile platform implementing the method of the present invention are as follows: Figure 2 As shown, the structure and coordinate system of the three-axis gimbal are as follows Figure 3 As shown, the space trajectory of this embodiment is as follows Figure 9 shown.

[0137] S2. Combined with the camera imaging model, the image features are designed and the image feature error dynamic equation is obtained, which is:

[0138] Let P ci =[x i ,y i ,z i ] T Indicates The coordinate vector of the i-th feature point in a set of n (3≤i≤n) feature points. According to the camera imaging principle, P ci The corresponding camera imaging pixel coordinates satisfy the following relationship:

[0139]

[0140] Among them, A is the camera intrinsic parameter matrix, which is in the form of:

[0141]

[0142] Where λ is the focal length of the camera, which is described in pixels in the camera coordinate system. (u0, v0) represents the pixel coordinates of the center point of the image.

[0143] For a fixed visual target in three-dimensional space, the velocity of the feature point has the following relationship with the spatial velocity of the camera:

[0144]

[0145] where v c =[v cx ,v cy ,v cz ] T and ω c =[ω cx ,ω cy ,ωcz ] T are the translation and rotation speeds of the camera respectively. Expanding the above formula, we have:

[0146]

[0147] Using pixel s i As an image feature, it is defined as:

[0148]

[0149] The image feature s i By taking the derivative of time and combining the relationship between the velocity of the image feature points and the spatial velocity of the camera, we can obtain:

[0150]

[0151] Arranged into a matrix form, the derivative of the image feature with respect to time, that is, the dynamic information, is:

[0152]

[0153] in Represented in the camera coordinate system The camera velocity vector under , including translation velocity and rotation velocity, It is called the image Jacobian matrix and is specifically expressed as:

[0154]

[0155] Considering that there will be n feature points in visual servoing, define an overall image feature vector and the overall image Jacobian matrix Then the image feature dynamic information can be written as:

[0156]

[0157] in

[0158] Since the accurate depth information of the image feature points cannot be obtained, and the image Jacobian matrix L that depends on the image depth information cannot be accurately obtained, the controller design needs to obtain an approximate value of L by estimation.

[0159] In this embodiment, the intrinsic parameter matrix A of the camera is:

[0160]

[0161] This embodiment designs n=4 feature points, and the image Jacobian matrix L can be obtained by approximately calculating the input of a fixed depth. The input fixed depth information is:

[0162] z d =[0.5,0.5,0.5,0.5] T

[0163] S3. Construct a visual servo feature error system, specifically:

[0164] The landing platform moves at a translation speed v in the camera coordinate system. m Movement, the translation speed of the feature points on it can also be expressed as v m , the roll, pitch and yaw angles of the mobile platform are all 0, that is, ω m =[0,0,0] T , then the speed of the landing platform in the camera coordinate system can be expressed as The speed difference of the mobile landing platform (V c -V m ), and reorganize the dynamic information of the fixed target image features to obtain the dynamic information of the image features in the case of a moving landing platform:

[0165]

[0166] Therefore, the image feature error can be defined as:

[0167] e s =ss d

[0168] in is the pixel coordinate error of the feature point. d is the desired feature point pixel coordinate, In the design of the visual servo landing task, the expected pixel coordinates of each feature point are artificially preset constants, that is, Therefore, the derivative of the feature point pixel coordinate error with respect to time can be expressed as:

[0169]

[0170] In this embodiment, the desired feature point pixel coordinates are designed as follows:

[0171] s d1 =[-120,120] T

[0172] s d2 =[120,120] T

[0173] s d3 =[-120,-120] T

[0174] sd4 =[120,-120] T

[0175] The simulation diagram of the motion trajectory of the image feature points in the camera field of view is as follows: Figure 5 As shown in the figure, the hollow circle is the initial position of the feature point, and the solid circle is the desired terminal position of the feature point pixel coordinate. It can be seen from the figure that the four feature points move to the ideal terminal position and maintain it in a consistent and smooth trajectory without exceeding the camera's field of view.

[0176] S4. Introduce the camera pixel coordinate limit range and define the feature point pixel coordinate error field constraint, specifically:

[0177] During the landing process of the drone, in order to ensure that the image features are within the camera's field of view, the feature pixel coordinates can be constrained. The upper and lower limits of the feature point pixel coordinates are defined as:

[0178] u min ≤u i ≤u max

[0179] v min ≤v i ≤v max

[0180] Among them, u min ,u max ,v min ,v max It is the pixel coordinate range preset by the designer, generally set to be less than or equal to the camera coordinate resolution.

[0181] Definition ui,0 , e vi,0 , are the initial upper and lower bounds of the feature point pixel coordinate error, that is,

[0182] e ui,0 =u di -u min

[0183]

[0184] e vi,0 =v di -v min

[0185]

[0186] When the image features at the initial moment are all within the preset field of view, the initial error will remain within the upper and lower limits of the field of view constraint.

[0187] In this embodiment, the pixel coordinate range is set as: min =-319,u max =319,v min =-239,v max =239.

[0188] S5. Based on the field of view constraint, a performance function is designed and fused with the image feature error to form a constrained conversion error. Specifically:

[0189] In order to ensure the predetermined performance of the image feature error in transient and steady states, the image feature error e si The constraints need to be met, and the following inequality is used to describe the error constraint range:

[0190]

[0191] Among them, e ui (t), e vi (t), is the preset performance function of the feature point pixel coordinate error, defined as:

[0192] e ui (t)=(e ui,0 -e ui,∞ )exp(-k ui t)+e ui,∞

[0193]

[0194] e vi (t)=(e vi,0 -e vi,∞ )exp(-k vi t)+e vi,∞

[0195]

[0196] Among them, the constant e hi,∞ , is the steady-state upper and lower bounds of the feature point pixel coordinate error, k hi , is the convergence rate of the feature point pixel coordinate error. h i,∞ , You can set e k i,∞ <e k i,0 and So that the error can converge effectively at a preset rate.

[0197] Then, the normalized error is defined as

[0198]

[0199] Introduce conversion error, defined as:

[0200]

[0201] Where ln(·) is the natural logarithm function. The overall feature point pixel coordinate conversion error vector is:

[0202] ε(s,t)=[ε u1 ,ε v1 ,…,ε un ,ε vn ] T

[0203] In this embodiment, the specific form of the performance function is:

[0204] e ui (t)=( e ui,0 -30)exp(-0.2t)+30

[0205]

[0206] e vi (t)=( e vi,0 -30)exp(-0.2t)+30

[0207]

[0208] Where i = 1,…,4,

[0209] Figure 6 and Figure 7 Shows the u of four feature points i and v i As for the pixel error, it can be seen that within the set time range, the image feature error can converge to near zero, and the error curve does not cross the boundary enclosed by the preset performance function of the feature point pixel coordinate error, which shows that the above-mentioned viewing angle constraint can always be met.

[0210] S6. Design a landing target speed estimator, specifically:

[0211] First, two first-order filters are designed to estimate e s and Derivative information with respect to time:

[0212] e f (0)=0

[0213] h(0)=0

[0214] Where μ>0 is the time constant of the filter. Then, let the unknown variable The filter state variables are Its dynamic equation is expressed as:

[0215] S(0)=0

[0216] therefore, It can be obtained through S, that is,

[0217]

[0218] Therefore, the landing visual target speed V can be defined as m Estimated value of

[0219]

[0220] in yes The pseudo-inverse matrix is ​​defined as:

[0221]

[0222] In this embodiment, the filter time constant μ is set to 0.5.

[0223] Figure 8 The designed landing target speed estimator is demonstrated to estimate the unknown speed. It can be seen that the preset speed of the landing target can be tracked within 5 seconds, proving its feasibility.

[0224] S7. Design a drone image visual servo landing controller to analyze the drone speed and gimbal angular velocity. Specifically:

[0225] First, the controller is designed as follows:

[0226]

[0227] Wherein, k>0 is a design control gain parameter. In this embodiment, the controller gain parameter k=10 is set.

[0228] Then, for the designed landing controller Decomposed into camera coordinate system Represents the camera translation speed v c and the rotational speed ω c .

[0229] According to the camera translation speed v cTo the drone in the global coordinate system The conversion relationship of the moving speed v under:

[0230] v=R·R out (ψ g )R mid (φ g )R in (θ g )·v c

[0231] According to the three-dimensional geometric relationship between the drone and the gimbal, the angular velocity ω of the camera can be obtained. c The relationship between:

[0232]

[0233] According to the existing measurable angle information, the gimbal angular velocity is defined as The angular velocity control input of the three-axis gimbal is obtained as follows:

[0234]

[0235] in

[0236]

[0237] From the above formula, we can see that when When , M will have singularity, so it is necessary to change φ in the implementation g Restricted to Inside.

[0238] This embodiment's image visual servo landing algorithm enables the servo control systems of the drone and three-axis gimbal camera to determine the position of the mobile platform solely through visual information, tracking the characteristic points of the landing area while gradually adjusting to a preset landing altitude. Even if the landing platform's direction and speed are unknown, the closed-loop system maintains the visual feature error within a preset range, allowing the drone to follow the landing platform horizontally and gradually approach the target altitude vertically.

[0239] This method converts the camera's field of view constraints into upper and lower bounds on the image feature error and applies a preset performance function for control. Landing position control is converted to controlling the positions of camera feature points, transforming the actual control problem into a numerical one. This method not only ensures that the constraints are consistently met but also allows for limited adjustment of the landing speed and ensures the safe operation of the landing program.

[0240] In addition, the UAV mobile platform landing control method based on three-axis gimbal visual servo proposed in the present invention can obtain accurate speed information of the landing platform without relying on information exchange methods. Instead, through the designed landing target speed estimator, the UAV itself observes the speed information of the mobile platform and compensates for the ideal reference speed, thereby ensuring that the present invention can be applied to a variety of practical scenarios.

[0241] It should be noted that, for the sake of convenience, the aforementioned method embodiments are all expressed as a series of action combinations, but those skilled in the art should know that the present invention is not limited to the described order of actions, because according to the present invention, certain steps can be performed in other orders or simultaneously.

[0242] This embodiment also provides a drone equipped with a controller, which is constructed using the above-mentioned drone mobile platform landing control method based on three-axis gimbal visual servoing, specifically:

[0243] (1) Define the coordinate relationship, define the physical quantities of the speed and attitude of the rotorcraft, and define the coordinate transformation relationship of the three-axis gimbal;

[0244] (2) Combined with the camera imaging model, the image features are designed and the image feature error dynamic equation is obtained;

[0245] (3) Construct a visual servo feature error system;

[0246] (4) Introducing the camera pixel coordinate limit range and defining the feature point pixel coordinate error field constraint;

[0247] (5) According to the field of view constraints, the performance function is designed and fused with the image feature error to form a constrained conversion error;

[0248] (6) Design a landing target velocity estimator;

[0249] (7) Design a UAV image visual servo landing controller to analyze the UAV speed and gimbal angular velocity;

[0250] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0251] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be considered as equivalent replacement methods and are included in the scope of protection of the present invention.

Claims

1. A landing control method for a UAV mobile platform based on three-axis gimbal visual servoing, characterized in that: The method comprises the following steps: (1) Define the coordinate relationship, the speed and attitude physical quantities of the rotorcraft, and the coordinate transformation relationship of the three-axis gimbal; (2) Design image features based on the preset camera imaging model and derive the image feature error dynamic equation; (3) Construct a visual servo feature error system to define the speed difference between the mobile platform and the UAV; (4) Introducing a camera pixel coordinate restriction range to constrain the feature point pixel error to be within a preset field of view; (5) According to the field of view constraints, the performance function is designed and fused with the image feature error to form a constrained conversion error; (6) Design a landing target velocity estimator to estimate the speed of the mobile platform in real time; (7) Design a UAV image visual servo landing controller to analyze the UAV speed and gimbal angular velocity, and control the UAV to track the mobile platform and land.

2. The method for controlling the landing of a UAV mobile platform based on three-axis gimbal visual servo according to claim 1, characterized in that: The step (1) is specifically as follows: First, define the coordinate system relationship: the global coordinate system pointing to the north and east The drone body coordinate system with its origin attached to the center of the drone pointing in the north-east direction Define the three-axis gimbal coordinate system The origin and initial state are the same as Coincidence; Camera Coordinate System Defined as the gimbal coordinate system coincide with each other, and the origin and initial state are Coincident; the north-east direction means that the X axis points straight ahead, the positive direction of the Y axis points to the right, and the positive direction of the Z axis is perpendicular to the XOY plane and downward; Then, the landing platform The unknown moving speed is defined as Drones in The speed under is defined as The UAV attitude vector described by Euler angles is η=[φ,θ,ψ] T , where φ, θ, ψ represent the roll, pitch, and yaw angles of the aircraft, respectively. Based on the premise that the changes in each angle are small, the change rate of the attitude angle is defined as The rotation matrix from the drone body coordinate system to the global coordinate system is expressed as: Then, define the angle vector of the three-axis gimbal as η g =[φ g ,θ g ,ψ g ] T , where each element represents the roll, pitch, and yaw angles of the gimbal respectively; the three-axis gimbal is divided into an outer frame, a middle frame, and an inner frame. The outer frame is the yaw frame, the middle frame is the roll frame, and the inner frame is the pitch frame. The outer frame motor shaft coincides with the Z axis, that is, the yaw angle rotates around the Z axis. The rotation matrix around the Z axis is: The middle frame motor axis coincides with the X-axis, that is, the roll angle rotates around the X-axis. The rotation matrix around the X-axis is: The inner frame motor shaft coincides with the Y axis, that is, the pitch angle rotates around the Y axis. The rotation matrix around the Y axis is:

3. The method for controlling the landing of a UAV mobile platform based on three-axis gimbal visual servo according to claim 2, characterized in that: The step (2) is specifically as follows: Let P ci =[x i ,y i ,z i ] T Indicates The coordinate vector of the i-th feature point in a set of n feature points, 3≤i≤n, according to the camera imaging principle, P ci The corresponding camera imaging pixel coordinates satisfy the following relationship: Among them, A is the camera intrinsic parameter matrix, which is in the form of: Where λ is the focal length of the camera, which is described in pixels in the camera coordinate system, and (u0, v0) represents the pixel coordinates of the center point of the image. For a fixed visual target in three-dimensional space, the velocity of the feature point has the following relationship with the spatial velocity of the camera: where v c =[v cx ,v cy ,v cz ] T and ω c =[ω cx ,ω cy ,ω cz ] T are the translation and rotation speeds of the camera respectively. Expanding the above formula, we have: Using pixel s i As image features, it is defined as: The image feature s i By taking the derivative of time and combining the relationship between the velocity of the image feature points and the spatial velocity of the camera, we can obtain: Arranged into a matrix form, the derivative of the image feature with respect to time, that is, the dynamic information, is: in Represented in the camera coordinate system The camera velocity vector under , including translation velocity and rotation velocity, It is called the image Jacobian matrix and is specifically expressed as: Considering that there will be n feature points in visual servoing, define an overall image feature vector and the overall image Jacobian matrix The image feature dynamic information is expressed as follows: in For depth estimation or approximation There is an estimate or approximation of the Jacobian matrix of the entire image 4. The method for controlling the landing of a UAV mobile platform based on three-axis gimbal visual servo according to claim 3, characterized in that: The step (3) is specifically as follows: The landing platform moves at a translation speed v in the camera coordinate system. m Movement, the translation speed of the feature point on it is expressed as v m , the roll, pitch and yaw angles of the mobile platform are all 0, that is, ω m =[0,0,0] T , then the speed of the landing platform in the camera coordinate system is expressed as The speed difference of the mobile landing platform (V c -V m ), reorganize the dynamic information of the fixed target image features to obtain the dynamic information of the image features in the case of the mobile landing platform: Therefore, the image feature error is defined as: e s =s-s d in is the pixel coordinate error of the feature point, s d is the desired feature point pixel coordinate, In the design of the visual servo landing task, the expected pixel coordinates of each feature point are artificially preset constants, that is, Therefore, the derivative of the feature point pixel coordinate error with respect to time is expressed as:

5. The method for controlling the landing of a UAV mobile platform based on three-axis gimbal visual servo according to claim 4, characterized in that: The step (4) is specifically as follows: The upper and lower limits of the feature point pixel coordinates are defined as: in min in i in max in min ≤in i ≤in max Among them, u min ,u max ,v min ,v max is the preset pixel coordinate range; Definition ui,0 , e vi,0 , are the initial upper and lower bounds of the feature point pixel coordinate error, that is, yes u and ,0 =in d and in m and n yes v and ,0 =v d iv m and n When the image features at the initial moment are all within the preset field of view, the initial error will remain within the upper and lower limits of the field of view constraint.

6. The method for controlling the landing of a UAV mobile platform based on three-axis gimbal visual servo according to claim 5, characterized in that: The step (5) is specifically as follows: Image feature error e si The following constraints need to be met: Among them, e ui (t), e vi (t), is the preset performance function of the feature point pixel coordinate error, defined as: yes ui (t)=(e ui,0 -e ui,∞ )exp(-k ui t)+e ui,∞ yes vi (t)=(e vi,0 -e vi,∞ )exp(-k vi t)+e vi,∞ Among them, the constant are the steady-state upper and lower bounds of the feature point pixel coordinate error, is the convergence rate of the feature point pixel coordinate error; set e k i,∞ <e k i,0 and So that the error can be effectively converged at a preset rate; Then, define the normalized error as: Introduce conversion error, defined as: Where ln(·) is the natural logarithm function, and the overall feature point pixel coordinate conversion error vector is: ε(s,t)=[ε u1 ,he v1 ,…,eh un ,he vn ] T 。 7. The method for controlling the landing of a UAV mobile platform based on three-axis gimbal visual servo according to claim 6, characterized in that: The step (6) is specifically as follows: First, two first-order filters are designed to estimate e s and Derivative information with respect to time: Then, let the unknown variable The filter state variables are Its dynamic equation is expressed as: therefore, Through S, we get Therefore, the landing visual target speed V is defined as m Estimated value of : in yes The pseudo-inverse matrix is ​​defined as:

8. The method for controlling the landing of a UAV mobile platform based on three-axis gimbal visual servoing according to claim 7, characterized in that: The specific steps in step (7) are: First, the controller is designed as follows: Where k>0 is a design control gain parameter; Then, for the designed landing controller Decomposed into camera coordinate system Represents the camera translation speed v c and the rotational speed ω c ; According to the camera translation speed v c To the drone in the global coordinate system The conversion relationship of the moving speed v under: v=R·R out (ψ g )R mid (f g )R in (i g )·v c According to the three-dimensional geometric relationship between the drone and the gimbal, the angular velocity ω of the camera is obtained. c The relationship between: Define the gimbal angular velocity as The angular velocity control input of the three-axis gimbal is obtained as follows: in: Therefore, when When , M will have singularity, so it is necessary to change φ in the implementation g Restricted to Inside.

9. A drone equipped with a controller, characterized in that: The controller executes the landing control method of the UAV mobile platform based on three-axis gimbal visual servoing according to any one of claims 1 to 8.

Images