A fixed-wing unmanned aerial vehicle target tracking method based on fast visual servo prediction control

Abstract

The application discloses a fixed-wing unmanned aerial vehicle target tracking method based on fast visual servo prediction control, which comprises the following steps: S1, the unmanned aerial vehicle captures target images through a gimbal camera; S2, feature point coordinates on a current image are obtained based on a target detection algorithm; S3, the influence of unmanned aerial vehicle posture and gimbal posture change on feature points is solved through posture compensation, virtual feature points are obtained, and an image kinematics model is constructed in combination with an IBVS method; and S4, an MPC optimization problem is constructed, the dynamic constraint of the fixed-wing unmanned aerial vehicle and the perception constraint of the camera are considered, the control law of the system is obtained by online solution, and the unmanned aerial vehicle is kept near the image center while continuously tracking the target. The application has the advantages of simple principle, wide application range, fast processing speed and improved target tracking effect.

Description

Technical Field

[0001] This invention mainly relates to the field of unmanned aerial vehicle (UAV) target tracking technology, specifically a fixed-wing UAV target tracking method based on fast visual servo predictive control. Background Technology

[0002] Target tracking is not only a fundamental aspect of UAVs' Earth observation missions but also the basis for subsequent intent analysis and situational awareness generation. It can be applied to border control, urban security, and aerial photography. Compared to fixed cameras, UAVs can significantly expand the field of view, thereby enhancing the ability to continuously monitor targets. Furthermore, considering that fixed-wing UAVs often have longer endurance and higher flight speeds than rotary-wing UAVs, researching target tracking for fixed-wing UAVs is highly significant.

[0003] Visual servoing methods can be used to achieve target tracking, leveraging visual sensors. Based on differences in feedback information, these methods are mainly divided into two categories: PBVS and IBVS. The PBVS method requires calculating the relative position of the target and the UAV after obtaining the target's feature information. Based on this method, some researchers have proposed technical solutions for tracking and estimating the motion of ground vehicles, providing real-time estimates of target position, velocity, and heading. Other researchers have proposed guidance laws for target tracking and UAV maneuvers under continuous excitation conditions, using nonlinear adaptive observers to estimate the target's state, parameters, and position. Furthermore, researchers have conducted flight experiments tracking moving ground targets using fixed-wing UAVs. However, due to significant errors in camera calibration, positioning deviations occur when using the PBVS method, affecting the UAV's tracking accuracy.

[0004] In contrast, the IBVS method does not require UAV localization; instead, it designs the controller directly on the image plane, thus exhibiting robustness to sensor model and calibration errors. However, the IBVS method only focuses on the motion of image feature information and cannot effectively handle constraints. For example, it cannot consider the dynamic constraints of fixed-wing UAVs and the visualization constraints of vision sensors, which may lead to the inability to obtain the optimal control solution, or even the problem of the target exceeding the camera's field of view during UAV tracking. Therefore, MPC is introduced to address constrained optimization problems, thereby improving the performance of the IBVS method. Currently, the application of MPC-based IBVS in UAVs is still limited, mainly due to the significant challenge of simultaneously considering the complex dynamic models of UAVs, the nonlinear IBVS system, and real-time requirements.

[0005] In summary, current target tracking solutions for fixed-wing UAVs have the following main shortcomings:

[0006] (1) After obtaining images of the target through airborne vision sensors, some studies use the PBVS method for control. That is, the camera intrinsic parameters are first calibrated, and then the relative position of the target with respect to the camera is calculated by combining image feature points. Then, the tracking control law is designed in Cartesian space. However, the camera calibration process often introduces large errors, resulting in large deviations in positioning accuracy, which in turn affects the tracking accuracy.

[0007] (2) When using the classic IBVS method to track a target by a UAV, it only focuses on the convergence from the current feature point to the desired feature point, but cannot handle UAV dynamic constraints and camera visualization constraints. This leads to the target possibly exceeding the camera's field of view during tracking, resulting in tracking failure.

[0008] (3) In traditional MPC optimization problems, if there are nonlinear hard constraints, the solution process will be very time-consuming or even impossible to obtain a feasible solution. Fixed-wing UAVs are in a state of high-speed flight and have high requirements for real-time performance. If they cannot be solved quickly online, the tracking effect will be poor or even the target will be lost.

[0009] (4) The current UAV tracking control law based on the IBVS method considers the convergence from the current feature point to the desired feature point. However, when the UAV is far away from the target, the convergence rate will not be significantly improved, which makes it difficult for the UAV to observe the detailed information of the target due to its inability to quickly approach the target, and may even affect the target recognition rate and lead to tracking failure. Summary of the Invention

[0010] The technical problem to be solved by this invention is: in view of the technical problems existing in the prior art, this invention provides a target tracking method for fixed-wing UAVs based on fast visual servo predictive control, which is simple in principle, has a wide range of applications, fast processing speed, and can improve the target tracking effect.

[0011] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0012] A target tracking method for a fixed-wing UAV based on fast visual servo predictive control, comprising:

[0013] Step S1: The drone captures target images using its gimbal camera;

[0014] Step S2: Obtain the coordinates of feature points on the current image based on the object detection algorithm;

[0015] Step S3: The impact of changes in UAV attitude and gimbal attitude on feature points is addressed through attitude compensation, and virtual feature points are obtained. Then, the image kinematic model is constructed by combining the IBVS method.

[0016] Step S4: Construct the MPC optimization problem, considering the dynamic constraints of the fixed-wing UAV and the perception constraints of the camera, and obtain the control law of the system through online solution, so that the UAV can keep the target near the center of the image while continuously tracking it.

[0017] As a further improvement to the method of the present invention, step S3 includes the following process:

[0018] Step S301: Describe the construction of the image kinematics model;

[0019] This represents the body coordinate system, where the origin is o. b Located at the center of gravity of the fixed-wing UAV, x b The axis is along the direction of the drone's nose, y b Axis perpendicular to x b The axis points to the left side of the fuselage, z b The axis is perpendicular to the fuselage and points upwards; This represents the image coordinate system, with the origin o. i Located at the center of the image; x i axis and y i The axes lie within the image and are parallel to the image's width and height, respectively. This represents the camera coordinate system, where the origin is o. c Located at the camera's optical center, z c The axis is along the optical axis and perpendicular to the image, x c axis and y c The axes are parallel to x. i axis and y i axis;

[0020] A two-degree-of-freedom gimbal camera has a yaw angle θ p and pitch angle θ t ; where θ p Around x p The axis rotates and x p The axis is perpendicular to the horizontal plane S p θ t Around x t The axis rotates and x t axis and x c Axle load combination located on horizontal plane S p Inside; because the gimbal can yaw in all directions, but the pitch angle is limited, therefore θ p ∈[-π,π],θ t ∈[γ1,γ2]; where And γ2>0, and note that when θ t =0 when z c The axis is located in the horizontal plane S p Inside.

[0021] Step S302: Construct the model;

[0022] A fixed-wing unmanned aerial vehicle (UAV) flies at a constant altitude and a constant speed, denoted as altitude (H) and speed (V). t Let the yaw, pitch, and roll angles of the UAV be represented by ψ, θ, and φ, respectively. Then, based on the unicycle overall system model, it can be described as follows:

[0023]

[0024] Step S303: Analyze the image kinematic model;

[0025] Define the goal in and The coordinates in P(x) are respectively c ,y c ,z c ) and s(u i ,v i ),based on and Based on the relationship and the principle of triangle similarity, we can conclude that:

[0026]

[0027] Where f represents the camera focal length;

[0028] Define the camera in The linear velocity and angular velocity in V are respectively c =[T x ,T y ,T z ] T and Ω c =[ω x ,ω y ,ω z ] T Then we have:

[0029]

[0030] By combining the formula, the relationship between the feature point change rate and the camera speed can be obtained:

[0031]

[0032] The image Jacobian matrix is:

[0033]

[0034] As a further improvement to the method of the present invention, step S4 includes the following process:

[0035] For the reference state, the corresponding system attitude is represented as follows:

[0036]

[0037] in It is a given constant;

[0038] Define the goal in The coordinates in the reference state are (u1, v1), and the coordinates in the reference state are (u2, v2). The constructed IBVS model is as follows:

[0039]

[0040] At the same time, the control inputs for the gimbal are given:

[0041]

[0042] Define s2 = [u2, v2] T Based on formula (7), the discrete IBVS model can be expressed as:

[0043]

[0044] The formula can be expressed as:

[0045] s2(k+1)=f s (s2(k),u ψ (k))

[0046] Since the yaw rate of a fixed-wing UAV is limited, the following constraints must be satisfied:

[0047] u ψ ∈U set U set =[-u max ,u max ]

[0048] To ensure u ψ The change is smooth, and the following acceleration constraints are given:

[0049] Δu∈a set ,a set =[-a max ,a max ]

[0050] in:

[0051] Δu(k+i)=u ψ (k+i)-u ψ (k+i-1).

[0052] As a further improvement to the method of the present invention: processing the camera's perceptual constraints includes:

[0053] First, transform the image plane containing s2 to a horizontal plane; define the coordinates of the feature point on this horizontal plane as s. ⊥ (u ⊥ ,v ⊥ Considering that the angle corresponding to the virtual image plane where s2 is located in the formula is θ t =-α, then s2 and s ⊥ The following relationship exists between them:

[0054]

[0055] in Solve for u using the above formula. ⊥ and v ⊥ The value;

[0056] If the flight altitude H is constant, then the camera's maximum sensing distance can be equivalent to its maximum horizontal distance R. M A target is within the camera's clear perception range if and only if the following conditions are met:

[0057]

[0058] In the defined reference state, α satisfies the following condition:

[0059]

[0060] For u ψ The following relations are also satisfied under the constraints:

[0061]

[0062] definition Then there is The perceptual constraint at time k is expressed as:

[0063] g1(k)≤0

[0064] in:

[0065]

[0066] The deviation of the feature points is defined as:

[0067] Δs(k+i|k)=s d -s2(k+i|k)

[0068] Where s d =[0,0] T Let s2 be the expected feature point corresponding to s2, s2(k|k)=s2(k) be the state observation at time k, and Δu(k|k=u ψ (k|k)-u ψIf (k-1|k-1), then the cost function of the optimization problem is defined as:

[0069]

[0070] in:

[0071] J r (k+i|k)=Δs(k+i|k) T ·Q s ·Δs(k+i|k)+Δu(k+i-1|k)·Q u ·Δu(k+i-1|k)

[0072] The control sequence is:

[0073] U r ={u ψ (k|k),u ψ (k+1|k),…,u ψ (k+N p -1|k)},

[0074] For the prediction domain, Q s =diag{q1,q2} is positive definite and

[0075] J1 = 0 if and only if for all i ∈ {1,…,N} p}, Δs(k+i|k)=0 and Δu(k+i-1|k)=0 hold true, which means that for a stationary target, the fixed-wing UAV will hover around it.

[0076] As a further improvement to the method of the present invention: the contraction constraint satisfies:

[0077] h1(k+N p +1|k)≤0,

[0078] in:

[0079] h1(k+N [ +1|k)=J r (k+N p +1|k)-λ·J r (k+1|k),

[0080] And λ∈(0,1); at the same time, the control sequence is redefined as:

[0081] U m ={U r ,u ψ (k+N p |k)}.

[0082] As a further improvement to the method of the present invention: the MPC optimization problem is constructed as follows:

[0083]

[0084] st

[0085] s2(k+i|k)=f s (s2(k+i-1|k),u ψ (k+i-1|k))

[0086] u ψ (k+i-1|k)∈U set

[0087] Δu(k+i-1|k)∈a set

[0088] g1(k+i|k)≤0

[0089] h1(k+N p +1|k)≤0

[0090] where i∈{1,2,…,N} p +1}.

[0091] As a further improvement to the method of the present invention: hard constraints are added to the cost function to transform it into soft constraints, the process of which includes:

[0092] Add the constraint g1(k+i|k)≤0 to J r From:

[0093] J m1 (k+i|k)=J r (k+i|k)+β1·max{g1(k+i|k),0}

[0094] in It is a constant;

[0095] Based on this, the corresponding contraction constraint is transformed into:

[0096] h2(k+N p +1|k)≤0

[0097] in:

[0098] h2(k+N p +1|k)=J m1 (k+N p +1|k)-λ·J m1 (k+1|k)

[0099] g1 is redefined as follows:

[0100]

[0101] Where ΔR>0; then formula (28) can be reformulated as:

[0102] J m1 (k+i|k)=J r (k+i|k)+β1·max{g2(k+i|k),0};

[0103] Secondly, constrain h1(k+N) p After adding +1|k), it is represented as:

[0104] J m2 (k+i|k)=J m1 (k+i|k)+β2·max{h2(k+N p +1|k),0}

[0105] in It is a constant;

[0106] Based on this, the cost function can be expressed as:

[0107]

[0108] The MPC optimization problem is refactored as follows:

[0109]

[0110] st

[0111] s2(k+i|k)=f s (s2(k+i-1|k),u ψ (k+i-1|k))

[0112] u ψ (k+i-1|k)∈U set

[0113] Δu(k+i-1|k)∈a set

[0114] where i∈{1,2,…,N} p +1}.

[0115] As a further improvement to the method of the present invention: a switching-based control strategy is adopted; when the relative distance exceeds R... M In order to accelerate the convergence of g1(k) to 0 when the perception constraint is not satisfied, a cost function is defined:

[0116]

[0117] in:

[0118] J n2 (k+i|k)=J n1 (k+i|k)+β2·max{h2(k+N p +1|k),0}

[0119] J n1 (k+i|k)=Δs ⊥ (k+i|k) T ·P s ·Δs ⊥ (k+i|k)+Δu(k+i-1|k)·Q u ·Δu(k+i-1|k)

[0120] P s =diag{p1,p2} is positive definite and:

[0121]

[0122] Where h2(k+N) p +1|k) can be represented as:

[0123] J n1 (k+N p +1|k)-λ·J n1 (k+1|k),g1(k)>0

[0124] Introducing a step function:

[0125]

[0126] This is used to construct the following switching-based MPC optimization problem:

[0127]

[0128] st

[0129] s2(k+i|k)=f s (s2(k+i-1|k),u ψ (k+i-1|k))

[0130] u ψ (k+i-1|k)∈U set

[0131] Δu(k+i-1|k)∈a set

[0132] where i∈{1,2,…,N} p +1}.

[0133] Compared with the prior art, the advantages of the present invention are as follows:

[0134] 1. This invention discloses a target tracking method for fixed-wing UAVs based on fast visual servoing predictive control. The method is simple in principle, widely applicable, fast in processing, and improves target tracking performance. Specifically, it addresses the integrated system of a fixed-wing UAV and a gimbal camera, proposing an image visual servoing (IBVS) method based on model predictive control (MPC) for target tracking. This invention considers the influence of camera perception capabilities, ensuring the target is centered in the image while remaining within the camera's clear perception range, thus generating more image feature points for accurate detection. In other words, this invention uses the IBVS method for model construction. After obtaining the target image, image detection algorithms are used to obtain the target's feature point information, which is then combined with the UAV's own attitude information to directly design the control law on the image plane. This avoids introducing errors in the position calculation stage, contributing to improved target tracking accuracy.

[0135] 2. This invention provides a target tracking method for a fixed-wing UAV based on fast visual servo predictive control, employing a fast MPC strategy to ensure real-time performance in practical applications. The invention first transforms the hard constraints in MPC into soft constraints, enabling feasible solutions to be obtained during the iteration process. Then, an initial feasible solution is provided to achieve a hot start in the optimization process, thereby accelerating the acquisition of the optimal solution to the MPC problem. In other words, this invention introduces MPC to improve the performance of the IBVS controller. After constructing the kinematic model of the image using IBVS, MPC is introduced to consider the dynamic constraints of the fixed-wing UAV and the visualization constraints of the camera, ensuring that the solution to the optimization problem satisfies the constraints. This helps ensure that the UAV can keep the target within the camera's field of view for an extended period, thus achieving continuous tracking.

[0136] 3. The fixed-wing UAV target tracking method based on fast visual servo predictive control of the present invention employs a switching-based control strategy to address the problem of quickly returning the target after it has exceeded the camera's clear perception range. This invention aims to avoid target loss due to the target being outside the camera's clear perception range for an extended period, or by exceeding it by too much, while also helping to reduce the online computational load on the processor.

[0137] 4. The target tracking method for fixed-wing UAVs based on fast visual servo predictive control of the present invention can solve the problem of continuous tracking of ground moving targets by fixed-wing UAVs. The present invention controls the overall system of the UAV and gimbal camera, enabling the UAV to keep the target near the center of the image while tracking it, and preventing the target from becoming undetectable due to its distance from the UAV.

[0138] 5. The target tracking method for fixed-wing UAVs based on fast visual servo predictive control of the present invention transforms nonlinear constraints into soft constraints by using a max function and adding them to the cost function. This ensures that the nonlinear constraints do not need to be strictly satisfied, but still need to be satisfied first due to their large weight in the cost function. In addition, the advantage of using the max function is that when the nonlinear constraint is satisfied, it is not necessary to calculate the gradient of the constraint, further reducing the amount of online computation. Attached Figure Description

[0139] Figure 1 This is a schematic diagram of the control principle of the method of the present invention.

[0140] Figure 2 This is a schematic diagram illustrating the principle of UAV target tracking in a specific application example of the present invention.

[0141] Figure 3 This is a schematic diagram of the gimbal model in a specific application example of the present invention.

[0142] Figure 4 This is a schematic diagram of the drone's perception range in a specific application example of the present invention.

[0143] Figure 5 This is an algorithm flowchart of the present invention in a specific application example.

[0144] Figure 6 This is a flowchart illustrating the method of the present invention. Detailed Implementation

[0145] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0146] This invention presents a target tracking method for fixed-wing UAVs based on fast visual servo predictive control. It introduces MPC (Multi-Performance Control) into the existing IBVS (Integrated Visual Servo Predictive Control) model to solve constrained target tracking problems. An MPC strategy is proposed for rapid online solution. Finally, a switching controller is designed to handle situations where the target is outside the camera's clear perception range, quickly bringing it back within that range and enabling continuous tracking by the UAV. This invention is the first to combine fast MPC with the IBVS method to solve the problem of continuous target tracking by a fixed-wing UAV.

[0147] like Figure 1 and Figure 6 As shown, the present invention provides a target tracking method for a fixed-wing UAV based on fast visual servo predictive control, the process of which includes:

[0148] Step S1: The drone first captures the target image using its gimbal camera;

[0149] Step S2: Obtain the coordinates of feature points on the current image based on the object detection algorithm;

[0150] Step S3: The impact of changes in UAV attitude and gimbal attitude on feature points is addressed through attitude compensation, and virtual feature points are obtained. Then, the image kinematic model is constructed by combining the IBVS method.

[0151] Step S4: Construct the MPC optimization problem, considering the dynamic constraints of the fixed-wing UAV and the perception constraints of the camera, and obtain the control law of the system through online solution, so that the UAV can keep the target near the center of the image while continuously tracking it.

[0152] In a specific application example, step S3 of the present invention includes the following process:

[0153] Step S301: Before designing the controller, describe the construction of the model.

[0154] like Figure 2 As shown, This represents the body coordinate system, where the origin is o. b Located at the center of gravity of the fixed-wing UAV, x b The axis is along the direction of the drone's nose, y b Axis perpendicular to x b The axis points to the left side of the fuselage, z b The axis is perpendicular to the fuselage and pointing upwards. This represents the image coordinate system, with the origin o. i Located at the center of the image; x i axis and y i The axes lie within the image and are parallel to the image's width and height, respectively. This represents the camera coordinate system, where the origin is o. c Located at the camera's optical center, z c The axis is along the optical axis and perpendicular to the image, x c axis and y c The axes are parallel to x. i axis and y i axis.

[0155] Furthermore, the present invention uses a two-degree-of-freedom gimbal camera, see [link / reference]. Figure 3 A two-degree-of-freedom gimbal camera has a yaw angle θ p and pitch angle θ t Where θ p Around x p The axis rotates and x p The axis is perpendicular to the horizontal plane S p θ t Around x t The axis rotates and x t axis and x c Axle load combination located on horizontal plane S pInside. Because this gimbal can yaw in all directions, but the pitch angle is limited, therefore θ p ∈[-π,π],θ t ∈[γ1,γ2]. And γ2>0, and note that when θ t =0 when z c The axis is located in the horizontal plane S p Inside.

[0156] Step S302: Construct the model;

[0157] The fixed-wing UAV under consideration flies at a constant altitude and a constant speed, denoted as H and V, respectively. t Let the yaw, pitch, and roll angles of the UAV be represented by ψ, θ, and φ, respectively. Then, based on the unicycle overall system model, it can be described as follows:

[0158]

[0159] Step S303: Analyze the image kinematic model;

[0160] Define the goal in and The coordinates in P(x) are respectively c ,y c ,z c ) and s(u i ,v i ),based on Figure 2 middle and Based on the relationship and the principle of triangle similarity, we can conclude that:

[0161]

[0162] Where f represents the camera's focal length. Further analysis requires considering the camera's motion.

[0163] Define the camera in The linear velocity and angular velocity in V are respectively c =[T x ,T y ,T z ] T and Ω c =[ω x ,ω y ,ω z ] T Then we have:

[0164]

[0165] Combining formulas (2) and (3), the relationship between the rate of change of feature points and the camera speed can be obtained:

[0166]

[0167] The image Jacobian matrix is:

[0168]

[0169] In a specific application example, step S4 of the present invention includes the following process:

[0170] A reference state was proposed to address the impact of attitude changes on feature points, thereby constructing a feature point change and UAV control input u ψ The relationship between them. For the reference state, its corresponding system attitude is represented as follows:

[0171]

[0172] in It is a given constant.

[0173] Define the goal in The coordinates in the reference state are (u1, v1), and the coordinates in the reference state are (u2, v2). The constructed IBVS model is as follows:

[0174]

[0175] At the same time, the control inputs for the gimbal are given:

[0176]

[0177] Define s2 = [u2, v2] T Based on formula (7), the discrete IBVS model can be expressed as:

[0178]

[0179] For ease of subsequent analysis, formula (9) is expressed as:

[0180] s2(k+1)=f s (s2(k),u ψ (k)) (10)

[0181] Since the yaw rate of a fixed-wing UAV is limited, the following constraints must be met:

[0182] u ψ ∈U set U set =[-u max ,u max (11)

[0183] In addition, to ensure u ψ The change is smooth, and the following acceleration constraints are given:

[0184] Δu∈a set ,a set =[-a max ,a max (12)

[0185] in:

[0186] Δu(k+i)=u ψ (k+i)-u ψ (k+i-1) (13)

[0187] As a preferred embodiment, the present invention further considers the camera's perception constraints. In fact, once the camera is selected, since the parameters remain unchanged, whether a target can be clearly observed and accurately identified in the image depends on its feature point information in the image. Specifically, it is related to the camera's maximum sensing distance, which is defined as L. M Assuming the gimbal's pitch allows the camera's field of view to cover directly below the drone, then combined with the gimbal's omnidirectional yaw, the drone's clear perception range is equivalent to... Figure 4 The cone shape shown.

[0188] To determine if the target is within this range, the image plane containing s2 needs to be transformed to a horizontal plane. The coordinates of the feature point on this horizontal plane are defined as s. ⊥ (u ⊥ ,v ⊥ Considering that the angle corresponding to the virtual image plane where s2 is located in the formula is θ t =-α, then s2 and s ⊥ The following relationship exists between them:

[0189]

[0190] in The third equation in the above formula can be used to solve for u. ⊥ and v ⊥ The value of .

[0191] Since the flight altitude H is constant in this study, the maximum sensing distance of the camera can be equivalent to the maximum horizontal distance R. M ,like Figure 4 As shown. Therefore, a target is considered to be within the camera's clear perception range if and only if the following condition is met:

[0192]

[0193] Furthermore, to ensure that the target remains within clear perception range when the UAV hovers around a stationary target, the α in the defined reference state (6) must satisfy the following condition:

[0194]

[0195] Meanwhile, considering the value of u in formula (11) ψ In addition to the constraints, the following relationships must also be satisfied:

[0196]

[0197] definition Then there is Based on formula (15), the perceptual constraint at time k can be expressed as:

[0198] g1(k)≤0 (18)

[0199] in:

[0200]

[0201] The deviation of the feature points is defined as:

[0202] Δs(k+i|k)=s d -s2(k+i|k) (20)

[0203] Where s d =[0,0] T Let s2 be the expected feature point corresponding to s2, s2(k|k)=s2(k) be the state observation at time k, and Δu(k|k=u ψ (k|k)-u ψ If (k-1|k-1), then the cost function of the optimization problem is defined as:

[0204]

[0205] in:

[0206]

[0207] The control sequence is:

[0208] U r ={u ψ (k|k),u ψ (k+1|k),…,u ψ (k+N p -1|k)}, (23)

[0209] For the prediction domain, Q s =diag{q1,q2} is positive definite and

[0210] Note that J1 = 0 if and only if for all i ∈ {1,…,N} p}, Δs(k+i|k)=0 and Δu(k+i-1|k)=0 hold true, which means that for a stationary target, the fixed-wing UAV will hover around it.

[0211] In addition, the following contraction constraints are also necessary to ensure system stability:

[0212] h1(k+N p +1|k)≤0, (24)

[0213] in:

[0214] h1(k+N p +1|k)=J r (k+N p +1|k)-λ·J r (k+1|k), (25)

[0215] And λ∈(0,1). Simultaneously, the control sequence is redefined as...

[0216] U m ={U r ,u ψ (k+N p |k)}。(26)

[0217] Therefore, based on formulas (10), (11), (12), (18), (21), and (24), the MPC optimization problem can be constructed as follows:

[0218]

[0219] st

[0220] s2(k+i|k)=f s (s2(k+i-1|k),u ψ (k+i-1|k))

[0221] u ψ (k+i-1|k)∈U set

[0222] Δu(k+i-1|k)∈a set

[0223] g1(k+i|k)≤0

[0224] h1(k+N p +1|k)≤0

[0225] where i∈{1,2,…,N}p +1}.

[0226] As a preferred embodiment, the present invention further employs a fast MPC strategy. This is because, with N p The increase in computational power is limited by the processor's computing performance, and the above nonlinear hard constraints g1(k+i|k)≤0 and h1(k+N) are solved online. p The optimization problem of +1|k) is very time-consuming, and may even lead to the inability to obtain a feasible solution within a preset time. To solve this problem, this invention adds hard constraints to the cost function, thereby transforming it into soft constraints. The process includes:

[0227] Add the constraint g1(k+i|k)≤0 to J r From this, we can obtain:

[0228] J m1 (k+i|k)=J r (k+i|k)+β1·max{g1(k+i|k),0} (28)

[0229] in It is a constant.

[0230] Based on this, the contraction constraint corresponding to the formula is transformed into:

[0231] h2(k+N p +1|k)≤0 (29)

[0232] in:

[0233] h2(k+N p +1|k)=J m1 (k+N p +1|k)-λ·J m1 (k+1|k) (30)

[0234] Furthermore, considering u ψ (k+i-1|k)∈U set and Δu(k+i-1|k)∈a set The corresponding UAV dynamics constraints and target motion are unknown, and it may not be possible to make g1(k+1)≤0 when g1(k)>0. Therefore, g1 in formula (19) is redefined as follows to make β1·max{·} in formula (28) work faster.

[0235]

[0236] Where ΔR>0. Then formula (28) can be reformulated as:

[0237] J m1 (k+i|k)=Jr (k+i|k)+β1·max{g2(k+i|k),0}. (32)

[0238] Secondly, constrain h1(k+N) p Adding +1|k) to (32) can be represented as:

[0239] J m2 (k+i|k)=J m1 (k+i|k)+β2·max{h2(k+N p +1|k),0} (33)

[0240] in It is a constant.

[0241] Based on this, the cost function can be expressed as:

[0242]

[0243] Compared to the soft constraint method that introduces slack variables in hard constraints, the max function used in this invention is more efficient. This is because when g(k+i|k)≤0, this nonlinear function does not participate in the gradient calculation for each iteration of the interior point method. However, when using slack variables, the gradient of this function needs to be calculated for each iteration, which is more time-consuming.

[0244] In summary, the MPC optimization problem can be refactored as:

[0245]

[0246] st

[0247] s2(k+i|k)=f s (s2(k+i-1|k),u ψ (k+i-1|k))

[0248] u ψ (k+i-1|k)∈U set

[0249] Δu(k+i-1|k)∈a set

[0250] where i∈{1,2,…,N} p +1}.

[0251] Furthermore, this invention also introduces a soft constraint method for acceleration. The control input of the UAV can be solved using formula (10) combined with the least squares method, and the obtained value is denoted as... Note The initial control sequence can then be represented as:

[0252]

[0253] It can be observed that U0 can easily satisfy the inequality constraint u. ψ (k+i-1|k)∈U set and Δu(k+i-1|k)∈a set This shows that U0 can be used as an initial feasible solution to the optimization problem. Furthermore, let the optimal solution at time k be U. * If (k), then the feasible solution U(k+1) at time k+1 can be defined as:

[0254]

[0255] As a preferred embodiment, the present invention further employs a switching-based control strategy. This is because as the target speed increases, it may exceed the clear perception range of the camera while moving away from the drone, making its details unobservable and even affecting the accuracy of target identification. In fact, the cost function J2 contains β1·max{g2(k+i|k),0}(i=1,…,N) p +1) helps prevent the relative distance from exceeding R. M However, the optimization problem (35) is still more concerned with the convergence of s2. Furthermore, regarding (u2,v2) and (u... ⊥ ,v ⊥ The gradients of all parameters need to be calculated online, which increases the computational load. To address this issue, this invention proposes a switching-based control method. When the relative distance does not exceed R... M At this time, consider optimization problem (35) to make the UAV hover around the target for tracking. However, when the relative distance exceeds R M At that time, consider another optimization problem to quickly bring it back to the camera's clear perception range.

[0256] According to formula (15), it can be found that the perceptual constraint directly depends on (u) ⊥ ,v ⊥ Therefore, to accelerate the convergence of g1(k) to 0 when the perceptual constraint is not satisfied, the following new cost function is defined:

[0257]

[0258] in:

[0259] J n2 (k+i|k)=J n1 (k+i|k)+β2·max{h2(k+N p +1|k),0} (39)

[0260] J n1 (k+i|k)=Δs⊥ (k+i|k) T ·P s ·Δs ⊥ (k+i|k)+Δu(k+i-1|k)·Q u ·Δu(k+i-1|k)(40)

[0261] P s =diag{p1,p2} is positive definite and:

[0262]

[0263] Note the h2(k+N) in formula (39) p +1|k) is different from that in formula (30), it is expressed as:

[0264] J n1 (k+N p +1|k)-λ·J n1 (k+1|k),g1(k)>0 (42)

[0265] Next, we introduce the step function:

[0266]

[0267] This is used to construct the following switching-based MPC optimization problem:

[0268]

[0269] st

[0270] s2(k+i|k)=f s (s2(k+i-1|k),u ψ (k+i-1|k))

[0271] u ψ (k+i-1|k)∈U set

[0272] Δu(k+i-1|k)∈a set

[0273] where i∈{1,2,…,N} p +1}.

[0274] Note that the cost function in equation (38) is a state-based switching function, which is sign-dependent on g1(k). Therefore, for i∈{1,2,…,N} p +1}, if and only if Δs ⊥ The cost function (38) reaches its minimum value of 0 when (k+i|k)=0 and Δu(k+i-1|k)=0. The algorithm in specific applications is as follows: Figure 5 As shown.

[0275] In practical applications, this invention further divides the switching-based MPC optimization problem corresponding to formula (44) into three steps to analyze its stability, corresponding to g1(k)≤0, g1(k>0, and the switching process, respectively. Next, the Lyapunov method will be used to prove its stability, where the Lyapunov function is the cost function corresponding to the formula.

[0276] Limited, when h2(k+N) p When +1|k)>0, we want its gradient to decrease rapidly. To achieve this, the parameter β2 needs to be set large enough to satisfy β2>>max{q1,q2,p1,p2,Q}. u ,β1}. At this time, once h2(k+N p +1|k)≤0 cannot be satisfied, so the cost function corresponding to formula (44) is equivalent to β2·h2(k+N) p +1|k), and use gradient descent to quickly satisfy this condition.

[0277] Next, when h2(k+N) p When +1|k)≤0, the stability of the closed-loop system is proven as follows:

[0278] (1) g1(k) ≤ 0;

[0279] According to the state equation model corresponding to formula (7), the following relationship exists:

[0280] s2(k+i|k+1)=s2(k+i|k),i=1,…,N p (45)

[0281] The optimal control sequence at time k is defined as follows:

[0282]

[0283] Its corresponding optimal cost is defined as The cost of the feasible control sequence obtained based on formula (38) is:

[0284]

[0285] because If and only if s2(k+1|k)=s d And Δu(k+1|k)=0. Therefore, combining formulas (29), (30) and (47), when At that time, there were:

[0286]

[0287] Since the optimal cost at time k+1 must not exceed Therefore, it can be deduced that:

[0288]

[0289] (2) g1(k) > 0;

[0290] Compared to the case where g1(k)≤0, the only difference here is the cost function; therefore, the proof is similar to the above. Referring to formula (48), when At that time, there were:

[0291]

[0292] Then we can obtain:

[0293]

[0294] (3) Switching process;

[0295] To analyze the stability of the closed-loop system at this stage, the optimal cost corresponding to formula (46) is assumed to satisfy the following condition:

[0296]

[0297] Then, the optimal cost at time k can be expressed as:

[0298]

[0299] Meanwhile, the cost of the feasible control sequence at time k+1, obtained based on formula (37), is expressed as:

[0300]

[0301] According to formulas (14), (19), (28) and (40), we can obtain:

[0302]

[0303] Based on the definition of the reference state in formula (6), when the target is initially located to the right of the UAV, the control input obtained based on formula (35) will keep the target to the right of the UAV. That is, during the switching process, v satisfies ⊥ If ≤0, then the range corresponding to v2 is At this point, the following relationship holds:

[0304] f·sinα-v ⊥ (k)·cosα≥f·sinα (56)

[0305]

[0306] The relevant parameter settings are as follows:

[0307]

[0308] in:

[0309]

[0310] Combining formulas (55), (56), (56), and (58), we can obtain:

[0311]

[0312] Furthermore, in formula (54) It can be written as:

[0313]

[0314] Similarly, we can obtain:

[0315]

[0316] In summary, based on formulas (49), (51) and (62), it can be proven that the closed loop of the control system formed by the method of the present invention is asymptotically stable.

[0317] By employing the method described above in this invention, namely the IBVS method based on MPC, the overall system of the fixed-wing UAV and gimbal is controlled, enabling the UAV to continuously track the target while keeping it near the center of the image. This method addresses the limitation of the classic IBVS method in considering constraints by introducing the MPC method to account for the fixed-wing UAV's velocity constraints, acceleration constraints, and camera perception constraints. This allows the UAV to keep the target within the camera's clear perception range during tracking, thereby mapping more feature information onto the image, facilitating the observation of target details and improving recognition accuracy.

[0318] For the MPC optimization problem with nonlinear hard constraints, the method described in this invention employs a fast MPC strategy based on soft constraints and warm start, enabling rapid online solution to meet the real-time target tracking requirements of fixed-wing UAVs. This invention considers that the camera's perception constraints and contraction constraints are nonlinear, which are time-consuming and may lead to infeasible solutions when solved as hard constraints. Therefore, a max function is introduced to add these constraints as soft constraints to the cost function, thereby ensuring the existence of a solution and the speed of the solution process.

[0319] By employing the method described above in this invention, a switching-based MPC control strategy enables a target to quickly return to the camera's clear perception range when it exceeds that range. Since the initially proposed MPC controller aims to achieve hovering tracking of the target by the UAV, it cannot quickly approach when the target exceeds the range, and the computational load of the designed controller also increases. The introduction of the switching controller helps reduce the online computational load at this time and speeds up the return of the target to the range. Furthermore, the stability of the switching controller has been demonstrated through analysis.

[0320] The above are merely preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should be considered within the scope of protection of the present invention.

Claims

1. A target tracking method for a fixed-wing unmanned aerial vehicle based on fast visual servo predictive control, characterized in that, include: Step S1: The drone captures target images using its gimbal camera; Step S2: Obtain the coordinates of feature points on the current image based on the object detection algorithm; Step S3: Address the impact of UAV and gimbal attitude changes on feature points through attitude compensation, obtain virtual feature points, and then construct an image kinematic model using the IBVS method; specifically, the following process is included: Step S301: Describe the construction of the image kinematics model; This represents the body coordinate system, where the origin is... Located at the center of gravity of a fixed-wing UAV, The axis is along the direction of the drone's nose. Axis perpendicular The axis points to the left side of the fuselage. Axis vertical fuselage upward Represents the image coordinate system, origin. Located at the center of the image; shaft and The axes lie within the image and are parallel to the image's width and height, respectively. This represents the camera coordinate system, where the origin is... Located at the camera's optical center, The axis is along the optical axis and perpendicular to the image. shaft and The axes are parallel to shaft and axis; A two-degree-of-freedom gimbal camera has a yaw angle. and pitch angle in, Around The axis rotates and The axis is perpendicular to the horizontal plane , Around The axis rotates and shaft and Axle load combination is located on the horizontal plane Internally; because the gimbal can yaw in all directions, but the pitch angle is limited, therefore... , in and And pay attention when hour The axis is located in the horizontal plane Inside; Step S302: Construct the model; A fixed-wing drone flies at a constant altitude and a constant speed, denoted as altitude and speed respectively. and The yaw, pitch, and roll angles of the UAV are respectively expressed as: The overall system model based on unicycle is described as follows: ； Step S303: Analyze the image kinematic model; Define the goal in and The coordinates in are respectively and ,based on and Based on the relationship and the principle of triangle similarity, we can conclude that: in Indicates the camera's focal length; Define the camera in The linear velocity and angular velocity in are respectively and Then we have: By combining the formula, the relationship between the feature point change rate and the camera speed can be obtained: The image Jacobian matrix is: ； Step S4: Construct the MPC optimization problem, considering the dynamic constraints of the fixed-wing UAV and the perception constraints of the camera, and obtain the control law of the system through online solution, so that the UAV can keep the target near the center of the image while continuously tracking the target.

2. The target tracking method for a fixed-wing UAV based on fast visual servo predictive control according to claim 1, characterized in that, Step S4 includes the following process: For the reference state, the corresponding system attitude is represented as follows: in It is a given constant; Define the goal in The coordinates in are Meanwhile, the coordinates under the reference state are The constructed IBVS model is as follows: At the same time, the control inputs for the gimbal are given: definition The discrete IBVS model can be represented as: The formula can be expressed as: Since the yaw rate of a fixed-wing UAV is limited, the following constraints must be satisfied: To ensure The change is smooth, and the following acceleration constraints are given: in: 。 3. The target tracking method for a fixed-wing UAV based on fast visual servo predictive control according to claim 2, characterized in that, Processing the camera's perceptual constraints includes: First The image plane is transformed to a horizontal plane; the coordinates of the feature points on this horizontal plane are defined as follows: Considering the formula The angle corresponding to the virtual image plane is ,but and The following relationship exists between them: in Solve according to the above formula and The value; Flight altitude If it is constant, then the camera's maximum sensing distance can be equivalent to its maximum horizontal distance. A target is within the camera's clear perception range if and only if the following conditions are met: In the defined reference state The following conditions must be met: right The following relations are also satisfied under the constraints: definition , Then there is ; The perceptual constraints at any given time are represented as: in: The deviation of the feature points is defined as: in yes The corresponding expected feature points, yes State observations at time, The cost function of the optimization problem is then defined as: in: The control sequence is: ， For the prediction domain, It is positive definite and ; If and only if for all ,have and This means that for stationary targets, fixed-wing drones will hover around them.

4. The fixed-wing UAV target tracking method based on fast visual servo predictive control according to claim 3, characterized in that, The contraction constraint satisfies: in: and Meanwhile, the control sequence is redefined as: 。 5. The target tracking method for a fixed-wing UAV based on fast visual servo predictive control according to claim 4, characterized in that, The MPC optimization problem is constructed as follows: st in .

6. The target tracking method for a fixed-wing UAV based on fast visual servo predictive control according to claim 5, characterized in that, Adding hard constraints to the cost function to transform it into soft constraints involves the following steps: Constraints Add to From: in It is a constant; Based on this, the corresponding contraction constraint is transformed into: in: Redefining it as follows: in Then formula (28) can be reformulated as: ； Secondly, constraints After adding, it will be displayed as: in It is a constant; Based on this, the cost function can be expressed as: The MPC optimization problem is refactored as follows: st in .

7. The fixed-wing UAV target tracking method based on fast visual servo predictive control according to claim 6, characterized in that, A switching-based control strategy is adopted; when the relative distance exceeds When the perceived constraints are not met, in order to accelerate The convergence to 0 defines the cost function: in: It is positive definite and: ； in, Represented as: Introducing a step function: This is used to construct the following switching-based MPC optimization problem: st in .

Images