A Visual Tracking Control Method for Non-Cooperative Targets with Velocity Constraints

Abstract

A non-cooperative target visual tracking control method with velocity constraints is proposed. This method constructs a target trajectory estimation algorithm to predict the projected trajectory of the non-cooperative target in real time, designs a nonlinear visual feedback control algorithm based on the observed trajectory and the star-turntable-camera dynamics, and ensures the velocity constraint through focal plane trajectory planning, thereby meeting the camera's requirements for high-precision, high-stability and fast-stability imaging of non-cooperative moving targets.

Description

Technical Field

[0001] This invention relates to a visual tracking control method for non-cooperative targets with velocity constraints, belonging to the field of spacecraft control technology. Background Technology

[0002] Rapid acquisition and high-precision tracking of space targets are the prerequisites and foundations for utilizing and controlling space. Space-based early warning systems combine the advantages of both space-based platforms and optical sensors, offering benefits such as being unrestricted by national borders, having a wide coverage area, high measurement accuracy, and strong concealment, and are increasingly attracting high attention from various countries.

[0003] To achieve the detection, tracking, and identification of ballistic missiles during their active and mid-course phases, one design approach involves using a two-dimensional turntable to drive a camera for highly stable target tracking. Throughout the tracking process, the satellite maintains a zero attitude to ensure the normal operation of other onboard payloads. This model can be equivalent to a three-link free-flying robot system. Unlike traditional satellite platform attitude control, the camera's target tracking process involves the kinematic characteristics of the non-cooperative target, as well as the parameters of the satellite platform, the two-dimensional turntable, and the camera. The object is complex, with more pronounced nonlinearities and uncertainties, posing challenges to the design of the control system.

[0004] In addition, to meet the requirements of high-precision pixel extraction by the camera, not only is it required that the miss distance in the control process track the target center as closely as possible, but also constraints are imposed on the speed of the miss distance movement in both dynamic and steady-state processes. Summary of the Invention

[0005] The technical problem to be solved by this invention is to overcome the shortcomings of the prior art and propose a non-cooperative target visual tracking control method with velocity constraints. The method is simple and effective and has strong engineering application value.

[0006] The technical solution of this invention is: a non-cooperative target visual tracking control method with velocity constraints, comprising:

[0007] Feature analysis is performed on the kinematic model of the task space in the focal plane, and a target trajectory observer is established by utilizing the property that the product of the model's composite matrix and any vector is zero.

[0008] The target trajectory observer is used to estimate the trajectory of a moving target online.

[0009] Considering the attitude coupling of the celestial body and the nonlinear kinematic relationship in image space, a visual feedback control law is designed based on the estimated trajectory of the moving target and the planned value of the focal plane trajectory. The control law is used to achieve the tracking of the position / velocity of the focal plane.

[0010] Preferably, the target trajectory observer is:

[0011]

[0012] in, For the target relative trajectory, K x >0 indicates an adjustable gain. The estimated value of the target's relative trajectory is B(q,y)=(P-ym). T C eo B is the composite matrix of the model. + =B T (BB T ) -1 It is a generalized inverse matrix. q s ∈R 3×1 Let q be the three-axis attitude angles of the satellite relative to its orbital system. z ∈R 2×1 Let y be the azimuth and pitch angle of the turntable, and y be the position of the target on the focal plane. eo Let P ∈ R be the transformation matrix of the end-camera coordinate system relative to the satellite's orbital system. 2×3 For camera projection parameters, m∈R 3×1 These are the depth projection parameters.

[0013] Preferably, the visual feedback control law is:

[0014]

[0015] in, C eb C is the transformation matrix between the camera coordinate system and the satellite body system. bo This is the transformation matrix of the satellite's intrinsic system relative to its orbital system. C represents eb For q z Taking the partial derivative, SumI is the integral of the image shift error, y r ∈R 2 Let y be the planned value of the focal plane movement trajectory, and y represent the projected position of the feature point on the camera's focal plane. d For the desired focal plane position, y r For the planned focal plane position, K I K is the integration parameter. y For proportional parameters, This is an estimate of the depth information.

[0016] Preferably, the expression for the planned value of the focal plane movement trajectory is:

[0017]

[0018] in, y0 represents the initial focal plane position of the target, y0(1) and y0(2) represent the row and column of the focal plane, respectively, v max This is the planned limit value.

[0019] Preferably, the tracking of the focal plane position / velocity using the control law is as follows:

[0020] Control law outputs turntable angular velocity command This command serves as the input to the turntable, driving it to perform servo control. The turntable then moves the camera to achieve target pointing and tracking. Once the camera captures the target, it tracks the target by ensuring that the target point is at the center of the focal plane.

[0021] Preferably, the focal plane position y of the target point is used as input to control the focal plane position y to converge to the planned value y. r This ensures that the target point is at the center of the focal plane, enabling the camera to track the target.

[0022] A non-cooperative target visual tracking control system with velocity constraints, comprising:

[0023] The moving target trajectory estimation module performs feature analysis on the focal plane task space kinematic model, and establishes a target trajectory observer by utilizing the property that the product of the model composite matrix and any vector is zero; the moving target trajectory is estimated online using the target trajectory observer.

[0024] The tracking control module considers the attitude coupling of the celestial body and the nonlinear kinematic relationship in image space. Based on the estimated trajectory of the moving target and the planned value of the focal plane trajectory, a visual feedback control law is designed, and the focal plane position / velocity is tracked using the control law.

[0025] Preferably, the target trajectory observer is:

[0026]

[0027] in, For the target relative trajectory, K x >0 indicates an adjustable gain. The estimated value of the target's relative trajectory is B(q,y)=(P-ym). T C eo B is the composite matrix of the model. + =B T (BB T ) -1 It is a generalized inverse matrix. q s ∈R 3×1 Let q be the three-axis attitude angles of the satellite relative to its orbital system. z ∈R 2×1Let y be the azimuth and pitch angle of the turntable, and y be the position of the target on the focal plane. eo Let P ∈ R be the transformation matrix of the end-camera coordinate system relative to the satellite's orbital system. 2×3 For camera projection parameters, m∈R 3×1 These are the depth projection parameters.

[0028] Preferably, the visual feedback control law is:

[0029]

[0030] in, C eb C is the transformation matrix between the camera coordinate system and the satellite body system. bo This is the transformation matrix of the satellite's intrinsic system relative to its orbital system. C represents eb For q z Taking the partial derivative, SumI is the integral of the image shift error, y r ∈R 2 Let y be the planned value of the focal plane movement trajectory, and y represent the projected position of the feature point on the camera's focal plane. d For the desired focal plane position, y r For the planned focal plane position, K I K is the integration parameter. y For proportional parameters, This is an estimate of the depth information.

[0031] Preferably, the expression for the planned value of the focal plane movement trajectory is:

[0032]

[0033] in, y0 represents the initial focal plane position of the target, y0(1) and y0(2) represent the row and column of the focal plane, respectively, v max This is the planned limit value.

[0034] Compared with the prior art, the present invention has the following advantages:

[0035] (1) This invention proposes a novel control method for the visual tracking problem of non-cooperative targets with focal plane image movement velocity output constraints. This method fully explores the characteristics of the focal plane kinematic model and designs a nonlinear visual feedback control law based on the observation trajectory combined with the dynamics of the star-turntable-camera. This method breaks through the limitations of the high precision, high stability and fast stability imaging requirements of the camera on the velocity constraints of the miss distance.

[0036] (2) This invention effectively improves upon traditional satellite control and traditional visual servo control by introducing moving target trajectory prediction and focal plane trajectory planning, thus solving the problem of non-cooperative target focal plane tracking control with dynamic image movement velocity constraints. By estimating the moving target trajectory and planning the focal plane trajectory, the system's fast and stable performance requirements can be effectively met. The entire algorithm design is simple, and the parameter debugging workload is small.

[0037] (3) This invention proposes a new solution for the problem of moving target tracking with visual servo tracking requirements. It does not require additional data input, is simple to calculate, and can be adapted to a large class of military and civilian satellite systems with non-cooperative moving target tracking requirements. It has strong engineering practicality. Attached Figure Description

[0038] Figure 1 This is a control block diagram of the present invention.

[0039] Figure 2 This is a schematic diagram of the camera's imaging modes.

[0040] Figure 3 The projection of the relative trajectories of the target and the satellite onto the satellite's orbital system.

[0041] Figure 4 The miss distance of the target at the focal plane.

[0042] Figure 5 The rate of change of the target's miss distance at the focal plane. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0044] The target's position projection on the focal plane is related not only to the kinematics of the star / turntable but also to the trajectory of the moving target. Since the model of the tracked target is unknown, this invention establishes a moving target trajectory predictor based on the target's projection kinematics in image space (this predictor is a nonlinear observer coupled with the star-turntable kinematics). Based on the predicted trajectory, a turntable pointing control law based on image space position feedback is designed. This algorithm considers the attitude coupling of the star and the nonlinear kinematic relationship in image space, enabling position / velocity tracking on the focal plane.

[0045] like Figure 1 As shown, this invention relates to a visual tracking control method for non-cooperative targets with velocity constraints, comprising the following steps:

[0046] Step 1: Analyze the kinematic model of the task space in the focal plane.

[0047] Simplified processing of camera imaging model, such as Figure 2 As shown. Imaging errors such as geometric distortion and spherical aberration of the optical system are ignored here.

[0048] The projection position of the feature point on the camera focal plane, y∈R 2 It can be represented as

[0049]

[0050] z(t) = m T C eo (q)x(t)

[0051] Among them, C eo This is the transformation matrix between the satellite orbital system and the end-camera coordinate system. q s ∈R 3×1 Let q be the three-axis attitude angles of the satellite relative to its orbital system. z ∈R 2×1 Let z(t) be the azimuth and pitch angle of the turntable, z(t) be the camera depth information, y be the target's position on the focal plane, and C be the position of the target. eo Let P ∈ R be the transformation matrix of the end-camera coordinate system relative to the satellite's orbital system. 2×3 For camera projection parameters, m∈R 3×1 These are the depth projection parameters; Further, we can obtain...

[0052]

[0053] in, Representing partial derivatives, the model composite matrix B = (P - ym) T C eo .

[0054] Characteristic analysis of the above model reveals that Bη = 0, where η is any 3*1 vector.

[0055] Step 2: Based on the focal plane kinematics model, perform online estimation of the moving target trajectory;

[0056] Based on focal plane kinematics, the target trajectory observer is designed as follows:

[0057]

[0058] in, x represents the relative trajectory of the target, K x >0 indicates an adjustable gain. The estimated value of the target's relative trajectory is B(q,y)=(P-ym). T C eo B is the composite matrix of the model. + =B T(BB T ) -1 It is a generalized inverse matrix;

[0059] Step 3: Design of visual feedback control law.

[0060] Based on the variable calculations in step two, the focal plane tracking controller is designed as follows:

[0061]

[0062] in, C eb C is the transformation matrix between the camera coordinate system and the satellite body system. bo This is the transformation matrix of the satellite's intrinsic system relative to its orbital system. C represents eb For q z Taking the partial derivative, SumI is the integral of the image shift error, y r ∈R 2 Let y be the planned value of the focal plane movement trajectory, and y represent the projected position of the feature point on the camera's focal plane. d For the desired focal plane position, y r For the planned focal plane position, K I K is the integration parameter. y For proportional parameters, This is an estimate of the depth information.

[0063] To meet the velocity constraint of the miss distance during the entire tracking process, a focal plane trajectory planning strategy based on image shift terminal deviation is further proposed, which effectively improves the tracking performance of the system. The expression for the planned focal plane movement trajectory is as follows:

[0064]

[0065] in, y0 represents the initial focal plane position of the target, y0(1) and y0(2) represent the row and column of the focal plane, respectively, v max The planned amplitude limit is determined by the focal plane change rate constraint.

[0066] Once the camera acquires the target, it uses the target's image plane position as input and tracks the target by ensuring the target is centered in the image space. The control law outputs the turntable angular velocity command. This command serves as the input to the turntable, driving it to perform servo control. By changing the rotation angle, it achieves target tracking. The control result, reflected in the focal plane, is the convergence of the focal plane position y to the planned value y. r The block diagram of the entire control loop is as follows: Figure 1 .

[0067] The present invention will be further described below with reference to the embodiments.

[0068] Example:

[0069] Taking a certain satellite as an example, a set of typical trajectories is used to simulate and verify the method proposed in this patent application. The projection of the relative positions of the target and the satellite in the satellite orbital system is as follows: Figure 3 As shown.

[0070] The simulation process considered the time delays of each sensor and actuator. The information processing delay of the focal plane was 400ms. The expected focal plane position yd = [-362, -298] pixels. The requirements were that the image movement velocity during the transition process of the focal plane should not exceed 65 pixels / s, the steady-state process should not exceed 13 pixels / s, and the steady-state tracking deviation of the focal plane should not exceed 5 pixels (the maneuvering angular velocity should not exceed 0.1° / s). The specific output results are as follows: Figures 4-5 As can be seen, the control method proposed in this paper can meet the above-mentioned index requirements.

[0071] The contents not described in detail in this specification are common knowledge to those skilled in the art.

[0072] Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make possible changes and modifications to the technical solutions of the present invention by utilizing the methods and techniques disclosed above without departing from the spirit and scope of the present invention. Therefore, any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solutions of the present invention shall fall within the protection scope of the technical solutions of the present invention.

Claims

1. A non-cooperative target visual tracking control method with velocity constraints, characterized in that... include: Feature analysis is performed on the kinematic model of the task space in the focal plane, and a target trajectory observer is established by utilizing the property that the product of the model's composite matrix and any vector is zero. The target trajectory observer is used to estimate the trajectory of a moving target online. Considering the attitude coupling of the celestial body and the nonlinear kinematic relationship in the image space, a visual feedback control law is designed based on the estimated trajectory of the moving target and the planned value of the focal plane trajectory. The control law is used to achieve the tracking of the position / velocity of the focal plane. The target trajectory observer is: in, , For the target relative trajectory, This is an estimate of the target's relative trajectory. For adjustable gain, For the model composite matrix, It is a generalized inverse matrix. , Let be the satellite's three-axis attitude angles relative to its orbital frame. For the turntable's azimuth and pitch angle, The position of the target on the focal plane, This is the transformation matrix of the end-camera coordinate system relative to the satellite's orbital system. For camera projection parameters, These are the depth projection parameters; The visual feedback control law is: in, , , Let be the transformation matrix of the camera coordinate system relative to the satellite coordinate system. This is the transformation matrix of the satellite's intrinsic system relative to its orbital system. express right Taking the partial derivative, SumI is the integral of the image shift error. The planned value for the focal plane movement trajectory. This indicates the projection position of the feature point onto the camera's focal plane. For the desired focal plane position, For the planned focal plane position, For integration parameters, For proportional parameters, , is the depth information estimate.

2. The visual tracking control method according to claim 1, characterized in that, The expression for the planned value of the focal plane movement trajectory is: in, , Let this be the initial focal plane position of the target. and These represent the rows and columns of the focal plane, respectively. This is the planned limit value.

3. The visual tracking control method according to claim 1, characterized in that, The control law is used to track the position / velocity of the focal plane as follows: Control law outputs turntable angular velocity command This command serves as the input to the turntable, driving the turntable to perform servo control. The turntable then moves the camera to achieve target pointing and tracking. Once the camera captures the target, it tracks the target by ensuring that the target point is at the center of the focal plane.

4. The visual tracking control method according to claim 3, characterized in that, With the focal plane position of the target point As input, control the focal plane position convergence to the planned value This ensures that the target point is at the center of the focal plane, enabling the camera to track the target.

5. A non-cooperative target visual tracking control system with velocity constraints, characterized in that... include: The moving target trajectory estimation module performs feature analysis on the focal plane task space kinematic model and establishes a target trajectory observer by utilizing the property that the product of the model composite matrix and any vector is zero. The target trajectory observer is used to estimate the trajectory of a moving target online. The tracking control module considers the attitude coupling of the celestial body and the nonlinear kinematic relationship in the image space. Based on the estimated trajectory of the moving target and the planned value of the focal plane trajectory, a visual feedback control law is designed, and the control law is used to realize the tracking of the position / velocity of the focal plane. The target trajectory observer is: in, , For the target relative trajectory, This is an estimate of the target's relative trajectory. For adjustable gain, For the model composite matrix, It is a generalized inverse matrix. , Let be the satellite's three-axis attitude angles relative to its orbital frame. For the turntable's azimuth and pitch angle, The position of the target on the focal plane, This is the transformation matrix of the end-camera coordinate system relative to the satellite's orbital system. For camera projection parameters, These are the depth projection parameters; The visual feedback control law is: in, , , Let be the transformation matrix of the camera coordinate system relative to the satellite coordinate system. This is the transformation matrix of the satellite's intrinsic system relative to its orbital system. express right Taking the partial derivative, SumI is the integral of the image shift error. The planned value for the focal plane movement trajectory. This indicates the projection position of the feature point onto the camera's focal plane. For the desired focal plane position, For the planned focal plane position, For integration parameters, For proportional parameters, , is the depth information estimate.

6. The visual tracking control system according to claim 5, characterized in that, The expression for the planned value of the focal plane movement trajectory is: in, , Let this be the initial focal plane position of the target. and These represent the rows and columns of the focal plane, respectively. This is the planned limit value.

Images