Non-cooperative target relative state tracking method, system, equipment and medium
Through multi-scale fusion deep learning network and Bayes filtering algorithm, combined with attention mechanism, the problem of traditional algorithms' robustness and low accuracy in complex space environments is solved, and high-precision relative state tracking is achieved, suitable for non-cooperation in space The close rendezvous and docking task of the target.
Patent Information
- Application Number
- CN202310639912.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-31
- Publication Date
- 2024-12-03
AI Technical Summary
Traditional pose tracking algorithms are low in robustness and low estimation accuracy in complex space environments, and cannot effectively estimate the relative state of space non-cooperation targets, especially in close rendezvous and docking tasks.
A multi-scale fusion lightweight deep learning network is designed, combined with Bayes filtering algorithm, and through the initial pose estimation and relative state estimation steps, high-precision and high-rootty spatial non-cooperation target relative state tracking is achieved. This method includes feature extraction network and prediction network, using channel attention module and spatial attention mechanism, constructing system state equations and measurement equations, and carrying out a deep coupled filtering framework for relative kinematics and dynamics models.
It improves the accuracy and filtering robustness of relative pose estimation, reduces network parameters, and improves inference efficiency, and is suitable for relative state tracking tasks in complex space environments.
Smart Images

Figure CN119068018A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of spacecraft control technology, and in particular to a method, system, device and medium for tracking the relative state of a non-cooperative target. Background Art
[0002] Most traditional pose tracking algorithms are based on the Bayesian filtering framework, and there is still insufficient research on the robustness of the filtering algorithm caused by feature point loss. In complex space environment conditions, the algorithm has problems such as low robustness and low estimation accuracy.
[0003] Existing related literature, [1] Yan Kun, Research on spatial non-cooperative target posture measurement technology based on binocular vision. 2018, University of Chinese Academy of Sciences (Institute of Optoelectronics Technology, Chinese Academy of Sciences), binocular vision is used to estimate the relative posture of spatial non-cooperative targets, and the baseline of binocular stereo vision is related to the measurement distance. The longer the detection distance, the longer the corresponding baseline, and the higher the hardware conditions are required. Related literature [2] Oestreich, C., TWLim, and R. Broussard, On-Orbit Relative Pose Initialization via Convolutional Neural Networks, in AIAAScitech 2020 Forum. 2020: Orlando, FL. Alexnet network is used to realize relative posture estimation, but relative posture estimation is not realized. For space close-range rendezvous and docking tasks, it is not enough to estimate the relative posture alone, and further estimation of the relative state is required. Summary of the invention
[0004] In view of the defects in the prior art, the present invention provides a method, system, device and medium for tracking the relative state of a non-cooperative target.
[0005] According to a non-cooperative target relative state tracking method, system, device and medium provided by the present invention, by designing a multi-scale fusion lightweight deep learning network and analyzing the distribution characteristics of the prediction error, deep coupling with the Bayes filter algorithm is achieved to achieve high-precision and high-robustness spatial non-cooperative target relative state estimation. The scheme is as follows:
[0006] In a first aspect, a non-cooperative target relative state tracking method is provided, the method comprising:
[0007] Initial pose estimation step: collecting image data, preprocessing the image data, and constructing and training an initial pose estimation model;
[0008] Relative state estimation step: construct a filtering model, which includes the system state equation and measurement equation. The relative pose estimated by the filtering model is fed back to the initial pose estimation step. The relative pose is predicted by the trained initial pose estimation model and sent to the filtering model to finally achieve relative state tracking.
[0009] Preferably, the initial pose estimation step comprises:
[0010] Step S1.1: Use a monocular camera to capture images with an image size of M×M;
[0011] Step S1.2: preprocessing the image, including denoising, normalization and cropping operations, wherein the image of size M×M is reduced to 256×256 in order to adapt to the input of different networks;
[0012] Step S1.3: Establishing an initial pose estimation model, wherein the initial pose estimation model includes: a feature extraction network and a prediction network;
[0013] The 3×M×M image is input into the feature extraction network, and the convolution, pooling and downsampling operations are implemented through the K-layer network to obtain an N×P×P feature map, which is then divided into P×P N-dimensional feature vectors, and P×P prediction outputs are obtained after passing through the prediction network;
[0014] The N-dimensional feature vector is input into the prediction network, which consists of multiple fully connected layers, including target position prediction, whether the target is included, and pose regression.
[0015] Preferably, a channel attention module and a spatial attention mechanism module are added after the convolution layer of the initial pose estimation model to assign different weights to different channel feature maps;
[0016] Among them, the input 3×M×M image passes through the feature extraction network to obtain N×P×P feature maps. The N feature maps capture target features from different scales. The channel attention mechanism is used to assign learnable weights to the N P×P feature maps to capture effective information.
[0017] For the spatial attention mechanism, learnable weights are assigned to the P×P pixel values on the feature map to capture effective information at different spatial locations.
[0018] Preferably, constructing the system state equation in the relative state estimation step includes:
[0019] The system state equation is expressed as:
[0020]
[0021] The measurement equation is expressed as:
[0022] z=h(x)+v(t) (2)
[0023] Where z represents the measured quantity, x represents the state quantity; f(x) and h(x) represent the process equation and the measurement equation respectively; w(t) and v(t) represent the process noise and the measurement noise;
[0024] The kinematic equation of the relative motion attitude is as follows:
[0025]
[0026] Among them, q represents quaternion, Q(q) represents quaternion matrix, R(q) represents rotation matrix, represents the relative angular velocity relative to the host spacecraft;
[0027] When the spacecraft is not disturbed by external forces, that is, N L =N F = 0, the dynamic model of relative angular velocity is further simplified:
[0028]
[0029] Among them, the matrix I F represents the inertia tensor matrix of the non-cooperative target, Indicates I F I L represents the moment of inertia matrix of the target spacecraft; N L and N F They represent the external forces on the target spacecraft and the non-cooperative target respectively; represents the angular velocity of the target spacecraft; represents the angular velocity of the non-cooperative target relative to the target spacecraft;
[0030] When the matrix I F When it is a diagonal matrix, there is an unobservable estimator. Assume that the inertia tensor matrix of the non-cooperative space target is not a diagonal matrix, but a symmetric matrix I F , further, rewrite I F For the matrix The ratio of the inertia tensor is as follows:
[0031]
[0032] Therefore, formula (4) is further rewritten as formula (6) as follows:
[0033]
[0034] The system status is as follows:
[0035]
[0036] Since ρ s Represents the coordinates of the feature point in the target coordinate system, and the spatial target object is a rigid body, so is the zero vector, and the moment of inertia ratio Therefore is also the zero vector;
[0037] In the relative kinematics equation of the feature point, the coordinates of the feature point must be solved in the camera coordinate system, so ρ c =Rρ s =[x c ,y c ,z c ] T Indicates that ρ s Transformed to the camera coordinate system, ρ c represents the vector of the feature point in the camera coordinate system, R represents the rotation matrix, x c ,y c 、z c Represents ρ c The components of , that is, the components in the x, y, and z directions; the system equation used is described by equation (8):
[0038]
[0039] Among them, x0, y0, and z0 represent the position components of the origin of the target coordinate system in the camera coordinate system respectively; They represent the velocity components of the origin of the target coordinate system in the camera coordinate system respectively; represents the first derivative of the true anomaly; represents the second derivative of the true anomaly; μ represents the earth's gravitational constant; r L The magnitude of the position vector of the main spacecraft in the Earth coordinate system; ω x ,ω y ,ω z are the angular velocities of the non-cooperative target relative to the target spacecraft. The amount of They represent angular acceleration, that is, The weight; 0 3×1 Represents a zero vector of length 3; 0 5×1 represents a zero vector of length 5; and are equivalent, indicating The component of , that is, the velocity component; They represent the acceleration components, that is, The amount.
[0040] Preferably, constructing the measurement equation in the relative state estimation step includes:
[0041] The measurement equation is a linear equation as shown in equation (9):
[0042]
[0043] Represents the unit matrix. According to the state estimation at the previous moment, the relative state at the current moment is predicted by combining the measurement equation and the process equation through equations (8) and (9).
[0044] Preferably, constructing a filtering model in the relative state estimation step includes:
[0045] State one-step prediction equation:
[0046]
[0047] State estimation equation:
[0048]
[0049] Filter gain:
[0050]
[0051] One-step forecast mean square error:
[0052]
[0053] Estimated mean square error
[0054] P k =(IK k H k ) k / k-1 (14)
[0055] in, represents the one-step state estimation, Φ k,k-1 represents the state transfer matrix; represents the state estimation at time k-1; represents the current state estimate at time k, K k represents the gain matrix, z k represents the quantity measurement at time k, I represents the unit matrix, P k represents the estimated mean square error, P k / k-1 represents the one-step prediction mean square error, H k represents the measurement matrix, Γ k-1 represents the process noise covariance matrix, R k represents the covariance matrix of the measurement noise, Q k-1 represents the covariance matrix of the process noise;
[0056] The relative posture feedback in the relative state estimation step includes:
[0057] At time k, the relative pose estimated by the filtering model is fed back to the initial pose estimation module to assist the relative pose estimation at time k+1. At time k+1, the relative pose at time k+1 is predicted by the trained initial pose estimation model and sent to the filtering model as a quantitative measurement.
[0058] Repeat the initial pose estimation step and the relative state estimation step to achieve relative state tracking.
[0059] In a second aspect, a non-cooperative target relative state tracking system is provided, the system comprising:
[0060] Initial pose estimation module: collects image data, preprocesses the image data, and constructs and trains an initial pose estimation model;
[0061] Relative state estimation module: Build a filter model, which includes the system state equation and measurement equation. The relative pose estimated by the filter model is fed back to the initial pose estimation module. The relative pose is predicted by the trained initial pose estimation model and sent to the filter model to finally achieve relative state tracking.
[0062] Preferably, the initial pose estimation module comprises:
[0063] Module M1.1: uses a monocular camera to collect images with an image size of M×M;
[0064] Module M1.2: preprocessing the image, including denoising, normalization and cropping operations, wherein the image of size M×M is reduced to 256×256 in order to adapt to the input of different networks;
[0065] Module M1.3: Establishing an initial pose estimation model, wherein the initial pose estimation model includes: a feature extraction network and a prediction network;
[0066] The 3×M×M image is input into the feature extraction network, and the convolution, pooling and downsampling operations are implemented through the K-layer network to obtain an N×P×P feature map, which is then divided into P×P N-dimensional feature vectors, and P×P prediction outputs are obtained after passing through the prediction network;
[0067] The N-dimensional feature vector is input into the prediction network, which consists of multiple fully connected layers, including target position prediction, whether the target is included, and pose regression;
[0068] Adding a channel attention module and a spatial attention mechanism module after the convolution layer of the initial pose estimation model to assign different weights to different channel feature maps;
[0069] Among them, the input 3×M×M image passes through the feature extraction network to obtain N×P×P feature maps. The N feature maps capture target features from different scales. The channel attention mechanism is used to assign learnable weights to the N P×P feature maps to capture effective information.
[0070] For the spatial attention mechanism, learnable weights are assigned to the P×P pixel values on the feature map to capture effective information at different spatial locations;
[0071] Constructing the system state equation in the relative state estimation module includes:
[0072] The system state equation is expressed as:
[0073]
[0074] The measurement equation is expressed as:
[0075] z=h(x)+v(t) (2)
[0076] Where z represents the measured quantity, x represents the state quantity; f(x) and h(x) represent the process equation and the measurement equation respectively; w(t) and v(t) represent the process noise and the measurement noise;
[0077] The kinematic equation of the relative motion attitude is as follows:
[0078]
[0079] Among them, q represents quaternion, Q(q) represents quaternion matrix, R(q) represents rotation matrix, represents the relative angular velocity relative to the host spacecraft;
[0080] When the spacecraft is not disturbed by external forces, that is, N L =N F = 0, the dynamic model of relative angular velocity is further simplified:
[0081]
[0082] Among them, the matrix I F represents the inertia tensor matrix of the non-cooperative target, Indicates I F I L represents the moment of inertia matrix of the target spacecraft; N L and N F They represent the external forces on the target spacecraft and the non-cooperative target respectively; represents the angular velocity of the target spacecraft; represents the angular velocity of the non-cooperative target relative to the target spacecraft;
[0083] When the matrix IF When it is a diagonal matrix, there is an unobservable estimator. Assume that the inertia tensor matrix of the non-cooperative space target is not a diagonal matrix, but a symmetric matrix I F , further, rewrite I F For the matrix The ratio of the inertia tensor is as follows:
[0084]
[0085] Therefore, formula (4) is further rewritten as formula (6) as follows:
[0086]
[0087] The system status is as follows:
[0088]
[0089] Since ρ s Represents the coordinates of the feature point in the target coordinate system, and the spatial target object is a rigid body, so is the zero vector, and the moment of inertia ratio Therefore is also the zero vector;
[0090] In the relative kinematics equation of the feature point, the coordinates of the feature point must be solved in the camera coordinate system, so ρ c =Rρ s =[x c ,y c ,z c ] T Indicates that ρ s Transformed to the camera coordinate system, ρ c represents the vector of the feature point in the camera coordinate system, R represents the rotation matrix, x c ,y c 、z c Represents ρ c The components of , that is, the components in the x, y, and z directions; the system equation used is described by equation (8):
[0091]
[0092] Among them, x0, y0, and z0 represent the position components of the origin of the target coordinate system in the camera coordinate system respectively; They represent the velocity components of the origin of the target coordinate system in the camera coordinate system respectively; represents the first derivative of the true anomaly; represents the second derivative of the true anomaly; μ represents the earth's gravitational constant; r L The magnitude of the position vector of the main spacecraft in the Earth coordinate system; ω x ,ωy ,ω z are the angular velocities of the non-cooperative target relative to the target spacecraft. The amount of They represent angular acceleration, that is, The weight; 0 3×1 Represents a zero vector of length 3; 0 5×1 represents a zero vector of length 5; and are equivalent, indicating The component of , that is, the velocity component; They represent the acceleration components, that is, The amount of
[0093] The measurement equation constructed in the relative state estimation module includes:
[0094] The measurement equation is a linear equation as shown in equation (9):
[0095]
[0096] represents the unit matrix. According to the state estimation at the previous moment, the relative state at the current moment is predicted by combining the measurement equation and the process equation through equations (8) and (9);
[0097] Constructing a filtering model in the relative state estimation module includes:
[0098] State one-step prediction equation:
[0099]
[0100] State estimation equation:
[0101]
[0102] Filter gain:
[0103]
[0104] One-step forecast mean square error:
[0105]
[0106] Estimated mean square error
[0107] P k =(IK k H k ) k / k-1 (14)
[0108] in, represents the one-step state estimation, Φk,k-1 represents the state transfer matrix; represents the state estimation at time k-1; represents the current state estimate at time k, K k represents the gain matrix, z k represents the quantity measurement at time k, I represents the unit matrix, P k represents the estimated mean square error, P k / k-1 represents the one-step prediction mean square error, H k represents the measurement matrix, Γ k-1 represents the process noise covariance matrix, R k represents the covariance matrix of the measurement noise, Q k-1 represents the covariance matrix of the process noise;
[0109] The relative posture feedback in the relative state estimation module includes:
[0110] At time k, the relative pose estimated by the filtering model is fed back to the initial pose estimation module to assist the relative pose estimation at time k+1. At time k+1, the relative pose at time k+1 is predicted by the trained initial pose estimation model and sent to the filtering model as a quantitative measurement.
[0111] Repeatedly trigger the initial pose estimation module and the relative state estimation module to achieve relative state tracking.
[0112] According to a third aspect, a computer-readable storage medium storing a computer program is provided, and when the computer program is executed by a processor, the steps in the non-cooperative target relative state tracking method are implemented.
[0113] In a fourth aspect, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the steps in the non-cooperative target relative state tracking method when executed by the processor.
[0114] Compared with the prior art, the present invention has the following beneficial effects:
[0115] 1. The present invention designs a separable convolutional network as a reference network to design a relative posture prediction model, which greatly reduces network parameters and improves inference efficiency;
[0116] 2. The target size changes in multiple scales as the relative position between the non-cooperative target and the camera changes. A dual attention mechanism is proposed to improve the accuracy of relative pose estimation.
[0117] 3. In order to solve the problem of poor robustness of traditional estimation methods based on feature points and models, we utilize the superior feature expression ability of deep learning networks and combine the relative kinematics and dynamics models of non-cooperative targets to design an integrated relative state tracking filtering framework to improve filtering robustness and estimation accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0118] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:
[0119] Figure 1 This is the overall block diagram of the system. DETAILED DESCRIPTION
[0120] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several changes and improvements can also be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.
[0121] The embodiment of the present invention provides a relative state tracking method for a non-cooperative target, which uses the relative pose estimation as the quantity measurement of the Bayes filter, and then realizes the relative state estimation by establishing a process model in a complex environment. The relative state tracking method includes two parts: 1) initial pose estimation; 2) relative state estimation. Figure 1 As shown, the method specifically includes:
[0122] Initial pose estimation step: collecting image data, preprocessing the image data, and constructing and training an initial pose estimation model.
[0123] Relative state estimation step: Construct a filter model. The filter model includes the system state equation and the measurement equation. The relative pose estimated by the filter model is fed back to the initial pose estimation step. The relative pose is predicted by the trained initial pose estimation model and sent to the filter model to finally achieve relative state tracking. Among them, the filter model realizes relative state estimation. The relative pose includes position and pose. The initial pose estimation can also be called initial relative pose estimation. We estimate the relative position relationship between the two targets, not the absolute position relationship. The relative state includes not only the pose but also the speed, angular velocity, etc. It is necessary to establish a dynamic equation to estimate it, so a filter model needs to be established.
[0124] Specifically, the initial pose estimation step includes:
[0125] Step S1.1: Use a monocular camera to capture images with an image size of M×M.
[0126] Step S1.2: Preprocess the image. In the complex space environment, the image is inevitably affected by factors such as sunlight, occlusion, and background noise. The acquired image needs to be preprocessed, including denoising, normalization, and cropping. The purpose of cropping is to adapt to the input of different networks and reduce the size of the M×M image to 256×256.
[0127] Step S1.3: Establish an initial pose estimation model, which includes: feature extraction network and prediction network. The 3×M×M image is input into the feature extraction network, and convolution, pooling and downsampling operations are performed through the K-layer network to obtain N×P×P feature maps, which are then divided into P×P N-dimensional feature vectors, and P×P prediction outputs are obtained after passing through the prediction network; the N-dimensional feature vectors are input into the prediction network, which consists of multiple fully connected layers, including target position prediction, whether the target is included, and pose regression.
[0128] It is easy to know that the deeper the network, the more abstract the output feature expression, corresponding to a larger receptive field, and has a good representation of large-scale targets. On the contrary, the shallow network corresponds to a smaller receptive field, pays more attention to the expression of details, and has a better representation of small-scale targets. Similarly, the target features represented by the feature maps of different channels are also different. Therefore, a channel attention module and a spatial attention mechanism module can be added after the convolution layer to assign different weights to the feature maps of different channels, thereby improving the feature extraction and prediction capabilities of the model. Specifically, after the input 3×M×M image passes through the feature extraction network, an N×P×P feature map is obtained. The N feature maps capture target features from different scales. The channel attention mechanism is used to assign learnable weights to the N P×P feature maps to capture effective information. For the spatial attention mechanism, learnable weights are assigned to the P×P pixel values on the feature map to capture effective information at different spatial positions, thereby improving the feature extraction and prediction capabilities of the model.
[0129] Specifically, the relative state estimation step includes:
[0130] Step S2.1: Construct the system state equation
[0131] The system state equation is expressed as:
[0132]
[0133] The measurement equation is expressed as:
[0134] z=h(x)+v(t) (2)
[0135] Where z represents the measured quantity, x represents the state quantity; f(x) and h(x) represent the process equation and the measurement equation respectively; w(t) and v(t) represent the process noise and the measurement noise;
[0136] Since the prior information about the structure and motion of the non-cooperative target is unknown in the application scenario of the present invention, the motion parameters of the non-cooperative target in the system kinematic model are also unknown. Further, the kinematic equation of the relative motion posture is as follows:
[0137]
[0138] Among them, q represents quaternion, Q(q) represents quaternion matrix, R(q) represents rotation matrix, Represents the relative angular velocity relative to the host spacecraft.
[0139] When the spacecraft is not disturbed by external forces, that is, N L =N F = 0, the dynamic model of relative angular velocity is further simplified:
[0140]
[0141] Among them, the matrix I F represents the inertia tensor matrix of the non-cooperative target, Indicates I F I L represents the moment of inertia matrix of the target spacecraft; N L and N F They represent the external forces on the target spacecraft and the non-cooperative target respectively; represents the angular velocity of the target spacecraft; represents the angular velocity of the non-cooperative target relative to the target spacecraft.
[0142] When the matrix I F When is a diagonal matrix, there is an unobservable estimator. In practical applications, due to the lack of prior knowledge of the geometric structure of non-cooperative space targets, the principal axis of the space target is difficult to determine. Therefore, this topic assumes that the inertia tensor matrix of the non-cooperative space target is not a diagonal matrix, but a symmetric matrix I F , further, rewrite I F For the matrix The ratio of the inertia tensor is as follows:
[0143]
[0144] Therefore, formula (4) is further rewritten as formula (6) as follows:
[0145]
[0146] The system status is as follows:
[0147]
[0148] Since ρ s Represents the coordinates of the feature point in the target coordinate system, and the spatial target object is a rigid body, so is the zero vector, and the moment of inertia ratio Therefore is also the zero vector;
[0149] In the relative kinematics equation of the feature point, the coordinates of the feature point must be solved in the camera coordinate system, so ρ c =Rρ s =[x c ,y c ,z c ] T Indicates that ρ s Transformed to the camera coordinate system, ρ c represents the vector of the feature point in the camera coordinate system, R represents the rotation matrix, x c ,y c 、z c Represents ρ c The components of , that is, the components in the x, y, and z directions; the system equation used is described by equation (8):
[0150]
[0151] Among them, x0, y0, and z0 represent the position components of the origin of the target coordinate system in the camera coordinate system respectively; They represent the velocity components of the origin of the target coordinate system in the camera coordinate system respectively; represents the first derivative of the true anomaly; represents the second derivative of the true anomaly; μ represents the earth's gravitational constant; r L The magnitude of the position vector of the main spacecraft in the Earth coordinate system; ω x ,ω y ,ω z are the angular velocities of the non-cooperative target relative to the target spacecraft. The weight of They represent angular acceleration, that is, The weight; 0 3×1 Represents a zero vector of length 3; 0 5×1 represents a zero vector of length 5; and are equivalent, indicating The component of , that is, the velocity component; They represent the acceleration components, that is, The amount.
[0152] Step S2.2 constructs the measurement equation including:
[0153] The measurement equation is a linear equation as shown in equation (9):
[0154]
[0155] Represents the unit matrix. According to the state estimation at the previous moment, the relative state at the current moment is predicted by combining the measurement equation and the process equation through equations (8) and (9).
[0156] Step S2.3: Constructing the filtering model includes:
[0157] State one-step prediction equation:
[0158]
[0159] State estimation equation:
[0160]
[0161] Filter gain:
[0162]
[0163] One-step forecast mean square error:
[0164]
[0165] Estimated mean square error
[0166] P k =(IK k H k ) k / k-1 (14)
[0167] in, represents the one-step state estimation, Φ k,k-1 represents the state transfer matrix; represents the state estimation at time k-1; represents the current state estimate at time k, K k represents the gain matrix, z k represents the quantity measurement at time k, I represents the unit matrix, P k represents the estimated mean square error, P k / k-1 represents the one-step prediction mean square error, H k represents the measurement matrix, Γ k-1 represents the process noise covariance matrix, R k represents the covariance matrix of the measurement noise, Q k-1 represents the covariance matrix of the process noise;
[0168] Step S2.4: Relative pose feedback:
[0169] At time k, the relative pose estimated by the filtering model is fed back to the initial pose estimation module to assist the relative pose estimation at time k+1; at time k+1, the relative pose at time k+1 is predicted by the trained initial pose estimation model and sent to the filtering model as a measurement; steps S1.1-S1.3 and S2.1-S2.4 are repeated to achieve relative state tracking.
[0170] The present invention also provides a non-cooperative target relative state tracking system, which can be implemented by executing the process steps of the non-cooperative target relative state tracking method, that is, those skilled in the art can understand the non-cooperative target relative state tracking method as a preferred implementation of the non-cooperative target relative state tracking system. The system specifically includes the following contents:
[0171] Initial pose estimation module: collects image data, preprocesses the image data, and constructs and trains an initial pose estimation model.
[0172] Relative state estimation module: Construct a filter model. The filter model includes the system state equation and the measurement equation. The relative pose estimated by the filter model is fed back to the initial pose estimation module. The relative pose is predicted by the trained initial pose estimation model and sent to the filter model to finally achieve relative state tracking. Among them, the filter model implements relative state estimation. The relative pose includes position and pose. The initial pose estimation can also be called initial relative pose estimation. We estimate the relative position relationship between the two targets, not the absolute position relationship. The relative state includes not only the pose but also the speed, angular velocity, etc. It is necessary to establish a dynamic equation to estimate it, so a filter model needs to be established.
[0173] Specifically, the initial pose estimation module includes:
[0174] Module M1.1: Use a monocular camera to capture images with an image size of M×M.
[0175] Module M1.2: Preprocess the image. In the complex space environment, the image is inevitably affected by factors such as sunlight, occlusion, and background noise. The acquired image needs to be preprocessed, including denoising, normalization, and cropping. The purpose of cropping is to adapt to the input of different networks and reduce the size of the M×M image to 256×256.
[0176] Module M1.3: Establish an initial pose estimation model, which includes: feature extraction network and prediction network. The 3×M×M image is input into the feature extraction network, and the convolution, pooling and downsampling operations are implemented through the K-layer network to obtain N×P×P feature maps, which are then divided into P×P N-dimensional feature vectors. After passing through the prediction network, P×P prediction outputs are obtained; the N-dimensional feature vectors are input into the prediction network, which consists of multiple fully connected layers, including target position prediction, whether the target is included, and pose regression.
[0177] It is easy to know that the deeper the network, the more abstract the output feature expression, corresponding to a larger receptive field, and has a good representation of large-scale targets. On the contrary, the shallow network corresponds to a smaller receptive field, pays more attention to the expression of details, and has a better representation of small-scale targets. Similarly, the target features represented by the feature maps of different channels are also different. Therefore, a channel attention module and a spatial attention mechanism module can be added after the convolution layer to assign different weights to the feature maps of different channels, thereby improving the feature extraction and prediction capabilities of the model. Specifically, after the input 3×M×M image passes through the feature extraction network, an N×P×P feature map is obtained. The N feature maps capture target features from different scales. The channel attention mechanism is used to assign learnable weights to the N P×P feature maps to capture effective information. For the spatial attention mechanism, learnable weights are assigned to the P×P pixel values on the feature map to capture effective information at different spatial positions, thereby improving the feature extraction and prediction capabilities of the model.
[0178] Specifically, the relative state estimation module includes:
[0179] Module M2.1: Constructing the system state equation
[0180] The system state equation is expressed as:
[0181]
[0182] The measurement equation is expressed as:
[0183] z=h(x)+v(t) (2)
[0184] Where z represents the measured quantity, x represents the state quantity; f(x) and h(x) represent the process equation and the measurement equation respectively; w(t) and v(t) represent the process noise and the measurement noise;
[0185] Since the prior information about the structure and motion of the non-cooperative target is unknown in the application scenario of the present invention, the motion parameters of the non-cooperative target in the system kinematic model are also unknown. Further, the kinematic equation of the relative motion posture is as follows:
[0186]
[0187] Among them, q represents quaternion, Q(q) represents quaternion matrix, R(q) represents rotation matrix, Represents the relative angular velocity relative to the host spacecraft.
[0188] When the spacecraft is not disturbed by external forces, that is, N L =N F = 0, the dynamic model of relative angular velocity is further simplified:
[0189]
[0190] Among them, the matrix I F represents the inertia tensor matrix of the non-cooperative target, Indicates I F I L represents the moment of inertia matrix of the target spacecraft; N L and N F They represent the external forces on the target spacecraft and the non-cooperative target respectively; represents the angular velocity of the target spacecraft; represents the angular velocity of the non-cooperative target relative to the target spacecraft.
[0191] When the matrix I F When is a diagonal matrix, there is an unobservable estimator. In practical applications, due to the lack of prior knowledge of the geometric structure of non-cooperative space targets, the principal axis of the space target is difficult to determine. Therefore, this topic assumes that the inertia tensor matrix of the non-cooperative space target is not a diagonal matrix, but a symmetric matrix I F , further, rewrite I F For the matrix The ratio of the inertia tensor is as follows:
[0192]
[0193] Therefore, formula (4) is further rewritten as formula (6) as follows:
[0194]
[0195] The system status is as follows:
[0196]
[0197] Since ρ s Represents the coordinates of the feature point in the target coordinate system, and the spatial target object is a rigid body, so is the zero vector, and the moment of inertia ratio Therefore is also the zero vector;
[0198] In the relative kinematics equation of the feature point, the coordinates of the feature point must be solved in the camera coordinate system, so ρ c =Rρ s =[x c ,y c ,z c ] T Indicates that ρ s Transformed to the camera coordinate system, ρ c represents the vector of the feature point in the camera coordinate system, R represents the rotation matrix, x c ,y c 、z c Represents ρ c The components of , that is, the components in the x, y, and z directions; the system equation used is described by equation (8):
[0199]
[0200] Among them, x0, y0, and z0 represent the position components of the origin of the target coordinate system in the camera coordinate system respectively; They represent the velocity components of the origin of the target coordinate system in the camera coordinate system respectively; represents the first derivative of the true anomaly; represents the second derivative of the true anomaly; μ represents the earth's gravitational constant; r L The magnitude of the position vector of the main spacecraft in the Earth coordinate system; ω x ,ω y ,ω z are the angular velocities of the non-cooperative target relative to the target spacecraft. The weight of They represent angular acceleration, that is, The weight; 0 3×1 Represents a zero vector of length 3; 0 5×1 represents a zero vector of length 5; and are equivalent, indicating The component of , that is, the velocity component; They represent the acceleration components, that is, The amount.
[0201] Module M2.2 builds measurement equations including:
[0202] The measurement equation is a linear equation as shown in equation (9):
[0203]
[0204] Represents the unit matrix. According to the state estimation at the previous moment, the relative state at the current moment is predicted by combining the measurement equation and the process equation through equations (8) and (9).
[0205] Module M2.3: Constructing the filter model includes:
[0206] State one-step prediction equation:
[0207]
[0208] State estimation equation:
[0209]
[0210] Filter gain:
[0211]
[0212] One-step forecast mean square error:
[0213]
[0214] Estimated mean square error
[0215] P k =(IK k H k ) k / k-1 (14)
[0216] in, represents the one-step state estimation, Φ k,k-1 represents the state transfer matrix; represents the state estimation at time k-1; represents the current state estimate at time k, K k represents the gain matrix, z k represents the quantity measurement at time k, I represents the unit matrix, P k represents the estimated mean square error, P k / k-1 represents the one-step prediction mean square error, H k represents the measurement matrix, Γ k-1 represents the process noise covariance matrix, R k represents the covariance matrix of the measurement noise, Q k-1 represents the covariance matrix of the process noise;
[0217] Module M2.4: Relative posture feedback:
[0218] At time k, the relative pose estimated by the filtering model is fed back to the initial pose estimation module to assist the relative pose estimation at time k+1. At time k+1, the relative pose at time k+1 is predicted by the trained initial pose estimation model and sent to the filtering model as a measurement. The modules M1.1-M1.3 and M2.1-M2.4 are repeatedly triggered to achieve relative state tracking.
[0219] The embodiments of the present invention provide a method, system, device and medium for relative state tracking of non-cooperative targets. In view of the high real-time requirements of space tasks, a separable convolutional network is designed as a reference network to design a relative pose prediction model, which greatly reduces network parameters and improves inference efficiency. The target size shows multi-scale changes with the relative position of the non-cooperative target and the camera, and a dual attention mechanism is proposed to improve the relative pose estimation accuracy. In view of the poor robustness of traditional estimation methods based on feature points and models, the superior feature expression ability of deep learning networks is utilized, combined with the relative kinematics and dynamics models of non-cooperative targets, an integrated relative state tracking filtering framework is designed to improve filtering robustness and estimation accuracy.
[0220] Those skilled in the art know that, in addition to realizing the system and its various devices, modules, and units provided by the present invention in a purely computer-readable program code, it is entirely possible to realize the same functions in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system and its various devices, modules, and units provided by the present invention can be considered as a hardware component, and the devices, modules, and units included therein for realizing various functions can also be regarded as structures within the hardware component; the devices, modules, and units for realizing various functions can also be regarded as both software modules for realizing the method and structures within the hardware component.
[0221] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. In the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.
Claims
1. A non-cooperative target relative state tracking method, characterized in that: include: Initial pose estimation step: collecting image data, preprocessing the image data, and constructing and training an initial pose estimation model; Relative state estimation step: construct a filtering model, which includes the system state equation and measurement equation. The relative pose estimated by the filtering model is fed back to the initial pose estimation step. The relative pose is predicted by the trained initial pose estimation model and sent to the filtering model to finally achieve relative state tracking.
2. The non-cooperative target relative state tracking method according to claim 1, characterized in that: The initial pose estimation step comprises: Step S1.1: Use a monocular camera to capture images with an image size of M×M; Step S1.2: preprocessing the image, including denoising, normalization and cropping operations, wherein the image of size M×P is reduced to 256×256 in order to adapt to the input of different networks; Step S1.3: Establishing an initial pose estimation model, wherein the initial pose estimation model includes: a feature extraction network and a prediction network; The 3×M×M image is input into the feature extraction network, and the convolution, pooling and downsampling operations are implemented through the K-layer network to obtain an N×P×P feature map, which is then divided into P×P N-dimensional feature vectors, and P×P prediction outputs are obtained after passing through the prediction network; The N-dimensional feature vector is input into the prediction network, which consists of multiple layers of fully connected layers, including target position prediction, whether the target is included, and pose regression.
3. The non-cooperative target relative state tracking method according to claim 2, characterized in that: Adding a channel attention module and a spatial attention mechanism module after the convolution layer of the initial pose estimation model to assign different weights to different channel feature maps; Among them, the input 3×M×M image passes through the feature extraction network to obtain N×P×P feature maps. The N feature maps capture target features from different scales. The channel attention mechanism is used to assign learnable weights to the N P×P feature maps to capture effective information. For the spatial attention mechanism, learnable weights are assigned to the P×P pixel values on the feature map to capture effective information at different spatial locations.
4. The non-cooperative target relative state tracking method according to claim 1, characterized in that: The relative state estimation step includes constructing the system state equation: The system state equation is expressed as: The measurement equation is expressed as: z=h(x)+v(t) (2) Where z represents the measured quantity, x represents the state quantity; f(x) and h(x) represent the process equation and the measurement equation respectively; w(t) and v(t) represent the process noise and the measurement noise; The kinematic equation of the relative motion attitude is as follows: Among them, q represents quaternion, Q(q) represents quaternion matrix, R(q) represents rotation matrix, represents the relative angular velocity relative to the host spacecraft; When the spacecraft is not disturbed by external forces, that is, N L =N F = 0, the dynamic model of relative angular velocity is further simplified: Among them, the matrix I F represents the inertia tensor matrix of the non-cooperative target, I F I L represents the moment of inertia matrix of the target spacecraft; N L and N F They represent the external forces on the target spacecraft and the non-cooperative target respectively; represents the angular velocity of the target spacecraft; represents the angular velocity of the non-cooperative target relative to the target spacecraft; When the matrix I F When it is a diagonal matrix, there is an unobservable estimator. Assume that the inertia tensor matrix of the non-cooperative space target is not a diagonal matrix, but a symmetric matrix I F , further, rewrite I F For the matrix The ratio of the inertia tensor is as follows: Therefore, formula (4) is further rewritten as formula (6) as follows: The system status is as follows: Since ρ s Represents the coordinates of the feature point in the target coordinate system, and the spatial target object is a rigid body, so is the zero vector, and the moment of inertia ratio Therefore is also the zero vector; In the relative kinematics equation of the feature point, the coordinates of the feature point must be solved in the camera coordinate system, so ρ c =Rρ s =[x c ,y c ,z c ] T Indicates that ρ s Transformed to the camera coordinate system, ρ c represents the vector of the feature point in the camera coordinate system, R represents the rotation matrix, x c ,y c 、z c Represents ρ c The components of , that is, the components in the x, y, and z directions; the system equation used is described by equation (8): Among them, x0, y0, and z0 represent the position components of the origin of the target coordinate system in the camera coordinate system respectively; They represent the velocity components of the origin of the target coordinate system in the camera coordinate system respectively; represents the first derivative of the true anomaly; represents the second derivative of the true anomaly; μ represents the earth's gravitational constant; r L The magnitude of the position vector of the main spacecraft in the Earth coordinate system; ω x ,ω y ,ω z are the angular velocities of the non-cooperative target relative to the target spacecraft. The weight of They represent angular acceleration, that is, The weight; 0 3×1 Represents a zero vector of length 3; 0 5×1 represents a zero vector of length 5; and are equivalent, indicating The component of , that is, the velocity component; They represent the acceleration components, that is, The amount.
5. The non-cooperative target relative state tracking method according to claim 4, characterized in that: The construction of the measurement equation in the relative state estimation step includes: The measurement equation is a linear equation as shown in equation (9): Represents the unit matrix. According to the state estimation at the previous moment, the relative state at the current moment is predicted by combining the measurement equation and the process equation through equations (8) and (9).
6. The non-cooperative target relative state tracking method according to claim 5, characterized in that: The relative state estimation step includes constructing a filtering model: State one-step prediction equation: State estimation equation: Filter gain: One-step forecast mean square error: Estimated mean square error P k =(I-K k H k )P k / k-1 (14) in, represents the one-step state estimation, Φ k,k-1 represents the state transfer matrix; represents the state estimation at time k-1; represents the current state estimate at time k, K k represents the gain matrix, z k represents the quantity measurement at time k, I represents the unit matrix, P k represents the estimated mean square error, P k / k-1 represents the one-step prediction mean square error, H k represents the measurement matrix, Γ k-1 represents the process noise covariance matrix, R k represents the covariance matrix of the measurement noise, Q k-1 represents the covariance matrix of the process noise; The relative posture feedback in the relative state estimation step includes: At time k, the relative pose estimated by the filtering model is fed back to the initial pose estimation module to assist the relative pose estimation at time k+1. At time k+1, the relative pose at time k+1 is predicted by the trained initial pose estimation model and sent to the filtering model as a quantitative measurement. Repeat the initial pose estimation step and the relative state estimation step to achieve relative state tracking.
7. A non-cooperative target relative state tracking system, characterized in that: include: Initial pose estimation module: collects image data, preprocesses the image data, and constructs and trains an initial pose estimation model; Relative state estimation module: Build a filter model, which includes the system state equation and measurement equation. The relative pose estimated by the filter model is fed back to the initial pose estimation module. The relative pose is predicted by the trained initial pose estimation model and sent to the filter model to finally achieve relative state tracking.
8. The non-cooperative target relative state tracking system according to claim 7, characterized in that: The initial pose estimation module comprises: Module M1.1: uses a monocular camera to collect images with an image size of M×M; Module M1.2: preprocessing the image, including denoising, normalization and cropping operations, wherein the image of size M×M is reduced to 256×256 in order to adapt to the input of different networks; Module M1.3: Establishing an initial pose estimation model, wherein the initial pose estimation model includes: a feature extraction network and a prediction network; The 3×M×M image is input into the feature extraction network, and the convolution, pooling and downsampling operations are implemented through the K-layer network to obtain an N×P×P feature map, which is then divided into P×P N-dimensional feature vectors, and P×P prediction outputs are obtained after passing through the prediction network; The N-dimensional feature vector is input into the prediction network, which consists of multiple fully connected layers, including target position prediction, whether the target is included, and pose regression; Adding a channel attention module and a spatial attention mechanism module after the convolution layer of the initial pose estimation model to assign different weights to different channel feature maps; Among them, the input 3×M×M image passes through the feature extraction network to obtain N×P×P feature maps. The N feature maps capture target features from different scales. The channel attention mechanism is used to assign learnable weights to the N P×P feature maps to capture effective information. For the spatial attention mechanism, learnable weights are assigned to the P×P pixel values on the feature map to capture effective information at different spatial locations; Constructing the system state equation in the relative state estimation module includes: The system state equation is expressed as: The measurement equation is expressed as: z=h(x)+v(t) (2) Where z represents the measured quantity, x represents the state quantity; f(x) and h(x) represent the process equation and the measurement equation respectively; w(t) and v(t) represent the process noise and the measurement noise; The kinematic equation of the relative motion attitude is as follows: Among them, q represents quaternion, Q(q) represents quaternion matrix, R(q) represents rotation matrix, represents the relative angular velocity relative to the host spacecraft; When the spacecraft is not disturbed by external forces, that is, N L =N F = 0, the dynamic model of relative angular velocity is further simplified: Among them, the matrix I F represents the inertia tensor matrix of the non-cooperative target, I F I L represents the moment of inertia matrix of the target spacecraft; N L and N F They represent the external forces on the target spacecraft and the non-cooperative target respectively; represents the angular velocity of the target spacecraft; represents the angular velocity of the non-cooperative target relative to the target spacecraft; When the matrix I F When it is a diagonal matrix, there is an unobservable estimator. Assume that the inertia tensor matrix of the non-cooperative space target is not a diagonal matrix, but a symmetric matrix I F , further, rewrite I F For the matrix The ratio of the inertia tensor is as follows: Therefore, formula (4) is further rewritten as formula (6) as follows: The system status is as follows: Since ρ s Represents the coordinates of the feature point in the target coordinate system, and the spatial target object is a rigid body, so is the zero vector, and the moment of inertia ratio Therefore is also the zero vector; In the relative kinematics equation of the feature point, the coordinates of the feature point must be solved in the camera coordinate system, so ρ c =Rρ s =[x c ,y c ,z c ] T Indicates that ρ s Transformed to the camera coordinate system, ρ c represents the vector of the feature point in the camera coordinate system, R represents the rotation matrix, x c ,y c 、z c Represents ρ c The components of , that is, the components in the x, y, and z directions; the system equation used is described by equation (8): Among them, x0, y0, and z0 represent the position components of the origin of the target coordinate system in the camera coordinate system respectively; They represent the velocity components of the origin of the target coordinate system in the camera coordinate system respectively; represents the first derivative of the true anomaly; represents the second derivative of the true anomaly; μ represents the earth's gravitational constant; r L The magnitude of the position vector of the main spacecraft in the Earth coordinate system; ω x ,ω y ,ω z are the angular velocities of the non-cooperative target relative to the target spacecraft. The weight of They represent angular acceleration, that is, The weight; 0 3×1 Represents a zero vector of length 3; 0 5×1 represents a zero vector of length 5; and are equivalent, indicating The component of , that is, the velocity component; They represent the acceleration components, that is, The weight of The measurement equation constructed in the relative state estimation module includes: The measurement equation is a linear equation as shown in equation (9): represents the unit matrix. According to the state estimation at the previous moment, the relative state at the current moment is predicted by combining the measurement equation and the process equation through equations (8) and (9); Constructing a filtering model in the relative state estimation module includes: State one-step prediction equation: State estimation equation: Filter gain: One-step forecast mean square error: Estimated mean square error P k =(I-K k H k )P k / k-1 (14) in, represents the one-step state estimation, Φ k,k-1 represents the state transfer matrix; represents the state estimation at time k-1; represents the current state estimate at time k, K k represents the gain matrix, z k represents the quantity measurement at time k, I represents the unit matrix, P k represents the estimated mean square error, P k / k-1 represents the one-step prediction mean square error, H k represents the measurement matrix, Γ k-1 represents the process noise covariance matrix, R k represents the covariance matrix of the measurement noise, Q k-1 represents the covariance matrix of the process noise; The relative posture feedback in the relative state estimation module includes: At time k, the relative pose estimated by the filtering model is fed back to the initial pose estimation module to assist the relative pose estimation at time k+1. At time k+1, the relative pose at time k+1 is predicted by the trained initial pose estimation model and sent to the filtering model as a quantitative measurement. Repeatedly trigger the initial pose estimation module and the relative state estimation module to achieve relative state tracking.
9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps in the non-cooperative target relative state tracking method according to any one of claims 1 to 6 are implemented.
10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the computer program is executed by a processor, the steps in the non-cooperative target relative state tracking method according to any one of claims 1 to 6 are implemented.
Citation Information
Patent Citations
Pose depth prediction method, visual odometer method, device, equipment and medium
CN111709984A
Equipment pose determination method and device, storage medium and electronic device
CN113096181A
Multi-scene monocular camera pose regression method based on graph convolutional network
CN113724325A
Monocular endoscope unsupervised depth and motion estimation method and device
CN114399527A
Visual positioning method based on point cloud map
CN114723920A
