A non-cooperative satellite attitude measurement method based on image sequence
By combining a two-stage network architecture with soft classification and an LSTM network, and utilizing temporal and physical information from sequential images, the problem of insufficient robustness and accuracy in non-cooperative satellite attitude measurement is solved, achieving higher-precision attitude estimation.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- NANJING UNIV OF SCI & TECH
- Filing Date
- 2026-01-05
- Publication Date
- 2026-06-02
AI Technical Summary
Existing technologies lack robustness and accuracy in non-cooperative satellite attitude measurement, especially under conditions of weak texture, uneven lighting, and complex spatial backgrounds. Two-stage methods rely on the target's 3D model and cannot be applied to unknown targets, while end-to-end methods lack geometric model constraints and have unstable attitude predictions. Existing methods do not effectively utilize the temporal information in sequential images.
A two-stage approach is adopted. First, a coarse estimation is performed using a pose measurement network based on soft classification. Then, a pose optimization network based on LSTM is used for refinement. By combining convolutional neural networks and long short-term memory networks to utilize the temporal information in the sequence images, a physical information loss function is introduced to guide pose optimization.
It improves the accuracy and stability of attitude estimation, significantly reduces the average attitude estimation error, and enhances measurement accuracy by utilizing the temporal and physical information in the sequence frames.
Smart Images
Figure CN122130102A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of non-cooperative satellite attitude measurement, and in particular, a non-cooperative satellite attitude measurement method based on sequence images. Background Technology
[0002] In space combat and on-orbit servicing missions, friendly satellites need to perform tasks such as circling, approaching, and rendezvous with non-cooperative target satellites. Accurate and real-time measurement of the target's attitude is a crucial technology. However, traditional vision-based attitude measurement methods rely on manually designed features, which are insufficient in terms of reliability and generalization when dealing with targets with weak textures, uneven lighting conditions, and complex space backgrounds.
[0003] With the outstanding performance of deep learning in many computer vision tasks, deep neural networks are increasingly being applied to non-cooperative satellite pose estimation. Deep learning-based pose estimation methods can be broadly categorized into two types: two-stage methods and end-to-end regression methods. Existing two-stage methods first use an object detection network to locate and extract satellite targets from images, then use a keypoint detection network to extract 2D keypoints from the images, and finally use the PnP algorithm to establish the relationship between the 2D keypoints and the target's 3D keypoints to calculate the target pose. However, because the PnP process is non-differentiable, it cannot be integrated into the network structure, resulting in a disconnect between feature extraction and pose calculation, reducing the robustness and accuracy of pose measurement. Furthermore, both-stage methods rely on accurate 3D target models, making them unsuitable for non-cooperative satellite pose estimation where the target model is unknown. Existing end-to-end methods do not rely on accurate 3D target models, using a deep learning model to directly regress or obtain the target pose through soft classification. However, lacking geometric model constraints, they are prone to pose prediction drift and instability in complex ambient lighting and symmetrical target conditions.
[0004] Patent document CN117351073A discloses a dual-channel transformer satellite attitude estimation algorithm based on a dimensionality discriminator. This method uses a two-branch approach to directly regress the target attitude and position, but the direct attitude regression has low accuracy. Patent document CN119085666A discloses a spatial non-cooperative target pose estimation method using a multi-scale dense structure neural network. This method uses a soft classification method to reduce the difficulty of fitting the target attitude to the network model and improve the attitude measurement accuracy, but it does not effectively utilize the temporal information and target satellite motion patterns in the sequence images. Patent document CN119068054A discloses a monocular image sequence satellite attitude estimation method. It uses a pre-trained semantic segmentation model to extract target solar panel features, then performs feature matching between adjacent frames to obtain the attitude change between adjacent frames, and combines the single-frame satellite attitude to obtain a series of satellite attitudes in the sequence images. However, this method suffers from cumulative error; the longer the sequence length, the lower the attitude estimation accuracy. Patent document CN119515973A discloses a space target adaptive pose estimation system based on deep learning, including a space non-cooperative target dataset unit, an image preprocessing unit, a deep learning network model unit, a model cross-domain adaptive unit, and a non-cooperative target high-precision pose estimation unit. However, the pose estimation algorithm used in this method relies on an accurate 3D target model. Patent document CN120576724A discloses an end-to-end satellite pose measurement method based on deep learning, using a differentiable EPro PnP module to replace the PnP algorithm in the two stages, realizing integrated network training for feature extraction and geometric pose calculation. However, it only uses single-frame information, ignoring the time-series information contained in the continuous motion of the target satellite. Patent document CN114266824A discloses a non-cooperative target relative pose measurement method and system based on deep learning. This method uses simulated satellite images and 6D pose labels to construct a dataset, obtains cropped images through target detection, regresses relative distance based on ResNet50 and estimates pose, and realizes pose measurement at long distance and close range. However, this method mainly relies on single-frame images and the pose measurement at close range directly regresses quaternion pose, resulting in low accuracy. Summary of the Invention
[0005] The purpose of this invention is to propose a non-cooperative satellite attitude measurement method based on sequential images. This method combines convolutional neural networks and long short-term memory neural networks, and utilizes the temporal information in the sequential images to improve the accuracy of attitude measurement.
[0006] The technical solution to achieve the purpose of this invention is as follows: the non-cooperative satellite attitude measurement is divided into two stages. In the first stage, a soft classification-based attitude measurement network is used to coarsely estimate the relative attitude of the target. In the second stage, an LSTM-based attitude optimization network is used to refine the coarse estimation results of the first stage, thereby improving the attitude estimation accuracy.
[0007] The specific implementation steps are as follows:
[0008] Step (1): Create a tumbling satellite attitude measurement dataset. First, import the 3D model of the non-cooperative target satellite into the Gazebo simulation environment; then, write a motion control algorithm in ROS to make the target perform nutation motion; finally, use a camera to acquire target images and use the ROS broadcast mechanism to synchronously acquire the target's relative attitude and angular velocity labels, where the target's relative attitude is represented by quaternions.
[0009] Step (2): Discretize the attitude quaternion space using a uniform random rotation algorithm, and then encode the target relative attitude quaternion labels using a soft classification method to generate the ground truth value of the weight distribution function for each quaternion label. Then, construct a soft classification attitude measurement network. The backbone network uses EfficientNetv2-m, and a soft classification head is added after the backbone network to estimate the weight distribution function of the target relative quaternion attitude. Construct a cross-entropy loss function, perform data augmentation and data preprocessing on the satellite image data in the dataset from step (1), use it as the network input data, and use the ground truth value of the quaternion weight distribution function generated in step (2) as the label to train the network to obtain a trained attitude measurement network.
[0010] Step (3): Input the satellite image data in the dataset of step (1) into the trained attitude measurement network frame by frame. The network outputs the estimated value of the quaternion weight distribution function of the image frame. Then, use the quaternion weighted average algorithm to decode the weight distribution function to obtain the relative quaternion attitude estimate of the target in the frame.
[0011] Step (4): Construct an LSTM-based pose optimization network with continuous input. The target relative pose is displayed in frames, and the output is continuous. The target's relative pose quaternion in the last frame of the frame. Then, a composite loss function is constructed. This includes weighted rotation loss. Weighted sequence rotation-angular velocity loss Weighted physical information loss Then, the target relative attitude quaternion estimated in step (3) is used as training data, and the target relative attitude quaternion and the ground truth value of angular velocity obtained in step (1) are used as training labels. The frames are stitched together into a sequence of data, and the pose optimization network is trained using a sliding window.
[0012] Step (5): Continuous The target satellite image of frame is sequentially input into the trained attitude measurement network and attitude optimization network to obtain the optimized first frame. Frame target relative attitude quaternion.
[0013] Compared with existing technologies, the significant advantages of this invention are: 1. It solves the problem of unstable attitude estimation by utilizing the temporal information in the sequence frames, thereby improving the accuracy of attitude estimation. 2. Based on the rigid body attitude kinematic equations, it introduces a physical information loss function to guide the training of the attitude optimization network, significantly reducing the average attitude estimation error. Attached Figure Description
[0014] Figure 1 This is a flowchart of a non-cooperative target space attitude measurement method based on sequential images according to the present invention.
[0015] Figure 2 is a schematic diagram of the dataset images generated in an embodiment of the present invention, wherein Figures a, b, and c are three consecutive frames of dataset images.
[0016] Figure 3 This is a schematic diagram of the attitude measurement network structure in this invention.
[0017] Figure 4 This is a schematic diagram of the attitude optimization network structure in this invention.
[0018] Figure 5 This is a schematic diagram of the overall network structure in this invention.
[0019] Figure 6 This is a diagram showing the attitude measurement results of an embodiment of the present invention. Detailed Implementation
[0020] The present invention will now be further described with reference to the accompanying drawings.
[0021] like Figure 1 As shown, this invention is a non-cooperative satellite attitude measurement method based on sequential images, comprising the following steps:
[0022] Step (1): Create a tumbling satellite attitude measurement dataset. This step specifically includes:
[0023] Step 1.1: Construct a 3D model of the satellite in FBX format in 3DS Max. In this embodiment, the 3D model of the Chang'e 2 satellite is used. The target model is imported into the Gazebo simulation environment, the camera is placed and a virtual deep space environment is set. The target system state is modeled by combining the rigid body attitude dynamics equation and kinematic equation. According to the target system model, a motion control algorithm is written in ROS to drive the target to perform nutation motion within the camera's field of view and save the target image. At the same time, the coordinate relationship tree between the target system and the camera system is established using ROS broadcasting to obtain the target's relative attitude and angular velocity information. Each target image corresponds to a target relative attitude and an angular velocity label. Figure 2 is a schematic diagram of the dataset images generated in this embodiment. Among them, the angular velocity is the vector representation of the angular velocity of the target system relative to the camera system in the target system. The attitude of the target system relative to the camera system is represented by a quaternion, defined as follows: (1) For the quaternion vector part, For the scalar part of a quaternion, due to the non-uniqueness of quaternions, this embodiment stipulates that the scalar part of the quaternion is always positive, i.e. .
[0024] (1)
[0025] Step 1.2: Following the dataset creation method in Step 1.1, randomly initialize the initial state of the target system model, generating 180 sets of sequence data. Each set of sequence data contains 700 frames, and each frame contains one target satellite image, one target relative attitude label, and one target relative angular velocity label. Divide the generated sequence data into training, validation, and test sets in an 8:1:1 ratio. The generated target image sequence is shown in Figure 2 below.
[0026] Step (2): Construct and train a soft-classification pose measurement network. This step specifically includes:
[0027] Step 2.1: First, discretize the attitude quaternion space using a uniform random rotation algorithm to obtain a space with a quantity of Quaternion array Then, based on the idea of soft classification, the discretized spatial quaternion array is used. The true value of the weight distribution function is obtained by performing soft classification encoding on the target relative pose quaternion labels. The coding formula is as follows (2), where Represents the nearest target quaternion of A set of quaternions express The Middle A quaternion, express The Middle The weights of the quaternions, in this embodiment , .
[0028] (2)
[0029] Step 2.2: First, use the alumments library to perform data augmentation on the image data generated in step (1), including adding Gaussian noise, motion blur, ISO noise, and contrast enhancement. Then, use the OpenCV library to normalize the data-enhanced image and uniformly scale it to a normal size. The size is used as input data for the subsequent attitude measurement network.
[0030] Step 2.3: Construct a pose measurement network based on soft classification, which consists of a backbone network and a soft classification head. The network structure is as follows: Figure 3 As shown, its backbone network adopts the standard EfficientNetv2-m network, and the soft classification head consists of three fully connected layers, with the first two fully connected layers activated by the ReLU function. The specific structure of the attitude measurement network is as follows. Figure 3 As shown.
[0031] Step 2.4: Construct the cross-entropy loss function To guide the training of the pose measurement network, as shown in equation (3), where The true value of the weight distribution function obtained in step 2.1 is... Let be the weight distribution function for the soft classifier head estimation, where Batch size:
[0032] (3)
[0033] Step 2.5: Use the target image data processed in Step 2.3 as training data, the ground truth value of the weight distribution function generated in Step 2.2 as training labels, and use the Adam optimizer and cosine annealing learning rate decay strategy. Set the initial learning rate to 0.001 and the batch size to 32 to train the pose measurement network.
[0034] Step (3): Input the satellite image data from the dataset in step (1) frame by frame into the trained attitude measurement network. The network outputs the estimated value of the quaternion weight distribution function of the image frame. Then, the weight distribution function is decoded using a quaternion weighted average algorithm to obtain the target's relative quaternion pose estimate for that frame. The formula for calculating the quaternion weighted average algorithm is as follows (4):
[0035] (4)
[0036] Step (4): Construct and train an LSTM-based pose optimization network. This step specifically includes:
[0037] Step 4.1: Construct an LSTM-based pose optimization network, the network structure of which is as follows: Figure 4 As shown, the network input is continuous. The frame target relative attitude quaternion, the input sequence data first passes through two LSTM layers to obtain temporal features, and then the temporal features are input into the attitude angular velocity prediction head. The prediction head consists of two fully connected layers and a quaternion normalization layer. The fully connected layers are activated using the tanh activation function. The function of the quaternion normalization layer is to normalize the quaternions to satisfy the quaternion constraint relationship, as shown in the following equation (5), where The quaternion estimate of the target relative attitude obtained in step (3) is... The quaternion estimate of the target's relative attitude after normalization:
[0038] (5)
[0039] Predictor output continuous The target's relative attitude and angular velocity in the frame are used as inputs, but the network only outputs the first frame. Frame target relative attitude quaternion, the rest Frame data is used to assist network training, in this embodiment The specific structure of the pose optimization network is as follows: Figure 4 As shown.
[0040] Step 4.2: Construct a composite loss function Including weighted rotation loss Weighted sequence rotation-angular velocity loss Weighted physical information loss The composite loss function is defined as follows (6), in this embodiment , , :
[0041] (6)
[0042] The weighted rotation loss is given by equation (7), where The batch size. These represent the first and second digits of the batch. Frame data number Frame target relative pose estimation and ground truth:
[0043] (7)
[0044] The weighted sequence rotation-angular velocity loss is given by equation (8), where They represent the first and second elements in this batch, respectively. Frame data number Frame target relative pose estimate and ground truth, They represent the first and second elements in this batch, respectively. Frame data number Frame target angular velocity estimate and true value:
[0045] (8)
[0046] In this embodiment, the attitude kinematics equation is used as the kinematic constraint term to define the physical information loss. The attitude kinematics equation is as follows (9), where For the target relative attitude quaternion, The relative angular velocity of the target for Regarding time The derivative of for The opposition to the formation, For unit array:
[0047] (9)
[0048] The weighted physical information loss is given by equation (10), where For direct use and The calculated first Frame data number The quaternion derivative of a frame, The first one calculated using equation (9) above Frame data number The quaternion derivative of a frame, and The definition is given in equation (11) below. The time interval between two frames of image data, in this embodiment, is... :
[0049] (10)
[0050] (11)
[0051] Step 4.3: Use the target relative attitude quaternion estimated in step (3) as training data and the target relative attitude quaternion and angular velocity ground truth obtained in step (1) as training labels. The frames are concatenated into a sequence of data. The Adam optimizer and cosine annealing learning rate decay strategy are used in a sliding window to train the pose optimization network. The initial learning rate is set to 0.001 and the batch size is set to 256.
[0052] Step (5): Continuous The target satellite image of the frame is input into the attitude measurement network to obtain the estimated quaternion weight distribution function, and the target relative quaternion attitude sequence is decoded according to step (3). This sequence is then input into the attitude optimization network to obtain the optimized first frame. The relative pose quaternion of the frame target, and the overall network structure diagram are as follows: Figure 5 As shown.
[0053] To demonstrate the effectiveness of the method of the present invention and fully showcase its superiority in non-cooperative target attitude measurement tasks, the following experiment was conducted. The experimental hardware platform used was as follows: CPU: Intel(R) Core(TM) i9-14900HX; GPU: NVIDIA RTX 4080; Memory: 32.0 GB; Operating System: Windows 11; Development Tool: PyCharm.
[0054] Based on steps (1) to (4) above, a dataset is constructed and the attitude measurement network and attitude training network are trained. A set of sequence data with a total of 700 frames is selected from the test set, and each set of 32 frames constitutes an input data. Referring to step (5), the target relative attitude is obtained. In this embodiment, the following formula (12) is used to represent the evaluation index of attitude measurement accuracy.
[0055] (12)
[0056] Appendix Figure 6 The image shows the attitude measurement results. The red curve represents the coarse estimate of the target's relative attitude predicted by the attitude measurement network, and the blue curve represents the refined estimate obtained by using the attitude optimization network. Table 1 below shows the attitude measurement results:
[0057] Table 1 Attitude Measurement Results
[0058] In summary, the present invention achieves good measurement results in non-cooperative target attitude measurement tasks.
[0059] Finally, it should be noted that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A non-cooperative satellite attitude measurement method based on sequential images, characterized in that, Includes the following steps: Step (1): Create a tumbling satellite attitude measurement dataset; first, import the 3D model of the non-cooperative target satellite into the Gazebo simulation environment; then, write a motion control algorithm in ROS, combine the rigid body attitude dynamics equation and kinematic equation to model the target system state, use the target system model to perform state recursion, and drive the target to perform nutation motion within the camera's field of view; finally, use the camera to acquire target images, and use the ROS broadcast mechanism to synchronously acquire the target's relative attitude and angular velocity labels, where the target's relative attitude is represented by quaternions; Step (2): Discretize the attitude quaternion space using a uniform random rotation algorithm, and then encode the target relative attitude quaternion labels using a soft classification method to generate the true value of the weight distribution function for each quaternion label; Then, a pose measurement network based on soft classification is constructed. Its backbone network uses EfficientNetv2-m, and a soft classification head is added after the backbone network to predict the weight distribution function of the target relative to the quaternion pose. A cross-entropy loss function is constructed. The satellite image data in the dataset of step (1) is processed by data augmentation and data preprocessing and used as the network input data. The true value of the quaternion weight distribution function generated in step (2) is used as the label to train the network to obtain a trained pose measurement network. Step (3): Input the satellite image data in the dataset from step (1) into the trained attitude measurement network. The network outputs the estimated quaternion weight distribution function. The estimation result is decoded by the quaternion weighted average algorithm to obtain the estimated relative attitude quaternion. Step (4): Construct an LSTM-based pose optimization network with continuous input. The frame target relative attitude quaternion, the output is continuous The target relative pose quaternion in the last frame; then construct the composite loss function. This includes weighted rotation loss. Weighted sequence rotation-angular velocity loss Physical information loss Then, the target relative attitude quaternion predicted in step (3) is used as training data, and the target relative attitude quaternion and the ground truth value of angular velocity obtained in step (1) are used as training labels. The frames are stitched together into a sequence of data, and the pose optimization network is trained using a sliding window. Step (5): Continuous The target satellite image of frame is sequentially input into the trained attitude measurement network and attitude optimization network to obtain the optimized first frame. Frame target relative attitude quaternion.
2. The non-cooperative satellite attitude measurement method based on sequence images as described in claim 1, characterized in that, In step (2), the relative pose quaternion space is discretized using a uniform random rotation algorithm, and then the target relative pose quaternion labels are encoded using a soft classification method to generate the true value of the weight distribution function for each quaternion label. The specific implementation is as follows: First, the relative attitude quaternion space is discretized using a uniform random rotation algorithm, resulting in a space with a quantity of... Quaternion array Then, based on the idea of soft classification, the discretized spatial quaternion array is used. The true value of the weight distribution function is obtained by performing soft classification encoding on the pose quaternion. The coding formula is as follows (1), where Represents the nearest target quaternion of A set of quaternions A spatial quaternion array The Middle A quaternion element of each pose The true value of the weight distribution function No. The weights of each element; (1)。 3. The non-cooperative satellite attitude measurement method based on sequence images as described in claim 2, characterized in that, In step (3), the quaternion weighted average algorithm is used to decode the quaternion weight distribution function to obtain the quaternion pose. The specific implementation process is as follows: Based on the quaternion weight distribution function output by the attitude prediction network The estimated value is then decoded using a quaternion weighted average algorithm to obtain the quaternion pose. The quaternion weighted average algorithm is calculated using the following formula (2), where... The representative unit is the sphere. Represents a spatial quaternion array The Middle A quaternion element of each pose: (2)。 4. The non-cooperative satellite attitude measurement method based on sequence images as described in claim 3, characterized in that, In step (4), an LSTM-based pose optimization network is constructed, specifically as follows: (1) Construct an LSTM-based pose optimization network with continuous input. The relative pose of the target frame is determined by the input sequence data, which first passes through two LSTM layers to obtain temporal features. These temporal features are then input into the attitude and angular velocity prediction head, which consists of two fully connected layers and a quaternion normalization layer. The fully connected layers are activated using the tanh activation function, and the quaternion normalization layer normalizes the quaternions to satisfy quaternion constraints. The prediction head outputs a continuous... The target's relative attitude and angular velocity in the frame are used as inputs, but the network only outputs the first frame. Frame target relative pose, the rest Frame data is used to assist in network training; (2) Construct a composite loss function Including weighted rotation loss Weighted sequence rotation-angular velocity loss Weighted physical information loss The composite loss function is defined as follows (4): (4) The weighted rotation loss is given by equation (5), where The batch size. These represent the first and second digits of the batch. Frame data number Frame target relative pose prediction and ground truth: (5) The weighted sequence rotation-angular velocity loss is given by equation (6), where They represent the first and second elements in this batch, respectively. Frame data number Frame target relative pose prediction and ground truth. They represent the first and second elements in this batch, respectively. Frame data number Frame target angular velocity prediction and true value: (6) The physical information loss is defined using the attitude kinematics equations as kinematic constraints. The attitude kinematics equations are as follows (7), where for The opposing formation: (7) The weighted physical information loss is as follows (8), where and The definition is given in equation (9) below. The time interval between two frames of image data: (8) (9) (3) Use the target relative attitude predicted in step (3) as training data and the ground truth values of the target relative attitude and angular velocity obtained in step (1) as training labels. The frames are concatenated into a sequence of data, and the pose optimization network is trained using the Adam optimizer and a cosine annealing learning rate decay strategy in a sliding window manner.
5. The non-cooperative satellite attitude measurement method based on sequence images as described in claim 4, characterized in that, In step (5), the continuous Frame target satellite images are input into a trained attitude measurement network to obtain continuous The frame quaternion weight distribution function estimate is used to decode the continuous [frame] using step (3). The frame quaternion weight distribution function estimation value yields the target continuity The coarse estimation results of the frame relative pose quaternions will be continuous The coarse frame estimation results are concatenated into a sequence of relative pose quaternions, which are then input into a trained pose optimization network to obtain the refined [frame number]. Frame target relative attitude quaternion.