Target pose estimation method for space missions
By assisting the network with shape prior information and using the encoding network and variational autoencoder structure to train the target pose estimation method for space missions, the pose estimation problem under complex lighting and stray light backgrounds is solved, and high-precision and robust pose measurement is achieved.
Patent Information
- Application Number
- CN202310191198.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-03-02
- Publication Date
- 2025-09-23
- Estimated Expiration
- 2043-03-02
AI Technical Summary
Existing technologies have poor robustness and low accuracy in target pose estimation in space missions under complex lighting environments and stray light backgrounds, making it difficult to meet the needs of on-orbit service missions for non-cooperative targets.
The shape prior information is used to assist the network, including the encoding network, sampling module and decoding network. By generating a shape prior image of the target image and using the variational autoencoder structure and loss function to train the network, the accuracy and robustness of pose estimation are improved.
It improves the accuracy and robustness of target pose estimation in space missions, is suitable for pose measurement tasks of small and medium-sized spacecraft, and can accurately estimate target pose in complex environments.
Smart Images
Figure CN116109703B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a target pose estimation method for aerospace missions, and belongs to the field of aerospace technology. Background Art
[0002] As countries around the world deepen their exploration of space resources, the scope of space missions continues to expand, and space mission applications are becoming more diverse. New missions such as on-orbit servicing, formation flying, and deep space exploration are being carried out on a large scale. These space missions involve non-cooperative target detection and identification, flyby approaches, rendezvous and docking, and other processes. Because ground tracking and control stations are limited by factors such as attitude and orbit determination accuracy and communication response speed, traditional satellite-to-ground loop control methods are not conducive to the completion of spacecraft autonomous missions. To ensure the successful completion of these new space missions, there is an urgent need to research methods for autonomous detection and tracking of space targets, trajectory planning, and autonomous control by spacecraft. Target pose estimation technology is one of the key technologies involved.
[0003] Target pose estimation technology for space missions faces challenges such as complex space environments and measurement targets without cooperative information. The space environment is characterized by a large number of stars and artificial celestial bodies, which form a stray light background. The reflection of the surface coating of the target being measured and the drastic changes in illumination caused by changes in the viewing angle make the target appear as a bright spot in the image and the illumination is complex and changeable. Due to the characteristics of visual sensors such as small size, light weight, low power consumption, and rich sensor information, vision-based measurement technology has a great advantage in the task of short-range high-precision space target pose estimation. With the substantial increase in processor computing power, the continuous development of image processing technology, and the rapid development of deep learning algorithms, space missions such as on-orbit assembly of spacecraft, repair of faulty or failed satellites, and space debris cleanup have an urgent need for space target pose measurement. Space target pose estimation technology based on computer vision has gradually become a research hotspot.
[0004] Since the advent of the AlexNet network in 2012, deep learning methods have demonstrated outstanding performance in a wide range of computer vision applications. Neural network-based approaches have also been used to address pose estimation for space missions. However, neural networks often focus on local image features, making them inadequate for spacecraft pose estimation in complex and changing space environments. These pose estimations often result in significant errors, making them difficult to meet the requirements of on-orbit servicing missions for non-cooperative targets. Summary of the Invention
[0005] In view of the current problem that pose estimation for aerospace missions has poor robustness and low accuracy due to the influence of complex lighting environment and stray light background, the present invention provides a target pose estimation method for aerospace missions.
[0006] A method for estimating a target pose for a space mission according to the present invention comprises:
[0007] S1. Generate a dataset of target pose estimation images for space missions, which includes target images and corresponding pose vector truth values;
[0008] S2. Preprocess the target image in the data set to obtain a shape prior image of the star area in the target image;
[0009] S3, using the preprocessed data set to train the shape prior information auxiliary network;
[0010] The shape prior information auxiliary network includes an encoding network, a sampling module and a decoding network;
[0011] The shape prior image of the star region in the target image is input to the encoding network, the encoding network outputs the position mean μ of the six degrees of freedom and the standard deviation σ of the six degrees of freedom, the position mean μ and the standard deviation σ are input to the sampling module, the sampling module processes to obtain a latent vector, the latent vector is input to the decoding network, and the decoding network generates the target image;
[0012] S4. Use the trained encoding network to estimate the pose of the target image.
[0013] Preferably, in S3, the sampling module processes and obtains the potential vector:
[0014]
[0015] Among them, z is the latent variable; is an auxiliary noise variable.
[0016] Preferably, in S3, using the preprocessed data set to train the shape prior information auxiliary network includes:
[0017] The data set is input into the shape prior information auxiliary network, and the loss function is used to train the parameters of the shape prior information auxiliary network;
[0018] The loss function is:
[0019]
[0020] Among them, γ1 and γ2 are hyperparameters for balancing the losses. is the KL-divergence loss, The cross entropy loss between the shape prior image and the target image generated by the decoding network, It is the mean absolute error loss between the true value of the pose vector and the pose estimation result output by the encoding network.
[0021] Preferably, the KL-divergence loss is:
[0022]
[0023] Among them, n b is the number of training samples in the dataset, and the KL divergence of the i-th training sample between the two normal distributions represents the normal distribution of the encoding network output.
[0024] As a preference, cross entropy loss for:
[0025]
[0026] Among them, n b is the number of training samples in the dataset, is the target image generated by the decoding network, and y is the shape prior image;
[0027] As a preference, the mean absolute error loss
[0028]
[0029] Among them, n b is the number of training samples in the dataset, the pose estimation result output by the encoding network is normally distributed, μ (i) represents the position mean of the i-th training sample, p (i) is the true value of the pose vector of the i-th training sample.
[0030] Preferably, the data set includes a training set and a test set, and S1 includes:
[0031] S11, constructing multiple virtual environments with different lighting conditions and different variance noises and defining coordinate systems, and importing the target 3D model into the virtual environments;
[0032] S12. Design the target motion range and motion path according to the actual requirements of the space mission;
[0033] S13, generating a simulated training set based on the virtual environment and target three-dimensional model of S11 and the target motion range and motion path designed in S12, including a target image and a corresponding true value of a pose vector;
[0034] S14. Based on the target motion range and motion path designed by the experimental system according to S12, a target experimental data set is obtained, including the target image and the corresponding pose vector true value, as a test set;
[0035] In S3, the shape prior information auxiliary network is trained using the preprocessed training set, and is verified using the preprocessed test set.
[0036] Preferably, S2 includes:
[0037] S21, read in the target pose estimation image dataset;
[0038] S22, reconstructing the target image to a set size;
[0039] S23, gray-scale the target image reconstructed in S22, and filter and remove noise;
[0040] S24, using Hough transform to extract straight line features in the filtered and denoised target image, obtain the star region, and generate an image mask;
[0041] S25. Generate an image mask to obtain a shape prior image.
[0042] The present invention has the following beneficial effects: It is highly universal and applicable to most pose measurement tasks targeting small and medium-sized spacecraft. It proposes a shape prior information network training framework that can adaptively modify the shape prior based on target characteristics. It also achieves high measurement accuracy, utilizing a variational autoencoder network structure and shape prior information to assist in training. Compared to existing deep learning methods, this method achieves improved measurement accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] Figure 1 It is a schematic diagram of the principle of the present invention;
[0044] Figure 2 This is the coordinate system definition of the present invention. DETAILED DESCRIPTION
[0045] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.
[0046] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.
[0047] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but they are not intended to limit the present invention.
[0048] The target pose estimation method for aerospace missions of this embodiment includes:
[0049] Step 1: Generate a dataset of target pose estimation images for space missions, which includes target images and corresponding pose vector truth values;
[0050] Images can be generated using physical engine simulation software and physical experiment systems;
[0051] Step 2: Preprocess the target image in the data set to obtain a shape prior image of the star area in the target image;
[0052] Step 3: Use the preprocessed data set to train the shape prior information auxiliary network;
[0053] Reference Figure 1 , the shape prior information auxiliary network includes an encoding network, a sampling module and a decoding network;
[0054] The shape prior image of the star region in the target image is input to the encoding network, which outputs the position mean μ and the standard deviation σ of the six degrees of freedom. The position mean and standard deviation are input to the sampling module, which processes them to obtain a latent vector. The latent vector is input to the decoding network, which generates the target image.
[0055] Step 4: Use the trained encoding network to estimate the pose of the target image.
[0056] This embodiment is designed to solve the problem of poor robustness and low accuracy of pose estimation for space missions currently affected by complex lighting environments and stray light backgrounds. This embodiment first uses a coding network to extract image information, performs efficient coding, and obtains the pose estimation result from the coding. The obtained feature vector is then resampled, and finally the resampled feature vector is restored by the decoding network to generate the image. Taking into account the measurement noise, this embodiment regards the pose vector as a latent variable, and assumes that the variable satisfies the conditional probability distribution, and uses a variational autoencoder structure based on variational inference to fit the true distribution to obtain the pose estimation result. In addition, this embodiment uses a shape prior image as the target image for generating the image, with the aim of introducing shape prior information and guiding the network to learn shape features. The network training method using shape prior information proposed in this embodiment can improve the effectiveness of model feature extraction and the accuracy of pose estimation.
[0057] In step 1 of this embodiment, the data set includes a training set and a test set, and step 1 includes:
[0058] Step 11: Build a virtual environment and define a coordinate system, import the target 3D model into the virtual environment, and simulate the space background environment, including but not limited to adding different lighting conditions and noise with different variances to obtain multiple virtual environments;
[0059] Step 12: Design the target motion range and motion path according to the actual requirements of the space mission, including but not limited to spin approach and nutation approach motion paths;
[0060] Step 13: Generate a simulated training set based on the virtual environment and target 3D model from step 11 and the target motion range and motion path designed in step 12, including a target image and the corresponding true pose vector. The image resolution is 1024*1024, including but not limited to a background-free random motion dataset, a space background random motion dataset, a background-free spin approach dataset, and datasets with added noise and illumination.
[0061] Step 14: Based on the target motion range and motion path designed by the experimental system according to step 12, a target experimental dataset is obtained, including the target image and the corresponding true value of the pose vector, as a test set; the test set includes a target model photographed by a real camera, the size of the target model is 700mm*1000mm, and the resolution of the camera image is 640*368.
[0062] Reference Figure 2 , the true value of the pose vector includes:
[0063] p=[ρ T ,ψ T ] T =[x,y,z,α,β,γ] T
[0064] where ρ = [x, y, z] T , x, y, z are the three-axis coordinates of relative position; ψ=[α, β, γ] T , α, β, γ are the pitch angle, yaw angle and roll angle respectively;
[0065] Camera coordinate system O C -X C Y C Z C With the target star coordinate system O B -X B Y B Z B The transformation relationship between the two coordinate systems is expressed as:
[0066] r C =Rr B +ρ
[0067] Among them, r C =[x C ,y C ,z C ] T ,r B =[x B ,y B ,z B ] T are points in the camera coordinate system and the star coordinate system respectively, ρ is the relative position vector, R is the relative rotation matrix,
[0068]
[0069] Step 2 of this embodiment includes:
[0070] S21, read in the target pose estimation image dataset;
[0071] S22, reconstructing the target image to a set size, where the image size is 224*224;
[0072] S23, gray-scaling the target image reconstructed in S22, and filtering and denoising the image, wherein the filtering method includes but is not limited to Gaussian filtering;
[0073] S24, using Hough transform to extract straight line features in the filtered and denoised target image, obtain the star region, and generate an image mask;
[0074] S25. Generate an image mask to obtain a shape prior image.
[0075] Step 3 of this embodiment is used to design and train a shape prior information-assisted pose estimation network. First, the shape prior information-assisted network structure needs to be determined. The shape prior information-assisted network includes an encoding network, a sampling module, and a decoding network. The encoding network and the decoding network are not limited to a fully connected network. The sampling module processes to obtain a latent vector: Among them, z is the latent variable; is an auxiliary noise variable. A loss function is then designed. The training set images and the corresponding true pose values are input, and the encoding and decoding network parameters are trained based on the designed loss function. The network model is then validated using the test set to evaluate model performance.
[0076] The loss function designed in this embodiment is:
[0077]
[0078] Among them, γ1 and γ2 are hyperparameters for balancing the losses. is the KL-divergence loss, The cross entropy loss between the shape prior image and the target image generated by the decoding network, It is the mean absolute error loss between the true value of the pose vector and the pose estimation result output by the encoding network.
[0079] The KL divergence between two normal distributions can be expressed as:
[0080]
[0081] in, represents the normal distribution of the encoding network output.
[0082] like For the standard normal distribution, the DL divergence can be simplified as:
[0083]
[0084] The KL-divergence loss of the i-th training sample is:
[0085]
[0086] n b is the number of training samples in the dataset.
[0087] Cross Entropy Loss for:
[0088]
[0089] in, is the target image generated by the decoding network, and y is the shape prior image;
[0090] Mean absolute error loss
[0091]
[0092] Among them, μ (i) represents the mean of the i-th training sample, p (i) is the true value of the pose vector of the i-th training sample.
[0093] In this embodiment, step 4 is used to obtain the pose vector of the target to be measured, which specifically includes: using a visual sensor to complete image acquisition of the target to be measured; performing image preprocessing based on step 2 to obtain an input image; inputting the preprocessed input image into the encoding network; and the encoding network outputting the pose estimation result.
[0094] This embodiment uses shape prior information to assist network training methods for space mission pose estimation to improve the pose estimation accuracy of space targets.
[0095] During training, images are first generated using physics engine simulation software and a physical experimental system. Traditional image transformation methods are then used for image preprocessing. Next, a shape prior information-assisted network is designed and trained. Finally, model performance is verified. If accuracy does not meet requirements, retraining can be performed by adjusting the network structure, training methods, and model parameters.
[0096] When applied, first, the airborne vision system collects images; then, the images are preprocessed and input into the trained network; finally, the encoding network outputs the image pose estimation results.
[0097] Although the present invention is described herein with reference to specific embodiments, it should be understood that these embodiments are merely illustrative of the principles and applications of the invention. It should be understood that many modifications may be made to the illustrative embodiments, and that other arrangements may be devised, without departing from the spirit and scope of the invention as defined by the appended claims. It should be understood that the various dependent claims and features described herein may be combined in ways other than those described in the original claims. It should also be understood that features described in conjunction with individual embodiments may be employed in conjunction with other described embodiments.
Claims
1. A target pose estimation method for space missions, characterized in that: The method comprises: S1. Generate a dataset of target pose estimation images for space missions, which includes target images and corresponding pose vector truth values; S2. Preprocess the target image in the data set to obtain a shape prior image of the star area in the target image; S3, using the preprocessed data set to train the shape prior information auxiliary network; The shape prior information auxiliary network includes an encoding network, a sampling module and a decoding network; The shape prior image of the star region in the target image is input to the encoding network, which outputs the position mean μ of the six degrees of freedom and the standard deviation σ of the six degrees of freedom. The position mean and standard deviation are input to the sampling module, which processes them to obtain a latent vector. The latent vector is input to the decoding network, which generates the target image. S4. Use the trained encoding network to estimate the pose of the target image.
2. The method for estimating target pose for aerospace missions according to claim 1, characterized in that: In S3, the sampling module processes the latent vector: Where z is the potential vector; is an auxiliary noise variable.
3. The method for estimating target pose for aerospace missions according to claim 1, characterized in that: In S3, the training of the shape prior information auxiliary network using the preprocessed dataset includes: The data set is input into the shape prior information auxiliary network, and the loss function is used to train the parameters of the shape prior information auxiliary network; The loss function is: Among them, γ1 and γ2 are hyperparameters for balancing the losses. is the KL-divergence loss, The cross entropy loss between the shape prior image and the target image generated by the decoding network, It is the mean absolute error loss between the true value of the pose vector and the pose estimation result output by the encoding network.
4. The method for estimating target pose for aerospace missions according to claim 3, characterized in that: The KL-divergence loss is: Among them, n b is the number of training samples in the dataset, and the KL divergence of the i-th training sample between the two normal distributions represents the normal distribution of the encoding network output.
5. The method for estimating target pose for aerospace missions according to claim 3, characterized in that: Cross Entropy Loss for: Among them, n b is the number of training samples in the dataset, is the target image generated by the decoding network, and y is the shape prior image.
6. The method for estimating target pose for aerospace missions according to claim 3, characterized in that: Mean absolute error loss Among them, n b is the number of training samples in the dataset, the pose estimation result output by the encoding network is normally distributed, μ (i) represents the position mean of the i-th training sample, p (i) is the true value of the pose vector of the i-th training sample.
7. The method for estimating target pose for aerospace missions according to claim 1, characterized in that: The data set includes a training set and a test set, and S1 includes: S11, constructing multiple virtual environments with different lighting conditions and different variance noises and defining coordinate systems, and importing the target 3D model into the virtual environments; S12. Design the target motion range and motion path according to the actual requirements of the space mission; S13, generating a simulated training set based on the virtual environment and target three-dimensional model of S11 and the target motion range and motion path designed in S12, including a target image and a corresponding true value of a pose vector; S14. Based on the target motion range and motion path designed by the experimental system according to S12, a target experimental data set is obtained, including the target image and the corresponding pose vector true value, as a test set; In S3, the shape prior information auxiliary network is trained using the preprocessed training set, and is verified using the preprocessed test set.
8. The method for estimating target pose for aerospace missions according to claim 7, characterized in that: The true value of the pose vector includes: p=[ρ T ,ψ T ] T =[x,y,z,a,b,c] T where ρ = [x, y, z] T , x, y, z are the three-axis coordinates of relative position; ψ=[α, β, γ] T , α, β, γ are the pitch angle, yaw angle and roll angle respectively; Camera coordinate system O C -X C Y C Z C With the target star coordinate system O B -X B Y B Z B The transformation relationship between the two coordinate systems is expressed as: r C =Rr B +r Among them, r C =[x C ,y C ,z C ] T ,r B =[x B ,y B ,z B ] T are points in the camera coordinate system and the star coordinate system respectively, ρ is the relative position vector, R is the relative rotation matrix, 9. The method for estimating target pose for aerospace missions according to claim 8, wherein S2 include: S21, read in the target pose estimation image dataset; S22, reconstructing the target image to a set size; S23, gray-scale the target image reconstructed in S22, and perform filtering and denoising; S24, using Hough transform to extract straight line features in the filtered and denoised target image, obtain the star region, and generate an image mask; S25. Obtain a shape prior image according to the generated image mask.
10. A target pose estimation device for a space mission, comprising a storage device, a processor, and a computer program stored in the storage device and executable on the processor, characterized in that: The processor executes the computer program to implement the target pose estimation method for aerospace missions as described in any one of claims 1 to 9.