A cross-domain spacecraft pose estimation method based on mask self-distillation domain adaptation
By designing a robust keypoint regression network and a mask self-distillation strategy, the performance degradation problem of spacecraft pose estimation models on real optical sensor images is solved, achieving efficient pose estimation applicable to non-cooperative spacecraft scenarios.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HARBIN INST OF TECH
- Filing Date
- 2025-09-03
- Publication Date
- 2026-04-21
AI Technical Summary
Existing spacecraft pose estimation models trained on synthetic images suffer from a significant performance degradation when processing real optical sensor images due to differences in image distribution.
A cross-domain spacecraft pose estimation method based on mask self-distillation domain adaptation is adopted. By designing a robust keypoint regression network, combined with data augmentation strategy, teacher-student model collaborative training mechanism, mask self-distillation strategy and cross-domain semantic consistency constraint, the domain difference is reduced, and robust cross-domain keypoint detection and pose estimation are achieved.
It improves the accuracy and robustness of spacecraft attitude estimation, reduces computational complexity and memory resource consumption, is applicable to scenarios involving non-cooperative spacecraft, and overcomes interference from image texture differences and lighting variations.
Smart Images

Figure CN121120776B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of spacecraft pose estimation technology, specifically relating to a cross-domain spacecraft pose estimation method based on mask self-distillation domain adaptation. Background Technology
[0002] Deep learning-based spacecraft pose estimation methods are mainly divided into two categories: direct methods and indirect methods. Much research began with SPEC2019, a challenge that used SPEED as the benchmark dataset. This dataset contains 15,000 synthetic images generated by simulators and 300 real images captured by SLAB's TRON facility, primarily for pose estimation. Direct methods typically rely on a single deep learning model for pose regression, requiring only pose labels and not camera parameters or a 3D spacecraft model. Indirect methods achieve pose estimation through multiple stages: spacecraft localization (optional), keypoint regression, and pose solving. For spacecraft localization, existing mature single-stage or multi-stage detectors can be selected based on mission requirements. Although indirect methods have longer inference times, their staged processing characteristics make them easier to modularly update and optimize performance.
[0003] Because synthetic data differs from real-world images in terms of lighting and texture, model generalization suffers. SPEC 2021 aimed to bridge the domain gaps in pose estimation. This spurred the development of numerous solutions. For example, adding data augmentation to the source domain and simultaneously adding multiple auxiliary tasks can effectively enhance the model's domain generalization in the main pose estimation task. Adversarial training uses a domain discriminator to distinguish between the source and target domains and confuses the feature extractor by generating indistinguishable features, thus aligning the distributions of the two domains. Self-training strategies iteratively generate pseudo-labels for the target domain and retrain the model, gradually improving its adaptability to the target domain. While these methods achieve good performance, they often focus on global features while neglecting semantic consistency, leading to insufficient cross-domain alignment.
[0004] The core objective of unsupervised domain adaptation (UDA) is to align the feature distributions of the source and target domains, enabling models trained in the source domain to generalize to the unlabeled target domain. Early research primarily used adversarial training and self-training, as well as discrepancy minimization (e.g., maximum mean difference, covariance alignment). However, adversarial methods are prone to negative transfer due to gradient instability, especially limiting their performance in complex scenarios. Discrepancy minimization methods often ignore local structural information. Self-training methods can lead to the accumulation of errors from pseudo-labels and overfitting. In recent years, self-supervised learning has provided new ideas for UDA by designing pre-training tasks (e.g., contrastive learning, rotation prediction) to learn general representations from unlabeled data. Traditional CNNs have limitations in global feature modeling, while Vision Transformer (ViT) can effectively capture long-range dependencies through its self-attention mechanism. Furthermore, the pre-trained weights of ViT naturally possess stronger domain invariance, providing a high starting point for UDA.
[0005] Therefore, based on the advantages of ViT's pre-trained weights and self-supervised learning, a cross-domain spacecraft pose estimation method based on mask self-distillation domain adaptation is proposed. Summary of the Invention
[0006] The purpose of this invention is to provide a cross-domain spacecraft pose estimation method based on mask self-distillation domain adaptation, which can solve the problem that the pose estimation performance of existing spacecraft pose estimation models trained on synthetic images is significantly reduced when processing real optical sensor images due to differences in image distribution, i.e., domain differences.
[0007] The specific technical solution adopted by this invention is as follows:
[0008] A cross-domain spacecraft pose estimation method based on mask self-distillation domain adaptation is proposed. This invention employs an indirect method to process optical sensor data. Its core lies in designing a novel keypoint regression network capable of robust heatmap regression. Specifically, through innovative data augmentation strategies, a teacher-student model collaborative training mechanism, a mask self-distillation strategy, and cross-domain semantic consistency constraints, domain differences are effectively reduced, thereby achieving robust cross-domain keypoint detection and ultimately improving the 6D pose estimation accuracy of the spacecraft. The method includes the following steps:
[0009] Step 1: Based on the complete image input, the Faster R-CNN object detection algorithm is used to locate and obtain the bounding box of the target spacecraft; during the training phase, a data augmentation strategy designed for the characteristics of the real orbital environment of the spacecraft is used to train the detection network. Based on the detected bounding box, the corresponding Region of Interest (ROI) of the target spacecraft in the image is extracted as the input data for the subsequent keypoint regression network.
[0010] Step 2: Divide the ROI input data obtained in Step 1 into labeled source domain data and unlabeled target domain data, keeping the phase spectrum unchanged, and reconstruct the image using the mixed amplitude spectrum and the original phase spectrum to obtain the enhanced mixed domain data; construct a teacher model and a student model with the same structure based on the pre-trained ViT model; input the enhanced, labeled source domain data into the student model, and fine-tune the student model by calculating the source domain supervised heatmap loss between the predicted heatmap and its ground truth annotation;
[0011] Step 3: The enhanced, unlabeled target domain data is processed with a random mask and then input into the student model for prediction. The original, unenhanced, unlabeled target domain data is input into the teacher model for prediction. A self-distillation mechanism is used to minimize the prediction heat of the teacher and student models. Figure 1 The loss of consistency guides the student model to learn local discriminative features and their spatial context information; at the same time, by introducing a set of optimizable shared prototype vectors, the features extracted by the teacher and student models are guided to move closer to these prototypes, thereby achieving semantic consistency matching.
[0012] Step 4: Jointly optimize the three loss functions; the three loss functions are: source domain supervised heatmap loss, target domain mask heatmap loss, and target domain mask heatmap loss. Figure 1 Consistency loss and cross-domain semantic consistency loss jointly guide the training process of the keypoint regression network. During training, the parameters of the teacher model are updated using an exponential moving average strategy. The teacher model predicts the complete target domain image, and the robust representation obtained can generate more accurate pseudo-labels for the target domain, realizing the gradual transfer of source domain labeled knowledge to the target domain.
[0013] Step 5: Based on the known 3D key point model of the spacecraft and the 2D image key points predicted in Step 4, establish the 3D-2D point correspondence; solve the 6D pose of the spacecraft relative to the camera coordinate system using the perspective N-point algorithm; the 6D pose includes 3D position and 3D attitude.
[0014] The technical effects achieved by this invention are as follows:
[0015] This invention addresses the common problems of high computational cost and insufficient semantic feature mining in traditional domain adaptation methods for spacecraft pose estimation. The method provided by this invention features a simple structure, ease of implementation, and excellent robustness on real images. Specifically, the beneficial effects of this invention are reflected in the following aspects:
[0016] 1. Enhance the model's ability to perceive local features and their spatial context: By processing the target domain image region through random masking, the student model is required to predict the occluded parts to generate a complete keypoint heatmap. This mechanism effectively enhances the network's ability to infer the overall keypoint distribution from visible local regions, as well as its ability to model the spatial context relationships between keypoints, thereby improving its robustness to occlusion and noise.
[0017] 2. Enhancing the Representation and Generalization Ability of Cross-Domain Semantic Features: An innovative cross-domain representation learning mechanism based on optimizable shared prototypes is introduced. This mechanism utilizes view-invariant constraints to encourage sample features from different domains to converge towards the same set of shared prototypes. This not only strengthens the model's representation of domain-invariant general semantic features but also significantly reduces its dependence on domain-sensitive features, thereby improving the model's generalization performance in the target domain.
[0018] 3. Optimize inference efficiency and resource consumption: After model training, only the encoder-decoder structure from the student model that completes the keypoint heatmap prediction task is retained for pose estimation inference of the target domain image. This design significantly reduces computational complexity and memory resource consumption during deployment, improving the practical application efficiency of the algorithm.
[0019] In summary, this invention is particularly applicable to scenarios involving non-cooperative spacecraft. By proposing a cross-domain spacecraft pose estimation method based on mask self-distillation, it can effectively overcome interference caused by external environmental noise such as differences in image texture and changes in lighting conditions, while ensuring high computational efficiency and low memory usage. Ultimately, it achieves robust six-degree-of-freedom pose estimation for the target spacecraft. Attached Figure Description
[0020] Figure 1 This is a flowchart of the present invention.
[0021] Figure 2 A rendering of spacecraft pose estimation in the Lightbox domain.
[0022] Figure 3 Image showing the spacecraft pose estimation in the Sunlamp domain.
[0023] Figure 4 Image showing the alignment effect between the source and target domains
[0024] Figure 5 The relationship between average pose error and mask ratio at different input resolutions is shown in the graph.
[0025] Figure 6 Error distribution diagram of Lightbox domain translation along three axes and rotation around three axes
[0026] Figure 7This is a graph showing the error distribution of the Sunlamp domain during translation along the three axes and rotation around the three axes. Detailed Implementation
[0027] To make the objectives and advantages of this invention clearer, the invention will be specifically described below with reference to embodiments. It should be understood that the following text is merely used to describe one or more specific embodiments of the invention and does not strictly limit the scope of protection specifically claimed by the invention.
[0028] like Figure 1 As shown, a cross-domain spacecraft pose estimation method based on mask self-distillation domain adaptation includes the following steps:
[0029] Step 1: Based on the complete image input, the Faster R-CNN object detection algorithm is used to locate and obtain the bounding box of the target spacecraft. During the training phase, a data augmentation strategy designed for the real orbital environment characteristics of spacecraft is employed to train the detection network. This data augmentation strategy includes simulating changes in space illumination, attitude changes, partial occlusion, and background interference to improve the model's robustness and generalization ability. Based on the detected bounding box, the corresponding Region of Interest (ROI) of the target spacecraft in the image is extracted and used as input data for the subsequent keypoint regression network.
[0030] In step one:
[0031] Step 1: To construct a robust geometric representation, the target spacecraft is represented as a set of N = 11 key points, whose homogeneous coordinates are represented as follows: (u i ,v i () represents the two-dimensional pixel coordinates of the i-th keypoint in the image. The correspondence between the three-dimensional structure and the two-dimensional observation is established through projection:
[0032] s i [u i v i 1] T =K[R∣t]P i
[0033] Where K is the intrinsic parameter matrix of the monocular camera, and R and t are the rotation matrix and translation vector, respectively. i It is the scaling factor.
[0034] Given 11 labeled 3D keypoints and attitude labels for the target spacecraft Tango, 2D keypoint coordinates are calculated through projection transformation. The target detection bounding box is generated by extending the minimum-maximum coordinates of the 2D keypoint set outward by 20 pixels, ensuring complete coverage of the spacecraft region.
[0035] In practical implementation, this invention employs Faster R-CNN to detect spacecraft bounding boxes. Data augmentation techniques include histogram equalization, solar flare correction, blurring, and noise reduction. The detector is trained using stochastic gradient descent with a learning rate of 0.02, a warm-up phase of 1000 steps, and a 0.1-fold decrease in learning rate at the 5th and 10th epochs. Momentum is set to 0.9, and weight decay is 0.0001. The batch size is set to 32, and the system runs on a single NVIDIA RTX 6000Ada 48GB GPU. Post-processing employs adaptive bounding box fusion: overlapping detections with a confidence score greater than 0.5 are merged into a single bounding box; if detection fails, a 600×600 pixel centrally cropped region is used as the default RoI.
[0036] Step 2: Divide the ROI input data obtained in Step 1 into labeled source domain data and unlabeled target domain data, keeping the phase spectrum unchanged. Reconstruct the image using the mixed amplitude spectrum and the original phase spectrum to obtain the enhanced mixed domain data. Construct a teacher model and a student model with the same structure based on the pre-trained ViT model; the teacher model has a momentum update mechanism, and the student model performs forward propagation and gradient update; input the enhanced, labeled source domain data into the student model, and fine-tune the student model by calculating the source domain supervised heatmap loss between the predicted heatmap and its ground truth annotations.
[0037] In step two: Fourier transform of an image x:
[0038]
[0039] Where (w,h) and (u,v) represent the spatial domain coordinates and frequency domain coordinates, respectively. The above equation can be expressed in complex form as follows:
[0040] F(x)(u,v)=A(x)(u,v)·e j·φ(x)(u,v)
[0041] The formulas for the amplitude spectrum and the phase spectrum are as follows:
[0042]
[0043] Images from two random domains and The mixed amplitude spectrum is obtained by mixing the amplitude spectra of the two domains using a scaling factor λ ~ U[0,1.0].
[0044]
[0045] The original phase spectrum and the mixed amplitude spectrum of the combined image are then subjected to inverse Fourier transform to obtain the enhanced source domain image and target domain image. In addition to the above-mentioned spectral mixing enhancement, the enhancement strategies for the source domain data also include simple histogram equalization and random erasure.
[0046] The model is trained using an asymmetric encoder and decoder in the source domain. The model processes the input image, cropped by object detection, with the encoder mapping the input image to high-order semantic features. The decoder outputs a multi-channel heatmap tensor. True Value Heatmap A two-dimensional Gaussian kernel with a standard deviation of σ = 2 is applied to the keypoint coordinates to generate the heatmap. The source domain heatmap loss function is defined as follows:
[0047]
[0048] The maximum response position of each key point is extracted by nonmaximum suppression as the two-dimensional detection result. The 2D-3D correspondence is transformed into the original image, and then the optimal pose parameters are solved based on the EPnP algorithm and RANSAC algorithm.
[0049] Step 3: The enhanced, unlabeled target domain data is processed with a random mask and then input into the student model for prediction. The original, unenhanced, unlabeled target domain data is input into the teacher model for prediction. A self-distillation mechanism is used to minimize the prediction heat of the teacher and student models. Figure 1 The system employs consistency loss to guide the student model in learning local discriminative features and their spatial contextual information. Simultaneously, by introducing a set of optimizable shared prototype vectors, the features extracted by both teacher and student models are guided to gravitate towards these prototypes, achieving semantic consistency matching.
[0050] Step 3 introduces a mask distillation method involving teachers and students. Through teacher-student prediction consistency loss, the target domain features are implicitly aligned to the source domain geometric space. The student network receives the mask input and randomly selects several non-contiguous patch regions for masking each sample in the input image sequence; this is called zeroing. The mask is obtained by randomly sampling from a uniform distribution i, and then element-wise multiplying the mask with the samples to obtain the mask image.
[0051] The teacher network receives a complete target domain image as input, and its network parameters... The weights are obtained by smoothing and aggregating the historical states of the student network parameters θ using an exponential moving average update.
[0052]
[0053] Where γ∈[0,1) is the smoothing factor. The teacher network continuously absorbs domain-invariant features from the student network optimization process, such as rigid body geometric relationships, and gradually improves its understanding of the target domain context.
[0054] Teacher-generated pseudo-label heatmap Through masking Figure 1 Consistency loss constrains student predictions, forcing the model to reconstruct the complete heatmap from locally visible regions. Suppress dependence on domain-sensitive features such as texture. The loss is as follows:
[0055]
[0056] Where q i The weights guided by teacher confidence are obtained by taking the maximum value of the spatial location of each channel as the confidence of the corresponding key point. This strategy prioritizes strengthening the supervision signal of high-confidence key points to improve target localization accuracy; at the same time, it weakens the interference of pseudo-labels in low-confidence regions, significantly improving the model's robustness to pose estimation of masked input and cross-domain data.
[0057] This invention transforms the features of the source and target domains to a shared prototype space, forcing samples from both domains to have the same semantic distribution in this space. This eliminates domain-sensitive features (such as background and lighting) and strengthens cross-domain shared semantic features (such as object shape and structure). First, a series of data augmentations are performed on the input image to generate multiple lower-resolution views. These views, along with the mask image from step two, form an anchor view. The image is passed from the student to the teacher, using the same unmasked image as the target view. Define a cross-domain shared prototype matrix p:
[0058]
[0059] Where k is the number of prototypes and l is the feature dimension. The prototype matrix is jointly optimized with the backbone network parameters. The [CLS] embeddings of the two views are the projected and normalized features:
[0060]
[0061] Calculate the similarity distribution vector based on the prototype matrix:
[0062]
[0063] Where τ a and τ t It is a temperature coefficient. To prevent prototype collapse, target sharpening is combined with mean-entropy maximization, always maintaining τ. t <τ a Using the soft distribution of the target view as a supervision signal, a stable target is constructed by stopping gradient operations, and the anchor view prediction is forced to match the target view distribution through cross-domain semantic consistency loss.
[0064]
[0065] Step 4: Jointly optimize the three loss functions; the three loss functions are: source domain supervised heatmap loss, target domain mask heatmap loss, and target domain mask heatmap loss. Figure 1 Consistency loss and cross-domain semantic consistency loss jointly guide the training process of the keypoint regression network. During training, the parameters of the teacher model are updated using an exponential moving average strategy. The teacher model predicts the complete target domain image, and the resulting robust representation can generate more accurate pseudo-labels for the target domain, realizing the gradual transfer of source domain annotation knowledge to the target domain.
[0066] In step four, the overall learning objective is:
[0067] L total =L src +L hc +αL sc
[0068] Where α = 0.001.
[0069] 47,966 images from the SPEED+ dataset were used as the training set, and 11,994 images were used as the validation set. Hardware-in-the-loop (HIL) domain images generated under different realistic lighting conditions using SLAB's TRON facility were used as the test set. These included 6,740 lightbox domain images with reflectivity simulation using lightboxes with diffusers, and 2,791 sunlamp domain images with fluorescent lamps to simulate direct, high-intensity uniform light from the sun.
[0070] Each network's encoder-decoder section follows the ViTPose-S architecture and has 11 output channels. The projector consists of a three-layer multilayer perceptron (MLP) with batch normalization (BatchNorm) and GELU activation functions. The system uses a set of 1024 learnable prototypes. During training, the same environment as for object detection is used, with student models configured for an input resolution of 384×384 and DeiT-III pre-trained weights. Optimization employs the AdamW algorithm, training for 30 epochs with a base learning rate of 0.0005, a batch size of 32, a 1000-step warm-up, and an exponential moving average factor γ = 0.999. The learning rate is reduced by a factor of 0.1 at epochs 10, 15, and 20.
[0071] Step 5: Based on the known 3D keypoint model of the spacecraft and the 2D image keypoints predicted in Step 4, establish the 3D-2D point correspondence. Using the Perspective-n-Point (PnP) algorithm, solve for the 6D pose of the spacecraft relative to the camera coordinate system; the 6D pose includes 3D position and 3D attitude.
[0072] In step five: only the encoder and decoder of the student model are retained to infer the real image and obtain two-dimensional pixel key points. Then, a reliable 3D-2D point correspondence is established based on the 3D model in step one. Once this set of matching point pairs is obtained, the solvePnPRansac function in OpenCV is used to solve for the 6-DOF pose of the spacecraft body coordinate system relative to the camera coordinate system.
[0073] In this invention: Figure 2 and Figure 3 The images show the spacecraft pose estimation results in the Lightbox and Sunlamp domains, respectively. Figure 4 This is a diagram showing the alignment effect between the source and target domains. Figure 5 The graph shows the relationship between average pose error and mask ratio at different input resolutions. Figure 6 and Figure 7 The error distribution diagrams for translation along the three axes and rotation around the three axes are shown for the Lightbox domain and the Sunlamp domain, respectively.
[0074] This invention addresses the common problems of high computational cost and insufficient semantic feature mining in traditional domain adaptation methods for spacecraft pose estimation. The method provided by this invention features a simple structure, ease of implementation, and excellent robustness on real images. Specifically, the beneficial effects of this invention are reflected in the following aspects:
[0075] This invention enhances the model's ability to perceive local features and their spatial context: by processing the target domain image region through random masking, the student model is required to predict the occluded parts to generate a complete keypoint heatmap. This mechanism effectively enhances the network's ability to infer the overall keypoint distribution from visible local regions, as well as its ability to model the spatial context relationships between keypoints, thus improving its robustness to occlusion and noise.
[0076] This invention enhances the representation and generalization capabilities of cross-domain semantic features by innovatively introducing a cross-domain representation learning mechanism based on optimizable shared prototypes. This mechanism utilizes view-invariant constraints to encourage sample features from different domains to converge towards the same set of shared prototypes. This not only strengthens the model's representation of domain-invariant general semantic features but also significantly reduces its dependence on domain-sensitive features, thereby improving the model's generalization performance in the target domain.
[0077] This invention optimizes inference efficiency and resource consumption: after model training, only the encoder-decoder structure from the student model that performs the keypoint heatmap prediction task is retained for pose estimation inference of the target domain image. This design significantly reduces computational complexity and memory resource consumption during deployment, improving the practical application efficiency of the algorithm.
[0078] In summary, this invention is particularly applicable to scenarios involving non-cooperative spacecraft. By proposing a cross-domain spacecraft pose estimation method based on mask self-distillation, it can effectively overcome interference caused by external environmental noise such as differences in image texture and changes in lighting conditions, while ensuring high computational efficiency and low memory usage. Ultimately, it achieves robust six-degree-of-freedom pose estimation for the target spacecraft.
[0079] The above description is merely a preferred embodiment of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention. Structures, devices, and operating methods not specifically described or explained in this invention are implemented according to conventional methods in the art unless otherwise specified or limited.
Claims
1. A cross-domain spacecraft pose estimation method based on mask self-distillation domain adaptation, characterized in that: Includes the following steps: Step 1: Based on the complete image input, use the Faster R-CNN object detection algorithm to locate and obtain the bounding box of the target spacecraft; During the training phase, a data augmentation strategy designed for the characteristics of the actual orbital environment of spacecraft is used to train the detection network. Based on the detected bounding boxes, the corresponding Region of Interest (ROI) of the target spacecraft in the image is extracted and used as input data for the subsequent keypoint regression network. Step 2: Divide the ROI input data obtained in Step 1 into labeled source domain data and unlabeled target domain data, keeping the phase spectrum unchanged, and reconstruct the image using the mixed amplitude spectrum and the original phase spectrum to obtain the enhanced mixed domain data; construct a teacher model and a student model with the same structure based on the pre-trained ViT model; input the enhanced, labeled source domain data into the student model, and fine-tune the student model by calculating the source domain supervised heatmap loss between the predicted heatmap and its ground truth annotation; In step two: An image Fourier transform: ; in and Let the coordinates in the spatial domain and frequency domain be represented respectively; express the above equation in complex form: ; The formulas for the amplitude spectrum and the phase spectrum are as follows: ; Images from two random domains and Use scaling factor The amplitude spectra of the two domains are mixed to obtain the mixed amplitude spectrum: ; The original phase spectrum and the mixed amplitude spectrum of the combined image are transformed by inverse Fourier transform to obtain the enhanced source domain image and target domain image. In addition to spectral mixing enhancement, the enhancement strategies for the source domain data also include simple histogram equalization and random erasure. The model is trained using an asymmetric encoder and decoder in the source domain; the model processes the input image cropped by object detection, and the encoder maps the input image to higher-order semantic features. Decoder outputs multi-channel heatmap tensors True Value Heatmap By applying standard deviation at key point coordinates Two-dimensional Gaussian kernel generation; defining the source domain heatmap loss function: ; The maximum response position of each key point is extracted by nonmaximum suppression as the two-dimensional detection result. The 2d-3d correspondence is transformed into the original image, and then the optimal pose parameters are solved based on the EPnP algorithm and RANSAC algorithm. Step 3: The enhanced, unlabeled target domain data is processed with a random mask and then input into the student model for prediction. The original, unenhanced, unlabeled target domain data is input into the teacher model for prediction. Using a self-distillation mechanism, the student model is guided to learn local discriminative features and their spatial contextual information by minimizing the consistency loss of the prediction heatmaps of the teacher and student models. At the same time, by introducing a set of optimizable shared prototype vectors, the features extracted by the teacher and student models are guided to move closer to these prototypes to achieve semantic consistency matching. Step 4: Jointly optimize the three loss functions; the three loss functions are: source domain supervised heatmap loss, target domain mask heatmap consistency loss, and cross-domain semantic consistency loss, which together guide the training process of the key point regression network; During training, the parameters of the teacher model are updated using an exponential moving average strategy. The teacher model predicts the complete target domain image, and the robust representation obtained can generate more accurate pseudo-labels for the target domain, realizing the gradual transfer of source domain labeled knowledge to the target domain. Step 5: Based on the known 3D key point model of the spacecraft and the 2D image key points predicted in Step 4, establish the 3D-2D point correspondence; solve the 6D pose of the spacecraft relative to the camera coordinate system using the perspective N-point algorithm; the 6D pose includes 3D position and 3D attitude.
2. The cross-domain spacecraft pose estimation method based on mask self-distillation domain adaptation according to claim 1, characterized in that: Step one specifically includes: To construct a robust geometric representation, the target spacecraft is represented as a set of N=11 key points, whose homogeneous coordinates are expressed as follows: , Indicates the first in the image Two-dimensional pixel coordinates of key points; establishing the correspondence between the three-dimensional structure and the two-dimensional observation through projection: ; in, This is the intrinsic parameter matrix of a monocular camera. and These are the rotation matrix and the translation vector, respectively. It is the scaling factor; Given 11 labeled 3D keypoints and attitude labels of the target spacecraft Tango, calculate the coordinates of the 2D keypoints through projection transformation.
3. The cross-domain spacecraft pose estimation method based on mask self-distillation domain adaptation according to claim 1, characterized in that: In step three: A masked teacher-student distillation method is introduced, using teacher-student prediction consistency loss to implicitly align target domain features to the source domain geometric space. The student network receives masked input and randomly selects several non-contiguous patch regions for masking each sample in the input image sequence. From a uniform distribution... A mask is obtained by random sampling, and the mask is multiplied element-wise with the sample to obtain the mask image; The teacher network receives a complete target domain image as input, and its network parameters... Updated via exponential moving average, its weights are determined by student network parameters. The smooth aggregation of historical states yields: ; in, As a smoothing factor; the teacher network continuously absorbs domain-invariant features from the student network optimization process; gradually improving its understanding of the target domain context; Teacher-generated pseudo-label heatmap By constraining student predictions with masked heatmap consistency loss, the model is forced to reconstruct the complete heatmap from locally visible regions. Suppress the dependence on texture-sensitive features; the loss is as follows: ; in The weights for teacher confidence are obtained by taking the maximum value of the spatial location of each channel as the confidence of the corresponding keypoint. First, a series of data augmentations are performed on the input image to generate multiple smaller-resolution views, which, together with the mask image from step two, form an anchor point view. The image is passed from the student to the teacher, using the same unmasked image as the target view. Define a cross-domain shared prototype matrix. : ; in For the number of prototypes, For feature dimension; prototype matrix and backbone network parameters are jointly optimized; [CLS] embeddings of the two views are projected and normalized features: ; Calculate the similarity distribution vector based on the prototype matrix: ; in and It is a temperature coefficient. To prevent prototype collapse, target sharpening is combined with mean-entropy maximization, always maintaining... Using the soft distribution of the target view as a supervision signal, a stable target is constructed by stopping gradient operations, and the anchor view prediction is forced to match the target view distribution through cross-domain semantic consistency loss. 。 4. The cross-domain spacecraft pose estimation method based on mask self-distillation domain adaptation according to claim 3, characterized in that: In step four, the overall learning objective is: ; in, ; 47,966 images from the SPEED+ dataset were used as the training set, and 11,994 images were used as the validation set. Hardware-in-the-loop domain images generated under different real-world lighting conditions through SLAB's TRON facility were used as the test set, which included 6,740 lightbox domain images that simulated reflectivity using lightboxes with diffusers, and 2,791 sunlamp domain images that simulated direct, high-intensity uniform light from the sun using fluorescent lamps. Each network's encoder-decoder section follows the ViTPose-S architecture and has 11 output channels; the projector consists of a three-layer multilayer perceptron with batch normalization and GELU activation functions; the system uses a set of 1024 learnable prototypes; during training, the same environment as for object detection is used, with student models configured with an input resolution of 384×384 and DeiT-III pre-trained weights; optimization employs the AdamW algorithm, training for 30 epochs with a base learning rate of 0.0005, a batch size of 32, a 1000-step warm-up, and an exponential moving average factor. The learning rate was reduced by 0.1x at the 10th, 15th, and 20th epochs.
5. The cross-domain spacecraft pose estimation method based on mask self-distillation domain adaptation according to claim 1, characterized in that: In step five: Only the encoder and decoder of the student model are retained to infer the real image and obtain two-dimensional pixel key points. Then, a reliable 3D-2D point correspondence is established based on the 3D model in step one. Once this set of matching point pairs is obtained, the solvePnPRansac function in OpenCV is used to solve the 6-DOF pose of the spacecraft body coordinate system relative to the camera coordinate system.
Citation Information
Patent Citations
Deep learning and geometric algorithm combined non-cooperative target relative pose estimation method
CN111862126A
Aerial image target detection method based on cross-level feature distillation
CN118691993A