Non-cooperative target 6D pose estimation method based on multi-spectrum feature interaction

Abstract

The invention relates to the technical field of aerospace robot docking, in particular to a non-cooperative target 6D pose estimation method based on multi-spectrum feature interaction, which comprises the following steps: acquiring and preprocessing image data, and establishing a training data set; according to the training data set, respectively obtaining low-frequency features and high-frequency features, interactively constructing a 6D pose estimation model, and designing a loss function to train the 6D pose estimation model; and performing pose prediction through the trained 6D pose estimation model, and outputting a prediction result to realize pose estimation of the non-cooperative target. The 6D pose estimation model comprehensively processes high-frequency features and low-frequency features of spacecraft image data so as to realize aggregation of long-range and long-distance context features; and the 6D pose estimation model has the characteristics of lightweight deployment, efficient algorithm and the like, can improve the efficiency and precision of non-cooperative spacecraft target pose estimation, and can be applied to 6D relative pose estimation for close-range spacecrafts and assist autonomous docking of spacecrafts.

Description

Technical Field

[0001] This invention relates to the field of aerospace robot docking technology, specifically to a non-cooperative target 6D pose estimation method based on multi-spectral feature interaction. Background Technology

[0002] To ensure the successful completion of space missions, it is necessary to obtain the motion status of spacecraft from both sides. However, most spacecraft are non-cooperative targets and cannot provide accurate attitude information in real time or communicate instantly. Therefore, actively estimating the accurate attitude of the target spacecraft is a key link in space offense and defense and on-orbit servicing.

[0003] The space environment is quite unique. Spacecraft targets generally have a wider range of motion, resulting in significant changes in size and position after imaging, and a higher probability of truncation. Furthermore, the harsh conditions of space imaging, the lack of atmospheric scattering, and the difficulty in collecting direct depth information pose significant challenges to the research.

[0004] To address these issues, deep learning-based pose estimation methods have become increasingly popular. Compared to publicly available datasets like YCB-Video (Yale-CMU-Berkeley-Video) and LINEMOD (Line Mode), which feature rich textures, edges, and other details of objects and contain more high-frequency feature regions, spacecraft datasets such as SPEED and SwissCube contain a large number of uniformly colored and textureless features, resulting in more low-frequency feature regions. While convolutional layers, acting as high-pass filters, can handle textured objects well, their ability to extract low-frequency features and mine global information from textureless spacecraft is limited. Therefore, methods that perform well on general datasets are not ideal for spacecraft pose estimation, and existing spacecraft pose estimation methods often neglect the interactions between frequency domain features. Summary of the Invention

[0005] To address the limitations of existing pose estimation methods in extracting low-frequency features and mining global information from textureless spacecraft, and the technical problem of ignoring frequency domain feature interactions, this invention aims to provide a non-cooperative target 6D pose estimation method based on multi-spectral feature interactions. The specific technical solution adopted is as follows:

[0006] Acquire image data and preprocess it to create a training dataset;

[0007] Low-frequency and high-frequency features are obtained from the training dataset and then interactively constructed to form a 6D pose estimation model. A loss function is designed to train the 6D pose estimation model.

[0008] Pose prediction is performed using a trained 6D pose estimation model, and the prediction results are output to achieve pose estimation for non-cooperative targets.

[0009] Preferably, image data is acquired and preprocessed to establish a training dataset, including:

[0010] Image data was obtained using the publicly available SwissCube and SPEED pose estimation datasets;

[0011] A training dataset is established by preprocessing image data using mosaic data augmentation and adaptive image scaling.

[0012] Preferably, low-frequency features and high-frequency features are obtained from the training dataset, and an interactive 6D pose estimation model is constructed. A loss function is designed to train the 6D pose estimation model, including:

[0013] High-frequency and low-frequency features of any image data in the training dataset are extracted using ResNet18 and DarkNet53 respectively.

[0014] The Transformer-based feature interaction module is designed to perform attention to high-frequency and low-frequency features and guide feature fusion. Counterfactual analysis is used to establish a 6D pose estimation model.

[0015] The loss function is designed to train the 6D pose estimation model, resulting in the trained 6D pose estimation model.

[0016] Preferably, high-frequency and low-frequency features of any image data in the training dataset are extracted using ResNet18 and DarkNet53 respectively, specifically as follows:

[0017] The corresponding edge image is obtained based on any image data. The edge image is then concatenated with the image data to form the input image. ResNet18 is used to extract the high-frequency features of the input image, denoted as... The image data was processed using DarkNet53 to obtain the corresponding low-frequency features, denoted as... .

[0018] Preferably, a Transformer-based feature interaction module is designed to perform attention to high-frequency and low-frequency features and guide feature fusion. Counterfactual analysis is used to establish a 6D pose estimation model, including:

[0019] Position encoding is performed on high-frequency and low-frequency features respectively to obtain the corresponding two-class flattened high-frequency and low-frequency features. Feature interaction is performed through attention blocks, and the low-frequency features are updated with information retrieved from the high-frequency features. The high-frequency and low-frequency features after interaction are output accordingly.

[0020] The design frequency-guided feature fusion module guides the fusion of high-frequency and low-frequency features after interaction, splices the high-frequency and low-frequency features after interaction, divides them into multiple sub-features, assigns a corresponding two-dimensional DCT frequency classification to each sub-feature, and connects them to obtain a compressed vector.

[0021] The enhanced frequency domain features are obtained by simplifying the compressed vector and then dot-producting it with the compressed vector.

[0022] A 6D pose estimation model is established by using counterfactual analysis to weaken the unbiased features after background interference.

[0023] Preferably, the high-frequency and low-frequency features after concatenation and interaction are divided into multiple sub-features, each sub-feature is assigned a corresponding two-dimensional DCT frequency classification, and the sub-features are concatenated to obtain a compressed vector, including:

[0024] The high-frequency and low-frequency features after the interaction are denoted as follows: And, after concatenation, are divided into multiple sub-features, denoted as ,in, The size is , , , Indicates the batch size of the sub-features; Indicates the number of sub-features; , , These represent the height, width, and number of channels, respectively. This indicates the sparsity of sub-features;

[0025] For each sub-feature, a corresponding two-dimensional DCT frequency classification is assigned, and the corresponding calculation formula is:

[0026]

[0027] in, Indicates the first The compressed vector of each sub-feature; Represents two dimensions; express The corresponding frequency component 2D index; , All indicate the first Individual characteristics; Describe the basis functions of the DCT;

[0028] The concatenation yields a compressed vector, and the corresponding calculation formula is:

[0029]

[0030] in, Represents a compressed vector; Indicates a splicing operation; These respectively represent the compressed vectors of the sub-features.

[0031] Preferably, the enhanced frequency domain features are obtained by simplifying based on the compressed vector and then dot-producting with the compressed vector, including:

[0032] Based on the simplification of the compressed vector, the corresponding calculation formula is:

[0033]

[0034] in, This represents the simplified multispectral features; Indicates the activation function; Indicates fully connected layer operations; Represents a compressed vector;

[0035] The enhanced frequency domain features are obtained by dot product with the compressed vector, denoted as . .

[0036] Preferably, a loss function is designed to train the 6D pose estimation model, resulting in a trained 6D pose estimation model, including:

[0037] A target segmentation module is introduced based on high-frequency features. The loss function includes 3D keypoint loss, target binary classification loss, difference loss generated in counterfactual analysis, and semantic segmentation loss.

[0038] A loss function is constructed to train the 6D pose estimation model, resulting in the trained 6D pose estimation model. The corresponding calculation formula is as follows:

[0039]

[0040] in, Represents the loss function; These correspond to 3D keypoint loss, target binary classification loss, difference loss generated in counterfactual analysis, and semantic segmentation loss, respectively. , All of these represent weighting coefficients.

[0041] Preferably, pose prediction is performed using a trained 6D pose estimation model, and the prediction result is output to achieve pose estimation for non-cooperative targets, specifically as follows:

[0042] The features output by the trained 6D pose estimation model are input into the anchor-based keypoint detector to predict 3D keypoints. The PnP algorithm is used to solve the 6D pose of the non-cooperative target to obtain the prediction result, thus realizing the pose estimation of the non-cooperative target.

[0043] The present invention has the following beneficial effects:

[0044] The 6D pose estimation model constructed in this application uses DarkNet-53 to extract low-frequency features from image data, combines them with edge images as input images for corresponding processing, thereby enhancing the high-frequency attention of CNN (Convolutional Neural Network); it uses ResNet18 to extract multi-scale high-frequency features, and uses the extracted features for non-cooperative target semantic segmentation; unlike existing methods that directly use Transformer as the baseline backbone to extract features, this application uses a Transformer-based feature interaction module to perform high-frequency and low-frequency attention on spacecraft images, thereby achieving long-range, long-distance aggregation of contextual features; and the 6D pose estimation model has the characteristics of lightweight deployment and high algorithm efficiency, which can improve the efficiency and accuracy of non-cooperative spacecraft target pose estimation, and can be applied to 6D relative pose estimation for close-range spacecraft, facilitating autonomous spacecraft docking. Attached Figure Description

[0045] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 A flowchart illustrating the steps of a non-cooperative target 6D pose estimation method based on multi-spectral feature interaction provided in one embodiment of the present invention;

[0047] Figure 2 This is a model architecture diagram of a 6D pose estimation model for a non-cooperative target 6D pose estimation method based on multi-spectral feature interaction provided in an embodiment of the present invention.

[0048] Figure 3 This is a schematic diagram of a Transformer-based feature interaction module for a non-cooperative target 6D pose estimation method based on multi-spectral feature interaction, provided in an embodiment of the present invention.

[0049] Figure 4 This is a schematic diagram illustrating guided feature fusion of a non-cooperative target 6D pose estimation method based on multi-spectral feature interaction, as provided in an embodiment of the present invention. Detailed Implementation

[0050] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of a non-cooperative target 6D pose estimation method based on multi-spectral feature interaction proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0051] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0052] The following description, in conjunction with the accompanying drawings, details the specific scheme of a non-cooperative target 6D pose estimation method based on multi-spectral feature interaction provided by the present invention.

[0053] Please combine Figure 1 and Figure 2 The diagrams show a flowchart of the steps and a model architecture diagram of a non-cooperative target 6D pose estimation method based on multi-spectral feature interaction provided in the first embodiment of the present invention. The method includes:

[0054] Step S1: Acquire image data and preprocess it to build a training dataset;

[0055] Step S2: Obtain low-frequency features and high-frequency features from the training dataset, and interactively construct a 6D pose estimation model. Design a loss function to train the 6D pose estimation model.

[0056] Step S3: Perform pose prediction using the trained 6D pose estimation model, output the prediction results, and achieve pose estimation for non-cooperative targets.

[0057] To better illustrate, a non-cooperative target refers to an object that has not established communication or cooperative work with a system beforehand; that is, in this field, a non-cooperative target refers to a spacecraft used to complete a space mission but not coordinated with existing systems. It cannot effectively exchange data and coordinate operations with other spacecraft or ground control centers, and may float freely in orbit or maintain an independent operating state when performing specific tasks. It increases the complexity of space operations. Therefore, a pose estimation method for non-cooperative targets is proposed to achieve rapid and accurate pose calculation of non-cooperative target spacecraft for autonomous rendezvous and docking, so as to provide accurate pose information and enable it to communicate in real time.

[0058] Further, step S1 includes:

[0059] Step S11: Obtain image data using the publicly available SwissCube and SPEED pose estimation datasets.

[0060] The SwissCube dataset simulates the relative positions of the Earth, the Sun, and the target spacecraft, as well as the operational data of the space environment. It contains 50,000 images across 500 scenes and 100 image sequences. The target spacecraft model in the SPEED dataset is scaled down to 1:2 scale based on the actual size of the Tango satellite from the PRISMA mission. It includes 12,000 composite images and 5 real images. The distance of the spacecraft in the composite images ranges from 3 to 40.5 m, while the distance in the real images ranges from 2.8 to 4.7 m.

[0061] Preferably, in this embodiment, the test set of the SPEED dataset contains 300 real images and 2998 synthetic images, with an image resolution of 1920×1200; the training set of the SwissCube dataset contains 40,000 images, and the test set contains 10,000 images, with an image resolution of 1024×1024.

[0062] Step S12: Perform preprocessing on the image data, including mosaic data augmentation and adaptive image scaling, to establish a training dataset.

[0063] It is explained that mosaic data augmentation is used to generate multiple sub-images of different sizes and perspectives by randomly cropping and scaling image data. It can simulate observation conditions in various complex environments and dynamically adjust the scaling ratio of image data to ensure that key details are preserved at different resolutions, thereby enhancing background diversity and improving the representation of small targets on spacecraft.

[0064] Further, step S2 includes:

[0065] Step S21: Extract high-frequency and low-frequency features from any image data in the training dataset using ResNet18 and DarkNet53 respectively.

[0066] It should be noted that ResNet18 and DarkNet53 are two different types of convolutional neural network architectures. ResNet18 is a residual network, while DarkNet53 is a network used for object detection and image recognition.

[0067] Furthermore, in step S21, specifically:

[0068] The corresponding edge image is obtained based on any image data. The edge image is then concatenated with the image data to form the input image. ResNet18 is used to extract the high-frequency features of the input image, denoted as... The image data was processed using DarkNet53 to obtain the corresponding low-frequency features, denoted as... .

[0069] To clarify, the input object includes image data and corresponding edge images; that is, the input image is the original image sample and its edge information, which can highlight the significant change areas in the image data. ResNet18 is used to extract multi-scale high-frequency features from the input image, denoted as... This allows CNNs to more effectively capture key features in images, enhancing the high-frequency focus of the CNN and helping to improve the model's recognition ability and accuracy in complex scenes; while low-frequency features... Extracted using DarkNet53; where high-frequency features refer to rapidly changing parts of the image data, such as edges, textures, or details, which are represented by sharp changes in pixel values ​​in the spatial domain, corresponding to high-frequency components; low-frequency features refer to slowly changing or smooth parts of the image data, such as large areas of sky, grass, or walls, which are represented by slow changes in pixel values ​​in the spatial domain, corresponding to low-frequency components.

[0070] Preferably, in this embodiment, high-frequency features Low-frequency characteristics middle This indicates the extracted multi-scale features, and high-frequency features are also used for spacecraft target semantic segmentation. High-frequency features can capture subtle changes and details in image data, or identify small target objects in complex backgrounds, thereby improving segmentation accuracy.

[0071] Please see Figure 3 The diagram illustrates a Transformer-based feature interaction module of a non-cooperative target 6D pose estimation method based on multi-spectral feature interaction provided in the first embodiment of the present invention.

[0072] Step S22: Design a Transformer-based feature interaction module to perform attention to high-frequency and low-frequency features, guide feature fusion, and use counterfactual analysis to establish a 6D pose estimation model.

[0073] It can be explained that, based on the two branches of high-frequency features and low-frequency features, a feature interaction module based on Transformer is designed to perform high-frequency attention and low-frequency attention on image data, so as to capture high-frequency details and low-frequency global information in image data respectively, and realize the aggregation of long-distance contextual features; among them, the feature interaction module of Transformer is able to efficiently capture the complex feature interaction relationship between high-frequency features and low-frequency features of the input through a multi-head self-attention mechanism.

[0074] Further, step S22 includes:

[0075] Step S221: Perform position encoding based on high-frequency features and low-frequency features respectively to obtain the corresponding two-class flattened high-frequency features and low-frequency features, and perform feature interaction through attention blocks. Update the low-frequency features with information retrieved from the high-frequency features, and output the interacted high-frequency features and low-frequency features accordingly.

[0076] Specifically, based on the high-frequency features output in this embodiment and low frequency characteristics Position encoding is performed separately to implicitly enhance positional consistency, enabling the model to understand the relative or absolute positions of elements within features and improving feature interaction. High-frequency features become more spatially ordered after processing, while low-frequency features are ensured global consistency through smooth positional encoding, resulting in flattened high-frequency and low-frequency features. This not only preserves the frequency characteristics of the original features but also improves their temporal coherence and accuracy. Next, features are interacted through attention blocks, retrieving visual information related to color, texture, or shape, and aggregating contextual features such as the current background environment, spatial relationships, and time series. This is known as inter-attention, effectively capturing complex relationships between features and improving the model's performance and accuracy when processing multimodal data. Finally, the information retrieved from high-frequency features updates the low-frequency features, and the resulting high-frequency and low-frequency features are output as follows: and This corresponds to intra-attention, which focuses on the cross-relationship between high-frequency and low-frequency features in the attention block to achieve bidirectional operation.

[0077] It can be explained that when performing bidirectional operations, i.e. calculating high-frequency features... Low-frequency characteristics When there is mutual attention, only low-frequency features are updated. The feature information, among which the core of the attention mechanism is to measure high-frequency features. and low frequency characteristics The similarity of features between them is used to determine the weights and retrieve information from the value matrix. The corresponding calculation formula is as follows:

[0078]

[0079] in, Indicates feature similarity; Indicates classifier operation; This represents the query matrix, i.e., the part currently being processed; This represents the key matrix, i.e., the part that is being queried; This represents a value matrix, containing information corresponding to the keys; Indicates transpose; This represents the dimension of the key vector, used to scale the dot product result and prevent the gradient vanishing problem caused by an excessively large result.

[0080] Please see Figure 4 It shows a schematic diagram of guided feature fusion of a non-cooperative target 6D pose estimation method based on multi-spectral feature interaction provided by the first embodiment of the present invention.

[0081] Step S222: Design a frequency-guided feature fusion module to guide the high-frequency and low-frequency features after interaction to perform feature fusion, splice the high-frequency and low-frequency features after interaction, divide them into multiple sub-features, assign a corresponding two-dimensional DCT frequency classification to each sub-feature, and connect them to obtain a compressed vector.

[0082] The frequency-guided feature fusion module further fuses the feature information of the two branches at the frequency domain level based on the high-frequency and low-frequency features after interaction, in order to aggregate multi-spectral features, namely the high-frequency and low-frequency features after interaction, so as to fully explore the feature information of low-visibility spacecraft images.

[0083] Further, in step S222, the concatenated high-frequency and low-frequency features are divided into multiple sub-features. Each sub-feature is assigned a corresponding two-dimensional DCT frequency classification, and the sub-features are concatenated to obtain a compressed vector, including:

[0084] The high-frequency and low-frequency features after the interaction are denoted as follows: and After concatenation, it is divided into multiple sub-features, denoted as... ,in, The size is , , , Indicates the batch size of the sub-features; Indicates the number of sub-features; , , These represent the height, width, and number of channels, respectively. This indicates the sparsity of sub-features;

[0085] For each sub-feature, a corresponding two-dimensional DCT frequency classification is assigned, and the corresponding calculation formula is:

[0086]

[0087] in, Indicates the first The compressed vector of each sub-feature; Represents two dimensions; express The corresponding frequency component 2D index; , All indicate the first Individual characteristics; Describe the basis functions of the DCT;

[0088] The concatenation yields a compressed vector, and the corresponding calculation formula is:

[0089]

[0090] in, Represents a compressed vector; Indicates a splicing operation; These respectively represent the compressed vectors of the sub-features.

[0091] Provide an explanation. The sparsity of sub-features is indicated by the value of 'sparseness', which determines the number of sub-features. A larger value indicates fewer sub-features per channel, resulting in a more concise and concentrated representation. Conversely, a smaller value indicates more sub-features per channel, leading to a more complex representation. Preferably, when performing two-dimensional DCT (Discrete Cosine Transform) frequency classification for each sub-feature, the top 16 frequency elements with the highest performance are selected.

[0092] Step S223: Simplify based on the compression vector and dot product with the compression vector to obtain the enhanced frequency domain features.

[0093] Further, step S223 includes:

[0094] Based on the simplification of the compressed vector, the corresponding calculation formula is:

[0095]

[0096] in, This represents the simplified multispectral features; Indicates the activation function; Indicates fully connected layer operations; Represents a compressed vector;

[0097] The enhanced frequency domain features are obtained by dot product with the compressed vector, denoted as . .

[0098] It can be explained that the multispectral attention map is obtained by simplifying the compressed vector, which is the simplified multispectral feature. By dot-productting the compressed vector with the multispectral attention map, the enhanced rich frequency domain feature information is obtained, i.e. .

[0099] Step S224: Using counterfactual analysis, the unbiased features after weakening background interference are weakened to establish a 6D pose estimation model.

[0100] It is explained that counterfactual analysis can effectively weaken the influence of background interference factors, thereby proposing purer and more unbiased features; that is, by constructing constructive scenarios as counterfactual scenarios to analyze the changes in the performance and characteristics of data under different conditions, we can better understand the true characteristics of the data, avoid the bias caused by background interference, and obtain more accurate and reliable analysis results.

[0101] Step S23: Design a loss function to train the 6D pose estimation model and obtain the trained 6D pose estimation model.

[0102] Furthermore, step S23 includes:

[0103] Step S231: Introduce a target segmentation module based on high-frequency features. The loss function includes 3D keypoint loss, target binary classification loss, difference loss generated in counterfactual analysis, and semantic segmentation loss.

[0104] Step S232: Construct a loss function to train the 6D pose estimation model, obtaining the trained 6D pose estimation model. The corresponding calculation formula is as follows:

[0105]

[0106] in, Represents the loss function; These correspond to 3D keypoint loss, target binary classification loss, difference loss generated in counterfactual analysis, and semantic segmentation loss, respectively. , All of these represent weighting coefficients.

[0107] It can be explained that the difference loss generated in counterfactual analysis refers to minimizing the difference loss between the counterfactual analysis and the features generated by the counterfactual analysis. Semantic segmentation loss uses binary cross-entropy loss to constrain the prediction results of semantic segmentation. The binary cross-entropy loss calculates a total loss value by comparing the predicted probability of each pixel with the true label point by point. Then, the model parameters are continuously adjusted through the backpropagation algorithm to minimize the loss. Preferably, in this embodiment, , The corresponding values ​​are set to 1, 1, 0.25, and 1, respectively.

[0108] Specifically, for the SPEED dataset, DarkNet-53 and ResNet18, pre-trained on ImageNet, were used as the backbone during the training phase. The network optimization adopted the Stochastic Gradient Descent (SGD) optimizer with an initial learning rate of 1e-3, momentum of 0.9, weight decay of 1e-4, and input size of 960×960. For the SwissCube dataset, the number of minibatches was set to 8, and the input size was set to 512×512.

[0109] Furthermore, in step S3, specifically:

[0110] The features output by the trained 6D pose estimation model are input into the anchor-based keypoint detector to predict 3D keypoints. The PnP algorithm is used to solve the 6D pose of the non-cooperative target to obtain the prediction result, thus realizing the pose estimation of the non-cooperative target.

[0111] The explanation is as follows: The anchor-based keypoint detector uses a sophisticated convolutional neural network structure to perform multi-level feature extraction and fusion on the features output by the trained 6D pose estimation model to accurately predict the 3D keypoints of non-cooperative targets of the spacecraft. Subsequently, the PnP (Perspective-n-Point) algorithm is used to efficiently solve the 6D pose of the non-cooperative target in three-dimensional space, including rotation and translation information, in combination with the 3D keypoints, so as to achieve accurate pose estimation of the non-cooperative target.

[0112] To better illustrate this, in practical applications, the 6D pose estimation model can be deployed in the system environment where it is needed. A deep learning environment can be built using Python and deep learning libraries such as PyTorch. The latest Python version 3.8.2 can be used, and PyTorch 1.12.0 can be used as a dependency library for implementing deep learning algorithms.

[0113] Understandably, the 6D pose estimation model constructed in this application uses DarkNet-53 to extract low-frequency features from image data, combines them with edge images as input images for corresponding processing, thereby enhancing the high-frequency attention of CNN (Convolutional Neural Network); it uses ResNet18 to extract multi-scale high-frequency features, and uses the extracted features for non-cooperative target semantic segmentation; unlike existing methods that directly use Transformer as the baseline backbone to extract features, this application uses a Transformer-based feature interaction module to perform high-frequency and low-frequency attention on spacecraft images, thereby achieving the aggregation of long-range, long-distance contextual features; and the 6D pose estimation model has the characteristics of lightweight deployment and high algorithm efficiency, which can improve the efficiency and accuracy of non-cooperative spacecraft target pose estimation, and can be applied to 6D relative pose estimation for close-range spacecraft, facilitating autonomous spacecraft docking.

[0114] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0115] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

Claims

1. A method for estimating the 6D pose of a non-cooperative target based on multi-spectral feature interaction, characterized in that, The method includes: Acquire image data and preprocess it to create a training dataset; Low-frequency and high-frequency features are obtained from the training dataset and then interactively constructed to form a 6D pose estimation model. A loss function is designed to train the 6D pose estimation model. Pose prediction is performed using a trained 6D pose estimation model, and the prediction results are output to achieve pose estimation for non-cooperative targets.

2. The non-cooperative target 6D pose estimation method based on multi-spectral feature interaction according to claim 1, characterized in that, Acquire image data and preprocess it to build a training dataset, including: Image data was obtained using the publicly available SwissCube and SPEED pose estimation datasets; A training dataset is established by preprocessing image data using mosaic data augmentation and adaptive image scaling.

3. The non-cooperative target 6D pose estimation method based on multi-spectral feature interaction according to claim 1, characterized in that, Low-frequency and high-frequency features were obtained from the training dataset and then interactively used to construct a 6D pose estimation model. A loss function was designed to train the 6D pose estimation model, including: High-frequency and low-frequency features of any image data in the training dataset are extracted using ResNet18 and DarkNet53 respectively. The Transformer-based feature interaction module is designed to perform attention to high-frequency and low-frequency features and guide feature fusion. Counterfactual analysis is used to establish a 6D pose estimation model. The loss function is designed to train the 6D pose estimation model, resulting in the trained 6D pose estimation model.

4. The non-cooperative target 6D pose estimation method based on multi-spectral feature interaction according to claim 3, characterized in that, High-frequency and low-frequency features of any image data in the training dataset are extracted using ResNet18 and DarkNet53 respectively, as follows: The corresponding edge image is obtained based on any image data. The edge image is then concatenated with the image data to form the input image. ResNet18 is used to extract the high-frequency features of the input image, denoted as... The image data was processed using DarkNet53 to obtain the corresponding low-frequency features, denoted as... .

5. The non-cooperative target 6D pose estimation method based on multi-spectral feature interaction according to claim 3, characterized in that, The Transformer-based feature interaction module is designed to perform attention to high-frequency and low-frequency features and guide feature fusion. Counterfactual analysis is used to establish a 6D pose estimation model, including: Position encoding is performed on high-frequency and low-frequency features respectively to obtain the corresponding two-class flattened high-frequency and low-frequency features. Feature interaction is performed through attention blocks, and the low-frequency features are updated with information retrieved from the high-frequency features. The high-frequency and low-frequency features after interaction are output accordingly. The design frequency-guided feature fusion module guides the fusion of high-frequency and low-frequency features after interaction, splices the high-frequency and low-frequency features after interaction, divides them into multiple sub-features, assigns a corresponding two-dimensional DCT frequency classification to each sub-feature, and connects them to obtain a compressed vector. The enhanced frequency domain features are obtained by simplifying the compressed vector and then dot-producting it with the compressed vector. A 6D pose estimation model is established by using counterfactual analysis to weaken the unbiased features after background interference.

6. The non-cooperative target 6D pose estimation method based on multi-spectral feature interaction according to claim 5, characterized in that, The high-frequency and low-frequency features after concatenation and interaction are divided into multiple sub-features. Each sub-feature is assigned a corresponding two-dimensional DCT frequency classification, and the sub-features are concatenated to obtain a compressed vector, including: The high-frequency and low-frequency features after the interaction are denoted as follows: and After concatenation, it is divided into multiple sub-features, denoted as... ,in, The size is , , , Indicates the batch size of the sub-features; Indicates the number of sub-features; , , These represent the height, width, and number of channels, respectively. This indicates the sparsity of sub-features; For each sub-feature, a corresponding two-dimensional DCT frequency classification is assigned, and the corresponding calculation formula is: ； in, Indicates the first The compressed vector of each sub-feature; Represents two dimensions; express The corresponding frequency component 2D index; , All indicate the first Individual characteristics; Describe the basis functions of the DCT; The concatenation yields a compressed vector, and the corresponding calculation formula is: ； in, Represents a compressed vector; Indicates a splicing operation; These respectively represent the compressed vectors of the sub-features.

7. The non-cooperative target 6D pose estimation method based on multi-spectral feature interaction according to claim 5, characterized in that, The enhanced frequency domain features are obtained by simplifying the compressed vector and then dot-producting it with the compressed vector, including: Based on the simplification of the compressed vector, the corresponding calculation formula is: ； in, This represents the simplified multispectral features; Indicates the activation function; Indicates fully connected layer operations; Represents a compressed vector; The enhanced frequency domain features are obtained by dot product with the compressed vector, denoted as . .

8. The non-cooperative target 6D pose estimation method based on multi-spectral feature interaction according to claim 3, characterized in that, The loss function is designed to train the 6D pose estimation model, resulting in the trained 6D pose estimation model, including: A target segmentation module is introduced based on high-frequency features. The loss function includes 3D keypoint loss, target binary classification loss, difference loss generated in counterfactual analysis, and semantic segmentation loss. A loss function is constructed to train the 6D pose estimation model, resulting in the trained 6D pose estimation model. The corresponding calculation formula is as follows: ； in, Represents the loss function; , , , These correspond to 3D keypoint loss, target binary classification loss, difference loss generated in counterfactual analysis, and semantic segmentation loss, respectively. , , , All of these represent weighting coefficients.

9. The non-cooperative target 6D pose estimation method based on multi-spectral feature interaction according to claim 1, characterized in that, Pose prediction is performed using a trained 6D pose estimation model, and the prediction results are output to achieve pose estimation for non-cooperative targets. Specifically: The features output by the trained 6D pose estimation model are input into the anchor-based keypoint detector to predict 3D keypoints. The PnP algorithm is used to solve the 6D pose of the non-cooperative target to obtain the prediction result, thus realizing the pose estimation of the non-cooperative target.

Images