Neural network image matching method for spacecraft rendezvous and docking optical auxiliary means

Abstract

The invention discloses a neural network image matching method for spacecraft rendezvous and docking optical auxiliary means, and the method comprises the following steps: S1, carrying out data collection and image preprocessing, and obtaining collected data and a preprocessed image; s2, performing feature matching on the acquired data and the preprocessed image according to a CtIFT algorithm to obtain a pixel-level high-precision matching point set, and outputting the pixel-level high-precision matching point set; s3, calculating the spatial relationship between the images through the high-precision matching point set, and further estimating and outputting the relative pose of the spacecraft; the problems that a traditional method and a part of deep learning methods based on an attention mechanism are limited in feature matching capacity for a low-texture region and cannot adapt to complex conditions such as view angle and illumination change in the spacecraft docking process are solved.

Description

Technical Field

[0001] The invention relates to the field of neural network image matching methods, and in particular to a neural network image matching method for optical auxiliary means of spacecraft rendezvous and docking. Background Art

[0002] Spacecraft rendezvous and docking is a critical technology in missions such as space station construction and satellite servicing, which requires two spacecraft to accurately approach and connect in orbit to form a stable structure. Highly accurate relative pose estimation is required in the process from long-distance approach to final docking. During the long-distance approach phase before the final docking phase, the relative speed of the spacecraft is high, reaching several meters per second, so it is a very challenging task to perform accurate attitude and position in a short time. At present, traditional rendezvous and docking methods mainly rely on inertial navigation systems, which use gyroscopes, accelerometers and star sensors to determine the relative position and attitude of spacecraft. Inertial navigation systems can achieve relatively robust results, but they also have certain limitations due to sensor errors, signal noise and environmental factors. These limitations are particularly critical in the final stage of rendezvous and docking, because accurate alignment is essential to ensure the safety and accuracy of the mission.

[0003] In order to address these constraints and incorporate more references and auxiliary means, computer vision (CV) technology has been introduced to enhance the relative pose estimation between spacecraft. In the field of CV, local feature extraction and automatic matching between images are the basic technologies for many 3D computer vision applications, including structure from motion (SfM), simultaneous localization and mapping (SLAM), and visual positioning tasks. As an optical image-based method, image local feature matching is closely related to the need to obtain accurate relative poses in rendezvous and docking missions. The typical matching process includes three different stages: feature detection, feature description, and feature matching. In the detection stage, salient features such as corners, endpoints, or spots are first identified in each image as key points, which are mainly used to obtain position information. Subsequently, local descriptors are generated in the neighborhood of the key points, giving these key points unique features as reference information for the subsequent matching stage. Then, correspondences are established between these points through nearest neighbor search or advanced matching techniques. Once the relationship between key points and positions is successfully established, the spatial correspondence between images can be calculated, providing a basis for subsequent spatial applications such as camera pose estimation.

[0004] Existing feature matching algorithms have several obvious defects in spacecraft rendezvous and docking missions, making it difficult to meet mission requirements. First, traditional feature matching methods, such as SIFT and SURF, mainly rely on key points and gradient descriptors of local features, and therefore perform poorly when dealing with complex scenes such as illumination changes, scale changes, and perspective shifts. During the spacecraft docking process, these algorithms find it difficult to maintain a stable matching effect. In addition, deep learning methods in recent years, such as D2-Net, SuperPoint, and LoFTR, although they perform well in some scenes, still have limitations in low-texture or highly repetitive areas, and usually require pre-detection of key points, and cannot effectively handle complex optical distortions in docking scenes.

[0005] More importantly, traditional methods and some deep learning methods based on attention mechanisms have limited feature matching capabilities for low-texture areas and cannot adapt to complex situations such as changes in perspective and illumination during spacecraft docking. For example, although LoFTR uses the attention mechanism of the Transformer structure, it is difficult to achieve consistent matching in cross-modal optical data due to the lack of support from multi-source data. Other methods, such as ASpanFormer, improve the matching effect of complex scenes by dynamically adjusting the attention range, but they do not have an advantage in real-time performance and are difficult to achieve the fast response speed required for docking tasks.

[0006] The present invention proposes a Class-Tuned Invariant Feature Transformer (CtIFT) network algorithm based on deep learning, which aims to improve the accuracy and robustness of optical image feature matching in spacecraft rendezvous and docking missions, and provide an effective optical assistance method. CtIFT achieves high-precision feature matching under different lighting and viewing angle change conditions through an improved symmetrical multi-layer cross-attention mechanism module. In addition, CtIFT introduces a feature matching method that integrates feature classification modules to filter out invalid features, enhance the directionality of feature matching, reduce the interference of invalid features, and effectively reduce the computational load, thereby improving the real-time performance and processing efficiency of the algorithm. By training on multi-source optical image data, the algorithm demonstrates excellent scene generalization and the ability to handle complex image distortions, thereby providing stable and efficient technical support for spacecraft docking missions. Summary of the invention

[0007] In order to solve the above technical problems, the present invention provides a neural network image matching method for optical auxiliary means of spacecraft rendezvous and docking, which solves the problem that traditional methods and some deep learning methods based on attention mechanism have limited feature matching capabilities in low-texture areas and cannot adapt to complex situations such as perspective and illumination changes during spacecraft docking.

[0008] The technical solution of the present invention is as follows:

[0009] The neural network image matching method for optical assistance means of spacecraft rendezvous and docking includes the following steps:

[0010] Step S1: data acquisition and image preprocessing, obtaining acquired data and preprocessed images;

[0011] Step S2: According to the CtIFT algorithm, feature matching is performed on the collected data and the preprocessed image to obtain and output a pixel-level high-precision matching point set;

[0012] Step S3: Calculate the spatial relationship between images through a set of high-precision matching points, and further estimate the relative position of the spacecraft and output it.

[0013] Preferably, the data collection in step S1 includes: a reference image and a real-time docking image.

[0014] Preferably, the image preprocessing in step S1 includes the following sub-steps:

[0015] Sub-step S11: performing normalization processing on the image to obtain a normalized image;

[0016] Sub-step S12: performing noise reduction processing on the normalized image to obtain a noise-reduced image;

[0017] Sub-step S13: rescaling the denoised image to obtain a rescaled image.

[0018] Preferably, step S2 includes the following sub-steps:

[0019] Sub-step S21: Perform multi-scale feature extraction on the collected data and pre-processed images. Combining the self-attention mechanism and the cross-attention mechanism, the model can generate feature expressions under complex lighting, viewing angles and low-texture scenes;

[0020] Sub-step S22: performing a rough matching phase on the feature expression to obtain matching points;

[0021] Sub-step S23: Perform a fine matching phase on the matching points to obtain a high-precision matching set.

[0022] Preferably, step S3 includes the following sub-steps:

[0023] Sub-step S31: combining the high-precision matching point set with the internal and external parameters of the camera, using the RANSAC algorithm to eliminate the mismatched points, calculating the transformation matrix, and then estimating the relative posture parameters between the spacecraft;

[0024] Sub-step S32: Combining the internal and external parameters of the camera and the calculation results, the relative posture parameters between the spacecraft are output in real time to provide key decision support for the control and navigation system during the docking process.

[0025] The technical effects of the neural network image matching method of the optical auxiliary means for spacecraft rendezvous and docking of the present invention are as follows:

[0026] 1. Significant improvement in matching accuracy

[0027] In four typical spacecraft docking scenarios, CtIFT's average number of correctly matched feature points (NCM) reached hundreds per frame, and even reached thousands in some scenarios, significantly surpassing other existing methods. For example, in experiments, traditional SIFT and some convolutional neural networks (such as D2-Net) could not successfully match under complex visual conditions, and SuperPoint+SuperGlue had a matching rate of only 17.6% in specific scenarios, while CtIFT's successful matching rate (SR) in all scenarios remained at 100%. These data show that CtIFT has extremely high matching accuracy in complex environments and has good generalization ability.

[0028] 2. Improved real-time processing capabilities

[0029] CtIFT performs well in terms of processing speed. On the NVIDIA RTX 4090D (24GB) GPU, 16-core Intel(R) Xeon(R) Platinum 8481C CPU platform, the average processing time per frame is less than 0.18 seconds, which can meet the requirements of real-time processing. CtIFT's feature classification module greatly reduces the computational load while ensuring high accuracy, thereby significantly improving processing efficiency. This feature makes CtIFT particularly suitable for real-time feature matching in rendezvous and docking tasks.

[0030] On the NVIDIA JetsonAGX Xavier, GPU 22TOPS (INT8), 2.3TFLOPS (FP32), 137.6GB / s (256-bit LPDDR4x) platform, CtIFT can achieve sub-second processing time per frame, indicating that the algorithm has the ability to be deployed on lightweight devices such as mobile and airborne. This feature enables CtIFT to be deployed in real-time feature matching in rendezvous and docking missions.

[0031] 3. Improved robustness in complex environments

[0032] CtIFT enhances its adaptability to complex visual scenes (such as illumination changes, scale changes, occlusion, etc.) through training with multi-source data and improved attention mechanisms. For example, in scenes with high noise and low texture, CtIFT's symmetrical multi-layer cross attention mechanism enables it to capture global information and maintain the correlation between features, thereby achieving high-precision matching under these conditions. Unlike traditional local feature detectors, CtIFT's detector-free architecture and Transformer-based attention mechanism perform particularly well in multimodal data.

[0033] 4. Optimized energy consumption and computing efficiency

[0034] CtIFT effectively filters irrelevant features through the feature classification module, reducing the energy consumption and burden of matching calculations. In the spacecraft docking mission, since most image areas do not contain matching features, the feature classification module of CtIFT can eliminate irrelevant features in the preprocessing stage, thereby greatly reducing the amount of calculation. Compared with traditional methods and some convolutional networks that require point-by-point calculations, CtIFT can complete the same task with lower energy consumption and is suitable for long-term space missions.

[0035] 5. Improved task adaptability and versatility

[0036] The present invention is trained on multi-source optical data, so that CtIFT can maintain high matching accuracy under different conditions such as illumination, radiation distortion and viewing angle changes. Experimental data show that CtIFT can also achieve a 100% success rate in complex simulated docking video data, which shows that it has strong task adaptability. In addition, CtIFT can be applied to a variety of complex scenes and visual matching tasks, and has strong versatility. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is the algorithm flow chart of the present invention. DETAILED DESCRIPTION

[0038] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0039] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

[0040] like Figure 1As shown, the neural network image matching method for optical auxiliary means of spacecraft rendezvous and docking includes the following steps:

[0041] Step S1: data acquisition and image preprocessing, obtaining acquired data and preprocessed images;

[0042] Step S2: According to the CtIFT algorithm, feature matching is performed on the collected data and the preprocessed image to obtain and output a pixel-level high-precision matching point set;

[0043] Step S3: Calculate the spatial relationship between images through a set of high-precision matching points, and further estimate the relative position of the spacecraft and output it.

[0044] The data collection in step S1 of this embodiment includes: a reference image and a real-time docking image.

[0045] The image preprocessing in step S1 of this embodiment includes the following sub-steps:

[0046] Sub-step S11: performing normalization processing on the image to obtain a normalized image;

[0047] Sub-step S12: performing noise reduction processing on the normalized image to obtain a noise-reduced image;

[0048] Sub-step S13: rescaling the denoised image to obtain a rescaled image.

[0049] Step S2 of this embodiment includes the following sub-steps:

[0050] Sub-step S21: Perform multi-scale feature extraction on the collected data and pre-processed images. Combining the self-attention mechanism and the cross-attention mechanism, the model can generate feature expressions under complex lighting, viewing angles and low-texture scenes;

[0051] Sub-step S22: performing a rough matching phase on the feature expression to obtain matching points;

[0052] Sub-step S23: Perform a fine matching phase on the matching points to obtain a high-precision matching set.

[0053] Step S3 of this embodiment includes the following sub-steps:

[0054] Sub-step S31: combining the high-precision matching point set with the internal and external parameters of the camera, using the RANSAC algorithm to eliminate the mismatched points, calculating the transformation matrix, and then estimating the relative posture parameters between the spacecraft;

[0055] Sub-step S32: Combining the internal and external parameters of the camera and the calculation results, the relative posture parameters between the spacecraft are output in real time to provide key decision support for the control and navigation system during the docking process.

[0056] When this implementation plan is implemented,

[0057] This invention proposes a Class-Tuned Invariant Feature Transformer (CtIFT) algorithm based on deep learning, focusing on solving the feature matching problem of complex optical imaging in spacecraft rendezvous and docking missions. The technical solution is mainly divided into three parts: data acquisition and preprocessing, feature matching based on the CtIFT algorithm, and pose estimation and output.

[0058] 1. Data collection and preprocessing

[0059] Data Entry

[0060] The input of this technical solution includes two types of images:

[0061] Reference image: A high-resolution target reference image captured in a standard posture, containing the feature distribution of the docking target, as a matching benchmark.

[0062] Real-time docking image: A dynamic image collected in real time during the docking process, containing target information under complex conditions such as lighting, viewing angle, and scale changes.

[0063] Image preprocessing

[0064] To ensure that the input image meets the requirements of subsequent algorithms, the following preprocessing methods are used:

[0065] Normalization: Adjust the pixel intensity range of an image to improve data consistency.

[0066] Noise reduction: Use filtering technology to remove random noise in the image.

[0067] Scaling: Uniformly scale images of different resolutions to ensure consistency during model training and inference.

[0068] 2. Feature matching based on CtIFT algorithm

[0069] Through the proposed Class-Tuned Invariant Feature Transformer neural network, the input preprocessed image is subjected to four sequential modules of multi-scale feature extraction-feature enhancement representation-coarse matching-fine matching to extract and match local features, and a high-precision matching point set at the pixel level is obtained and output.

[0070] 3. Pose estimation and output

[0071] After completing feature matching, the spatial relationship between images is calculated through a high-precision matching point set, and the relative pose of the spacecraft is further estimated. First, the matching point set is combined with the internal and external parameters of the camera, and the RANSAC algorithm is used to eliminate the mismatched points, calculate the transformation matrix, and then estimate the relative pose (including position and direction) between the spacecraft. Finally, combined with the internal parameters of the camera and the calculation results, the relative pose parameters of the spacecraft are output in real time, providing key decision support for the control and navigation system during the docking process.

[0072] Example 1

[0073] The specific implementation of the present invention includes three steps: data collection and preprocessing, CtIFT algorithm for feature matching, and pose estimation and output:

[0074] Data collection and preprocessing

[0075] In the spacecraft rendezvous and docking mission, a high-resolution optical camera is used to obtain reference images and real-time docking images. The reference image is collected in advance under standard attitude conditions and contains the characteristic distribution of the docking target as a benchmark for matching. The real-time docking image is collected in real time during the dynamic docking process.

[0076] In order to ensure that the input data can meet the processing requirements of the CtIFT algorithm, the image is first preprocessed. The pixel values ​​of all images are normalized to the range of [0,1] to eliminate the influence of light intensity; then, Gaussian filtering is used to remove noise in the image; finally, the width and height of the image are uniformly adjusted to multiples of 8 to meet the input requirements of the CtIFT model.

[0077] CtIFT algorithm for feature matching

[0078] After the preprocessing is completed, the CtIFT algorithm is used to perform feature matching between the reference image and the real-time docking image. The specific steps are as follows:

[0079] 2.1 Multi-scale feature extraction

[0080] Through the multi-scale feature extraction module based on ResUNet, feature maps of 1 / 2, 1 / 4 and 1 / 8 scales are generated for the input image. These feature maps capture information from local details to global structures and adapt to possible scale changes and perspective shifts during the docking process. In the experiment, the 1 / 8 scale feature map performed best in capturing complex scene changes and was further optimized for matching.

[0081] 2.2 Feature Enhanced Representation

[0082] Based on the extracted multi-scale features, the present invention uses the GIFE (global invariant feature encoding) module to further process the features. This module combines the self-attention mechanism and the cross-attention mechanism. The self-attention mechanism is used to capture the long-range dependencies and global context information within a single image, thereby enhancing the expressiveness of the features. Subsequently, the feature interaction between the two images is realized through the cross-attention mechanism, so that the features are associated layer by layer and the matching correlation is enhanced. Through the alternating self-attention and cross-attention, the model can generate more robust and discriminative feature expressions in complex lighting, viewing angles and low-texture scenes.

[0083] 2.3 Rough Matching Stage

[0084] In the rough matching stage, preliminary matching points are selected through feature classification and dual-softmax operation:

[0085] First, feature classification is performed in combination with the feature classification module to divide the extracted features into valid features and invalid features. Valid features enter the matching process, and invalid features are eliminated. Then, the confidence matrix is ​​generated through the dual-softmax operation to screen high-confidence matching points.

[0086] 2.4 Fine matching stage

[0087] Based on the rough matching results, a small area (such as 5×5 pixels) centered on each rough matching point is generated, and the lightweight GIFE module is used to enhance the feature representation in these areas. Finally, the position of the matching point is optimized through uncertainty calculation to form a high-precision matching set.

[0088] Pose estimation and output

[0089] After completing feature matching, the spatial relationship between images is calculated through a set of high-precision matching points, and the relative position of the spacecraft is further estimated:

[0090] First, the RANSAC algorithm is used to eliminate mismatched points by combining the set of matching points with the internal and external parameters of the camera, and the transformation matrix is ​​calculated to estimate the relative posture (including position and direction) between the spacecraft. Finally, the relative posture parameters of the spacecraft are output in real time by combining the internal parameters of the camera and the calculation results, providing key decision support for the control and navigation system during the docking process.

[0091] The key points of the present invention are:

[0092] 1. Symmetric multi-layer cross attention mechanism

[0093] The CtIFT algorithm of the present invention introduces a symmetrical multi-layer cross-attention mechanism. This mechanism can gradually enhance the interrelationships between image features at different levels, allowing the algorithm to maintain the accuracy and robustness of feature matching in complex scenes such as illumination changes, perspective changes, and low-texture areas. This design significantly improves the adaptability of the algorithm in spacecraft rendezvous and docking missions.

[0094] 2. Feature classification module

[0095] The feature classification module of CtIFT adopts a multi-label classification strategy to classify the extracted features into "valid features" and "invalid features". This module filters out features that are not relevant to the match, thereby reducing the computational load and improving the accuracy and real-time performance of the match. This function is particularly critical in spacecraft docking missions because most image areas do not contain significant texture features.

[0096] 3. Multi-source data training for spacecraft docking missions

[0097] In order to enhance the adaptability of the model in complex docking scenarios, the present invention trains CtIFT on multi-source optical data to enable it to cope with different radiation distortions and scale changes. By training on multi-source datasets with similar distortions, CtIFT can adapt to typical visual environment changes during spacecraft docking, showing strong generalization capabilities. At the same time, the present invention introduces the optical image feature matching method based on deep learning into the spacecraft rendezvous and docking mission for the first time, providing a new technical perspective and solution for related research and engineering fields.

Claims

1. A neural network image matching method for optical assistance of spacecraft rendezvous and docking, characterized in that: The following steps are involved: Step S1: data acquisition and image preprocessing, obtaining acquired data and preprocessed images; Step S2: According to the CtIFT algorithm, feature matching is performed on the collected data and the preprocessed image to obtain and output a pixel-level high-precision matching point set; Step S3: Calculate the spatial relationship between images through a set of high-precision matching points, and further estimate the relative position of the spacecraft and output it.

2. The neural network image matching method for optical assistance of spacecraft rendezvous and docking according to claim 1 is characterized in that: The data collection in step S1 includes: a reference image and a real-time docking image.

3. The neural network image matching method for optical assistance of spacecraft rendezvous and docking according to claim 1, characterized in that: The image preprocessing in step S1 includes the following sub-steps: Sub-step S11: performing normalization processing on the image to obtain a normalized image; Sub-step S12: performing noise reduction processing on the normalized image to obtain a noise-reduced image; Sub-step S13: rescaling the denoised image to obtain a rescaled image.

4. The neural network image matching method for optical assistance of spacecraft rendezvous and docking according to claim 1, characterized in that: The step S2 comprises the following sub-steps: Sub-step S21: Perform multi-scale feature extraction on the collected data and pre-processed images. Combining the self-attention mechanism and the cross-attention mechanism, the model can generate feature expressions under complex lighting, viewing angles and low-texture scenes; Sub-step S22: performing a rough matching phase on the feature expression to obtain matching points; Sub-step S23: Perform a fine matching phase on the matching points to obtain a high-precision matching set.

5. The neural network image matching method for optical assistance of spacecraft rendezvous and docking according to claim 1, characterized in that: The step S3 comprises the following sub-steps: Sub-step S31: combining the high-precision matching point set with the internal and external parameters of the camera, using the RANSAC algorithm to eliminate the mismatched points, calculating the transformation matrix, and then estimating the relative posture parameters between the spacecraft; Sub-step S32: Combining the internal and external parameters of the camera and the calculation results, the relative posture parameters between the spacecraft are output in real time to provide key decision support for the control and navigation system during the docking process.

Images