Non-cooperative space target relative pose estimation method and system based on monocular vision

Abstract

The application discloses a non-cooperative space target relative pose estimation method and system based on monocular vision, comprising: pre-acquiring space image data sets, training and reasoning, and obtaining pre-training weights; using a multi-level convolution feature extraction network for reasoning, and outputting key point coordinates, key point confidence, key point descriptors and key point brightness information; constructing a dynamic space weight matching constraint matrix in the form of Gaussian distribution; inputting an image sequence pair to be matched into a feature matching algorithm, and outputting a preliminary matching point pair; through key point brightness information, the preliminary matching point pair is post-processed and optimized to obtain a fine feature matching point pair; and the fine feature matching point pair is input through geometric verification and model expansion, and the relative pose between space target sequence frames is solved. The application can realize stable sparse three-dimensional reconstruction of a non-cooperative space target monocular image sequence and high-precision solving of the relative pose between frames without cooperative identification and manual annotation.

Description

Technical Field

[0001] This invention relates to the field of space target perception technology, and in particular to a method and system for estimating the relative pose of non-cooperative space targets based on monocular vision. Background Technology

[0002] Space target relative pose estimation is one of the core technologies of space optoelectronic measurement and aerospace sensing, with applications covering on-orbit servicing (OOS), active debris removal, on-orbit docking and maintenance, close-range mapping, and formation flying. With the rapid growth of commercial spaceflight and on-orbit servicing demands, the requirements for autonomous sensing, close-range navigation, and precise maneuvering of servicing spacecraft are becoming increasingly stringent. Monocular cameras, due to their small size, low power consumption, and light weight, have become one of the most attractive sensing solutions for various mission phases such as hovering, fly-around, and approach. This is accompanied by a strong market and engineering demand for algorithms that can operate in real-time or near-real-time on-site and provide high-precision inter-frame pose estimation even under conditions of extremely limited or unavailable prior information. These algorithms must balance the limitations of onboard computing resources with robustness under extreme lighting, strong radiation, and complex target surface materials.

[0003] Currently, technological development in this field presents two main routes: On the one hand, there are traditional measurement / visual methods based on accurate models or cooperative identifiers, suitable for targets with known models or pre-installed identifiers, capable of high-precision measurement using structured light, LiDAR, or calibration geometry; however, these methods lose their applicability when facing non-cooperative targets with unknown models. On the other hand, there are image-based visual 3D reconstruction and learning-based feature processing methods. Among these, text-based hand-designed features such as Harris, SIFT, and ORB, and nearest neighbor (NN)-based matching methods perform stably in everyday scenarios, but in aerospace scenarios, due to low lighting, low signal-to-noise ratio, surface specular reflection, and sparse texture, keypoint detection and descriptor discrimination rates decrease significantly, leading to missed detections, mismatches, and reconstruction failures. In recent years, deep learning technologies, such as the well-regarded self-supervised detection and description framework, have made breakthroughs in general vision tasks. However, directly transferring them to spatial target scenes still faces challenges such as insufficient data, difficulty in obtaining real pose annotations, and generalization problems caused by domain differences. Meanwhile, some learning-based advanced matchers, such as graph network-based methods, have improved matching quality, but they often have high computational and storage costs, making it difficult to meet real-time and resource constraints on spaceborne platforms.

[0004] Furthermore, while existing benchmarks and competitions, such as several public challenges and datasets, have driven research, most methods rely on prior information provided by the competitions or manually labeled pairings in evaluation or implementation, thus masking the problems faced by algorithms in real-world, unlabeled, non-cooperative environments. 3D reconstruction pipelines such as Colmap perform well in everyday ground scenes with traditional features like SIFT / NN and under sufficient lighting conditions. However, in space with low illumination, highly symmetrical structures, and limited target textures, the stability of traditional features is insufficient, and the outlier rate of matching is high, leading to sparse reconstruction failures or excessively sparse point clouds, ultimately failing to guarantee accurate inter-frame relative pose estimation. Moreover, spatial targets often possess high symmetry and repetitive textures; a nearest neighbor strategy based solely on local descriptors is prone to many-to-one or mismatch problems, while matchers based on dense meshes or complex graph optimizations may introduce computational bottlenecks, making them unsuitable for real-time spaceborne operation.

[0005] Therefore, current technologies have several significant shortcomings: First, they lack the ability to detect key points and discriminate descriptors in low-light, low-signal-to-noise-ratio regions, resulting in substandard quality of basic data input for 3D reconstruction. Second, they lack training strategies and synthetic data generation methods that can effectively train and transfer data to space scenes without pose ground truth and manual annotation. Third, feature matching algorithms do not fully utilize temporal continuity and motion smoothness when processing sequential flyby or hovering images, making it difficult to suppress outlier matching and structural misjudgments. Fourth, they lack efficient post-processing mechanisms that combine brightness information and spatial priors to filter out low-confidence matches, thus failing to meet the trade-off between accuracy and resources on spaceborne platforms. At the market and engineering levels, there is an urgent need for end-to-end or modular visual perception solutions that can operate in resource-constrained spaceborne computing environments without manual intervention and have good generalization capabilities for various non-cooperative targets, in order to support an increasing number of on-orbit services and debris management tasks. Summary of the Invention

[0006] In view of this, the purpose of this invention is to provide a method and system for estimating the relative pose of non-cooperative spatial targets based on monocular vision. By constructing a robust feature extraction and sequence constraint feature matching mechanism for low-light and prior information-deficient scenarios, it is possible to achieve stable sparse 3D reconstruction of non-cooperative spatial target monocular image sequences and high-precision calculation of inter-frame relative pose without the need for cooperative labeling and manual annotation.

[0007] In a first aspect, embodiments of the present invention provide a method for estimating the relative pose of a non-cooperative spatial target based on monocular vision, comprising:

[0008] S100 involves pre-collecting synthetic image datasets and public spatial target datasets, and then training and inferring using a pre-trained feature extraction model to obtain pre-trained weights.

[0009] S200, based on the pre-trained weights, a multi-level convolutional feature extraction network is used to infer the input spatial target sequence, and the key point coordinates, key point confidence and key point descriptors of each image frame are output as spatial target features, while the key point brightness information is also output.

[0010] S300, based on the assumption of inter-frame motion smoothness of the spatial target features, construct a dynamic spatial weight matching constraint matrix in Gaussian distribution form.

[0011] S400: Input the image sequence pairs to be matched into the feature matching algorithm, perform the initial matching using the nearest neighbor algorithm, calculate the basic similarity matrix using the key point descriptor, calculate the dynamic spatial weight value of each pair of matching points using the dynamic spatial weight matching constraint matrix, and output the preliminary matching point pairs in combination with the scene confidence level judgment.

[0012] S500, using the brightness information of the key points, perform brightness consistency verification and anomaly suppression on the preliminary matching point pairs to obtain fine feature matching point pairs.

[0013] S600, the fine feature matching point pairs are input into the Colmap 3D reconstruction architecture, and the relative poses between the spatial target sequence frames are calculated through geometric verification and model expansion.

[0014] In conjunction with the first aspect, this embodiment of the invention provides a first possible implementation of the first aspect, wherein, in S100, the process of obtaining the pre-trained weights includes:

[0015] S110, pre-acquire synthetic image datasets and public spatial target datasets.

[0016] S120 utilizes a low-light scene training strategy to perform the first round of training, inference phase, and second round of training for the feature extraction model.

[0017] S130, in the first and second rounds of training, adaptive specular reflection enhancement processing is used to enhance the selected high-brightness areas. Random sampling of the specular reflection light source position is performed, and a Gaussian function is used to simulate the light intensity attenuation and generate a brightness distribution function. , will reflect the component The enhanced target region brightness is output by weighted fusion with the normalized image and then subjected to gamma correction. , wherein the reflection component The calculation formula is: , For gamma correction parameters, Fill the target with texture. The fusion weights are the original image and the reflected image.

[0018] S140, In the second round of training, a phased illumination enhancement preprocessing strategy is adopted to perform primary and secondary enhancement on the original input image I.

[0019] In conjunction with the first aspect, this embodiment of the invention provides a second possible implementation of the first aspect, wherein, in S110, the generation process of the synthetic image dataset includes a background generation stage, a foreground rendering stage, and a post-processing stage.

[0020] The background generation stage generates a simulated spatial background with a preset grayscale range based on the spatial background information in the publicly available spatial target dataset, and sets the contrast between the grayscale difference between the foreground target and the background.

[0021] The foreground rendering stage generates a target fill texture with controllable lighting stripes and fractal noise. The calculation formula is: ,in, The dynamic reference grayscale of the foreground target is based on the background grayscale. For the light stripe section, The dynamic strength coefficient, This is the noise term for fractal Brownian motion (FBM).

[0022] The post-processing stage adds zero-mean random Gaussian noise to the entire image.

[0023] In conjunction with the first aspect, this embodiment of the invention provides a third possible implementation of the first aspect, wherein, in S200, the process of reasoning about the input spatial target sequence using a multi-level convolutional feature extraction network includes:

[0024] S210, Construct a multi-level convolutional feature extraction network, which includes a multi-level convolutional feature encoder, a Transformer attention module, a multi-task output head, and a post-processing module.

[0025] S220 inputs the spatial target sequence into a multi-level convolutional feature extraction network, and extracts core features and multi-scale high-level information through a multi-level convolutional feature encoder.

[0026] S230, the core features are enhanced by positional encoding and input into the Transformer attention module to generate global semantic enhancement features.

[0027] S240, the global semantic enhancement features are input into the multi-task output head, key points are detected by the key point heatmap decoder, descriptors are generated by the descriptor decoder, and the original output results of the network are output.

[0028] S250, the original output of the network enters the post-processing module, which outputs key point coordinates, key point confidence, key point descriptors, and key point brightness information.

[0029] This invention utilizes a self-designed, supervised multi-level convolutional feature extraction network (DarkPoint network, an improved model based on the typical SuperPoint architecture, with enhanced backbone depth, attention mechanism, and multi-task output head optimization for low-light spatial environments) to effectively extract key points in low signal-to-noise ratio regions on the SHIRT public spatial target dataset. Compared to the SuperPoint network, it achieves a 31% improvement in registration rate in the semi-physical simulation image domain, a 386.6% increase in effective point cloud density, and a 50.31% reduction in reconstruction pose error. Ablation experiments further validate the enhancement effects of the Transformer attention module, high-level feature expansion module, and the proposed low-light scene training strategy (DT training strategy, whose core is to improve the model's robustness to extreme lighting conditions without prior information through synthetic texture generation and adaptive specular reflection enhancement). This invention provides a highly robust feature extraction scheme for spatial target relative pose estimation tasks, addressing to some extent the two core challenges of missing prior information for non-cooperative targets and feature degradation in low-light scenes.

[0030] In conjunction with the first aspect, this embodiment of the invention provides a fourth possible implementation of the first aspect, wherein, in S250, the process by which the post-processing module constructs the key point brightness information includes:

[0031] The image of the original output result of the network Generate normalized offset The calculation formula is:

[0032] ,

[0033] in, For the defined neighborhood window size, Image width, This represents the image height.

[0034] For each key point Sampling neighborhood brightness The calculation formula is:

[0035] ,

[0036] in, For the current batch, This is the current key point number. The index of the pixel within the neighborhood window. and This is the normalized offset.

[0037] The brightness of the current key point is calculated using Gaussian weighting. The calculation formula is:

[0038] ,

[0039] in, These are Gaussian weighting coefficients, calculated using the following formula:

[0040] ,

[0041] in, To control the standard deviation of the weight distribution.

[0042] In conjunction with the first aspect, this embodiment of the invention provides a fifth possible implementation of the first aspect, wherein, in S300, the formula for constructing the Gaussian-distributed dynamic spatial weight matching constraint matrix is:

[0043] ,

[0044] Among them, The difference in feature point coordinates. The formula for calculating the dynamically adjusted Gaussian kernel standard deviation is as follows: , The time difference between the two frames. Based on the scope of the basic spatial constraints, For time constraint parameters, For the maximum reference frame interval, This is the maximum scaling factor.

[0045] Specifically, by analyzing the characteristics of space targets, the following hypothesis on inter-frame motion smoothness is proposed: the displacement of the three-dimensional feature points of the target involved in the space target image sequence is small between adjacent frames, and increases with the increase of frame interval (increasing time difference); the short-term motion of the space target relative to the servicing spacecraft has no abrupt characteristics, the offset of feature points within several adjacent frames approximately follows a normal distribution, and its standard deviation is related to the time difference (frame number difference). Based on this, combined with the inter-frame time difference... Difference with feature point coordinates Construct a dynamic space weight matching constraint matrix in Gaussian distribution form. , as a matching constraint based on spatiotemporal consistency.

[0046] In conjunction with the first aspect, this embodiment of the invention provides a sixth possible implementation of the first aspect, wherein, in S400, the feature matching algorithm includes:

[0047] S410, the nearest neighbor algorithm is used for initial matching, and the basic similarity matrix is ​​calculated using the key point descriptors. .

[0048] S420, calculate the dynamic spatial weight value of each pair of matching points using the dynamic spatial weight matching constraint matrix. .

[0049] D430 performs scene classification judgment, defining a high-confidence scene as one where the proportion of matching pairs with high initial matching confidence exceeds a preset proportion, and a low-confidence scene otherwise.

[0050] S440, for high-confidence scenarios, sets a spatial weight threshold. A spatial verification and review mechanism is used to filter the matching results and output preliminary matching point pairs.

[0051] For low-confidence scenarios, S450 employs a dual-strategy fusion method combining descriptor similarity S and spatial weight W, calculating similarity using an exponential fusion strategy. Similarity enhancement with double saliency regions Select the confidence score corresponding to the higher similarity score after normalization as the matching result, and output the preliminary matching point pair.

[0052] The index fusion strategy calculates similarity. The calculation formula is: ,in, This is the spatial weighting index.

[0053] The double salient regions enhance similarity. The calculation formula is: ,in, For the salient region of the descriptor, For spatially significant regions, It is an enhancing factor.

[0054] In conjunction with the first aspect, this embodiment of the invention provides a seventh possible implementation of the first aspect, wherein, in S500, the step of performing brightness consistency verification and anomaly suppression on the preliminary matching point pair using the key point brightness information to obtain a fine feature matching point pair includes:

[0055] S510, if the source frame's first Key points and target frame Neighborhood brightness values ​​of key matching point pairs and All below the threshold Then for the matching score To suppress it, the calculation formula is:

[0056] .

[0057] S520, after bidirectional consistency verification, yields the final matching score. The calculation formula is:

[0058] ,

[0059] in, This indicates a forward match from the source frame to the target frame. This indicates a negative match between the target frame and the source frame.

[0060] S530: Extract the set of point pairs that satisfy the positive and negative bidirectional matching consistency constraint to obtain fine feature matching point pairs.

[0061] In conjunction with the first aspect, this embodiment of the invention provides an eighth possible implementation of the first aspect, wherein, in S600, the step of inputting the fine feature matching point pairs into the Colmap 3D reconstruction architecture, and calculating the relative poses between the spatial target sequence frames through geometric verification and model expansion includes:

[0062] S610, the fine feature matching points are geometrically verified using the RANSAC algorithm in two views to remove outliers.

[0063] S620, perform incremental 3D reconstruction, select the model with the most images in multiple sets of reconstruction results as the spatial perception database of the target, and obtain the relative pose between the spatial target sequence frames.

[0064] This invention proposes a feature matching algorithm based on neural networks (NNs), which is optimized and improved in three aspects: First, for the sequence characteristics during hovering or circling phases, an inter-frame motion smoothness assumption is introduced to construct dynamic spatial weight matching constraints to suppress outlier matching errors. Second, to alleviate the problem of decreased descriptor discriminability under low illumination, a confidence-based hierarchical matching strategy is designed, and spatial weights are fused through a differentiated strategy. Third, for mismatches between symmetrical structures of spatial targets and low signal-to-noise ratio regions, a post-processing optimization algorithm based on the original brightness information of the feature point neighborhood is proposed. Finally, the proposed matching algorithm is fused with the feature extraction algorithm, and sparse reconstruction of spatial targets is achieved through the Colmap framework to obtain the relative poses between registered image sequences.

[0065] Experiments show that the proposed feature matching algorithm achieves a 100% image registration rate in the first 200 image sequences of the ROE2 trajectory in the SHIRT public spatial target dataset. Compared with SuperGlue and NN algorithms, it reduces the average pose error by 95.17% and 34.67%, respectively. Even in the extremely low-light scenario of the ROE1 trajectory, it maintains a relatively low matching error. This invention provides a robust feature matching scheme for spatial target relative pose estimation tasks, eliminating the need to rely on large-scale labeled data for model training. It addresses, to some extent, the two key problems of missing prior information for non-cooperative targets and feature degradation in low-light scenes.

[0066] Secondly, embodiments of the present invention also provide a non-cooperative spatial target relative pose estimation system based on monocular vision, comprising:

[0067] The model training and weight acquisition module is used to pre-collect synthetic image datasets and public spatial target datasets, and to train and infer pre-trained weights through a pre-trained feature extraction model.

[0068] The multi-task feature extraction module is used to infer the input spatial target sequence based on the pre-trained weights using a multi-level convolutional feature extraction network, and output the key point coordinates, key point confidence, and key point descriptors of each image frame as spatial target features, while also outputting key point brightness information.

[0069] The spatiotemporal constraint matrix construction module is used to construct a Gaussian-distributed dynamic spatial weight matching constraint matrix based on the inter-frame motion smoothness assumption of the spatial target features.

[0070] The confidence-driven hierarchical matching module is used to input the image sequence pairs to be matched into the feature matching algorithm, perform the initial matching using the nearest neighbor algorithm, calculate the basic similarity matrix through the key point descriptor, calculate the dynamic spatial weight value of each pair of matching points through the dynamic spatial weight matching constraint matrix, and output the preliminary matching point pairs in combination with the scene confidence level determination.

[0071] The brightness verification and anomaly suppression module is used to perform brightness consistency verification and anomaly suppression on the preliminary matching point pairs using the brightness information of the key points, so as to obtain fine feature matching point pairs.

[0072] The incremental 3D reconstruction and pose calculation module is used to input the fine feature matching point pairs into the Colmap 3D reconstruction architecture, and calculate the relative pose between the spatial target sequence frames through geometric verification and model expansion.

[0073] The beneficial effects of the embodiments of the present invention are:

[0074] The non-cooperative spatial target relative pose estimation method and system based on monocular vision of this invention has high versatility and practicality. The entire technical process has a clear architecture, with only the feature extraction stage involving model training. Moreover, after training once using existing datasets, it can be stably applied to various typical spatial target sequences without the need for repeated training for different targets or a large amount of manual intervention. This effectively overcomes the problem of severe lack of prior information and significantly improves the system's ease of deployment and generalization ability in real-world tasks.

[0075] In the core feature extraction stage, the DarkPoint network and its training strategy designed in this invention fundamentally improve the robustness and accuracy of the system in harsh space environments. The network enhances its understanding of the overall geometric structure of the target through a reconstructed backbone architecture and attention mechanism. Combined with a multi-scale feature fusion strategy, the generated keypoints and descriptors are insensitive to scale changes and low-light interference. The specially introduced brightness information output provides effective auxiliary constraints for subsequent matching. Coupled with a specially designed DT training strategy, by simulating extreme space lighting and surface reflection characteristics, the network is guided to learn stable, light-independent feature representations, thereby ensuring high repeatability and consistency of feature extraction under complex lighting conditions and laying a reliable foundation for subsequent processing.

[0076] In the feature matching and 3D reconstruction stages, the improved algorithm proposed in this invention significantly enhances the final accuracy and reliability of pose estimation. By utilizing the temporal continuity of the sequence to construct dynamic spatial weight constraints and employing a confidence-driven hierarchical matching strategy, false matching caused by motion blur, illumination variations, and target symmetry is effectively suppressed. Post-processing optimization based on neighborhood brightness information further cleans the matching results. These measures collectively ensure accurate and stable feature correspondences even under low signal-to-noise ratio conditions. High-quality sparse 3D reconstruction is then achieved through the mature Colmap framework, ultimately outputting high-precision inter-frame relative pose estimation results. The entire system ensures high performance while also addressing the potential efficiency requirements of spaceborne platforms. Attached Figure Description

[0077] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0078] Figure 1 This is a flowchart of the non-cooperative spatial target relative pose estimation method based on monocular vision according to the present invention;

[0079] Figure 2This is a schematic diagram of the multi-level convolutional feature extraction network architecture in the non-cooperative spatial target relative pose estimation method based on monocular vision of the present invention. Detailed Implementation

[0080] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0081] Please refer to Figures 1 to 2 The first embodiment of the present invention provides a non-cooperative spatial target relative pose estimation method based on monocular vision, comprising: S100, pre-collecting a synthetic image dataset and a public spatial target dataset, training and inferring through a pre-trained feature extraction model to obtain pre-trained weights; S200, based on the pre-trained weights, using a multi-level convolutional feature extraction network to infer the input spatial target sequence, outputting the keypoint coordinates, keypoint confidence, and keypoint descriptors of each image frame as spatial target features, and simultaneously outputting keypoint brightness information; S300, based on the inter-frame motion smoothness assumption of the spatial target features, constructing a Gaussian-distributed dynamic spatial weight matching method. S400: Input the image sequence pairs to be matched into the feature matching algorithm, perform initial matching using the nearest neighbor algorithm, calculate the basic similarity matrix using the keypoint descriptors, calculate the dynamic spatial weight value of each matching point pair using the dynamic spatial weight matching constraint matrix, and output the preliminary matching point pairs based on scene confidence level judgment; S500: Perform brightness consistency verification and anomaly suppression on the preliminary matching point pairs using the keypoint brightness information to obtain fine feature matching point pairs; S600: Input the fine feature matching point pairs into the Colmap 3D reconstruction architecture, and calculate the relative pose between the spatial target sequence frames through geometric verification and model expansion.

[0082] In step S100, the process of acquiring the pre-trained weights includes: S110, pre-collecting a synthetic image dataset and a public spatial target dataset; S120, using a low-light scene training strategy to perform a first round of training, an inference phase, and a second round of training on the feature extraction model; wherein, the first round of training uses the synthetic image dataset to train the feature extraction model, initially training the network's key point detection capability; the inference phase uses the weights after the first round of training to perform inference on the public spatial target dataset, generating pseudo-real values ​​through homography transformation; the second round of training uses the public spatial target dataset and the corresponding pseudo-real values ​​to jointly train key points and descriptors; S130, in the first and second rounds of training, adaptive specular reflection enhancement processing is used to enhance the selected high-brightness areas. Random sampling of the specular reflection light source position is performed, and a Gaussian function is used to simulate the light intensity attenuation and generate a brightness distribution function. , will reflect the component The enhanced target region brightness is output by weighted fusion with the normalized image and then subjected to gamma correction. , wherein the reflection component The calculation formula is: , For gamma correction parameters, Fill the target with texture. The fusion weights for the original image and the reflected image are defined. Specifically, adaptive specular reflection enhancement processing is applied in two rounds of training to simulate the specular reflection characteristics of low-light surfaces. The core process includes four steps: target region selection, light source position sampling, Gaussian reflection modeling, and reflection fusion and correction. High-brightness regions M are calculated for selection and localization. The center coordinates of the direct position of the specular reflection light source are randomly generated within these high-brightness regions M. A Gaussian function is used to simulate the light intensity attenuation law, yielding the light source brightness distribution function. , will reflect the component The image is weighted and fused with the normalized original image, and the brightness gradient is enhanced through gamma correction to output the enhanced brightness of the target area. S140, In the second round of training, a staged illumination enhancement preprocessing strategy is adopted to perform primary and secondary enhancements on the original input image I. The primary enhancement includes at least one of brightness adjustment, contrast perturbation, motion blur, Gaussian noise, and salt-and-pepper noise. Local Gaussian blurring is achieved through random masking. After the original input image I undergoes homography transformation, the generated matching image pair I is... ’ Secondary enhancement processing is performed, which includes parameter range processing different from that of the primary enhancement.

[0083] In step S110, the process of generating the synthetic image dataset includes a background generation stage, a foreground rendering stage, and a post-processing stage. The background generation stage generates a simulated spatial background with a preset grayscale range based on the spatial background information in the publicly available spatial target dataset, and sets the contrast between the grayscale difference between the foreground target and the background. The foreground rendering stage generates a target filling texture with controllable illumination stripes and fractal noise. The calculation formula is: ,in, The dynamic reference grayscale of the foreground target is based on the background grayscale. For the illumination fringe term, consisting of the main fringe and auxiliary stripes Superimposed to form the dominant stripes in the dominant direction Simulate the arrangement direction of the main grooves or textures on the surface, and supplement with auxiliary stripes in orthogonal directions. Simulate secondary directions, such as random scratches, etc. The dynamic strength coefficient, The Fractal Brownian Motion (FBM) noise term is used to further simulate the irregularity of the material surface and balance the target size and the intensity of the filling texture; the post-processing stage adds zero-mean random Gaussian noise to the entire image.

[0084] In step S200, the process of reasoning about the input spatial target sequence using a multi-level convolutional feature extraction network includes: S210, constructing a multi-level convolutional feature extraction network, which includes a multi-level convolutional feature encoder, a Transformer attention module, a multi-task output head, and a post-processing module; S220, inputting the spatial target sequence into the multi-level convolutional feature extraction network, and extracting core features and multi-scale high-level information through the multi-level convolutional feature encoder; S230, the core features are enhanced by position encoding and input into the Transformer attention module to generate global semantic enhancement features; S240, the global semantic enhancement features are input into the multi-task output head, and keypoints are detected by the keypoint heatmap decoder and descriptors are generated by the descriptor decoder using a multi-scale feature pyramid (FPN) fusion strategy, outputting the original network output result; S250, the original network output result is input into the post-processing module, which outputs keypoint coordinates, keypoint confidence, keypoint descriptors, and keypoint brightness information.

[0085] This invention utilizes a self-designed, supervised multi-level convolutional feature extraction network (DarkPoint network, an improved model based on the typical SuperPoint architecture, with enhanced backbone depth, attention mechanism, and multi-task output head optimization for low-light spatial environments) to effectively extract key points in low signal-to-noise ratio regions on the SHIRT public spatial target dataset. Compared to the SuperPoint network, it achieves a 31% improvement in registration rate in the semi-physical simulation image domain, a 386.6% increase in effective point cloud density, and a 50.31% reduction in reconstruction pose error. Ablation experiments further validate the enhancement effects of the Transformer attention module, high-level feature expansion module, and the proposed low-light scene training strategy (DT training strategy, whose core is to improve the model's robustness to extreme lighting conditions without prior information through synthetic texture generation and adaptive specular reflection enhancement). This invention provides a highly robust feature extraction scheme for spatial target relative pose estimation tasks, addressing to some extent the two core challenges of missing prior information for non-cooperative targets and feature degradation in low-light scenes.

[0086] In S250, the process by which the post-processing module constructs the key point brightness information includes: in the image of the original output result of the network... Generate normalized offset The calculation formula is: ,in, For the defined neighborhood window size, Image width, Image height; for each keypoint Sampling neighborhood brightness The calculation formula is: ,in, For the current batch, This is the current key point number. The index of the pixel within the neighborhood window. and For normalized offset, Gaussian weighted calculation is used to obtain the brightness of the current keypoint. The calculation formula is: ,in, These are Gaussian weighting coefficients, calculated using the following formula: ,in, To control the standard deviation of the weight distribution.

[0087] In S300, the formula for constructing the Gaussian-distributed dynamic spatial weight matching constraint matrix is ​​as follows: ,in, The difference in feature point coordinates. The formula for calculating the dynamically adjusted Gaussian kernel standard deviation is as follows: , The time difference between the two frames. Based on the scope of the basic spatial constraints, For time constraint parameters, For the maximum reference frame interval, This is the maximum scaling factor.

[0088] Specifically, by analyzing the characteristics of space targets, the following hypothesis on inter-frame motion smoothness is proposed: the displacement of the three-dimensional feature points of the target involved in the space target image sequence is small between adjacent frames, and increases with the increase of frame interval (increasing time difference); the short-term motion of the space target relative to the servicing spacecraft has no abrupt characteristics, the offset of feature points within several adjacent frames approximately follows a normal distribution, and its standard deviation is related to the time difference (frame number difference). Based on this, combined with the inter-frame time difference... Difference with feature point coordinates Construct a dynamic space weight matching constraint matrix in Gaussian distribution form. , as a matching constraint based on spatiotemporal consistency.

[0089] In step S400, the feature matching algorithm includes: S410, performing initial matching using the nearest neighbor algorithm, and calculating the basic similarity matrix using the key point descriptors. S420, calculate the dynamic spatial weight value of each pair of matching points using the dynamic spatial weight matching constraint matrix. D430: Perform scene classification judgment, defining a high-confidence scene as one where the proportion of matching pairs with high initial matching confidence exceeds a preset ratio, and a low-confidence scene otherwise; S440: For high-confidence scenes, set a spatial weight threshold. A spatial verification and review mechanism is used to filter the matching results and output preliminary matching point pairs. For low-confidence scenarios, a dual-strategy fusion method combining descriptor similarity S and spatial weight W is adopted, calculating the similarity using an exponential fusion strategy. Similarity enhancement with double saliency regions The confidence score corresponding to the higher similarity score after normalization is selected as the matching result, and the preliminary matching point pair is output; wherein, the index fusion strategy calculates the similarity. The calculation formula is: ,in, The spatial weight index; the double salient regions enhance similarity. The calculation formula is: ,in, For the salient region of the descriptor, For spatially significant regions, It is an enhancing factor.

[0090] In S500, the step of performing brightness consistency verification and anomaly suppression on the preliminary matching point pair using the key point brightness information to obtain a fine feature matching point pair includes: S510, if the source frame's... Key points and target frame Neighborhood brightness values ​​of key matching point pairs and All below the threshold Then for the matching score To suppress it, the calculation formula is: S520, after bidirectional consistency verification, yields the final matching score. The calculation formula is: ,in, This indicates a forward match from the source frame to the target frame. S530: Indicates a negative match from the target frame to the source frame; S530: Extract the set of point pairs that satisfy the consistency constraint of positive and negative bidirectional matching to obtain fine feature matching point pairs.

[0091] In step S600, the step of inputting the fine feature matching point pairs into the Colmap 3D reconstruction architecture and calculating the relative poses between the spatial target sequence frames through geometric verification and model expansion includes: S610, performing two-view geometric verification on the fine feature matching points using the RANSAC algorithm to remove outliers; S620, performing incremental 3D reconstruction, selecting the model with the most images in multiple reconstruction results as the spatial perception database of the target, and obtaining the relative poses between the spatial target sequence frames.

[0092] This invention proposes a feature matching algorithm based on neural networks (NNs), which is optimized and improved in three aspects: First, for the sequence characteristics during hovering or circling phases, an inter-frame motion smoothness assumption is introduced to construct dynamic spatial weight matching constraints to suppress outlier matching errors. Second, to alleviate the problem of decreased descriptor discriminability under low illumination, a confidence-based hierarchical matching strategy is designed, and spatial weights are fused through a differentiated strategy. Third, for mismatches between symmetrical structures of spatial targets and low signal-to-noise ratio regions, a post-processing optimization algorithm based on the original brightness information of the feature point neighborhood is proposed. Finally, the proposed matching algorithm is fused with the feature extraction algorithm, and sparse reconstruction of spatial targets is achieved through the Colmap framework to obtain the relative poses between registered image sequences.

[0093] Experiments show that the proposed feature matching algorithm achieves a 100% image registration rate in the first 200 image sequences of the ROE2 trajectory in the SHIRT public spatial target dataset. Compared with SuperGlue and NN algorithms, it reduces the average pose error by 95.17% and 34.67%, respectively. Even in the extremely low-light scenario of the ROE1 trajectory, it maintains a relatively low matching error. This invention provides a robust feature matching scheme for spatial target relative pose estimation tasks, eliminating the need to rely on large-scale labeled data for model training. It addresses, to some extent, the two key problems of missing prior information for non-cooperative targets and feature degradation in low-light scenes.

[0094] A second embodiment of the present invention provides a non-cooperative spatial target relative pose estimation system based on monocular vision, comprising: a model training and weight acquisition module, used to pre-collect synthetic image datasets and public spatial target datasets, train and infer through a pre-trained feature extraction model to obtain pre-trained weights; a multi-task feature extraction module, used to infer the input spatial target sequence based on the pre-trained weights using a multi-level convolutional feature extraction network, outputting keypoint coordinates, keypoint confidence, and keypoint descriptors of each image frame as spatial target features, and simultaneously outputting keypoint brightness information; and a spatiotemporal constraint matrix construction module, used to construct a Gaussian-distributed dynamic spatial weight matching constraint matrix based on the inter-frame motion smoothness assumption of the spatial target features. The confidence-driven hierarchical matching module is used to input the image sequence pairs to be matched into the feature matching algorithm, perform initial matching using the nearest neighbor algorithm, calculate the basic similarity matrix through the key point descriptors, calculate the dynamic spatial weight value of each matching point pair through the dynamic spatial weight matching constraint matrix, and output the preliminary matching point pairs in combination with scene confidence level judgment. The brightness verification and anomaly suppression module is used to perform brightness consistency verification and anomaly suppression on the preliminary matching point pairs through the brightness information of the key points to obtain fine feature matching point pairs. The incremental 3D reconstruction and pose calculation module is used to input the fine feature matching point pairs into the Colmap 3D reconstruction architecture, and calculate the relative pose between the spatial target sequence frames through geometric verification and model expansion.

[0095] The embodiments of the present invention aim to protect a method and system for estimating the relative pose of non-cooperative spatial targets based on monocular vision, which has the following effects:

[0096] 1. This invention proposes a general relative pose estimation scheme that overcomes the lack of a large amount of prior information for non-cooperative spatial targets. The entire process architecture only involves model training in the feature extraction algorithm, and the relative pose estimation of various typical spatial target sequences can be achieved with a single training on existing datasets, demonstrating strong generalization ability.

[0097] 2. The improved feature extraction algorithm of this invention can extract stable features independent of illumination, and obtain more accurate keypoint and descriptor results for spatial target scenes. Under the constraint of missing keypoint labels, it guides the network to focus on the overall geometric structure of the spatial target, achieving joint extraction of spatial target keypoint detection and feature descriptors.

[0098] 3. The improved feature matching algorithm of this invention can fully utilize the temporal continuity and motion smoothness of the flight sequence of spatial targets to achieve accurate feature matching between image pairs. Through post-processing optimization and anomaly suppression algorithms, the problem of mismatch between symmetrical structures of spatial targets and low signal-to-noise ratio regions is solved.

[0099] The computer program product of the non-cooperative spatial target relative pose estimation method and apparatus based on monocular vision provided in the embodiments of the present invention includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods in the preceding method embodiments. For specific implementation, please refer to the method embodiments, which will not be repeated here.

[0100] Specifically, the storage medium can be a general-purpose storage medium, such as a portable disk or hard disk. When the computer program on the storage medium is run, it can execute the above-mentioned method for relative pose estimation of non-cooperative spatial targets based on monocular vision, thereby enabling stable 3D reconstruction and high-precision inter-frame relative pose estimation of monocular image sequences of non-cooperative spatial targets.

[0101] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0102] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of protection of the claims.

Claims

1. A method for estimating the relative pose of a non-cooperative spatial target based on monocular vision, characterized in that, include: S100: Pre-collect synthetic image datasets and public spatial target datasets, and train and infer pre-trained feature extraction models to obtain pre-trained weights; S200, based on the pre-trained weights, a multi-level convolutional feature extraction network is used to infer the input spatial target sequence, and the key point coordinates, key point confidence and key point descriptors of each image frame are output as spatial target features, and key point brightness information is output at the same time. S300, Based on the inter-frame motion smoothness assumption of the spatial target features, construct a dynamic spatial weight matching constraint matrix in Gaussian distribution form; S400, input the image sequence pairs to be matched into the feature matching algorithm, use the nearest neighbor algorithm for initial matching, calculate the basic similarity matrix through the key point descriptor, calculate the dynamic spatial weight value of each pair of matching points through the dynamic spatial weight matching constraint matrix, and output the preliminary matching point pairs in combination with scene confidence level judgment. S500, using the brightness information of the key points, perform brightness consistency verification and anomaly suppression on the preliminary matching point pairs to obtain fine feature matching point pairs; S600, the fine feature matching point pairs are input into the Colmap 3D reconstruction architecture, and the relative poses between the spatial target sequence frames are calculated through geometric verification and model expansion.

2. The method for estimating the relative pose of a non-cooperative spatial target based on monocular vision according to claim 1, characterized in that, In S100, the process of obtaining the pre-trained weights includes: S110, pre-collected synthetic image dataset and public spatial target dataset; S120 utilizes a low-light scene training strategy to perform the first round of training, inference phase, and second round of training on the feature extraction model; S130, in the first and second rounds of training, adaptive specular reflection enhancement processing is used to enhance the selected high-brightness areas. Random sampling of the specular reflection light source position is performed, and a Gaussian function is used to simulate the light intensity attenuation and generate a brightness distribution function. , will reflect the component The enhanced target region brightness is output by weighted fusion with the normalized image and then subjected to gamma correction. , wherein the reflection component The calculation formula is , For gamma correction parameters, Fill the target with texture. The fusion weights for the original image and the reflected image; S140, In the second round of training, a phased illumination enhancement preprocessing strategy is adopted to perform primary and secondary enhancement on the original input image I.

3. The method for estimating the relative pose of a non-cooperative spatial target based on monocular vision according to claim 2, characterized in that, In S110, the process of generating the synthetic image dataset includes a background generation stage, a foreground drawing stage, and a post-processing stage; The background generation stage generates a simulated spatial background with a preset grayscale range based on the spatial background information in the publicly available spatial target dataset, and sets the contrast between the grayscale difference between the foreground target and the background. The foreground rendering stage generates a target fill texture with controllable lighting stripes and fractal noise. The calculation formula is: ,in, The dynamic reference grayscale of the foreground target is based on the background grayscale. For the light stripe section, The dynamic strength coefficient, This is the fractal Brownian motion noise term; The post-processing stage adds zero-mean random Gaussian noise to the entire image.

4. The method for estimating the relative pose of a non-cooperative spatial target based on monocular vision according to claim 1, characterized in that, In S200, the process of reasoning about the input spatial target sequence using a multi-level convolutional feature extraction network includes: S210, Construct a multi-level convolutional feature extraction network, which includes a multi-level convolutional feature encoder, a Transformer attention module, a multi-task output head, and a post-processing module; S220 inputs the spatial target sequence into a multi-level convolutional feature extraction network, and extracts core features and multi-scale high-level information through a multi-level convolutional feature encoder. S230, the core features are enhanced by positional encoding and input into the Transformer attention module to generate global semantic enhancement features; S240, the global semantic enhancement features are input into the multi-task output head, key points are detected by the key point heatmap decoder, descriptors are generated by the descriptor decoder, and the original output results of the network are output. S250, the original output of the network enters the post-processing module, which outputs key point coordinates, key point confidence, key point descriptors, and key point brightness information.

5. The method for estimating the relative pose of a non-cooperative spatial target based on monocular vision according to claim 4, characterized in that, In S250, the process by which the post-processing module constructs the key point brightness information includes: The image of the original output result of the network Generate normalized offset The calculation formula is: ， in, For the defined neighborhood window size, Image width, Image height; For each key point Sampling neighborhood brightness The calculation formula is: ， in, For the current batch, This is the current key point number. The index of the pixel within the neighborhood window. and For normalized offset; The brightness of the current key point is calculated using Gaussian weighting. The calculation formula is: ， in, These are Gaussian weighting coefficients, calculated using the following formula: ， in, To control the standard deviation of the weight distribution.

6. The method for estimating the relative pose of a non-cooperative spatial target based on monocular vision according to claim 1, characterized in that, In S300, the formula for constructing the Gaussian-distributed dynamic spatial weight matching constraint matrix is ​​as follows: ， in, The difference in feature point coordinates. The formula for calculating the dynamically adjusted Gaussian kernel standard deviation is as follows: , The time difference between the two frames. Based on the scope of the basic spatial constraints, For time constraint parameters, For the maximum reference frame interval, This is the maximum scaling factor.

7. The method for estimating the relative pose of a non-cooperative spatial target based on monocular vision according to claim 6, characterized in that, In S400, the feature matching algorithm includes: S410, the nearest neighbor algorithm is used for initial matching, and the basic similarity matrix is ​​calculated using the key point descriptors. ; S420, calculate the dynamic spatial weight value of each pair of matching points using the dynamic spatial weight matching constraint matrix. ; D430 performs scene classification judgment, defining a high-confidence scene as one where the proportion of matching pairs with high initial matching confidence exceeds a preset proportion, and a low-confidence scene otherwise. S440, for high-confidence scenarios, sets a spatial weight threshold. A spatial verification and review mechanism is used to filter the matching results and output preliminary matching point pairs; For low-confidence scenarios, S450 employs a dual-strategy fusion method combining descriptor similarity S and spatial weight W, calculating similarity using an exponential fusion strategy. Similarity enhancement with double saliency regions Select the confidence score corresponding to the higher similarity score after normalization as the matching result, and output the preliminary matching point pair; The index fusion strategy calculates the similarity. The calculation formula is ,in, Spatial weight index; The double salient regions enhance similarity. The calculation formula is ,in, For the salient region of the descriptor, For spatially significant regions, It is an enhancing factor.

8. The method for estimating the relative pose of a non-cooperative spatial target based on monocular vision according to claim 7, characterized in that, In S500, the step of performing brightness consistency verification and anomaly suppression on the preliminary matching point pair using the key point brightness information to obtain a fine feature matching point pair includes: S510, if the source frame's first Key points and target frame Neighborhood brightness values ​​of key matching point pairs and All below the threshold Then for the matching score To suppress it, the calculation formula is: ； S520, after bidirectional consistency verification, yields the final matching score. The calculation formula is: ， in, This indicates a forward match from the source frame to the target frame. This indicates a negative match from the target frame to the source frame; S530: Extract the set of point pairs that satisfy the positive and negative bidirectional matching consistency constraint to obtain fine feature matching point pairs.

9. The method for estimating the relative pose of a non-cooperative spatial target based on monocular vision according to claim 1, characterized in that, In S600, the step of inputting the fine feature matching point pairs into the Colmap 3D reconstruction architecture and calculating the relative poses between the spatial target sequence frames through geometric verification and model expansion includes: S610, Perform two-view geometric verification on the fine feature matching points using the RANSAC algorithm to remove outliers; S620, perform incremental 3D reconstruction, select the model with the most images in multiple sets of reconstruction results as the spatial perception database of the target, and obtain the relative pose between the spatial target sequence frames.

10. A non-cooperative spatial target relative pose estimation system based on monocular vision, characterized in that, include: The model training and weight acquisition module is used to pre-collect synthetic image datasets and public space target datasets, and to train and infer pre-trained weights through a pre-trained feature extraction model. The multi-task feature extraction module is used to reason about the input spatial target sequence based on the pre-trained weights using a multi-level convolutional feature extraction network, and output the key point coordinates, key point confidence and key point descriptors of each image frame as spatial target features, while also outputting key point brightness information. The spatiotemporal constraint matrix construction module is used to construct a Gaussian-distributed dynamic spatial weight matching constraint matrix based on the inter-frame motion smoothness assumption of the spatial target features. The confidence-driven hierarchical matching module is used to input the image sequence pairs to be matched into the feature matching algorithm, perform the initial matching using the nearest neighbor algorithm, calculate the basic similarity matrix through the key point descriptor, calculate the dynamic spatial weight value of each pair of matching points through the dynamic spatial weight matching constraint matrix, and output the preliminary matching point pairs in combination with the scene confidence level judgment. The brightness verification and anomaly suppression module is used to perform brightness consistency verification and anomaly suppression on the preliminary matching point pair using the brightness information of the key points, so as to obtain a fine feature matching point pair. The incremental 3D reconstruction and pose calculation module is used to input the fine feature matching point pairs into the Colmap 3D reconstruction architecture, and calculate the relative pose between the spatial target sequence frames through geometric verification and model expansion.

Images