A real-time posture tracking method and system for non-cooperative targets in space

Abstract

The present invention discloses a real-time posture tracking method for non-cooperative targets in space, comprising: performing target detection and instance segmentation on the original RGB image captured by a camera to determine the target area of ​​the object to be tracked; extracting key point information of the target area, wherein the key point information includes the position of the key point and the feature vector of the key point; matching the key points of the current frame with the previous frame, that is, updating the key point feature values ​​of the current frame and the previous frame, matching the updated key points of the current frame with the key points of the previous frame, and eliminating mismatched point pairs to obtain an optimized matching result; calculating the posture of the current frame; and optimizing the posture of the current frame using data from historical frames. The present invention calculates relative posture by matching key points of adjacent frames, and does not require real-time target modeling during the tracking process, effectively avoiding the introduction of modeling errors, improving the posture generation rate, and reducing fuel consumption.

Description

Technical Field

[0001] The present invention relates to the field of tracking and positioning technology, and in particular to a real-time posture tracking method and system for a non-cooperative target in space. Background Art

[0002] Powered by deep learning networks, pose tracking technology has achieved unprecedented breakthroughs over the past few decades. Deep features free themselves from the limitations of traditional features, freeing tracking effectiveness from the constraints of manually designed features. End-to-end models can even generate pose results directly from images. Existing technologies can be divided into two categories: one requires a known 3D model of the object to be tracked and compares the real-time image data with a standard 3D model of the object to be tracked to determine the pose result for the current frame; the other does not require a 3D model of the object to be tracked, but continuously generates 3D models of the object during the tracking process and then compares the real-time data with the modeled results.

[0003] However, in real-world applications, the shape and contours of the objects being tracked are often unknown. This is particularly true in the aerospace sector, where 3D models of non-cooperative targets in space are classified and inaccessible, making the first type of technology unsuitable for this scenario. The complex space environment requires frequent orbital changes for multi-dimensional observation and modeling of the objects being tracked, resulting in high fuel consumption and low security. Therefore, the second type of technology also fails to meet practical application requirements. Summary of the Invention

[0004] In view of the fact that existing technologies are unable to meet the application needs in the aerospace field, the present invention proposes a real-time posture tracking method and system for non-cooperative targets in space.

[0005] The object of the present invention is achieved through the following technical solutions:

[0006] A real-time pose tracking method for a non-cooperative target in space, comprising the following steps:

[0007] Step 1: Perform target detection and instance segmentation on the original RGB image captured by the camera to determine the target area of ​​the object to be tracked;

[0008] Step 2: extracting key point information of the target area obtained in step 1, wherein the key point information includes the position of the key point and the feature vector of the key point;

[0009] Step 3: Match the key points of the current frame with the previous frame; the specific operations are: normalize the key point features of the current frame, establish the adjacency relationship between the normalized key points of the current frame, and construct the normalized key point graph structure, update the feature vector of the center point of the normalized key point graph structure, and obtain the new key point of the current frame; based on the current frame, forward match the new key point in the previous frame that matches the current frame; based on the previous frame, reverse match the new key point in the current frame that matches the previous frame; based on the forward matching and the reverse matching, preliminarily construct the matching relationship between the new key point in the current frame and the new key point in the previous frame, and then eliminate the mismatch to obtain the optimized matching result;

[0010] Step 4: Calculate the pose of the current frame;

[0011] Step 5: Using the data of the historical frames, optimize the pose of the current frame obtained in step 4.

[0012] Furthermore, the step three is specifically implemented through the following sub-steps:

[0013] (3.1) Normalize key point features: Perform zero-mean normalization on the feature vector of the current frame, keeping the dimension of the feature vector of the current frame unchanged, making the mean of each dimension 0 and the variance 1; record the normalized key points as feature points;

[0014] (3.2) Establishing the adjacency relationship between the feature points of the current frame: Calculate the distance and cosine similarity between the feature points, and mark the two feature points whose horizontal and vertical distances do not exceed the set threshold and whose cosine similarity is higher than the set threshold as adjacent points; each feature point has at most 10 adjacent points, and all the adjacent points of each feature point constitute the neighborhood information of the feature point; for a feature point, if there are more than 10 feature points that meet the adjacent point conditions, only the 10 feature points with the closest Euclidean distance to it are retained as adjacent points; if there are no feature points that meet the adjacent point conditions, it is considered that the feature point has no adjacent points;

[0015] (3.3) Constructing a feature point graph structure: establishing a connection relationship between each feature point and each adjacent point in its neighborhood, and making the feature point connected to each adjacent point in the neighborhood the central feature point; one end of each connecting edge is connected to the central feature point, and the other end of the connecting edge is connected to one of the adjacent points in the neighborhood of the central feature point, and each connecting edge only connects two points; the length of each connecting edge is inversely proportional to the cosine similarity of the two feature points connected by the connecting edge; performing zero-mean normalization on the lengths of all connecting edges of the central feature point so that the mean length of all connecting edges of the central feature point is 0 and the variance is 1;

[0016] (3.4) updating the feature vector of the central feature point so that the value of the feature vector of the central feature point in each dimension after the update is equal to the weighted sum of the value of the feature vector of the central feature point in the corresponding dimension before the update and the value of the feature vector of the adjacent points in the neighborhood of the central feature point in the corresponding dimension; and the weight of the feature vector of the central feature point before the update is 1, and the weight of the feature vector of the adjacent points in the neighborhood of the central feature point is equal to the length of the corresponding connecting edge of the adjacent points; the updated central feature point is recorded as the new key point;

[0017] (3.5) Forward matching of new key points: Based on the current frame, find the points in the new key points of the previous frame that match the new key points of the current frame. The specific steps are as follows: for each new key point in the current frame, compare it with all the new key points in the previous frame. Among all the new key points in the previous frame whose horizontal and vertical distances to a new key point in the current frame do not exceed the set threshold and whose cosine similarity is higher than the set threshold, select the new key point with the largest cosine similarity as the matching point of the new key point in the current frame. If there is no point that meets the requirements, it is considered that there is no matching point for the new key point in the current frame.

[0018] (3.6) Reverse matching of new key points: Based on the previous frame, refer to the steps described in (3.5) to find the points in the new key points of the current frame that match the new key points of the previous frame;

[0019] (3.7) Preliminary construction of matching relationship: For the matching relationship p1-p2 in the forward matching, if there is also a matching relationship p2-p1 in the reverse matching, then p1 and p2 are considered to have a matching relationship, and p1 and p2 form a set of matching point pairs. Otherwise, p1 is considered to have no matching points; p1 is the new key point of the current frame, and p2 is the new key point of the previous frame;

[0020] (3.8) Eliminate false matches: Randomly select no less than four groups of matching point pairs from all matching point pairs that satisfy (3.7) and calculate the homography matrix; calculate whether all matching point pairs satisfy the homography matrix, obtain the optimal homography matrix, eliminate the matching point pairs that do not satisfy the optimal homography matrix, and complete the optimization of the matching results.

[0021] Furthermore, step one is specifically implemented by the following steps: training a target detection and instance segmentation network on a large-scale dataset including images and true values ​​of the poses of tracked objects, and then fine-tuning the trained target detection and instance segmentation network based on a small number of aerospace sample images; using the fine-tuned target detection and instance segmentation network to obtain the position mask of the non-cooperative target, and determining the target area of ​​the object to be tracked based on the mask information.

[0022] Furthermore, the step 2 is specifically implemented by the following steps: training a key point extraction network on a large-scale data set including images and true values ​​of the tracked object pose, and then fine-tuning the trained key point extraction network based on a small number of aerospace sample images; using the fine-tuned key point extraction network, extracting key points in the target area obtained in the step 1, and the extraction results include the positions and feature vectors of multiple key points.

[0023] Furthermore, step four is specifically implemented by the following steps: based on the key point matching result obtained in step three, combined with the depth information of the original RGB image captured by the camera and the calibration parameters of the camera, the 6D pose of the current frame is calculated.

[0024] Furthermore, the step five is specifically implemented by the following steps: screening out neighboring frames with a posture close to that of the current frame in the historical frame database, optimizing the posture results of the current frame and the neighboring frames using a posture graph optimization method, and storing the posture optimization results of the current frame and the neighboring frames in the historical frame database.

[0025] Furthermore, the definition of the adjacent points is specifically as follows: two feature points whose horizontal and vertical distances do not exceed 10 mm and whose cosine similarity is higher than 0.7 are marked as adjacent points; in the step (3.5), the specific selection area of ​​the matching points is: all new key points in the previous frame whose horizontal and vertical distances to a new key point in the current frame do not exceed 10 mm and whose cosine similarity is higher than 0.7.

[0026] A real-time posture tracking system for non-cooperative targets in space, which adopts the real-time posture tracking method for non-cooperative targets in space, comprises:

[0027] The object detection and instance segmentation module is used to train the object detection and instance segmentation network on a large-scale dataset, and then fine-tune the trained object detection and instance segmentation network based on a small number of aerospace sample images. The fine-tuned object detection and instance segmentation network is used to obtain the position mask of non-cooperative targets, and the target area of ​​the object to be tracked is determined based on the mask information.

[0028] A key point extraction module is used to train a key point extraction network on a large-scale dataset and then fine-tune the trained key point extraction network based on a small number of aerospace sample images. The fine-tuned key point extraction network is used to extract key points within the target area obtained by the target detection and instance segmentation module. The extraction results include the positions and feature vectors of multiple key points.

[0029] Key point matching module, used to match the key points of the current frame with the previous frame;

[0030] A pose estimation module is used to calculate the pose of the current frame: based on the key point matching results obtained by the key point matching module, combined with the depth information of the original RGB image captured by the camera and the calibration parameters of the camera, the pose transformation matrix of the current frame relative to the previous frame is calculated, and then multiplied by the absolute pose of the previous frame to obtain the pose estimation result of the current frame;

[0031] The posture optimization module calculates the relative posture transformation, finds the neighboring frames that are close to the posture estimation results of the current frame in the historical frame database, optimizes the posture of the current frame and its neighboring frames, and stores the posture optimization results of the current frame and the neighboring frames in the historical frame database.

[0032] A computer-readable storage medium stores a program, which, when executed by a processor, implements a real-time posture tracking method for a non-cooperative target in space.

[0033] The beneficial effects of the present invention are:

[0034] (1) The present invention uses a model-free tracking method that does not require a 3D model of the target being tracked and can directly track non-cooperative unknown spacecraft in space.

[0035] (2) The present invention calculates relative pose by matching key points of adjacent frames, which eliminates the need to model the target in real time during tracking. This effectively avoids the introduction of modeling errors, greatly improves the pose generation rate, and reduces fuel consumption. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 This is a flow chart of the real-time posture tracking method for non-cooperative targets in space according to the present invention.

[0037] Figure 2 It is a flowchart for matching the key points of the current frame with the previous frame.

[0038] Figure 3 It is a flow chart for eliminating error matching.

[0039] Figure 4 It is a schematic diagram of the real-time posture tracking system for non-cooperative targets in space according to the present invention. DETAILED DESCRIPTION

[0040] The present invention will be described in detail below based on the accompanying drawings and preferred embodiments. The purpose and effects of the present invention will become more apparent. The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only for explaining the present invention and are not intended to limit the present invention.

[0041] like Figure 1As shown, a real-time pose tracking method for non-cooperative targets in space includes the following steps:

[0042] Step 1: Perform object detection and instance segmentation on the original RGB image captured by the camera.

[0043] The target detection and instance segmentation network is trained on a large-scale dataset that includes images (generally, each frame of a video is extracted and saved in the form of a series of images in chronological order) and the true value (accurate numerical value) of the position of the tracked object, and then the trained target detection and instance segmentation network is fine-tuned based on a small number of aerospace sample images. The fine-tuned target detection and instance segmentation network is used to obtain the position mask of the non-cooperative target, and the target area of ​​the object to be tracked is determined based on the mask information. When training the network, the present invention uses a common large-scale dataset to train the network, and then uses a small number of spacecraft samples for fine-tuning. The training process does not rely on a large number of spacecraft images, avoiding the problem of insufficient training due to insufficient spacecraft information.

[0044] In this embodiment, the Mask R-CNN network is pre-trained on the COCO dataset, and then the trained network is fine-tuned using spacecraft simulation data.

[0045] Step 2: Extract key point information of the target area.

[0046] A key point extraction network is trained on a large-scale dataset that includes images (generally, every frame of a video is extracted and saved in chronological order as a series of images) and the true values ​​(accurate numerical values) of the tracked object's pose. The trained key point extraction network is then fine-tuned based on a small number of aerospace sample images. The fine-tuned key point extraction network is used to extract key points A within the target area obtained in step 1. The extraction results include the positions and feature vectors of multiple key points.

[0047] In this embodiment, the VGG16 network is used to generate multiple depth features in the target area, the FAST corner feature extraction algorithm is used to find the key point A in the target area, and the depth features of the corresponding positions are recorded.

[0048] Step 3: Match the key points of the current frame and the previous frame.

[0049] The graph attention network is used to fine-tune the key point feature values ​​of the current frame and the previous frame, and the key points of the fine-tuned current frame are matched with the key points of the previous frame, and the mismatched point pairs are eliminated to obtain the optimized matching results. In this embodiment, the neighborhood information is obtained based on the positional relationship and cosine similarity between the key points, and a neighborhood relationship graph is constructed. The feature points in the neighborhood are used to update the current feature points. The bidirectional KNN (K-NearestNeighbor) voting algorithm is used to match the key points of the current frame and the previous frame, and the RANSAC (Random Sample Consensus) algorithm is used to eliminate mismatched point pairs.

[0050] like Figure 2 As shown, step three is implemented through the following sub-steps:

[0051] (3.1) Normalized key point features: Perform zero-mean normalization on the feature vector of the current frame, keeping the dimension of the feature vector of the current frame unchanged, making the mean of each dimension 0 and the variance 1. The normalized key point is recorded as feature point A′.

[0052] (3.2) Establish the adjacency relationship between the feature points of the current frame.

[0053] Calculate the distance and cosine similarity between feature points A′. Mark any two feature points A′ whose horizontal and vertical distances do not exceed 10 mm and whose cosine similarity is greater than 0.7 as neighbors. Each feature point A′ has at most 10 neighbors. If more than 10 feature points A′ meet the requirements, only the 10 with the closest Euclidean distance are retained as neighbors. All neighbors of each feature point A′ constitute its neighborhood information. If no feature point A′ meets the requirements, it is considered to have no neighbors.

[0054] (3.3) Construct feature point graph structure.

[0055] Establish a connection relationship between each feature point A′ and each adjacent point in its neighborhood, and let the feature point A′ connected to each adjacent point in the neighborhood be the central feature point. One end of each connecting edge is connected to the central feature point, and the other end is connected to a nearby point in the neighborhood of the central feature point, and each connecting edge only connects two points.

[0056] The length of each connecting edge is inversely proportional to the cosine similarity of the two feature points connected by the connecting edge.

[0057] For the central feature point, the lengths of all its connecting edges are normalized to zero mean, that is, the lengths of all the connecting edges of the central feature point have a mean of 0 and a variance of 1.

[0058] (3.4) Fine-tune the feature vector of the central feature point.

[0059] The updated feature vector of the central feature point should meet the following conditions: the value of each dimension of the updated feature vector of the central feature point is equal to the weighted sum of the value of the feature vector of the central feature point in the corresponding dimension before the update and the value of the feature vector of the adjacent points in the neighborhood of the central feature point in the corresponding dimension; the weight of the feature vector of the central feature point before the update is 1, and the weight of the feature vector of the adjacent points in the neighborhood of the central feature point is equal to the length of the corresponding connecting edge. The expression of the feature vector of the fine-tuned central feature point is:

[0060]

[0061] Where i is the index value of a certain dimension of the feature vector of the central feature point before updating, F i is the value of the i-th dimension of the feature vector of the central feature point before updating, F i ′ is the value of the i-th dimension of the feature vector of the updated central feature point, n is the number of edges connecting the central feature point, W k is the weight of the adjacent points connected by the kth edge, F ki It is the value of the i-th dimension of the feature vector of the adjacent points connected by the k-th edge before updating.

[0062] The updated central feature point is recorded as the new key point A″.

[0063] Steps (3.1) to (3.4) are all performed on the current frame at time T. After saving the results, at time T+1, we can directly retrieve the results at time T using time T+1 as the current frame. In other words, when running to the current frame, these operations have already been performed on all the historical frames.

[0064] (3.5) Forward matching of key points: Based on the current frame, find the point in the new key point A″ of the previous frame that matches the new key point A″ of the current frame. The specific steps are as follows: For each new key point A″ in the current frame, compare it with all the new key points A″ in the previous frame. Among all the new key points A″ in the previous frame whose horizontal and vertical distances to a new key point A″ in the current frame are no more than 10mm and whose cosine similarity is higher than 0.7, select the point with the largest cosine similarity as the matching point of the new key point A″ in the current frame; if there is no point that meets the requirements, it is considered that the new key point A″ in the current frame has no matching point.

[0065] (3.6) Reverse key point matching: Based on the previous frame, use the method described in (3.5) to find the points in the new key point A″ of the current frame that match the new key point A″ of the previous frame.

[0066] (3.7) Preliminary construction of matching relationships: For a matching relationship p1-p2 in the forward matching (p1 is the new key point A″ of the current frame, p2 is the new key point A″ of the previous frame), if a matching relationship p2-p1 also exists in the reverse matching, then p1 and p2 are considered to have a matching relationship, and p1 and p2 form a set of matching point pairs. Otherwise, p1 is considered to have no matching points. The number of all point pairs that satisfy the matching relationship is counted as N.

[0067] (3.8) Eliminate mismatches: Randomly select four sets of matching point pairs from all matching point pairs and calculate the homography matrix H; calculate whether all matching point pairs satisfy the homography matrix, obtain the optimal homography matrix, and eliminate the matching point pairs that do not satisfy the optimal homography matrix. The specific process is as follows: Figure 3 As shown, the expression is as follows:

[0068] Dst′=H*Src (2)

[0069] Dst[0]′=Dst[0]′ / Dst[2]′ (3)

[0070] Dst[1]′=Dst[1]′ / Dst[2]′ (4)

[0071] E=(|Dst[0]′- Dst[0]|+|Dst[1]′- Dst[1]|) / 2 (5)

[0072] Where Src is the coordinate vector of the key point p1 in the current frame, with a dimension of 3. The values ​​of each dimension are the value in the x direction, the value in the y direction, and 1 respectively; Dst is the coordinate vector of the key point p2 corresponding to p1 in the previous frame, and the value of each dimension is taken in the same way as Src; Dst′ is the coordinate vector of the key point p2 calculated according to the homography matrix, and Dst[0]′, Dst[1]′, and Dst[2]′ are the values ​​of the coordinate vector of the matching point p2 calculated according to the homography matrix in the x direction, y direction, and z direction respectively; E is the coordinate vector of the key point p2 calculated according to the homography matrix, and the mean error between it and the actual coordinate vector of the key point p2 in each dimension.

[0073] If the error mean E is less than 0.5 mm, the point pair is considered to conform to the homography matrix, otherwise it is considered not to conform. Count the number of point pairs that conform to the homography matrix among all point pairs;

[0074] Then randomly select four new sets of matching point pairs and repeat the above steps (3.8) until all point pairs conform to the calculated homography matrix, or the number of loops reaches 1000, then end the loop. Take the homography matrix that all point pairs conform to, or the homography matrix with the largest number of matching point pairs, as the optimal homography matrix. Delete the matching point pairs that do not conform to the optimal homography matrix (i.e., the error mean E is greater than 0.5mm), and complete the optimization of the matching results.

[0075] Step 4: Calculate the pose of the current frame.

[0076] The 6D pose of the current frame is calculated by combining the RGB image, depth information, and camera calibration parameters. In this embodiment, the 6D pose of the current frame is calculated using the PnP (Perspective-n-Points) algorithm.

[0077] Step 5: Use historical frame data to perform multi-frame pose optimization.

[0078] The historical frames with similar poses to the current frame are selected from the historical frame database, and the pose graph optimization method is used to optimize the pose results of the current frame and the historical frames. The pose optimization results of the current frame and the historical frames are stored in the historical frame database.

[0079] like Figure 4 Figure 2 shows a schematic diagram of a real-time pose tracking system for non-cooperative targets in space, which includes the following five modules:

[0080] The target detection and instance segmentation module is used to train the target detection and instance segmentation network on a large-scale dataset, and then fine-tune the trained target detection and instance segmentation network based on a small number of aerospace sample images; the fine-tuned target detection and instance segmentation network is used to obtain the position mask of non-cooperative targets, and the target area of ​​the object to be tracked is determined based on the mask information.

[0081] The key point extraction module is used to train the key point extraction network on a large-scale dataset and then fine-tune the trained key point extraction network based on a small number of aerospace sample images. The fine-tuned key point extraction network is used to extract key points within the target area obtained by the target detection and instance segmentation modules. The extraction results include the positions and feature vectors of multiple key points.

[0082] The key point matching module is used to match the key points of the current frame with the previous frame. The specific operations are as follows: normalize the key point features of the current frame, establish the adjacency relationship between the normalized key points of the current frame, and construct the normalized key point graph structure, update the feature vector of the center point of the normalized key point graph structure, and obtain the new key point of the current frame; based on the current frame, forward match the new key point in the previous frame that matches the current frame; based on the previous frame, reverse match the new key point in the current frame that matches the previous frame; based on the forward matching and reverse matching, preliminarily construct the matching relationship between the new key point in the current frame and the new key point in the previous frame, and then eliminate the false matches to obtain the optimized matching results.

[0083] The pose estimation module is used to calculate the pose of the current frame: based on the key point matching results obtained by the key point matching module, combined with the depth information of the original RGB image captured by the camera and the camera calibration parameters, the pose transformation matrix of the current frame relative to the previous frame is calculated, and then multiplied by the absolute pose of the previous frame to obtain the pose estimation result of the current frame.

[0084] The pose optimization module calculates the relative pose transformation, finds the neighboring frames in the historical frame database that are close to the pose estimation results of the current frame, optimizes the pose of the current frame and its neighboring frames, and stores the pose optimization results of the current frame and the neighboring frames in the historical frame database.

[0085] Those skilled in the art will understand that the foregoing descriptions are merely preferred embodiments of the invention and are not intended to limit the invention. Although the invention has been described in detail with reference to the foregoing examples, those skilled in the art will still be able to modify the technical solutions described in the foregoing examples or substitute equivalents for some of the technical features therein. Any modifications, equivalent substitutions, etc. made within the spirit and principles of the invention shall be included within the scope of protection of the invention.

Claims

1. A real-time pose tracking method for non-cooperative targets in space, characterized in that: The following steps are involved: Step 1: Perform target detection and instance segmentation on the original RGB image captured by the camera to determine the target area of ​​the object to be tracked; Step 2: extracting key point information of the target area obtained in step 1, wherein the key point information includes the position of the key point and the feature vector of the key point; Step 3: Match the key points of the current frame and the previous frame; the specific operation is: normalize the key point features of the current frame, record them as feature points, establish the adjacency relationship between the feature points of the current frame, and construct the feature point graph structure. Specifically, two feature points whose horizontal and vertical distances do not exceed the set threshold and whose cosine similarity is higher than the set threshold are marked as adjacent points, and each feature point is connected to each adjacent point in its neighborhood. The feature point connected to each adjacent point in the neighborhood is the central feature point; One end of each connecting edge is connected to the central feature point, and the other end is connected to a nearby point in the neighborhood of the central feature point, and each connecting edge only connects two points; the length of each connecting edge is inversely proportional to the cosine similarity of the two feature points connected by the connecting edge; the lengths of all connecting edges of the central feature point are normalized to zero mean; Update the feature vector of the center point of the feature point graph structure so that the value of the feature vector of the center feature point in each dimension after the update is equal to the weighted sum of the value of the feature vector of the center feature point in the corresponding dimension before the update and the value of the feature vector of the adjacent points in the neighborhood of the center feature point in the corresponding dimension; and the weight of the feature vector of the center feature point before the update is 1, and the weight of the feature vector of the adjacent points in the neighborhood of the center feature point is equal to the length of the corresponding connecting edge of the adjacent points; the updated center feature point is the new key point of the current frame; based on the current frame, forward match the new key point in the previous frame that matches the current frame; based on the previous frame, reverse match the new key point in the current frame that matches the previous frame; based on the forward matching and the reverse matching, preliminarily construct a matching relationship between the new key point in the current frame and the new key point in the previous frame, and then eliminate false matches to obtain an optimized matching result; Step 4: Calculate the pose of the current frame; Step 5: Using the data of the historical frames, optimize the pose of the current frame obtained in step 4.

2. The real-time pose tracking method for non-cooperative targets in space according to claim 1, characterized in that: The step three is specifically implemented through the following sub-steps: (3.1) Normalize key point features: Perform zero-mean normalization on the feature vector of the current frame, keeping the dimension of the feature vector of the current frame unchanged, making the mean of each dimension 0 and the variance 1; record the normalized key points as feature points; (3.2) Establishing the adjacency relationship between the feature points of the current frame: Calculate the distance and cosine similarity between the feature points, and mark the two feature points whose horizontal and vertical distances do not exceed the set threshold and whose cosine similarity is higher than the set threshold as adjacent points; each feature point has at most 10 adjacent points, and all the adjacent points of each feature point constitute the neighborhood information of the feature point; for a feature point, if there are more than 10 feature points that meet the adjacent point conditions, only the 10 feature points with the closest Euclidean distance to it are retained as adjacent points; if there are no feature points that meet the adjacent point conditions, the feature point is considered to have no adjacent points; (3.3) Construct feature point graph structure; (3.4) updating the feature vector of the central feature point; (3.5) Forward matching of new key points: Based on the current frame, find the points in the new key points of the previous frame that match the new key points of the current frame. The specific steps are as follows: for each new key point in the current frame, compare it with all the new key points in the previous frame. Among all the new key points in the previous frame whose horizontal and vertical distances to a new key point in the current frame do not exceed the set threshold and whose cosine similarity is higher than the set threshold, select the new key point with the largest cosine similarity as the matching point of the new key point in the current frame. If there is no point that meets the requirements, it is considered that there is no matching point for the new key point in the current frame. (3.6) Reverse matching of new key points: Based on the previous frame, refer to the steps described in (3.5) to find the points in the new key points of the current frame that match the new key points of the previous frame; (3.7) Preliminary construction of matching relationships: For the matching relationship p1-p2 in the forward matching, if there is also a matching relationship p2-p1 in the reverse matching, then p1 and p2 are considered to have a matching relationship, and p1 and p2 form a set of matching point pairs. Otherwise, p1 is considered to have no matching points; p1 is the new key point of the current frame, and p2 is the new key point of the previous frame; (3.8) Eliminate false matches: Randomly select no less than four groups of matching point pairs from all matching point pairs that satisfy (3.7) and calculate the homography matrix; calculate whether all matching point pairs satisfy the homography matrix, obtain the optimal homography matrix, eliminate the matching point pairs that do not satisfy the optimal homography matrix, and complete the optimization of the matching results.

3. The real-time pose tracking method for non-cooperative targets in space according to claim 1, characterized in that: The step 1 is specifically implemented by the following steps: training a target detection and instance segmentation network on a large-scale dataset including images and true values ​​of the tracked object poses, and then fine-tuning the trained target detection and instance segmentation network based on a small number of aerospace sample images; The fine-tuned target detection and instance segmentation network is used to obtain the location mask of the non-cooperative target, and the target area of ​​the object to be tracked is determined based on the mask information.

4. The real-time pose tracking method for non-cooperative targets in space according to claim 1, characterized in that: The second step is specifically implemented by the following steps: training a key point extraction network on a large-scale dataset including images and true values ​​of the position and pose of the tracked object, and then fine-tuning the trained key point extraction network based on a small number of aerospace sample images; The fine-tuned key point extraction network is used to extract key points in the target area obtained in step 1. The extraction results include the positions and feature vectors of multiple key points.

5. The real-time pose tracking method for non-cooperative targets in space according to claim 1, characterized in that: The fourth step is specifically implemented by the following steps: based on the key point matching result obtained in the third step, combined with the depth information of the original RGB image captured by the camera and the calibration parameters of the camera, the 6D pose of the current frame is calculated.

6. The real-time pose tracking method for non-cooperative targets in space according to claim 1, characterized in that: The step five is specifically implemented by the following steps: screening out neighboring frames with a posture close to that of the current frame in the historical frame database, optimizing the posture results of the current frame and the neighboring frames using a posture graph optimization method, and storing the posture optimization results of the current frame and the neighboring frames in the historical frame database.

7. The real-time pose tracking method for non-cooperative targets in space according to claim 2, characterized in that: The definition of the adjacent points is specifically as follows: marking two feature points whose horizontal and vertical distances do not exceed 10 mm and whose cosine similarity is higher than 0.7 as adjacent points; in the step (3.5), the specific selection area of ​​the matching points is: all new key points in the previous frame whose horizontal and vertical distances to a new key point in the current frame do not exceed 10 mm and whose cosine similarity is higher than 0.

7.

8. A real-time position and posture tracking system for a non-cooperative target in space using the real-time position and posture tracking method for a non-cooperative target in space according to any one of claims 1 to 7, characterized in that: include: The object detection and instance segmentation module is used to train the object detection and instance segmentation network on a large-scale dataset, and then fine-tune the trained object detection and instance segmentation network based on a small number of aerospace sample images; Use the fine-tuned object detection and instance segmentation network to obtain the location mask of the non-cooperative target, and determine the target area of ​​the object to be tracked based on the mask information; The key point extraction module is used to train the key point extraction network on a large-scale dataset and then fine-tune the trained key point extraction network based on a small number of aerospace sample images; Using the fine-tuned key point extraction network, key points are extracted within the target area obtained by the target detection and instance segmentation module. The extraction results include the positions and feature vectors of multiple key points. Key point matching module, used to match the key points of the current frame with the previous frame; A pose estimation module is used to calculate the pose of the current frame: based on the key point matching results obtained by the key point matching module, combined with the depth information of the original RGB image captured by the camera and the calibration parameters of the camera, the pose transformation matrix of the current frame relative to the previous frame is calculated, and then multiplied by the absolute pose of the previous frame to obtain the pose estimation result of the current frame; The posture optimization module calculates the relative posture transformation, finds the neighboring frames that are close to the posture estimation results of the current frame in the historical frame database, optimizes the posture of the current frame and its neighboring frames, and stores the posture optimization results of the current frame and the neighboring frames in the historical frame database.

9. A computer-readable storage medium, characterized in that A program is stored thereon, and when the program is executed by the processor, the real-time posture tracking method for non-cooperative targets in space according to any one of claims 1 to 7 is implemented.

Images