A spatial unknown target recognition method based on foreground metric learning
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- NORTHWESTERN POLYTECHNICAL UNIV
- Filing Date
- 2024-04-23
- Publication Date
- 2026-08-07
AI Technical Summary
目前主流的空间目标检测识别算法基于深度算法实现，只能识别训练数据集中存在的目标，对真实空间场景下真实出现的未知目标难以识别
[0037]本发明方法能够实现空间目标检测模型对未知样本的有效识别，从而提高了模型在真实环境下的泛化检测精度，最后采用领域自适应技术，利用新类数据集进行领域自适应，进一步提高了模型的目标检测精度和可靠性。
Smart Images
Figure CN118537618B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of target detection technology, specifically relating to a spatial unknown target recognition method based on foreground metric learning. Background Technology
[0002] In recent years, the number of spacecraft launches worldwide has increased rapidly, leading to an explosive growth in demand for on-orbit space services. Accurately obtaining the relative positions between the service satellite and the target satellite during on-orbit service implementation has become extremely urgent, as it is a crucial guarantee for subsequent guidance and control. Currently, mainstream space target detection and recognition algorithms are based on deep learning algorithms, which can only identify targets present in the training dataset and struggle to identify unknown targets that actually appear in real space scenarios. Existing methods for detecting and recognizing unknown targets rely on limited known data labels, resulting in biased detection results and high false negative rates. Therefore, there is an urgent need for a learning algorithm that can reliably detect unknown space targets. This invention proposes a space unknown target recognition method based on foreground metric learning. It constructs foreground and background metric learning methods, utilizes foreground confidence constraints to enable the model to recognize objects outside the domain, and introduces a transfer learning algorithm to guide the model to generalize across multiple datasets, improving the detection accuracy of the deep learning model in new categories. Summary of the Invention
[0003] To overcome the shortcomings of existing technologies, this invention provides a spatial unknown target recognition method based on foreground metric learning. First, a dataset containing multiple typical spatial targets is constructed. Then, the dataset is divided into a training dataset and a test dataset. Next, a target detection model based on foreground metric learning is constructed and trained using the training dataset. Then, an unsupervised adaptive model is used to perform unsupervised domain adaptation on the test set, reducing the domain difference between the test and training sets. Finally, the model performs inference on the test set, labeling the unknown categories, and comparing the results with the true values to provide a quantitative evaluation of the model. This invention utilizes a novel dataset for domain adaptation, further improving the target detection accuracy and reliability of the model.
[0004] The technical solution adopted by this invention to solve its technical problem is as follows:
[0005] Step 1: Using Blender software, import the CAD model and construct a dataset containing multiple typical spatial targets;
[0006] Step 2: Use a% of the classes in the dataset as the training dataset, and the remaining 1-a% of the classes as the test dataset; the target classes in the test dataset have not appeared in the training dataset.
[0007] Step 3: Construct an object detection model based on foreground metric learning. Train the object detection model using the training dataset. The training process uses foreground scores as quantification to quantify the presence or absence of an object.
[0008] Step 4: Use an unsupervised adaptive model to perform unsupervised domain adaptation on the test set to reduce the domain difference between the test set and the training set;
[0009] Step 5: After domain adaptation is completed, the model performs inference on the test set, labels the unknown categories, compares the results with the true values, and provides a quantitative evaluation of the model.
[0010] Preferably, step 3 is as follows:
[0011] Step 3-1: After the image is input, it first enters the YOLO-based feature extractor to extract features, using the C3 and SPP models to extract the multi-scale depth features of the model;
[0012] Step 3-2: After feature extraction, candidate boxes are generated. The candidate box generation module consists of two parts: the convolutional part is composed of an FPN feature pyramid structure based on the YOLO model, which mines multiple scale spaces of deep features to obtain candidate boxes at multiple levels.
[0013] We introduce random candidate boxes based on physical parameters. These boxes are generated by random numbers after collecting targets. The physical parameters include the pixel size of typical spatial targets in the image, which is calculated by querying the perception distance of actual non-cooperative targets and the actual size of spatial targets. Then, we randomly generate 200 candidate boxes and merge them with the candidate boxes extracted from depth features.
[0014] Step 3-3: After candidate boxes are generated, the RPN process is used to generate multiple detector heads. Each detector head undergoes a convolution process to obtain position, type, and foreground confidence modules. The convolution process for category and position is the same as in the YOLO algorithm. The foreground confidence module is used to evaluate the standardized distance from the current position to the target center of the previous position, i.e.:
[0015]
[0016] Among them l * ,r * and t * ,b * These correspond to the coordinates of the four directions (left, right, up, down) of the current target detection box; the foreground confidence ranges from 0 to 1, and the binary cross-entropy (BCE) loss is used for training.
[0017] Steps 3-4: The final network output is trained using a loss function to iterate the model parameters; the loss function is constructed as follows:
[0018]
[0019] in and ... The calculation is as follows:
[0020]
[0021] in For cross-entropy loss, These represent the category output by the model and the true category, respectively. These represent the foreground confidence and the true confidence of the model output, respectively; λ represents the weight error. This represents the location regression error; These represent the four locations of the actual and estimated detection boxes, respectively.
[0022] For unknown targets and background bounding boxes, regression is performed only on their categories;
[0023] loss function It is expressed as follows:
[0024]
[0025] Where UN represents an unknown target, BG represents a background target, and P represents an unknown target. U and P B Each represents a different candidate box;
[0026] Steps 3-5: During the testing phase, the judgment of unknown targets is composed of the following formulas:
[0027] C = C1 & (C2 | C3)
[0028] Where C1 represents the portion of all candidate boxes with a confidence threshold greater than 0.8 after removing the known target; C2 represents the top 3 candidate boxes after removing the known target; C3 represents the portion of all candidate boxes with an FP score greater than 0.9 after removing the known target; the final selected candidate boxes are recorded as Unknown, as unknown candidate boxes that do not exist in the training set.
[0029] Steps 3-6: Test the model results using real bounding box labels on the test dataset.
[0030] Preferably, step 4 is as follows:
[0031] Step 4-1: First, use the pre-trained model to perform inference on the test dataset. After inference, labels are generated for the test dataset. Using the generated FP labels, construct the FP loss function and train the model again. The loss function is in the form of cross-entropy loss.
[0032] Step 4-2: Use the current FP value to filter the test dataset; based on the TOPK algorithm, use the candidate boxes corresponding to the top N FP scores as the target set, and delete the rest of the data;
[0033] Step 4-3: Fine-tune the model using the constructed dataset; the fine-tuning refers to fixing the feature extraction model and only performing it on the candidate box generation module and the detection head module, while setting the learning rate to 1e-5;
[0034] Step 4-4: Repeat steps 4-1 to 4-3 until the required number of iterations is reached; save the model and output it.
[0035] Preferably, the location regression error adopts the IOU loss function.
[0036] The beneficial effects of this invention are as follows:
[0037] The method of this invention enables the spatial target detection model to effectively identify unknown samples, thereby improving the model's generalization detection accuracy in real-world environments. Finally, the method employs domain adaptation technology, utilizing a new class dataset for domain adaptation, further enhancing the model's target detection accuracy and reliability. Attached Figure Description
[0038] Figure 1 This is a schematic diagram of spatial unknown target recognition based on foreground metric learning.
[0039] Figure 2 This is a schematic diagram of a foreground measurement target detection model.
[0040] Figure 3 This is a schematic diagram of an adaptive process in an unsupervised domain.
[0041] Figure 4 A schematic diagram of targets in training data for non-cooperative targets in space.
[0042] Figure 5 This is a schematic diagram of a target in the test data for non-cooperative targets in space. Detailed Implementation
[0043] The present invention will be further described below with reference to the accompanying drawings and embodiments.
[0044] When satellites encounter new target detection categories during space environment perception, targets are affected by various factors such as different lighting conditions and space background, resulting in different feature distributions. This leads to significant differences in target features across different categories in different tasks. Deep learning models often experience a sharp decline in their ability to identify new categories during non-cooperative space target detection, misidentifying unknown targets as background noise and greatly weakening the generalization ability of the target detection model. The technical solution in this invention solves these problems, achieving the ability of the target detection model to identify targets not present in the training samples and labeling these targets as "unknown." This approach offers advantages such as high detection accuracy and strong model adaptability.
[0045] To effectively improve the adaptability of target detection models to unknown targets in space, this invention proposes a spatial unknown target recognition method based on foreground metric learning, which improves the efficiency and detection generalization of the model in detecting and recognizing unknown spatial targets.
[0046] The technical method solution of the present invention includes the following steps:
[0047] Step 1: Using Blender software, import the CAD model and construct a dataset containing multiple typical spatial targets;
[0048] Step 2: Use 80% of the categories as the training dataset and the remaining 20% as the test dataset; the target categories in the test set have not appeared in the training set.
[0049] Step 3: Construct a target detection model based on foreground metric learning, train the model using the training dataset, and use foreground scores as quantification during the training process to quantify the presence of a target;
[0050] Step 4: Use an unsupervised adaptive model to perform unsupervised domain adaptation on the test set to reduce the domain difference between the test set and the training set;
[0051] Step 5: After domain adaptation is completed, the model performs inference on the test set, labels the unknown categories, compares the results with the true values, and provides a quantitative evaluation of the model.
[0052] Example:
[0053] Figure 1This invention relates to the overall structure of a spatial unknown target recognition method based on foreground metric learning. First, using Blender software, a CAD model is imported to construct a dataset containing multiple typical spatial targets. 80% of the categories are used as the training dataset, and the remaining 20% as the test dataset. The target categories in the test set do not appear in the training set. A target detection model based on foreground metric learning is constructed and trained using the training dataset. Foreground scores are used as quantification during training to quantify the presence of a target. An unsupervised adaptive model is used to perform unsupervised domain adaptation on the test set, reducing the domain difference between the test and training sets. After domain adaptation, the model performs inference on the test set, labeling the unknown categories. The results are compared with the true values to provide a quantitative evaluation of the model.
[0054] Figure 2 This is a schematic diagram of a foreground measurement object detection model.
[0055] The details are as follows:
[0056] ① After the image is input, it first enters the feature extractor for feature extraction. Here, a YOLO-based feature extractor is used, which extracts multi-scale depth features of the model using the C3 and SPP models.
[0057] ② Following feature extraction, a candidate box generation module is implemented. This module consists of two parts: the convolutional part comprises a YOLO-based FPN feature pyramid structure, which mines multiple scales of deep features to obtain candidate boxes at multiple levels. To effectively extract candidate boxes for unknown targets, this project also introduces random candidate boxes based on physical parameters, reducing the probability of missing unknown target candidate boxes during training. Random candidate boxes are generated using random numbers after target collection. The physical parameters include the pixel dimensions of typical spatial targets in the image, calculated by querying the perceptual distance of actual non-cooperative targets and the actual size of spatial targets. Then, based on empirical sizes, 200 candidate boxes are randomly generated and merged with the candidate boxes extracted from deep features.
[0058] ③ After the candidate boxes are generated, the RPN process produces multiple detector heads. Each detector head undergoes a convolution process to obtain a position, type, and foreground confidence module. The convolution process for category and position is similar to that of the YOLO algorithm. The foreground confidence module is used to evaluate the standardized distance from the current position to the center of the target responsible for the previous position, i.e.
[0059]
[0060] Among them l * ,r * and t * ,b* These correspond to the coordinates of the current target detection box in the four directions: left, right, up, and down. The foreground confidence ranges from 0 to 1, so binary cross-entropy (BCE) loss is used for training.
[0061] ④ Finally, the network's output is fed into the loss function for training, thereby iterating the model parameters, such as... Figure 4 and Figure 5 As shown. The loss function is constructed as follows:
[0062]
[0063] in and ... The calculation is as follows:
[0064]
[0065] in For cross-entropy loss, These represent the category output by the model and the true category, respectively. These represent the foreground confidence and the true confidence of the model output, respectively. λ represents the weight error. For the location regression error, the classic IOU loss function is used here. These represent the four locations of the actual and estimated detection boxes, respectively.
[0066] For unknown targets and background detection boxes, since there is no ground truth alignment for position regression, we only perform category regression. Loss function It is expressed as follows:
[0067]
[0068] Where UN represents an unknown target and BG represents a background target. P U and P B These represent candidate boxes.
[0069] ⑤ During the testing phase, considering that there are generally few targets in space, the judgment of unknown targets is composed of the following formula.
[0070] C = C1 & (C2 | C3)
[0071] Where C1 represents the candidate boxes with a confidence threshold greater than 0.8 after removing the known target. C2 represents the top 3 candidate boxes after removing the known target. C3 represents the candidate boxes with a FP score greater than 0.9 after removing the known target. The final selected candidate boxes are recorded as "Unknown," representing unknown candidate boxes that do not exist in the training set.
[0072] ⑥ Test the model on the test dataset using real bounding box labels to evaluate the model results.
[0073] like Figure 3 As shown, the pre-trained model is first used to perform inference on the test dataset. After inference, labels are generated for the test dataset; here, only FP labels are considered, not class labels. Using the generated FP labels, an FP loss function is constructed and applied to the model again. The loss function is in the form of cross-entropy loss.
[0074] The test dataset is filtered using the current FP (First Form) value. Based on the TOPK (Top-N-Rank) algorithm, the candidate boxes corresponding to the top N FP scores are used as the target set, and the remaining data are discarded.
[0075] Fine-tune the model using the constructed dataset. Here, fine-tune the fixed feature extraction model, only for the candidate box generation module and the detection head module, with the learning rate set to 1e-5. Repeat the above process until the required number of iterations is reached. Save the model and output it.
Claims
1. A spatial unknown target recognition method based on foreground metric learning, characterized in that, Includes the following steps: Step 1: Using Blender software, import the CAD model and construct a dataset containing multiple typical spatial targets; Step 2: Use a% of the classes in the dataset as the training dataset, and the remaining 1-a% of the classes as the test dataset; The target categories in the test dataset did not appear in the training dataset. Step 3: Construct an object detection model based on foreground metric learning. Train the model using the training dataset, employing foreground scores as quantification to quantify the presence of a target. Details are as follows: Step 3-1: After the image is input, it first enters the YOLO-based feature extractor to extract features, using the C3 and SPP models to extract the multi-scale depth features of the model; Step 3-2: After feature extraction, candidate boxes are generated. The candidate box generation module consists of two parts: the convolutional part is composed of an FPN feature pyramid structure based on the YOLO model, which mines multiple scale spaces of deep features to obtain candidate boxes at multiple levels. We introduce random candidate boxes based on physical parameters. These boxes are generated by random numbers after collecting targets. The physical parameters include the pixel size of typical spatial targets in the image, which is calculated by querying the perception distance of actual non-cooperative targets and the actual size of spatial targets. Then, we randomly generate 200 candidate boxes and merge them with the candidate boxes extracted from depth features. Step 3-3: After candidate boxes are generated, the RPN process is used to generate multiple detector heads. Each detector head undergoes a convolution process to obtain position, type, and foreground confidence modules. The convolution process for category and position is the same as in the YOLO algorithm. The foreground confidence module is used to evaluate the standardized distance from the current position to the target center of the previous position, i.e.: in as well as These correspond to the coordinates of the current target detection box in the four directions: left, right, up, and down. The foreground confidence ranges from 0 to 1, and the binary cross-entropy (BCE) loss is used for training. Steps 3-4: The final network output is trained using a loss function to iterate the model parameters; the loss function is constructed as follows: in and These represent the losses for targets with and without known foregrounds, respectively. The adjustment weight between the two losses; The calculation is as follows: in For cross-entropy loss, These represent the category output by the model and the true category, respectively. These represent the foreground confidence and the true confidence of the model output, respectively. This represents the weighting error; This represents the location regression error; These represent the four locations of the actual and estimated detection boxes, respectively. For unknown targets and background bounding boxes, regression is performed only on their categories; loss function It is expressed as follows: Where UN represents an unknown target and BG represents a background target. and Each represents a different candidate box; Steps 3-5: During the testing phase, the judgment of unknown targets is composed of the following formulas: in C1 This represents the portion of all candidate boxes with a confidence threshold greater than 0.8 after excluding known targets. C2 This represents the top 3 candidate boxes after removing known targets; C3 This represents the portion of all candidate boxes with an FP score greater than 0.9 after removing known targets; the final selected candidate boxes are recorded as "Unknown" and are considered as unknown candidate boxes that do not exist in the training set. Steps 3-6: Test the model results using real bounding box labels on the test dataset; Step 4: Use an unsupervised adaptive model to perform unsupervised domain adaptation on the test set to reduce the domain difference between the test set and the training set; Step 5: After domain adaptation is completed, the model performs inference on the test set, labels the unknown categories, compares the results with the true values, and provides a quantitative evaluation of the model.
2. The spatial unknown target recognition method based on foreground metric learning according to claim 1, characterized in that, Step 4 is described in detail below: Step 4-1: First, use the pre-trained model to perform inference on the test dataset. After inference, labels are generated for the test dataset. Using the generated FP labels, construct the FP loss function and train the model again. The loss function is in the form of cross-entropy loss. Step 4-2: Use the current FP value to filter the test dataset; based on the TOPK algorithm, use the candidate boxes corresponding to the top N FP scores as the target set, and delete the rest of the data; Step 4-3: Fine-tune the model using the constructed dataset; the fine-tuning refers to fixing the feature extraction model and only performing it on the candidate box generation module and the detection head module, while setting the learning rate to 1e-5; Step 4-4: Repeat steps 4-1 to 4-3 until the required number of iterations is reached; save the model and output it.
3. The spatial unknown target recognition method based on foreground metric learning according to claim 2, characterized in that, The location regression error is calculated using the IOU loss function.