Spacecraft target detection method considering similarity and diversity
By building a spacecraft target detection network that considers similarity and diversity, the problem of insufficient robustness of spacecraft target detection in the prior art is solved, and real-time, high-accuracy target detection effect is achieved in complex contexts.
Patent Information
- Application Number
- CN202411703772.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-26
- Publication Date
- 2025-03-14
AI Technical Summary
The existing spacecraft target detection methods are not robust enough to deal with low signal-to-noise ratio and high contrast characteristics in complex backgrounds, making it difficult to meet the competitive needs of real-time processing and high accuracy.
A spacecraft object detection method considering similarity and diversity is proposed. By acquiring the spacecraft target images and posture points, a zero-sample object detection network is built, including an image processing module, a feature processing module, a vector generation module and a filtering fusion module. The root mean square error and adversarial learning loss function are used, and the target detection is achieved by combining geometric constraints and similarity selection techniques.
It significantly improves the accuracy and robustness of spacecraft target detection, and can achieve real-time processing and high accuracy in complex contexts, adapting to the needs of different fields.
Smart Images
Figure CN119625264A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the field of aerospace technology, and in particular to a spacecraft target detection method considering similarity and diversity. Background Art
[0002] With the rapid development of space technology, the tasks related to space situational awareness, autonomous on-orbit servicing and active debris removal are growing exponentially. Among these tasks, spacecraft target detection is a key technology that can quickly detect targets, ensure mission safety, and effectively manage the dynamics and complexity of space operations.
[0003] Sensors used for spacecraft target detection include radar and vision sensors. Vision sensors have become the preferred sensors for space target monitoring, space object monitoring, and autonomous rendezvous and docking missions due to their simple design, low cost, low power consumption, wide range of uses, and impressive real-time application capabilities. In recent years, with the advancement of computer vision, monocular vision technology has attracted widespread attention in space applications. Traditional target detection methods rely on hand-crafted features, but these features are unstable under strong lighting conditions, variable lighting conditions, low signal-to-noise ratio, and high-contrast features encountered in space images. Deep learning models have shown potential in improving target detection performance, and two main learning methods have emerged: single-stage and multi-stage detection methods. Multi-stage detectors propose areas where objects may exist and classify the areas in the first stage, and then classify these areas. On the other hand, single-stage detectors directly map image pixels to bounding box estimates, confidence, and classification results, using only one neural network, and some single-stage detectors have higher efficiency and are more suitable for real-time applications compared to some multi-stage detectors.
[0004] In the aerospace field, several datasets have been established that have significantly promoted the utilization of deep learning based methods. Previous studies have explored various deep learning methods for spacecraft target detection. However, for spacecraft target detection, existing methods are not robust enough and cannot adequately address the unique challenges in the space environment. Despite the success of deep learning methods in ground target detection tasks and initial attempts in space target detection tasks, some challenges remain for the following reasons: (1) spacecraft targets usually have low signal-to-noise ratio and high contrast features and are often observed under different lighting conditions, which may cause difficulties for accurate detection; (2) deep learning methods may have difficulty in meeting the competing requirements of real-time processing and high accuracy in spacecraft target detection tasks.
[0005] Therefore, there is an urgent need for a spacecraft target detection method that takes similarity and diversity into consideration to meet the competing requirements of real-time processing and high accuracy of spacecraft target detection tasks in complex backgrounds. Summary of the invention
[0006] In order to solve the problems existing in the prior art such as difficulty in obtaining training data and poor model domain adaptability, this application proposes a spacecraft target detection method considering similarity and diversity, including the following steps:
[0007] S1. Obtain a spacecraft target image, and obtain the spacecraft position and attitude through a simulation environment or hardware platform, calculate the target frame based on the position and attitude points, and generate a data set for spacecraft target detection based on the target image and target frame;
[0008] S2. Constructing a spacecraft target detection framework that considers similarity and diversity, wherein the spacecraft target detection framework that considers similarity and diversity is a zero-shot target detection network;
[0009] S3. Use the spacecraft target detection framework to perform target detection on the target image.
[0010] Preferably, the specific content of the target frame calculated according to the pose points includes:
[0011]
[0012] in, represents the two-dimensional coordinates of the i-th point obtained from the j-th image, P i is the i-th feature point; K is the camera internal matrix; represents the three-dimensional homogeneous coordinates in B; [R BC |t BC ] j is the position vector and direction matrix of the jth image, where R BC is the rotation matrix of the image coordinate system relative to the target body coordinate system, t BC is the translation vector, q is a quaternion, q=[q w ,q x ,q y ,q z ].
[0013] Preferably, the specific contents of the spacecraft target detection framework considering similarity and diversity include:
[0014] Image processing module: The image processing module is used to obtain a target image, crop and downsample the target image to obtain an input image;
[0015] Feature processing module: The feature processing module is used to extract and fuse the features of the input image to obtain the feature vector of the image. The feature processing module is composed of multiple feature extraction networks. A single feature extraction network extracts the features of the input image to obtain a single feature vector. The feature extraction network can be trained by a data set to obtain different network units. Multiple feature extraction units are fused to obtain the feature vector of the image.
[0016] Vector generation module: The vector generation module is composed of multiple vector generation networks. The feature vector of the image is input into the vector generation network and the multiple vector generation networks are overlapped to generate an overlapping vector O(B1, B2) = [o1, o2, o3, o4], and the candidate target frame is obtained according to the overlapping vector solution;
[0017] Filtering and fusion module: Filter and fuse the candidate target boxes to obtain the target bounding box.
[0018] Preferably, the specific content of obtaining the target image and performing cropping and downsampling processing on the target image to obtain the input image is:
[0019] Get the target image and crop the source image L0 of size 1×w0×h0 to get a size of 1×w s ×h s Image L s ;
[0020] For image L s The down-sampling process is performed to convert it into the input image of the feature extraction network.
[0021] Preferably, the feature processing module includes a feature extraction unit and a feature fusion unit;
[0022] The feature extraction unit uses different convolutional layers to better adapt to different object sizes to extract features L of various scales. F ;
[0023] The convolutional layer includes different padding parameters, stride parameters and kernel parameters, denoted as C mnp L, where m, n, and p represent padding, stride, and kernel size, respectively;
[0024] The activation function used in the convolutional layer adopts LeakyReLU;
[0025] The feature fusion unit is used to fuse the feature L F , to provide for L F A comprehensive representation of
[0026] Preferably, the downsampling process involves a uniform downsampling method, and the specific content of the uniform downsampling method is:
[0027]
[0028] in, is the nth sample cropping L of the original image L0 s The coordinate of the upper left corner, U[·] represents uniform distribution, u ll ,u rr , v ll and v rr are the left and right outer boundaries of the clipping area, and u lr ,u rl , v lr and v rl is the inner boundary;
[0029] The coordinates of the boundary are calculated as follows:
[0030]
[0031] Where (u t ,v t ) is the coordinate of the upper left corner of the target box, and its size is w t ×h t .
[0032] Preferably, the vector generation module selects root mean square error RMSE as the loss function;
[0033] The expression of root mean square error RMSE is:
[0034]
[0035] Preferably, the vector generation module adopts random discarding and adversarial learning methods, and introduces adversarial learning loss L A , where L A is defined as follows:
[0036]
[0037] Among them, O i is the label of the overlapping vector, δ = 1e -7 ;
[0038] The total loss is defined as:
[0039] L=L O +L A .
[0040] Preferably, the filtering and fusion module uses geometric constraints and similarity selection technology to set constraints, and filters the target bounding box through the constraints;
[0041] The constraints of geometric constraints are:
[0042] o1+o2≥w s +t w ;
[0043] o3+o4≥h s +t h ;
[0044] Among them, t w and t h are the minimum width and height of the target respectively.
[0045] A spacecraft target detection device that takes similarity and diversity into consideration includes a storage device, a processor, and a computer program stored in the storage device and executable on the processor. The device is characterized in that the processor executes the computer program to implement a spacecraft target detection method for aerospace missions.
[0046] In summary, the spacecraft target detection method of the present invention considering similarity and diversity has excellent performance in prediction accuracy and processing speed compared with traditional technologies, and has strong practicality in real-time applications. The present invention utilizes the diversity and similarity of models in target and suspicious areas, and this fusion method significantly improves the accuracy and robustness of target detection, especially under domain differences. The multi-model fusion strategy of the present invention, centered on diversity and similarity, proves its greater adaptability in different fields.
[0047] The technical method of the present invention is further described in detail below through the drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] Figure 1 The overall architecture diagram of a spacecraft target detection method considering similarity and diversity of the present invention;
[0049] Figure 2 This is a diagram of an overlapping vector estimation network architecture of a spacecraft target detection method considering similarity and diversity according to the present invention;
[0050] Figure 3 This is a feature extraction network architecture diagram of a spacecraft target detection method considering similarity and diversity according to the present invention;
[0051] Figure 4 The invention discloses a relationship diagram between overlapping vectors describing candidate target frames and candidate prediction frames of a spacecraft target detection method considering similarity and diversity. DETAILED DESCRIPTION
[0052] The technical method of the present invention is further described below by means of the accompanying drawings and embodiments. It should be noted that unless otherwise specifically stated, the relative arrangement of the components and steps, numerical expressions and numerical values described in these embodiments do not limit the scope of the present application.
[0053] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present application, its application, or uses.
[0054] Technologies, systems, and devices known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, systems, and devices should be considered part of the specification.
[0055] In all examples shown and discussed herein, any specific values should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.
[0056] Unless otherwise defined, technical or scientific terms used in the present invention shall have the common meanings understood by one having ordinary skills in the field to which the present invention belongs.
[0057] like Figure 1 As shown, the present invention provides a method for detecting spacecraft targets taking similarity and diversity into consideration, comprising the following steps:
[0058] S1. Obtain a spacecraft target image, and obtain the spacecraft position and attitude through a simulation environment or hardware platform, calculate the target frame based on the position and attitude points, and generate a data set for spacecraft target detection based on the target image and target frame;
[0059] The spacecraft target image refers to the image of the target spacecraft, and the spacecraft target image is obtained by observing the target spacecraft with a camera carried by the tracking spacecraft.
[0060] The data set consists of a training set and a test set. The S3 test actually tests the test set.
[0061] Furthermore, the specific contents of the target frame calculated according to the pose points include:
[0062]
[0063] in, represents the two-dimensional coordinates of the i-th point obtained from the j-th image, P i is the i-th feature point; K is the camera internal matrix; represents the three-dimensional homogeneous coordinates in B; [R BC |t BC ] j is the position vector and direction matrix of the jth image, where RBC is the rotation matrix of the image coordinate system relative to the target body coordinate system, t BC is the translation vector, q is a quaternion, q=[q w ,q x ,q y ,q z ].
[0064] S2. Constructing a spacecraft target detection framework that considers similarity and diversity, wherein the spacecraft target detection framework that considers similarity and diversity is a zero-shot target detection network;
[0065] like Figure 2 , Figure 3 , Figure 4 As shown in FIG. 1 , further, the specific contents of the spacecraft target detection framework considering similarity and diversity include:
[0066] Image processing module: The image processing module is used to obtain a target image, crop the target image, and perform downsampling processing to obtain an input image. The downsampling processing will reduce the scale of the input image.
[0067] Feature processing module: The feature processing module is used to extract and fuse the features of the input image to obtain the feature vector of the image. The feature processing module is composed of multiple feature extraction networks. A single feature extraction network extracts the features of the input image to obtain a single feature vector. The feature extraction network can be trained by a data set to obtain different network units. Multiple feature extraction units are fused to obtain the feature vector of the image.
[0068] Vector generation module: The vector generation module is composed of multiple vector generation networks. The feature vector of the image is input into the vector generation network and the multiple vector generation networks are overlapped to generate an overlapping vector O(B1, B2) = [o1, o2, o3, o4], and the candidate target frame can be obtained by solving the overlapping vector;
[0069] like Figure 4 As shown, the overlap vector describes the relationship between the candidate target box and the candidate prediction box, which is defined as:
[0070]
[0071] Among them, o1, o2 are width components, o3, o4 are length components, l T ,u T ,w T ,h T and l P ,u P ,w P ,h P They are candidate target boxes BT and candidate prediction box B P The left, top, width, and height of the .
[0072] Filtering and fusion module: Filter and fuse the candidate target boxes to obtain the target bounding box.
[0073] Furthermore, the target image is obtained, and the specific contents of the input image obtained by cropping and downsampling the target image are as follows:
[0074] Get the target image and crop the source image L0 of size 1×w0×h0 to get a size of 1×w s ×h s Image L s ;
[0075] For image L s The down-sampling process is performed to convert it into the input image of the feature extraction network.
[0076] Further, the feature processing module includes a feature extraction unit and a feature fusion unit;
[0077] The feature extraction unit uses different convolutional layers to better adapt to different object sizes to extract features L of various scales. F ;
[0078] The convolutional layer includes different padding parameters, stride parameters and kernel parameters, denoted as C mnp L, where m, n, and p represent padding, stride, and kernel size, respectively;
[0079] The activation function used in the convolutional layer adopts LeakyReLU;
[0080] The feature fusion unit is used to fuse the feature L F , to provide for L F A comprehensive representation of
[0081] Furthermore, the downsampling process involves a uniform downsampling method, and the specific content of the uniform downsampling method is:
[0082]
[0083] in, is the nth sample crop L of the original image L0 s The coordinate of the upper left corner, U[·] represents uniform distribution, u ll ,u rr , v ll and v rr are the left and right outer boundaries of the clipping area, and u lr ,u rl , v lr and vrl is the inner boundary;
[0084] The coordinates of the boundary are calculated as follows:
[0085]
[0086] Where (u t ,v t ) is the coordinate of the upper left corner of the target box, and its size is w t ×h t .
[0087] Furthermore, the vector generation module selects root mean square error RMSE as a loss function;
[0088] The expression of root mean square error RMSE is:
[0089]
[0090] Furthermore, the vector generation module adopts random dropout and adversarial learning methods, and introduces adversarial learning loss L A , where L A is defined as follows:
[0091]
[0092] Among them, O i is the label of the overlapping vector, δ = 1e -7 ;
[0093] The total loss is defined as:
[0094] L=L O +L A .
[0095] Furthermore, the filtering and fusion module uses geometric constraints and similarity selection technology to set constraints, and filters the target bounding box through the constraints;
[0096] The constraints of geometric constraints are:
[0097] o1+o2≥w s +t w ;
[0098] o3+o4≥h s +t h ;
[0099] Among them, t w and t h are the minimum width and height of the target respectively.
[0100] S3. Use the spacecraft target detection framework to perform target detection on the target image.
[0101] In S3 of this embodiment, the calculation method of F-score in target detection of the test data set using the zero-sample spacecraft target detection system is:
[0102]
[0103] Among them: F-score is an indicator for evaluating target detection results, Precision is the accuracy, and Recall is the recall rate;
[0104]
[0105] Among them, TP represents the correct classification of the true bounding box, FP represents the wrong classification, missing or misplaced object; FN represents the missing classification of the true object.
[0106] A spacecraft target detection device that takes similarity and diversity into consideration includes a storage device, a processor, and a computer program stored in the storage device and executable on the processor. The device is characterized in that the processor executes the computer program to implement a spacecraft target detection method for aerospace missions.
[0107] Finally, it should be noted that the above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical method of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical method to deviate from the spirit and scope of the technical method of the present invention.
Claims
1. A spacecraft target detection method considering similarity and diversity, characterized in that: The following steps are involved: S1. Obtain a spacecraft target image, and obtain the spacecraft position and attitude through a simulation environment or hardware platform, calculate the target frame based on the position and attitude points, and generate a data set for spacecraft target detection based on the target image and target frame; S2. Constructing a spacecraft target detection framework that considers similarity and diversity, wherein the spacecraft target detection framework that considers similarity and diversity is a zero-shot target detection network; S3. Use the spacecraft target detection framework to detect the target image.
2. A method for detecting spacecraft targets taking similarity and diversity into consideration according to claim 1, characterized in that: The specific contents of the target frame calculated based on the pose points include: in, represents the two-dimensional coordinates of the i-th point obtained from the j-th image, P i is the i-th feature point; K is the camera internal matrix; represents the three-dimensional homogeneous coordinates in B; [R BC |t BC ] j is the position vector and direction matrix of the jth image, where R BC is the rotation matrix of the image coordinate system relative to the target body coordinate system, t BC is the translation vector, q is a quaternion, q=[q w ,q x ,q y ,q z ].
3. The method for detecting spacecraft targets taking similarity and diversity into consideration according to claim 2, characterized in that: The specific contents of the spacecraft target detection framework considering similarity and diversity include: Image processing module: The image processing module is used to read the target image, and perform cropping and downsampling processing on the target image to obtain an input image; Feature processing module: The feature processing module is used to extract and fuse the features of the input image to obtain the feature vector of the image. The feature processing module is composed of multiple feature extraction networks. A single feature extraction network extracts the features of the input image to obtain a single feature vector. The feature extraction network can be trained by a data set to obtain different network units. Multiple feature extraction units are fused to obtain the feature vector of the image. Vector generation module: The vector generation module is composed of multiple vector generation networks. The feature vector of the image is input into the vector generation network and the multiple vector generation networks are overlapped to generate an overlapping vector O(B1, B2) = [o1, o2, o3, o4], and the candidate target frame is obtained according to the overlapping vector solution; Filtering and fusion module: Filter and fuse the candidate target boxes to obtain the target bounding box.
4. The method for detecting spacecraft targets taking similarity and diversity into consideration according to claim 3, characterized in that: Get the target image, crop and downsample the target image to get the specific content of the input image: Get the target image and crop the source image L0 of size 1×w0×h0 to get a size of 1×w s ×h s Image L s ; For image L s The down-sampling process is performed to transform it into the input image of the feature extraction network.
5. The method for detecting spacecraft targets taking similarity and diversity into consideration according to claim 4, characterized in that: The feature processing module includes a feature extraction unit and a feature fusion unit; The feature extraction unit uses different convolutional layers to adapt to different object sizes to extract features L of various scales. F ; The convolutional layer includes different padding parameters, stride parameters and kernel parameters, denoted as C mnp L, where m, n, and p represent padding, stride, and kernel size, respectively; The activation function used in the convolutional layer adopts LeakyReLU; The feature fusion unit is used to fuse the feature L F , to provide for L F comprehensive representation.
6. A method for detecting spacecraft targets taking similarity and diversity into consideration according to claim 5, characterized in that: The downsampling process involves a uniform downsampling method, and the specific contents of the uniform downsampling method are: in, is the nth sample crop L of the original image L0 s The coordinate of the upper left corner, U[·] represents uniform distribution, u ll ,u rr , v ll and v rr are the left and right outer boundaries of the clipping area, and u lr ,u rl , v lr and v rl is the inner boundary; The coordinates of the boundary are calculated as follows: Where (u t ,v t ) is the coordinate of the upper left corner of the target box, and its size is w t ×h t .
7. A method for detecting spacecraft targets taking similarity and diversity into consideration according to claim 6, characterized in that: The vector generation module selects the root mean square error RMSE as the loss function; The expression of root mean square error RMSE is:
8. The method for detecting spacecraft targets taking similarity and diversity into consideration according to claim 7, characterized in that: The vector generation module adopts random dropout and adversarial learning methods and introduces adversarial learning loss L A , where L A is defined as follows: Among them, O i is the label of the overlapping vector, δ = 1e -7 ; The total loss is defined as: L=L O +L A .
9. The method for detecting spacecraft targets taking similarity and diversity into consideration according to claim 8, characterized in that: The filtering and fusion module uses geometric constraints and similarity selection technology to set constraints, and filters the target bounding box through the constraints; The constraints of geometric constraints are: o1+o2≥w s +t w ; o3+o4≥h s +t h ; Among them, t w and t h are the minimum width and height of the target respectively.
10. A spacecraft target detection device considering similarity and diversity, comprising a storage device, a processor, and a computer program stored in the storage device and executable on the processor, characterized in that: The processor executes the computer program to implement any spacecraft target detection method for aerospace missions as described in claims 1-9.