Zero-sample spacecraft target detection method combining semantic segmentation and image semantic mapping large model
By combining semantic segmentation and image semantic mapping large model, the problem of non-cooperative object detection in complex space environments is solved, and robust and efficient object detection under zero-sample conditions is achieved.
Patent Information
- Application Number
- CN202411703777.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-26
- Publication Date
- 2025-03-14
AI Technical Summary
Spacecraft target detection faces challenges in complex and changing space environments, especially detection of non-cooperative targets, making it difficult to achieve effective detection without cooperative information and severe lighting changes.
A zero-sample spacecraft object detection method combining semantic segmentation and image semantic mapping large model is adopted to generate candidate masks through semantic segmentation, and the target matching mask is selected using image semantic mapping to finally solve the target detection position.
It realizes robust and generalizable detection of spacecraft targets under zero sample data conditions, which is suitable for complex space environments and improves domain adaptability and detection accuracy.
Smart Images
Figure CN119625309A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of aerospace technology, and in particular to a zero-sample spacecraft target detection method combining semantic segmentation and an image semantic mapping large model. Background Art
[0002] As the importance of space exploration continues to increase, the number of tasks related to space situational awareness and on-orbit servicing is also growing. Among them, spacecraft target detection is a key component of space missions. In general, spacecraft can be divided into cooperative targets and non-cooperative targets, depending on whether their three-dimensional models are known or whether they have cooperative markers. The detection of cooperative targets is relatively mature and has been successfully applied in orbit. However, due to the inability to communicate and the lack of prior information and cooperative markers. The detection of non-cooperative targets (such as space debris and failed satellites) remains a major challenge, especially in complex and changing space environments.
[0003] Spacecraft target detection technology faces challenges such as complex space environment and measurement targets without cooperative information. There are a large number of stray light backgrounds formed by stars and artificial celestial bodies in the space environment. The reflection of the surface coating of the measured target and the drastic changes in illumination caused by the change of the line of sight angle make the target appear bright in the image and the illumination is complex and changeable. Deep learning methods can solve the robustness problem in complex environments to a certain extent. However, these methods require a large number of training data sets for specific scenes to operate effectively. Due to the limitations of computing resources and communication costs, it is very difficult to obtain a large number of comprehensive data sets for spacecraft target detection tasks. Therefore, how to achieve a robust and generalizable target detection model in a specific context using small or zero sample data without additional training has become a key problem that needs to be solved urgently.
[0004] Therefore, there is an urgent need for a zero-shot spacecraft target detection method that combines semantic segmentation and image semantic mapping models to complete the spacecraft target detection task and obtain a large number of comprehensive data sets. Summary of the invention
[0005] In order to solve the problems existing in the prior art such as difficulty in obtaining training data and poor model domain adaptability, this application proposes a zero-shot spacecraft target detection method combining semantic segmentation and image semantic mapping large model, including the following steps:
[0006] S1. Obtain a spacecraft target image, analyze the spacecraft image to obtain pose points, calculate a target frame based on the pose points, and generate a data set for spacecraft target detection based on the target image and the target frame;
[0007] S2, design model parameters and combine semantic segmentation and image semantic mapping large model to build a zero-shot spacecraft target detection system;
[0008] S3. Use the zero-shot spacecraft target detection system to perform target detection on the dataset.
[0009] Preferably, the specific content of the target frame calculated according to the pose points includes:
[0010]
[0011] in, represents the two-dimensional coordinates of the i-th point obtained from the j-th image, P i is the i-th feature point, K is the camera internal matrix; represents the three-dimensional homogeneous coordinates in B; [R BC |t BC ] j is the position vector and direction matrix of the jth image; where R BC is the rotation matrix of the image coordinate system relative to the target body coordinate system, t BC is the translation vector, q is a quaternion, q=[q w ,q x ,q y ,q z ].
[0012] Preferably, the zero-shot target detection network includes a prompt information generation module, a candidate mask generation module, an optimal mask selection module, and a target position solution module;
[0013] The prompt information generation module is used to screen the target image to obtain the prompt point P;
[0014] The candidate mask generation module obtains the candidate mask using the prompt point P;
[0015] The optimal mask selection module selects a target matching mask using a zero-shot spacecraft target detection system;
[0016] The target position solving module uses the target matching mask to solve and obtain the target detection position.
[0017] Preferably, the specific contents of screening the target image to obtain the target prompt point P include:
[0018] Input the target image into the prompt information generation module to obtain a number of given pixels;
[0019] For a given pixel, the accelerated fragment test point detection method is used to check the intensity of the pixel and surrounding pixels in the circular pattern, and the pixels that meet the conditions are screened to obtain the corner points;
[0020] Use non-maximum suppression to obtain the target corner points, remove the weaker corner points, and calculate the scores of the pixels corresponding to the target corner points;
[0021] The candidate point p with a score higher than all neighboring candidate points within the target radius is retained as the key point, i.e., the target prompt point, and the target prompt point P is output.
[0022] Preferably, the specific steps of obtaining the candidate mask using the target cue point P include:
[0023] Read in the dataset of the target pose estimation image and adjust the input image size;
[0024] Feed the target cue point P into the candidate mask generation module;
[0025] Using the semantic segmentation large model, candidate masks are generated.
[0026] Preferably, the specific steps of selecting a target matching mask using a zero-shot spacecraft target detection system include:
[0027] The optimal mask selection module reads the candidate masks and designs text prompt information according to the task scenario. The text prompt information is to analyze the current scene and design semantic information to distinguish the target from other objects.
[0028] The textual prompt information is fed into the zero-shot spacecraft object detection system, and the object matching mask is selected based on the image-text similarity.
[0029] Preferably, the expression for calculating the fraction of pixels corresponding to the target corner point is:
[0030]
[0031] Among them, I p is the absolute intensity of the candidate pixel; is the absolute intensity of the surrounding pixels in the S region; N s is the number of surrounding pixels.
[0032] Preferably, the specific content of the image-text similarity is the cosine similarity between the mask image code and the prompt point P code;
[0033] The formula for calculating cosine similarity is:
[0034]
[0035] Among them, M i is the candidate mask; t j is the prompt text; VE() is the mask image encoding; TE() is the prompt text encoding.
[0036] Preferably, the target position solving module uses the target matching mask to solve the target detection position expression:
[0037]
[0038] Among them, M best is the optimal mask, M best is the optimal mask, x, y are pixel coordinates, x min ,y min They are the minimum pixel coordinates of the target box, i.e. the lower left corner, x max ,y max are the maximum pixel coordinates of the target box respectively.
[0039] A zero-sample spacecraft target detection device that combines semantic segmentation and a large model of image semantic mapping includes a storage device, a processor, and a computer program stored in the storage device and executable on the processor. The device is characterized in that the processor executes the computer program to implement a zero-sample spacecraft target detection method that combines semantic segmentation and a large model of image semantic mapping.
[0040] In summary, the zero-sample spacecraft target detection method of the present invention, which combines semantic segmentation and image semantic mapping large models, has strong universality compared to traditional technologies. It is not only applicable to most target detection tasks targeting small and medium-sized spacecraft, but also applicable to the general zero-sample / small-sample target detection field. The present invention integrates the semantic segmentation large model and the image and semantic mapping large model to provide a zero-sample target detection solution for dynamic and complex space environments. Since the present invention uses a large model to train weights and does not rely on specific data to retrain the model, the present invention has good domain adaptability.
[0041] The technical method of the present invention is further described in detail below through the drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] Figure 1 This is a schematic diagram of the principle of a zero-sample spacecraft target detection method that combines semantic segmentation and an image semantic mapping large model in the present invention. DETAILED DESCRIPTION
[0043] The technical method of the present invention is further described below by means of the accompanying drawings and embodiments. It should be noted that unless otherwise specifically stated, the relative arrangement of the components and steps, numerical expressions and numerical values described in these embodiments do not limit the scope of the present application.
[0044] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present application, its application, or uses.
[0045] Technologies, systems, and devices known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, systems, and devices should be considered part of the specification.
[0046] In all examples shown and discussed herein, any specific values should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.
[0047] Unless otherwise defined, technical or scientific terms used in the present invention shall have the common meanings understood by one having ordinary skills in the field to which the present invention belongs.
[0048] The main purpose of the present invention is to solve some defects existing in the existing image compression technology, especially the problem that when compressing images in machine vision applications, the relationship between the image volume after compression and the retention of effective visual data information of the image is not fully considered, and then compressing the image according to a fixed compression ratio leads to the loss of effective visual information after image compression.
[0049] The present invention provides a zero-shot spacecraft target detection method combining semantic segmentation and image semantic mapping large model, comprising the following steps:
[0050] S1. Obtain a spacecraft target image, analyze the spacecraft image to obtain pose points, calculate a target frame based on the pose points, and generate a data set for spacecraft target detection based on the target image and the target frame;
[0051] Preferably, the specific content of the target frame calculated according to the pose points includes:
[0052]
[0053] in, represents the two-dimensional coordinates of the i-th point obtained from the j-th image, P i is the i-th feature point; K is the camera internal matrix; represents the three-dimensional homogeneous coordinates in B; [R BC |t BC ] j is the position vector and direction matrix of the jth image, where R BC is the rotation matrix of the image coordinate system relative to the target body coordinate system, t BC is the translation vector, q is a quaternion, q=[q w ,q x ,q y ,q z ].
[0054] S2, constructing a zero-shot spacecraft target detection system by network mapping the semantic segmentation model and the image semantic model;
[0055] like Figure 1As shown, the zero-shot target detection network includes a prompt information generation module, a candidate mask generation module, an optimal mask selection module, and a target position solution module;
[0056] The prompt information generation module is used to screen the target image to obtain the prompt point P;
[0057] The candidate mask generation module obtains the candidate mask using the prompt point P;
[0058] The optimal mask selection module selects a target matching mask using a zero-shot spacecraft target detection system;
[0059] The target position solving module uses the target matching mask to solve and obtain the target detection position.
[0060] Preferably, the specific contents of screening the target image to obtain the target prompt point P include:
[0061] Input the target image into the prompt information generation module to obtain a number of given pixels;
[0062] For a given pixel, the accelerated fragment test point detection method is used to check the intensity of the pixel and surrounding pixels in the circular pattern, and the pixels that meet the conditions are screened to obtain the corner points;
[0063] For a given pixel, its intensity is I p , consider N circles of radius r around a given pixel s pixels, these given pixels form a set If there is T in S c pixels, the intensity of these given pixels is higher than I p +T i , or below I p -T i , that is, the target threshold range K is, K>I P +T i Or K<I P -T i ; where T i is the intensity threshold, T c is the counting threshold, then the given point is classified as a corner point, S i is the set of surrounding pixels, T c is the corner point judgment threshold, I p is the intensity corresponding to pixel p, T i is the intensity screening threshold p is a corner point if and only if: I Sk >I p +T i or I Sk <I p -T i .
[0064] Use non-maximum suppression to obtain the target corner points, remove the weaker corner points, and calculate the scores of the pixels corresponding to the target corner points;
[0065] The candidate point p with a score higher than all neighboring candidate points within the target radius is retained as the key point, i.e., the target prompt point, and the target prompt point P is output.
[0066] Preferably, the specific steps of obtaining the candidate mask using the target cue point P include:
[0067] Read in the dataset of the target pose estimation image and adjust the input image size;
[0068] Feed the target cue point P into the candidate mask generation module;
[0069] Using the semantic segmentation large model, candidate masks are generated.
[0070] Preferably, the specific steps of selecting a target matching mask using a zero-shot spacecraft target detection system include:
[0071] The optimal mask selection module reads the candidate masks and designs text prompt information according to the task scenario. The text prompt information is to analyze the current scene and design semantic information to distinguish the target from other objects.
[0072] The textual prompt information is fed into the zero-shot spacecraft object detection system, and the object matching mask is selected based on the image-text similarity.
[0073] Preferably, the expression for calculating the fraction of pixels corresponding to the target corner point is:
[0074]
[0075] Among them, I p is the absolute intensity of the candidate pixel; is the absolute intensity of the surrounding pixels in the S region; N s is the number of surrounding pixels.
[0076] Preferably, the specific content of the image-text similarity is the cosine similarity between the mask image code and the prompt point P code;
[0077] The formula for calculating cosine similarity is:
[0078]
[0079] Among them, M i is the candidate mask; t j is the prompt text; VE() is the mask image encoding; TE() is the prompt text encoding.
[0080] Preferably, the target position solving module uses the target matching mask to solve the target detection position expression:
[0081]
[0082] Among them, M best is the optimal mask, x, y are pixel coordinates, x min ,y min They are the minimum pixel coordinates of the target box, i.e. the lower left corner, x max ,y max are the maximum pixel coordinates of the target box respectively.
[0083] S3. Use the zero-shot spacecraft target detection system to perform target detection on the dataset.
[0084] In S3 of this embodiment, the calculation method of F-score in target detection of a data set using a zero-sample spacecraft target detection system is:
[0085]
[0086] Among them: F-score is an indicator for evaluating target detection results, Precision is the accuracy, and Recall is the recall rate;
[0087]
[0088]
[0089] Among them, TP represents the correct classification of the true bounding box, FP represents the wrong classification, missing or misplaced object; FN represents the missing classification of the true object.
[0090] A zero-sample spacecraft target detection device that combines semantic segmentation and a large model of image semantic mapping includes a storage device, a processor, and a computer program stored in the storage device and executable on the processor. The device is characterized in that the processor executes the computer program to implement a zero-sample spacecraft target detection method that combines semantic segmentation and a large model of image semantic mapping.
[0091] Finally, it should be noted that the above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical method of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical method to deviate from the spirit and scope of the technical method of the present invention.
Claims
1. A zero-shot spacecraft target detection method combining semantic segmentation and image semantic mapping large model, characterized in that: The following steps are involved: S1. Obtain a spacecraft target image, and obtain the spacecraft position and attitude through a simulation environment or hardware platform, calculate the target frame based on the position and attitude points, and generate a data set for spacecraft target detection based on the target image and target frame; S2, constructing a zero-shot spacecraft target detection system by network mapping the semantic segmentation model and the image semantic model; S3. Use the zero-shot spacecraft target detection system to perform target detection on the dataset.
2. A zero-shot spacecraft target detection method combining semantic segmentation and image semantic mapping large model according to claim 1, characterized in that: The specific contents of the target frame calculated based on the pose points include: in, represents the two-dimensional coordinates of the i-th point obtained from the j-th image, P i is the i-th feature point; K is the camera internal matrix; represents the three-dimensional homogeneous coordinates in B; [R BC |t BC ] j is the position vector and direction matrix of the jth image, where R BC is the rotation matrix of the image coordinate system relative to the target body coordinate system, t BC is the translation vector, q is a quaternion, q=[q w ,q x ,q y ,q z ].
3. The zero-shot spacecraft target detection method combining semantic segmentation and image semantic mapping large model according to claim 2 is characterized in that: The zero-shot target detection network includes a prompt information generation module, a candidate mask generation module, an optimal mask selection module, and a target position solution module; The prompt information generation module is used to screen the target image to obtain the prompt point P; The candidate mask generation module obtains the candidate mask using the prompt point P; The optimal mask selection module selects a target matching mask using a zero-shot spacecraft target detection system; The target position solving module uses the target matching mask to solve and obtain the target detection position.
4. The zero-shot spacecraft target detection method combining semantic segmentation and image semantic mapping large model according to claim 3 is characterized in that: The specific contents of screening the target image to obtain the target prompt point P include: Input the target image into the prompt information generation module to obtain a number of given pixels; For a given pixel, the intensity of the pixel and surrounding pixels in the circular pattern is checked using the accelerated fragment test point detection method. A target threshold range is preset, and pixels within the target threshold range are screened to obtain corner points. Use non-maximum suppression to obtain the target corner points, remove the weaker corner points, and calculate the scores of the pixels corresponding to the target corner points; The candidate point p with a score higher than that of all neighboring candidate points within the target radius is retained as the key point, i.e., the target prompt point P, and the target prompt point P is output.
5. The zero-shot spacecraft target detection method combining semantic segmentation and image semantic mapping large model according to claim 4 is characterized in that: The specific steps of using the target cue point P to obtain the candidate mask include: Read in the dataset of the target pose estimation image and adjust the input image size; Feed the target cue point P into the candidate mask generation module; Using the semantic segmentation large model, candidate masks are generated.
6. The zero-shot spacecraft target detection method combining semantic segmentation and image semantic mapping large model according to claim 5, characterized in that: The specific steps of selecting the target matching mask using the zero-shot spacecraft object detection system include: The optimal mask selection module reads the candidate masks and designs text prompt information according to the task scenario. The text prompt information analyzes the current scene and designs semantic information to distinguish the target from other objects. The textual prompt information is fed into the zero-shot spacecraft object detection system, and the object matching mask is selected based on the image-text similarity.
7. The zero-shot spacecraft target detection method combining semantic segmentation and image semantic mapping large model according to claim 6, characterized in that: The expression for calculating the score of the pixel corresponding to the target corner point is: Among them, p is a candidate cue point, I p is the absolute intensity of the candidate pixel; I si is the absolute intensity of the surrounding pixels in the S region; N s is the number of surrounding pixels.
8. The zero-shot spacecraft target detection method combining semantic segmentation and image semantic mapping large model according to claim 7, characterized in that: The specific content of image-text similarity is the cosine similarity between the masked image encoding and the prompt point P encoding; The formula for calculating cosine similarity is: Among them, M i is the candidate mask; t j is the prompt text; VE() is the mask image encoding; TE() is the prompt text encoding.
9. The zero-shot spacecraft target detection method combining semantic segmentation and image semantic mapping large model according to claim 8, characterized in that: The target position solving module uses the target matching mask to solve the expression of the target detection position: Among them, M best is the optimal mask, x, y are pixel coordinates, x min ,y min They are the minimum pixel coordinates of the target box, i.e. the lower left corner, x max ,y max are the maximum pixel coordinates of the target box respectively.
10. A zero-shot spacecraft target detection device combining semantic segmentation and image semantic mapping large model, comprising a storage device, a processor, and a computer program stored in the storage device and executable on the processor, characterized in that: The processor executes the computer program to implement any spacecraft target detection method for aerospace missions as described in claims 1-9.