A non-cooperative space target template fast matching method and device

Abstract

The application discloses a kind of non-cooperative space target template fast matching method and device, belong to space detection and image processing field.The method includes: according to the three-dimensional scattering model of non-cooperative space target, according to the angle resolution of pre-set generation image template library covering different observation angle and illumination angle;All template images in the image template library and the multiple real-time target images of the target to be estimated pose are sequentially carried out image segmentation and structure analysis, respectively to obtain the structural characteristic value of the template image and the target image;Randomly match an initial template image for each target image, and according to the structural characteristic value, the target image and the initial template image are iteratively matched to determine the best matching template of the target image;According to the observation angle and the illumination angle corresponding to the best matching template, the real-time pose of the target to be estimated pose is calculated.The application can improve the pose matching estimation speed of non-cooperative space target.

Description

Technical Field

[0001] This invention relates to the field of space exploration and image processing technology, and in particular to a method and apparatus for rapid matching of non-cooperative space target templates. Background Technology

[0002] Non-cooperative space target attitude estimation refers to determining the three-dimensional attitude of a target spacecraft during its orbital operation through remote sensing observations such as optics and radar, without relying on any prior information or cooperative signals from the target spacecraft. Such targets are typically defunct satellites, space debris, etc. The core technology lies in extracting information from the target's appearance, structure, and motion characteristics to calculate its tumbling state. This requires extremely high autonomy from the sensing algorithm. Non-cooperative space target attitude estimation is crucial for space safety and mission execution, and is a core prerequisite for critical operations such as collision warning and faulty target acquisition.

[0003] Currently, the main method for estimating an object's attitude is to observe the target using ground-based or space-based telescopes and then infer its attitude based on the acquired images. This is the most direct and effective observation and estimation path under current technological conditions. However, since the image template library can be tens or even millions of images, each image to be matched needs to be searched and compared individually, resulting in an excessively long overall time consumption, making it difficult to meet real-time requirements.

[0004] Therefore, there is an urgent need for a method and apparatus for rapid matching of non-cooperative spatial target templates to solve the above-mentioned technical problems. Summary of the Invention

[0005] This invention provides a method and apparatus for fast template matching of non-cooperative space targets, which can improve the speed of attitude matching and estimation of non-cooperative space targets. The technical solution is as follows: On the one hand, a fast matching method for target templates in non-cooperative spaces is provided, the method comprising: Based on the three-dimensional scattering model of non-cooperative spatial targets, an image template library covering different observation angles and illumination angles is generated according to a preset angular resolution. Image segmentation and structural analysis are performed sequentially on all template images in the image template library and multiple real-time target images of the target whose pose is to be estimated, respectively, to obtain the structural feature values ​​of the template images and the target images. For each target image, an initial template image is randomly matched, and the target image and the initial template image are iteratively matched based on structural feature values ​​to determine the best matching template for the target image; The real-time attitude of the target to be estimated is calculated based on the observation angle and illumination angle corresponding to the best matching template.

[0006] On the other hand, a rapid matching device for non-cooperative spatial target templates is provided, the device comprising: The generation module is used to generate an image template library covering different observation angles and illumination angles based on the three-dimensional scattering model of non-cooperative spatial targets and according to a preset angular resolution. The parsing module is used to sequentially perform image segmentation and structural parsing on all template images in the image template library and multiple real-time target images of the target to be estimated, so as to obtain the structural feature values ​​of the template images and the target images respectively; The matching module is used to randomly match an initial template image for each target image, and to perform iterative matching between the target image and the initial template image based on structural feature values ​​to determine the best matching template for the target image; The calculation module is used to calculate the real-time attitude of the target to be estimated based on the observation angle and illumination angle corresponding to the best matching template.

[0007] On the other hand, a computer device is provided, the computer device including a memory and a processor, the memory for storing a computer program, and the processor for executing the computer program stored in the memory to implement the steps of the non-cooperative spatial target template fast matching method described above.

[0008] On the other hand, a computer-readable storage medium is provided, wherein a computer program is stored therein, and when the computer program is executed by a processor, it implements the steps of the above-described fast matching method for non-cooperative spatial target templates.

[0009] On the other hand, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the fast matching method for non-cooperative spatial target templates described above.

[0010] The technical solution provided by this invention can bring at least the following beneficial effects: First, based on the three-dimensional scattering model of the target, an image template library covering different illumination angles and observation angles is generated; then, for each template image in the image template library, the search range is narrowed based on the matching result of the previous frame image by utilizing the continuous attitude change of the image sequence of non-cooperative targets; finally, the target image whose attitude is to be estimated is compared one by one with the images in the template library, and the target attitude is determined by similarity. This method utilizes attitude continuity to narrow the search range, significantly improves the template matching speed, meets the real-time estimation requirements, and is suitable for scenarios such as space debris early warning and faulty satellite capture. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a flowchart of a method for fast matching of non-cooperative spatial target templates provided in an embodiment of the present invention; Figure 2 This is a structural diagram of a non-cooperative spatial target template rapid matching device provided in an embodiment of the present invention; Figure 3 This is a hardware architecture diagram of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0014] As mentioned earlier, traditional matching methods have image template libraries that are tens or even millions of images, requiring each image to be searched and compared individually, resulting in excessively long overall processing time and making it difficult to meet real-time requirements.

[0015] Based on this, the concept of the present invention is to narrow the search range based on the matching results of the previous frame image, thereby greatly improving the template matching speed and meeting the real-time estimation requirements.

[0016] The following describes the specific implementation of the above concept.

[0017] Please refer to Figure 1 The present invention provides a method for fast matching of target templates in non-cooperative space, the method comprising: Step 100: Based on the three-dimensional scattering model of the non-cooperative space target, generate an image template library covering different observation angles and illumination angles according to a preset angular resolution; Step 102: Perform image segmentation and structural analysis on all template images in the image template library and multiple real-time target images of the target to be estimated, respectively, to obtain the structural feature values ​​of the template images and the target images; Step 104: Randomly match an initial template image for each target image, and perform iterative matching between the target image and the initial template image based on structural feature values ​​to determine the best matching template for the target image; Step 106: Calculate the real-time attitude of the target to be estimated based on the observation angle and illumination angle corresponding to the best matching template.

[0018] In this embodiment of the invention, an image template library covering different illumination and observation angles is first generated based on the target's three-dimensional scattering model. Then, for each template image in the library, the search range is narrowed based on the matching results of the previous frame, utilizing the continuous attitude variation of the image sequence of non-cooperative targets. Finally, the target image whose attitude is to be estimated is compared one by one with the images in the template library, and the target attitude is determined through similarity. This method utilizes attitude continuity to narrow the search range, significantly improving template matching speed, meeting real-time estimation requirements, and is suitable for scenarios such as space debris early warning and faulty satellite capture.

[0019] The following description Figure 1 The execution method of each step is shown.

[0020] First, for step 100, based on the three-dimensional scattering model of the non-cooperative spatial target, an image template library covering different observation angles and illumination angles is generated according to a preset angular resolution.

[0021] In this embodiment of the invention, an image template library covering different observation angles and illumination angles is generated based on a preset angular resolution (e.g., illumination angle 30° / step, observation angle 10° / step) and a non-cooperative target three-dimensional scattering model. Each template corresponds to a unique index value and can be directly called through the index.

[0022] Then, for step 102, image segmentation and structural analysis are performed sequentially on all template images in the image template library and multiple real-time target images of the target to be estimated, respectively, to obtain the structural feature values ​​of the template images and the target images.

[0023] In this embodiment of the invention, for all template images in the image template library and multiple real-time target images of the target whose pose needs to be estimated, the target regions are extracted by image segmentation algorithm, and then the target structure is analyzed by structural analysis algorithm. Finally, structural feature values ​​are extracted as the basis for similarity calculation.

[0024] Specifically, target extraction uses threshold segmentation or edge detection algorithms, while structural analysis uses contour extraction or component recognition algorithms.

[0025] The structural feature values ​​include at least one of the target profile moment and the relative position parameters of the components.

[0026] For step 104, an initial template image is randomly matched for each target image, and the target image and the initial template image are iteratively matched according to the structural feature value to determine the best matching template for the target image.

[0027] In this embodiment of the invention, the optimal matching template is obtained in the following manner: Step S21: Iteratively match the target images in ascending order of image frame number; Step S22: Perform iterative matching again on the target images that have undergone the iterative matching, according to the order of image frame number from largest to smallest; Step S23: Repeat steps S21-S22 until the matching results of all target images and template images remain unchanged for two consecutive rounds, and determine the template image at the end of the iteration as the best matching template for each target image.

[0028] Specifically, firstly, an initial template image is randomly matched for each target image, and the index value of the initial template image is assigned to the corresponding target image as its initial index value.

[0029] Furthermore, the images are processed sequentially, from frame 1 to frame n: First, determine whether the target image is the first frame image. If so, determine the candidate template image for the first frame image based on the first frame image and the corresponding initial template image.

[0030] Specifically, based on the preset search radius, random spatial matching of the target is performed with the angle data of the first frame image and the corresponding initial template image as the center, resulting in multiple template images to be matched.

[0031] In this embodiment, it is assumed that the angle of the initial template image is In four-dimensional space, with this as the center, the search radius is set to 10 angular resolution units (e.g., if the angular resolution is 10°, then the search radius is 100°). Within this radius, 10 random matches are performed. Each search randomly selects a template library index within a fixed radius to obtain multiple template images to be matched.

[0032] The similarity between the first frame image and the initial template image and all template images to be matched is calculated based on the structural feature values. Based on the calculation results, the template image with the highest similarity is determined as the candidate template image of the first frame image.

[0033] In this embodiment of the invention, similarity calculation uses Euclidean distance or cosine similarity of structural feature values.

[0034] Next, if the target image is not the first frame image, the preferred template image of the current frame image is determined based on the current frame image, the corresponding initial template image, and the candidate template image of the previous frame image.

[0035] In this embodiment, angles are combined based on the neighborhood values ​​of the angle data of the candidate template image in the previous frame in four-dimensional space to generate multiple sets of derived angle data.

[0036] In this embodiment, the angle data includes the illumination azimuth angle, the illumination elevation angle, the observation azimuth angle, and the observation elevation angle.

[0037] For example, suppose the angle of the candidate template image in the previous frame is the illumination azimuth angle. Illumination pitch angle Observation azimuth and observation pitch angle In four-dimensional space, take All the nearest neighbor angle combinations, for example, taking all the direct neighbors of each angle (each dimension takes three values: -1, 0, +1), a total of 27 angle combinations are obtained.

[0038] Next, the derived template image corresponding to each set of derived angle data is determined, and the similarity between the current frame image and the corresponding initial template image and all derived template images is calculated based on the structural feature value. Based on the calculation results, the template image with the highest similarity is determined as the initial candidate template of the current frame image.

[0039] For example, these 27 angle combinations correspond to 27 template images. The similarity calculation result between the current image and these 27 template images is denoted as...

[0040] Similarity of initial candidate templates in the current frame image It can then be calculated using the following formula: in, This represents the similarity between the current frame image and the corresponding initial template image.

[0041] Subsequently, similar to the process of processing the first frame image, based on the preset search radius, random spatial matching of the target number is performed with the angle data of the current frame image and the corresponding initial template image as the center, to obtain multiple template images to be matched; The similarity between the first frame image and the initial candidate template image and all template images to be matched is calculated based on the structural feature values. Based on the calculation results, the template image with the highest similarity is determined as the candidate template image for the current frame image.

[0042] After completing the above forward iterative matching, repeat the above reverse iterative matching process in reverse order of image frame number, i.e., frame n → frame 1, to further optimize the index value based on the pose continuity of the reverse sequence.

[0043] Finally, by alternately performing forward and reverse iterative matching, until the matching results of all target images remain unchanged for two consecutive rounds, that is, the index value corresponding to the target image remains unchanged for two consecutive rounds, the matching is determined to be converged. At this time, the template image pointed to by the index of each image is the best matching template.

[0044] After completing the above matching process, the real-time attitude of the non-cooperative space target can be obtained based on the observation angle and illumination angle parameters corresponding to the best matching template of each target image.

[0045] The feasibility of the above method is verified by an example below: Template library generation: Based on a 3D model of a non-cooperative spatial target, the angular resolution is set to 30° for illumination azimuth, 30° for illumination elevation, 10° for observation azimuth, and 10° for illumination elevation. A template library covering illumination azimuth 0~360°, illumination elevation -90°~90°, observation azimuth 0°~180°, and observation elevation -90°~90° is generated, with a total of 46,656 templates and index values ​​from 1 to 46,656. Image sequence to be processed: n=10000 images of non-cooperative space targets taken by space telescopes, with a frame interval attitude change angle of 1°; Feature extraction: Canny edge detection was used to extract the target contour, and seven invariant moments were calculated as structural feature values; Iteration parameter settings: search radius 100°, 4 iterations (2 forward and 2 backward); Matching results: After one round of forward iteration, 80% of the image indices converged; after one round of backward iteration, 95% of the image indices converged. After four rounds of iteration, the indexes of all 10,000 images remained unchanged, the matching time was reduced from 120,000 seconds in the traditional method to 12 seconds, and the pose estimation error was better than 10°, meeting the requirements for real-time early warning.

[0046] Please refer to Figure 2 This invention provides a rapid matching device for non-cooperative spatial target templates, the device comprising: The generation module 200 is used to generate an image template library covering different observation angles and illumination angles according to a preset angular resolution based on the three-dimensional scattering model of the non-cooperative space target. The parsing module 202 is used to sequentially perform image segmentation and structural analysis on all template images in the image template library and multiple real-time target images of the target to be estimated, so as to obtain the structural feature values ​​of the template images and the target images respectively; The matching module 204 is used to randomly match an initial template image for each target image, and to perform iterative matching between the target image and the initial template image based on structural feature values ​​to determine the best matching template for the target image; The calculation module 206 is used to calculate the real-time attitude of the target to be estimated based on the observation angle and illumination angle corresponding to the best matching template.

[0047] In this embodiment of the invention, the step of iteratively matching the target image and the initial template image based on structural feature values ​​to determine the best matching template for the target image includes: step S21, iteratively matching the target images in ascending order of image frame number; step S22, iteratively matching the target images that have undergone the iterative matching again in descending order of image frame number; step S23, repeating steps S21-S22 until the matching results of all target images and template images remain unchanged for two consecutive rounds, and determining the template image at the end of the iteration as the best matching template for each target image.

[0048] In this embodiment of the invention, the iterative matching of the target image includes: determining whether the target image is a first frame image; if so, determining a candidate template image of the first frame image based on the first frame image and the corresponding initial template image; otherwise, determining a preferred template image of the current frame image based on the current frame image, the corresponding initial template image, and the candidate template image of the previous frame image.

[0049] In this embodiment of the invention, determining the candidate template image of the first frame image based on the first frame image and the corresponding initial template image includes: performing random spatial matching of the target number of times based on the angle data of the first frame image and the corresponding initial template image as the center according to a preset search radius, to obtain multiple template images to be matched; calculating the similarity between the first frame image and the initial template image and all template images to be matched based on structural feature values, and determining the template image with the highest similarity as the candidate template image of the first frame image based on the calculation results.

[0050] In this embodiment of the invention, determining the preferred template image of the current frame image based on the current frame image, the corresponding initial template image, and the candidate template images of the previous frame image includes: combining angles based on the neighborhood values ​​of the angle data of the candidate template images of the previous frame image in four-dimensional space to generate multiple sets of derived angle data; determining the derived template image corresponding to each set of derived angle data, and calculating the similarity between the current frame image and the corresponding initial template image and all derived template images based on structural feature values, and determining the template image with the highest similarity as the initial candidate template image of the current frame image based on the calculation results; performing random spatial matching of the target number of times based on the angle data of the current frame image and the corresponding initial template image as the center according to a preset search radius to obtain multiple template images to be matched; calculating the similarity between the first frame image and the initial candidate template image and all template images to be matched based on structural feature values, and determining the template image with the highest similarity as the candidate template image of the current frame image based on the calculation results.

[0051] In this embodiment of the invention, the angle data includes the illumination azimuth angle, the illumination elevation angle, the observation azimuth angle, and the observation elevation angle.

[0052] It should be noted that the non-cooperative space target template rapid matching device provided in the above embodiments is only an example of the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the non-cooperative space target template rapid matching device and the non-cooperative space target template rapid matching method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process is detailed in the method embodiments, which will not be repeated here.

[0053] Embodiments of this application also provide a computer device, please refer to... Figure 3 The computer device includes a processor and a memory, the memory storing at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, at least one program, code set, or instruction set is loaded and executed by the processor to implement the non-cooperative spatial target template fast matching method provided in the above-described method embodiments.

[0054] Embodiments of this application also provide a computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, at least one program, code set, or instruction set is loaded and executed by a processor to implement the non-cooperative spatial target template fast matching method provided in the above-described method embodiments.

[0055] Embodiments of this application also provide a computer program product, which includes a computer program. A processor of a computer device reads the computer program from a computer-readable storage medium and executes the computer program, causing the computer device to perform any of the non-cooperative spatial target template fast matching methods described in the above embodiments.

[0056] For ease of description, the above systems or devices are described separately as various modules or units based on their functions. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware components.

[0057] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0058] Finally, it should be noted that in this document, relational terms such as first, second, third, and fourth are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0059] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for fast matching of non-cooperative spatial target templates, characterized in that, The method includes: Based on the three-dimensional scattering model of non-cooperative spatial targets, an image template library covering different observation angles and illumination angles is generated according to a preset angular resolution. Image segmentation and structural analysis are performed sequentially on all template images in the image template library and multiple real-time target images of the target whose pose is to be estimated, respectively, to obtain the structural feature values ​​of the template images and the target images. For each target image, an initial template image is randomly matched, and the target image and the initial template image are iteratively matched based on structural feature values ​​to determine the best matching template for the target image; The real-time attitude of the target to be estimated is calculated based on the observation angle and illumination angle corresponding to the best matching template.

2. The method as described in claim 1, characterized in that, The step of iteratively matching the target image and the initial template image based on structural feature values ​​to determine the optimal matching template for the target image includes: Step S21: Iteratively match the target images in ascending order of image frame number; Step S22: Perform iterative matching again on the target images that have undergone the iterative matching, according to the order of image frame number from largest to smallest; Step S23: Repeat steps S21-S22 until the matching results of all target images and template images remain unchanged for two consecutive rounds, and determine the template image at the end of the iteration as the best matching template for each target image.

3. The method as described in claim 2, characterized in that, The iterative matching of the target image includes: Determine whether the target image is the first frame image. If so, determine the candidate template image for the first frame image based on the first frame image and the corresponding initial template image. Otherwise, the preferred template image for the current frame image is determined based on the current frame image, the corresponding initial template image, and the candidate template image of the previous frame image.

4. The method as described in claim 3, characterized in that, The step of determining the candidate template image for the first frame image based on the first frame image and the corresponding initial template image includes: Based on the preset search radius, random spatial matching of the target is performed with the angle data of the first frame image and the corresponding initial template image as the center, resulting in multiple template images to be matched; The similarity between the first frame image and the initial template image and all template images to be matched is calculated based on the structural feature values. Based on the calculation results, the template image with the highest similarity is determined as the candidate template image of the first frame image.

5. The method as described in claim 3, characterized in that, The step of determining the preferred template image for the current frame image based on the current frame image, the corresponding initial template image, and the candidate template image of the previous frame image includes: Based on the angle data of the candidate template image in the previous frame, the angle is combined in the neighborhood value in four-dimensional space to generate multiple sets of derived angle data; Determine the derived template image corresponding to each set of derived angle data, calculate the similarity between the current frame image and the corresponding initial template image and all derived template images based on the structural feature values, and determine the template image with the highest similarity as the initial candidate template image of the current frame image based on the calculation results; Based on the preset search radius, random spatial matching of the target is performed with the angle data of the current frame image and the corresponding initial template image as the center, resulting in multiple template images to be matched; The similarity between the first frame image and the initial candidate template image and all template images to be matched is calculated based on the structural feature values. Based on the calculation results, the template image with the highest similarity is determined as the candidate template image for the current frame image.

6. The method as described in claim 4 or 5, characterized in that, The angle data includes the illumination azimuth angle, the illumination elevation angle, the observation azimuth angle, and the observation elevation angle.

7. A rapid matching device for non-cooperative spatial target templates, characterized in that, The device includes: The generation module is used to generate an image template library covering different observation angles and illumination angles based on the three-dimensional scattering model of non-cooperative spatial targets and according to a preset angular resolution. The parsing module is used to sequentially perform image segmentation and structural parsing on all template images in the image template library and multiple real-time target images of the target to be estimated, so as to obtain the structural feature values ​​of the template images and the target images respectively; The matching module is used to randomly match an initial template image for each target image, and to perform iterative matching between the target image and the initial template image based on structural feature values ​​to determine the best matching template for the target image; The calculation module is used to calculate the real-time attitude of the target to be estimated based on the observation angle and illumination angle corresponding to the best matching template.

8. A computer device, characterized in that, The computer device includes a memory and a processor. The memory is used to store computer programs, and the processor is used to execute the computer programs stored in the memory to implement the steps of the method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the steps of the method described in any one of claims 1-6.

10. A computer program product, characterized in that, Includes a computer program, which, when executed by a processor, implements the steps of the method according to any one of claims 1-6.

Images