Target identification matching method for star group collaborative optical observation

Abstract

The invention provides a target identification matching method for star group collaborative optical observation, which comprises the following steps: designing a star group configuration, establishing an inter-satellite communication network, and determining a master satellite and a slave satellite of a star group; an optical observation camera of a satellite group is utilized to complete observation image acquisition of a specified airspace, an image key frame is extracted, a key frame image of a slave satellite is transmitted to a master satellite, and key frame image alignment is implemented on the master satellite; carrying out target extraction on the key frame images of the primary satellite and the secondary satellite, and then carrying out primary satellite-secondary satellite target matching and secondary satellite-secondary satellite target matching to obtain a target identification matching result; and updating the star group configuration according to a target identification matching result, so that the star groups all point to the target. The target identification matching method provided by the invention has the autonomous information processing capability of multiple micro-nano satellites, and realizes multi-target identification of multi-view collaborative fusion.

Description

Technical Field

[0001] The present invention belongs to the technical field of space multi-satellite collaborative perception and control, and particularly relates to a method for target recognition and matching in constellation collaborative optical observation. Background Art

[0002] Based on the optical observation information of a microsatellite constellation, obtaining multi-view observation images of a target and achieving collaborative recognition of space targets through the matching of multi-view observation images is an important development direction for future space situation awareness.

[0003] The difficulties in target recognition and matching based on constellation collaborative optical observation lie in constellation configuration design, redundant images for each view, target matching between multiple views, etc., resulting in slow processing speed and low recognition accuracy of existing methods, and they still cannot meet space-based applications. Therefore, how to reasonably and effectively design a constellation configuration that meets the observation requirements of a specified space area, and then design mechanisms such as constellation communication networks, compression of multi-view redundant images, and cyclic discrimination of multiple targets between views, remains a difficult problem. Summary of the Invention

[0004] In order to overcome the deficiencies in the prior art, the inventors of the present invention have conducted intensive research and provided a method for target recognition and matching in constellation collaborative optical observation, which has the ability of autonomous information processing for multiple microsatellites and realizes multi-target recognition with multi-view collaborative fusion.

[0005] The technical solution provided by the present invention is as follows:

[0006] In a first aspect, a method for target recognition and matching in constellation collaborative optical observation includes:

[0007] Design a constellation configuration, build an inter-satellite communication network, and determine the master satellite and slave satellites of the constellation;

[0008] Use the optical observation cameras of the constellation to complete the acquisition of observation images in a specified airspace, extract the key frames of the images, transmit the key frame images of the slave satellites to the master satellite, and perform key frame image alignment on the master satellite;

[0009] Extract targets from the key frame images of the master satellite and slave satellites, and then perform two target matches, namely master satellite - slave satellite and slave satellite - slave satellite, to obtain the target recognition and matching results;

[0010] According to the target recognition and matching results, update the constellation configuration so that the entire constellation points to the target.

[0011] In combination with the first aspect, the steps of designing the star cluster configuration, establishing an inter-satellite communication network, and determining the master star and slave stars of the star cluster are specifically implemented as follows: Considering the parameters of the optical observation cameras carried by each satellite, design the star cluster observation configuration, determine the spatial position distribution of each satellite, and determine the inter-satellite distances; Establish an inter-satellite communication network and determine the pose transformation matrix between the optical cameras carried by each satellite; Select one satellite at the center of the spatial position as the master star, and the satellites other than the master star are defined as slave stars.

[0012] In combination with the first aspect, the star cluster configuration is as follows: The number of stars in the star cluster is designed to be 9, and the observation configuration is a quadrilateral; Two satellites are placed at each corner of the quadrilateral, and one satellite is placed at the center point of the quadrilateral; The satellites placed at each corner of the quadrilateral are defined as slave stars, and one satellite placed at the center point of the quadrilateral is defined as the master star;

[0013] The communication capacity of the inter-satellite communication network is set to an image transmission capacity of at least 1 Mb / s, and the maximum communication distance is greater than the maximum satellite spacing;

[0014] The pose transformation matrix between the optical cameras carried by each satellite is determined as follows: Using the inter-satellite communication network, determine the pose transformation matrix between the star clusters, and in combination with the installation matrix of the camera on the satellite, determine the pose transformation matrix converted between each optical observation camera.

[0015] In combination with the first aspect, the steps of extracting the key frames of the images are specifically implemented as follows:

[0016] Match the front and back frames of the observation images collected by each optical observation camera. If the number of ORB feature point matches is less than the set number of feature points, and the pose change between the front and back frames is greater than the set pose change amount, then the back frame is retained as a key frame, otherwise it is deleted; If the number of observation images after screening is greater than the preset number, reduce the set number of feature points and / or increase the set pose change amount, and continue to match the front and back frames until key frame images that meet the preset number are obtained.

[0017] In combination with the first aspect, in the step of transmitting the key frame images of the slave stars to the master star, the key frame image transmission method is: Binary-encode the key frame images of the slave stars, with an encoding length of 11 bits, and transmit them to the master star through the inter-satellite communication network.

[0018] In combination with the first aspect, in the step of aligning the key frame images on the master star, the key frame image alignment method is: The master star receives the binary-encoded images and decodes them, and uses the wavelet transform method to align all the key frame images of the slave stars.

[0019] Combined with the first aspect, the steps of performing target extraction on the key-frame images of the master satellite and the slave satellites, and then implementing two target matches, namely master satellite - slave satellite and slave satellite - slave satellite, to obtain the target recognition and matching results are specifically implemented in the following manner:

[0020] Encode and group the aligned key-frame images according to the slave satellite numbers they belong to; use the background subtraction method to perform target extraction on the key-frame images of the master satellite and the encoded and grouped key-frame images;

[0021] Use the pose transformation matrix to sequentially map the targets extracted from the key-frame images of the master satellite to all the encoded and grouped key-frame images, and take the targets that match the target extraction results of the key-frame images of the master satellite and the encoded and grouped key-frame images as the preliminary target recognition results;

[0022] Use the pose transformation matrix to sequentially match the target extraction results of the encoded and grouped key-frame images. When the extracted target continuously appears in the key-frame images of 2 encoded groups, this target is determined as the newly added target recognition result;

[0023] Perform a final match between the preliminary target recognition results and the newly added target recognition results, and delete the duplicate matching targets to obtain the final target recognition and matching results.

[0024] Combined with the first aspect, the steps of updating the constellation configuration according to the target recognition and matching results so that all satellites in the constellation point to the target are specifically implemented in the following manner:

[0025] Determine the spatial positions of all targets according to the target tracking results;

[0026] Judge the satellites whose recognized target quantity is lower than the minimum recognition threshold and the satellites whose recognized target quantity is greater than the maximum recognition threshold according to the final target recognition and matching results;

[0027] Use the spatial positions of the targets and the pose transformation matrix between the optical cameras carried by each satellite to reallocate the above satellites again so that each satellite can recognize a target quantity between the minimum recognition threshold and the maximum recognition threshold;

[0028] Use the inter-satellite communication network to transmit the planning instructions to update the constellation configuration.

[0029] Combined with the first aspect, the standard for updating the constellation configuration is: adjust the positions of the satellites whose recognized target quantity is lower than the minimum recognition threshold and plan them around the satellites whose recognized target quantity is greater than the maximum recognition threshold.

[0030] In the second aspect, a target recognition and matching device for collaborative optical observation of a constellation includes:

[0031] One or more processors;

[0032] A storage device for storing one or more programs,

[0033] When the one or more programs are executed by the one or more processors, the one or more processors implement the target recognition and matching method for constellation collaborative optical observation described in the first aspect.

[0034] In a third aspect, a readable storage medium has a computer program stored thereon, and when the program is executed by a processor, it implements the target recognition and matching method for constellation collaborative optical observation described in the first aspect.

[0035] In a fourth aspect, a computer program product includes: a computer program (which may also be referred to as code or instructions), and when the computer program is run, it executes the target recognition and matching method for constellation collaborative optical observation described in the first aspect.

[0036] A target recognition and matching method for constellation collaborative optical observation provided by the present invention has the following

[0037] Beneficial effects:

[0038] (1) For the target recognition and matching method for constellation collaborative optical observation provided by the present invention, the constellation configuration is designed as a quadrilateral deformation, with slave stars placed at each corner of the quadrilateral and a master star placed at the center point of the quadrilateral, having the advantages of specific stable configuration and wide observation angle coverage;

[0039] (2) For the target recognition and matching method for constellation collaborative optical observation provided by the present invention, the inter-satellite communication network is used to synchronously trigger the optical observation cameras of the constellation to turn on, ensuring the time synchronization of the images. At the same time, the master star uses the wavelet transform method to align the key frame images of all slave stars, ensuring the spatial synchronization of the images. Finally, the spatio-temporal (time and space) consistency of the images is achieved, improving the reliability of the target;

[0040] (3) For the target recognition and matching method for constellation collaborative optical observation provided by the present invention, the key frame images of the slave stars are transmitted using binary coding, ensuring the stability and accuracy of the image transmission;

[0041] (4) For the target recognition and matching method for constellation collaborative optical observation provided by the present invention, the pose transformation matrix of the optical observation camera is used to achieve two target matches between the master and slave stars and between the slave stars, accelerating the speed and accuracy of the target matching and recognition;

[0042] (5) The object recognition and matching method for star cluster collaborative optical observation provided by the present invention sets the communication ability of the network to an image transmission ability of at least 1 Mb / s, and the maximum communication distance is greater than the maximum satellite spacing, such as 50 km. The inter-satellite communication network has strong information transmission ability and large information transmission coverage space, greatly improving the efficiency of multi-view information interaction and transmission of the star cluster.

[0043] (6) The object recognition and matching method for star cluster collaborative optical observation provided by the present invention uses inter-satellite communication and camera installation matrix to determine the pose transformation matrix of all optical observation cameras. Compared with the traditional pose measurement method, it can achieve a pose measurement ability of more than 20 km, greatly improving the robustness of multi-view interaction.

[0044] (7) The object recognition and matching method for star cluster collaborative optical observation provided by the present invention uses the object recognition and tracking results to dynamically plan the positions of the star cluster, enabling each satellite to recognize the number of targets between the minimum recognition threshold and the maximum recognition threshold, such as 2 to 5 targets, balancing the computing resources within the group and having the ability of coordinated pointing. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 is a flowchart of the object recognition and matching method for star cluster collaborative optical observation of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0046] The present invention will be described in detail below, and the features and advantages of the present invention will become clearer and more definite with these descriptions.

[0047] Here, the special term "exemplary" means "serving as an example, embodiment or illustration". Any embodiment described as "exemplary" here does not have to be construed as superior to or better than other embodiments.

[0048] The present invention provides an object recognition and matching method for star cluster collaborative optical observation, as Figure 1 shown, including the following steps:

[0049] Step (1), star cluster configuration design

[0050] This step completes the confirmation of the position and spatial layout of the star cluster for observing space targets, and the confirmation of the main star and the slave stars.

[0051] Specifically: Considering the parameters of the optical observation cameras carried by each star (including focal length, observation field of view and aperture), design the star cluster observation configuration, determine the spatial position distribution of each star, and determine the inter-satellite distance; Build an inter-satellite communication network, determine the pose transformation matrix between the optical cameras carried by each star; Select a certain star at the center of the spatial position as the main star, and the stars other than the main star are defined as slave stars.

[0052] Preferably, the number of satellites in the constellation is designed to be 9, and the observation configuration is a quadrilateral; the side length of the quadrilateral is 1.5 km to 2.5 km, such as 2 km; 2 satellites are placed at each corner of the quadrilateral, and the satellite spacing between the 2 satellites is 150 m to 250 m, such as 200 m, and 1 satellite is placed at the center point of the quadrilateral; the weight of the satellite is designed to be 50 kg, and the weight of the optical observation camera carried is a visible light camera designed to be 10 kg.

[0053] Build an inter-satellite communication network, and set the communication capacity of the network to at least 1 Mb / s for image transmission capacity, and the maximum communication distance is greater than the maximum satellite spacing, such as 50 km.

[0054] The satellites placed at each corner of the quadrilateral are defined as slave satellites, and the 1 satellite placed at the center point of the quadrilateral is defined as the master satellite.

[0055] Using the inter-satellite communication network, determine the pose transformation matrix between constellations, and combine with the installation matrix of the camera on the satellite to determine the pose transformation matrix between each optical observation camera.

[0056] Step (2), multi-view image alignment

[0057] This step completes image acquisition of each satellite, extraction of image key frames, transmission of key frame images from slave satellites to the master satellite, and alignment of key frame images.

[0058] Specifically: use the inter-satellite communication network to synchronously trigger the optical observation cameras of the constellation to turn on; the optical observation cameras of the constellation image a specified airspace at a set frequency, such as 10 Hz, and continuously acquire multi-view observation images for a set duration, such as 5 minutes; use the key frame selection method to compress the multi-view observation images, and only extract a set number of key frame images, such as 50, from the observation perspective of each satellite, and transmit the key frame images of the slave satellites to the master satellite through the inter-satellite communication network, and the master satellite uses the wavelet transform method to align all the images.

[0059] Preferably, the key frame selection method is: match the front and back frames of the observation images collected by each optical observation camera. If the number of ORB feature point matches is less than the first number of feature points, such as 200, and the pose change between the front and back frames is greater than the first pose change amount, such as 0.5°, then the back frame is retained as a key frame, otherwise it is deleted; if the number of observation images after screening is greater than the preset number, perform a second match on all the key frames. If the number of ORB feature point matches is less than the second number of feature points, such as 50, and the pose change between the front and back frames is greater than the second pose change amount, such as 1°, then the back frame is retained as a key frame, otherwise it is deleted. Finally, only retain a preset number of key frame images, such as 50.

[0060] Preferably, the method for transmitting key frame images from slave satellites to the master satellite is: binary encode the key frame images of the slave satellites, with an encoding length of 11 bits, and transmit them to the master satellite through the inter-satellite communication network.

[0061] Preferably, the key-frame image alignment method is as follows: the master satellite receives and decodes the binary-coded image, and uses the wavelet transform method to align the key-frame images of all slave satellites.

[0062] Step (3), target recognition and matching

[0063] This step completes target recognition and matching through the input key-frame aligned images.

[0064] Specifically: the aligned key-frame images are encoded and grouped according to the serial numbers of the slave satellites they belong to; the background subtraction method is used to extract the targets from the key-frame image of the master satellite and the key-frame images after encoding and grouping;

[0065] Using the pose transformation matrix, the targets extracted from the key-frame image of the master satellite are sequentially mapped to all key-frame images after encoding and grouping, and the targets whose target extraction results of the key-frame image of the master satellite match those of the key-frame images after encoding and grouping are used as the preliminary target recognition results;

[0066] Using the pose transformation matrix, the target extraction results of the key-frame images after encoding and grouping are sequentially matched. When the extracted target continuously appears in the key-frame images of 2 encoded groups, this target is determined as the new target recognition result;

[0067] The preliminary target recognition results and the new target recognition results are finally matched, and the duplicate-matched targets are deleted to obtain the final target recognition and matching results.

[0068] Using the pose transformation matrix and through the inter-satellite communication network, the target recognition and matching results are sequentially assigned to the slave satellites corresponding to the encoded groups to perform target tracking.

[0069] Step (4), constellation configuration update

[0070] This step updates the constellation configuration according to the target recognition and matching results, so that the entire constellation points to the target.

[0071] Specifically: according to the target tracking results, determine the spatial positions of all targets; according to the final target recognition and matching results, judge the satellites with the number of recognized targets less than the minimum recognition threshold, such as 2, and the satellites with the number of recognized targets greater than the maximum recognition threshold, such as 5; using the spatial positions of the targets and the pose transformation matrix between the optical cameras carried by each satellite, reallocate the above satellites so that each satellite can recognize a number of targets between the minimum recognition threshold and the maximum recognition threshold, such as 2 - 5 targets; use the inter-satellite communication network to transmit the planning instructions to update the constellation configuration.

[0072] Preferably, the criteria for updating the constellation configuration are as follows: satellites with the number of identified targets lower than the minimum identification threshold, such as 2, are repositioned and planned to be around satellites with the number of identified targets greater than the maximum identification threshold, such as 5, and the inter-satellite distance is designed to be 200 m to 300 m.

[0073] The present invention also provides an apparatus for target recognition and matching in constellation collaborative optical observation, including:

[0074] One or more processors;

[0075] A storage device for storing one or more programs,

[0076] When the one or more programs are executed by the one or more processors, the one or more processors implement the method for target recognition and matching in constellation collaborative optical observation as described above.

[0077] The present invention also provides a readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the method for target recognition and matching in constellation collaborative optical observation as described above is implemented.

[0078] The readable storage medium includes but is not limited to: various media such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs that can store program codes.

[0079] The present invention also provides a computer program product, which includes: a computer program (which can also be referred to as code or instructions), and when the computer program is run, the method for target recognition and matching in constellation collaborative optical observation as described above is executed.

[0080] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium.

[0081] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0082] The present invention has been described in detail above in conjunction with specific embodiments and exemplary examples. However, these descriptions should not be construed as limiting the present invention. Those skilled in the art understand that, without departing from the spirit and scope of the present invention, various equivalent substitutions, modifications, or improvements can be made to the technical solution of the present invention and its implementation manners, and all of these fall within the scope of the present invention. The protection scope of the present invention is subject to the appended claims.

[0083] The content not detailedly described in the specification of the present invention belongs to the well-known technology of those skilled in the art.

Claims

1. A target recognition and matching method for collaborative optical observation of a star cluster, characterized in that, Including: Design the constellation configuration, build the inter-satellite communication network, and determine the master satellite and slave satellites of the constellation; Use the optical observation cameras of the constellation to complete the acquisition of observation images in the specified airspace, extract the key frames of the images, transmit the key frame images of the slave satellites to the master satellite, and perform key frame image alignment on the master satellite; Extract targets from the key frame images of the master satellite and slave satellites, and then perform two target matches of master satellite - slave satellite and slave satellite - slave satellite to obtain the target recognition and matching results; According to the target recognition and matching results, update the constellation configuration so that the entire constellation points to the target.

2. The target recognition and matching method for star cluster collaborative optical observation according to claim 1, wherein The steps of designing the constellation configuration, building the inter-satellite communication network, and determining the master satellite and slave satellites of the constellation are specifically implemented in the following way: Considering the parameters of the optical observation cameras carried by each satellite, design the constellation observation configuration, determine the spatial position distribution of each satellite, and determine the inter-satellite distance; Build the inter-satellite communication network and determine the pose transformation matrix between the optical cameras carried by each satellite; Select a certain satellite at the center of the spatial position as the master satellite, and define the satellites other than the master satellite as slave satellites.

3. The object recognition and matching method for star cluster collaborative optical observation according to claim 2, characterized in that, The constellation configuration is: The number of satellites in the constellation is designed to be 9, and the observation configuration is a quadrilateral; Place 2 satellites at each corner of the quadrilateral and 1 satellite at the center point of the quadrilateral; The satellites placed at each corner of the quadrilateral are defined as slave satellites, and 1 satellite placed at the center point of the quadrilateral is defined as the master satellite; The communication capacity of the inter-satellite communication network is set to an image transmission capacity of at least 1 Mb / s, and the maximum communication distance is greater than the maximum satellite spacing; The pose transformation matrix between the optical cameras carried by each satellite is determined in the following way: Use the inter-satellite communication network to determine the pose transformation matrix between the constellations, and combine the installation matrix of the camera on the satellite to determine the pose transformation matrix converted between each optical observation camera.

4. The target recognition and matching method for star cluster collaborative optical observation according to claim 1, wherein The steps of extracting the key frames of the images are specifically implemented in the following way: Match the front and back frames of the observation images collected by each optical observation camera. If the number of ORB feature point matches is less than the set number of feature points, and the pose change between the front and back frames is greater than the set pose change amount, then the back frame is retained as the key frame, otherwise it is deleted; If the number of observation images after screening is greater than the preset number, reduce the set number of feature points and / or increase the set pose change amount, and continue to match the front and back frames until the key frame images that meet the preset number are obtained.

5. The object recognition and matching method for star cluster collaborative optical observation according to claim 1, wherein In the step of transmitting the key frame images of the slave satellites to the master satellite, the key frame image transmission method is: Binary-encode the key frame images of the slave satellites and transmit them to the master satellite through the inter-satellite communication network.

6. The object recognition and matching method for constellation collaborative optical observation according to claim 1, characterized in that, In the step of performing key frame image alignment on the master satellite, the key frame image alignment method is: The master satellite receives the binary-encoded image and decodes it, and uses the wavelet transform method to align the key frame images of all slave satellites.

7. The object recognition and matching method for star cluster collaborative optical observation according to claim 1, characterized in that The steps of extracting targets from the key frame images of the master satellite and slave satellites, and then performing two target matches of master satellite - slave satellite and slave satellite - slave satellite to obtain the target recognition and matching results are specifically implemented in the following way: Encode and group the aligned key frame images according to the numbers of the slave satellites they belong to; Use the background subtraction method to extract targets from the key frame images of the master satellite and the encoded and grouped key frame images; Using the pose transformation matrix, the targets extracted from the key frames of the main satellite are sequentially mapped to all the key frames of the encoded grouped images, and the targets that match the target extraction results of the key frames of the main satellite and the target extraction results of the key frames of the encoded grouped images are used as the preliminary target recognition results; Using the pose transformation matrix, the target extraction results of the key frames of the encoded grouped images are sequentially matched. When the extracted target continuously appears in the key frames of 2 encoded grouped images, the target is determined as the new target recognition result; The preliminary target recognition results and the new target recognition results are finally matched, and the duplicate matching targets are deleted to obtain the final target recognition matching results.

8. The target recognition and matching method for constellation collaborative optical observation according to claim 1, characterized in that The step of updating the constellation configuration according to the target recognition matching results so that all the satellites in the constellation point to the target is specifically implemented in the following manner: According to the target tracking results, determine the spatial positions of all the targets; According to the final target recognition matching results, judge the satellites with the number of recognized targets lower than the minimum recognition threshold and the number of recognized targets greater than the maximum recognition threshold; Using the spatial positions of the targets and the pose transformation matrix between the optical cameras carried by each satellite, reallocate the above satellites so that each satellite can recognize the number of targets between the minimum recognition threshold and the maximum recognition threshold; Use the inter-satellite communication network to transmit the planning instructions to update the constellation configuration.

9. The target recognition and matching method for constellation collaborative optical observation according to claim 8, characterized in that, The standard for updating the constellation configuration is: adjust the positions of the satellites with the number of recognized targets lower than the minimum recognition threshold and plan them around the satellites with the number of recognized targets greater than the maximum recognition threshold.

10. A computer program product, characterized in that, The computer program product includes: a computer program, which when run, executes the above-mentioned target recognition matching method for cooperative optical observation of the constellation.

Images