Space target detection system and method based on time-of-flight dual-mode integrated photoelectronic imaging

Abstract

The invention provides a space target detection system and method based on time-of-flight dual-mode integrated photoelectric imaging, and the system employs a time-of-flight dual-mode integrated photoelectric detection scheme to detect a space target, and collects the gray image data and depth image data of the space target. And after data acquisition is completed, the spaceborne computer is matched with a space target detection algorithm to obtain relative motion information of the target. According to the invention, the three-dimensional relative motion information of the target can be synchronously and directly measured in the two-dimensional gray domain and the three-dimensional space domain, and favorable conditions are provided for accurate measurement of the space target in a space environment.

Description

Technical Field

[0001] This invention relates to the field of space optoelectronic detection technology, and in particular to a target detection system and method based on time-of-flight dual-mode integrated optoelectronic imaging. Background Technology

[0002] Time-of-flight imaging cameras acquire depth information by measuring the phase difference of light. The camera is equipped with a high-frequency modulated laser source that emits light signals. These signals are reflected and diverged by the target scene, with a portion reflected into the camera. The demodulated pixels in the camera's imaging sensor receive the raw exposure image, and the phase difference of the propagating light signals is then calculated from this image to obtain depth information. Because time-of-flight imaging cameras can acquire both grayscale and depth images within the same exposure cycle, they are feature-rich and highly reliable, making them widely used in fields such as intelligent sensing.

[0003] Space target detection systems based on monocular imaging cameras only reflect the target's illumination intensity information in their grayscale images, and cannot directly measure accurate three-dimensional relative motion information. Summary of the Invention

[0004] The technical problem solved by this invention is to overcome the shortcomings of the prior art and provide a space target detection system and method based on time-of-flight dual-mode integrated photoelectric imaging, which can simultaneously and directly measure the three-dimensional relative motion information of the target in the two-dimensional grayscale domain and the three-dimensional spatial domain, providing favorable conditions for accurate measurement of space targets in the space environment.

[0005] The technical solution of the present invention is: a space target detection system based on time-of-flight dual-mode integrated optoelectronic imaging, comprising: a laser circuit, a transmitting optical system, a receiving optical system, a time-of-flight optoelectronic imaging sensor, a preprocessing FPGA module, and a data processing module; The laser circuit emits narrowband laser light driven by the modulation signal from the preprocessing FPGA module; The transmitting optical system shapes and homogenizes the narrowband laser, which is then irradiated onto the target and reflected off the surface of the target. The receiving optical system receives the laser reflected from the target and images it, then sends the image to the time-of-flight photoelectric imaging sensor. The time-of-flight photoelectric imaging sensor receives images transmitted by the optical system and generates phase image data. The preprocessing FPGA module calculates grayscale image data and depth image data based on the phase image data and sends them to the data processing module. The data processing module receives grayscale image data and depth image data, and calculates the relative position data and relative attitude data of the target under test.

[0006] Furthermore, the data processing module uses a grayscale image target recognition algorithm to calculate the imaging area of ​​the target in the grayscale image, including: An adaptive threshold is calculated based on the grayscale image data, and the grayscale image data is binarized to obtain a foreground binary image. Using the foreground binary image, grayscale image data, and prior distance as parameters, we extract all connected regions in the grayscale image, calculate the average grayscale, number of pixels, and distance between connected regions, and return the set of filtered valid regions. Each retained region within the effective region set is scored and sorted by score, and the top K regions are extracted as candidate regions; The centroid distance threshold is dynamically set based on the prior distance, and the candidate regions are merged to obtain the final target region; The final target region on the grayscale image is mapped onto the depth image to obtain the corresponding three-dimensional point set of the target region; the points in the three-dimensional point set of the target region are filtered to remove invalid points and points with coordinates (0,0,0), and the filtered target point set is output.

[0007] Furthermore, the comprehensive score formula for evaluating each retained region within the effective region set is as follows:

[0008] Where: score i For the first i The score for each region, brightness i The mean grayscale value of the region, contrast_score i The difference in grayscale between the inside and outside of the area; area i The area is the region. i =1,2,3,…,N, where N represents the number of regions in the effective region set.

[0009] Furthermore, the data processing module uses a point cloud relative pose algorithm to calculate the relative position and attitude data of the target under test, including: Principal component analysis is performed on the target point set to obtain the initial centroid, principal direction of the point set, and eigenvalues; When the average Euclidean distance of the target point set is greater than the threshold a, the initial centroid of the target and the main direction of the target point set are output; when the average distance of the target point set is less than the threshold a, the relative attitude is calculated to obtain the initial transformation matrix. Using the point-to-plane ICP registration function, the initial transformation matrix is ​​used as the initial pose. The pre-stored reference target is fitted to the target point set. The point-to-plane ICP registration function outputs the final pose transformation matrix and root mean square error. The centroid of the target function is extracted and calculated from the final pose transformation matrix.

[0010] Furthermore, attitude calculation determines the geometric shape of the point cloud based on eigenvalues: If the ratio of the first two eigenvalues ​​is in the range of 0.9 to 1.1, then the principal direction of the target point set is considered to be pointing towards the camera. The third principal direction of the principal component analysis (PCA) result is taken as the length direction of the target, and the rotation matrix R is reconstructed. Otherwise, the first two principal directions of the PCA are used to construct the rotation matrix R.

[0011] A method for space target detection based on the above-mentioned time-of-flight dual-mode integrated optoelectronic imaging system includes: The laser circuit emits a narrowband laser under the modulation signal of the preprocessing FPGA module. The emitting optical system shapes and homogenizes the narrowband laser, which is then irradiated onto the target under test. The laser is reflected from the surface of the target under test and imaged by the receiving optical system, generating phase image data on the time-of-flight photoelectric imaging sensor. The preprocessing FPGA module calculates grayscale image data and depth image data based on the phase image data; The data processing module receives grayscale image data and depth image data, uses a grayscale image target recognition algorithm to calculate the imaging area of ​​the target in the grayscale image, and uses a point cloud relative pose algorithm to calculate the relative position data and relative pose data of the target.

[0012] The advantages of this invention compared to the prior art are: (1) Based on the physical size characteristics and surface optical characteristics of actual space targets, the present invention selects a detection method based on time-of-flight dual-mode integrated photoelectric imaging, which effectively improves the detection capability of space targets; (2) The method of the present invention combines grayscale image target recognition with depth image target measurement, thereby reducing the three-dimensional measurement error of the centroid of spatial targets; Attached Figure Description Figure 1 This is a schematic diagram of the space target detection system based on time-of-flight dual-mode integrated photoelectric imaging according to the present invention; Figure 2 This is a flowchart of the spatial target recognition method based on grayscale images of the present invention; Figure 3 This is a flowchart of the spatial target relative motion information calculation method based on depth images according to the present invention; Figure 4 This is an example diagram showing the results of a ground-based physical experiment for measuring space targets according to the present invention. Detailed Implementation

[0013] To better understand the above technical solutions, the technical solutions of this application will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of this application and the specific features in the embodiments are detailed descriptions of the technical solutions of this application, rather than limitations on the technical solutions of this application. In the absence of conflict, the embodiments of this application and the technical features in the embodiments can be combined with each other.

[0014] The present application will be further described in detail below with reference to the accompanying drawings.

[0015] A space target detection system based on time-of-flight dual-mode integrated optoelectronic imaging includes: Laser circuitry, used to modulate the emission and drive of illumination lasers; An optical system for transmitting light, used for beam expansion and shaping of the illumination laser; A receiving optical system is used to receive the modulated illumination laser reflected back by the target, and to clearly image it on the imaging sensor. The operating spectral band is the same as that of the modulated illumination laser. A time-of-flight photoelectric imaging sensor is used to acquire photons in the modulated beam reflected back by the target and output phase image data; The preprocessing FPGA module is used to connect with the modulation light source transmitter and the time-of-flight photoelectric imaging sensor, drive the laser circuit and the time-of-flight photoelectric imaging sensor to illuminate and expose according to a fixed period, and synchronize and communicate with the satellite controller, and receive grayscale image data and depth image data transmitted by the time-of-flight photoelectric imaging sensor. The data processing module implements grayscale image target recognition algorithms and depth image relative pose algorithms; The grayscale image includes the intensity value of the photoelectric signal of the spatial target acquired within an exposure time. The depth image includes a depth value calculated from the phase change value of the modulated beam after the pulse beam emitted by the transmitter is reflected by the target and acquired by the receiving optical system and the time-of-flight photoelectric imaging sensor. A grayscale image-based target recognition method includes: determining a target region in space, wherein the target region is analyzed by adaptive brightness thresholding and connected component analysis of the grayscale image to which it belongs; obtaining candidate regions and scoring them using brightness / area / contrast as features; and merging the candidate regions to form the minimum bounding rectangle of the target.

[0016] A method for calculating relative motion information based on depth images includes: extracting effective points from the depth image using the minimum bounding rectangle of the target to obtain a target point set, fitting the target point set to a three-dimensional centroid, and outputting the target's position information and three-dimensional pointing information.

[0017] Example: like Figure 1As shown, the space target detection system based on time-of-flight dual-mode integrated optoelectronic imaging mainly includes: a laser circuit, a transmitting optical system, a receiving optical system, a time-of-flight optoelectronic imaging sensor, a preprocessing FPGA module, and a data processing module. Driven by the modulation signal of the preprocessing FPGA module, the laser circuit emits a narrowband laser with a spectral range of 850nm±10nm. The emitted laser is shaped and homogenized by the emitting optical system and then illuminates the target. Reflected from the target surface, the laser is imaged by the receiving optical system, generating phase image data on the time-of-flight photoelectric imaging sensor. The preprocessing FPGA module calculates grayscale and depth image data from the phase image data and sends them to the data processing module to calculate the target's relative position and attitude data. The grayscale image target recognition algorithm and the point cloud relative pose algorithm are stored and run on the SOC component of the data processing circuit 60.

[0018] Grayscale image target recognition methods are used to detect the energy and edge information of spatial targets in grayscale images, and to obtain the imaging area of ​​the spatial target in the grayscale image. The workflow is as follows: Figure 2 As shown, the steps are as follows: (1) Image preprocessing and binarization First, an adaptive threshold is calculated based on the grayscale image data, and the grayscale image data is binarized to obtain a foreground binary image.

[0019] (2) Connectivity analysis Using the foreground binary image, grayscale image data, and prior distance as parameters, all connected regions in the grayscale image are extracted, and their attributes are calculated, including the average grayscale of the connected regions, the number of pixels, and the distance between connected regions. Finally, a set of filtered valid regions is returned. If this set is empty, the processing terminates.

[0020] (3) Regional scoring and ranking For each retained region, the following comprehensive score formula applies:

[0021] in: score i For the first i Score for each region brightness i Mean gray level of the region contrast_score i Gray-scale difference between inside and outside the region area i Area (in pixels) Extract scores from all regions and select the top K regions, where K=3. The specific method is as follows: Suppose a total of N regions are extracted, and each region i There are corresponding scores s i This forms the score sequence: scores=[ s 1, s 2,…, s N ].

[0022] Construct the index sequence: idx=[1,2,3,…, N ].

[0023] Sort the index array idx so that the corresponding scores[idx] are arranged in descending order.

[0024] The sorting method is either selection sort or bubble sort.

[0025] After sorting, select the top K regions by score to form a candidate region set.

[0026] (4) Regional merger The centroid distance threshold is dynamically set based on the prior distance, and candidate regions are merged to obtain the final target region. If the final target region merging fails or the result is empty, it reverts to the single candidate region with the highest score.

[0027] (5) Target point cloud extraction and filtering This step is based on the final target region. The final target region on the grayscale image is mapped onto the depth image to obtain the corresponding three-dimensional point set of the target region. These points are filtered to remove invalid points (NaN, Inf) and points with coordinates (0,0,0), and the filtered target point set is output.

[0028] Point cloud relative pose algorithms are used to match the pose information of spatial target point clouds, obtaining the position and pose information of the spatial target in three-dimensional space. The workflow is as follows: Figure 3 As shown, the steps are as follows: (1) Coarse registration First, principal component analysis is performed on the target point set to obtain the initial centroid, principal orientation, and eigenvalues. When the average Euclidean distance of the target point set is greater than 4 m, the initial centroid and principal orientation of the target point set are output; when the average Euclidean distance of the target point set is less than 4 m, attitude calculation is performed to obtain the initial transformation matrix.

[0029] Attitude calculation determines the geometric shape of the point cloud based on eigenvalues. If the first two eigenvalues ​​are very close (i.e., the ratio of the first two eigenvalues ​​is within the range of 0.9 to 1.1), the target's principal direction is considered to be pointing towards the camera. The third principal direction from the principal component analysis (PCA) result is then used as the target's length direction (i.e., the Z-axis), and the rotation matrix R is reconstructed. Otherwise, the first two principal directions from the PCA are used to construct R. The third principal direction refers to the normal direction of the plane formed by the two directions represented by the two eigenvalues.

[0030] The initial transformation matrix combines the constructed rotation matrix R and the centroid into a 4×4 homogeneous transformation matrix.

[0031] (2) Precise registration This step uses a point-to-plane ICP registration function, with the initial transformation matrix as the initial pose, to accurately fit the reference target to the target point set. The registration function outputs the final pose transformation matrix and root mean square error.

[0032] (3) Result extraction and judgment The centroid of the precise functional target is extracted and calculated from the final pose transformation matrix.

[0033] (5) Output Output the final, precise centroid. For example... Figure 4 The image shown is an example of the results of a ground-based physical experiment for measuring space targets.

[0034] A method for space target detection based on the above-mentioned time-of-flight dual-mode integrated photoelectric imaging system includes the following steps: The laser circuit emits a narrowband laser under the modulation signal of the preprocessing FPGA module. The emitting optical system shapes and homogenizes the narrowband laser, which is then irradiated onto the target under test. The laser is reflected from the surface of the target under test and imaged by the receiving optical system, generating phase image data on the time-of-flight photoelectric imaging sensor. The preprocessing FPGA module calculates grayscale image data and depth image data based on the phase image data; The data processing module receives grayscale image data and depth image data, uses a grayscale image target recognition algorithm to calculate the imaging area of ​​the target in the grayscale image, and uses a point cloud relative pose algorithm to calculate the relative position data and relative pose data of the target.

[0035] Those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

[0036] The contents not described in detail in this specification are common knowledge to those skilled in the art.

Claims

1. A space target detection system based on time-of-flight dual-mode integrated photoelectric imaging, characterized in that, include: Laser circuit, transmitting optical system, receiving optical system, time-of-flight photoelectric imaging sensor, preprocessing FPGA module and data processing module; The laser circuit emits narrowband laser light driven by the modulation signal from the preprocessing FPGA module; The transmitting optical system shapes and homogenizes the narrowband laser, which is then irradiated onto the target and reflected off the surface of the target. The receiving optical system receives the laser reflected from the target and images it, then sends the image to the time-of-flight photoelectric imaging sensor. The time-of-flight photoelectric imaging sensor receives images transmitted by the optical system and generates phase image data. The preprocessing FPGA module calculates grayscale image data and depth image data based on the phase image data and sends them to the data processing module. The data processing module receives grayscale image data and depth image data, and calculates the relative position data and relative attitude data of the target under test.

2. The space target detection system based on time-of-flight dual-mode integrated photoelectric imaging according to claim 1, characterized in that, The data processing module uses a grayscale image target recognition algorithm to calculate the imaging region of the target in the grayscale image, including: An adaptive threshold is calculated based on the grayscale image data, and the grayscale image data is binarized to obtain a foreground binary image. Using the foreground binary image, grayscale image data, and prior distance as parameters, we extract all connected regions in the grayscale image, calculate the average grayscale, number of pixels, and distance between connected regions, and return the set of filtered valid regions. Each retained region within the effective region set is scored and sorted by score, and the top K regions are extracted as candidate regions; The centroid distance threshold is dynamically set based on the prior distance, and the candidate regions are merged to obtain the final target region; The final target region on the grayscale image is mapped onto the depth image to obtain the corresponding three-dimensional point set of the target region; the points in the three-dimensional point set of the target region are filtered to remove invalid points and points with coordinates (0,0,0), and the filtered target point set is output.

3. A space target detection system based on time-of-flight dual-mode integrated photoelectric imaging according to claim 2, characterized in that, The comprehensive score formula for evaluating each retained region within the valid region set is as follows: Where: score i For the first i The score for each region, brightness i The mean grayscale value of the region, contrast_score i The difference in grayscale between the inside and outside of the area; area i The area is the region. i =1,2,3,…,N, where N represents the number of regions in the effective region set.

4. A space target detection system based on time-of-flight dual-mode integrated photoelectric imaging according to claim 3, characterized in that, The data processing module uses a point cloud relative pose algorithm to calculate the relative position and attitude data of the target under test, including: Principal component analysis is performed on the target point set to obtain the initial centroid, principal direction of the point set, and eigenvalues; When the average Euclidean distance of the target point set is greater than the threshold a, the initial centroid of the target and the main direction of the target point set are output; when the average distance of the target point set is less than the threshold a, the relative attitude is calculated to obtain the initial transformation matrix. Using the point-to-plane ICP registration function, the initial transformation matrix is ​​used as the initial pose. The pre-stored reference target is fitted to the target point set. The point-to-plane ICP registration function outputs the final pose transformation matrix and root mean square error. The centroid of the target function is extracted and calculated from the final pose transformation matrix.

5. A space target detection system based on time-of-flight dual-mode integrated photoelectric imaging according to claim 4, characterized in that, Pose calculation determines the geometric shape of a point cloud based on eigenvalues: If the ratio of the first two eigenvalues ​​is in the range of 0.9 to 1.1, then the principal direction of the target point set is considered to be pointing towards the camera. The third principal direction of the principal component analysis (PCA) result is taken as the length direction of the target, and the rotation matrix R is reconstructed. Otherwise, the first two principal directions of the PCA are used to construct the rotation matrix R.

6. A space target detection method based on time-of-flight dual-mode integrated photoelectric imaging, characterized in that, include: The laser circuit emits a narrowband laser under the modulation signal of the preprocessing FPGA module. The emitting optical system shapes and homogenizes the narrowband laser, which is then irradiated onto the target under test. The laser is reflected from the surface of the target under test and imaged by the receiving optical system, generating phase image data on the time-of-flight photoelectric imaging sensor. The preprocessing FPGA module calculates grayscale image data and depth image data based on the phase image data; The data processing module receives grayscale image data and depth image data, uses a grayscale image target recognition algorithm to calculate the imaging area of ​​the target in the grayscale image, and uses a point cloud relative pose algorithm to calculate the relative position data and relative pose data of the target.

7. A space target detection method based on time-of-flight dual-mode integrated photoelectric imaging according to claim 6, characterized in that, The imaging region of the target in the grayscale image is calculated using a grayscale image target recognition algorithm, including: An adaptive threshold is calculated based on the grayscale image data, and the grayscale image data is binarized to obtain a foreground binary image. Using the foreground binary image, grayscale image data, and prior distance as parameters, we extract all connected regions in the grayscale image, calculate the average grayscale, number of pixels, and distance between connected regions, and return the set of filtered valid regions. Each retained region within the effective region set is scored and sorted by score, and the top K ranked regions are extracted as candidate regions; The centroid distance threshold is dynamically set based on the prior distance, and the candidate regions are merged to obtain the final target region; The final target region calculated from the grayscale image is mapped onto the depth image to obtain the corresponding three-dimensional point set of the target region. The points in the three-dimensional point set of the target region are filtered to remove invalid points and points with coordinates (0,0,0), and the filtered target point set is output.

8. A space target detection method based on time-of-flight dual-mode integrated photoelectric imaging according to claim 7, characterized in that, The comprehensive score formula for evaluating each retained region within the valid region set is as follows: Where: score i For the first i The score for each region, brightness i The mean grayscale value of the region, contrast_score i The difference in grayscale between the inside and outside of the area; area i Let i be the area of ​​the region; i = 1, 2, 3, ..., N, where N represents the number of regions in the set of valid regions.

9. A space target detection method based on time-of-flight dual-mode integrated photoelectric imaging according to claim 8, characterized in that, The relative position and attitude data of the target under test are calculated using a point cloud relative pose algorithm, including: Principal component analysis is performed on the target point set to obtain the initial centroid, principal direction of the point set, and eigenvalues; When the average Euclidean distance of the target point set is greater than the threshold a, the initial centroid of the target and the main direction of the target point set are output; when the average distance of the target point set is less than the threshold a, the relative attitude is calculated to obtain the initial transformation matrix. Using the point-to-plane ICP registration function, the initial transformation matrix is ​​used as the initial pose. The pre-stored reference target is fitted to the target point set. The point-to-plane ICP registration function outputs the final pose transformation matrix and root mean square error. The centroid of the target is extracted and calculated from the final pose transformation matrix.

10. A space target detection method based on time-of-flight dual-mode integrated photoelectric imaging according to claim 9, characterized in that, Pose calculation determines the geometric shape of a point cloud based on eigenvalues: If the ratio of the first two eigenvalues ​​is in the range of 0.9 to 1.1, then the principal direction of the target point set is considered to be pointing towards the camera. The third principal direction of the principal component analysis (PCA) result is taken as the length direction of the target, and the rotation matrix R is reconstructed. Otherwise, the first two principal directions of the PCA are used to construct the rotation matrix R.

Images