A method and system for rapid 3D imaging of space targets based on a combination of active and passive methods

Abstract

This disclosure provides a method and system for rapid three-dimensional imaging of space targets based on a combination of active and passive optics, applicable to the field of imaging detection. The method includes: acquiring images of the target object using an imaging device based on the target's line-of-sight direction, obtaining a vertical image and a horizontal image, wherein the vertical image is obtained based on an optical camera and passive optics technology, and the plane containing the vertical image is perpendicular to the target's line-of-sight direction; determining the first relative motion information between the target object and the optical camera; determining the second relative motion information between the target object and a synthetic aperture lidar; calibrating the dimensions of the vertical and horizontal images based on the first and second relative motion information, obtaining a target vertical image and a target horizontal image; and orthogonally fusing the target vertical and horizontal images based on the correlation information between them to obtain a target three-dimensional image.

Description

Technical Field

[0001] This disclosure relates to the field of imaging detection, and more specifically, to a method and system for rapid three-dimensional imaging of space targets based on a combination of active and passive methods. Background Technology

[0002] Acquiring a 3D image of a target is crucial for determining its type and function. Optical images, compared to radar images, are closer to human perception and easier to interpret. Traditional optical imaging techniques observe targets using visible light or infrared cameras, requiring wide-angle and multi-angle observations to obtain a 3D image, making them unsuitable for complex scenarios involving moving targets. Furthermore, optical cameras rely on lighting conditions, making it difficult to obtain target image information in shadowed areas or under backlight. Synthetic aperture lidar, a newly developed technology in recent years, is independent of lighting conditions during imaging, producing results that more closely resemble optical images.

[0003] However, whether it is optical passive imaging or synthetic aperture imaging, the imaging effect is to obtain two-dimensional image information of the target. It is still necessary to conduct multi-angle observation, and it is difficult and inefficient to quickly obtain a three-dimensional image of the target. Summary of the Invention

[0004] In view of this, this disclosure provides a method and system for rapid three-dimensional imaging of space targets based on a combination of active and passive methods.

[0005] One aspect of this disclosure provides a rapid three-dimensional imaging method for space targets based on a combination of active and passive optics, comprising: acquiring images of the target object using an imaging device based on the target's line-of-sight direction to obtain a vertical image and a horizontal image, wherein the imaging device includes an optical camera and a synthetic aperture lidar; the vertical image is obtained based on the optical camera and passive optics, and the plane containing the vertical image is perpendicular to the target's line-of-sight direction; the horizontal image is obtained based on the synthetic aperture lidar and active optics, and the plane containing the horizontal image is parallel to the target's line-of-sight direction; determining first relative motion information between the target object and the optical camera; determining second relative motion information between the target object and the synthetic aperture lidar; calibrating the dimensions of the vertical image and the horizontal image according to the first and second relative motion information to obtain a target vertical image and a target horizontal image; and orthogonally fusing the target vertical image and the target horizontal image based on the correlation information between them to obtain a target three-dimensional image.

[0006] According to an embodiment of this disclosure, the second relative motion information includes relative distance information; wherein, according to the first relative motion information and the second relative motion information, the vertical image and the horizontal image are sized to obtain the target vertical image and the target horizontal image respectively, including: obtaining the first dimension and the second dimension of the target object according to the field of view parameters of the optical camera and the relative distance information; and sized the vertical image based on the first dimension and the second dimension to obtain the target vertical image.

[0007] According to embodiments of this disclosure, the first relative motion information includes relative angle change information; wherein, the process of calibrating the vertical image and the horizontal image according to the first relative motion information and the second relative motion information to obtain the target vertical image and the target horizontal image further includes: obtaining the first dimension and the third dimension of the target object according to the relative angle change information and the relative distance information; and calibrating the horizontal image according to the first dimension and the third dimension to obtain the target horizontal image.

[0008] According to embodiments of this disclosure, obtaining the first and third dimensions of a target object based on relative angle change information and relative distance information includes: determining Doppler information of a horizontal image; obtaining relative vector velocity information of the target object based on the relative angle change information and Doppler information; and obtaining the first and third dimensions of the target object based on the relative vector velocity information and relative distance information.

[0009] According to embodiments of this disclosure, the target vertical image is constructed based on the first and second dimension information of the target object, and the target horizontal image is constructed based on the first and third dimension information of the target object. The correlation information between the target vertical image and the target horizontal image includes the first dimension information. Specifically, based on the correlation information between the target vertical image and the target horizontal image, orthogonally fusing the target vertical image and the target horizontal image to obtain a target three-dimensional image includes: based on the first dimension information, performing coordinate matching on the pixels in the target vertical image and the pixels in the target horizontal image to obtain an initial three-dimensional image; and processing the initial three-dimensional image using a distortion correction algorithm to obtain the target three-dimensional image.

[0010] According to embodiments of this disclosure, processing an initial three-dimensional image using a distortion correction algorithm to obtain a target three-dimensional image includes: performing shape correction on the initial three-dimensional image to obtain an intermediate three-dimensional image; and performing texture correction on the intermediate three-dimensional image to obtain the target three-dimensional image.

[0011] Another aspect of this disclosure provides a rapid three-dimensional imaging system for space targets based on a combined active and passive imaging approach, comprising:

[0012] The acquisition module is used to acquire images of the target object based on the target observation line of sight using imaging equipment, and obtain vertical and horizontal images. The imaging equipment includes an optical camera and a synthetic aperture lidar. The vertical image is obtained based on the optical camera and passive optics technology, and the plane of the vertical image is perpendicular to the target observation line of sight. The horizontal image is obtained based on synthetic aperture lidar and active technology, and the plane of the horizontal image is parallel to the target observation line of sight.

[0013] The first determining module is used to determine the first relative motion information between the target object and the optical camera;

[0014] The second determining module is used to determine the second relative motion information between the target object and the synthetic aperture lidar.

[0015] The calibration module is used to calibrate the dimensions of the vertical image and the horizontal image according to the first relative motion information and the second relative motion information, respectively, to obtain the target vertical image and the target horizontal image;

[0016] The fusion module is used to orthogonally fuse the target vertical image and the target horizontal image based on the correlation information between them to obtain a target 3D image.

[0017] Another aspect of this disclosure provides an electronic device comprising:

[0018] One or more processors;

[0019] Memory, used to store one or more programs.

[0020] When one or more programs are executed by one or more processors, the one or more processors implement the above-mentioned fast three-dimensional imaging method for space targets based on active-passive composite.

[0021] Another aspect of this disclosure provides a computer-readable storage medium storing computer-executable instructions, which, when executed, are used to implement the above-described method for rapid three-dimensional imaging of space targets based on a combined active and passive approach.

[0022] Another aspect of this disclosure provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method for rapid three-dimensional imaging of space targets based on a combination of active and passive methods.

[0023] According to the active-passive composite spatial target rapid 3D imaging method and system provided in this disclosure, the vertical and horizontal images of the target object are acquired by imaging equipment based on the target observation line of sight. The vertical and horizontal images are then calibrated according to first and second relative motion information to obtain the target vertical and horizontal images. Based on correlation information, the target vertical and horizontal images are orthogonally fused to obtain the target 3D image. Since an optical camera and synthetic aperture lidar simultaneously acquire vertical and horizontal images based on the same target observation line of sight, and then the orthogonal characteristics of the images are used to fuse them to obtain the target 3D image, 3D information and 3D images are obtained from a single observation direction. This eliminates the need for large-angle and multi-angle image acquisition of the target object, reducing imaging complexity and improving imaging efficiency. Furthermore, the synthetic aperture lidar imaging method is not limited by lighting conditions, allowing for high-resolution horizontal imaging of the target object in environments such as shadow areas, backlighting, and long distances, thus enabling the fusion of high-resolution target 3D images. Attached Figure Description

[0024] The above and other objects, features, and advantages of this disclosure will become clearer from the following description of embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0025] Figure 1 A flowchart of a rapid three-dimensional imaging method for space targets based on a combined active and passive approach according to an embodiment of the present disclosure is shown;

[0026] Figure 2 An example schematic diagram of image acquisition according to an embodiment of the present disclosure is shown;

[0027] Figure 3 An example schematic diagram of generating a target three-dimensional image according to an embodiment of the present disclosure is shown;

[0028] Figure 4 An example schematic diagram illustrating the generation of an intermediate three-dimensional image according to an embodiment of the present disclosure is shown;

[0029] Figure 5 A block diagram of a rapid three-dimensional imaging system for space targets based on a combined active and passive imaging approach according to an embodiment of the present disclosure is shown; and

[0030] Figure 6 A block diagram of an electronic device suitable for implementing a rapid three-dimensional imaging method for space targets based on a combined active and passive approach, according to an embodiment of the present disclosure, is shown. Detailed Implementation

[0031] The embodiments of the present disclosure will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the disclosure. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the present disclosure for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concepts of the present disclosure.

[0032] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. The terms “comprising,” “including,” etc., as used herein indicate the presence of features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0033] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0034] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0035] In the process of developing this disclosure, it was discovered that synthetic aperture lidar (SAR) is a newly developed technology in recent years, combining microwave synthetic aperture with laser technology to achieve imaging through the relative motion between the radar and the target. Because SAR operates at a high wavelength, minimal relative motion between the radar and the target is sufficient to meet the synthetic aperture conditions and achieve imaging. Furthermore, since SAR operates in the laser band, its imaging effect is closer to that of optical images. However, both optical passive imaging and SAR imaging only obtain two-dimensional image information of the target, still requiring multi-angle observation. Rapidly obtaining a three-dimensional image of the target is difficult and inefficient.

[0036] In view of this, embodiments of this disclosure provide a method and system for rapid three-dimensional imaging of a space target based on a combination of active and passive optics. The method includes: acquiring images of the target object using an imaging device based on the target's line-of-sight direction to obtain a vertical image and a horizontal image, wherein the vertical image is obtained based on an optical camera and passive optics technology, and the plane containing the vertical image is perpendicular to the target's line-of-sight direction; determining first relative motion information between the target object and the optical camera; determining second relative motion information between the target object and a synthetic aperture lidar; calibrating the dimensions of the vertical and horizontal images according to the first and second relative motion information to obtain a target vertical image and a target horizontal image; and orthogonally fusing the target vertical and horizontal images based on the correlation information between them to obtain a target three-dimensional image.

[0037] In the technical solution disclosed herein, the user information (including but not limited to user personal information, user image information, user device information, such as location information) and data (including but not limited to data used for analysis, stored data, and displayed data) involved are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of related data all comply with relevant laws, regulations, and standards, necessary confidentiality measures have been taken, and they do not violate public order and good morals. Corresponding operation entry points are provided for users to choose to authorize or refuse.

[0038] Figure 1 A flowchart of a rapid three-dimensional imaging method for space targets based on a combination of active and passive methods according to an embodiment of this disclosure is shown.

[0039] like Figure 1 As shown, the method 100 includes operations S110 to S150.

[0040] During operation S110, based on the target observation line of sight, the imaging device is used to acquire images of the target object, obtaining vertical and horizontal images.

[0041] According to embodiments of this disclosure, the target observation line-of-sight direction characterizes the line-of-sight direction of the imaging device when photographing the target object. The imaging device includes an optical camera and a synthetic aperture lidar. The target object can be a car, a satellite, etc.

[0042] According to embodiments of this disclosure, the optical camera can be a visible light camera or an infrared camera, utilizing passive optics to obtain a two-dimensional vertical image of the target object perpendicular to the target's line of sight. The plane containing the vertical image is perpendicular to the target's line of sight.

[0043] For example, if the two-dimensional directions of the target object perpendicular to the direction of the target's line of sight are the X-axis and Y-axis, then the vertical image contains information about the target object in the X-axis and Y-axis directions.

[0044] According to embodiments of this disclosure, the active technology is inverse synthetic aperture technology or synthetic aperture laser technology. Using inverse synthetic aperture technology or synthetic aperture laser technology, a two-dimensional horizontal image of the target object along the target's line of sight is obtained through the relative motion between the synthetic aperture lidar and the target object. The plane containing the horizontal image is parallel to the target's line of sight.

[0045] For example, if the two-dimensional directions of the target object along the target observation line of sight are the X-axis and Z-axis, then the horizontal image contains information about the target object in the X-axis and Z-axis directions.

[0046] For example, if the two-dimensional directions of the target object along the target observation line of sight are the Y-axis and Z-axis, then the horizontal image contains information about the target object in the Y-axis and Z-axis directions.

[0047] According to embodiments of this disclosure, horizontal images can be obtained even with very small relative motion using inverse synthetic aperture technology or synthetic aperture laser technology. Therefore, the imaging time of synthetic aperture lidar is comparable to that of an optical camera, and vertical and horizontal images can be acquired simultaneously.

[0048] According to embodiments of this disclosure, the imaging resolution using inverse synthetic aperture technology or synthetic aperture laser technology is not affected by the detection distance. Even when the target object is moving at a distance, a horizontal image with super-diffraction limit resolution can be obtained, thereby obtaining a high-resolution three-dimensional image of the target.

[0049] Figure 2 An example schematic diagram of image acquisition according to an embodiment of the present disclosure is shown.

[0050] like Figure 2 As shown, the imaging device includes an optical camera 1 and a synthetic aperture lidar 2. The target object 3 moves relative to the imaging device in direction 5. The optical camera 1 uses passive optics to obtain a vertical image of the target object 3 perpendicular to the target observation line of sight 4; the synthetic aperture lidar obtains a horizontal image of the target object 3 along the target observation line of sight 4.

[0051] In operation S120, the first relative motion information between the target object and the optical camera is determined.

[0052] According to embodiments of this disclosure, the first relative motion information characterizes the state information of the target object when it moves relative to the optical camera. For example, it can be angle change information, texture change information, etc.

[0053] According to embodiments of this disclosure, after acquiring a vertical image, first relative motion information is obtained from an optical camera.

[0054] In operation S130, the second relative motion information between the target object and the synthetic aperture lidar is determined.

[0055] According to embodiments of this disclosure, the second relative motion information characterizes the state information of the target object when it is moving relative to the synthetic aperture lidar. For example, it can be velocity change information, distance change information, etc.

[0056] According to embodiments of this disclosure, after acquiring a horizontal image, second relative motion information is obtained from a synthetic aperture lidar.

[0057] In operation S140, based on the first relative motion information and the second relative motion information, the vertical image and the horizontal image are sized respectively to obtain the target vertical image and the target horizontal image.

[0058] According to embodiments of this disclosure, the vertical image is sized based on the first relative motion information and the second relative motion information to determine the actual size information of the target object in the X-axis and Y-axis directions, thereby determining the scale of the vertical image and the target object in the X-axis and Y-axis directions. The vertical image is then adjusted based on the scale to obtain the target vertical image.

[0059] According to embodiments of this disclosure, the horizontal image is sized based on the first relative motion information and the second relative motion information. This allows the determination of the actual size information of the target object in the X-axis and Z-axis directions or the Y-axis and Z-axis directions. Consequently, the scale of the horizontal image relative to the target object in the X-axis and Z-axis directions or the Y-axis and Z-axis directions is determined. The horizontal image is then adjusted based on the scale to obtain the target horizontal image.

[0060] In operation S150, based on the correlation information between the target vertical image and the target horizontal image, the target vertical image and the target horizontal image are orthogonally fused to obtain the target three-dimensional image.

[0061] According to embodiments of this disclosure, the association information represents the same information about the target object contained in the target vertical image and the target horizontal image. For example, it can be information about the target object in the X-axis direction.

[0062] According to embodiments of this disclosure, the target vertical image and the target horizontal image are orthogonal in dimensions. Therefore, based on the correlation information between the target vertical image and the target horizontal image, orthogonal fusion of the target vertical image and the target horizontal image is performed to obtain a target three-dimensional image.

[0063] According to embodiments of this disclosure, the target 3D image contains 3D information of the target object in the X-axis direction, Y-axis direction, and Z-axis direction.

[0064] According to the embodiments of this disclosure, since an optical camera and a synthetic aperture lidar simultaneously acquire vertical and horizontal images based on the same target observation line of sight, and then utilize the orthogonal characteristics of the images to fuse them to obtain a three-dimensional image of the target, it is possible to obtain three-dimensional information and three-dimensional images from a single observation direction. This eliminates the need for large-angle and multi-angle image acquisition of the target object, reducing imaging complexity and improving imaging efficiency. In addition, the synthetic aperture lidar imaging method is not limited by lighting conditions. It can perform high-resolution horizontal imaging of the target object in environments such as ground shadow areas, backlighting, and long distances, thereby fusing to obtain a high-resolution three-dimensional image of the target.

[0065] Figure 3 An example schematic diagram of generating a target three-dimensional image is shown according to an embodiment of the present disclosure.

[0066] like Figure 3 As shown, vertical image 6 contains information about the target object in the X-axis and Y-axis directions, and horizontal image 7 contains information about the target object in the X-axis and Z-axis directions, or in the Y-axis and Z-axis directions. Based on the first relative motion information and the second relative motion information, the vertical image 6 and horizontal image 7 are sized to obtain a target vertical image and a target horizontal image. Based on the correlation information between the target vertical image and the target horizontal image, orthogonal fusion 8 is performed on the target vertical image and the target horizontal image to obtain a target three-dimensional image 9.

[0067] According to an embodiment of this disclosure, the second relative motion information includes relative distance information; wherein, according to the first relative motion information and the second relative motion information, the vertical image and the horizontal image are sized to obtain the target vertical image and the target horizontal image respectively, including: obtaining the first dimension and the second dimension of the target object according to the field of view parameters of the optical camera and the relative distance information; and sized the vertical image based on the first dimension and the second dimension to obtain the target vertical image.

[0068] According to embodiments of this disclosure, the relative distance information is the distance information of the target object relative to the synthetic aperture lidar when it is moving.

[0069] According to embodiments of this disclosure, the field of view parameter characterizes the field of view set when the optical camera acquires a vertical image.

[0070] According to embodiments of this disclosure, the first dimension represents the actual size of the target object in the first dimension direction, and the second dimension represents the actual size of the target object in the second dimension direction. For example, with the target object as the center, the first dimension direction can be the X-axis direction, and the second dimension direction can be the Y-axis direction.

[0071] For example, if the first dimension can be the Y-axis, then the second dimension can be the X-axis.

[0072] According to embodiments of this disclosure, the field-of-view parameters and relative distance information of the optical camera are processed using a size function to obtain the first dimension and the second dimension of the target object.

[0073] According to embodiments of this disclosure, a first-dimensional scale of the vertical image is determined based on a first-dimensional size and a size on the first dimension of the vertical image; a second-dimensional scale is determined based on a second-dimensional size and a size on the second dimension of the vertical image.

[0074] According to embodiments of this disclosure, a target vertical image is obtained by adjusting a vertical image based on a first-dimensional scale and a second-dimensional scale.

[0075] According to embodiments of this disclosure, the first relative motion information includes relative angle change information; wherein, the process of calibrating the vertical image and the horizontal image according to the first relative motion information and the second relative motion information to obtain the target vertical image and the target horizontal image further includes: obtaining the first dimension and the third dimension of the target object according to the relative angle change information and the relative distance information; and calibrating the horizontal image according to the first dimension and the third dimension to obtain the target horizontal image.

[0076] According to embodiments of this disclosure, the relative angle change information is the orientation change information of the target object relative to the optical camera during its motion.

[0077] According to embodiments of this disclosure, the third dimension characterizes the actual size of the target object in the third dimension direction. For example, with the target object as the center, the third dimension direction can be the Z-axis direction.

[0078] According to embodiments of this disclosure, relative angle change information and relative distance information are processed using a size function to obtain the first dimension and third dimension of the target object.

[0079] According to embodiments of this disclosure, a first-dimensional scale of the horizontal image is determined based on a first-dimensional dimension and a dimension on the first dimension of the horizontal image; a third-dimensional scale is determined based on a third-dimensional dimension and a dimension on the third dimension of the horizontal image.

[0080] According to embodiments of this disclosure, a target horizontal image is obtained by adjusting the horizontal image based on a first-dimensional scale and a third-dimensional scale.

[0081] According to embodiments of this disclosure, obtaining the first and third dimensions of the target object based on relative angle change information and relative distance information includes:

[0082] Determine the Doppler information of the horizontal image; obtain the relative vector velocity information of the target object based on the relative angle change information and the Doppler information; obtain the first dimension and third dimension of the target object based on the relative vector velocity information and the relative distance information.

[0083] According to embodiments of this disclosure, the horizontal image is acquired based on signals emitted by a synthetic aperture lidar towards a target object, and the Doppler information of the horizontal image characterizes the changes in wavelength and frequency when the wave source generated by the synthetic aperture lidar moves relative to the target object.

[0084] According to embodiments of this disclosure, the relative angle change information and Doppler information are processed using a Doppler function to obtain the relative vector velocity information of the target object. The relative vector velocity information characterizes the direction and magnitude of the velocity of the target object relative to the synthetic aperture lidar.

[0085] According to embodiments of this disclosure, relative vector velocity information and relative distance information are processed using a size function to obtain the first dimension and third dimension of the target object.

[0086] According to embodiments of this disclosure, by using a detection method that fuses an optical camera and a synthetic aperture lidar, multi-dimensional information such as relative distance information, relative vector velocity information, and relative angle change information can be obtained, which can be conveniently used for precise measurement of the trajectory of a target object.

[0087] According to embodiments of this disclosure, the target vertical image is constructed based on the first and second dimension information of the target object, and the target horizontal image is constructed based on the first and third dimension information of the target object. The correlation information between the target vertical image and the target horizontal image includes the first dimension information. Specifically, based on the correlation information between the target vertical image and the target horizontal image, orthogonal fusion is performed on the target vertical image and the target horizontal image to obtain a target three-dimensional image, including:

[0088] Based on the first-dimensional information, coordinate matching is performed on the pixels in the vertical image and the horizontal image of the target to obtain an initial 3D image; the initial 3D image is then processed using a distortion correction algorithm to obtain the target 3D image.

[0089] According to embodiments of this disclosure, the pixels in the target vertical image include first-dimensional information and second-dimensional information of the target object. The first-dimensional information represents the feature information of the target object in the first-dimensional direction, and the second-dimensional information represents the feature information of the target object in the second-dimensional direction.

[0090] According to embodiments of this disclosure, the pixels in the target horizontal image include first-dimensional information and third-dimensional information of the target object, wherein the third-dimensional information characterizes the feature information of the target object in the third-dimensional direction.

[0091] According to embodiments of this disclosure, the association information between the target vertical image and the target horizontal image includes first dimension information, such as feature information in the X-axis direction.

[0092] For example, the first dimension information is the information in the X-axis direction. Both the target vertical image and the target horizontal image contain information about the target object in the X-axis direction. The X-axis coordinates of the pixels in the target vertical image and the target horizontal image are matched. After a successful match, the Y-axis and Z-axis coordinates of the pixels are fused to obtain the initial three-dimensional image.

[0093] According to embodiments of this disclosure, a distortion correction algorithm is used to process pixels in an initial 3D image for denoising, texture correction, and other processes to obtain a target 3D image.

[0094] According to embodiments of this disclosure, processing an initial 3D image using a distortion correction algorithm to obtain a target 3D image includes:

[0095] Shape correction is performed on the initial 3D image to obtain an intermediate 3D image; texture correction is performed on the intermediate 3D image to obtain the target 3D image.

[0096] According to embodiments of this disclosure, the shape of the target object depicted in the initial 3D image may be significantly damaged, for example, deformed. Shape correction of the initial 3D image can correct this image damage, resulting in an intermediate 3D image.

[0097] According to embodiments of this disclosure, texture correction can be performed on intermediate 3D images based on convolutional networks to supplement texture details and obtain a corrected target 3D image.

[0098] Figure 4 An example schematic diagram illustrating the generation of an intermediate three-dimensional image according to an embodiment of the present disclosure is shown.

[0099] like Figure 4As shown, based on the field of view parameter 10 of the optical camera and the relative distance information, the vertical image 6 is sized to obtain the target vertical image 11; based on the relative angle change information 12 and the relative distance information, the horizontal image is sized to obtain the target horizontal image 13; based on the first dimension information, the pixels in the target vertical image 11 and the pixels in the target horizontal image 13 are matched to obtain the initial three-dimensional image 14; the initial three-dimensional image 14 is shaped to obtain the intermediate three-dimensional image 15.

[0100] Based on the aforementioned rapid three-dimensional imaging method for space targets using a combined active and passive approach, this disclosure also provides a rapid three-dimensional imaging system for space targets based on a combined active and passive approach. The following will be combined with... Figure 5 The system is described in detail.

[0101] Figure 5 A structural block diagram of a rapid three-dimensional imaging system for space targets based on a combined active and passive approach according to an embodiment of the present disclosure is shown.

[0102] like Figure 5 As shown, the active-passive composite rapid three-dimensional imaging system 500 for space targets in this embodiment includes an acquisition module 510, a first determination module 520, a second determination module 530, a calibration module 540, and a fusion module 550.

[0103] The acquisition module 510 is used to acquire images of the target object based on the target's line-of-sight direction using an imaging device, obtaining vertical and horizontal images. The imaging device includes an optical camera and a synthetic aperture lidar. The vertical image is obtained using the optical camera and passive optics technology, and the plane containing the vertical image is perpendicular to the target's line-of-sight direction. The horizontal image is obtained using the synthetic aperture lidar and active technology, and the plane containing the horizontal image is parallel to the target's line-of-sight direction. In one embodiment, the acquisition module 510 can be used to perform the operation S110 described above, which will not be repeated here.

[0104] The first determining module 520 is used to determine the first relative motion information between the target object and the optical camera. In one embodiment, the first determining module 520 can be used to perform the operation S120 described above, which will not be repeated here.

[0105] The second determining module 530 is used to determine the second relative motion information between the target object and the synthetic aperture lidar. In one embodiment, the second determining module 530 can be used to perform the operation S130 described above, which will not be repeated here.

[0106] The calibration module 540 is used to calibrate the dimensions of the vertical image and the horizontal image respectively based on the first relative motion information and the second relative motion information to obtain the target vertical image and the target horizontal image. In one embodiment, the calibration module 540 can be used to perform the operation S140 described above, which will not be repeated here.

[0107] The fusion module 550 is used to orthogonally fuse the target vertical image and the target horizontal image based on the correlation information between them to obtain a target 3D image. In one embodiment, the fusion module 550 can be used to perform the operation S150 described above, which will not be repeated here.

[0108] According to embodiments of this disclosure, calibration module 540 includes a first calibration submodule and a second calibration submodule.

[0109] The first calibration submodule is used to obtain the first and second dimensions of the target object based on the field of view parameters and relative distance information of the optical camera.

[0110] The second calibration submodule is used to calibrate the vertical image based on the first and second dimensions to obtain the target vertical image.

[0111] According to embodiments of this disclosure, calibration module 540 further includes a third calibration submodule and a fourth calibration submodule.

[0112] The third calibration submodule is used to obtain the first and third dimensions of the target object based on the relative angle change information and the relative distance information.

[0113] The fourth calibration submodule is used to calibrate the dimensions of the horizontal image based on the first and third dimensions to obtain the target horizontal image.

[0114] According to embodiments of this disclosure, the third calibration submodule includes a first calibration unit, a second calibration unit, and a third calibration unit.

[0115] The first calibration unit is used to determine the Doppler information of the horizontal image.

[0116] The second calibration unit is used to obtain the relative vector velocity information of the target object based on the relative angle change information and Doppler information.

[0117] The third calibration unit is used to obtain the first and third dimensions of the target object based on the relative vector velocity information and the relative distance information.

[0118] According to embodiments of this disclosure, the fusion module 550 includes a first fusion submodule and a second fusion submodule.

[0119] The first fusion submodule is used to perform coordinate matching of pixels in the target vertical image and pixels in the target horizontal image based on the first dimension information to obtain an initial three-dimensional image.

[0120] The second fusion submodule is used to process the initial 3D image using a distortion correction algorithm to obtain the target 3D image.

[0121] According to embodiments of this disclosure, the second fusion submodule includes a first fusion unit and a second fusion unit.

[0122] The first fusion unit is used to perform shape correction on the initial 3D image to obtain an intermediate 3D image.

[0123] The second fusion unit is used to perform texture correction on the intermediate 3D image to obtain the target 3D image.

[0124] Any one or more of the modules, submodules, units, and subunits according to embodiments of the present disclosure, or at least part of the functions of any one or more of them, can be implemented in one module. Any one or more of the modules, submodules, units, and subunits according to embodiments of the present disclosure can be implemented by dividing them into multiple modules. Any one or more of the modules, submodules, units, and subunits according to embodiments of the present disclosure can be at least partially implemented as hardware circuitry, such as a Field-Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a System-on-Chip, a System-on-a-Substrate, a System-on-Package, an Application-Specific Integrated Circuit (ASIC), or implemented in hardware or firmware by any other reasonable means of integrating or packaging circuitry, or implemented in software, hardware, or firmware, or in any suitable combination of any of these three implementation methods. Alternatively, one or more of the modules, submodules, units, and subunits according to embodiments of the present disclosure can be at least partially implemented as computer program modules, which, when run, can perform corresponding functions.

[0125] For example, any plurality of the acquisition module 510, the first determination module 520, the second determination module 530, the calibration module 540, and the fusion module 550 can be combined into one module / unit / subunit, or any one of these modules / units / subunits can be split into multiple modules / units / subunits. Alternatively, at least part of the functionality of one or more of these modules / units / subunits can be combined with at least part of the functionality of other modules / units / subunits and implemented in one module / unit / subunit. According to embodiments of this disclosure, at least one of the acquisition module 510, the first determination module 520, the second determination module 530, the calibration module 540, and the fusion module 550 can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or any other reasonable means of integrating or packaging the circuitry, or implemented in software, hardware, or firmware, or in any suitable combination of any of these three implementation methods. Alternatively, at least one of the acquisition module 510, the first determination module 520, the second determination module 530, the calibration module 540, and the fusion module 550 can be at least partially implemented as a computer program module, which can perform corresponding functions when the computer program module is run.

[0126] Figure 6 A block diagram of an electronic device suitable for implementing a rapid three-dimensional imaging method for space targets based on a combined active and passive method according to an embodiment of the present invention is shown.

[0127] Figure 6 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0128] like Figure 6 As shown, a computer electronic device 600 according to an embodiment of the present invention includes a processor 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage portion 608 into a random access memory (RAM) 603. The processor 601 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 601 may also include onboard memory for caching purposes. The processor 601 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0129] RAM 603 stores various programs and data required for the operation of electronic device 600. Processor 601, ROM 602, and RAM 603 are interconnected via bus 604. Processor 601 executes various operations of the method flow according to embodiments of the present invention by executing programs in ROM 602 and / or RAM 603. It should be noted that programs may also be stored in one or more memories other than ROM 602 and RAM 603. Processor 601 may also execute various operations of the method flow according to embodiments of the present invention by executing programs stored in one or more memories.

[0130] Optionally, the electronic device 600 may also include an input / output (I / O) interface 605, which is also connected to the bus 604. The electronic device 600 may also include one or more of the following components connected to the input / output (I / O) interface 605: an input section 606 including a keyboard, mouse, etc.; an output section 607 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the input / output (I / O) interface 605 as needed. A removable medium 611, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 610 as needed so that computer programs read from it can be installed into the storage section 608 as needed.

[0131] Optionally, the method flow according to embodiments of the present invention can be implemented as a computer software program. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a computer-readable storage medium, the computer program containing program code for performing the method shown in the flowchart. In such embodiments, the computer program can be downloaded and installed from a network via communication section 609, and / or installed from removable medium 611. When the computer program is executed by processor 601, it performs the functions defined in the system of embodiments of the present invention. Optionally, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0132] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the target three-dimensional image generation method according to embodiments of the present invention.

[0133] Optionally, the computer-readable storage medium can be a non-volatile computer-readable storage medium. Examples include, but are not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this invention, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0134] For example, optionally, the computer-readable storage medium may include ROM 602 and / or RAM 603 and / or one or more memories other than ROM 602 and RAM 603 as described above.

[0135] Embodiments of the present invention also include a computer program product comprising a computer program containing program code for performing the methods provided in the embodiments of the present invention. When the computer program product is run on an electronic device, the program code is used to enable the electronic device to implement the fast three-dimensional imaging method for space targets based on active-passive composite provided in the embodiments of the present invention.

[0136] When the computer program is executed by the processor 601, it performs the functions defined in the system / apparatus of this embodiment of the invention. Optionally, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0137] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and downloaded and installed via the communication section 609, and / or installed from the removable medium 611. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0138] Optionally, program code for executing the computer programs provided in the embodiments of the present invention can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0139] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions. Those skilled in the art will understand that the features described in the various embodiments of the present disclosure can be combined and / or combined in various ways, even if such combinations are not explicitly described in the present disclosure. In particular, the features described in the various embodiments of this disclosure may be combined and / or combined in various ways without departing from the spirit and teachings of this disclosure. All such combinations and / or combinations fall within the scope of this disclosure.

[0140] The embodiments of this disclosure have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of this disclosure. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of this disclosure, and all such substitutions and modifications should fall within the scope of this disclosure.

Claims

1. A rapid three-dimensional imaging method for space targets based on a combination of active and passive methods, characterized in that, The method includes: Based on the target observation line of sight, an imaging device is used to acquire images of the target object, resulting in vertical and horizontal images. The imaging device includes an optical camera and a synthetic aperture lidar. The vertical image is obtained based on the optical camera and passive optics technology, and the plane containing the vertical image is perpendicular to the target observation line of sight. The horizontal image is obtained based on the synthetic aperture lidar and active technology, and the plane containing the horizontal image is parallel to the target observation line of sight. Determine the first relative motion information between the target object and the optical camera, wherein the first relative motion information includes relative angle change information; Determine the second relative motion information between the target object and the synthetic aperture lidar, the second relative motion information including relative distance information; Based on the first relative motion information and the second relative motion information, the vertical image and the horizontal image are sized respectively to obtain the target vertical image and the target horizontal image, including: Based on the field of view parameters of the optical camera and the relative distance information, the first dimension and the second dimension of the target object are obtained; Based on the first dimension and the second dimension, the vertical image is sized to obtain the target vertical image; Based on the relative angle change information and the relative distance information, the first dimension and third dimension of the target object are obtained as follows: Determine the Doppler information of the horizontal image; Based on the relative angle change information and the Doppler information, the relative vector velocity information of the target object is obtained; Based on the relative vector velocity information and the relative distance information, the first dimension and the third dimension of the target object are obtained; Based on the first dimension and the third dimension, the horizontal image is sized to obtain the target horizontal image; Based on the correlation information between the target vertical image and the target horizontal image, the target vertical image and the target horizontal image are orthogonally fused to obtain a target three-dimensional image.

2. The method according to claim 1, characterized in that, The target vertical image is constructed based on the first and second dimension information of the target object, and the target horizontal image is constructed based on the first and third dimension information of the target object. The association information between the target vertical image and the target horizontal image includes the first dimension information. Specifically, based on the correlation information between the target vertical image and the target horizontal image, orthogonal fusion is performed on the target vertical image and the target horizontal image to obtain a target 3D image, including: Based on the first dimension information, coordinate matching is performed on the pixels in the target vertical image and the pixels in the target horizontal image to obtain an initial three-dimensional image; The initial 3D image is processed using a distortion correction algorithm to obtain the target 3D image.

3. The method according to claim 2, characterized in that, The initial 3D image is processed using a distortion correction algorithm to obtain the target 3D image, including: The initial 3D image is shaped and corrected to obtain an intermediate 3D image; The intermediate 3D image is texture-corrected to obtain the target 3D image.

4. A rapid three-dimensional imaging system for space targets based on a combination of active and passive methods, characterized in that, The system includes: The acquisition module is used to acquire images of the target object based on the target observation line of sight using an imaging device, obtaining vertical and horizontal images. The imaging device includes an optical camera and a synthetic aperture lidar. The vertical image is obtained based on the optical camera and passive optics technology, and the plane containing the vertical image is perpendicular to the target observation line of sight. The horizontal image is obtained based on the synthetic aperture lidar and active technology, and the plane containing the horizontal image is parallel to the target observation line of sight. The first determining module is used to determine the first relative motion information between the target object and the optical camera, wherein the first relative motion information includes relative angle change information. The second determining module is used to determine the second relative motion information between the target object and the synthetic aperture lidar, the second relative motion information including relative distance information; The calibration module is used to calibrate the dimensions of the vertical image and the horizontal image respectively based on the first relative motion information and the second relative motion information to obtain the target vertical image and the target horizontal image. The calibration module includes a first calibration submodule, a second calibration submodule, a third calibration submodule and a fourth calibration submodule. The first calibration submodule is used to obtain the first dimension and the second dimension of the target object based on the field of view parameters of the optical camera and the relative distance information. The second calibration submodule is used to perform size calibration on the vertical image based on the first dimension and the second dimension to obtain the target vertical image; The third calibration submodule is used to obtain the first dimension and the third dimension of the target object based on the relative angle change information and the relative distance information. The third calibration submodule includes a first calibration unit, a second calibration unit, and a third calibration unit. The first calibration unit is used to determine the Doppler information of the horizontal image; The second calibration unit is used to obtain the relative vector velocity information of the target object based on the relative angle change information and the Doppler information. The third calibration unit is used to obtain the first dimension and the third dimension of the target object based on the relative vector velocity information and the relative distance information. The fourth calibration submodule is used to calibrate the size of the horizontal image based on the first dimension and the third dimension to obtain the target horizontal image; The fusion module is used to orthogonally fuse the target vertical image and the target horizontal image based on the correlation information between them to obtain a target three-dimensional image.

5. An electronic device, comprising: One or more processors; Memory, used to store one or more programs. Wherein, when one or more programs are executed by one or more processors, the one or more processors implement the method of any one of claims 1 to 3.

6. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, cause the processor to perform the method of any one of claims 1 to 3.

7. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 3.

Images