Space target rapid three-dimensional imaging method and system based on active and passive combination

Abstract

The invention provides a space target rapid three-dimensional imaging method and system based on active and passive combination, and can be applied to the field of imaging detection. The method comprises steps that based on a target observation line-of-sight direction, an imaging device is utilized to carry out image acquisition on a target object, a vertical image and a horizontal image are acquired, the vertical image is acquired based on an optical camera and a passive optical technology, and a plane where the vertical image is located is vertical to the target observation line-of-sight direction; determining first relative motion information of the target object and the optical camera; determining second relative motion information of the target object and the synthetic aperture laser radar; performing size calibration on the vertical image and the horizontal image according to the first relative motion information and the second relative motion information to obtain a target vertical image and a target horizontal image; and based on the association information between the target vertical image and the target horizontal image, carrying out orthogonal fusion on the target vertical image and the target horizontal image to obtain a target three-dimensional image.

Description

Technical Field

[0001] The present disclosure relates to the field of imaging detection, and more specifically, to a method and system for rapid three-dimensional imaging of space targets based on active-passive combination. Background Art

[0002] Obtaining a three-dimensional image of the target is an important requirement for determining the type and function of the target. Optical images are closer to people's real feelings than radar images, and are easier to interpret. Traditional optical imaging technology observes the target through visible light or infrared cameras. In order to obtain a three-dimensional image of the target, the target needs to be observed at large angles and multiple angles, which is difficult to apply to complex scenes of moving targets. At the same time, the imaging process of optical cameras depends on lighting conditions, and it is difficult to obtain target image information in the shadow area and backlighting conditions of space. Synthetic aperture lidar is a newly developed technical means in recent years. The imaging process does not depend on lighting conditions, and the imaging effect is closer to optical images.

[0003] However, whether it is optical passive imaging or synthetic aperture imaging, the imaging effect is to obtain two-dimensional image information of the target, and multi-angle observation is still required. It is difficult and inefficient to quickly obtain a three-dimensional image of the target. Summary of the invention

[0004] In view of this, the present disclosure provides a method and system for rapid three-dimensional imaging of space targets based on active and passive combination.

[0005] One aspect of the present disclosure provides a method for rapid three-dimensional imaging of space targets based on active-passive combination, comprising: based on the target observation line of sight direction, using an imaging device to collect images of the target object to obtain a vertical image and a horizontal image, wherein the imaging device includes an optical camera and a synthetic aperture laser radar, the vertical image is obtained based on the optical camera and passive optical technology, and the plane where the vertical image is located is perpendicular to the target observation line of sight direction, and the horizontal image is obtained based on the synthetic aperture laser radar and active technology, and the plane where the horizontal image is located is parallel to the target observation line of sight direction; determining first relative motion information between the target object and the optical camera; determining second relative motion information between the target object and the synthetic aperture laser radar; calibrating the size of the vertical image and the horizontal image respectively according to the first relative motion information and the second relative motion information to obtain a target vertical image and a target horizontal image; and orthogonally fusing the target vertical image and the target horizontal image based on the association information between the target vertical image and the target horizontal image to obtain a target three-dimensional image.

[0006] According to an embodiment of the present disclosure, the second relative motion information includes relative distance information; wherein, according to the first relative motion information and the second relative motion information, the vertical image and the horizontal image are respectively calibrated to obtain the target vertical image and the target horizontal image, including: according to the field of view angle parameters and the relative distance information of the optical camera, the first dimension size and the second dimension size of the target object are obtained; based on the first dimension size and the second dimension size, the vertical image is calibrated to obtain the target vertical image.

[0007] According to an embodiment of the present disclosure, the first relative motion information includes relative angle change information; wherein, according to the first relative motion information and the second relative motion information, the vertical image and the horizontal image are calibrated respectively to obtain the target vertical image and the target horizontal image, which also includes: according to the relative angle change information and the relative distance information, the first dimension size and the third dimension size of the target object are obtained; based on the first dimension size and the third dimension size, the horizontal image is calibrated to obtain the target horizontal image.

[0008] According to an embodiment of the present disclosure, obtaining the first dimension and the third dimension of the target object according to the relative angle change information and the relative distance information includes: determining the Doppler information of the horizontal image; obtaining the relative vector velocity information of the target object according to the relative angle change information and the Doppler information; and obtaining the first dimension and the third dimension of the target object according to the relative vector velocity information and the relative distance information.

[0009] According to an embodiment of the present disclosure, a target vertical image is composed based on first dimensional information and second dimensional information of the target object, a target horizontal image is composed based on first dimensional information and third dimensional information of the target object, and association information between the target vertical image and the target horizontal image includes first dimensional information; wherein, based on the association information between the target vertical image and the target horizontal image, orthogonally fusion of the target vertical image and the target horizontal image is performed to obtain a target three-dimensional image, including: based on the first dimensional information, coordinate matching of pixel points in the target vertical image and pixel points in the target horizontal image is performed to obtain an initial three-dimensional image; and processing the initial three-dimensional image using a distortion correction algorithm to obtain a target three-dimensional image.

[0010] According to an embodiment of the present disclosure, using a distortion correction algorithm to process an initial three-dimensional image to obtain a target three-dimensional image includes: performing shape correction on the initial three-dimensional image to obtain an intermediate three-dimensional image; and performing texture correction on the intermediate three-dimensional image to obtain a target three-dimensional image.

[0011] Another aspect of the present disclosure provides a space target rapid three-dimensional imaging system based on active-passive combination, comprising:

[0012] An acquisition module is used to acquire images of a target object using an imaging device based on the target observation sight line direction, and obtain a vertical image and a horizontal image, wherein the imaging device includes an optical camera and a synthetic aperture laser radar, the vertical image is obtained based on the optical camera and passive optical technology, and the plane where the vertical image is located is perpendicular to the target observation sight line direction, and the horizontal image is obtained based on the synthetic aperture laser radar and active technology, and the plane where the horizontal image is located is parallel to the target observation sight line direction;

[0013] A first determination module, used to determine first relative motion information between the target object and the optical camera;

[0014] A second determination module, used to determine second relative motion information between the target object and the synthetic aperture laser radar;

[0015] A calibration module, used to calibrate the size of the vertical image and the horizontal image according to the first relative motion information and the second relative motion information, so as to obtain a target vertical image and a target horizontal image;

[0016] The fusion module is used to perform orthogonal fusion of the target vertical image and the target horizontal image based on the association information between the target vertical image and the target horizontal image to obtain a target three-dimensional image.

[0017] Another aspect of the present disclosure provides an electronic device, comprising:

[0018] one or more processors;

[0019] a memory for storing one or more programs,

[0020] When one or more programs are executed by one or more processors, the one or more processors implement the above-mentioned method for rapid three-dimensional imaging of space targets based on active and passive combination.

[0021] Another aspect of the present disclosure provides a computer-readable storage medium storing computer-executable instructions, which, when executed, are used to implement the above-mentioned method for rapid three-dimensional imaging of space targets based on active and passive combination.

[0022] Another aspect of the present disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements the above-mentioned method for rapid three-dimensional imaging of space targets based on active and passive combination.

[0023] According to the method and system for rapid three-dimensional imaging of space targets based on active and passive combination provided by the present disclosure, the vertical image and the horizontal image are obtained by using an imaging device to collect images of the target object based on the target observation line of sight direction; the vertical image and the horizontal image are calibrated according to the first relative motion information and the second relative motion information respectively to obtain the target vertical image and the target horizontal image; the target vertical image and the target horizontal image are orthogonally fused based on the associated information to obtain the target three-dimensional image. Since the optical camera and the synthetic aperture laser radar are used to simultaneously obtain the vertical image and the horizontal image based on the same target observation line of sight direction, and then the target three-dimensional image is obtained by fusing the images using the characteristics of mutual orthogonality, it is realized to obtain three-dimensional information and three-dimensional images from a single observation direction, and it is not necessary to collect large-angle and multi-angle images of the target object, which reduces the complexity of imaging and improves the imaging efficiency; in addition, the synthetic aperture laser radar imaging method, the imaging process is not limited by lighting conditions, and the target object can be imaged horizontally with high resolution in environments such as ground shadow areas, backlight, and long distances, thereby fusing to obtain a high-resolution target three-dimensional image. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The above and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0025] Figure 1 A flow chart of a method for rapid three-dimensional imaging of space targets based on active-passive combination according to an embodiment of the present disclosure is shown;

[0026] Figure 2 An example schematic diagram of image acquisition according to an embodiment of the present disclosure is shown;

[0027] Figure 3 An example schematic diagram of generating a target three-dimensional image according to an embodiment of the present disclosure is shown;

[0028] Figure 4 An example schematic diagram of generating an intermediate three-dimensional image according to an embodiment of the present disclosure is shown;

[0029] Figure 5 A block diagram of a space target rapid three-dimensional imaging system based on active-passive combination according to an embodiment of the present disclosure is shown; and

[0030] Figure 6 A block diagram of an electronic device suitable for implementing a method for rapid three-dimensional imaging of space targets based on active-passive combination according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0031] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0032] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0033] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0034] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0035] In the process of realizing the present disclosure, it is found that synthetic aperture laser radar is a newly developed technical means in recent years. It combines microwave synthetic aperture with laser technology to achieve imaging through the relative movement between radar and target. Since synthetic aperture laser radar operates in a high band, a very small relative movement between radar and target can meet the synthetic aperture conditions and achieve imaging. At the same time, since synthetic aperture laser radar operates in the laser band, its imaging effect is closer to optical images. However, whether it is optical passive imaging or synthetic aperture imaging, the imaging effect is to obtain two-dimensional image information of the target, and multi-angle observation is still required. It is difficult and inefficient to quickly obtain a three-dimensional image of the target.

[0036] In view of this, the embodiments of the present disclosure provide a method and system for fast three-dimensional imaging of space targets based on active-passive combination. The method includes: based on the target observation line of sight direction, using an imaging device to collect images of the target object to obtain a vertical image and a horizontal image, wherein the vertical image is obtained based on an optical camera and passive optical technology, and the plane where the vertical image is located is perpendicular to the target observation line of sight direction; determining the first relative motion information between the target object and the optical camera; determining the second relative motion information between the target object and the synthetic aperture laser radar; according to the first relative motion information and the second relative motion information, respectively calibrating the size of the vertical image and the horizontal image to obtain a target vertical image and a target horizontal image; based on the association information between the target vertical image and the target horizontal image, orthogonally fusing the target vertical image and the target horizontal image to obtain a target three-dimensional image.

[0037] In the technical solution of the present disclosure, the user information (including but not limited to user personal information, user image information, user device information, such as location information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with relevant laws, regulations and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0038] Figure 1 A flow chart of a method for rapid three-dimensional imaging of space targets based on active and passive combination according to an embodiment of the present disclosure is shown.

[0039] like Figure 1 As shown, the method 100 includes operations S110 to S150.

[0040] In operation S110, based on the target observation sight line direction, an imaging device is used to capture an image of the target object to obtain a vertical image and a horizontal image.

[0041] According to an embodiment of the present disclosure, the target observation sight direction represents the sight direction of the imaging device photographing the target object. The imaging device includes an optical camera and a synthetic aperture laser radar. The target object can be a car, a satellite, etc.

[0042] According to an embodiment of the present disclosure, the optical camera may be a visible light camera or an infrared camera, and passive optical technology is used to obtain a two-dimensional vertical image of the target object perpendicular to the target observation line of sight. The plane where the vertical image is located is perpendicular to the target observation line of sight.

[0043] For example, the two-dimensional directions of the target object perpendicular to the target observation line of sight are the X-axis direction and the Y-axis direction, and the vertical image contains the information of the target object in the X-axis direction and the Y-axis direction.

[0044] According to an embodiment of the present disclosure, the active technology is an inverse synthetic aperture technology or a synthetic aperture laser technology, which uses the inverse synthetic aperture technology or the synthetic aperture laser technology to obtain a two-dimensional horizontal image of the target object along the target observation line of sight through the relative motion between the synthetic aperture laser radar and the target object. The plane where the horizontal image is located is parallel to the target observation line of sight.

[0045] For example, the two-dimensional directions of the target object along the target observation line of sight are the X-axis direction and the Z-axis direction, and the horizontal image contains information of the target object in the X-axis direction and the Z-axis direction.

[0046] For example, the two-dimensional directions of the target object along the target observation line of sight are the Y-axis direction and the Z-axis direction, and the horizontal image contains information of the target object in the Y-axis direction and the Z-axis direction.

[0047] According to the embodiments of the present disclosure, a horizontal image can be obtained even for very small relative motion by using inverse synthetic aperture technology or synthetic aperture laser technology. Therefore, the imaging time of the synthetic aperture lidar is comparable to that of an optical camera, and vertical and horizontal images can be acquired synchronously.

[0048] According to the embodiments of the present disclosure, the imaging resolution using the inverse synthetic aperture technology or the synthetic aperture laser technology is not affected by the detection distance, and a horizontal image of the target with a resolution exceeding the diffraction limit can be obtained even when the target object moves relative to the target over a long distance, thereby obtaining a high-resolution three-dimensional image of the target.

[0049] Figure 2 An example schematic diagram of image acquisition according to an embodiment of the present disclosure is shown.

[0050] like Figure 2 As shown, the imaging device includes an optical camera 1 and a synthetic aperture laser radar 2. The target object 3 moves in a direction 5 relative to the imaging device. The optical camera 1 uses passive optical technology to obtain a vertical image of the target object 3 perpendicular to the target observation line of sight direction 4; and obtains a horizontal image of the target object 3 along the target observation line of sight direction 4 through the synthetic aperture laser radar.

[0051] In operation S120 , first relative motion information between the target object and the optical camera is determined.

[0052] According to an embodiment of the present disclosure, the first relative motion information represents state information of the target object and the optical camera when in relative motion, for example, it may be angle change information, texture change information, and the like.

[0053] According to an embodiment of the present disclosure, after the vertical image is captured, first relative motion information is acquired from the optical camera.

[0054] In operation S130, second relative motion information between the target object and the synthetic aperture laser radar is determined.

[0055] According to an embodiment of the present disclosure, the second relative motion information represents the state information of the target object and the synthetic aperture laser radar when they are in relative motion, for example, it may be speed change information, distance change information, etc.

[0056] According to an embodiment of the present disclosure, after collecting the horizontal image, the second relative motion information is obtained from the synthetic aperture laser radar.

[0057] In operation S140, the vertical image and the horizontal image are calibrated according to the first relative motion information and the second relative motion information to obtain a target vertical image and a target horizontal image.

[0058] According to an embodiment of the present disclosure, by calibrating the size of the vertical image based on the first relative motion information and the second relative motion information, the actual size information of the target object in the X-axis direction and the Y-axis direction can be determined, thereby determining the scale of the vertical image and the target object in the X-axis direction and the Y-axis direction, and adjusting the vertical image based on the scale to obtain the target vertical image.

[0059] According to an embodiment of the present disclosure, the horizontal image is calibrated according to the first relative motion information and the second relative motion information, and the actual size information of the target object in the X-axis, Z-axis direction or the Y-axis, Z-axis direction can be determined, thereby determining the scale of the horizontal image and the target object in the X-axis, Z-axis direction or the Y-axis, Z-axis direction, and adjusting the horizontal image based on the scale to obtain the target horizontal image.

[0060] In operation S150, based on the association information between the target vertical image and the target horizontal image, the target vertical image and the target horizontal image are orthogonally fused to obtain a target three-dimensional image.

[0061] According to an embodiment of the present disclosure, the association information represents the same information of the target object contained in the target vertical image and the target horizontal image, for example, the information of the target object in the X-axis direction.

[0062] According to an embodiment of the present disclosure, the target vertical image and the target horizontal image have the characteristic that their dimensions are mutually orthogonal. Therefore, based on the association information between the target vertical image and the target horizontal image, the target vertical image and the target horizontal image are orthogonally fused to obtain a target three-dimensional image.

[0063] According to an embodiment of the present disclosure, the target three-dimensional image includes three-dimensional information of the target object in the X-axis direction, the Y-axis direction, and the Z-axis direction.

[0064] According to the embodiments of the present disclosure, an optical camera and a synthetic aperture lidar are used to simultaneously obtain vertical and horizontal images based on the same target observation line of sight direction, and then the images are fused to obtain a three-dimensional image of the target by utilizing the orthogonal characteristics of each other, thereby achieving three-dimensional information and a three-dimensional image from a single observation direction. There is no need to perform large-angle and multi-angle image acquisition on the target object, thereby reducing the complexity of imaging and improving imaging efficiency. In addition, in the synthetic aperture lidar imaging method, the imaging process is not limited by lighting conditions, and high-resolution horizontal imaging of the target object can be performed in environments such as shadow areas, backlighting, and long distances, thereby obtaining a high-resolution three-dimensional image of the target through fusion.

[0065] Figure 3 An example schematic diagram of generating a target three-dimensional image according to an embodiment of the present disclosure is shown.

[0066] like Figure 3 As shown, the vertical image 6 contains information of the target object in the X-axis direction and the Y-axis direction, and the horizontal image 7 contains information of the target object in the X-axis direction and the Z-axis direction, or information in the Y-axis direction and the Z-axis direction. According to the first relative motion information and the second relative motion information, the vertical image 6 and the horizontal image 7 are calibrated respectively to obtain a target vertical image and a target horizontal image. Based on the association information between the target vertical image and the target horizontal image, the target vertical image and the target horizontal image are orthogonally fused 8 to obtain a target three-dimensional image 9.

[0067] According to an embodiment of the present disclosure, it is characterized in that the second relative motion information includes relative distance information; wherein, according to the first relative motion information and the second relative motion information, the vertical image and the horizontal image are respectively calibrated to obtain the target vertical image and the target horizontal image, including: according to the field of view angle parameters and the relative distance information of the optical camera, the first dimension size and the second dimension size of the target object are obtained; based on the first dimension size and the second dimension size, the vertical image is calibrated to obtain the target vertical image.

[0068] According to an embodiment of the present disclosure, the relative distance information is the distance information of the target object when it moves relative to the synthetic aperture lidar.

[0069] According to an embodiment of the present disclosure, the field of view angle parameter represents the field of view range set when the optical camera captures a vertical image.

[0070] According to an embodiment of the present disclosure, the first dimension size represents the actual size of the target object in the first dimension direction, and the second dimension size represents the actual size of the target object in the second dimension direction. For example, with the target object as the center, the first dimension direction may be the X-axis direction, and the second dimension direction may be the Y-axis direction.

[0071] For example, the first dimension direction may also be the Y-axis direction, and the second dimension direction may be the X-axis direction.

[0072] According to an embodiment of the present disclosure, a size function is used to process the field angle parameters and relative distance information of an optical camera to obtain a first dimension size and a second dimension size of a target object.

[0073] According to an embodiment of the present disclosure, the first dimension scale of the vertical image is determined based on the first dimension size and the size of the vertical image in the first dimension; the second dimension scale is determined based on the second dimension size and the size of the vertical image in the second dimension.

[0074] According to an embodiment of the present disclosure, a vertical image is adjusted based on a first dimensional scale and a second dimensional scale to obtain a target vertical image.

[0075] According to an embodiment of the present disclosure, the first relative motion information includes relative angle change information; wherein, according to the first relative motion information and the second relative motion information, the vertical image and the horizontal image are calibrated respectively to obtain the target vertical image and the target horizontal image, which also includes: according to the relative angle change information and the relative distance information, the first dimension size and the third dimension size of the target object are obtained; based on the first dimension size and the third dimension size, the horizontal image is calibrated to obtain the target horizontal image.

[0076] According to an embodiment of the present disclosure, the relative angle change information is the position change information of the target object when it moves relative to the optical camera.

[0077] According to an embodiment of the present disclosure, the third dimension size represents the actual size of the target object in the third dimension direction. For example, with the target object as the center, the third dimension direction may be the Z-axis direction.

[0078] According to an embodiment of the present disclosure, a size function is used to process relative angle change information and relative distance information to obtain a first dimension size and a third dimension size of the target object.

[0079] According to an embodiment of the present disclosure, the first dimension scale of the horizontal image is determined based on the first dimension size and the size of the horizontal image in the first dimension; the third dimension scale is determined based on the third dimension size and the size of the horizontal image in the third dimension.

[0080] According to an embodiment of the present disclosure, the horizontal image is adjusted based on the first dimension scale and the third dimension scale of the horizontal image to obtain a target horizontal image.

[0081] According to an embodiment of the present disclosure, obtaining the first dimension size and the third dimension size of the target object according to the relative angle change information and the relative distance information includes:

[0082] Determine the Doppler information of the horizontal image; obtain the relative vector velocity information of the target object according to the relative angle change information and the Doppler information; obtain the first dimension size and the third dimension size of the target object according to the relative vector velocity information and the relative distance information.

[0083] According to an embodiment of the present disclosure, a horizontal image is collected based on a signal emitted by a synthetic aperture laser radar to a target object, and Doppler information of the horizontal image represents information on changes in wavelength and frequency when a wave source generated by the synthetic aperture laser radar moves relative to the target object.

[0084] According to an embodiment of the present disclosure, the relative angle change information and the Doppler information are processed using a Doppler function to obtain the relative vector velocity information of the target object. The relative vector velocity information represents the velocity direction and magnitude information of the target object when it moves relative to the synthetic aperture laser radar.

[0085] According to an embodiment of the present disclosure, a size function is used to process relative vector velocity information and relative distance information to obtain a first dimension size and a third dimension size of the target object.

[0086] According to the embodiments of the present disclosure, based on the detection method of the fusion of optical camera and synthetic aperture lidar, multi-dimensional information such as relative distance information, relative vector velocity information, relative angle change information, etc. can be obtained, which can be conveniently used for fine orbit measurement of target objects.

[0087] According to an embodiment of the present disclosure, a target vertical image is formed based on first dimensional information and second dimensional information of a target object, a target horizontal image is formed based on first dimensional information and third dimensional information of the target object, and association information between the target vertical image and the target horizontal image includes the first dimensional information; wherein, based on the association information between the target vertical image and the target horizontal image, orthogonally fusion of the target vertical image and the target horizontal image is performed to obtain a target three-dimensional image, including:

[0088] Based on the first dimension information, coordinate matching is performed on pixel points in the target vertical image and pixel points in the target horizontal image to obtain an initial three-dimensional image; the initial three-dimensional image is processed using a distortion correction algorithm to obtain a target three-dimensional image.

[0089] According to an embodiment of the present disclosure, the pixel points in the target vertical image include first dimensional information and second dimensional information of the target object, the first dimensional information represents feature information of the target object in the first dimensional direction, and the second dimensional information represents feature information of the target object in the second dimensional direction.

[0090] According to an embodiment of the present disclosure, the pixel points in the target horizontal image include first dimensional information and third dimensional information of the target object, and the third dimensional information represents feature information of the target object in the third dimensional direction.

[0091] According to an embodiment of the present disclosure, the association information between the target vertical image and the target horizontal image includes first dimensional information, for example, may be feature information in the X-axis direction.

[0092] For example, the first dimension information is the information in the X-axis direction. Both the target vertical image and the target horizontal image contain the information of the target object in the X-axis direction. The X-axis coordinates of the pixel points in the target vertical image and the pixel points in the target horizontal image are matched. After the match is successful, the Y-axis and Z-axis coordinates of the pixel points are fused to obtain the initial three-dimensional image.

[0093] According to an embodiment of the present disclosure, a distortion correction algorithm is used to perform denoising, texture correction and other processing on the pixels in the initial three-dimensional image to obtain a target three-dimensional image.

[0094] According to an embodiment of the present disclosure, processing an initial three-dimensional image using a distortion correction algorithm to obtain a target three-dimensional image includes:

[0095] The initial three-dimensional image is subjected to shape correction to obtain an intermediate three-dimensional image; the intermediate three-dimensional image is subjected to texture correction to obtain a target three-dimensional image.

[0096] According to an embodiment of the present disclosure, the shape of the target object depicted in the initial three-dimensional image may be significantly damaged, for example, deformed. The initial three-dimensional image may be subjected to shape correction to correct the image damage and obtain an intermediate three-dimensional image.

[0097] According to the embodiments of the present disclosure, texture correction can be performed on the intermediate three-dimensional image based on a convolutional network to supplement texture details and obtain a corrected target three-dimensional image.

[0098] Figure 4 An example schematic diagram of generating an intermediate three-dimensional image according to an embodiment of the present disclosure is shown.

[0099] like Figure 4As shown, according to the field of view angle parameter 10 of the optical camera and the relative distance information, the vertical image 6 is calibrated to obtain the target vertical image 11; according to the relative angle change information 12 and the relative distance information, the horizontal image is calibrated to obtain the target horizontal image 13; based on the first dimension information, the pixel points in the target vertical image 11 and the pixel points in the target horizontal image 13 are matched to obtain the initial three-dimensional image 14; the initial three-dimensional image 14 is shape corrected to obtain the intermediate three-dimensional image 15.

[0100] Based on the above-mentioned space target rapid three-dimensional imaging method based on active and passive combination, the present disclosure also provides a space target rapid three-dimensional imaging system based on active and passive combination. Figure 5 The system is described in detail.

[0101] Figure 5 A structural block diagram of a space target rapid three-dimensional imaging system based on active-passive combination according to an embodiment of the present disclosure is shown.

[0102] like Figure 5 As shown, the space target rapid three-dimensional imaging system 500 based on active-passive combination of this embodiment includes an acquisition module 510 , a first determination module 520 , a second determination module 530 , a calibration module 540 and a fusion module 550 .

[0103] The acquisition module 510 is used to acquire images of the target object using an imaging device based on the target observation line of sight direction to obtain a vertical image and a horizontal image, wherein the imaging device includes an optical camera and a synthetic aperture laser radar, the vertical image is obtained based on the optical camera and passive optical technology, and the plane where the vertical image is located is perpendicular to the target observation line of sight direction, and the horizontal image is obtained based on the synthetic aperture laser radar and active technology, and the plane where the horizontal image is located is parallel to the target observation line of sight direction. In one embodiment, the acquisition module 510 can be used to perform the operation S110 described above, which will not be repeated here.

[0104] The first determination module 520 is used to determine the first relative motion information between the target object and the optical camera. In one embodiment, the first determination module 520 can be used to perform the operation S120 described above, which will not be described in detail here.

[0105] The second determination module 530 is used to determine the second relative motion information between the target object and the synthetic aperture laser radar. In one embodiment, the second determination module 530 can be used to perform the operation S130 described above, which will not be described in detail here.

[0106] The calibration module 540 is used to calibrate the size of the vertical image and the horizontal image according to the first relative motion information and the second relative motion information to obtain a target vertical image and a target horizontal image. In one embodiment, the calibration module 540 can be used to perform the operation S140 described above, which will not be described in detail here.

[0107] The fusion module 550 is used to perform orthogonal fusion of the target vertical image and the target horizontal image based on the association information between the target vertical image and the target horizontal image to obtain a target three-dimensional image. In one embodiment, the fusion module 550 can be used to perform the operation S150 described above, which will not be repeated here.

[0108] According to an embodiment of the present disclosure, the calibration module 540 includes a first calibration submodule and a second calibration submodule.

[0109] The first calibration submodule is used to obtain the first dimension size and the second dimension size of the target object according to the field angle parameter and relative distance information of the optical camera.

[0110] The second calibration submodule is used to calibrate the size of the vertical image based on the first dimension size and the second dimension size to obtain a target vertical image.

[0111] According to an embodiment of the present disclosure, the calibration module 540 further includes a third calibration submodule and a fourth calibration submodule.

[0112] The third calibration submodule is used to obtain the first dimension size and the third dimension size of the target object according to the relative angle change information and the relative distance information.

[0113] The fourth calibration submodule is used to calibrate the size of the horizontal image based on the first dimension and the third dimension to obtain a target horizontal image.

[0114] According to an embodiment of the present disclosure, the third calibration submodule includes a first calibration unit, a second calibration unit and a third calibration unit.

[0115] The first calibration unit is used to determine Doppler information of the horizontal image.

[0116] The second calibration unit is used to obtain the relative vector velocity information of the target object according to the relative angle change information and the Doppler information.

[0117] The third calibration unit is used to obtain the first dimension size and the third dimension size of the target object according to the relative vector velocity information and the relative distance information.

[0118] According to an embodiment of the present disclosure, the fusion module 550 includes a first fusion submodule and a second fusion submodule.

[0119] The first fusion submodule is used to perform coordinate matching on pixel points in the target vertical image and pixel points in the target horizontal image based on the first dimensional information to obtain an initial three-dimensional image.

[0120] The second fusion submodule is used to process the initial three-dimensional image using a distortion correction algorithm to obtain a target three-dimensional image.

[0121] According to an embodiment of the present disclosure, the second fusion submodule includes a first fusion unit and a second fusion unit.

[0122] The first fusion unit is used to perform shape correction on the initial three-dimensional image to obtain an intermediate three-dimensional image.

[0123] The second fusion unit is used to perform texture correction on the intermediate three-dimensional image to obtain a target three-dimensional image.

[0124] According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits, or at least part of the functions of any one of them can be implemented in one module. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be split into multiple modules for implementation. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be at least partially implemented as hardware circuits, such as field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), systems on chips, systems on substrates, systems on packages, application specific integrated circuits (ASICs), or can be implemented by hardware or firmware in any other reasonable way of integrating or packaging circuits, or can be implemented in any one of the three implementation methods of software, hardware, and firmware, or in any appropriate combination of any of them. Alternatively, according to the embodiments of the present invention, one or more of the modules, submodules, units, and subunits can be at least partially implemented as computer program modules, and when the computer program modules are run, the corresponding functions can be performed.

[0125] For example, any of the acquisition module 510, the first determination module 520, the second determination module 530, the calibration module 540, and the fusion module 550 can be combined in one module / unit / sub-unit for implementation, or any of the modules / units / sub-units can be split into multiple modules / units / sub-units. Alternatively, at least part of the functions of one or more of these modules / units / sub-units can be combined with at least part of the functions of other modules / units / sub-units and implemented in one module / unit / sub-unit. According to an embodiment of the present disclosure, at least one of the acquisition module 510, the first determination module 520, the second determination module 530, the calibration module 540, and the fusion module 550 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or by any one of the three implementation methods of software, hardware, and firmware, or by a suitable combination of any of them. Alternatively, at least one of the acquisition module 510, the first determination module 520, the second determination module 530, the calibration module 540 and the fusion module 550 may be at least partially implemented as a computer program module, and when the computer program module is executed, the corresponding function may be executed.

[0126] Figure 6 A block diagram of an electronic device suitable for implementing a method for rapid three-dimensional imaging of space targets based on active and passive combination according to an embodiment of the present invention is shown.

[0127] Figure 6 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0128] like Figure 6 As shown, a computer electronic device 600 according to an embodiment of the present invention includes a processor 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage part 608 into a random access memory (RAM) 603. The processor 601 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 601 may also include an onboard memory for caching purposes. The processor 601 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0129] In RAM 603, various programs and data required for the operation of electronic device 600 are stored. Processor 601, ROM 602 and RAM 603 are connected to each other via bus 604. Processor 601 performs various operations of the method flow according to the embodiment of the present invention by executing the programs in ROM 602 and / or RAM 603. It should be noted that the program can also be stored in one or more memories other than ROM 602 and RAM 603. Processor 601 can also perform various operations of the method flow according to the embodiment of the present invention by executing the programs stored in one or more memories.

[0130] Optionally, the electronic device 600 may further include an input / output (I / O) interface 605, which is also connected to the bus 604. The electronic device 600 may further include one or more of the following components connected to the input / output (I / O) interface 605: an input portion 606 including a keyboard, a mouse, etc.; an output portion 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 608 including a hard disk, etc.; and a communication portion 609 including a network interface card such as a LAN card, a modem, etc. The communication portion 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the input / output (I / O) interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed, so that a computer program read therefrom is installed into the storage portion 608 as needed.

[0131] Optionally, the method flow according to the embodiment of the present invention can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 609, and / or installed from the removable medium 611. When the computer program is executed by the processor 601, the above-mentioned functions defined in the system of the embodiment of the present invention are executed. Optionally, the system, equipment, device, module, unit, etc. described above can be implemented by a computer program module.

[0132] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the target three-dimensional image generation method according to the embodiment of the present invention is implemented.

[0133] Optionally, the computer-readable storage medium may be a non-volatile computer-readable storage medium. For example, it may include, but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, apparatus, or device.

[0134] For example, optionally, the computer-readable storage medium may include the ROM 602 and / or the RAM 603 described above and / or one or more memories other than the ROM 602 and the RAM 603 .

[0135] An embodiment of the present invention also includes a computer program product, which includes a computer program, which contains program code for executing the method provided by the embodiment of the present invention. When the computer program product runs on an electronic device, the program code is used to enable the electronic device to implement the method for rapid three-dimensional imaging of space targets based on active and passive combination provided by the embodiment of the present invention.

[0136] When the computer program is executed by the processor 601, the above functions defined in the system / device of the embodiment of the present invention are performed. Optionally, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0137] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 609, and / or installed from a removable medium 611. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0138] Optionally, the program code for executing the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages, specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, such as Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0139] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram may represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions. It can be understood by those skilled in the art that the features recorded in the various embodiments of the present disclosure can be combined and / or combined in a variety of ways, even if such a combination or combination is not explicitly recorded in the present disclosure. In particular, without departing from the spirit and teaching of the present disclosure, the features described in the various embodiments of the present disclosure may be combined and / or combined in a variety of ways. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0140] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A method for rapid three-dimensional imaging of space targets based on active and passive combination, characterized in that: The method comprises: Based on the target observation line of sight direction, an imaging device is used to collect images of the target object to obtain a vertical image and a horizontal image, wherein the imaging device includes an optical camera and a synthetic aperture laser radar, the vertical image is obtained based on the optical camera and passive optical technology, and the plane where the vertical image is located is perpendicular to the target observation line of sight direction, and the horizontal image is obtained based on the synthetic aperture laser radar and active technology, and the plane where the horizontal image is located is parallel to the target observation line of sight direction; Determining first relative motion information between the target object and the optical camera; Determining second relative motion information between the target object and the synthetic aperture laser radar; According to the first relative motion information and the second relative motion information, the vertical image and the horizontal image are calibrated to obtain a target vertical image and a target horizontal image respectively; Based on the association information between the target vertical image and the target horizontal image, the target vertical image and the target horizontal image are orthogonally fused to obtain a target three-dimensional image.

2. The method according to claim 1, characterized in that The second relative motion information includes relative distance information; According to the first relative motion information and the second relative motion information, the vertical image and the horizontal image are calibrated respectively to obtain a target vertical image and a target horizontal image, which includes: Obtaining a first dimension size and a second dimension size of the target object according to the field of view angle parameter of the optical camera and the relative distance information; Based on the first dimension size and the second dimension size, the vertical image is calibrated to obtain a target vertical image.

3. The method according to claim 2, characterized in that The first relative motion information includes relative angle change information; Wherein, according to the first relative motion information and the second relative motion information, respectively calibrating the size of the vertical image and the horizontal image to obtain a target vertical image and a target horizontal image further comprises: Obtaining the first dimension size and the third dimension size of the target object according to the relative angle change information and the relative distance information; Based on the first dimension and the third dimension, the horizontal image is calibrated to obtain a target horizontal image.

4. The method according to claim 3, characterized in that: Obtaining the first dimension size and the third dimension size of the target object according to the relative angle change information and the relative distance information includes: determining Doppler information of the horizontal image; Obtaining relative vector velocity information of the target object according to the relative angle change information and the Doppler information; The first dimension size and the third dimension size of the target object are obtained according to the relative vector velocity information and the relative distance information.

5. The method according to claim 3, characterized in that: The target vertical image is composed based on the first dimension information and the second dimension information of the target object, the target horizontal image is composed based on the first dimension information and the third dimension information of the target object, and the association information between the target vertical image and the target horizontal image includes the first dimension information; Wherein, based on the association information between the target vertical image and the target horizontal image, orthogonally fusing the target vertical image and the target horizontal image to obtain the target three-dimensional image comprises: Based on the first dimensional information, coordinate matching is performed on pixel points in the target vertical image and pixel points in the target horizontal image to obtain an initial three-dimensional image; The initial three-dimensional image is processed using a distortion correction algorithm to obtain the target three-dimensional image.

6. The method according to claim 5, characterized in that Processing the initial three-dimensional image using a distortion correction algorithm to obtain the target three-dimensional image includes: Performing shape correction on the initial three-dimensional image to obtain an intermediate three-dimensional image; Texture correction is performed on the intermediate three-dimensional image to obtain the target three-dimensional image.

7. A space target rapid three-dimensional imaging system based on active and passive combination, characterized in that: The system comprises: A collection module, used for collecting images of a target object using an imaging device based on a target observation sight line direction, to obtain a vertical image and a horizontal image, wherein the imaging device includes an optical camera and a synthetic aperture laser radar, the vertical image is obtained based on the optical camera and passive optical technology, and the plane where the vertical image is located is perpendicular to the target observation sight line direction, and the horizontal image is obtained based on the synthetic aperture laser radar and active technology, and the plane where the horizontal image is located is parallel to the target observation sight line direction; A first determination module, used to determine first relative motion information between the target object and the optical camera; A second determination module, used to determine second relative motion information between the target object and the synthetic aperture laser radar; A calibration module, configured to calibrate the size of the vertical image and the horizontal image according to the first relative motion information and the second relative motion information, respectively, to obtain a target vertical image and a target horizontal image; The fusion module is used to perform orthogonal fusion on the target vertical image and the target horizontal image based on the association information between the target vertical image and the target horizontal image to obtain a target three-dimensional image.

8. An electronic device comprising: one or more processors; a memory for storing one or more programs, When one or more programs are executed by one or more processors, the one or more processors implement the method of any one of claims 1 to 6.

9. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to implement the method of any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images