A real-time multi-view three-dimensional imaging system and imaging method

Abstract

The application discloses a kind of real-time multi-view three-dimensional imaging system and imaging method, solve the problem that the cost of obtaining different view three-dimensional information is too high, and system is not enough integrated.Specifically including the first beam splitter, second beam splitter, third beam splitter and fourth beam splitter of rectangular structure distribution, first encoding plate is arranged between second beam splitter and fourth beam splitter, second encoding plate is arranged between third beam splitter and fourth beam splitter, in addition still including light source, first imaging lens, second imaging lens, stripe camera and processing unit;The application adopts three beam splitters and a fourth beam splitter to make that stripe camera can obtain the 3D information of different view in single time, and the first encoding plate and second encoding plate of transmission type are used to encode the 3D information of different view respectively, simplify system structure, beneficial to system integration, while also simplify the energy loss of DMD encoding device, reduce signal detection cost.

Description

Technical Field

[0001] This invention relates to a real-time imaging device, specifically to a real-time multi-view three-dimensional imaging system and imaging method. Background Technology

[0002] Striking laser lidar is an important means of acquiring three-dimensional information about targets. Due to its high temporal resolution, low noise, and large dynamic range, it is widely used in underwater detection, helicopter-borne lidar, unmanned aerial vehicles, terrain and geomorphological exploration, ocean exploration, and military defense. However, traditional striking cameras only have one-dimensional imaging capabilities. How to obtain more data in a single acquisition using striking cameras has always been an important research direction for striking camera-based lidar.

[0003] The conventional method for achieving 2D imaging is to combine it with rotating mirror scanning. While this can obtain more data, it cannot obtain complete 3D depth information and also significantly sacrifices the imageable depth range. On the other hand, the image resolution based on fiber optic conversion is extremely limited, usually less than 25*25.

[0004] Currently, the two-dimensional imaging technology of stripe cameras based on compressed sensing (compressed ultrafast imaging technology, CUP) has made it possible for stripe cameras to obtain high-resolution three-dimensional information of targets in a single shot. Through encoding and decoding, stripe cameras can achieve true real-time three-dimensional imaging.

[0005] However, with the development of technologies such as multi-target 3D detection and autonomous driving, the demand for multi-view 3D imaging is becoming increasingly strong. Currently, a single 3D detection device cannot acquire 3D information from different perspectives in real time. Achieving this through the accumulation of detection devices increases the cost of signal detection and also hinders system integration. Summary of the Invention

[0006] The purpose of this invention is to provide a real-time multi-view three-dimensional imaging system and imaging method to solve the technical problem that in the prior art, in order to obtain three-dimensional information from different perspectives in real time, it is necessary to accumulate detection devices, which not only increases the cost of signal detection, but also hinders system integration.

[0007] To achieve the above objectives, the present invention provides a real-time multi-view three-dimensional imaging system, which is characterized by including a first beam splitter, a second beam splitter, a third beam splitter, a fourth beam splitter, a light source, a first imaging lens, a second imaging lens, a stripe camera, a first encoding board, a second encoding board, and a processing unit.

[0008] The first, second, third, and fourth beam splitters are arranged in a rectangular structure along the optical path, with the first and fourth beam splitters being diagonally opposite each other; the first encoding plate is disposed on the optical path between the second and fourth beam splitters; the second encoding plate is disposed on the optical path between the third and fourth beam splitters; the second encoding plate is independent of the first encoding plate.

[0009] The light source emits a first beam, and a first beam splitter is placed in the optical path of the first beam to split the first beam into a second beam and a third beam; the second beam is reflected by the second beam splitter to form a fourth beam.

[0010] The first imaging lens is positioned in the optical path of the fourth beam; the fourth beam passes through the first imaging lens and reaches the first target, and is then reflected by the first target to form the sixth beam; the sixth beam passes sequentially through the first imaging lens, the second beam splitter, and the first encoding plate to reach the fourth beam splitter; the third beam is reflected by the third beam splitter to form the fifth beam.

[0011] The second imaging lens is positioned in the optical path of the fifth beam; the fifth beam passes through the second imaging lens to reach the second target, and is reflected by the second target to form the seventh beam; the seventh beam passes sequentially through the second imaging lens, the third beam splitter, and the second encoding plate to reach the fourth beam splitter; the fourth beam splitter superimposes the sixth beam and the seventh beam to form the eighth beam.

[0012] The stripe camera is positioned in the optical path of the eighth beam.

[0013] The processing unit is connected to the output of the stripe camera and is used to construct a three-dimensional image based on the data from the stripe camera.

[0014] Furthermore, it also includes a light homogenizer;

[0015] The light homogenizer is positioned in the optical path between the light source and the first beam splitter.

[0016] Furthermore, it also includes the first f system;

[0017] The first f system is set in the optical path between the fourth beam splitter and the stripe camera, with its incident end facing the fourth beam splitter and its exit end facing the stripe camera.

[0018] Furthermore, it also includes CCD cameras;

[0019] The second beam splitter further splits the sixth beam into a ninth beam; the ninth beam is incident on the first beam splitter.

[0020] The third beam splitter further splits the seventh beam into a tenth beam; the tenth beam is incident on the first beam splitter; the first beam splitter superimposes the ninth beam and the tenth beam to form an eleventh beam;

[0021] The CCD camera is positioned in the output light path of the eleventh beam; the output end of the CCD camera is connected to the processing unit.

[0022] The processing unit constructs a three-dimensional image based on data from the stripe camera and the CCD camera.

[0023] Furthermore, it also includes a second f system;

[0024] The second f system is set in the optical path between the CCD camera and the first beam splitter, with its incident end facing the first beam splitter and its exit end facing the CCD camera.

[0025] Furthermore, the light source is a picosecond laser with a wavelength of nm.

[0026] Furthermore, both the first and second encoding boards have a resolution of * and are made of chrome-plated quartz glass.

[0027] Furthermore, the focal lengths of both the first f system and the second f system are greater than mm.

[0028] Meanwhile, the present invention also provides a real-time multi-view 3D imaging method, based on the above-mentioned real-time multi-view 3D imaging system, which is characterized by including the following steps:

[0029] Step 1: The light source emits the first beam of light;

[0030] Step 2: The first beam splitter splits the first beam into a second beam and a third beam;

[0031] Step 3: The second beam splitter receives the second beam and reflects it to form a fourth beam; the fourth beam passes through the first imaging lens to reach the first target, and is reflected by the first target to form a sixth beam; the sixth beam passes through the first imaging lens, the second beam splitter, and the first encoding plate in sequence to reach the fourth beam splitter.

[0032] The third beam splitter receives the third beam and reflects it to form the fifth beam; the fifth beam passes through the second imaging lens to reach the second target, and is reflected by the second target to form the seventh beam; the seventh beam passes through the second imaging lens, the third beam splitter, and the second encoding plate in sequence to reach the fourth beam splitter.

[0033] Step 4: The fourth beam splitter superimposes the sixth beam and the seventh beam to form the eighth beam, and the streak camera acquires the eighth beam;

[0034] Step 5: Construct a three-dimensional image based on the data acquired by the stripe camera.

[0035] Furthermore, in step 3, the second beam splitter further splits the sixth beam into a ninth beam; the ninth beam is incident on the first beam splitter.

[0036] Furthermore, the third beam splitter also splits the seventh beam into a tenth beam; the tenth beam is incident on the first beam splitter.

[0037] Step 4 further includes: the first beam splitter superimposes the ninth beam and the tenth beam to form the eleventh beam, and the CCD camera acquires the eleventh beam;

[0038] Step 5 specifically involves constructing a three-dimensional image based on the data acquired by the stripe camera and the CCD camera.

[0039] The beneficial effects of this invention are:

[0040] 1. The real-time multi-view 3D imaging system provided by the present invention uses three beam splitters and one fourth beam splitter to enable the streak camera to obtain 3D information from different viewpoints in a single operation. The system uses a transmissive first encoding board and a second encoding board to encode the 3D information from different viewpoints, which simplifies the system structure, facilitates system integration, and also simplifies the energy loss of the DMD encoding device and reduces the cost of signal detection.

[0041] 2. The present invention also includes a CCD camera to acquire two-dimensional aliasing information from different perspectives, which assists the reconstruction algorithm and improves the image reconstruction quality.

[0042] 3. The three-dimensional imaging system of the present invention can realize low-bandwidth, low-cost multi-view real-time three-dimensional imaging. That is, as the solution algorithm capability improves, the viewpoint can be expanded to multiple, and combined with miniaturized stripe tubes, it can be used in more scenarios, such as multi-angle three-dimensional ranging for autonomous driving, thereby reducing the commercial cost of three-dimensional detection technology.

[0043] 4. This invention employs a beam homogenizer. The first beam emitted by the light source can generate uniformly expanded beam light after passing through the beam homogenizer (ED) for use by each beam splitter, thereby improving the quality of the constructed image. Attached Figure Description

[0044] Figure 1 This is a schematic diagram of the structure of an embodiment of the real-time multi-view three-dimensional imaging system of the present invention;

[0045] Figure 2 This is an optical path diagram of an embodiment of a real-time multi-view three-dimensional imaging system according to the present invention.

[0046] Icon labels:

[0047] 1-First beam splitter, 2-Second beam splitter, 3-Third beam splitter, 4-Fourth beam splitter, 5-Light source, 6-First imaging lens, 7-Second imaging lens, 8-Stripe camera, 9-First encoding plate, 10-Second encoding plate, 11-Light homogenizer, 12-First 4F system, 13-CCD camera, 14-Second 4F system, 15-First target, 16-Second target;

[0048] 01-First beam, 02-Second beam, 03-Third beam, 04-Fourth beam, 05-Fifth beam, 06-Sixth beam, 07-Seventh beam, 08-Eighth beam, 09-Ninth beam, 010-Tenth beam, 011-Eleventh beam. Detailed Implementation

[0049] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0050] This invention extends the spatial three-dimensional information aliasing capability and provides a real-time multi-view three-dimensional imaging system. In the system design, to ensure the imaging efficiency of the lidar, this invention proposes a compact MV-Lidar system. Traditional compressed ultrafast imaging systems often employ DMD encoding methods, which are not only complex but also inefficient. The system in this invention uses transmissive encoding to process information from different viewpoints separately, simplifying the system structure and reducing the laser energy loss of the DMD encoding device. Furthermore, an external CDD camera acquires two-dimensional aliasing information from different viewpoints to assist the reconstruction algorithm. The specific structure is as follows:

[0051] like Figure 1 As shown, the three-dimensional imaging system includes a first beam splitter 1, a second beam splitter 2, a third beam splitter 3, a fourth beam splitter 4, a light source 5, a first imaging lens 6, a second imaging lens 7, a stripe camera 8, a first encoding plate 9, a second encoding plate 10, a light homogenizer 11, a first 4f system 12, a CCD camera 13, a second 4f system 14, and a processing unit.

[0052] The first beam splitter 1, the second beam splitter 2, the third beam splitter 3, and the fourth beam splitter 4 are arranged in a rectangular structure, with the first beam splitter 1 and the fourth beam splitter 4 being diagonally opposite each other, and the second beam splitter 2 and the third beam splitter 3 being diagonally opposite each other. The first encoding plate 9 is set in the optical path between the second beam splitter 2 and the fourth beam splitter 4; the second encoding plate 10 is set in the optical path between the third beam splitter 3 and the fourth beam splitter 4. The resolution of both the first encoding plate 9 and the second encoding plate 10 is 250*250, and their material is chrome-plated quartz glass. It should be noted that the first encoding plate 9 and the second encoding plate 10 are not related. Specifically, the encoding of the first encoding plate 9 and the second encoding plate 10 adopts a pseudo-random distribution encoding mode of "0" and "1". "0" indicates that the pixel is chrome-plated, and "1" indicates that the pixel is not chrome-plated. The first encoding plate 9 and the second encoding plate 10 each adopt two pseudo-random encoding modes. A light source 5 is positioned on the side of the first beam splitter 1 away from the second beam splitter 2, with its active end facing the first beam splitter 1. This light source 5 is a picosecond laser with a wavelength of 532nm. A beam homogenizer 11 is positioned in the optical path between the light source 5 and the first beam splitter 1 to uniformly expand the first beam 01 emitted by the light source. A first imaging lens 6 is positioned in the optical path on the side of the second beam splitter 2 away from the fourth beam splitter 4. A second imaging lens 7 is positioned in the optical path on the side of the third beam splitter 3 away from the fourth beam splitter 4. A streak camera 8 is positioned in the optical path on the side of the fourth beam splitter 4 away from the third imaging lens 7, with its active end facing the fourth beam splitter 4. A first 4f system 12 is positioned in the optical path between the fourth beam splitter 4 and the streak camera 8, with its incident end facing the fourth beam splitter 4 and its exit end facing the streak camera 8. A CCD camera 13 is positioned in the optical path on the side of the first beam splitter 1 away from the third beam splitter 3. The second 4f system 14 is positioned in the optical path between the CCD camera 13 and the first beam splitter 1, with its incident end facing the first beam splitter 1 and its exit end facing the CCD camera 13. The focal lengths of both the first 4f system 12 and the second 4f system 14 are greater than 100mm. The processing unit is connected to the output ends of the stripe camera 8 and the CCD camera 13, respectively, and is used to construct a three-dimensional image based on the data output by the stripe camera 8 and the CCD camera 13. A picosecond laser emits pulsed light in the 532nm band. This light first passes through an optical diffuser (ED) to generate uniformly expanded beam light. The expanded beam light is split into two identical beams by the first beam splitter 1, and then each beam passes through a beam splitter (the second beam splitter 2 and the third beam splitter 3) and an imaging lens (the first imaging lens 6 and the second imaging lens 7) to reach the surfaces of the first target 15 and the second target 16 at viewing angles 1 and 2, respectively. Subsequently, the first imaging lens 6 and the second imaging lens 7 acquire the returned light signals, which are then returned to the corresponding beam splitters (the second beam splitter 2 and the third beam splitter 3) and split into two beams.Two beams (the sixth beam 06 and the seventh beam 07) are superimposed at the fourth beam splitter 4 and acquired by the streak camera, while the other two beams (the ninth beam 09 and the tenth beam 010) are superimposed at the first beam splitter 1 and acquired by the CCD camera 13. Finally, the processing unit constructs a three-dimensional image based on the data acquired by the streak camera 8 and the CCD camera 13, resulting in significantly improved image quality. Compressed ultrafast imaging technology combines encoding with a streak camera. The acquisition process mainly consists of three steps: encoding operator T, scanning voltage deflection operator S, and CCD time-domain superposition operator C. Establishing a spatial coordinate system, the acquired signals from different encodings can be represented as T1SCI1(x,y,t) and T2SCI2(x,y,t), where (x,y) represents the spatial coordinates and t represents the time coordinates. Assuming the operation operators T1SC = A1, T2SC = A2, and ∫I1(x,y,t)dt = A. C I1,∫I2(x,y,t)dt=A C I2; the two sets of measurement signals are M sc (Striped camera) and M ccd (CCD camera) The original 3D signals I1 and I2 can be solved using the following optimization formula:

[0053]

[0054] Where TV(*) represents the total variation function, and this optimization problem can be solved by compressed sensing algorithms, such as the two-step soft thresholding algorithm (TwIst) and the generalized projection algorithm (GAP-TV).

[0055] The imaging method of the aforementioned real-time multi-view 3D imaging system specifically includes the following steps:

[0056] Step 1: Light source 5 emits first beam 01, and first beam 01 is uniformly expanded by a beam homogenizer;

[0057] Step 2: The first beam splitter 1 splits the first beam 01 into the second beam 02 and the third beam 03;

[0058] Step 3: The second beam splitter 2 receives the second beam 02 and reflects it to form the fourth beam 04; the fourth beam 04 passes through the first imaging lens 6 and reaches the first target 15, and is reflected by the first target 15 to form the sixth beam 06; the sixth beam 06 passes through the first imaging lens 6, the second beam splitter 2, and the first encoding plate 9 in sequence to reach the fourth beam splitter 4; at the same time, the second beam splitter 2 also splits the sixth beam 06 into the ninth beam 09; the ninth beam 09 is incident on the first beam splitter 1;

[0059] The third beam splitter 3 receives the third beam 03 and reflects it to form the fifth beam 05; the fifth beam 05 passes through the second imaging lens 7 and reaches the second target 16, and is reflected by the second target 16 to form the seventh beam 07; the seventh beam 07 passes through the second imaging lens 7, the third beam splitter 3, and the second encoding plate 10 in sequence to reach the fourth beam splitter 4. At the same time, the third beam splitter 3 also splits the seventh beam 07 into the tenth beam 010; the tenth beam 010 is incident on the first beam splitter 1.

[0060] Step 4: The fourth beam splitter 4 superimposes the sixth beam 06 and the seventh beam 07 to form the eighth beam 08. The eighth beam 08 is acquired by the stripe camera 8 after passing through the first and second 4f system 12. The first beam splitter 1 superimposes the ninth beam 09 and the tenth beam 010 to form the eleventh beam 011. The eleventh beam 011 is acquired by the CCD camera 13 after passing through the second 4f system 14.

[0061] Step 5: The processing unit constructs a 3D image based on the data acquired by the stripe camera 8 and the CCD camera 13.

[0062] This invention expands the spatial 3D information aliasing capability by proposing a single-shot, multi-view 3D imaging scheme. It achieves 3D imaging of targets from multiple perspectives through the encoding and aliasing of 3D target information from different orientations using a single streak camera, and through the separation of information using compressed sensing algorithms. This method reconstructs the 3D information of the target by combining photons arriving at different times captured by compressed ultrafast imaging technology. Furthermore, it distinguishes information from different perspectives by independently encoding 3D information from different scenes. This method enables real-time 3D detection from multiple orientations and reduces hardware costs for 3D perception in the field of autonomous driving.

[0063] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present invention should be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A real-time multi-view three-dimensional imaging system, characterized by: The system comprises a first beam splitter (1), a second beam splitter (2), a third beam splitter (3), a fourth beam splitter (4), a light source (5), a first imaging lens (6), a second imaging lens (7), a stripe camera (8), a first encoding plate (9), a second encoding plate (10) and a processing unit. The first beam splitter (1), the second beam splitter (2), the third beam splitter (3) and the fourth beam splitter (4) are arranged in a rectangular structure along the light path, and the first beam splitter (1) and the fourth beam splitter (4) are diagonally opposite to each other; the first encoding plate (9) is arranged on the light path between the second beam splitter (2) and the fourth beam splitter (4); the second encoding plate (10) is arranged on the light path between the third beam splitter (3) and the fourth beam splitter (4); the second encoding plate (10) is not correlated with the first encoding plate (9). The light source (5) emits a first light beam (01), and the first beam splitter (1) is arranged on the light path of the first light beam (01) to divide the first light beam (01) into a second light beam (02) and a third light beam (03); the second light beam (02) is reflected by the second beam splitter (2) to form a fourth light beam (04); The first imaging lens (6) is arranged on the light path of the fourth light beam (04); the fourth light beam (04) passes through the first imaging lens (6) to reach a first target (15), and then is reflected by the first target (15) to form a sixth light beam (06); the sixth light beam (06) passes through the first imaging lens (6), the second beam splitter (2) and the first encoding plate (9) in sequence to reach the fourth beam splitter (4); the third light beam (03) is reflected by the third beam splitter (3) to form a fifth light beam (05); The second imaging lens (7) is arranged on the light path of the fifth light beam (05); the fifth light beam (05) passes through the second imaging lens (7) to reach a second target (16), and then is reflected by the second target (16) to form a seventh light beam (07); the seventh light beam (07) passes through the second imaging lens (7), the third beam splitter (3) and the second encoding plate (10) in sequence to reach the fourth beam splitter (4); the fourth beam splitter (4) superimposes the sixth light beam (06) and the seventh light beam (07) to form an eighth light beam (08); The stripe camera (8) is arranged on the light path of the eighth light beam (08); The processing unit is connected with the output end of the stripe camera (8) and is used for constructing a three-dimensional image according to the data of the stripe camera (8).

2. The real-time multi-view three-dimensional imaging system according to claim 1, further comprising a light homogenizing plate (11). The light homogenizing plate (11) is arranged on the light path between the light source (5) and the first beam splitter (1).

3. The real-time multi-view three-dimensional imaging system according to claim 1 or 2, further comprising a first 4f system (12). The first 4f system (12) is arranged on the light path between the fourth beam splitter (4) and the stripe camera (8), and the incident end of the first 4f system (12) is opposite to the fourth beam splitter (4), and the exit end of the first 4f system (12) is opposite to the stripe camera (8).

4. The real-time multi-view three-dimensional imaging system according to claim 3. ​ ​ The CCD camera (13) is further included; The second beam splitter (2) further splits the sixth light beam (06) into a ninth light beam (09); the ninth light beam (09) is incident to the first beam splitter (1); The third beam splitter (3) further splits the seventh light beam (07) into a tenth light beam (010); the tenth light beam (010) is incident to the first beam splitter (1); the first beam splitter (1) superimposes the ninth light beam (09) and the tenth light beam (010) to form an eleventh light beam (011); The CCD camera (13) is arranged on the light path of the eleventh light beam (011); the output end of the CCD camera (13) is connected with the processing unit; The processing unit constructs a three-dimensional image according to the data of the stripe camera (8) and the CCD camera (13).

5. The real-time multi-view three-dimensional imaging system according to claim 4, further comprising a second 4f system (14); The second 4f system (14) is arranged on the light path between the CCD camera (13) and the first beam splitter (1), the incident end of the second 4f system (14) is opposite to the first beam splitter (1), and the outgoing end of the second 4f system (14) is opposite to the CCD camera (13). The light source (5) is a picosecond laser, and the wavelength is 532 nm.

6. The real-time multi-view three-dimensional imaging system of claim 5, wherein:

7. The real-time multi-view three-dimensional imaging system according to claim 6, wherein: The resolution of the first encoding plate (9) and the second encoding plate (10) is 250*250, and the material of the first encoding plate (9) and the second encoding plate (10) is chromium-plated quartz glass.

8. The real-time multi-view three-dimensional imaging system according to claim 7, wherein: The focal length of the first 4f system (12) and the second 4f system (14) is greater than 100 mm. The steps include:

9. A real-time multi-view three-dimensional imaging method based on the real-time multi-view three-dimensional imaging system according to any one of claims 1-8, characterized in that, Step 1, the light source (5) emits a first light beam (01); Step 2, the first beam splitter (1) splits the first light beam (01) into a second light beam (02) and a third light beam (03); Step 3, the second beam splitter (2) receives the second light beam (02) and reflects it to form a fourth light beam (04); the fourth light beam (04) passes through the first imaging lens (6) to reach the first target (15), is reflected by the first target (15) to form a sixth light beam (06); the sixth light beam (06) passes through the first imaging lens (6), the second beam splitter (2) and the first encoding plate (9) in sequence to reach the fourth beam splitter (4); The third beam splitter (3) receives the third light beam (03) and reflects it to form a fifth light beam (05); the fifth light beam (05) passes through the second imaging lens (7) to reach the second target (16), is reflected by the second target (16) to form a seventh light beam (07); the seventh light beam (07) passes through the second imaging lens (7), the third beam splitter (3) and the second encoding plate (10) in sequence to reach the fourth beam splitter (4); Step 4, the fourth beam splitter (4) superimposes the sixth light beam (06) and the seventh light beam (07) to form an eighth light beam (08), and the stripe camera (8) acquires the eighth light beam (08); Step 5, a three-dimensional image is constructed according to the data acquired by the stripe camera (8). ​ 10. The method of claim 9, wherein: In step 3, the second beam splitter (2) also splits the sixth light beam (06) into a ninth light beam (09); the ninth light beam (09) is incident to the first beam splitter (1); And the third beam splitter (3) also splits the seventh light beam (07) into a tenth light beam (010); the tenth light beam (010) is incident to the first beam splitter (1); Then step 4 further comprises: the first beam splitter (1) superimposes the ninth light beam (09) and the tenth light beam (010) to form an eleventh light beam (011), and a CCD camera (13) acquires the eleventh light beam (011); Then step 5 is specifically: constructing a three-dimensional image according to the data acquired by the stripe camera (8) and the CCD camera (13).

Images