Solid-state laser radar based on iTOF and dTOF point cloud fusion

Abstract

The utility model relates to the field of laser radars, in particular to a solid-state laser radar based on iTOF and dTOF point cloud fusion, which is characterized in that different working wavelength sums are set to ensure that an iTOF laser radar module and a dTOF laser radar module do not interfere with each other when working at the same time; through setting a control and data processing module, work of a first light source and a second light source is controlled, imaging data of an iTOF detector and a dTOF detector are received, a structured light algorithm is operated, multi-source data fusion is carried out, and finally point cloud data in a space environment is obtained. The distance measurement information of the dTOF laser radar module is adopted to assist in eliminating the multipath crosstalk problem of the iTOF laser radar; a low-emissivity target area which cannot be seen clearly by the iTOF laser radar is filled through distance measurement information of the dTOF laser radar module; the problems that an iTOF laser radar has multi-path interference and poor sensitivity, and when the target reflectivity is low or the distance is long, the ranging performance is sharply reduced are solved.

Description

Technical Field

[0001] This utility model relates to the field of lidar technology, and in particular to a solid-state lidar based on iTOF and dTOF point cloud fusion. Background Technology

[0002] Currently, mass-produced iTOF array detectors can achieve a resolution of 640x480 with pixel sizes within 10µm, but they do not use detectors with single-photon sensitivity. In contrast, mass-produced dTOF single-photon array detectors, if they achieve tens of thousands of pixels, are expensive and consume a lot of power. Currently, dTOF single-photon array detectors with assisted autofocus, which are already widely used in mobile applications, generally have a resolution of 40x30. The multipath interference problem in iTOF has long plagued the industry. Although more than a decade of image processing algorithm development has improved the quality of iTOF point clouds, the elimination of multipath interference has not been significantly improved. Furthermore, the sensitivity of iTOF detectors is far lower than that of single-photon detection arrays, resulting in a sharp decline in iTOF ranging performance when dealing with low-reflectivity targets or at slightly longer working distances.

[0003] Based on this, a solid-state lidar based on iTOF and dTOF point cloud fusion is proposed. Utility Model Content

[0004] To address the issues of multipath interference, poor sensitivity, and the sharp decline in ranging performance when the target has low reflectivity or is at a long distance, iTOF lidar is being developed.

[0005] This invention provides a solid-state lidar based on iTOF and dTOF point cloud fusion, including an iTOF lidar module, a dTOF lidar module, a beam splitter, and a control and data processing module. The iTOF lidar module includes a first transmitter and a first detector. The first transmitter includes a first light source, which emits light with a center wavelength of [missing information]. The illumination light is provided by a first detection end comprising a first lens and an iTOF detector; the dTOF lidar module comprises a second transmitter and a second detector, the second transmitter comprising a second light source, the second light source being used to emit light with a center wavelength of... The lighting light, Not equal to The second detection end includes a second lens and a dTOF detector; the beam splitter is used to receive the illumination light reflected by the target, and to divide the light with a center wavelength of... The illumination light is reflected to the first detector, with a center wavelength of The illumination light is directly transmitted to the second detector end; the control and data processing module is used to control the operation of the first and second light sources, receive the imaging data of the iTOF detector and dTOF detector, run the structured light algorithm, perform multi-source data fusion, and finally obtain point cloud data in the space environment.

[0006] Preferably, the focal length of the first lens Focal length of the second lens The ratio of the size of the iTOF detector to the size of the iTOF detector and the size of the dTOF detector The ratio is directly proportional, that is k is a positive number.

[0007] Preferably, both the iTOF detector and the dTOF detector employ a single-photon detector array, with the dTOF detector integrating a TDC timing unit.

[0008] Preferably, the field of view of each 16x16 pixel sub-region of the iTOF detector matches the field of view of each pixel of the dTOF detector.

[0009] Preferably, the control and data processing module includes a timing control interface, a data communication interface, and a central processing unit. The timing control interface is electrically connected to the first light source, the second light source, the iTOF detector, and the dTOF detector, and the data communication interface is electrically connected to the iTOF detector and the dTOF detector.

[0010] Preferably, a first narrowband bandpass filter is disposed between the first lens and the iTOF detector, and a second narrowband bandpass filter is disposed between the second lens and the dTOF detector. The center wavelength of the first narrowband bandpass filter is [missing information]. The center wavelength of the second narrowband bandpass filter lens is .

[0011] The beneficial effects of this invention are reflected in the setting of different working wavelengths. and This system ensures that the iTOF and dTOF lidar modules operate simultaneously without interference. By setting up a control and data processing module, it controls the operation of the first and second light sources, receives imaging data from the iTOF and dTOF detectors, and runs a structured light algorithm to perform multi-source data fusion, ultimately obtaining point cloud data of the space environment. The ranging information from the dTOF lidar module is used to help eliminate multipath crosstalk issues in the iTOF lidar. The ranging information from the dTOF lidar module is also used to fill in low-emissivity target areas that the iTOF lidar cannot see clearly. This solves the problems of multipath interference, poor sensitivity, and a sharp decline in ranging performance when the target reflectivity is low or the distance is far, inherent in iTOF lidar. Furthermore, it can utilize the large-area array information of the iTOF lidar module to achieve super-resolution point cloud reconstruction, solving the problem that dTOF lidar cannot achieve high-resolution point clouds. Attached Figure Description

[0012] Figure 1 This is a schematic diagram of the structure of a lidar based on iTOF and dTOF point cloud fusion provided by this utility model.

[0013] Figure 2 This is a schematic diagram of the field-of-view matching of the iTOF detector array and the dTOF detector array provided by this utility model.

[0014] In the diagram: 1-iTOF lidar module; 11-first light source; 12-first lens; 13-iTOF detector; 14-first narrowband bandpass filter lens; 2-dTOF lidar module; 21-second light source; 22-second lens; 23-dTOF detector; 24-second narrowband bandpass filter lens; 3-beam splitter prism; 4-control and data processing module. Detailed Implementation

[0015] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0016] Reference Figures 1-2 A solid-state lidar based on iTOF and dTOF point cloud fusion includes an iTOF lidar module 1, a dTOF lidar module 2, a beam splitter prism 3, and a control and data processing module 4. The iTOF lidar module 1 includes a first transmitter and a first detector. The first transmitter includes a first light source 11, which emits light with a center wavelength of... The illumination light is provided by the first detection end, which includes a first lens 12 and an iTOF detector 13; the dTOF lidar module 2 includes a second transmitter and a second detector, the second transmitter including a second light source 21, which is used to emit a center wavelength. For illumination light, Not equal to The second detection end includes a second lens 22 and a dTOF detector 23; the beam splitter 3 is used to receive the illumination light reflected by the target and to split the light with a center wavelength of... The illumination light is reflected to the first detector, with a center wavelength of The illumination light is directly transmitted to the second detection end; the control and data processing module 4 is used to control the operation of the first light source 11 and the second light source 21, receive the imaging data of the iTOF detector 13 and the dTOF detector 23 and run the structured light algorithm to perform multi-source data fusion, and finally obtain point cloud data in the space environment.

[0017] Set different operating wavelengths and This system ensures that the iTOF lidar module 1 and dTOF lidar module 2 operate simultaneously without interference. A control and data processing module 4 controls the operation of the first light source 11 and the second light source 21, receives imaging data from the iTOF detector 13 and the dTOF detector 23, and runs a structured light algorithm to perform multi-source data fusion, ultimately obtaining point cloud data of the space environment. The ranging information from the dTOF lidar module 2 is used to help eliminate multipath crosstalk in the iTOF lidar. The ranging information from the dTOF lidar module 2 is used to fill in low-emissivity target areas that the iTOF lidar cannot see. This solves the problems of multipath interference, poor sensitivity, and a sharp decline in ranging performance when the target reflectivity is low or the distance is far, inherent in iTOF lidar. Furthermore, the large-area array information from the iTOF lidar module 1 can be used to achieve super-resolution point cloud reconstruction, solving the problem that dTOF lidar cannot achieve high-resolution point clouds.

[0018] In some embodiments, the focal length of the first lens 12 The focal length of the second lens 22 The ratio is related to the size of the iTOF detector 13. and the size of the dTOF detector 23 The ratio is directly proportional, that is, k is a positive number.

[0019] By adjusting the focal length of the first lens 12 The focal length of the second lens 22 The ratio is used to match the size of the iTOF detector 13. and the size of the dTOF detector 23 This ensures that the receiving field of view positions and sizes of the iTOF detector 13 and dTOF detector 23 are matched at the same imaging distance, which helps to maintain consistency when fusing data acquired by the two technologies. The iTOF detector 13 is more suitable for high-resolution imaging at close range, while the dTOF detector 23 is more suitable for ranging at long distance. By adjusting the focal length ratio, the resolution and ranging capabilities of the two can be optimally balanced at different distances.

[0020] For example, when K is 1, the array formed by the iTOF detector 13 is 640 x 480 with a pixel size of 3.5µm, and the array formed by the dTOF detector 23 is 40 x 30 with a pixel size of 20µm. Then the first lens 12 corresponds to a focal length... The focal length corresponding to the second lens 22 ratio Need equals .

[0021] The specific algorithm is as follows:

[0022] The horizontal dimensions of the iTOF detector 13 are 640 × 3.5 μm = 2.24 mm.

[0023] The vertical dimensions of the iTOF detector 13 are 480 × 3.5 μm = 1.68 mm.

[0024] The size of the iTOF detector 13 ;

[0025] The horizontal dimensions of the dTOF detector 23 are 40 × 20 μm = 0.8 mm;

[0026] The vertical dimension of the dTOF detector 23 is 30 × 20 μm = 0.6 mm;

[0027] Size of dTOF detector 23 ;

[0028] ;

[0029] Preferably, both the iTOF detector 13 and the dTOF detector 23 employ a single-photon detector array, and the dTOF detector 23 integrates a TDC timing unit.

[0030] The single-photon detector array can detect signals at the single-photon level, significantly improving the sensitivity of the iTOF detector 13 and dTOF detector 23. This high sensitivity is essential in low-light conditions or long-distance measurements, especially when dealing with low-reflectivity targets. The TDC unit provides extremely high-precision time measurement, accurately measuring the round-trip time of the light pulse, thereby improving ranging accuracy. This high precision is crucial for applications requiring accurate distance measurement, such as autonomous vehicles and robot navigation.

[0031] More preferably, the field of view of each 16x16 pixel sub-region of the iTOF detector 13 matches the field of view of each pixel of the dTOF detector 23.

[0032] Ensure consistency in the field of view angle between iTOF detector 13 and dTOF detector 23. Within the same observation area, iTOF detector 13 and dTOF detector 23 can capture an area of ​​the same size, facilitating subsequent data fusion and processing.

[0033] In some embodiments, the control and data processing module 4 includes a timing control interface, a data communication interface, and a central processing unit. The timing control interface is electrically connected to the first light source 11, the second light source 21, the iTOF detector 13, and the dTOF detector 23, and the data communication interface is electrically connected to the iTOF detector 13 and the dTOF detector 23.

[0034] Precise time synchronization is achieved through the timing control interface, and high-speed, reliable data transmission is achieved through the data communication interface. This ensures that the data collected by the iTOF detector 13 and the dTOF detector 23 can be transmitted to the central processing unit for processing in a timely manner.

[0035] In some embodiments, a first narrowband bandpass filter 14 is disposed between the first lens 12 and the iTOF detector 13, and a second narrowband bandpass filter 24 is disposed between the second lens 22 and the dTOF detector 23. The center wavelength of the first narrowband bandpass filter 14 is [missing information]. The center wavelength of the second narrowband bandpass filter lens 24 is .

[0036] By setting a first narrowband bandpass filter 14 and a second narrowband bandpass filter 24 with different center wavelengths, crosstalk between different wavelengths is avoided, ensuring that the iTOF detector 13 and dTOF detector 23 only receive their corresponding light source signals, effectively reducing the influence of ambient light and improving the signal-to-noise ratio of the signal.

[0037] Specific working methods:

[0038] a) Handling iTOF ranging anomalies caused by multipath interference: such as Figure 2 As shown, the field of view of each 16x16 pixel sub-region of the iTOF detector 13 matches the field of view of each pixel of the dTOF detector 23. The dTOF detector 23 uses a single-photon detector, such as a SPAD, and integrates a TDC timing unit. For example, the distances detected by pixels 1 and 2 of the dTOF detector 23 are d1 and d2, respectively, and d1 equals d2; the average distances detected by pixel sub-regions 1 and 2 of the iTOF detector 13 are D1 and D2, respectively. D1 is close to d1, but differs greatly from D2. Therefore, the data in sub-region 2 corresponding to D2 needs to be processed in detail, and d2 is used as input information to start the corresponding data processing algorithm. The advantage of this is that targeted post-processing guided by dTOF ranging values ​​can be carried out on the region of most interest (ROI) to eliminate ranging deviations introduced by iTOF during the measurement process. ROI-based data processing greatly reduces the amount of computation and power consumption compared to processing the entire frame point cloud.

[0039] b) Handling low ranging confidence of iTOF due to low emissivity targets: In addition to generating depth information, the iTOF detector 13 also generates grayscale images. Typically, iTOF systems use 850nm or 940nm illumination sources, and the corresponding grayscale images are IR images. If the ranging confidence of D1 and D2 and the corresponding IR image intensity value confidence are both lower than the preset threshold, d1 and d2 are directly activated to fill the ranging information of the corresponding pixel area. If only the ranging confidence of D1 and D2 is lower than the preset threshold, point cloud reconstruction or super-resolution point cloud reconstruction can be performed by combining d1 and IR images to fill the ranging information of the corresponding pixel area.

[0040] c) Super-resolution point cloud reconstruction: For sub-pixel regions with high confidence levels in both iTOF and dTOF ranging and IR image intensity values, point clouds with resolutions exceeding 16x16 (e.g., 32x32) can be reconstructed using appropriate super-resolution algorithms. For sub-pixel regions with high confidence levels in both dTOF ranging and IR image intensity, or only high confidence levels in dTOF ranging, the goal is to maximize the 16x16 ranging confidence level, without considering super-resolution reconstruction.

[0041] d) Multi-source image information fusion: Compared with point cloud generation by a single-photon dTOF detector array with a resolution of 640x480, the multi-source image fusion scheme of 640x480 iTOF + 640x480 IR image + 40x30 dTOF is more conducive to dealing with complex and changing target scenes, and can improve the adaptability of ranging and the confidence of point cloud.

[0042] This application proposes fusing point clouds obtained from iTOF and dTOF ranging technologies to achieve complementary advantages of the two technologies. The main focus is on leveraging the high spatial resolution advantage of iTOF's large-area array, while using a small-area dTOF single-photon detector array to compensate for iTOF's multipath ranging problem and difficulty in detecting low-reflectivity targets. Ultimately, this enables a solid-state lidar that integrates iTOF IR image output.

[0043] A solid-state LiDAR system combining mature, mass-produced iTOF and dTOF LiDAR technologies fully leverages the advantages of both approaches to achieve higher 3D perception performance. Furthermore, multi-source image information fusion utilizes computing power and algorithms to achieve higher imaging metrics, rather than solely relying on hardware upgrades.

[0044] Although embodiments of the present invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the present invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A solid-state lidar based on iTOF and dTOF point cloud fusion, characterized in that: include: The iTOF lidar module includes a first transmitter and a first detector. The first transmitter includes a first light source, which emits light with a center wavelength of [missing information]. The illumination light, the first detection end includes a first lens and an iTOF detector; The dTOF lidar module includes a second transmitter and a second detector. The second transmitter includes a second light source, which emits light with a center wavelength of [missing information]. The lighting light, Not equal to The second detection end includes a second lens and a dTOF detector; A beam splitter is used to receive illumination light reflected from a target and to split the light with a center wavelength of... The illumination light is reflected to the first detector, with a center wavelength of The illumination light is directly transmitted to the second detection end; The control and data processing module is used to control the operation of the first and second light sources, receive imaging data from the iTOF detector and dTOF detector, run the structured light algorithm, perform multi-source data fusion, and finally obtain point cloud data in the space environment.

2. A solid-state lidar based on iTOF and dTOF point cloud fusion as described in claim 1, characterized in that: Focal length of the first lens Focal length of the second lens The ratio of the size of the iTOF detector to the size of the iTOF detector and the size of the dTOF detector The ratio is directly proportional, that is k is a positive number.

3. A solid-state lidar based on iTOF and dTOF point cloud fusion according to claim 2, characterized in that: Both the iTOF and dTOF detectors use single-photon detector arrays, with the dTOF detector integrating a TDC timing unit.

4. A solid-state lidar based on iTOF and dTOF point cloud fusion according to claim 3, characterized in that: The field of view of each 16x16 pixel sub-region of the iTOF detector matches the field of view of each pixel of the dTOF detector.

5. A solid-state lidar based on iTOF and dTOF point cloud fusion according to claim 1, characterized in that: The control and data processing module includes a timing control interface, a data communication interface, and a central processing unit. The timing control interface is electrically connected to the first light source, the second light source, the iTOF detector, and the dTOF detector. The data communication interface is electrically connected to the iTOF detector and the dTOF detector.

6. A solid-state lidar based on iTOF and dTOF point cloud fusion according to claim 1, characterized in that: A first narrowband bandpass filter is disposed between the first lens and the iTOF detector, and a second narrowband bandpass filter is disposed between the second lens and the dTOF detector. The center wavelength of the first narrowband bandpass filter is [missing information]. The center wavelength of the second narrowband bandpass filter lens is .

Images