System for realizing fusion of lidar point cloud and image

Abstract

The present application relates to the technical field of laser radar depth information and image information fusion, and particularly relates to a system capable of realizing laser radar point cloud and image fusion, a laser emission module emits laser and processes the laser, a beam splitter transmits the laser emitted by the laser emission module, and reflects the laser reflected from a measured object and visible light to an optical receiving module, a beam control module operates the laser transmitted from the beam splitter, the optical receiving module focuses the laser and visible light reflected from the beam control module to a photosensitive chip module to realize imaging, and the photosensitive chip module comprises a column of linear array laser photosensitive chips and a column of linear array visible light photosensitive chips arranged in parallel.Compared with the prior art, the system capable of realizing laser radar point cloud and image fusion simplifies the system structure, improves the stability of the system, reduces the hardware cost, reduces the complexity of system calibration, and reduces the difficulty of image fusion due to the coincidence of detection information.

Description

[Technical Field]

[0001] This invention relates to the field of fusion technology of lidar depth information and image information, and particularly to a system that can realize the fusion of lidar point cloud and image. [Background Technology]

[0002] LiDAR can detect the three-dimensional information of an object and generate a depth point cloud map of the object. The image information of the object provides feature information such as texture and color, which is convenient for identification. By fusing the depth information of the LiDAR with the feature information of the image, a color three-dimensional image is formed, which is beneficial for the recognition, processing and application of three-dimensional images.

[0003] Existing fusion solutions use two independent component systems: LiDAR and camera. This results in complex structural design, high stability requirements, and the need to calibrate both the LiDAR and camera image systems before acquiring point clouds and images of the object under test. The image fusion algorithm is complex, and the two systems also increase costs. [Summary of the Invention]

[0004] To overcome the above problems, this invention proposes a system that can effectively solve the above problems and realize the fusion of lidar point clouds and images.

[0005] The present invention provides a technical solution to solve the above-mentioned technical problems: a system for fusing lidar point clouds and images, comprising a light emitting module, a beam splitter, a beam manipulation module, an optical receiving module, and a photosensitive chip module. The laser emitting module is used to emit laser light and process the emitted laser beam. The beam splitter is used to transmit the laser light emitted by the laser emitting module and reflect the laser light and visible light reflected from the object being measured back to the optical receiving module. The beam manipulation module is used to operate the laser light transmitted through the beam splitter. The optical receiving module is used to focus the laser light and visible light reflected from the beam manipulation module onto the photosensitive chip module to achieve imaging. The photosensitive chip module includes a linear array of laser photosensitive chips and a linear array of visible light photosensitive chips, which are arranged in parallel.

[0006] Preferably, the laser photosensitive chip and the visible light photosensitive chip are flush, and both the laser photosensitive chip and the visible light photosensitive chip are rectangular chips of the same size.

[0007] Preferably, the distance between adjacent sides of the laser photosensitive chip and the visible light photosensitive chip is D1, and the distance between adjacent sides of two adjacent laser photosensitive chips or two adjacent visible light photosensitive chips is D2.

[0008] Preferably, D1 and D2 are both 0.

[0009] Preferably, D1 and D2 are 20-30 micrometers.

[0010] Preferably, the laser photosensitive chip includes a laser photosensitive pixel unit, which is used to receive laser light and convert the laser light into an electrical signal; the visible light photosensitive chip includes a visible light photosensitive pixel unit, which is used to receive visible light and convert the visible light into an electrical signal.

[0011] Preferably, the number of laser photosensitive chips and visible light photosensitive chips are equal.

[0012] Preferably, the laser emitting module, the beam splitter, and the beam control module are located on the same straight line, the beam splitter is located between the laser emitting module and the beam control module, the beam splitter, the optical receiving module, and the photosensitive chip module are located on the same straight line, and the optical receiving module is located between the beam splitter and the photosensitive chip module.

[0013] Preferably, the laser emitting module processes the laser beam by collimating or expanding the laser beam.

[0014] Preferably, the beam manipulation module operates the laser by reflecting the emitted laser onto an object for detection or by rotating and reflecting the emitted laser onto an object to achieve scanning and detection of the object, as well as reflecting the laser and visible light reflected from the detected object onto a beam splitter.

[0015] Compared with existing technologies, the system of the present invention that can realize the fusion of lidar point cloud and image integrates lidar and image detection components into one system, which simplifies the system structure, improves the system stability, and reduces the hardware cost of lidar point cloud and image fusion. The linear array laser photosensitive chip and the visible light photosensitive chip are arranged in parallel and share optical receiving components, which reduces the complexity of system calibration and the overlap of detection information reduces the difficulty of image fusion. [Attached Image Description]

[0016] Figure 1 This is a structural framework diagram of the system of the present invention that can realize the fusion of lidar point cloud and image;

[0017] Figure 2 This is a structural diagram of the photosensitive chip module of the system of the present invention that can realize the fusion of lidar point cloud and image.

Detailed Implementation Methods

[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the invention.

[0019] It should be noted that in the embodiments of the present invention, all directional indications (such as up, down, left, right, front, back, etc.) are limited to relative positions on the specified view, rather than absolute positions.

[0020] Furthermore, in this invention, descriptions involving "first," "second," etc., are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0021] Please see Figure 1 and Figure 2 The present invention provides a system for fusing lidar point clouds and images, comprising a laser emitting module, a beam splitter, a beam control module, an optical receiving module, and a photosensitive chip module. The laser emitting module is used to emit laser light and process the emitted laser beam. The beam splitter is used to transmit the laser light emitted by the laser emitting module and reflect the laser light and visible light reflected from the object being measured back to the optical receiving module. The beam control module is used to manipulate the laser light transmitted through the beam splitter. The optical receiving module is used to focus the laser light and visible light reflected from the beam control module onto the photosensitive chip module to achieve imaging.

[0022] The photosensitive chip module includes a linear array of laser photosensitive chips and a linear array of visible light photosensitive chips, which are arranged in parallel.

[0023] The laser photosensitive chip includes a laser photosensitive pixel unit, which is used to receive laser light and convert the laser light into an electrical signal.

[0024] The visible light photosensitive chip includes a visible light photosensitive pixel unit, which is used to receive visible light and convert the visible light into an electrical signal.

[0025] The laser emitting module processes the laser beam by performing processes such as collimation and beam expansion.

[0026] The beam manipulation module operates the laser by reflecting the emitted laser onto an object for detection or by rotating and reflecting the emitted laser onto an object to achieve scanning and detection of the object, as well as reflecting the laser and visible light reflected from the detected object onto a beam splitter.

[0027] In this embodiment, the linear array of laser photosensitive chips forms a 1*10 laser photosensitive chip array, and the linear array of visible light photosensitive chips forms a 1*10 visible light photosensitive chip array, both packaged on the same substrate. The number of laser photosensitive chips and visible light photosensitive chips can be more than 10, and can be increased or decreased according to actual needs. The number of laser photosensitive chips and visible light photosensitive chips is equal. During operation, the scanning interval of the emitted laser is controlled by the beam manipulation module, so that the object currently being scanned by the laser coincides with the object detected by visible light at the previous scan position; thus, except for the first and last scan positions, the same measured position can undergo one laser scan detection and one visible light detection, and the image information detected by the laser photosensitive chip and the visible light photosensitive chip can overlap.

[0028] The laser photosensitive chip and the visible light photosensitive chip are arranged in parallel and are aligned with each other. Both the laser photosensitive chip and the visible light photosensitive chip are rectangular chips of the same size.

[0029] The laser photosensitive chip can be a photodiode, avalanche photodiode, etc., to detect laser light. The visible light photosensitive chip can be a CMOS or CCD to receive visible light reflected or generated by the object being measured and generate an image of the object.

[0030] The distance between adjacent sides of the laser photosensitive chip and the visible light photosensitive chip is D1, and the distance between adjacent sides of two adjacent laser photosensitive chips or two adjacent visible light photosensitive chips is D2. D1 and D2 are determined according to different packaging processes. Theoretically, D1 and D2 being 0 is optimal, allowing for a more compact structure and improved light energy utilization. D1 and D2 are preferably 20-30 micrometers, which satisfies process requirements, controls process costs, and maintains a relatively compact structure while retaining high light energy utilization.

[0031] In this embodiment, the laser emitting module, the beam splitter, and the beam control module are located on the same straight line. The beam splitter is located between the laser emitting module and the beam control module. The beam splitter, the optical receiving module, and the photosensitive chip module are located on the same straight line. The optical receiving module is located between the beam splitter and the photosensitive chip module.

[0032] Compared with existing technologies, the system of the present invention that can realize the fusion of lidar point cloud and image integrates lidar and image detection components into one system, which simplifies the system structure, improves the system stability, and reduces the hardware cost of lidar point cloud and image fusion. The linear array laser photosensitive chip and the visible light photosensitive chip are arranged in parallel and share optical receiving components, which reduces the complexity of system calibration and the overlap of detection information reduces the difficulty of image fusion.

[0033] The above description is only a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any modifications, equivalent substitutions and improvements made within the concept of the present invention should be included within the patent protection scope of the present invention.

Claims

1. A system capable of fusing lidar point clouds with images, characterized in that, The system includes a laser emitting module, a beam splitter, a beam control module, an optical receiving module, and a photosensitive chip module. The laser emitting module emits a laser beam and processes it. The beam splitter transmits the laser emitted by the laser emitting module and reflects the laser and visible light reflected from the object being measured back to the optical receiving module. The beam control module manipulates the laser transmitted through the beam splitter. The optical receiving module focuses the laser and visible light reflected from the beam control module onto the photosensitive chip module to achieve imaging. The photosensitive chip module includes a linear array of laser photosensitive chips and a linear array of visible light photosensitive chips. The laser photosensitive chips and the visible light photosensitive chips are arranged in parallel and share an optical receiving component. The laser photosensitive chip and the visible light photosensitive chip are aligned with each other, and both the laser photosensitive chip and the visible light photosensitive chip are rectangular chips of the same size; The distance between adjacent sides of the laser photosensitive chip and the visible light photosensitive chip is D1, and the distance between adjacent sides of two adjacent laser photosensitive chips or two adjacent visible light photosensitive chips is D2. The D1 and D2 are 0 or the D1 and D2 are 20-30 micrometers; The number of laser photosensitive chips and visible light photosensitive chips are equal; During operation, the beam control module controls the scanning interval of the emitted laser, so that the object currently being scanned by the laser coincides with the object detected by visible light at the previous scan position. Thus, except for the first and last scan positions, the same measured position can be scanned by laser and detected by visible light once, and the image information detected by the laser photosensitive chip and the visible light photosensitive chip can overlap. The laser photosensitive chip includes a laser photosensitive pixel unit, which is used to receive laser light and convert it into an electrical signal; the visible light photosensitive chip includes a visible light photosensitive pixel unit, which is used to receive visible light and convert it into an electrical signal. The laser emitting module, beam splitter, and beam control module are located on the same straight line. The beam splitter is located between the laser emitting module and the beam control module. The beam splitter, optical receiving module, and photosensitive chip module are located on the same straight line. The optical receiving module is located between the beam splitter and the photosensitive chip module. The laser emitting module processes the laser beam by collimating or expanding it. The beam manipulation module operates the laser by reflecting the emitted laser onto an object for detection or by rotating and reflecting the emitted laser onto an object to achieve scanning and detection of the object, as well as reflecting the laser and visible light reflected from the detected object onto a beam splitter.

Images