Vehicle-mounted LiDAR, processing methods, systems and vehicles
By using different types of laser detectors for data fusion processing in vehicle-mounted LiDAR, the problems of insufficient detection data and false target detection are solved, and the matching of detection data with intelligent driving is realized, thereby improving the safety and reliability of intelligent driving.
Patent Information
- Application Number
- CN202611080575.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-20
- Publication Date
- 2026-11-17
AI Technical Summary
Existing vehicle-mounted LiDAR systems suffer from insufficient detection data or a high probability of false target detection, resulting in a mismatch between the perception requirements of intelligent driving systems.
Data fusion processing is performed using different types of laser detectors (such as SPAD and SIPM), including pixel fusion and waveform fusion, leveraging their complementary advantages to eliminate ghosting points and ensure the accuracy of the real target.
By ensuring a sufficient amount of detection data, the probability of false target detection is reduced, thereby improving the safety and reliability of intelligent driving.
Smart Images

Figure CN123112866A_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of automotive technology, and more particularly to an onboard lidar, processing method, system, and vehicle. Background Technology
[0002] Current automotive LiDAR typically employs either Single Photon Avalanche Diode (SPAD) or Silicon Photomultiplier (SIPM) LiDAR. For SPAD LiDAR, due to its limited array size and multi-channel concurrent transmission and detection method, multiple receiving channels are covered within a single transmission zone. Strong echo signals from high-reflectivity targets not only trigger the avalanche effect in the corresponding pixel but also cause optical and electrical crosstalk between adjacent pixels, generating crosstalk echo signals. This results in high-reflectivity ghosting in the point cloud, leading to a higher probability of false detections. As for SIPM LiDAR, its limited array size results in insufficient point count. This insufficiency makes it difficult for target detection algorithms to acquire enough observation data, limiting the effective output of target detection data and consequently affecting the reliable perception of small targets at long distances by intelligent driving systems. Therefore, existing vehicle-mounted LiDAR systems suffer from insufficient data collection or a high probability of false target detection, resulting in a mismatch between the detection data and the actual needs of intelligent driving. Summary of the Invention
[0003] This disclosure provides an in-vehicle lidar, processing method, system, and vehicle that can reduce the probability of false target detection while ensuring sufficient detection data, thereby improving the matching degree between detection data and the actual needs of intelligent driving and enhancing the safety and reliability of intelligent driving.
[0004] To achieve the above objectives, according to a first aspect of this disclosure, a vehicle-mounted lidar is provided, comprising: The first laser detector is used to collect the first detection data; The second laser detector is used to collect second detection data, wherein the first laser detector and the second laser detector are different types of detectors; The vehicle-mounted lidar sends the first detection data and the second detection data to the processor, so that the processor can perform fusion processing on the first detection data and the second detection data, the fusion processing including pixel fusion processing and / or waveform fusion processing.
[0005] In some embodiments, the first laser detector includes a first detector array, and the second laser detector includes a second detector array, wherein the first detector array and the second detector array have different detector types, and the number of first pixels corresponding to the first detector array is greater than the number of second pixels corresponding to the second detector array.
[0006] In some embodiments, the first laser detector and the second laser detector are integrated within the same housing and share an optical system; the vehicle-mounted lidar includes the processor, or the processor is disposed in the vehicle; or, The first laser detector and the second laser detector are both independent devices, and the processor is located in the vehicle.
[0007] In some embodiments, the processor is disposed in the vehicle, and the vehicle-mounted lidar further includes: The data transmission interface is communicatively connected to the first laser detector, the second laser detector, and the processor, respectively, and is used to transmit the first detection data and the second detection data to the processor.
[0008] In some embodiments, the first field of view of the first laser detector and the second field of view of the second laser detector partially or completely overlap. When the first field of view and the second field of view partially overlap, the first optical center direction of the first laser detector forms a first angle with the vehicle's direction of travel, and the second optical center direction of the second laser detector forms a second angle with the vehicle's direction of travel. The first optical center direction and the second optical center direction are located on the left and right sides of the vehicle's direction of travel, respectively.
[0009] According to a second aspect of this disclosure, a data processing method is provided, applied to an onboard lidar provided in the first aspect or a vehicle including the onboard lidar, the method comprising: The first detection data and the second detection data are fused together, wherein the fusion process includes pixel fusion processing and / or waveform fusion processing.
[0010] In some embodiments, the first detection data includes first point cloud data, the second detection data includes second point cloud data, and the fusion processing includes pixel fusion processing. The fusion processing of the first detection data and the second detection data to obtain target point cloud data includes: Based on a preset pixel matching relationship, the first sub-pixel in the first point cloud data and the second sub-pixel in the second point cloud data are matched to obtain at least one set of pixel pairs; For each pair of pixels, the distance value of the fused pixel corresponding to the pair of pixels is determined based on the difference between the first distance value of the first sub-pixel in the pair of pixels and the second distance value of the second sub-pixel in the pair of pixels. The target point cloud data is obtained based on the ranging value of the corresponding fused pixel points for each group of pixels.
[0011] In some embodiments, determining the pixel value of the fused pixel corresponding to each pixel pair based on the difference in the ranging values of that pixel pair includes: For each pair of pixels, if the difference in the measured values of the pair of pixels is less than a preset threshold, then the measured value of the fused pixel corresponding to the pair of pixels is the first measured value of the first sub-pixel in the pair of pixels; if the difference in the measured values of the pair of pixels is not less than the preset threshold, then the measured value of the fused pixel corresponding to the pair of pixels is a first set value.
[0012] In some embodiments, in the pixel matching relationship, one second sub-pixel point matches multiple first sub-pixel points.
[0013] In some embodiments, the first detection data includes first echo data, the second detection data includes second echo data, and the fusion processing includes waveform fusion processing. The fusion processing of the first and second detection data to obtain target point cloud data includes: The first echo data and the second echo data after time synchronization are subjected to waveform superposition or waveform weighted fusion processing to obtain fused waveform data; The ranging value of each fused pixel is determined based on the waveform of each fused pixel in the fused wave data and the preset detection threshold of each fused pixel; The target point cloud data is obtained based on the ranging value of each fused pixel.
[0014] In some embodiments, determining the ranging value of each fused pixel based on the waveform of each fused pixel in the fused waveform data and a preset detection threshold includes: For each fused pixel in the fused waveform data, if the maximum value of the waveform of the fused pixel is greater than the preset detection threshold of the fused pixel, then the ranging value of the fused pixel is the ranging value in the first echo data corresponding to the waveform of the fused pixel; if the maximum value of the waveform of the fused pixel is not greater than the preset detection threshold of the fused pixel, then the ranging value of the fused pixel is a second set value.
[0015] In some embodiments, the preset detection threshold of the fused pixel is the product of the detection threshold of the waveform of the fused pixel in the first echo data and a preset coefficient, wherein the preset coefficient is greater than 1.
[0016] In some embodiments, the method further includes: Based on the vehicle's operating condition information and / or environmental information, a fusion strategy is determined for fusing the first detection data and the second detection data.
[0017] In some embodiments, determining the fusion strategy for fusing the first detection data and the second detection data based on the vehicle's environmental information includes: If the environmental information determines that the vehicle is in a backlight or strong light scene, the first weight of the first detection data is reduced and the second weight of the second detection data is increased; and if the environmental information determines that the vehicle is in a low light or nighttime scene, the first weight is increased.
[0018] According to a third aspect of this disclosure, a data processing system is provided, comprising: The first laser detector is used to collect the first detection data; The second laser detector is used to collect second detection data, wherein the first laser detector and the second laser detector are different types of detectors; A processor, connected to the first laser detector and the second laser detector, is used to perform fusion processing on the first detection data and the second detection data to obtain target point cloud data. The fusion processing includes pixel fusion processing and / or waveform fusion processing.
[0019] According to a fourth aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps of the method as described in any of the second aspects.
[0020] According to a fifth aspect of this disclosure, a vehicle is provided, such as the vehicle-mounted lidar provided in the first aspect, or, performing any of the data processing methods provided in the second aspect, or including the data processing system provided in the third aspect.
[0021] According to a sixth aspect of this disclosure, a computer program product is provided, comprising a computer program or instructions that, when executed by a processor, implement the steps of any of the methods provided in the second aspect.
[0022] In summary, in this embodiment of the present disclosure, the vehicle-mounted LiDAR includes a first laser detector for collecting first detection data and a second laser detector for collecting second detection data, wherein the first laser detector and the second laser detector are detectors of different types; and the first detection data and the second detection data are sent to a processor so that the processor performs fusion processing on the first detection data and the second detection data, the fusion processing including pixel fusion processing and / or waveform fusion processing; by adopting the above technical solution, by setting different types of detectors, a high-line-count detector can ensure sufficient detection data, while a low-line-count detector provides a ghost-free reference benchmark. Pixel fusion and / or waveform fusion of the two sets of detection data can cross-verify and eliminate ghost points and ensure that real targets are not mistakenly deleted, thereby reducing the probability of false detection while ensuring sufficient detection data, making the fused detection data more compatible with the perception requirements of intelligent driving, and thus improving the safety and reliability of intelligent driving.
[0023] Other features and advantages of this disclosure will be described in detail in the following detailed description section. Attached Figure Description
[0024] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are merely some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.
[0025] To gain a more complete understanding of this disclosure and its beneficial effects, the following description will be made in conjunction with the accompanying drawings, wherein the same reference numerals denote the same parts in the following description.
[0026] Figure 1 This is a schematic diagram of a vehicle-mounted lidar provided in an exemplary embodiment of this disclosure; Figure 2 This is a schematic diagram of a vehicle-mounted lidar including a processor provided in an exemplary embodiment of this disclosure; Figure 3 This is a schematic diagram of a vehicle-mounted lidar including a data transmission interface provided in an exemplary embodiment of this disclosure; Figure 4 This is a schematic diagram of a vehicle-mounted lidar system comprising two independent lidars, provided in an exemplary embodiment of this disclosure. Figure 5 This is a schematic diagram of a structure in which two lidars are mounted on the roof of a vehicle, as provided in an exemplary embodiment of this disclosure. Figure 6a and Figure 6b This is a schematic diagram of a structure in an exemplary embodiment of the present disclosure where the first field of view and the second field of view partially overlap; Figure 7a and Figure 7b This is another structural schematic diagram showing the partial overlap of the first and second field of view provided in the exemplary embodiments of this disclosure; Figure 8 This is a schematic diagram of an embodiment of pixel fusion of first point cloud data and second point cloud data provided in the exemplary embodiments of this disclosure; Figure 9a This is a schematic diagram of a target scenario provided in an exemplary embodiment of this disclosure; Figure 9b This is a schematic diagram of the structure of the first point cloud data corresponding to the target scene provided in the exemplary embodiments of this disclosure; Figure 9c This is a schematic diagram of the structure of the second point cloud data corresponding to the target scene provided in the exemplary embodiments of this disclosure; Figure 9d This is a schematic diagram of the structure of target point cloud data corresponding to the target scene provided in the exemplary embodiments of this disclosure; Figure 10 This is a schematic diagram of an embodiment of waveform fusion processing of the first echo data and the second echo provided in the exemplary embodiments of this disclosure; Figure 11 This is a flowchart illustrating a data processing method provided in an exemplary embodiment of this disclosure; Figure 12 This is a system architecture diagram of the data processing system provided in the exemplary embodiments of this disclosure; Figure 13 This is a schematic diagram of the vehicle architecture provided in an exemplary embodiment of this disclosure. Detailed Implementation
[0027] The technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without creative effort are within the protection scope of this disclosure.
[0028] The vehicle-mounted LiDAR described in this specification can be applied to a vehicle, which can be any of the following: a hybrid vehicle, an electric vehicle, or a gasoline vehicle. This specification does not impose any specific limitations.
[0029] Please see Figure 1This disclosure provides an embodiment of a vehicle-mounted lidar, comprising a first laser detector 10 for collecting first detection data and a second laser detector 20 for collecting second detection data, wherein the first laser detector 10 and the second laser detector 20 are detectors of different types; and the vehicle-mounted lidar sends the first detection data and the second detection data to a processor 30, so that the processor 30 performs fusion processing on the first detection data and the second detection data, the fusion processing including pixel fusion processing and / or waveform fusion processing.
[0030] In some embodiments, the first laser detector 10 and the second laser detector 20 may be an integrated device or separate devices; this specification does not impose specific limitations.
[0031] In some embodiments, the first laser detector 10 can be any one of SPAD lidar, SIPM lidar, avalanche photodiode (APD) lidar, frequency-modulated continuous wave (FMCW) lidar, indirect time-of-flight (iTOF) lidar, etc.; correspondingly, the second laser detector 20 can be any one of SPAD lidar, SIPM lidar, APD lidar, FMCW lidar, and iTOF lidar, etc., and the first laser detector 10 and the second laser detector 20 are detectors of different types. For example, if the first laser detector 10 is a SPAD lidar, then the second laser detector 20 can be any one of SIPM lidar, APD lidar, FMCW lidar, and iTOF lidar, etc. Thus, since the first laser detector 10 and the second laser detector 20 are different types of detectors, their response characteristics to the same detection area differ. Each type of detector has its own advantages and disadvantages (for example, SPAD has the advantage of high line count but suffers from inter-channel crosstalk, SIPM has the advantage of no crosstalk but limited line count, APD has the advantage of high reflectivity measurement accuracy, and FMCW has the advantage of simultaneous ranging and velocity measurement). By utilizing these differences to fuse the two sets of detection data, the advantages of different types of detectors can be complemented, preserving their respective strengths while suppressing the inherent defects of a single type of detector. This reduces the probability of false detections while ensuring sufficient detection data, making the detection data more aligned with the actual needs of intelligent driving and improving the safety and reliability of intelligent driving.
[0032] In some embodiments, the first detection data is first point cloud data or first echo data, and the second detection data is second point cloud data or second echo data. The point cloud data is obtained by detecting the echo data. Thus, when the first detection data is first point cloud data and the second detection data is second point cloud data, the processor 30 can perform pixel fusion processing on the first and second point cloud data. This allows the processor to use two different types of detectors to independently measure the distance to the same spatial target. By comparing the consistency of the two sets of ranging results, it can determine whether a real target exists in the pixel. If the two sets of ranging results are consistent, the pixel is determined to be a real target and retained; otherwise, it is determined to be a ghost noise point and removed. Therefore, the above technical solution can remove ghost points and ensure that real targets are not mistakenly deleted. This reduces the probability of false detection while ensuring sufficient detection data, making the fused detection data more compatible with the perception requirements of intelligent driving, thereby improving the safety and reliability of intelligent driving.
[0033] Accordingly, when the first detection data is the first echo data and the second detection data is the second echo data, the processor 30 can perform echo fusion processing on the first echo data and the second echo data. Then, it can utilize the difference in response characteristics of two different types of detectors to the echo signal of the same target, enhance the signal strength of the real target by waveform superposition, and at the same time, prevent the ghost signal that has not been enhanced from being detected by increasing the detection threshold. Thus, the above technical solution can also eliminate ghost points and ensure that the real target is not mistakenly deleted. This can reduce the probability of false detection while ensuring that the amount of detection data is sufficient, making the fused detection data more compatible with the perception requirements of intelligent driving, thereby improving the safety and reliability of intelligent driving.
[0034] In some embodiments, the first laser detector 10 includes a first detector array, and the second laser detector 20 includes a second detector array, wherein the detector types of the first detector array and the second detector array are different, and the number of first pixels corresponding to the first detector array is greater than the number of second pixels corresponding to the second detector array.
[0035] Specifically, if the first laser detector 10 is a SPAD lidar, then the first laser detector 10 includes a SPAD array with a pixel count of, for example, 300 or 400 lines; if the second laser detector 20 is a SIPM lidar, then the second laser detector 20 includes a SIPM array with a pixel count of, for example, 100 or 80 lines. The SPAD array has a large pixel scale, enabling high line counts (e.g., 400 lines or even higher). However, because the SPAD array uses a few partitions and multiple channels for concurrent transmission and detection, multiple receiving channels are covered within a single transmission partition. Strong echo signals from high-reflectivity targets not only trigger the avalanche effect of the corresponding SPAD pixels but also cause optical and electrical crosstalk between adjacent SPAD pixels, generating crosstalk echo signals. These crosstalk echoes form high-reflectivity dilatation ghosts in the point cloud, leading to a higher probability of false target detection. Meanwhile, some SPAD LiDARs employ high-reflectivity dilatation filtering algorithms to suppress ghosting, actively filtering low-reflectivity targets surrounding high-reflectivity targets. However, in scenarios where low-reflectivity targets (such as pedestrians in black clothing or tires) exist around a real high-reflectivity target, this can lead to false filtering of the real target, resulting in a higher probability of false detections. Furthermore, SIPM arrays typically use point-to-point time-division multiplexing for transmission and reception, preventing adjacent channels from illuminating simultaneously at the receiver, thus avoiding crosstalk between channels at the source. However, the individual pixel size of a SIPM array is much larger than that of a SPAD array, limiting array size. Under current automotive LiDAR size constraints, achieving a high line count (typically <200 lines) is difficult, resulting in insufficient vertical resolution and a limited number of effective points for distant, small targets. Therefore, a single type of detector cannot simultaneously meet the requirements of both high line count and no crosstalk.
[0036] The present invention uses two different types of detectors, SPAD array and SIPM array, to achieve high line count detection to ensure sufficient detection data. The SIPM array is used as a verification benchmark because it has no crosstalk between channels. The processor 30 performs fusion processing to remove crosstalk ghosts and restore the real targets that were mistakenly deleted. This reduces the probability of false detection and false negative detection while ensuring sufficient detection data.
[0037] In some embodiments, the first laser detector 10 and the second laser detector 20 are not limited to SPAD lidar and SIPM lidar, but can also be at least two different types of lidar, such as SIPM lidar + SPAD lidar + APD lidar, SIPM lidar + SPAD lidar + FMCW lidar, APD lidar + FMCW lidar, and SIPM lidar + SPAD lidar + ITOF lidar. Different types of lidar have different advantages and disadvantages, and fusion can achieve more comprehensive performance complementarity. For example, the advantage of APD lidar is its high accuracy in reflectivity measurement, the advantage of FMCW lidar is that it can measure speed while measuring distance, providing an additional dimension of information, and the advantage of ITOF lidar is its high accuracy in short-range distance measurement.
[0038] In some embodiments, the first laser detector 10 and the second laser detector 20 are integrated within the same housing and share an optical system, and the vehicle-mounted LiDAR includes a processor 30, which is a hardware module within the vehicle-mounted LiDAR. Specifically, see [link to documentation]. Figure 2 The vehicle-mounted LiDAR includes a processor 30, which is connected to a first laser detector 10 and a second laser detector 20. The processor 30 acquires first detection data collected by the first laser detector 10 and second detection data collected by the second laser detector 20. The vehicle-mounted LiDAR also includes an optical system 40, which aligns the optical paths of the first laser detector 10 and the second laser detector 20. After fusing the first and second detection data, the processor 30 obtains target point cloud data and transmits it to the vehicle's domain controller 50, enabling the domain controller 50 to utilize the target point cloud data for intelligent driving. At this point, because the vehicle-mounted LiDAR includes a processor 30, the amount of target point cloud data after fusion processing is significantly reduced compared to the first and second detection data. This effectively reduces data transmission latency, improves the real-time performance of the acquired target point cloud data, and because the data transmission volume from the vehicle-mounted LiDAR to the domain controller 50 is small, the bandwidth requirement for transmission from the vehicle-mounted LiDAR to the domain controller 50 is low. Furthermore, since data fusion has been completed at the vehicle-mounted LiDAR end, the domain controller 50 can directly use the target point cloud data, resulting in low computing power requirements for the domain controller 50. Additionally, the internal assembly process of the vehicle-mounted LiDAR involves pixel intrinsic parameter alignment, requiring high production line control and high alignment accuracy.
[0039] In some embodiments, the processor 30 may be any of a field-programmable gate array (FPGA), a microcontroller unit (MCU), and a digital signal processor (DSP), and the domain controller 50 may be any of a vehicle's infotainment system, central processing unit, and intelligent driving domain controller, etc., without specific limitations in this specification.
[0040] In some embodiments, the first laser detector 10 and the second laser detector 20 are integrated within the same housing and share an optical system, and the processor 30 is disposed in the vehicle. In this case, the processor 30 can be a hardware module in the vehicle's domain controller, or it can be a separate hardware device in the vehicle. Specifically, see [link to documentation]. Figure 3 Taking processor 30 as a hardware module in domain controller 50 as an example, the vehicle-mounted LiDAR transmits the first detection data and the second detection data to processor 30. Processor 30 can utilize the abundant computing resources of domain controller 50 to fuse the first and second detection data. Due to the abundant computing resources of domain controller 50, complex fusion algorithm strategies can be deployed in processor 30 to further improve fusion processing performance. Furthermore, the vehicle-mounted LiDAR also includes an optical system 40 for aligning the optical paths of the first laser detector 10 and the second laser detector 20.
[0041] In some embodiments, such as Figure 3 As shown, the vehicle-mounted LiDAR also includes a data transmission interface 60, which is communicatively connected to the first laser detector 10, the second laser detector 20, and the processor 30, respectively, for transmitting the first and second detection data to the processor 30 via the domain controller. Specifically, the first laser detector 10 and the second laser detector 20 output the collected first and second detection data to the domain controller 50 via the data transmission interface 60, respectively, and then the processor 30 in the domain controller 50 performs fusion processing. In this way, the stability of the transmission of the first and second detection data can be ensured through the data transmission interface 60. The data transmission interface 60 can be any of a Universal Serial Bus (USB) interface, an Ethernet interface, or a Controller Area Network (CAN) interface.
[0042] In some embodiments, the first laser detector 10 and the second laser detector 20 are both independent devices, and the processor 30 is located in the vehicle. In this case, the processor 30 can be a hardware module in the vehicle's domain controller, or it can be a separate hardware device in the vehicle. Specifically, see [link to documentation]. Figure 4 The first laser detector 10 is an independent first lidar, and the second laser detector 20 is an independent second lidar. The processor 30 is a hardware module within the domain controller 50. The first laser detector 10 and the second laser detector 20 each output their first and second detection data to the processor 30 in the domain controller 50 through their respective data transmission interfaces. This dual lidar approach achieves hardware redundancy, strong environmental robustness, and is unaffected by human eye safety limitations, while also allowing for more flexible placement and pixel fusion schemes. Furthermore, since the first laser detector 10 and the second laser detector 20 are independent lidars, each has its own optical system and does not need to share one.
[0043] In some embodiments, both the first laser detector 10 and the second laser detector 20 include a transmitter and a receiver, wherein the transmitter is used to emit a detection laser and the receiver is used to receive detection data.
[0044] In some embodiments, the first field of view of the first laser detector 10 and the second field of view of the second laser detector 20 partially or completely overlap. When the first field of view and the second field of view partially overlap, the first optical center direction of the first laser detector 10 forms a first angle with the vehicle's direction of travel, and the second optical center direction of the second laser detector 20 forms a second angle with the vehicle's direction of travel. The first optical center direction and the second optical center direction are located on the left and right sides of the vehicle's direction of travel, respectively.
[0045] Specifically, when the first and second field of view angles completely overlap, the optical centers of both the first laser detector 10 and the second laser detector 20 are parallel to the vehicle's direction of travel (i.e., the x-direction). Both the first laser detector 10 and the second laser detector 20 are mounted on the top of the vehicle (e.g., in front of the roof), arranged side-by-side or along the vehicle's central axis. At this time, the field of view coverage of the first laser detector 10 and the second laser detector 20 is the same, all pixels correspond one-to-one, and the processor 30 performs fusion processing on all pixels. Thus, when the first and second field of view angles completely overlap, the two laser detectors synchronously detect the same spatial area, and the processor 30 can perform pixel fusion and / or waveform fusion processing on all pixels. This avoids target misses and false detections caused by inter-channel crosstalk throughout the entire field of view, ensuring detection reliability across the entire field of view.
[0046] Furthermore, when the first and second field of view partially overlap, the first optical center direction of the first laser detector 10 forms a first angle with the vehicle's direction of travel, and the second optical center direction of the second laser detector 20 forms a second angle with the vehicle's direction of travel. The first and second optical center directions are located to the left and right of the vehicle's direction of travel, respectively. For example, the first laser detector 10 can be mounted on the left side of the vehicle roof, with its optical center direction deflected to the left of the vehicle's direction of travel by a first angle (e.g., 30°); the second laser detector 20 can be mounted on the right side of the vehicle roof, with its optical center direction deflected to the right of the vehicle's direction of travel by a second angle (e.g., 30°), and the first and second angles are equal. Thus, the central regions of the horizontal field of view of the first laser detector 10 and the second laser detector 20 form an overlapping region, while non-overlapping regions are formed on both sides, thereby expanding the total horizontal field of view (e.g., from 120° for a single laser radar to 180°). Thus, when the first and second field of view partially overlap, the two laser detectors simultaneously detect the same target, expanding the total horizontal field of view of the vehicle-mounted LiDAR. This, in turn, extends the environmental perception coverage around the vehicle, facilitating the early detection of vehicles, pedestrians, or obstacles approaching from the side, and providing the intelligent driving system with more time for decision-making and planning. Furthermore, the processor 30 can fuse pixels within the overlapping area, using consistency verification of the two sets of detection data to remove ghost noise and restore mistakenly deleted real targets, thereby ensuring high detection reliability within the core area of interest in the vehicle's direction of travel.
[0047] It should be noted that the aforementioned 30° value is merely illustrative. This disclosure does not limit the specific angle values of the first and second included angles, as long as the first and second optical center directions are located on the left and right sides of the vehicle's travel direction, respectively. In practical applications, the specific angles of the first and second included angles can be flexibly configured according to the vehicle model, the size of the lidar's field of view, and perception requirements. For example, they can be any angle such as 10°, 20°, 30°, or 45°, and they can be equal or unequal. Furthermore, the aforementioned "left side of the roof" and "right side of the roof" are merely illustrative arrangements provided by this disclosure. Those skilled in the art will understand that the first laser detector 10 and the second laser detector 20 can also be arranged at any suitable detection location, such as the vehicle's front bumper, front grille, or the inside of the windshield, as long as their field of view angles completely or partially overlap. This disclosure does not limit the specific installation positions of the two laser detectors.
[0048] Taking a lidar system where the first and second field of view partially overlap, and the first laser detector 10 and the second laser detector 20 are independent, as an example, see [link to relevant documentation]. Figure 5The vehicle 70 has a first lidar 71 and a second lidar 72 mounted on its roof. The first lidar 71 has a first field of view, and the second lidar 72 has a second field of view. For example, see... Figure 6a The vehicle 70 is equipped with a first lidar 71 and a second lidar 72. The first field of view of the first lidar 71 is represented by a sector 11, and the second field of view of the second lidar 72 is represented by a sector 12. The overlapping portion 13 between sector 11 and sector 12 is the overlapping area of the first and second field of view. In this case, the overlapping area between the first and second field of view is relatively large. When the overlapping area between the first and second field of view is relatively small, see... Figure 6b The first field of view of the first lidar 71 is represented by sector 14, and the second field of view of the second lidar 72 is represented by sector 15. The overlapping portion 16 between sector 14 and sector 15 is the overlapping area of the first and second field of view. In this case, the overlapping area between the first and second field of view is small, which reduces the number of fused pixels and also reduces the corresponding computing power consumption. The pixels in the overlapping area between the first and second field of view can be fused using a fusion strategy to avoid target missed / false detection problems caused by crosstalk between channels. At the same time, the non-overlapping area between the first and second field of view itself has relatively low requirements for the lidar's missed / false detection and does not need to be fused. In this way, by installing the first lidar 71 and the second lidar 72 in an offset manner, the field of view coverage of the dual lidars is expanded (e.g., from 120° to 180°).
[0049] In some embodiments, taking a first field of view and a second field of view as examples, when the first field of view and the second field of view completely overlap, the coverage areas of the first field of view and the second field of view are specifically as follows: Figure 7a As shown. When the first and second field of view partially overlap, the coverage areas of the first and second field of view are specifically as follows. Figure 7b As shown, the coverage area of the first field of view is region 17, the coverage area of the second field of view is region 18, and the overlapping area between the first and second field of view is region 19.
[0050] In some embodiments, if the first detection data includes first point cloud data and the second detection data includes second point cloud data, the fusion process includes pixel fusion processing. In this case, based on a preset pixel matching relationship, the first sub-pixel in the first point cloud data and the second sub-pixel in the second point cloud data can be matched to obtain at least one set of pixel pairs. For each set of pixel pairs, the ranging value of the fused pixel corresponding to the set of pixel pairs is determined according to the ranging value difference between the first ranging value of the first sub-pixel in the set of pixel pairs and the second ranging value of the second sub-pixel in the set of pixel pairs. The target point cloud data is obtained according to the ranging value of the fused pixel corresponding to each set of pixel pairs.
[0051] In some embodiments, pixel matching relationships can be achieved through matching and intrinsic parameter calibration of the internal optomechanical structure of the vehicle-mounted LiDAR, or through extrinsic parameter calibration. Specifically, in a pixel matching relationship, one second sub-pixel matches multiple first sub-pixels.
[0052] Specifically, when the first laser detector 10 includes a SPAD array (400 lines of pixels) and the second laser detector 20 includes a SIPM array (100 lines of pixels), since the number of pixels in the SPAD array is higher than that in the SIPM array, one second sub-pixel is matched with four first sub-pixels. For example, taking pixels A-1 to A-400 in the first point cloud data and pixels B-1 to B-100 in the second point cloud data as an example, pixels A-1 to A-4 in the first point cloud data are matched with pixel B-1 in the second point cloud data as a group, pixels A5 to A-8 are matched with pixel B-2 as a group, and so on.
[0053] Furthermore, the aforementioned matching relationship of "one second sub-pixel matching multiple first sub-pixels" is merely illustrative, and in practical applications, the matching relationship is not limited to this. For example, in other embodiments, if the first laser detector 10 is a SIPM array (low line count) and the second laser detector 20 is a SPAD array (high line count), the first sub-pixel and the second sub-pixel can be matched according to other ratios. Those skilled in the art will understand that the preset pixel matching relationship can be flexibly configured and adjusted according to the pixel ratio of the first laser detector 10 and the second laser detector 20, the detector type, the installation position, and the specific application scenario, as long as a pixel correspondence between the two point cloud data can be established. This disclosure does not limit this.
[0054] In some embodiments, after acquiring each set of pixel pairs, for each set of pixel pairs, the processor 30 acquires the first ranging value of the first sub-pixel and the second ranging value of the second sub-pixel in the set of pixel pairs, and calculates the absolute value of the difference between the two. Then, based on the difference in ranging values, the ranging value of the fused pixel corresponding to the set of pixel pairs is determined.
[0055] Specifically, for each pair of pixels, if the difference in the measured values of the pair is less than a preset threshold, then the measured value of the fused pixel corresponding to the pair is the first measured value of the first sub-pixel in the pair (or the second measured value of the second sub-pixel, or a weighted average of the two). If the difference in the measured values of the pair is not less than the preset threshold, then the measured value of the fused pixel corresponding to the pair is a first set value. The first set value is a preset invalid value, which can be set according to actual needs, for example, it can be 0 or empty.
[0056] In some embodiments, the preset threshold can be calibrated and set according to the inherent ranging deviation of the first laser detector 10 and the second laser detector 20, the system noise level, and the required target detection sensitivity, and this disclosure does not limit it. For example, the preset threshold can be set to 0.2 meters, or it can be set to other values such as 0.1 meters, 0.3 meters, or 0.5 meters depending on the accuracy level of different detectors.
[0057] In some embodiments, target point cloud data is obtained based on the ranging values of the corresponding fused pixels for each group of pixels. Specifically, when the first and second field of view completely overlap, all pixels in the first and second point cloud data are fused, thus the target point cloud data can be obtained based on the ranging values of all fused pixels. When the first and second field of view partially overlap, pixels located in the overlapping area of the first and second point cloud data are fused, while pixels located in the non-overlapping area are not fused. Thus, the target point cloud data can be obtained based on the ranging values of all fused pixels in the overlapping area and the ranging values of pixels in the non-overlapping area.
[0058] Taking the complete overlap of the first and second field of view as an example, the first laser detector 10 includes a SPAD array (400 pixels), and the second laser detector 20 includes a SIPM array (100 pixels). Figure 8As shown, the pixel points in the first point cloud data are A-1 to A-400, and the pixel points in the second point cloud data are B-1 to B-100, wherein the pixel points A-1 to A-4 in the first point cloud data all match the pixel point B-1 in the second point cloud data; correspondingly, the pixel points A-5 to A-8 all match the pixel point B-2, and so on, until the pixel points A-397 to A-400 all match the pixel point B-100. Taking the matching of pixel points A-1 to A-4 all with the pixel point B-1 in the second point cloud data as an example, 4 sets of pixel point pairs are obtained as (A-1, B-1), (A-2, B-1), (A-3, B-1) and (A-4, B-1) respectively; correspondingly, each set of pixel point pairs can be acquired, and the fused pixel point corresponding to each set of pixel point pairs is denoted as C-1 to C-400, wherein (A-1, B-1) corresponds to C-1, (A-2, B-1) corresponds to C-2, and so on, until (A-400, B-100) corresponds to C-400.
[0059] Wherein, the SPAD array measures a first ranging value for each of the pixel points A-1 to A-400, and the SIPM array measures a second ranging value for each of the pixel points B-1 to B-100. In this way, after each set of pixel point pairs is acquired, the ranging value of the fused pixel point corresponding to each set of pixel point pairs can be acquired according to the ratio of the ranging value difference of each set of pixel point pairs to a preset threshold. Taking the fused pixel point C-1 corresponding to a set of pixel point pairs (A-1, B-1) as an example, wherein the ranging value of the pixel A-1 is D_A-1, the ranging value of the pixel B-1 is D_B-1, and the preset threshold is D_th, if |D_A-1-D_B-1|<D_th, the ranging value D_C-1 of the pixel C-1 is equal to D_A-1; if |D_A-1-D_B-1|≥D_th, the ranging value of the pixel C-1 is 0.
[0060] Specifically, in a high reflectance expansion scenario, the target reflectance is on the high side, leading to low-reflectivity noise point ghost images (false points) around the high-reflectivity target. Due to the dense arrangement of pixels in the SPAD array, crosstalk occurs between vertical channels. At this time, real target point clouds exist at pixels A-1 to A-4, while ghost noise points caused by crosstalk exist at pixels A-5 to A-8. Assume that the ranging values of pixels A-1 to A-8 are all D0, wherein pixels A-5 to A-8 are misidentified as targets by the LiDAR. In addition, since the SiPM array adopts time-sharing emission and the spacing between pixels is large, there is no vertical channel crosstalk, so that pixel B-1 is a real target point cloud, and pixel B-2 has no point cloud (NAN point), that is, the ranging value D_B-1 of pixel B-1 is approximately equal to D0. In this way, it can be determined that |D0-D_B-1| < D_th, and further determined that the ranging values of pixels C-1 to C-4 are all D0; and since the ranging value D_B-2 of pixel B-2 is 0, it can be determined that |D0 -0| > D_th, and further determined that the ranging values of pixels C-5 to C-8 are all 0. Since the ranging values of C-5 to C-8 are all 0, the ghost noise points existing at A-5 to A-8 can be avoided, and the false detection rate of the target is effectively reduced.
[0061] In some embodiments, in order to improve the accuracy of the ranging value of the fused pixel obtained as described above, the preset threshold is usually set to a value greater than the inherent error. The inherent deviation is pre-calibrated, and can be calibrated by testing the same target with the first laser detector 10 and the second laser detector 20.
[0062] In practical applications, under high-reflectivity dilation scenarios, SPAD LiDAR experiences crosstalk between vertical channels due to the dense pixel arrangement. For example, pixels A-1 to A-4 contain real target point clouds (high reflectivity), while pixels A-5 to A-8 contain ghost noise caused by crosstalk (low reflectivity), and pixels A-9 to A-12 contain real targets (low reflectivity). In this case, the high-reflectivity dilation filtering algorithm is activated, filtering out the low-reflectivity point clouds adjacent to the high-reflectivity pixels. This results in both the low-reflectivity ghost noise and the low-reflectivity real targets in pixels A-5 to A-12 being filtered out, leading to missed target detection in pixels A-9 to A-12. In contrast, SIPM LiDAR, due to time-division multiplexing and large pixel spacing, does not experience vertical channel crosstalk. Pixel B-1 contains a real target point cloud, pixel B-2 contains an empty point cloud (NAN point), and pixel B-3 contains a low-reflectivity point cloud; however, pixel B-3 only has one point, resulting in insufficient target detection points. Through the aforementioned pixel fusion processing, pixels C-1 to C-4 are identified as true high-reflectivity target point clouds, C-5 to C-8 as empty point clouds (NAN points), and pixels C-9 to C-12 as low-reflectivity true target point clouds. Thus, the technical solution disclosed herein avoids ghosting noise similar to A-5 to A-8 before high-reflectivity dilation filtering, reducing the false detection rate of targets; it also avoids the target under-detection problem of A-9 to A-12 after high-reflectivity dilation filtering, thereby avoiding the problem of insufficient pixel count in B-3 and reducing the target under-detection rate.
[0063] Taking SPAD and SIPM lidar as examples, SPAD and SIPM lidars have the same field of view. Figure 9a Taking the target scene shown as an example, the SPAD lidar performs laser detection on the target scene to obtain the first point cloud data, specifically as follows: Figure 9b As shown; the SIPM lidar performs laser detection on the target scene to obtain the second point cloud data, specifically as follows: Figure 9c As shown; then the first point cloud data and the second point cloud data are pixel-wise fused to obtain the target point cloud data, specifically as follows. Figure 9d As shown, this results in a higher degree of matching between the target point cloud data and the target scene. This reduces the probability of false detections while ensuring a sufficient amount of detection data, leading to a higher degree of matching between the detection data and the actual needs of intelligent driving, thus improving the safety and reliability of intelligent driving.
[0064] In some embodiments, the first detection data includes first echo data, the second detection data includes second echo data, and the fusion processing includes waveform fusion processing. In this case, waveform superposition or waveform weighted fusion processing can be performed on the time-synchronized first echo data and second echo data to obtain fused waveform data. The ranging value of each fused pixel is determined according to the waveform of each fused pixel in the fused waveform data and the preset detection threshold of each fused pixel.
[0065] In some embodiments, the first echo data is the original echo signal waveform acquired by the first laser detector 10, including the echo signal amplitude sequence of each pixel in the first laser detector 10 on the time axis; the second echo data is the original echo signal waveform acquired by the second laser detector 20, including the echo signal amplitude sequence of each pixel in the second laser detector 20 on the time axis. The echo data retains the complete waveform information of the original signal. Compared to point cloud data, which only contains ranging values, the echo data contains richer signal features, which is beneficial for more refined signal processing.
[0066] Specifically, since the first laser detector 10 and the second laser detector 20 are physically independent, there may be a deviation in the time reference for the echo data they acquire. If the two sets of echo data are directly superimposed, the time deviation will cause waveform misalignment, making effective superposition impossible, and may even produce false peaks due to waveform misalignment. Therefore, the processor 30 first performs time synchronization alignment processing on the first and second echo data. Time synchronization can be achieved through hardware synchronization signals, or it can be achieved through software algorithms (such as cross-correlation methods) to calculate and compensate for the time offset between the two sets of echo signals; this disclosure does not limit this approach. Through time synchronization processing, the waveforms of the two sets of echo signals are ensured to be precisely aligned on the time axis, laying the foundation for subsequent waveform superposition or fusion.
[0067] Furthermore, in the waveform overlay processing, the processor 30 directly adds the amplitude of the echo signal waveform of each pixel in the first echo data to the amplitude of the echo signal waveform of the corresponding pixel in the second echo data at the same time point to obtain the fused waveform. Thus, waveform fusion can further improve ranging accuracy and anti-interference capability, making it suitable for target scenarios in complex environments. Especially for weak signal targets, waveform overlay can enhance the effective signal and increase the detection probability.
[0068] In some embodiments, during waveform weighted fusion processing, the two echo signal waveforms can be weighted and fused according to a preset fusion weight, wherein the fusion weight can be set according to actual needs. Specifically, the calculation formula corresponding to waveform weighted fusion processing can be expressed by formula (1), as follows: Waveform C = a × Waveform A + b × Waveform B (1) Where 'a' represents the first weight, 'b' represents the second weight, waveform C represents the waveform of the fused pixel, waveform A represents the waveform of a pixel in the first echo data, and waveform B represents the waveform of a pixel in the second echo data corresponding to waveform A. For example, waveform C-1 = a × A-1 waveform + b × B-1 waveform, and waveform C-4 = a × A-4 waveform + b × B-1 waveform.
[0069] Specifically, waveform weighted fusion allows for differentiated adjustment of the contribution of the two waveforms. For example, in backlight or strong light scenarios, the signal-to-noise ratio of the SPAD LiDAR echo signal is low. The processor 30 can reduce the weight 'a' of the first echo data and increase the weight 'b' of the second echo data, thereby suppressing the noise component from the SPAD in the fused waveform while preserving the effective signal component from the SIPM LiDAR. In nighttime or low-light scenarios, the SPAD LiDAR is no longer affected by sunlight interference, and the processor 30 can increase the weight 'a' of the first echo data, fully leveraging the high line count advantage of the SPAD.
[0070] In some embodiments, for each fused pixel in the fused waveform data, if the maximum value of the waveform of the fused pixel is greater than a preset detection threshold for the fused pixel, then the ranging value of the fused pixel is the ranging value corresponding to the waveform of the fused pixel in the first echo data; if the maximum value of the waveform of the fused pixel is not greater than the preset detection threshold for the fused pixel, then the ranging value of the fused pixel is a second set value. The preset detection threshold for the fused pixel is the product of the detection threshold of the waveform of the fused pixel in the first echo data and a preset coefficient, where the preset coefficient is greater than 1.
[0071] Specifically, when determining the ranging value of each fused pixel, the processor 30 can acquire the fused waveform of each fused pixel in the fused waveform data and extract the preset detection threshold of the fused pixel. Then, the processor 30 compares the maximum value of the fused waveform with the preset detection threshold. If the maximum value of the waveform of the fused pixel is greater than the preset detection threshold of the fused pixel, the processor 30 determines that there is a target in the fused pixel and determines that the ranging value of the fused pixel is the ranging value corresponding to the fused pixel in the first echo data (or the ranging value corresponding to the fused pixel in the second echo data, or the weighted average of the two). If the maximum value of the waveform of the fused pixel is less than or equal to the preset detection threshold of the fused pixel, the processor 30 determines that there is no target in the fused pixel and determines the ranging value of the fused pixel as a second set value (i.e., an invalid value, indicating that there is no valid target in the pixel).
[0072] The maximum value of the waveform refers to the maximum amplitude of the fused pixel across all sampling points on the time axis, and its corresponding time position is the flight time of the echo signal. The maximum value of the waveform is the most commonly used feature in detection processing. Compared with other feature values such as the average value and integral area, the maximum value is the most direct and sensitive reflection of the echo signal strength, and can effectively distinguish between strong and weak echoes.
[0073] In some embodiments, taking the waveform fusion of SPAD and SIPM lidar as an example, in a high anti-expansion scenario, pixels A-1 to A-4 of the SPAD lidar exhibit real target echoes (waveforms significantly higher than the detection threshold), while pixels A-5 to A-8 exhibit ghost noise echoes caused by crosstalk (waveforms also higher than the detection threshold). Meanwhile, the SIPM lidar does not exhibit inter-channel crosstalk; pixel B-1 exhibits a real target echo (the target's flight time and position show a pulse waveform), while pixel B-2 has no effective echo (only background noise).
[0074] After waveform superposition processing of the first and second echo data, the fused waveforms of pixels C-1 to C-4 (corresponding to the superposition of pixels A-1 to A-4 and pixel B-1) are enhanced at the target time-of-flight position, and the amplitude of the real target signal increases due to the in-phase superposition of the two sets of waveforms. However, the fused waveforms of pixels C-5 to C-8 (corresponding to the superposition of pixels A-5 to A-8 and pixel B-2) are not enhanced at the target time-of-flight position—the ghost signal remains unchanged because there is an echo only on the SPAD side and no echo on the SIPM side. Accordingly, the preset detection threshold C is set higher than the original detection threshold A of SPAD (i.e., preset detection threshold C = r × detection threshold A, r > 1). Then, the maximum waveform value corresponding to each fused pixel is compared with C. The maximum value of the fused waveform of pixels C-1 to C-4 is still higher than C, so it is determined that a target exists and a valid ranging value is output; the maximum value of the fused waveform of pixels C-5 to C-8 is lower than C, so it is determined that no target exists and the ranging value is set to an invalid value. As a result, ghost noise was successfully suppressed, effectively reducing the false detection rate of the target.
[0075] In some embodiments, the preset detection threshold of the fused pixel is the product of the detection threshold of the waveform corresponding to the fused pixel in the first echo data and a preset coefficient, wherein the preset coefficient is greater than 1, such that the preset detection threshold of each fused pixel is higher than the detection threshold of the corresponding pixel in the first laser detector 10. In this way, waveform superposition can be used to enhance the signal amplitude of the real target while simultaneously increasing the preset detection threshold, ensuring that only the enhanced real target signal can cross the increased detection threshold, while unenhanced ghost signals are filtered out because they cannot cross the threshold.
[0076] In one embodiment, the specific value of the preset coefficient can be adjusted according to the actual application scenario. For example, in backlight or strong light scenarios, due to the large ambient light noise, a larger preset coefficient (e.g., 1.5 or 2.0) can be set to improve noise immunity, significantly increasing the preset detection threshold and effectively suppressing ambient light noise and ghosting noise. In nighttime or low-light scenarios, due to the low ambient light noise and weak echo signals from the actual target, a smaller preset coefficient (e.g., 1.1 or 1.2) can be set to avoid missing the actual target due to an excessively high threshold. Furthermore, the specific value of the preset coefficient can be adaptively adjusted according to different detector combinations, different target reflectivities, and different detection distances; this disclosure does not limit this adjustment.
[0077] It should be noted that the above-mentioned setting of the preset detection threshold is an optional implementation method, and this disclosure is not limited thereto. For example, in other embodiments, the preset detection threshold may also be the product of the detection threshold of the second echo data and a preset coefficient, or the weighted average of the detection thresholds of the first echo data and the second echo data multiplied by the preset coefficient, all of which fall within the protection scope of this disclosure.
[0078] Taking the waveform fusion of SPAD and SIPM lidar as an example, the waveform and detection threshold of each pixel in the first echo data are as follows: Figure 10 As shown, the detection threshold for pixels A-1 to A-8 is A; the waveform and detection threshold for each pixel in the second echo data are as follows. Figure 10 As shown, the detection thresholds for pixels B-1 and B-2 are both B. Then, the first and second echo data are fused to obtain fused waveform data. The waveform of each fused pixel and the preset detection threshold for each fused pixel are specifically shown below. Figure 10 As shown. Since pixels A-1 to A-4 contain real target point clouds, the corresponding waveforms of pixels A-1 to A-4 are as follows. Figure 10 As shown, the waveform is significantly higher than the detection threshold A; however, pixels A-5 to A-8 exhibit ghost noise due to crosstalk, as detailed in [link to documentation]. Figure 10 The waveforms of pixels A-5 to A-8 are also higher than the detection threshold A. Because the SIPM lidar uses time-division multiplexing and has large pixel spacing, there is no vertical channel crosstalk, making pixel B-1 a true target point cloud. The corresponding waveform of pixel B-1 is as follows... Figure 10 As shown, a pulse waveform exists at the target's flight time position; pixel B-2 has no point cloud (NAN point), see details. Figure 10The waveform of pixel B-2 shows no echo signal. After waveform fusion processing, the waveforms of pixels C-1 to C-4 at the target flight time position are enhanced, while the waveforms of C-5 to C-8 at the target flight time position are not enhanced. And according to the preset detection threshold C = r... Since r is greater than 1, the preset detection threshold C will also be higher than the detection threshold A. Therefore, Figure 10 The target waveforms of pixels C-1 to C-4 are still higher than the preset detection threshold C, while the target waveforms of pixels C-5 to C-8 are lower than the preset detection threshold C. This ensures that the echoes of pixels C-5 to C-8 will not be detected as ghost noise point clouds, effectively reducing the false detection rate of the target.
[0079] For each fused pixel in the fused waveform data, if the maximum value of the waveform of the fused pixel is greater than the preset detection threshold of the fused pixel, then the ranging value of the fused pixel is the ranging value corresponding to the waveform of the fused pixel in the first echo data; if the maximum value of the waveform of the fused pixel is not greater than the preset detection threshold of the fused pixel, then the ranging value of the fused pixel is the second set value.
[0080] In some embodiments, the processor 30 is disposed in a vehicle, whereby the processor 30 is configured to determine a fusion strategy for fusing the first detection data and the second detection data based on the vehicle's operating condition information and / or environmental information.
[0081] In some embodiments, environmental information and operating condition information may be collected by vehicle sensors or actuators. Sensors may include, for example, cameras, temperature sensors, humidity sensors, positioning sensors, and speed sensors. Actuators may include, for example, windshield wipers. Environmental information may include one or more of the following: vehicle position, current weather, wiper status, temperature and humidity, and light intensity. Operating condition information may include one or more of the following: vehicle speed, yaw rate, and braking information.
[0082] In some embodiments, the processor 30 is configured to reduce the first weight of the first detection data and increase the second weight of the second detection data when it is determined from environmental information that the vehicle is in a backlight or strong light scene; and to increase the first weight when it is determined from environmental information that the vehicle is in a low light or night scene.
[0083] Specifically, when the processor 30 determines that the vehicle is in a backlight or strong light scene based on environmental information (such as light sensor data), since the SPAD LiDAR will generate a lot of noise in the strong light environment, the confidence of the point cloud will decrease and the signal-to-noise ratio of the echo signal will decrease, while the SIPM LiDAR is relatively insensitive to strong light, the processor 30 reduces the first weight of the first detection data (SPAD) and increases the second weight of the second detection data (SIPM).
[0084] In some embodiments, the first weight and the second weight can take values between 0 and 1, and their sum is 1. Under normal lighting conditions, both the first weight and the second weight can be 0.5. When the processor 30 detects a strong light signal, the first weight is reduced to 0.2 and the second weight is increased to 0.8. When the strong light signal further intensifies, the first weight is reduced to 0 and the second weight is increased to 1. In this way, the fusion strategy is adapted to the strong light environment, ensuring the reliability of all-weather detection.
[0085] Specifically, when the processor 30 determines that the vehicle is in a low-light or nighttime scene based on environmental information (such as light sensor data or GPS positioning information combined with time information), the SPAD LiDAR is no longer affected by sunlight, and its high line count advantage is fully utilized. Therefore, the processor 30 increases the first weight of the first detection data (SPAD) to leverage the high line count advantage of the SPAD LiDAR and effectively improve the ranging capability and detection point count of the LiDAR in nighttime scenes. For example, when the processor 30 detects that the ambient light intensity is lower than a preset light threshold, it increases the first weight to 0.8 and decreases the second weight to 0.2 to fully utilize the high line count and high resolution advantages of the SPAD LiDAR, ensuring reliable detection of distant small targets (such as pedestrians, cones, and debris) during nighttime driving.
[0086] In some embodiments, the processor 30 can also determine the current driving condition of the vehicle based on the operating condition information, and then dynamically adjust the fusion strategy according to the current driving condition. Specifically, when the current driving condition is determined to be a high-speed driving condition based on the vehicle speed, the vehicle's requirements for long-range detection capabilities are significantly increased. The processor 30 can increase the weight of the high-line-count SPAD LiDAR to enhance the number of detection points for small targets at long distances, providing more sufficient response time for the intelligent driving system. When the current driving condition is determined to be a low-speed driving condition based on the vehicle speed (such as urban congestion), the vehicle has higher requirements for short-range detection accuracy and wide field of view. The processor 30 can increase the weight of the SIPM LiDAR or decrease the weight of the SPAD to utilize the crosstalk-free advantage of SIPM to improve the stability and accuracy of the short-range point cloud.
[0087] In some embodiments, after determining the current driving condition of the vehicle based on the operating condition information, the processor 30 can also dynamically adjust the fusion strategy based on the current driving condition and environmental information. When the current driving condition is determined to be a high-speed driving condition based on the vehicle speed, and the environmental information indicates that the vehicle is in a closed or semi-closed environment such as a tunnel or underground garage, the processor 30 can increase the first weight and ensure that the second weight remains unchanged.
[0088] In some embodiments, the processor 30 can also determine the weather conditions of the vehicle based on environmental information and dynamically adjust the fusion strategy according to the weather conditions. Specifically, when the processor 30 determines that the weather is rainy or snowy based on the wiper activation status and / or rain sensor information, the signal-to-noise ratio of the echo signal will decrease due to the scattering and attenuation of the laser signal by raindrops and snowflakes. The processor 30 can increase the weight of waveform fusion processing, enhance the weak signal by waveform superposition in waveform fusion to compensate for the signal attenuation caused by rain and snow, and at the same time appropriately increase the preset coefficient of the detection threshold to suppress the noise caused by raindrop scattering, thereby ensuring the detection performance under adverse weather conditions.
[0089] In some embodiments, the processor 30 can also determine the vehicle's location information based on environmental information, and then dynamically adjust the fusion strategy based on the location information. Specifically, when the processor 30 determines from GPS positioning information that the vehicle is in a closed or semi-closed environment such as a tunnel or underground parking garage, the ambient lighting conditions may change abruptly and the GPS signal may weaken or be lost. The processor 30 can comprehensively judge the environmental state based on the location information and the lighting sensor information, and dynamically adjust the fusion strategy to ensure the detection stability of the lidar in scenarios with drastic changes in lighting, such as tunnel entrances / exits.
[0090] In some embodiments, the processor 30 can also perform more refined scene recognition and strategy selection based on a combination of various environmental information such as illumination information and vehicle location information. Specifically, during dusk or dawn, when illumination conditions change rapidly and vehicles may simultaneously face alternating backlight and low light, the processor 30 can comprehensively determine the current scene category by fusing multiple environmental information (illuminance change rate, GPS time, solar azimuth angle, etc.) and select a preset fusion strategy that matches the scene.
[0091] It should be noted that the specific values of the aforementioned weight adjustments (such as 0.2, 0.8, etc.) are merely illustrative examples. This disclosure does not limit the specific values of the first and second weights, as long as adaptive adjustment can be achieved in different scenarios. In practical applications, weight adjustment can be continuous (such as linear interpolation based on light intensity) or discrete (such as preset multiple levels), both of which fall within the protection scope of this disclosure.
[0092] According to a second aspect of this disclosure, a data processing method is provided, applied to an onboard lidar or a vehicle including an onboard lidar. The onboard lidar includes a first laser detector for acquiring first detection data and a second laser detector for acquiring second detection data. The first laser detector and the second laser detector are detectors of different types, such as... Figure 11 As shown, the method includes: Step S110: Perform fusion processing on the first detection data and the second detection data to obtain target point cloud data, wherein the fusion processing includes pixel fusion processing and / or waveform fusion processing.
[0093] In some embodiments, the first laser detector includes a first detector array, and the second laser detector includes a second detector array, wherein the detector types of the first detector array and the second detector array are different, and the number of first pixels corresponding to the first detector array is greater than the number of second pixels corresponding to the second detector array.
[0094] In some embodiments, the first laser detector and the second laser detector are integrated within the same housing and share an optical system.
[0095] In some embodiments, the first laser detector and the second laser detector are integrated within the same housing and share an optical system.
[0096] In some embodiments, the first laser detector and the second laser detector are both independent devices.
[0097] In some embodiments, the vehicle-mounted lidar further includes a data transmission interface, which is communicatively connected to the first laser detector, the second laser detector, and the processor.
[0098] In some embodiments, the first field of view of the first laser detector and the second field of view of the second laser detector partially or completely overlap. When the first field of view and the second field of view partially overlap, the first optical center direction of the first laser detector forms a first angle with the vehicle's direction of travel, and the second optical center direction of the second laser detector forms a second angle with the vehicle's direction of travel. The first optical center direction and the second optical center direction are located on the left and right sides of the vehicle's direction of travel, respectively.
[0099] In some embodiments, the first detection data includes first point cloud data, the second detection data includes second point cloud data, and the fusion processing includes pixel fusion processing. The first detection data and the second detection data are fused to obtain target point cloud data, including: Based on the preset pixel matching relationship, the first sub-pixel in the first point cloud data and the second sub-pixel in the second point cloud data are matched to obtain at least one set of pixel pairs; For each pair of pixels, the distance value of the fused pixel corresponding to the pair of pixels is determined based on the difference between the first distance value of the first sub-pixel in the pair of pixels and the second distance value of the second sub-pixel in the pair of pixels. The target point cloud data is obtained based on the distance measurement value of the corresponding fused pixel points for each group of pixels.
[0100] In some embodiments, for each pair of pixels, the pixel value of the fused pixel corresponding to the pair of pixels is determined based on the difference in the ranging values of the pair of pixels, including: For each pair of pixels, if the difference in the measured values of the pair of pixels is less than a preset threshold, then the measured value of the fused pixel corresponding to the pair of pixels is the first measured value of the first sub-pixel in the pair of pixels; if the difference in the measured values of the pair of pixels is not less than the preset threshold, then the measured value of the fused pixel corresponding to the pair of pixels is the first set value.
[0101] In some embodiments, in a pixel matching relationship, one second sub-pixel matches multiple first sub-pixels.
[0102] In some embodiments, the first detection data includes first echo data, the second detection data includes second echo data, and the fusion processing includes waveform fusion processing. The first detection data and the second detection data are fused to obtain target point cloud data, including: The first and second echo data after time synchronization are subjected to waveform superposition or waveform weighted fusion to obtain fused waveform data. The ranging value of each fused pixel is determined based on the waveform of each fused pixel in the fused wave data and the preset detection threshold of each fused pixel; The target point cloud data is obtained based on the ranging value of each fused pixel.
[0103] In some embodiments, determining the ranging value of each fused pixel based on the waveform of each fused pixel in the fused waveform data and a preset detection threshold includes: For each fused pixel in the fused waveform data, if the maximum value of the waveform of the fused pixel is greater than the preset detection threshold of the fused pixel, then the ranging value of the fused pixel is the ranging value corresponding to the waveform of the fused pixel in the first echo data; if the maximum value of the waveform of the fused pixel is not greater than the preset detection threshold of the fused pixel, then the ranging value of the fused pixel is the second set value.
[0104] In some embodiments, the preset detection threshold of the fused pixel is the product of the detection threshold of the waveform of the fused pixel in the first echo data and a preset coefficient, wherein the preset coefficient is greater than 1.
[0105] In some embodiments, the method further includes: Based on the vehicle's operating condition information and / or environmental information, a fusion strategy is determined for fusing the first and second detection data.
[0106] In some embodiments, a fusion strategy for fusing the first detection data and the second detection data is determined based on vehicle or environmental information, including: When environmental information determines that the vehicle is in a backlight or strong light scene, the first weight of the first detection data is reduced and the second weight of the second detection data is increased; and when environmental information determines that the vehicle is in a low light or nighttime scene, the first weight is increased.
[0107] According to a third aspect of this disclosure, a data processing system is provided, such as Figure 12 As shown, it includes: The first laser detector 10 is used to collect the first detection data; The second laser detector 20 is used to collect second detection data, wherein the first laser detector 10 and the second laser detector 20 are detectors of different types; The processor 30 is connected to the first laser detector 10 and the second laser detector 20, and is used to perform fusion processing on the first detection data and the second detection data to obtain target point cloud data. The fusion processing includes pixel fusion processing and / or waveform fusion processing.
[0108] In some embodiments, when the vehicle-mounted lidar is composed of a first laser detector 10, a second laser detector 20, and a processor 30, the data processing system further includes a domain controller disposed in the vehicle, which is communicatively connected to the processor 30, specifically as follows: Figure 2 As shown.
[0109] In some embodiments, when an on-board lidar is composed of a first laser detector 10 and a second laser detector 20, the processor 30 is disposed in the vehicle. Specifically, the processor 30 can be a hardware module of the vehicle's domain controller, such as... Figure 3 As shown. Of course, the processor 30 can also be a separate hardware device in the vehicle.
[0110] In some embodiments, when both the first laser detector 10 and the second laser detector 20 are separate lidar units, the processor 30 is located in the vehicle. Specifically, the processor 30 can be a hardware module of the vehicle's domain controller, such as... Figure 4 As shown. Of course, the processor 30 can also be a separate hardware device in the vehicle.
[0111] This disclosure also provides a computer-readable storage medium storing instructions that, when executed by a processor, configure the processor to perform the data processing method described above.
[0112] Those skilled in the art will understand that embodiments of this disclosure can be provided as methods, systems, or computer program products. Therefore, this disclosure can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this disclosure can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0113] This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0114] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0115] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0116] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0117] Memory may include non-persistent memory in computer-readable media, random access memory (RAM), and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.
[0118] Computer-readable media include both permanent and non-permanent, removable and non-removable media, which can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated communication signals and carrier waves.
[0119] This disclosure also provides a controller, a processor, and a memory storing a computer program that, when executed by the processor, implements the data processing method described above.
[0120] like Figure 13 The diagram shown is a schematic representation of a vehicle architecture according to an embodiment of this disclosure. In this embodiment, vehicle 70 includes the aforementioned onboard LiDAR. In this embodiment, vehicle 70 can be a gasoline-powered vehicle, a plug-in hybrid electric vehicle, or an electric vehicle, etc., and this disclosure does not specifically limit its application.
[0121] In some embodiments, the vehicle 70 may also perform the above-described data processing method, or the vehicle 70 may also include the above-described data processing system.
[0122] In the description of this disclosure, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more features. In the description of this disclosure, "a plurality of" means two or more, unless otherwise explicitly specified.
[0123] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.
[0124] The embodiments, implementation methods and related technical features disclosed herein can be combined and substituted for each other without conflict.
[0125] The above are merely preferred embodiments of this disclosure and are not intended to limit this disclosure in any way. Any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of this disclosure without departing from the content of the technical solution of this disclosure shall still fall within the scope of the technical solution of this disclosure.
Claims
1. A vehicle-mounted lidar, characterized in that, include: The first laser detector is used to collect the first detection data; The second laser detector is used to collect second detection data, wherein the first laser detector and the second laser detector are different types of detectors; The vehicle-mounted lidar sends the first detection data and the second detection data to the processor, so that the processor can perform fusion processing on the first detection data and the second detection data to obtain target point cloud data. The fusion processing includes pixel fusion processing and / or waveform fusion processing.
2. The vehicle-mounted lidar as described in claim 1, characterized in that, The first laser detector includes a first detector array, and the second laser detector includes a second detector array. The detector types of the first detector array and the second detector array are different, and the number of first pixels corresponding to the first detector array is greater than the number of second pixels corresponding to the second detector array.
3. The vehicle-mounted lidar as described in claim 1, characterized in that, The first laser detector and the second laser detector are integrated into the same housing and share an optical system; the vehicle-mounted lidar includes the processor, or the processor is disposed in the vehicle; or, The first laser detector and the second laser detector are both independent devices, and the processor is located in the vehicle.
4. The vehicle-mounted lidar as described in claim 1, characterized in that, The processor is disposed in the vehicle, and the vehicle-mounted lidar further includes: The data transmission interface is communicatively connected to the first laser detector, the second laser detector, and the processor, respectively, and is used to transmit the first detection data and the second detection data to the processor.
5. The vehicle-mounted lidar as described in claim 1, characterized in that, The first field of view of the first laser detector and the second field of view of the second laser detector partially or completely overlap. When the first field of view and the second field of view partially overlap, the first optical center direction and the second optical center direction are located on the left and right sides of the vehicle's direction of travel, respectively.
6. A data processing method, characterized in that, Applied to an automotive lidar as described in any one of claims 1-5, or a vehicle including said automotive lidar, the method comprises: The first and second detection data are fused to obtain target point cloud data, wherein the fusion process includes pixel fusion processing and / or waveform fusion processing.
7. The method as described in claim 6, characterized in that, The first detection data includes first point cloud data, the second detection data includes second point cloud data, and the fusion processing includes pixel fusion processing. The fusion processing of the first and second detection data to obtain target point cloud data includes: Based on a preset pixel matching relationship, the first sub-pixel in the first point cloud data and the second sub-pixel in the second point cloud data are matched to obtain at least one set of pixel pairs. In the pixel matching relationship, one second sub-pixel matches multiple first sub-pixels. For each pair of pixels, the distance value of the fused pixel corresponding to the pair of pixels is determined based on the difference between the first distance value of the first sub-pixel in the pair of pixels and the second distance value of the second sub-pixel in the pair of pixels. The target point cloud data is obtained based on the ranging value of the corresponding fused pixel points for each group of pixels.
8. The method as described in claim 7, characterized in that, For each pair of pixels, determining the pixel value of the corresponding fused pixel based on the difference in the ranging values of that pair includes: For each pair of pixels, if the difference in the measured values of the pair of pixels is less than a preset threshold, then the measured value of the fused pixel corresponding to the pair of pixels is the first measured value of the first sub-pixel in the pair of pixels; if the difference in the measured values of the pair of pixels is not less than the preset threshold, then the measured value of the fused pixel corresponding to the pair of pixels is a first set value.
9. The method as described in claim 6, characterized in that, The first detection data includes first echo data, the second detection data includes second echo data, and the fusion processing includes waveform fusion processing. The fusion processing of the first and second detection data to obtain target point cloud data includes: The first echo data and the second echo data after time synchronization are subjected to waveform superposition or waveform weighted fusion processing to obtain fused waveform data; The ranging value of each fused pixel is determined based on the waveform of each fused pixel in the fused wave data and the preset detection threshold of each fused pixel; The target point cloud data is obtained based on the ranging value of each fused pixel.
10. The method as described in claim 9, characterized in that, The step of determining the ranging value of each fused pixel based on the waveform of each fused pixel in the fused waveform data and a preset detection threshold includes: For each fused pixel in the fused waveform data, if the maximum value of the waveform of the fused pixel is greater than the preset detection threshold of the fused pixel, then the ranging value of the fused pixel is the ranging value in the first echo data corresponding to the waveform of the fused pixel; if the maximum value of the waveform of the fused pixel is not greater than the preset detection threshold of the fused pixel, then the ranging value of the fused pixel is a second set value.
11. The method as described in claim 10, characterized in that, The preset detection threshold of the fused pixel is the product of the detection threshold of the waveform of the fused pixel in the first echo data and a preset coefficient.
12. The method as described in claim 6, characterized in that, The method further includes: Based on the vehicle's operating condition information and / or environmental information, a fusion strategy is determined for fusing the first detection data and the second detection data.
13. The vehicle-mounted lidar as described in claim 12, characterized in that, The step of determining a fusion strategy for fusing the first detection data and the second detection data based on the vehicle's environmental information includes: If the environmental information determines that the vehicle is in a backlight or strong light scene, the first weight of the first detection data is reduced and the second weight of the second detection data is increased; and if the environmental information determines that the vehicle is in a low light or nighttime scene, the first weight is increased.
14. A data processing system, characterized in that, include: The first laser detector is used to collect the first detection data; The second laser detector is used to collect second detection data, wherein the first laser detector and the second laser detector are different types of detectors; A processor, connected to the first laser detector and the second laser detector, is used to perform fusion processing on the first detection data and the second detection data to obtain target point cloud data. The fusion processing includes pixel fusion processing and / or waveform fusion processing.
15. A vehicle, characterized in that, Includes the vehicle-mounted lidar as described in any one of claims 1-5, or performs the data processing method as described in any one of claims 6-13, or includes the data processing system as described in claim 14.
