Point cloud data processing method, system, device and equipment of laser radar

Abstract

This application discloses a method, system, device, and equipment for processing point cloud data of a lidar. The lidar includes multiple lasers, including a first laser and at least one second laser. A polarizer is provided on the laser emission path of the first laser, and an analyzer is provided on the receiving optical path of the corresponding channel of the first laser inside the lidar. The method includes: acquiring first point cloud data corresponding to the first laser and second point cloud data corresponding to the second laser; calculating the distance between each second target point in the second point cloud data and its multiple neighboring points to filter out at least one second target point whose distance is greater than a preset value, thereby obtaining updated second point cloud data; and determining the first point cloud data and the updated second point cloud data as the final point cloud data acquired by the lidar. This method can improve the accuracy of detecting and filtering noise in the point cloud data while reducing the computational load.

Description

Technical Field

[0001] This application belongs to the field of point cloud data processing technology, and in particular relates to a point cloud data processing method, system, device and equipment for lidar. Background Technology

[0002] Autonomous vehicles are equipped with lidar, which can collect point cloud data within the field of view.

[0003] In rainy or foggy weather, vehicles ahead of autonomous vehicles will splash water. During data collection, the radar also collects point cloud data corresponding to these splashes and other contaminants. This point cloud data contains noise, which can affect the LiDAR's perception system and cause misjudgments. Therefore, it is necessary to remove noise from the point cloud data. Existing methods detect and filter noise by finding outliers in the global point cloud, but this method has low accuracy and high computational cost. Summary of the Invention

[0004] This application provides a method, system, device, and equipment for processing point cloud data of lidar, which can improve the accuracy of detecting and filtering noise in point cloud data and reduce the amount of computation.

[0005] On one hand, this application provides a point cloud data processing method for a lidar. The lidar is installed on a vehicle and includes multiple lasers, including a first laser and at least one second laser. The field of view corresponding to the first laser is the area in front of the vehicle. A polarizer is provided on the laser emission path of the first laser, and an analyzer is provided on the receiving path of the corresponding channel of the first laser inside the lidar. The polarizer is used to convert the emission light of the first laser into linearly polarized light, and the analyzer is used to filter the linearly polarized light. The field of view corresponding to the second laser is the area other than the area in front of the vehicle.

[0006] Point cloud data processing methods include:

[0007] Obtain the first point cloud data corresponding to the first laser and the second point cloud data corresponding to the second laser;

[0008] Determine the second target point from the second point cloud data;

[0009] Search for multiple neighboring points that are close to the second target point from the second point cloud data;

[0010] Calculate the distance between the second target point and each of its neighboring points;

[0011] When at least one distance is greater than a preset threshold, the second target point is removed from the second point cloud data to obtain the updated second point cloud data;

[0012] Determine the second target point from the updated second point cloud data, and return multiple neighboring points that are close to the second target point from the second point cloud data, until every second point in the second point cloud data is determined to be the second target point;

[0013] The first point cloud data and the updated second point cloud data are identified as the final point cloud data collected by the lidar.

[0014] In some embodiments of this application, determining the second target point from the second point cloud data includes:

[0015] Based on the second point cloud data and the projection relationship, a distance-depth map corresponding to the forward-looking direction of the second point cloud data is generated;

[0016] Based on the point cloud spawn order in the distance-depth map, a point cloud structure map is generated. The point cloud structure map includes a first region and a second region. The height of the points in the first region is greater than the first height, and the height of the points in the second region is less than or equal to the first height.

[0017] Determine the second target point from the first region and the second region respectively;

[0018] Search for multiple neighboring points in the second point cloud data that are close to the second target point, including:

[0019] For the second target point in the first region, search for a first number of neighboring points that are close to the second target point from the point cloud structure map;

[0020] For the second target point in the second region, search for a second number of neighboring points that are close to the second target point from the point cloud structure map;

[0021] The first quantity is less than the second quantity.

[0022] In some embodiments of this application, the point cloud structure diagram further includes a third region and a fourth region, wherein the height of the points in the third region is greater than the second height, the height of the points in the fourth region is less than or equal to the third height, and the height of the points in the first region is less than or equal to the second height, and the height of the points in the second region is greater than the third height.

[0023] The third height is less than the first height, and the first height is less than the second height.

[0024] In some embodiments of this application, the distance between the second target point and each neighboring point is calculated, including:

[0025] Calculate the Euclidean distance between the second target point and each of its neighboring points.

[0026] In some embodiments of this application, the first height, the second height, and the third height are determined based on the height of the vehicle.

[0027] On the other hand, embodiments of this application provide a point cloud data processing system for a lidar, including a lidar and a processor.

[0028] The lidar is installed on the vehicle and includes multiple lasers, including a first laser and at least one second laser. The field of view corresponding to the first laser is the area in front of the vehicle. A polarizer is set in the laser emission path of the first laser, and an analyzer is set in the receiving path of the corresponding channel of the first laser inside the lidar. The polarizer is used to convert the emission light into linearly polarized light, and the analyzer is used to filter the linearly polarized light. The field of view corresponding to the second laser is the area other than the area in front of the vehicle.

[0029] The processor is used to execute the point cloud data processing method of any of the above embodiments.

[0030] On the other hand, this application provides a point cloud data processing device for a lidar. The lidar is installed on a vehicle and includes multiple lasers, including a first laser and at least one second laser. The field of view corresponding to the first laser is the area in front of the vehicle. A polarizer is provided on the laser emission path of the first laser, and an analyzer is provided on the receiving path of the corresponding channel of the first laser inside the lidar. The polarizer is used to convert the emitted light into linearly polarized light, and the analyzer is used to filter the linearly polarized light. The field of view corresponding to the second laser is the area other than the area in front of the vehicle.

[0031] The point cloud data processing device includes:

[0032] The acquisition module is used to acquire the first point cloud data corresponding to the first laser and the second point cloud data corresponding to the second laser.

[0033] The first determining module is used to determine the second target point from the second point cloud data;

[0034] The search module is used to search for multiple neighboring points that are close to the second target point from the second point cloud data;

[0035] The calculation module is used to calculate the distance between the second target point and each of its neighboring points.

[0036] The update module is used to remove the second target point from the second point cloud data when at least one distance is greater than a preset threshold, so as to obtain the updated second point cloud data.

[0037] Determine the second target point from the updated second point cloud data, and return multiple neighboring points that are close to the second target point from the second point cloud data, until every second point in the second point cloud data is determined to be the second target point;

[0038] The second determining module is used to determine the first point cloud data and the updated second point cloud data as the final point cloud data collected by the lidar.

[0039] On the other hand, embodiments of this application provide a point cloud data processing device for lidar, the point cloud data processing device including: a processor and a memory storing computer program instructions;

[0040] The processor executes computer program instructions to implement the point cloud data processing method of any of the above embodiments.

[0041] On the other hand, embodiments of this application provide a computer-readable storage medium storing computer program instructions, which, when executed by a processor, implement the point cloud data processing method of any of the above embodiments.

[0042] On the other hand, embodiments of this application provide a computer program product, wherein when the instructions in the computer program product are executed by the processor of an electronic device, the electronic device performs the point cloud data processing method of any of the above embodiments.

[0043] According to an embodiment of this application, a point cloud data processing method, system, device, and equipment for a lidar are provided. The lidar is installed on a vehicle and includes multiple lasers, including a first laser and at least one second laser. The field of view corresponding to the first laser is the area in front of the vehicle. A polarizer is provided on the laser emission path of the first laser, and an analyzer is provided on the receiving path of the corresponding channel of the first laser inside the lidar. The polarizer is used to convert the emitted light into linearly polarized light, and the analyzer is used to filter the linearly polarized light. The field of view corresponding to the second laser is the area other than the area in front of the vehicle. The process involves acquiring first point cloud data corresponding to a first laser and second point cloud data corresponding to a second laser. In processing the first point cloud data corresponding to the first laser, by setting up a polarizer and analyzer, point clouds generated by raindrops in the field of view directly in front of the vehicle corresponding to the first laser can be detected and filtered out more accurately. This reduces noise in the first point cloud data at its source and improves the accuracy of detecting and filtering noise. In processing the second point cloud data corresponding to the second laser, the distance between each second target point in the second point cloud data and its multiple neighboring points is calculated to filter out at least one second target point whose distance is greater than a preset value, thereby filtering out noise in the second point cloud data. Compared with existing technologies that detect and filter noise in point cloud data by finding outliers in the global point cloud, this method reduces the computational load. Attached Figure Description

[0044] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 This is a schematic diagram illustrating the working principle of a single laser within a lidar system.

[0046] Figure 2 This is a diagram showing the field of view of multiple lasers in the first type of lidar.

[0047] Figure 3 This is a diagram illustrating the field of view stitching of multiple lasers within the second type of lidar.

[0048] Figure 4 This is a schematic diagram showing the installation of the polarizer and analyzer in a lidar system.

[0049] Figure 5 A flowchart illustrating a point cloud data processing method for a lidar provided in an embodiment of this application;

[0050] Figure 6A flowchart illustrating another point cloud data processing method for lidar provided in an embodiment of this application;

[0051] Figure 7 A flowchart illustrating another point cloud data processing method for lidar provided in an embodiment of this application;

[0052] Figure 8 A schematic diagram illustrating the first nearest-neighbor search method provided in this application;

[0053] Figure 9 A schematic diagram illustrating the second nearest-point search method provided in this application;

[0054] Figure 10 A schematic diagram illustrating the third nearest-point search method provided in this application;

[0055] Figure 11 A schematic diagram of the structure of a point cloud data processing device for a lidar provided in an embodiment of this application;

[0056] Figure 12 This is a schematic diagram of the structure of a point cloud data processing device for a lidar provided in an embodiment of this application. Detailed Implementation

[0057] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.

[0058] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0059] To facilitate understanding, we will first introduce the structure of lidar. The lidar referred to in this article is MEMS (Micro-Electro-Mechanical System) lidar, which includes multiple lasers, multiple receivers, a beam splitter, and a MEMS micro-mirror.

[0060] Figure 1 This is a schematic diagram illustrating the working principle of a single laser within a lidar system.

[0061] The laser beam emitted by laser 110 is irradiated onto the corresponding area on MEMS micro-mirror 130 after passing through beam splitter 120. Different excitation frequencies are set for the fast and slow axes of MEMS micro-mirror 130, causing MEMS micro-mirror 130 to oscillate. After being reflected by MEMS micro-mirror 130, the laser beam is emitted to the outside world, forming the scanning area corresponding to laser 110. This scanning area is the field of view (Fov) corresponding to laser 110. The light in this scanning area is reflected by external objects and returns along the original path to be received by receiver 140. After processing, a point cloud is formed. Each point in the point cloud contains three-dimensional coordinate information and reflection intensity information.

[0062] Figure 2 This is a diagram illustrating the field of view of multiple lasers within the first type of lidar.

[0063] Figure 3 This is a diagram illustrating the field of view of multiple lasers within the second type of lidar.

[0064] Because the swing range of MEMS micromirrors is limited, generally within 8 degrees, lidar cannot achieve a larger field of view. In order to meet the observation range of 360 degrees for autonomous driving, multiple lidars are deployed at different positions of autonomous vehicles. At the same time, multiple sub-sectors are designed inside the lidar. Generally, 5-8 sub-sectors are designed. The laser corresponding to each sector works independently. The field of view of each sub-sector is determined by the placement position of the laser. The sparseness density of the point cloud in each sub-sector is determined by the frequency of the fast and slow axes of the MEMS micromirror.

[0065] For example, Figure 2 The corresponding LiDAR has a field of view of approximately 120 degrees, divided into 5 sub-sectors. Typically, 3 such LiDARs are installed on the roof of a vehicle. One of these LiDARs scans the area in front of the vehicle, with laser 3 scanning the area directly in front of the vehicle. The other lasers, namely lasers 1, 2, 4, and 5, scan the area excluding the area directly in front of the vehicle. Figure 3The corresponding lidar is a blind spot radar, which is generally installed on both sides of the front bumper of the vehicle, one unit on each side. The scanning area corresponding to laser 3 and laser 7 is the area directly in front of the vehicle. The scanning areas corresponding to the other lasers, namely laser 1, laser 2, laser 4, laser 5, laser 6 and laser 8, are the areas other than the area directly in front of the vehicle.

[0066] In rainy or foggy weather, when a vehicle in front of the autonomous vehicle is traveling at high speed, it will splash water. Some of the light emitted by the lidar installed on the vehicle hits the raindrops and is reflected multiple times before returning along the original path and being received by the receiver. The other part of the light passes through the raindrops and hits the target vehicle in front, and after reflection, it returns along the original path and is received by the receiver. The two parts of reflected light form point cloud data. The point cloud reflected by the raindrops will form a part of the trail near the point cloud of the target vehicle, which can easily lead to misjudgment of the target recognition by the perception system of the autonomous vehicle.

[0067] To address the issue that water splashes from vehicles in front of autonomous vehicles can easily cause misjudgments in the vehicle's perception system during rainy or foggy weather, this application provides a method, system, device, and equipment for processing point cloud data from a lidar system. This method removes noise from point clouds generated by raindrops in rainy conditions. Compared to existing technologies, this method improves the accuracy of detecting and filtering noise in point cloud data while reducing computational load.

[0068] Figure 4 This is a schematic diagram of the installation of the polarizer and analyzer in a lidar system.

[0069] Figure 5 This is a flowchart illustrating a point cloud data processing method for lidar provided in an embodiment of this application.

[0070] Figure 6 This is a flowchart illustrating another point cloud data processing method for LiDAR provided in an embodiment of this application.

[0071] Figure 7 This is a flowchart illustrating another point cloud data processing method for lidar provided in an embodiment of this application.

[0072] This application provides a method for processing point cloud data from a lidar system. The lidar is installed on a vehicle and includes multiple lasers, including a first laser and at least one second laser. The field of view corresponding to the first laser is the area in front of the vehicle. A polarizer 150 is provided on the laser emission path of the first laser, and an analyzer 160 is provided on the receiving path of the corresponding channel of the first laser inside the lidar. The polarizer 150 is used to convert the emission light from the first laser into linearly polarized light, and the analyzer 160 is used to filter the linearly polarized light. The field of view corresponding to the second laser is the area other than the area in front of the vehicle.

[0073] As mentioned earlier, three LiDAR sensors are typically installed on the roof of a vehicle. One of these LiDAR sensors scans the area in front of the vehicle, with laser 3 specifically targeting the area directly in front of the vehicle. Laser 3 from the LiDAR sensor used for scanning the area in front of the vehicle can be designated as the first laser, while the lasers used for other areas, such as lasers 1, 2, 4, and 5, can be designated as second lasers.

[0074] The polarizer 150 is placed in the laser output path of the first laser, and the analyzer 160 is placed in the receiving path of the corresponding channel of the first laser inside the lidar. Specifically, referring to the figure, the polarizer 150 is placed in the laser output path corresponding to the area of ​​the laser 110 corresponding to the MEMS micro-mirror 130, and the analyzer 160 is placed in the receiving path of the receiver 140 receiving the reflected light.

[0075] In this way, the laser beam emitted by the laser 110 passes through the beam splitter 120 and illuminates the corresponding area on the MEMS micro-mirror 130. After being reflected by the MEMS micro-mirror 130, the laser beam passes through the polarizer 150, which converts the light into linearly polarized light before illuminating the outside world, forming the scanning area corresponding to the laser 110. The light in this scanning area is reflected by external objects and returns along the original path. After being filtered by the analyzer 160, the linearly polarized light is received by the receiver 140. After processing, the first point cloud data corresponding to the first laser is formed.

[0076] The following explains how the polarizer 150 and analyzer 160 filter raindrop point clouds.

[0077] A vertical transmission axis can be set for the polarizer 150. After the laser beam is reflected by the MEMS micro-mirror 130, it passes through the polarizer 150 and becomes vertically linearly polarized light. When the linearly polarized light is transmitted in rain or fog, part of the linearly polarized light hits the raindrops and undergoes multiple reflections inside the raindrops, changing into horizontally linearly polarized light. The other part of the linearly polarized light does not enter the raindrops but hits the vehicle in front directly. After being reflected by the vehicle, it remains vertically linearly polarized light.

[0078] The analyzer 160 can be configured with a vertical transmission axis. The light that hits the target vehicle and is reflected back is vertically polarized light with only a vertical component. The linearly polarized light that hits the raindrops and is reflected back has only a horizontal component. Therefore, by configuring the analyzer 160 with a vertical transmission axis, the linearly polarized light with a horizontal component can be detected and filtered out, allowing only the linearly polarized light with a vertical component to pass through. This filters out the raindrop point cloud, so the receiver 140 receives point cloud data without noise.

[0079] By setting up polarizer 150 and analyzer 160, point clouds generated by raindrops in the field of view directly in front of the vehicle corresponding to the first laser can be detected and filtered out more accurately. This reduces noise in the first point cloud data at the source and improves the accuracy of detecting and filtering noise in the point cloud data. Moreover, since polarizers and analyzers are only set in the output and reflection optical paths of one laser, and not in the output and reflection optical paths of other lasers, costs are reduced, and the size of the lidar is not affected.

[0080] If it is a blind spot radar, polarizers and analyzers can be set on the output optical path and the reflected optical path corresponding to laser 3 and laser 7.

[0081] Point cloud data processing methods include the following steps:

[0082] Step 101: Obtain the first point cloud data corresponding to the first laser and the second point cloud data corresponding to the second laser.

[0083] Laser 3 in the LiDAR used to scan the area in front of the vehicle can be designated as the first laser, and lasers in other areas besides laser 3, such as laser 1, laser 2, laser 4, and laser 5, can be designated as second lasers. This results in one first laser and four second lasers. The first point cloud data can be acquired by the first receiver corresponding to the first laser, and the second point cloud data can be acquired by the second receiver corresponding to the second laser. Therefore, the corresponding point cloud data can be obtained from both the first and second receivers.

[0084] Step 102: Determine the second target point from the second point cloud data.

[0085] Determine the second target point from the second point cloud data, such as Figure 6 As shown, it includes:

[0086] Step 601: Based on the second point cloud data and the projection relationship, generate a distance-depth map in the forward-looking direction corresponding to the second point cloud data.

[0087] The grayscale value of each pixel in the distance-depth map can be used to characterize the distance of a point in the scene from the LiDAR. Based on this, the distance-depth map in the forward-looking direction can be obtained by transforming the second point cloud data through coordinate transformation.

[0088] Step 602: Generate a point cloud structure map based on the point cloud spawn order in the distance-depth map. The point cloud structure map includes a first region and a second region. The height of the points in the first region is greater than the first height, and the height of the points in the second region is less than or equal to the first height.

[0089] Within the field of view of a laser, since the light rays within that field of view are emitted by the same laser in sequence according to the laser emission interval, the point cloud received by the receiver is also obtained in sequence. Because the excitation frequencies of the fast and slow axes of the MEMS micro-mirrors inside the lidar are set differently—generally, a high-frequency sine wave excitation is applied to the fast axis, and a lower-frequency sawtooth wave excitation is applied to the slow axis—the total number of points N in the vertical direction and the total number of points M in the horizontal direction in the reflected point cloud data are fixed, and the total scanning time is generally a fixed value, typically 90ms. Therefore, the time interval between adjacent points in the horizontal direction is fixed, Δt1, and the time interval between adjacent points in the vertical direction is also fixed, Δt2. Δt1 / Δt2 are both related to the number of points M in the horizontal direction and the number of points N in the vertical direction. Based on this, a point cloud structure diagram is constructed according to the point cloud's output order. Each point in the point cloud structure contains output order information. After determining a point, points with adjacent output orders can be searched.

[0090] For raindrops floating in the air, the resulting raindrop point clouds are mainly concentrated in the upper region of the point cloud structure diagram, and are relatively sparse. For water splashed up by fast-moving vehicles, the resulting raindrop point clouds are mainly concentrated in the lower region of the point cloud structure diagram. The point cloud structure diagram includes a first region and a second region. The height of the points in the first region is greater than the first height, while the height of the points in the second region is less than or equal to the first height. The raindrop point clouds corresponding to the first region are mainly formed by raindrops floating in the air, while the raindrop point clouds corresponding to the second region are mainly formed by water splashed up by vehicles ahead.

[0091] Step 603: Determine the second target point from the first region and the second region respectively.

[0092] We can first determine the second target point from the first region. Once every second point in the first region has been determined as a second target point, we can then determine the second target point from the second region. Starting from the first point in the upper left of the first region, we determine the second target points in the first region sequentially from left to right and from top to bottom. The same applies to the second region.

[0093] Step 103: Search for multiple neighboring points that are close to the second target point from the second point cloud data.

[0094] Search for multiple neighboring points that are close to the second target point from the second point cloud data, such as... Figure 7 As shown, it includes:

[0095] Step 701: For the second target point in the first region, search for a first number of neighboring points that are close to the second target point from the point cloud structure map.

[0096] Figure 8 A schematic diagram illustrating the first nearest-neighbor search method provided in this application.

[0097] For the second target point in the first region, according to Figure 8 The method shown searches for neighboring points of the second target point. A total of 4 neighboring points were found. Since the raindrop cloud corresponding to the first region is relatively sparse, only a few neighboring points need to be searched for distance calculation, so that raindrop cloud with a distance greater than the distance threshold can be filtered out.

[0098] Wherein, point_N_M is the second target point, point_N_M-1 and point_N_M+1 are points that are adjacent to the second target point in the horizontal direction by one time interval, and point_N-1_M and point_N+1_M are points that are adjacent to the second target point in the vertical direction by one time interval.

[0099] Step 702: For the second target point in the second region, search for a second number of neighboring points that are close to the second target point from the point cloud structure map, wherein the first number is less than the second number.

[0100] Figure 9 This is a schematic diagram illustrating the second nearest-point search method provided in this application.

[0101] Wherein, point_N_M is the second target point, point_N_M-1 and point_N_M+1 are points that are adjacent to the second target point in the horizontal direction by one time interval, point_N-1_M and point_N+1_M are points that are adjacent to the second target point in the vertical direction by one time interval, and point_N-2_M and point_N+2_M are points that are adjacent to the second target point in the vertical direction by two time intervals.

[0102] Figure 10 A schematic diagram illustrating the third nearest-point search method provided in this application.

[0103] Wherein, point_N_M is the second target point, point_N_M-1 and point_N_M+1 are points that are adjacent to the second target point in the horizontal direction by one time interval, point_N-1_M and point_N+1_M are points that are adjacent to the second target point in the vertical direction by one time interval, point_N-2_M and point_N+2_M are points that are adjacent to the second target point in the vertical direction by two time intervals, and point_N-1_M-1 and point_N+1_M+1 are points that are adjacent to the second target point in both the vertical and horizontal directions by one time interval.

[0104] For the second target point in the second region, according to Figure 9 or Figure 10 Search for the nearest points of the second target point in the manner shown, according to Figure 9 The nearest neighbor search was performed in this manner, and a total of 6 nearest neighbor points were found that are close to the second target point. Figure 10 The nearest point search was performed using a specific method, finding a total of 10 nearest points to the second target point. Because the raindrop cloud corresponding to the second region is relatively dense, it is necessary to search for more nearest points for distance calculation to further improve the accuracy of raindrop cloud filtering.

[0105] Step 104: Calculate the distance between the second target point and each neighboring point.

[0106] The Euclidean distance between the second target point and each neighboring point can be calculated separately.

[0107] Step 105: When at least one distance is greater than a preset threshold, remove the second target point from the second point cloud data to obtain the updated second point cloud data.

[0108] When at least one distance is greater than a preset threshold, it indicates that the second target point is an outlier, i.e., a noise point. Therefore, the second target point needs to be removed from the second point cloud data. The preset threshold can be obtained from experiments.

[0109] Step 106: Determine the second target point from the updated second point cloud data, and return to search for multiple neighboring points that are close to the second target point from the second point cloud data, until every second point in the second point cloud data is determined to be the second target point.

[0110] After obtaining the updated second point cloud data, the system continues to determine new second target points from the updated second point cloud data. It then continues to search for multiple neighboring points that are close to the second target point from the second point cloud data, calculates the distance, and removes at least one second target point whose distance is greater than a preset threshold, until every second point in the second point cloud data is determined to be a second target point. At that point, all raindrop point clouds are filtered out.

[0111] Step 107: Determine the first point cloud data and the updated second point cloud data as the final point cloud data collected by the lidar.

[0112] Because a polarizer and analyzer are set up, the first point cloud data itself does not contain raindrop point clouds, so there is no need to process the first point cloud data. The updated second point cloud data has also had raindrop point clouds filtered out by the above method, so the second point cloud data no longer contains raindrop point clouds. Thus, neither the first point cloud data nor the updated second point cloud data contains noise, and the first point cloud data and the updated second point cloud data can be identified as the point cloud data finally collected by the lidar.

[0113] According to an embodiment of this application, a point cloud data processing method for a lidar is provided, wherein the lidar is installed on a vehicle and includes multiple lasers, including a first laser and at least one second laser. The field of view corresponding to the first laser is the area in front of the vehicle. A polarizer is provided on the laser emission path of the first laser, and an analyzer is provided on the receiving path of the corresponding channel of the first laser inside the lidar. The polarizer is used to convert the emission light into linearly polarized light, and the analyzer is used to filter the linearly polarized light. The field of view corresponding to the second laser is the area other than the area in front of the vehicle. The process involves acquiring first point cloud data corresponding to a first laser and second point cloud data corresponding to a second laser. In processing the first point cloud data corresponding to the first laser, by setting up a polarizer and analyzer, point clouds generated by raindrops in the field of view directly in front of the vehicle corresponding to the first laser can be detected and filtered out more accurately. This reduces noise in the first point cloud data at its source and improves the accuracy of detecting and filtering noise. In processing the second point cloud data corresponding to the second laser, the distance between each second target point in the second point cloud data and its multiple neighboring points is calculated to filter out at least one second target point whose distance is greater than a preset value, thereby filtering out noise in the second point cloud data. Compared with existing technologies that detect and filter noise in point cloud data by finding outliers in the global point cloud, this method reduces the computational load.

[0114] By using different processing methods for different fields of view of the lidar, the system complexity and high cost caused by using multiple polarization systems are avoided. For non-key areas, scanning is performed according to the same laser at the same time interval, and only the position of points under adjacent timestamps is determined, avoiding the time overhead caused by global search.

[0115] In some embodiments of this application, the point cloud structure diagram further includes a third region and a fourth region, wherein the height of the points in the third region is greater than the second height, the height of the points in the fourth region is less than or equal to the third height, and the height of the points in the first region is less than or equal to the second height, and the height of the points in the second region is greater than the third height, wherein the third height is less than the first height, and the first height is less than the second height.

[0116] In some embodiments of this application, the first height, the second height, and the third height are determined based on the height of the vehicle.

[0117] For example, for a typical sedan, the first, second, and third heights are determined based on the vehicle's height. The first height can be 80cm, the second height can be 3m, and the third height can be 50cm. Areas greater than 3m are considered the third region. The point cloud data in the third region has minimal impact on target recognition and therefore does not require processing. Areas less than or equal to 3m but greater than 80cm are considered the first region. The point cloud data corresponding to the first region contains relatively sparse raindrop point clouds, and therefore... Figure 8 The search is performed using the nearest neighbor search method shown; areas less than or equal to 80cm but greater than 50cm are designated as the second region. In the point cloud data corresponding to the second region, raindrop point clouds are relatively dense, and therefore... Figure 9 or Figure 10 The search is performed using the nearest neighbor method shown; areas less than or equal to 50cm are designated as the fourth region. The fourth region has a relatively small impact on target recognition, so it can be left unprocessed.

[0118] For SUVs (Sport Utility Vehicles), the body height is relatively high. The first height, second height, and third height can be determined based on the height of the SUV body, which will not be described in detail in this application.

[0119] On the other hand, embodiments of this application provide a point cloud data processing system for a lidar, including a lidar and a processor.

[0120] The lidar is installed on the vehicle and includes multiple lasers, including a first laser and at least one second laser. The field of view corresponding to the first laser is the area in front of the vehicle. A polarizer is set in the laser emission path of the first laser, and an analyzer is set in the receiving path of the corresponding channel of the first laser inside the lidar. The polarizer is used to convert the emission light into linearly polarized light, and the analyzer is used to filter the linearly polarized light. The field of view corresponding to the second laser is the area other than the area in front of the vehicle.

[0121] The processor is used to execute the point cloud data processing method of any of the above embodiments.

[0122] According to an embodiment of this application, a point cloud data processing system for a lidar is provided, wherein the lidar is installed on a vehicle and includes multiple lasers, including a first laser and at least one second laser. The field of view corresponding to the first laser is the area in front of the vehicle. A polarizer is provided on the laser emission path of the first laser, and an analyzer is provided on the receiving path of the corresponding channel of the first laser inside the lidar. The polarizer is used to convert the emitted light into linearly polarized light, and the analyzer is used to filter the linearly polarized light. The field of view corresponding to the second laser is the area other than the area in front of the vehicle. The process involves acquiring first point cloud data corresponding to a first laser and second point cloud data corresponding to a second laser. In processing the first point cloud data corresponding to the first laser, by setting up a polarizer and analyzer, point clouds generated by raindrops in the field of view directly in front of the vehicle corresponding to the first laser can be detected and filtered out more accurately. This reduces noise in the first point cloud data at its source and improves the accuracy of detecting and filtering noise. In processing the second point cloud data corresponding to the second laser, the distance between each second target point in the second point cloud data and its multiple neighboring points is calculated to filter out at least one second target point whose distance is greater than a preset value, thereby filtering out noise in the second point cloud data. Compared with existing technologies that detect and filter noise in point cloud data by finding outliers in the global point cloud, this method reduces the computational load.

[0123] On the other hand, this application provides a point cloud data processing device for a lidar. The lidar is installed on a vehicle and includes multiple lasers, including a first laser and at least one second laser. The field of view corresponding to the first laser is the area in front of the vehicle. A polarizer is provided on the laser emission path of the first laser, and an analyzer is provided on the receiving path of the corresponding channel of the first laser inside the lidar. The polarizer is used to convert the emitted light into linearly polarized light, and the analyzer is used to filter the linearly polarized light. The field of view corresponding to the second laser is the area other than the area in front of the vehicle.

[0124] Figure 11 This is a schematic diagram of the structure of a point cloud data processing device for a lidar provided in an embodiment of this application.

[0125] The point cloud data processing device includes:

[0126] The acquisition module 801 is used to acquire the first point cloud data corresponding to the first laser and the second point cloud data corresponding to the second laser.

[0127] The first determining module 802 is used to determine the second target point from the second point cloud data.

[0128] Search module 803 is used to search for multiple neighboring points that are close to the second target point from the second point cloud data.

[0129] The calculation module 804 is used to calculate the distance between the second target point and each neighboring point.

[0130] The update module 805 is used to remove the second target point from the second point cloud data when at least one distance is greater than a preset threshold, so as to obtain the updated second point cloud data.

[0131] It is also used to determine the second target point from the updated second point cloud data, and return multiple neighboring points that are close to the second target point from the second point cloud data, until every second point in the second point cloud data is determined to be the second target point.

[0132] The second determining module 806 is used to determine the first point cloud data and the updated second point cloud data as the point cloud data finally collected by the lidar.

[0133] In some embodiments of this application, the first determining module 802 includes:

[0134] The first determining unit is used to generate a distance-depth map in the forward-looking direction corresponding to the second point cloud data based on the second point cloud data and the projection relationship.

[0135] The generation unit is used to generate a point cloud structure map based on the point cloud spawn order in the distance-depth map. The point cloud structure map includes a first region and a second region. The height of the points in the first region is greater than the first height, and the height of the points in the second region is less than or equal to the first height.

[0136] The second determining unit is used to determine the second target point from the first region and the second region respectively.

[0137] In some embodiments of this application, the search module 803 includes:

[0138] The first search unit is used to search for a first number of neighboring points that are close to the second target point in the point cloud structure map for the second target point in the first region.

[0139] The second search unit is used to search for a second number of neighboring points in the point cloud structure map for a second target point in the second region, wherein the first number is less than the second number.

[0140] According to an embodiment of this application, a point cloud data processing device for a lidar is provided, wherein the lidar is installed on a vehicle and includes multiple lasers, including a first laser and at least one second laser. The field of view corresponding to the first laser is the area in front of the vehicle. A polarizer is provided on the laser emission path of the first laser, and an analyzer is provided on the receiving path of the corresponding channel of the first laser inside the lidar. The polarizer is used to convert the emitted light into linearly polarized light, and the analyzer is used to filter the linearly polarized light. The field of view corresponding to the second laser is the area other than the area in front of the vehicle. The process involves acquiring first point cloud data corresponding to a first laser and second point cloud data corresponding to a second laser. In processing the first point cloud data corresponding to the first laser, by setting up a polarizer and analyzer, point clouds generated by raindrops in the field of view directly in front of the vehicle corresponding to the first laser can be detected and filtered out more accurately. This reduces noise in the first point cloud data at its source and improves the accuracy of detecting and filtering noise. In processing the second point cloud data corresponding to the second laser, the distance between each second target point in the second point cloud data and its multiple neighboring points is calculated to filter out at least one second target point whose distance is greater than a preset value, thereby filtering out noise in the second point cloud data. Compared with existing technologies that detect and filter noise in point cloud data by finding outliers in the global point cloud, this method reduces the computational load.

[0141] Figure 12 This is a schematic diagram of the structure of a point cloud data processing device for a lidar provided in an embodiment of this application.

[0142] The point cloud data processing device may include a processor 901 and a memory 902 storing computer program instructions.

[0143] Specifically, the processor 901 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0144] Memory 902 may include mass storage for data or instructions. For example, and not limitingly, memory 902 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 902 may include removable or non-removable (or fixed) media. Where appropriate, memory 902 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 902 is non-volatile solid-state memory.

[0145] Memory may include read-only memory (ROM), random access memory (RAM), disk storage media devices, optical storage media devices, flash memory devices, and electrical, optical, or other physical / tangible memory storage devices. Therefore, typically, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the methods according to one aspect of this disclosure.

[0146] The processor 901 reads and executes computer program instructions stored in the memory 902 to implement the point cloud data processing method in the above embodiments.

[0147] In one example, the point cloud data processing device may further include a communication interface 903 and a bus 910. For example, Figure 12 As shown, the processor 901, memory 902, and communication interface 903 are connected through bus 910 and complete communication with each other.

[0148] The communication interface 903 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0149] Bus 910 includes hardware, software, or both, that couples components of a point cloud data processing device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 710 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, any suitable bus or interconnect is contemplated herein.

[0150] The point cloud data processing device executes the point cloud data processing method in the embodiments of this application, thereby achieving... Figure 5 , Figure 6 , Figure 7 The described point cloud data processing method.

[0151] Furthermore, in conjunction with the point cloud data processing methods in the above embodiments, this application embodiment can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the point cloud data processing methods in the above embodiments.

[0152] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0153] The functional blocks shown in the above-described structural diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0154] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0155] The aspects of this disclosure have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by special-purpose hardware performing the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0156] The above description is merely a specific implementation of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A method for processing point cloud data from a lidar system, characterized in that, The lidar is installed on the vehicle and includes multiple lasers, including a first laser and at least one second laser. The field of view corresponding to the first laser is the area in front of the vehicle. A polarizer is provided on the laser emission path of the first laser, and an analyzer is provided on the receiving path of the corresponding channel of the first laser inside the lidar. The polarizer is used to convert the emission light of the first laser into linearly polarized light, and the analyzer is used to filter the linearly polarized light. The field of view corresponding to the second laser is the area other than the area in front of the vehicle. The method includes: Obtain the first point cloud data corresponding to the first laser and the second point cloud data corresponding to the second laser; Determine the second target point from the second point cloud data; Search for multiple neighboring points that are close to the second target point from the second point cloud data; Calculate the distance between the second target point and each of its neighboring points; When at least one distance is greater than a preset threshold, the second target point is removed from the second point cloud data to obtain updated second point cloud data; The second target point is determined from the updated second point cloud data, and the search for multiple neighboring points that are close to the second target point is returned from the second point cloud data until every second point in the second point cloud data is determined as the second target point. The first point cloud data and the updated second point cloud data are determined as the final point cloud data collected by the lidar.

2. The point cloud data processing method according to claim 1, characterized in that, Determining the second target point from the second point cloud data includes: Based on the second point cloud data and the projection relationship, a distance-depth map corresponding to the second point cloud data in the forward-looking direction is generated; A point cloud structure map is generated based on the point cloud spawn order in the distance-depth map. The point cloud structure map includes a first region and a second region. The height of the points in the first region is greater than the first height, and the height of the points in the second region is less than or equal to the first height. The second target point is determined from the first region and the second region, respectively; The step of searching for multiple neighboring points in the second point cloud data that are close to the second target point includes: For the second target point in the first region, search for a first number of neighboring points that are close to the second target point from the point cloud structure map; For the second target point in the second region, search for a second number of neighboring points that are close to the second target point from the point cloud structure map; Wherein, the first quantity is less than the second quantity.

3. The point cloud data processing method according to claim 2, characterized in that, The point cloud structure also includes a third region and a fourth region. The height of the points in the third region is greater than the second height, the height of the points in the fourth region is less than or equal to the third height, and the height of the points in the first region is less than or equal to the second height, while the height of the points in the second region is greater than the third height. The third height is less than the first height, and the first height is less than the second height.

4. The point cloud data processing method according to claim 1, characterized in that, The step of calculating the distance between the second target point and each neighboring point includes: Calculate the Euclidean distance between the second target point and each neighboring point.

5. The point cloud data processing method according to claim 3, characterized in that, The first height, the second height, and the third height are determined based on the height of the vehicle.

6. A point cloud data processing system for a lidar system, comprising a lidar and a processor. The lidar is installed on the vehicle and includes multiple lasers, including a first laser and at least one second laser. The field of view corresponding to the first laser is the area in front of the vehicle. A polarizer is provided on the laser emission path of the first laser, and an analyzer is provided on the receiving path of the corresponding channel of the first laser inside the lidar. The polarizer is used to convert the emitted light into linearly polarized light, and the analyzer is used to filter the linearly polarized light. The field of view corresponding to the second laser is the area other than the area in front of the vehicle. The processor is used to execute the method according to any one of claims 1-5.

7. A point cloud data processing device for lidar, characterized in that, The lidar is installed on the vehicle and includes multiple lasers, including a first laser and at least one second laser. The field of view corresponding to the first laser is the area in front of the vehicle. A polarizer is provided on the laser emission path of the first laser, and an analyzer is provided on the receiving path of the corresponding channel of the first laser inside the lidar. The polarizer is used to convert the emitted light into linearly polarized light, and the analyzer is used to filter the linearly polarized light. The field of view corresponding to the second laser is the area other than the area in front of the vehicle. The device includes: The acquisition module is used to acquire the first point cloud data corresponding to the first laser and the second point cloud data corresponding to the second laser; The first determining module is used to determine the second target point from the second point cloud data; The search module is used to search for multiple neighboring points that are close to the second target point from the second point cloud data; The calculation module is used to calculate the distance between the second target point and each neighboring point respectively; The update module is used to remove the second target point from the second point cloud data when at least one distance is greater than a preset threshold, so as to obtain updated second point cloud data; The second target point is determined from the updated second point cloud data, and the search for multiple neighboring points that are close to the second target point is returned from the second point cloud data until every second point in the second point cloud data is determined as the second target point. The second determining module is used to determine the first point cloud data and the updated second point cloud data as the point cloud data finally collected by the lidar.

8. A point cloud data processing device for lidar, characterized in that, The device includes: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, it implements the point cloud data processing method as described in any one of claims 1-5.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the point cloud data processing method as described in any one of claims 1-5.

10. A computer program product, characterized in that, When the instructions in the computer program product are executed by the processor of the electronic device, the electronic device performs the point cloud data processing method as described in any one of claims 1-5.

Images