Rapid rainy day laser radar point cloud generation method

Abstract

The invention discloses a rapid rainy day laser radar point cloud generation method, which comprises the following steps of S1, deploying a laser radar in a real rainy day environment, and collecting original point cloud data containing raindrops; s2, collecting laser radar point cloud data in sunny days; s3, aiming at each effective point in the point cloud in the sunny day, taking the corresponding laser beam emission direction as an axis, and constructing a slender prism region around the axis in a three-dimensional space; and S4, for each sub-region, calculating the number of raindrops with different diameters in unit volume of air under the current rainfall rate by using a raindrop particle size distribution function according to the rainfall rate specified by a user, and normalizing the number as the conditional probability that the raindrops with specific diameters are hit by the laser beam in the sub-region. According to the invention, starting from the data acquired in the real scene of the sunny day, the data of the rainy day is generated on the basis, so that the automatic driving algorithm for training the generated data can be directly applied to the actual scene.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and in particular to a rapid method for generating lidar point clouds in rainy weather. Background Technology

[0002] Autonomous driving technology is currently in the advanced driver assistance stage. To advance this technology, testing and validation of algorithms in various driving scenarios is essential. Currently, LiDAR data for rainy conditions only appears in limited quantities in publicly available datasets, and even if automakers collect relevant data, it cannot cover all driving scenarios.

[0003] Human drivers constantly monitor their surroundings while driving, noting the presence of other vehicles and pedestrians. For LiDAR sensors in autonomous vehicles, the system constructs a three-dimensional image of the surrounding environment by emitting and receiving laser beams. While not as detailed as what the naked eye sees, the general outlines of objects are obtained. Rainy day LiDAR data is typically collected directly from the LiDAR itself during rainfall within the corresponding driving scenario. However, autonomous driving technology requires testing in a wide variety of scenarios, and direct LiDAR data collection cannot cover all rainy conditions. Therefore, the mainstream technology utilizes simulators to construct virtual test scenarios, generating raindrops within these scenarios, and then emitting laser beams from the simulated LiDAR to generate simulated rainy day LiDAR data.

[0004] Existing technologies primarily rely on simulators, which utilize computer graphics and the physical properties of electromagnetic wave reflection and diffraction. Using game engines like Unreal Engine 4, they generate driving scenarios that closely resemble real-world conditions. When computational resources are abundant, these simulated scenarios are visually indistinguishable to the human eye. Currently, game scenarios are all implemented using simulators. The closest technical solution to this invention is PCSim, released by the Shanghai Artificial Intelligence Laboratory. Its core idea is to use the CARLA simulator to generate autonomous driving test scenarios such as roads, buildings, and vehicles with textures similar to real-world object surfaces. It also generates raindrops splashed up by wheels driving over rainy surfaces within these scenarios, simulating the emission of laser beams from a LiDAR system in the virtual environment. When the intensity of the reflected laser beam exceeds a given threshold, the reflection point is considered valid, and its 3D position information is read and written into the LiDAR data. Furthermore, PCSim uses neural networks to further refine the intensity information in the LiDAR data, making the values ​​even closer to those actually collected.

[0005] Current technologies rely on simulators, which are entirely software-based. The generated data does not match the distribution of real-world data, thus affecting the accuracy of autonomous driving algorithms. In particular, while test scenarios generated by game engines are realistic, they only reflect human vision. Furthermore, simulated scenarios cannot perfectly replicate real-world scenes, especially in terms of the actual reflectivity of surrounding vehicles and the true texture of the road. Therefore, the data collected by LiDAR in simulated scenarios is not accurate for autonomous driving algorithms. Summary of the Invention

[0006] To address the shortcomings of existing technologies, the purpose of this invention is to provide a rapid method for generating rainy-day lidar point clouds. Based on collected clear-day data, this method utilizes probability and the characteristics of lidar's active laser beam emission to generate corresponding rainy-day environmental data, thus solving the problem of incomplete coverage by lidar data in rainy weather. To achieve the above objective and other advantages, this invention provides a rapid method for generating rainy-day lidar point clouds, comprising: S1. Deploy lidar in a real rainy environment to collect raw point cloud data containing raindrops and calculate the spatial distribution probability of raindrops; S2. Collect point cloud data from lidar in clear weather; S3. For each valid point in the clear sky point cloud, with its corresponding laser beam emission direction as the axis, construct a slender prism region around the axis in three-dimensional space. The side length of the cross-section of the region is greater than the spot diameter of the laser beam at that distance. Divide the region into several equally spaced sub-regions along the laser beam propagation direction. S4. For each sub-region, based on the arbitrary rainfall rate specified by the user, use the raindrop size distribution function to calculate the number of raindrops of different diameters in a unit volume of air at the current rainfall rate, and normalize the number as the conditional probability that a raindrop of a specific diameter will be hit by the laser beam in that sub-region. S5. Based on the technical specifications of the lidar equipment, determine the upper and lower limits of its effective detection range; within the defined slender prism area, according to the calculated raindrop spatial distribution probability model, randomly sample the sub-region index corresponding to a potential raindrop location; at the same time, within the effective range of the raindrop diameter, randomly select a raindrop diameter according to the particle size distribution model, calculate the prior probability of the existence of a raindrop at that location and the conditional probability of having the selected diameter at that location, and the product of the two constitutes the joint probability that the raindrop is effectively detected by the lidar. S6. For each laser beam path, if the calculated joint probability is greater than the preset probability threshold, it is determined that the laser beam interacts with the raindrop during propagation. S7. For the newly added raindrop reflection points, the intensity of their echo is calculated using the intensity attenuation model; S8. For the original sunny day points that have not been replaced by raindrops, perform the rain intensity correction operation; S9. Encapsulate the above processing flow into a parallel-executable computing unit, and simultaneously perform raindrop insertion, point removal, and intensity recalculation operations on all laser beam paths in the point cloud. S10: Output complete point cloud data simulated by the rain effect.

[0007] Preferably, in step S1, a lidar is used in a rainy test scenario to collect driving scenario data for a specific lidar, and then the probability function of raindrop distribution is statistically obtained based on the raindrop position and intensity information in the lidar data.

[0008] Preferably, step S2 specifically includes using a lidar to collect test scene data on a sunny day, or using sunny scene data from a publicly available dataset, and then obtaining the theoretical emission angle of the laser beam of the corresponding lidar based on the lidar model used for data collection.

[0009] Preferably, step S3 specifically includes traversing the points in the lidar clear sky data, and for each point, there is a laser beam emitted by the lidar; in three-dimensional space, starting from the lidar, a cuboid with a minimum length and width is constructed along the laser beam, where the laser beam is the height and the laser beam is located at the center of this cuboid.

[0010] Preferably, step S4 specifically includes, for each small cuboid, calculating the parameter value of the formula using Ulbricht's gamma distribution function or Marshall-Palmer's exponential distribution function for a given arbitrary rainfall rate.

[0011] Preferably, for a given raindrop diameter, the ratio of the number of raindrops within a unit cube to the total number of raindrops is used as the conditional probability that a raindrop of a given diameter will be hit by a laser beam.

[0012] Preferably, step S5 specifically includes determining the total number of small cuboids as N based on the maximum detection distance given in the lidar manual; and randomly selecting a value in the interval [1, N] according to the probability function of raindrop distribution, and defining this value as the position O of the raindrop.

[0013] Preferably, a value is randomly selected between the minimum and maximum values ​​of the raindrop, and this value is defined as the diameter D of the raindrop. The ratio between the position O of the raindrop and the total number N is denoted as the probability P(O) that the raindrop is hit at position O. The ratio of the number of raindrops with diameter D in a unit cube to the total number of raindrops is denoted as the conditional probability P(D|O) that the raindrop has diameter D when it appears at position O.

[0014] Preferably, for each laser beam, P(D|O)P(O) is calculated, and a probability threshold is set based on empirical values.

[0015] Compared with the prior art, the advantages and positive effects of the present invention are: This invention directly generates rainy day data based on sunny day data, using probability and intensity formulas. The advantages include: (1) Data collected on a sunny day includes the error of the lidar itself, so this type of error does not need to be simulated, but the simulator needs to simulate this type of error. (2) Data collected on sunny days corresponds to the real three-dimensional world. Data generated on rainy days is simply replacing some points of objects with raindrops and generating new reflection intensity values. Therefore, it can reflect the real scene, but the simulator is troubled by this. (3) The method proposed in this invention can be implemented directly on the CPU without relying on the GPU, reducing the overhead of computing resources and achieving the purpose of rapid generation, while the simulator must use the GPU due to rendering requirements; (4) In addition to the data collected by itself, there is also data from various public datasets, so the scope is wider. When the simulator simulates a single scene, it needs to manually build the simulated scene first, which requires high manpower and other resources and has extremely low generation efficiency. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] Relevant terms used in this application: Autonomous driving: Based on ordinary vehicles, various types of sensors, such as lidar and cameras, are installed to perceive the surrounding environment. Then, algorithms are used to identify static and dynamic obstacles in the surrounding environment, enabling the vehicle to achieve conventional driving functions such as autonomous obstacle avoidance and automatic parking without a driver.

[0018] LiDAR: LiDAR is one of the key sensors for autonomous driving. It emits laser beams into the surrounding environment and receives the laser beams reflected by objects in the environment to calculate the position of objects and the intensity of their reflection, thereby characterizing environmental information during driving.

[0019] Conditional probability: Given that event A has occurred, the probability that event B, which is associated with event A, will occur.

[0020] Ulbricht's gamma distribution function: used to describe the distribution characteristics of raindrops, the specific formula is as follows: .

[0021] Here, D represents the diameter of the raindrop, and N0 is an empirical constant. and It is a parameter calculated from an empirical formula related to rainfall rate, where N(D) represents the number of raindrops of a given diameter D within a unit cube. And when... When it is 0, the above The formula degenerates into the Marshall-Palmer exponential distribution function.

[0022] Strength formula: Here, d represents the distance between a point in three-dimensional space and the lidar. I(d) represents the reflection intensity at a distance d, and C is a constant related to the characteristics of the lidar. Let represent the average reflectance at a distance d, and r represent the rainfall rate. For simplicity, here... It can be further simplified to an empirical constant value.

[0023] This invention abandons the idea of ​​using simulators to simulate test scenarios, and can obtain near-realistic rainy-day LiDAR data. Simulator-generated test scenarios, besides not being able to perfectly replicate real-world objects, resulting in significant differences between the generated and actual data distributions, also require GPU rendering to render the simulated scene. This invention, however, directly generates rainy-day data based on sunny-day data, using probability and intensity formulas.

[0024] A fast method for generating point clouds for lidar in rainy weather, comprising: (1) Using LiDAR in a rainy test scenario, driving scenario data is collected for a specific LiDAR. Then, based on the raindrop position and intensity information in the LiDAR data, the detection distance of the LiDAR is divided into uniform intervals, and the number of raindrops in each interval is counted. Thus, the probability function of raindrop distribution is obtained, which solves the drawback of the existing method of using uniform distribution, which results in unrealistic raindrop distribution. For LiDAR data in sunny and rainy weather, in addition to the data collected by ourselves, we can also use publicly available autonomous driving datasets, such as nuScenes, to count the probability of raindrop distribution in the publicly available datasets, or directly use the probability distribution in our own collected data. Using the method mentioned in this invention, rainy weather data can be generated.

[0025] (2) Use a lidar to collect test scene data on a clear day, or use clear day scene data from a publicly available dataset, and then obtain the theoretical emission angle of the laser beam of the corresponding lidar based on the lidar model used for data collection. If copyright licensing is involved and the theoretical emission angle of the laser beam cannot be obtained, the emission angle of the corresponding laser beam can be calculated using the polar coordinate calculation formula based on the three-dimensional coordinates of each point in the actual collected data.

[0026] (3) Traverse the points in the lidar data under clear weather conditions. For each point, there is a laser beam emitted by the lidar. In three-dimensional space, starting from the lidar, construct a cuboid with a minimum length and width along the laser beam, where the laser beam is the height and is located at the center of the cuboid. Based on this, continuously cut the cuboid into multiple smaller cuboids with a height of 1 along the direction of the laser beam. Compared with the previous cuboid, the smaller cuboids are all 1 in height. If the height of the last smaller cuboid is less than 1, then automatically extend the length of the laser beam to ensure that the height of the last smaller cuboid is also 1. The cuboids and smaller cuboids here are viewed from the lidar's perspective.

[0027] (4) For each small cuboid, for a given rainfall rate, the parameter values ​​of the formula are calculated using empirical formulas based on Ulbricht's gamma distribution function or Marshall-Palmer's exponential distribution function. For a given raindrop diameter, the ratio of the number of raindrops in the unit cube to the total number of raindrops is used as the conditional probability that a raindrop of a given diameter will be hit by the laser beam. In addition to using Ulbricht's gamma distribution function or Marshall-Palmer's exponential distribution function, other functions used to describe the characteristics of raindrop distribution, including existing ones and those derived by ourselves, can replace the distribution function in (4).

[0028] (5) Based on the minimum and maximum detection distances given in the lidar manual, determine the total number of small cuboids as N. According to the probability function in (1), randomly select a value in the interval [1, N] and define this value as the position O of the raindrop. At the same time, randomly select a value between the minimum and maximum values ​​of the raindrop and define this value as the diameter D of the raindrop. Based on this, the ratio between the position O of the raindrop and the total number N is denoted as the probability P(O) of the raindrop being hit at position O, and the ratio of the number of raindrops with diameter D in the unit cube to the total number of raindrops is denoted as the conditional probability P(D|O) of the raindrop having diameter D when it appears at position O. Therefore, the probability of the raindrop appearing at position O and having diameter D is the product of the above two probabilities, denoted as P(D|O)P(O).

[0029] (7) For each laser beam, calculate P(D|O)P(O) and set a probability threshold based on empirical values. If the calculated P(D|O)P(O) is greater than the set probability threshold, and compared with the point position collected on the laser beam under sunny conditions, the raindrop is closer to the lidar than the collected point, then it is considered that under rainy conditions, the laser beam that should have hit the object point in three-dimensional space hit the raindrop during propagation and returned the position and reflection intensity of the raindrop to the lidar. Therefore, for the object point in the sunny data, the corresponding rainy data does not exist.

[0030] (8) For each laser beam, after obtaining the position of the raindrop, calculate the reflection intensity of the raindrop. According to the intensity formula in the terminology, given the distance d of the raindrop and the rainfall rate r, calculate the reflection intensity value of the raindrop.

[0031] (9) For each laser beam, the original point that has not been replaced by raindrops can be recalculated using the intensity formula, or an empirical coefficient less than 1 can be used to directly multiply the intensity value of a sunny day to obtain the corresponding reflection intensity value of a rainy day.

[0032] (10) The above process is for each laser beam. In actual implementation, in order to speed up the calculation, parallel computing can be used to process these laser beams simultaneously.

[0033] Besides replacing existing points with raindrops, we can also add laser beams that should exist in the sunny data but are directly ignored by the lidar due to weak reflection intensity or other factors, based on the theoretical angle of the laser beam. Then, according to Part 6, we can determine whether raindrops appear on these laser beams. If so, we add them to the rainy data.

[0034] In summary, the present invention has the following effects: (1) It gets rid of the dependence on simulators and no longer simulates the whole process of raindrops being hit. Instead, it uses probability to determine whether raindrops are hit and reflected back to lidar. (2) Using real sunny day data, which naturally covers a wide variety of driving scenarios around the world due to the existence of public datasets; (3) It is implemented directly on the CPU without relying on the GPU, which reduces the computational overhead and ensures the rapid generation of data.

[0035] The number of devices and processing scale described herein are for simplification purposes. Applications, modifications, and variations of this invention will be readily apparent to those skilled in the art. Although embodiments of the invention have been disclosed above, they are not limited to the applications listed in the specification and embodiments. It can be applied to various fields suitable for this invention, and further modifications can be readily implemented by those skilled in the art. Therefore, without departing from the general concept defined by the claims and their equivalents, this invention is not limited to the specific details and those shown herein.

Claims

1. A rapid method for generating point clouds using lidar in rainy weather, characterized in that, Includes the following steps: S1. Deploy lidar in a real rainy environment to collect raw point cloud data containing raindrops and calculate the spatial distribution probability of raindrops; S2. Collect point cloud data from lidar in clear weather; S3. For each valid point in a clear sky cloud, with its corresponding laser beam emission direction as the axis, construct a slender prism region around the axis in three-dimensional space. The side length of the cross-section of the region is greater than the spot diameter of the laser beam at that distance. The region is divided into several equally spaced sub-regions along the direction of laser beam propagation; S4. For each sub-region, based on the arbitrary rainfall rate specified by the user, use the raindrop size distribution function to calculate the number of raindrops of different diameters in a unit volume of air at the current rainfall rate, and normalize the number as the conditional probability that a raindrop of a specific diameter will be hit by the laser beam in that sub-region. S5. Based on the technical specifications of the lidar equipment, determine the upper and lower limits of its effective detection range; within the defined slender prism area, according to the calculated raindrop spatial distribution probability model, randomly sample the sub-region index corresponding to a potential raindrop location; at the same time, within the effective range of the raindrop diameter, randomly select a raindrop diameter according to the particle size distribution model, calculate the prior probability of a raindrop existing at that location and the conditional probability of having the selected diameter at that location, and the product of the two constitutes the joint probability that the raindrop is effectively detected by the lidar. S6. For each laser beam path, if the calculated joint probability is greater than the preset probability threshold, it is determined that the laser beam interacts with the raindrop during propagation. S7. For the newly added raindrop reflection points, the intensity of their echo is calculated using the intensity attenuation model; S8. For the original sunny day points that have not been replaced by raindrops, perform the rain intensity correction operation; S9. Encapsulate the above processing flow into a parallel-executable computing unit, and simultaneously perform raindrop insertion, point removal, and intensity recalculation operations on all laser beam paths in the point cloud. S10: Output complete point cloud data simulated by the rain effect.

2. The method for rapid rain-day lidar point cloud generation as described in claim 1, characterized in that, In step S1, a lidar is used in a rainy test scenario to collect driving scenario data for a specific lidar. Then, based on the raindrop position and intensity information in the lidar data, a probability function of raindrop distribution is statistically obtained.

3. The rapid rain-day lidar point cloud generation method as described in claim 1, characterized in that, Step S2 specifically includes using a lidar to collect test scene data on a sunny day, or using sunny scene data from a publicly available dataset, and then obtaining the theoretical emission angle of the laser beam of the corresponding lidar based on the lidar model used for data collection.

4. The method for rapid rain-day lidar point cloud generation as described in claim 1, characterized in that, Step S3 specifically includes traversing the points in the lidar clear sky data, and for each point, there is a laser beam emitted by the lidar; in three-dimensional space, starting from the lidar, a cuboid with a minimum length and width is constructed along the laser beam, where the laser beam is the height and the laser beam is located at the center of this cuboid.

5. The rapid rain-day lidar point cloud generation method as described in claim 1, characterized in that, Step S4 specifically includes, for each small cuboid, for a given arbitrary rainfall rate, calculating the parameter values ​​of the formula using Ulbricht's gamma distribution function or Marshall-Palmer's exponential distribution function.

6. The rapid rain-day lidar point cloud generation method as described in claim 5, characterized in that, For a given raindrop diameter, the ratio of the number of raindrops of that diameter within a unit cube to the total number of raindrops is used as the conditional probability that a raindrop of that diameter will be hit by a laser beam.

7. The rapid rain-day lidar point cloud generation method as described in claim 5, characterized in that, Step S5 specifically includes determining the total number of small cuboids as N based on the maximum and minimum detection distances given in the lidar manual; and randomly selecting a value in the interval [1, N] according to the probability function of raindrop distribution, and defining this value as the position O of the raindrop.

8. A rapid method for generating point clouds for rainy lidar as described in claim 7, characterized in that, A random value is also selected between the minimum and maximum values ​​of the raindrop, and this value is defined as the diameter D of the raindrop. The ratio between the position O of the raindrop and the total number N is denoted as the probability P(O) that the raindrop is hit at position O. The ratio of the number of raindrops with diameter D in a unit cube to the total number of raindrops is denoted as the conditional probability P(D|O) that the raindrop has diameter D given that it appears at position O.

9. A rapid method for generating point clouds for rainy lidar as described in claim 8, characterized in that, For each laser beam, calculate P(D|O)P(O) and set a probability threshold based on empirical values.