A laser radar rain and snow noise filtering method, system, medium and device

Abstract

The application provides a laser radar rain and snow noise filtering method, system, medium and equipment. Through two-dimensional ray distance image conversion on the motion-compensated point cloud data, distance information and dispersion are calculated by constructing a ray field, a ray space-time consistency index, a shelter violation residual and a ray internal shape stability index are calculated based on the distance information and the dispersion, and the noise probability and the uncertainty index of the motion point cloud data are calculated according to the ray space-time consistency index, the shelter violation residual and the ray internal shape stability index, and then the category of the point cloud is determined, thereby reducing the interference of false obstacles, and based on the ray space-time consistency and the shelter physical constraint, the adaptation ability to the long-distance sparse point cloud and the cross-domain data is stronger, thereby improving the reliability of automatic driving.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, specifically to a method, system, medium, and device for filtering rain and snow noise in lidar. Background Technology

[0002] Light Detection and Ranging (LiDAR), a core perception sensor in autonomous driving systems, actively scans the environment to acquire high-precision 3D point cloud data in real time. This point cloud data accurately depicts the 3D structure, spatial location, and semantic category of target objects, providing fundamental support for key perception tasks such as obstacle detection, target tracking, and high-precision map construction. In recent years, with the rapid development of autonomous driving technology, LiDAR has made significant progress in miniaturization, resolution improvement, and cost control, resulting in continuously enhanced hardware performance.

[0003] However, in complex environments, especially in severe weather scenarios such as snowfall, the perception performance of lidar still faces serious challenges. During snowfall, the snowflake particle size is on the same order of magnitude as the laser wavelength commonly used in lidar, triggering a strong Mie scattering effect. When the laser beam comes into contact with snowflakes, it undergoes strong scattering and partial absorption, resulting in numerous noise points in the point cloud formed by snowflake reflections. These noise points are typically sparse, close-range, and unstable in their inter-frame distribution, severely weakening the effective signal and reducing perception accuracy. They also significantly shorten the effective detection range of lidar, posing a direct threat to the safety and reliability of autonomous driving systems.

[0004] To address the aforementioned issues, some studies have attempted to incorporate deep learning methods to solve the snow noise filtering problem. However, the performance of existing deep learning models is limited by the scarcity of large-scale, diverse labeled datasets for snowy weather, and they struggle to generalize effectively in unseen environments. More critically, existing methods generally fail to fully utilize the "interpretable consistency" information provided by the continuous motion of autonomous vehicles and multi-frame observations, and they also struggle to output uncertainty metrics that can be used for redundant control in safety-critical systems. Therefore, under adverse weather conditions, existing methods are highly susceptible to false detection of obstacles or accidental deletion of real traffic participants, making it difficult to meet the stringent requirements of perception robustness and reliability for high-safety-level autonomous driving systems. Summary of the Invention

[0005] To address the aforementioned technical problems, this application is proposed. Embodiments of this application provide a method, system, medium, and device for filtering rain and snow noise in lidar.

[0006] According to one aspect of this application, a method for filtering rain and snow noise in lidar is provided, comprising: acquiring multiple frames of lidar point cloud data and the current pose information of a vehicle, and performing motion compensation on the multiple frames of lidar point cloud data based on the current pose information to obtain moving point cloud data; mapping the moving point cloud data into a two-dimensional ray distance image based on the scanning structure of the moving point cloud data; determining the ray region of the moving point cloud data based on the two-dimensional ray distance image and calculating the distance information and dispersion of the ray region; calculating a ray spatiotemporal consistency index based on the distance information and distance tolerance of the ray region; calculating the occlusion violation residual of a stable background based on the distance information and dispersion of the ray region; calculating an intra-ray morphological stability index based on the two-dimensional ray distance image; calculating a noise probability and uncertainty index of the moving point cloud data based on the ray spatiotemporal consistency index, the occlusion violation residual, and the intra-ray morphological stability index; and determining the category of the moving point cloud data based on the noise probability and uncertainty index of the moving point cloud data.

[0007] In one embodiment, the step of performing motion compensation on the multi-frame LiDAR point cloud data based on the current pose information to obtain motion point cloud data includes: performing motion compensation on each LiDAR point cloud within a set time window based on the transformation matrix corresponding to the current pose information to obtain the motion point cloud data; wherein, the motion compensation formula for the LiDAR point cloud is: ;in, To compensate for the previous k Frame number i A LiDAR point cloud, To compensate for the first k Frame number i One motion point cloud data, t This serves as the reference frame for the corresponding time window. For the first t The transpose of the frame's transformation matrix. For the first k The transformation matrix of the frame.

[0008] In one embodiment, the two-dimensional ray distance image includes the channel number and azimuth of the moving point cloud data; wherein, determining the ray region of the moving point cloud data based on the two-dimensional ray distance image and calculating the distance information and dispersion of the ray region includes: determining the ray region of the moving point cloud data based on the channel number and azimuth of the moving point cloud data; calculating the stable distance of the ray region based on the distances of all point cloud data within the ray region; and calculating the dispersion of the ray region based on the distances of all point cloud data within the ray region and the stable distance of the ray region.

[0009] In one embodiment, calculating the ray spatiotemporal consistency index based on the distance information and distance tolerance of the ray domain includes: the calculation formula for the ray spatiotemporal consistency index is: ;in, For the first i The spatiotemporal consistency index of rays in the ray domain of a point cloud. For the first t Frame number i The distance of a point cloud, For the first i The stable distance of the ray domain of a point cloud. For the first t Frame number i Distance tolerance of point clouds, ,for Linear growth term, This is a secondary growth term.

[0010] In one embodiment, calculating the occlusion violation residual of the stable background based on the distance information and dispersion of the ray domain includes: determining whether a stable background exists for the corresponding ray direction based on the dispersion of the ray domain; if a stable background exists, calculating the occlusion violation residual of the stable background based on the distance information of the ray domain; wherein the calculation formula for the occlusion violation residual is: ;in, To conceal violations of residual tolerances, For the first t Frame number i The distance of a point cloud, For the first i The stable distance of the ray domain of a point cloud. To minimize shading margin, , This is the proportionality coefficient.

[0011] In one embodiment, calculating the intra-ray morphological stability index based on the two-dimensional ray distance image includes: calculating the distance change between point clouds at adjacent azimuth angles on the same scan line based on the two-dimensional ray distance image to obtain the intra-ray morphological stability index.

[0012] In one embodiment, calculating the noise probability and uncertainty index of the moving point cloud data based on the ray spatiotemporal consistency index, the occlusion violation residual, and the ray intramorphic stability index includes: inputting the ray spatiotemporal consistency index, the occlusion violation residual, and the ray intramorphic stability index into a trained regression function to calculate the noise probability of the moving point cloud data; and calculating the variance of multiple acquisitions of the same point cloud to obtain the uncertainty index.

[0013] According to another aspect of this application, a lidar rain and snow noise filtering system is provided, comprising: a point cloud data compensation module, used to acquire multiple frames of lidar point cloud data and the current pose information of a vehicle, and to perform motion compensation on the multiple frames of lidar point cloud data based on the current pose information to obtain motion point cloud data; a ray image construction module, used to map the motion point cloud data into a two-dimensional ray distance image based on the scanning structure of the motion point cloud data; a ray neighborhood determination module, used to determine the ray neighborhood of the motion point cloud data based on the two-dimensional ray distance image and to calculate the distance information and dispersion of the ray neighborhood; and a consistency calculation module, used to... The distance information and distance tolerance of the ray domain are used to calculate the spatiotemporal consistency index of the ray; the occlusion residual calculation module is used to calculate the occlusion violation residual of the stable background based on the distance information and dispersion of the ray domain; the stability calculation module is used to calculate the intra-ray morphological stability index based on the two-dimensional ray distance image; the noise probability calculation module is used to calculate the noise probability and uncertainty index of the moving point cloud data based on the spatiotemporal consistency index of the ray, the occlusion violation residual, and the intra-ray morphological stability index; the point cloud category determination module is used to determine the category of the moving point cloud data based on the noise probability and uncertainty index of the moving point cloud data.

[0014] According to another aspect of this application, a computer-readable storage medium is provided, the storage medium storing a computer program for performing any of the methods described above.

[0015] According to another aspect of this application, an electronic device is provided, comprising: a processor; a memory for storing processor-executable instructions; the processor being configured to perform any of the methods described above.

[0016] This application provides a method, system, medium, and device for filtering rain and snow noise in lidar. It acquires multiple frames of lidar point cloud data and the current pose information of a vehicle, and performs motion compensation on the multiple frames of lidar point cloud data based on the current pose information to obtain moving point cloud data. Based on the scanning structure of the moving point cloud data, it maps the moving point cloud data into a two-dimensional ray distance image. Based on the two-dimensional ray distance image, it determines the ray region of the moving point cloud data and calculates the distance information and dispersion of the ray region. Based on the distance information and distance tolerance of the ray region, it calculates the ray spatiotemporal consistency index. Based on the distance information and dispersion of the ray region, it calculates the occlusion violation residual of the stable background. Based on the two-dimensional ray distance image, it calculates the morphological stability index within the ray. Based on the ray spatiotemporal consistency index, occlusion violation residual, and morphological stability within the ray, it calculates the morphological stability index within the ray. The system calculates the noise probability and uncertainty of moving point cloud data. Based on these metrics, it determines the category of the moving point cloud data. By performing a two-dimensional ray distance image transformation on the motion-compensated point cloud data, it constructs a ray domain to calculate distance information and dispersion. Based on this distance information and dispersion, it calculates the ray spatiotemporal consistency index, occlusion violation residual, and intra-ray morphological stability index. Then, based on these indices, it calculates the noise probability and uncertainty of the moving point cloud data, thereby determining the point cloud category. This reduces interference from false obstacles and, based on ray spatiotemporal consistency and occlusion physical constraints, demonstrates stronger adaptability to long-distance sparse point clouds and cross-domain data, thus improving the reliability of autonomous driving. Attached Figure Description

[0017] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0018] Figure 1 This is a schematic flowchart of a lidar rain and snow noise filtering method provided in an exemplary embodiment of this application.

[0019] Figure 2 This is a schematic diagram of the structure of a lidar rain and snow noise filtering system provided in an exemplary embodiment of this application.

[0020] Figure 3 This is a structural diagram of an electronic device provided in an exemplary embodiment of this application. Detailed Implementation

[0021] Hereinafter, exemplary embodiments according to this application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments of this application. It should be understood that this application is not limited to the exemplary embodiments described herein.

[0022] Figure 1 This is a schematic flowchart of a lidar rain and snow noise filtering method provided in an exemplary embodiment of this application. Figure 1 As shown, the lidar rain and snow noise filtering method includes the following steps: Step 110: Acquire multiple frames of LiDAR point cloud data and the vehicle's current pose information, and perform motion compensation on the multiple frames of LiDAR point cloud data based on the current pose information to obtain motion point cloud data.

[0023] By acquiring multiple frames of LiDAR point cloud data and the vehicle's current pose information, and performing motion compensation on the multiple frames of LiDAR point cloud data based on the current pose information, motion point cloud data is obtained.

[0024] Step 120: Based on the scanning structure of the motion point cloud data, map the motion point cloud data into a two-dimensional ray distance image.

[0025] Ray structure indexing is used for efficient and interpretable cross-frame matching on compensated point clouds. This application maps each frame of point cloud to a two-dimensional ray distance image. ,in For the ring field of point cloud data, This is a discrete azimuth index. For each pixel, the nearest echo distance and point index along the ray direction are stored first. If multiple echoes are retained, multiple candidate values ​​sorted by distance can be stored, and the minimum distance is selected as the foreground representative in subsequent statistics. This ray index structure allows cross-frame queries to be accessed in constant time on a two-dimensional grid, avoiding high-overhead structures and facilitating real-time deployment on vehicles.

[0026] Step 130: Based on the two-dimensional ray distance image, determine the ray domain of the moving point cloud data and calculate the distance information and dispersion of the ray domain.

[0027] Based on the two-dimensional ray distance image, the ray domain of the moving point cloud data is determined and the distance information and dispersion of the ray domain are calculated.

[0028] Step 140: Calculate the spatiotemporal consistency index of the ray based on the distance information and distance tolerance in the ray domain.

[0029] Based on the distance information and distance tolerance in the ray field, the spatiotemporal consistency index of the ray is calculated.

[0030] Step 150: Calculate the occlusion violation residual of the stable background based on the distance information and dispersion of the ray domain.

[0031] Based on the distance information and dispersion of the ray domain, the occlusion violation residual of the stable background is calculated.

[0032] Step 160: Calculate the morphological stability index within the ray based on the two-dimensional ray distance image.

[0033] Based on the two-dimensional ray distance image, calculate the morphological stability index within the ray.

[0034] Step 170: Calculate the noise probability and uncertainty index of the moving point cloud data based on the ray spatiotemporal consistency index, occlusion violation residual and ray intramorphic stability index.

[0035] Based on the spatiotemporal consistency index of rays, the occlusion violation residual, and the morphological stability index within rays, the noise probability and uncertainty index of moving point cloud data are calculated.

[0036] Step 180: Determine the category of motion point cloud data based on the noise probability and uncertainty index of the motion point cloud data.

[0037] The category of motion point cloud data is determined based on the noise probability and uncertainty index of the motion point cloud data.

[0038] This application provides a method for filtering rain and snow noise in lidar. It acquires multiple frames of lidar point cloud data and the vehicle's current pose information, and performs motion compensation on the multiple frames of lidar point cloud data based on the current pose information to obtain moving point cloud data. Based on the scanning structure of the moving point cloud data, it maps the moving point cloud data into a two-dimensional ray distance image. Based on the two-dimensional ray distance image, it determines the ray region of the moving point cloud data and calculates the distance information and dispersion of the ray region. Based on the distance information and distance tolerance of the ray region, it calculates the ray spatiotemporal consistency index. Based on the distance information and dispersion of the ray region, it calculates the occlusion violation residual of the stable background. Based on the two-dimensional ray distance image, it calculates the morphological stability index within the ray. Based on the ray spatiotemporal consistency index, occlusion violation residual, and morphological stability index within the ray, it calculates... The method calculates the noise probability and uncertainty index of moving point cloud data; based on the noise probability and uncertainty index of moving point cloud data, it determines the category of moving point cloud data; by performing two-dimensional ray distance image conversion on the motion-compensated point cloud data, it constructs the ray domain to calculate distance information and dispersion, and calculates the ray spatiotemporal consistency index, occlusion violation residual, and ray intramorphic stability index based on the distance information and dispersion. Then, based on the ray spatiotemporal consistency index, occlusion violation residual, and ray intramorphic stability index, it calculates the noise probability and uncertainty index of moving point cloud data, and then determines the category of point cloud, thereby reducing the interference of false obstacles. Furthermore, based on the ray spatiotemporal consistency and occlusion physical constraints, it has a stronger adaptability to long-distance sparse point clouds and cross-domain data, thereby improving the reliability of autonomous driving.

[0039] In one embodiment, step 110 can be implemented as follows: based on the transformation matrix corresponding to the current pose information, motion compensation is performed on each lidar point cloud within a set time window to obtain motion point cloud data; wherein, the motion compensation formula for the lidar point cloud is: ;in, To compensate for the previous k Frame number i A LiDAR point cloud, To compensate for the first k Frame number i One motion point cloud data, t This serves as the reference frame for the corresponding time window. For the first t The transpose of the frame's transformation matrix. For the first k The transformation matrix of the frame.

[0040] Specifically, let the current frame be the th frame. t The frame, the captured point cloud is , The total number of point clouds, where each point contains at least three-dimensional coordinates. The point cloud data typically contains a ring field (ring number), with a value ranging from 0 to 31, used to identify 32 laser beam numbers. This application directly uses this ring field as the vertical ray index. Azimuth of each point can be The calculated distance r can be The vehicle pose is obtained as a homogeneous transformation matrix, which can be derived from inertial navigation fusion positioning, wheel speed odometer, or point cloud odometer. This application employs a sliding window processing method with a window length of [missing information]. K Frame, in which K Choosing a value between 3 and 7 strikes a balance between real-time performance and robustness, and a window spanning a time range on the order of 0.3 seconds is usually sufficient to distinguish transient scattering noise from the real structure.

[0041] Multi-frame motion compensation is used to unify the point clouds of each frame within a window to a reference frame. t A coordinate system is used to eliminate relative displacement caused by vehicle movement. For the first... k any point in the frame The compensation point is obtained by transforming (homogeneous coordinate representation). Preferably, this application may also use the transformation matrix corresponding to the interpolated pose to replace the first... k The frame transformation matrix is ​​used to further suppress the disruption of consistency statistics by motion distortion within a single scan.

[0042] In one embodiment, the two-dimensional ray distance image includes the channel number and azimuth of the moving point cloud data; wherein, the specific implementation of step 130 above may be: determining the ray domain of the moving point cloud data based on the channel number and azimuth of the moving point cloud data; calculating the stable distance of the ray domain based on the distance of all point cloud data within the ray domain; and calculating the dispersion of the ray domain based on the distance of all point cloud data within the ray domain and the stable distance of the ray domain.

[0043] Specifically, for the current frame point Define its ray index as This application constructs a ray neighborhood set on each frame within the window. ,in Take 1, The value is set to 2, corresponding to the joint statistics of an adjacent beam in the vertical direction and two adjacent azimuth bins in the horizontal direction, respectively. To suppress the influence of scattering anomalies on the statistical values, this application defines the stable distance estimate using the median. ,in, The dispersion is defined by the median absolute deviation. .when When the number of samples is less than the minimum (e.g., 3), there is insufficient evidence of consistency. In such cases, the uncertainty can be increased or the conservative branch can be directly entered to avoid accidental deletion.

[0044] In one embodiment, step 140 can be specifically implemented as follows: the formula for calculating the ray spatiotemporal consistency index is: ;in, For the first i The spatiotemporal consistency index of rays in the ray domain of a point cloud. For the first t Frame number i The distance of a point cloud, For the first i The stable distance of the ray domain of a point cloud. For the first t Frame number i Distance tolerance of point clouds, ,for Linear growth term, This is a secondary growth term.

[0045] Ray spatiotemporal consistency is used to measure whether a current point can be explained by a stable surface. This application calculates the ray spatiotemporal consistency index using the ratio of distance deviation to distance tolerance, where a linearly growing term... Characterizing the random error in ranging as a function of distance, a quadratic growth term. Characterizing angle discretization, attitude interpolation residuals, and long-distance geometric magnification, we can take... , Using this as an initial value, data was collected under a clear, static scene. The calibration was performed by fitting the compensated distance error distribution of a real static surface, ensuring that the real surface satisfies or approaches 1 within a 95% confidence range. The physical meaning of this residual is clear: when it is significantly greater than 1, it indicates that the point is difficult to explain by a stable surface and is more likely to be scattering noise or a spurious echo.

[0046] In one embodiment, step 150 can be implemented as follows: based on the dispersion of the ray domain, determine whether a stable background exists for the corresponding ray direction; if a stable background exists, calculate the occlusion violation residual of the stable background based on the distance information of the ray domain; wherein, the formula for calculating the occlusion violation residual is: ;in, To conceal violations of residual tolerances, For the first t Frame number i The distance of a point cloud, For the first i The stable distance of the ray domain of a point cloud. To minimize shading margin, , This is the proportionality coefficient.

[0047] Specifically, occlusion physical constraints are used to prove the existence of outliers located before a stable background but lacking sustained occlusion behavior. First, the existence of a stable background surface is determined in the ray domain. This application uses a discreteness threshold to determine stability. It is assumed that a stable surface exists along the direction of this ray, and the background distance is defined as... The stability threshold adopts an adaptive form related to distance tolerance. , In the presence of a stable background, this application employs the minimum explainable occlusion margin. This reflects the increase in the tolerance of shielding thickness as the laser beam coverage cone grows larger with increasing distance. Take 0.02. When the condition is met... At this point, the current point is geometrically in front of the background. To reflect the continuity of occlusion, foreground support counting is introduced. ,in Indicates only by the first k The set of distances obtained from the frame neighborhood. If Therefore, it is considered that there is a lack of evidence of persistent occlusion at this close range. M Option 3 is acceptable. Occlusion violates residuals only under certain conditions. It takes effect when established; otherwise, it will occlude objects that violate the behavior of setting the residual to zero or attenuating to protect the real foreground objects. The larger the residual, the stronger the sudden forward shift of the relatively stable background at that point, and the less it conforms to the real object occlusion behavior, thus the higher the noise confidence level.

[0048] In one embodiment, the specific implementation of step 160 above may be: based on the two-dimensional ray distance image, calculate the distance change between point clouds of adjacent azimuth angles on the same scan line to obtain the morphological stability index within the ray.

[0049] In complex traffic scenarios, lidar point clouds not only contain static background surfaces but also a large number of moving targets (such as vehicles and pedestrians) and random scattering points introduced by adverse weather conditions such as rain and snow. When relying solely on multi-frame distance consistency or occlusion relationships for discrimination, moving targets, due to their distance changing over time, may be misclassified as unstable measurements; while random scattering points often exhibit instantaneous appearance and spatial disorder. To further distinguish between real object structures (including moving targets) and random scattering noise, this application introduces a discrimination method that does not rely on the assumption of constant time-distance but focuses on characterizing the stability of local spatial structures. Specifically, on the same scan line, the distance change of adjacent azimuth angles is used to define a first-order difference to characterize the structural consistency of points. The current frame ray distance image is defined as follows: Index of the current point This application defines the morphological stability index within the ray as: In actual measurements, the stability metric of real targets changes relatively little; even for moving targets, the local morphology remains consistent across adjacent azimuth angles. However, random scattering points and pseudo-echoes introduced by meteorological conditions such as rain and snow, due to the lack of physical constraints on their echo positions and morphologies, cause drastic fluctuations in the first-order difference, thus forming a significant distinguishing signal.

[0050] Since the morphological stability index within the ray depends only on the local difference relationship of the ray range image and not on point density, reflection intensity, or normal vector estimation, it is inherently robust to moving targets and can effectively suppress random scattering noise. Furthermore, its calculation process involves only local access to the ray range image, consistent with the structure of LiDAR scanning, resulting in low computational overhead and suitability for real-time implementation in autonomous driving perception systems.

[0051] In one embodiment, step 170 can be implemented by inputting the ray spatiotemporal consistency index, occlusion violation residual, and ray intramorphic stability index into a trained regression function to calculate the noise probability of the moving point cloud data; and calculating the variance of multiple acquisitions of the same point cloud to obtain the uncertainty index.

[0052] Specifically, the aforementioned ray spatiotemporal consistency index, occlusion violation residual, and ray intramorphic stability index are used to construct a low-dimensional feature vector. The noise probability is calculated. To ensure real-time performance and interpretability in the vehicle environment, logistic regression is used in this community. The formula for calculating the noise probability is as follows: ,in For the Sigmoid function, These are trainable parameters. The parameters can be trained using a self-supervised approach, that is, by automatically generating pseudo-labels using extremely consistent and extremely inconsistent samples for training. For example, when... and Simultaneously significantly large and When used as a noisy positive sample, Clearly small and with a stable background and The time is used as a clean negative sample, and thus the result is obtained by optimization using cross-entropy.

[0053] The uncertainty index is output to meet the conservative decision-making requirements of autonomous driving. This application employs multiple random inactivation inferences during the inference phase to obtain the probability distribution, thereby deriving the uncertainty index. Specifically, for the same point... The reasoning yielded the following result. The formula for calculating the uncertainty index is: ,in, Take 8 to 12.

[0054] The final decision adopts a hierarchical strategy based on probability and uncertainty indicators to balance denoising effectiveness and the risk of accidental deletion. A high threshold is defined. Low threshold With uncertainty threshold ,when and Points are identified as noise and removed when... The time point is retained, and the remaining points are retained but marked as uncertain so that downstream modules can adopt robust loss or risk mitigation strategies. .

[0055] Figure 2 This is a schematic diagram of the structure of a lidar rain and snow noise filtering system provided in an exemplary embodiment of this application. Figure 2 As shown, the lidar rain and snow noise filtering system 20 includes: a point cloud data compensation module 21, used to acquire multiple frames of lidar point cloud data and the vehicle's current pose information, and to perform motion compensation on the multiple frames of lidar point cloud data based on the current pose information to obtain motion point cloud data; a ray image construction module 22, used to map the motion point cloud data into a two-dimensional ray distance image based on the scanning structure of the motion point cloud data; a ray neighborhood determination module 23, used to determine the ray neighborhood of the motion point cloud data based on the two-dimensional ray distance image and to calculate the distance information and dispersion of the ray neighborhood; and a consistency calculation module 24, used to determine the ray neighborhood based on the ray neighborhood. The system calculates the ray spatiotemporal consistency index based on the distance information and distance tolerance of the ray domain; the occlusion residual calculation module 25 is used to calculate the occlusion violation residual of the stable background based on the distance information and dispersion of the ray domain; the stability calculation module 26 is used to calculate the morphological stability index within the ray based on the two-dimensional ray distance image; the noise probability calculation module 27 is used to calculate the noise probability and uncertainty index of the moving point cloud data based on the ray spatiotemporal consistency index, occlusion violation residual, and morphological stability index within the ray; and the point cloud category determination module 28 is used to determine the category of the moving point cloud data based on the noise probability and uncertainty index of the moving point cloud data.

[0056] This application provides a lidar rain and snow noise filtering system. A point cloud data compensation module 21 acquires multiple frames of lidar point cloud data and the vehicle's current pose information, and performs motion compensation on the multiple frames of lidar point cloud data based on the current pose information to obtain moving point cloud data. A ray image construction module 22 maps the moving point cloud data into a two-dimensional ray distance image based on the scanning structure of the moving point cloud data. A ray neighborhood determination module 23 determines the ray neighborhood of the moving point cloud data based on the two-dimensional ray distance image and calculates the distance information and dispersion of the ray neighborhood. A consistency calculation module 24 calculates the ray spatiotemporal consistency index based on the distance information and distance tolerance of the ray neighborhood. An occlusion residual calculation module 25 calculates the occlusion violation residual of the stable background based on the distance information and dispersion of the ray neighborhood. A stability calculation module 26 calculates the intra-ray morphological stability index based on the two-dimensional ray distance image. A noise probability calculation module 27... The noise probability and uncertainty index of moving point cloud data are calculated based on the spatiotemporal consistency index of rays, occlusion violation residuals, and intra-ray morphological stability index. The point cloud category determination module 28 determines the category of moving point cloud data based on the noise probability and uncertainty index of moving point cloud data. By performing two-dimensional ray distance image conversion on the motion-compensated point cloud data, distance information and dispersion are calculated by constructing the ray domain. Based on the distance information and dispersion, the spatiotemporal consistency index of rays, occlusion violation residuals, and intra-ray morphological stability index are calculated. Based on the spatiotemporal consistency index of rays, occlusion violation residuals, and intra-ray morphological stability index, the noise probability and uncertainty index of moving point cloud data are calculated, and the category of point cloud is determined. This reduces the interference of false obstacles. Furthermore, based on the spatiotemporal consistency index of rays and the physical constraints of occlusion, it has a stronger adaptability to long-distance sparse point clouds and cross-domain data, thereby improving the reliability of autonomous driving.

[0057] In one embodiment, the point cloud data compensation module 21 can be further configured to: perform motion compensation on each lidar point cloud within a set time window based on the transformation matrix corresponding to the current pose information, to obtain motion point cloud data; wherein, the motion compensation formula for the lidar point cloud is: ;in, To compensate for the previous k Frame number i A LiDAR point cloud, To compensate for the first k Frame number i One motion point cloud data, t This serves as the reference frame for the corresponding time window. For the first t The transpose of the frame's transformation matrix. For the first k The transformation matrix of the frame.

[0058] In one embodiment, the two-dimensional ray distance image includes the channel number and azimuth of the moving point cloud data; wherein, the ray domain determination module 23 can be further configured to: determine the ray domain of the moving point cloud data based on the channel number and azimuth of the moving point cloud data; calculate the stable distance of the ray domain based on the distance of all point cloud data within the ray domain; and calculate the dispersion of the ray domain based on the distance of all point cloud data within the ray domain and the stable distance of the ray domain.

[0059] In one embodiment, the consistency calculation module 24 can be further configured such that the calculation formula for the ray spatiotemporal consistency index is: ;in, For the first i The spatiotemporal consistency index of rays in the ray domain of a point cloud. For the first t Frame number i The distance of a point cloud, For the first i The stable distance of the ray domain of a point cloud. For the first t Frame number i Distance tolerance of point clouds, ,for Linear growth term, This is a secondary growth term.

[0060] In one embodiment, the occlusion residual calculation module 25 can be further configured to: determine whether a stable background exists for the corresponding ray direction based on the dispersion of the ray domain; if a stable background exists, calculate the occlusion violation residual of the stable background based on the distance information of the ray domain; wherein, the calculation formula for the occlusion violation residual is: ;in, To conceal violations of residual tolerances, For the first t Frame number i The distance of a point cloud, For the first i The stable distance of the ray domain of a point cloud. To minimize shading margin, , This is the proportionality coefficient.

[0061] In one embodiment, the stability calculation module 26 can be further configured to: calculate the distance change between point clouds of adjacent azimuth angles on the same scan line based on the two-dimensional ray distance image, and obtain the morphological stability index within the ray.

[0062] In one embodiment, the noise probability calculation module 27 can be further configured to: input the ray spatiotemporal consistency index, occlusion violation residual and ray intramorphic stability index into a trained regression function to calculate the noise probability of the moving point cloud data; calculate the variance of multiple acquisitions of the same point cloud to obtain the uncertainty index.

[0063] Below, for reference Figure 3 This application describes an electronic device according to embodiments thereof. The electronic device may be either or both of a first device and a second device, or a standalone device independent of them, which may communicate with the first device and the second device to receive acquired input signals from them.

[0064] Figure 3 A block diagram of an electronic device according to an embodiment of this application is illustrated.

[0065] like Figure 3 As shown, the electronic device 10 includes one or more processors 11 and memory 12.

[0066] The processor 11 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.

[0067] The memory 12 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 11 may execute the program instructions to implement the methods of the various embodiments of this application described above and / or other desired functions. Various contents such as input signals, signal components, and noise components may also be stored in the computer-readable storage medium.

[0068] In one example, the electronic device 10 may also include an input device 13 and an output device 14, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0069] When the electronic device is a standalone device, the input device 13 can be a communication network connector for receiving the collected input signals from the first device and the second device.

[0070] In addition, the input device 13 may also include, for example, a keyboard, a mouse, etc.

[0071] The output device 14 can output various information to the outside, including determined distance information, direction information, etc. The output device 14 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0072] Of course, for the sake of simplicity, Figure 3 Only some of the components of the electronic device 10 relevant to this application are shown in this illustration; components such as buses, input / output interfaces, etc., are omitted. In addition, the electronic device 10 may include any other suitable components depending on the specific application.

[0073] In addition to the methods and apparatus described above, embodiments of this application may also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform the steps in the methods according to various embodiments of this application described in the "Exemplary Methods" section above.

[0074] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0075] Furthermore, embodiments of this application may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps in the methods according to various embodiments of this application described in the "Exemplary Methods" section above.

[0076] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0077] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the application to the necessity of employing the aforementioned specific details for implementation.

[0078] The block diagrams of devices, apparatuses, devices, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0079] It should also be noted that in the apparatus, equipment, and methods of this application, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions of this application.

[0080] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0081] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this application to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A method for filtering rain and snow noise in lidar, characterized in that, include: Acquire multiple frames of lidar point cloud data and the vehicle's current pose information, and perform motion compensation on the multiple frames of lidar point cloud data based on the current pose information to obtain motion point cloud data; Based on the scanning structure of the motion point cloud data, the motion point cloud data is mapped into a two-dimensional ray distance image; Based on the two-dimensional ray distance image, the ray region of the moving point cloud data is determined and the distance information and dispersion of the ray region are calculated; Based on the distance information and distance tolerance in the ray field, the spatiotemporal consistency index of the ray is calculated. Based on the distance information and dispersion of the ray domain, the occlusion violation residual of the stable background is calculated; Based on the two-dimensional ray distance image, the morphological stability index within the ray is calculated; Based on the spatiotemporal consistency index of the ray, the occlusion violation residual and the intra-ray morphological stability index, the noise probability and uncertainty index of the moving point cloud data are calculated. The category of the motion point cloud data is determined based on the noise probability and uncertainty index of the motion point cloud data.

2. The lidar rain and snow noise filtering method according to claim 1, characterized in that, The motion compensation process based on the current pose information to obtain motion point cloud data from the multi-frame LiDAR point cloud data includes: Based on the transformation matrix corresponding to the current pose information, motion compensation is performed on each lidar point cloud within a set time window to obtain the motion point cloud data; wherein, the motion compensation formula for the lidar point cloud is: ； in, To compensate for the previous k Frame number i A LiDAR point cloud, To compensate for the first k Frame number i One motion point cloud data, t The reference frame for the corresponding time window. For the first t The transpose of the frame's transformation matrix. For the first k The transformation matrix of the frame.

3. The lidar rain and snow noise filtering method according to claim 1, characterized in that, The two-dimensional ray distance image includes the channel number and azimuth of the moving point cloud data; wherein, the step of determining the ray region of the moving point cloud data based on the two-dimensional ray distance image and calculating the distance information and dispersion of the ray region includes: Based on the channel number and azimuth of the motion point cloud data, the ray domain of the motion point cloud data is determined; Based on the distances of all point cloud data within the ray field, calculate the stable distance of the ray field; The dispersion of the ray field is calculated based on the distances of all point cloud data within the ray field and the stable distance of the ray field.

4. The lidar rain and snow noise filtering method according to claim 1, characterized in that, The calculation of the spatiotemporal consistency index of the ray based on the distance information and distance tolerance in the ray field includes: The formula for calculating the spatiotemporal consistency index of the ray is: ； in, For the first i The spatiotemporal consistency index of rays in the ray domain of a point cloud. For the first t Frame number i The distance of a point cloud, For the first i The stable distance of the ray domain of a point cloud. For the first t Frame number i Distance tolerance of point clouds, ,for Linear growth term, This is a secondary growth term.

5. The lidar rain and snow noise filtering method according to claim 1, characterized in that, The calculation of the occlusion violation residual of the stable background based on the distance information and dispersion of the ray domain includes: Based on the dispersion of the ray field, determine whether there is a stable background for the corresponding ray direction; If a stable background exists, the occlusion violation residual of the stable background is calculated based on the distance information in the ray field. The formula for calculating the occlusion violation residual is as follows: ； in, To conceal violations of residual tolerances, For the first t Frame number i The distance of a point cloud, For the first i The stable distance of the ray domain of a point cloud. To minimize shading margin, , This is the proportionality coefficient.

6. The lidar rain and snow noise filtering method according to claim 1, characterized in that, The calculation of the intra-ray morphological stability index based on the two-dimensional ray distance image includes: Based on the two-dimensional ray distance image, the distance change between point clouds at adjacent azimuth angles on the same scan line is calculated to obtain the morphological stability index within the ray.

7. The lidar rain and snow noise filtering method according to claim 1, characterized in that, The calculation of the noise probability and uncertainty index of the moving point cloud data based on the spatiotemporal consistency index of the ray, the occlusion violation residual, and the intra-ray morphological stability index includes: The spatiotemporal consistency index of the ray, the occlusion violation residual and the intra-ray morphological stability index are input into the trained regression function to calculate the noise probability of the moving point cloud data. The variance of multiple data collections of the same point cloud is calculated to obtain the uncertainty index.

8. A lidar rain and snow noise filtering system, characterized in that, include: The point cloud data compensation module is used to acquire multiple frames of lidar point cloud data and the vehicle's current pose information, and to perform motion compensation on the multiple frames of lidar point cloud data based on the current pose information to obtain motion point cloud data. A ray image construction module is used to map the motion point cloud data into a two-dimensional ray distance image based on the scanning structure of the motion point cloud data; A ray domain determination module is used to determine the ray domain of the moving point cloud data based on the two-dimensional ray distance image and to calculate the distance information and dispersion of the ray domain; The consistency calculation module is used to calculate the spatiotemporal consistency index of the ray based on the distance information and distance tolerance of the ray field; The occlusion residual calculation module is used to calculate the occlusion violation residual of the stable background based on the distance information and dispersion of the ray domain. The stability calculation module is used to calculate the morphological stability index within the ray based on the two-dimensional ray distance image. The noise probability calculation module is used to calculate the noise probability and uncertainty index of the moving point cloud data based on the ray spatiotemporal consistency index, the occlusion violation residual and the ray intramorphic stability index. The point cloud category determination module is used to determine the category of the moving point cloud data based on the noise probability and uncertainty index of the moving point cloud data.

9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program for performing the method described in any one of claims 1-7.

10. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is used to execute the method described in any one of claims 1-7.

Images