Lidar point cloud abnormal data processing method and device

Abstract

The application provides a laser radar point cloud abnormal data processing method and device, image information is converted into a pseudo-depth map by acquiring laser radar point cloud and image information of a target area. The difference between each three-dimensional point in the laser radar point cloud and the neighborhood point in the neighborhood in the geometric feature is obtained, and the voting score of the three-dimensional point is obtained based on the difference. The real depth order of the three-dimensional point in the laser radar point cloud is obtained, and the pseudo-depth order of the three-dimensional point after being mapped to the pseudo-depth map is obtained. The depth difference is obtained based on the real depth order and the pseudo-depth order. Whether the three-dimensional point is an abnormal point is judged by combining the voting score and the depth difference. In the scheme, the pseudo-depth map is introduced on the basis of the neighborhood voting, the depth consistency is combined for abnormal point detection, the abnormal point can be effectively identified, and the defect that the denoising effect is poor in the prior art which only depends on the laser radar point cloud is avoided.

Description

Technical Field

[0001] This invention relates to the field of data processing, and more specifically, to a method and apparatus for processing abnormal point cloud data from lidar. Background Technology

[0002] LiDAR point clouds are needed in scenarios such as autonomous driving, robot perception, and 3D reconstruction. However, the collected LiDAR point clouds often contain some outliers, such as multipath points and noisy points. Therefore, it is necessary to identify and effectively remove these outliers from the LiDAR point clouds.

[0003] In existing technologies, methods for identifying and removing outliers in LiDAR point clouds mainly include statistical outlier removal methods, radius outlier removal methods, and robot learning-based filtering methods. Statistical outlier removal methods identify outliers by calculating the statistical characteristics of the distance between a point and its neighbors. Radius outlier removal methods filter points by determining the number of neighbors within a specified radius. In addition, there are deep learning-based methods that learn the features of the point cloud to achieve noise reduction.

[0004] Existing lidar point cloud denoising methods rely solely on lidar point cloud data. When faced with multipath effects, adverse weather conditions (such as rain or fog), or inherent sensor noise, they struggle to effectively distinguish between real and outlier points, resulting in unreliable denoising performance. Summary of the Invention

[0005] The purpose of this invention is to provide a method and apparatus for processing abnormal LiDAR point cloud data, which can effectively identify abnormal points and avoid the shortcomings of existing methods that rely solely on LiDAR point clouds and have poor noise reduction effects.

[0006] In a first aspect, the present invention provides a method for processing abnormal point cloud data from a lidar system, the method comprising: Acquire lidar point cloud and image information of the target area and convert the image information into a pseudo depth map; For each three-dimensional point in the lidar point cloud, the differences in geometric features between the three-dimensional point and its neighboring points are obtained, and the voting score of the three-dimensional point is obtained based on the differences. The true depth order of the three-dimensional points in the lidar point cloud is obtained, and the pseudo depth order of the three-dimensional points after being mapped to the pseudo depth map is obtained. The depth difference is obtained based on the true depth order and the pseudo depth order. The three-dimensional point is determined to be an anomaly by combining the voting score and the depth difference.

[0007] In an optional implementation, the method further includes: For 3D points identified as abnormal, at least one of the set verification methods is used to verify the 3D points; If the verification result indicates that the three-dimensional point is abnormal, then the determination that the three-dimensional point is an abnormal point is maintained. If the verification results show that the three-dimensional point is not abnormal, then the three-dimensional point will be corrected to a normal point.

[0008] In an optional implementation, the step of obtaining the geometric differences between the 3D point and its neighboring points includes: The geometric features of the three-dimensional point and its neighboring points are obtained as well as the differences in the consistency of the normal vectors and the differences in the continuity of the intensity. By combining the differences in dispersion, normal vector consistency, and intensity continuity, a comprehensive difference is obtained.

[0009] In an optional implementation, the dispersion difference is obtained in the following manner: Calculate the distance between the 3D point and its neighboring points within its neighborhood; calculate the mean and standard deviation of all distances; and obtain the dispersion difference based on the distances, mean distance, and standard deviation. The normal vector consistency difference is obtained in the following way: Calculate the local principal normal vector of the neighboring points within the neighborhood of the three-dimensional point; calculate the angle between the normal vector of each neighboring point and the local principal normal vector; obtain the consistency difference of the normal vector based on the angle; The intensity continuity difference is obtained in the following way: The strength, mean strength, and standard deviation of the strength of the neighboring points within the neighborhood of the three-dimensional point are obtained; the strength deviation rate is calculated based on the strength, mean strength, and standard deviation of the strength; and the strength continuity difference is obtained based on the strength deviation rate.

[0010] In an optional implementation, the step of obtaining the true depth order of the three-dimensional points in the lidar point cloud includes: Obtain the true depth value of the three-dimensional point and each neighboring point within its neighborhood range in the lidar point cloud; The three-dimensional point and its neighboring points are sorted based on their actual depth values. Obtain the sorted position of the three-dimensional points, and obtain the true depth sort of the three-dimensional points based on the sorted position.

[0011] In an optional implementation, the step of obtaining the pseudo-depth sort of the 3D points after mapping to the pseudo-depth map includes: The laser radar point cloud is projected onto the image coordinate system to obtain the pixel coordinates of the corresponding pixel points of each three-dimensional point in the image coordinate system. Obtain the pseudo-depth value in the pseudo-depth map of the pixel coordinates corresponding to the three-dimensional point and its neighboring points within its neighborhood range. The three-dimensional point and its neighboring points are sorted based on their pseudo-depth values. Obtain the sorting position of the three-dimensional points, and obtain the pseudo-depth sorting of the three-dimensional points based on the sorting position.

[0012] In an optional implementation, the multiple verification methods include an isolated verification method; The steps for verifying the three-dimensional points using an isolated verification method include: Count the number of neighboring points that are identified as abnormal points within the neighborhood of the three-dimensional point; Based on the quantity, determine the proportion of outliers within the neighborhood range, and obtain the nearest distance between the three-dimensional point and the nearest outlier within the neighborhood range; The three-dimensional point is verified as an anomaly by combining the percentage of outliers and the nearest distance.

[0013] In an optional implementation, the multiple verification methods include an intensity clustering verification method; The steps for verifying the three-dimensional points using an intensity clustering verification method include: Obtain the intensity data of the neighboring points that are determined to be non-anomalies within the neighborhood of the three-dimensional point; Clustering is performed based on the intensity data to obtain multiple intensity cluster centers; Calculate the distance between the intensity value of the three-dimensional point and each intensity cluster center, and verify whether the three-dimensional point is an outlier based on the distance.

[0014] In an optional implementation, the multiple verification methods include spatial correlation verification methods; The steps for verifying the three-dimensional points using a spatial correlation verification method include: The true depth information of the three-dimensional point and its neighboring points within its neighborhood range in the lidar point cloud is obtained, as well as the pseudo depth information mapped to the pseudo depth map. Based on the pseudo-depth information, pseudo-depth fluctuation values ​​and pseudo-depth trends are obtained, and based on the real depth information, real depth trends are obtained. By combining the pseudo-depth trend, the true depth trend, and the pseudo-depth fluctuation value, it is verified whether the three-dimensional point is an anomaly.

[0015] In a second aspect, the present invention provides a lidar point cloud anomaly data processing device, the device comprising: The acquisition module is used to acquire the lidar point cloud of the target area and the image information of the target area, and convert the image information into a pseudo-depth map; The voting score acquisition module is used to obtain the difference in geometric features between each three-dimensional point and its neighboring points in the lidar point cloud, and to obtain the voting score of the three-dimensional point based on the difference. The depth difference acquisition module is used to obtain the true depth order of the three-dimensional points in the lidar point cloud, and to obtain the pseudo depth order of the three-dimensional points after mapping to the pseudo depth map, and to obtain the depth difference based on the true depth order and the pseudo depth order. The judgment module is used to determine whether the three-dimensional point is an anomaly point by combining the voting score and the depth difference.

[0016] This invention provides a method and apparatus for processing abnormal data from LiDAR point clouds. It acquires LiDAR point clouds and image information of a target area, converting the image information into a pseudo-depth map. The method obtains the differences in geometric features between each 3D point in the LiDAR point cloud and its neighboring points, and calculates a voting score for each 3D point based on these differences. It then obtains the true depth ranking of the 3D points in the LiDAR point cloud and the pseudo-depth ranking of the 3D points after mapping to the pseudo-depth map. The method calculates the depth difference based on the true depth ranking and the pseudo-depth ranking. Finally, it combines the voting score and the depth difference to determine whether a 3D point is an anomaly. This solution introduces a pseudo-depth map based on neighborhood voting to combine depth consistency for anomaly detection, effectively identifying anomalies and avoiding the poor denoising effect of existing methods that rely solely on LiDAR point clouds. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments of the present invention will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A structural block diagram of an electronic device provided in an embodiment of the present invention; Figure 2 A flowchart of a lidar point cloud anomaly data processing method provided in an embodiment of the present invention; Figure 3 for Figure 2 A flowchart of the sub-steps included in S12; Figure 4 for Figure 2 A flowchart of the sub-steps included in S13; Figure 5 for Figure 2 A flowchart of the sub-steps included in S14; Figure 6 A flowchart of the verification method in the lidar point cloud abnormal data processing method provided in the embodiments of the present invention; Figure 7 This is a functional block diagram of a lidar point cloud anomaly data processing device provided in an embodiment of the present invention. Detailed Implementation

[0019] The technical solutions of the present invention will now be described with reference to the accompanying drawings in the embodiments of the present invention.

[0020] Please see Figure 1 The present invention provides an electronic device, and the lidar point cloud anomaly data processing method provided in the present invention can be applied to this electronic device. The electronic device includes a memory, a processor, and a communication module. The memory, processor, and communication module are electrically connected directly or indirectly to each other to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines.

[0021] The memory is used to store programs or data. The memory can be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc.

[0022] The processor is used to read / write data or programs stored in memory and to perform corresponding functions.

[0023] The communication module is used to establish communication connections between electronic devices and other communication terminals via a network, and to send and receive data via the network.

[0024] It should be understood that, Figure 1 The structure shown is only a schematic diagram of an electronic device; the electronic device may also include components that are larger than those shown. Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown. Figure 1 The components shown can be implemented using hardware, software, or a combination thereof.

[0025] In some embodiments, the electronic device can be mounted on a carrier equipped with devices such as cameras and lidar, and can serve as the control system or data processing system of that carrier to execute the lidar point cloud anomaly data processing method provided in this embodiment of the invention, and to perform real-time detection and identification of anomalies for better subsequent applications. The carrier can be, but is not limited to, subways, robots, automobiles, aircraft, etc.

[0026] The following, combined with Figure 2 The method for processing abnormal point cloud data of lidar provided in the embodiments of the present invention will be described. Figure 2 This is a flowchart of a method for processing abnormal point cloud data provided by an embodiment of the present invention. The method for processing abnormal point cloud data includes the following steps: S11: Acquire the lidar point cloud and image information of the target area, and convert the image information into a pseudo depth map; S12: For each 3D point in the lidar point cloud, obtain the difference in geometric features between the 3D point and its neighboring points, and obtain the voting score of the 3D point based on the difference. S13, obtain the true depth order of 3D points in the lidar point cloud; S14, obtain the pseudo-depth sort of the 3D points after mapping to the pseudo-depth map; S15, depth difference is obtained based on real depth sorting and pseudo depth sorting; S16, combine voting scores and depth differences to determine whether a 3D point is an outlier.

[0027] In one possible scenario, such as a vehicle or robot in motion, images and lidar point clouds of a target area are collected along the direction of travel using cameras and lidar on the vehicle. The areas collected by the cameras and lidar are the same.

[0028] For the obtained lidar point cloud, the lidar point cloud is first preprocessed, including invalid point filtering, coordinate transformation and voxelization.

[0029] In the invalid point filtering process, the ranging range of the lidar can be obtained, for example, from 0.5m to 100m. Three-dimensional points whose coordinate values ​​are not within this ranging range are considered invalid points. Therefore, three-dimensional points whose coordinate values ​​are not within the ranging range can be identified as invalid points and filtered out. Furthermore, the reflection intensity of each three-dimensional point can be obtained, and three-dimensional points with reflection intensity less than a preset value can be filtered out. This eliminates sensor noise inherent in the lidar itself.

[0030] In coordinate transformation processing, the extrinsic parameters of the lidar can be used to transform the lidar point cloud from the sensor coordinate system to the world coordinate system. The coordinate transformation can be characterized as follows: P world = R×P lidar + T Among them, P world P represents the coordinate values ​​in the world coordinate system. lidar R and T represent the coordinate values ​​in the sensor coordinate system, and R and T represent the external parameters of the lidar.

[0031] In voxelization, the voxel size can be dynamically adjusted according to the density of local point clouds in the lidar point cloud. For example, the voxel size is set to be smaller in areas with higher density (e.g., 0.03m to 0.05m), and the voxel size is set to be larger in areas with lower density (e.g., 0.08m to 0.1m). The density of the local point cloud can be obtained by the number of 3D points within the local point cloud.

[0032] By adaptively adjusting the voxel size, the corresponding local point cloud is voxelized. This avoids the problems of missed multipath points or accidental deletion of real points caused by using a fixed voxel size. In this embodiment, voxelizing the lidar point cloud can reduce the amount of subsequent processing while preserving the geometric and spatial distribution features of the point cloud.

[0033] In addition, in this embodiment, image preprocessing is performed on the obtained image information. This can be done by first performing distortion processing on the image information, such as removing distorted pixels at the image edges.

[0034] Based on this, image information can be converted into pseudo-depth maps. Deep learning networks can be used to convert RGB format image information into pseudo-depth maps. A pseudo-depth map is a depth map predicted based on a 2D image, reflecting the relative distances between points in a scene.

[0035] To eliminate scale differences in the output of deep learning networks, the pseudo-depth values ​​of each pixel in the pseudo-depth map can be normalized to a uniform range, such as [0,1]. A pseudo-depth value closer to 0 indicates that the corresponding pixel is closer to the carrier, while a pseudo-depth value closer to 1 indicates that the corresponding pixel is farther from the carrier.

[0036] Furthermore, in this embodiment, bilateral filtering can be applied to the pseudo-depth map. The filtering parameters are as follows: a filtering window of 5×5, color standard deviation sigma_color=0.1, and spatial standard deviation sigma_space=2. This filtering process preserves edge features in the pseudo-depth map and suppresses isolated noise points, preventing misidentification as multipath points due to pseudo-depth noise.

[0037] Based on this, the coordinates of each valid pixel (u, v) in the pseudo-depth map are converted to relative 3D reference coordinates using the camera's intrinsic parameters. The conversion formula is as follows: X_rel = (u - c x )×D_norm / (f x ) Y_rel = (v - c )×D_norm / (f ) Z_rel = D_norm Where D_norm is the normalized pseudo-depth value, X_rel, Y_rel, and Z_rel represent the transformed coordinate values ​​relative to the 3D reference coordinates, and c x f x c f This refers to the camera's internal parameters.

[0038] Based on the extrinsic parameters of the camera and LiDAR, the relative three-dimensional reference coordinates are transformed to the world coordinate system, thereby generating a pseudo LiDAR reference point cloud.

[0039] Based on the above, for each 3D point in the lidar point cloud, a voting score that can characterize its degree of anomaly is obtained based on the difference in geometric features between it and its neighboring points in the neighborhood.

[0040] Specifically, please refer to Figure 3 In this embodiment, the step of obtaining the geometric differences between a 3D point and its neighboring points can be achieved in the following way: S121, obtain the geometric feature differences, normal vector consistency differences, and intensity continuity differences between a 3D point and its neighboring points; S122, combining the differences in dispersion, the differences in normal vector consistency, and the differences in intensity continuity, yields a comprehensive difference.

[0041] In lidar point clouds, anomalous points (such as multipath points) typically exhibit spatially discrete distribution, disordered surface normal vectors, and abnormal emission intensity compared to true points (i.e., non-anomalous points). Based on this, this embodiment calculates the differences in geometric features, normal vector consistency, and intensity continuity between a 3D point and its neighboring points, and then combines these three types of differences to obtain a comprehensive difference.

[0042] The dispersion difference is obtained in the following way: Calculate the distance between a 3D point and its neighboring points within its neighborhood; calculate the mean and standard deviation of all distances; and obtain the dispersion difference based on the distances, mean distances, and standard deviations.

[0043] Specifically, for any 3D point Pi to be determined in the LiDAR point cloud, a search is conducted within the LiDAR point cloud for neighboring points centered on Pi. The number of neighboring points can be set according to requirements, for example, any number between 15 and 40. For any neighboring point Pj found, Pj has real spatial coordinates and reflection intensity, which is used to provide local spatial feature references for the multipath determination of Pi.

[0044] In this embodiment, when performing neighborhood point search, a KD tree can be constructed for the three-dimensional points in the lidar point cloud, and the neighborhood point search can be performed based on the KD tree.

[0045] Calculate the distances between a 3D point Pi and its neighboring points Pj. For example, the Euclidean distance can be calculated, and the resulting distance is denoted as d_ij. Then, the average of all distances within the obtained neighborhood is calculated to obtain the mean distance d_mean. In addition, the standard deviation can be calculated to obtain the standard deviation of the distance d_std.

[0046] Based on the obtained distance, mean distance, and standard deviation of distance, the dispersion difference is calculated using the following formula: W_spread = max (0, 1 - |d_ij - d_mean| / (2×d_std)) The dispersion difference obtained from the above formula can be used as the subsequent dispersion weight coefficient. The above formula can be understood as follows: if the distance between a 3D point and its neighboring points is closer to the mean distance, the spatial dispersion is lower, the dispersion weight coefficient is larger, and the probability that the 3D point is a real point is higher.

[0047] Furthermore, the normal vector consistency difference is obtained in the following way: Calculate the local principal normal vector of the neighboring points within the neighborhood of the 3D point; calculate the angle between the normal vector of each neighboring point and the local principal normal vector; obtain the consistency difference of the normal vector based on the angle.

[0048] In this embodiment, for the neighboring points within the searched neighborhood range, all neighboring points constitute a neighborhood point set. The local principal normal vector N_main of the neighboring points in the neighborhood point set can be calculated using the covariance matrix.

[0049] In this embodiment, the search of the neighborhood point set can be determined based on the average distance obtained above. For example, it can be the average distance of a set multiple, such as 0.3, etc., to adapt to different local point cloud densities.

[0050] For each neighboring point Pj, the normal vector of the neighboring point Pj can be obtained, and the angle θ_ij between the normal vector of the neighboring point Pj and the local principal normal vector N_main can be calculated, where the obtained angle θ_ij ∈ [0, 90°].

[0051] Based on the obtained angle, the consistency difference of the normal vector is calculated using the following formula: W_normal = cos (θ_ij×π / 180) The above formula can be understood as follows: the smaller the included angle, the higher the consistency of the normal vector, and the higher the probability that the three-dimensional point Pi is the real point.

[0052] In this embodiment, the intensity continuity difference is obtained in the following way: The strength, mean strength, and standard deviation of the strength of the neighboring points within the neighborhood of the three-dimensional point are obtained; the strength deviation rate is calculated based on the strength, mean strength, and standard deviation of the strength; and the strength continuity difference is obtained based on the strength deviation rate.

[0053] In this embodiment, the intensity of each neighboring point within the neighborhood range is obtained; this intensity is the reflection intensity. The mean intensity I_mean is obtained by averaging the intensities of all neighboring points, and the standard deviation I_std is calculated based on the standard deviation of the intensities of all neighboring points. The intensity deviation rate is obtained based on the intensity of the neighboring points, the mean intensity, and the standard deviation of the intensity using the following formula: δ_j = |I_j - I_mean| / I_std Based on the obtained strength deviation rate, the strength continuity difference is obtained according to the following formula: W_intensity = max (0, 1 - δ_j) The above formula can be understood as follows: if the strength deviation rate is less than 1 and the difference between it and 1 is larger, then the strength continuity difference is positive and the larger the value, the higher the probability that the three-dimensional point is the real point.

[0054] After obtaining the dispersion difference, normal vector consistency difference, and intensity continuity difference through the above methods, these three types of differences can be weighted and summed to obtain the comprehensive difference, as shown below: W_ij = 0.4×W_spread + 0.3×W_normal + 0.3×W_intensity Then, based on the comprehensive differences obtained, the voting scores for the three-dimensional points are calculated, as shown below: S_i = (1 / K)×Σ(W_ij) Where S_i ∈ [0,1], and K is the number of neighborhood points. The lower S_i is, the higher the probability that the 3D point P_i is a multipath point. For example, the S_i corresponding to a true point is usually greater than 0.5, while the S_i corresponding to a multipath point is usually less than 0.4.

[0055] The above is a voting score based on the differences in geometric features between a 3D point and its neighboring points within its range. This voting score, to some extent, characterizes whether a 3D point exhibits anomalies. However, when faced with factors such as multipath effects, severe weather, or inherent sensor noise, relying solely on the lidar point cloud data for anomaly identification is insufficient to effectively distinguish between real and anomalous points. For instance, multipath points may resemble real points in geometric features, making misjudgments likely based solely on lidar point cloud neighborhood information.

[0056] Based on the above considerations, in this embodiment, a pseudo-depth map is introduced, and the depth information provided by the pseudo-depth map is combined to obtain the depth difference between the 3D points in the lidar point cloud and the pseudo-depth map. This depth difference is then used to achieve accurate identification of anomalies.

[0057] In this embodiment, the true depth ranking of 3D points in the lidar point cloud is obtained. Please refer to [link / reference needed]. Figure 4 Specifically, this step can be achieved in the following way: S131, obtain the true depth value of a 3D point and its neighboring points in the lidar point cloud. S132, Sort the 3D point and its neighboring points based on the true depth values ​​of the 3D point and each neighboring point; S133, obtain the sorted position of the 3D points, and obtain the true depth sort of the 3D points based on the sorted position.

[0058] In this embodiment, in the lidar point cloud, for a three-dimensional point Pi, the true depth value of the three-dimensional point Pi and each neighboring point within its neighborhood range is obtained. The true depth value can be obtained based on the three-dimensional coordinates of the three-dimensional point and each neighboring point.

[0059] Sort the 3D point and its neighboring points in ascending or descending order of their true depth values. Obtain the sorted positions of the 3D points and determine their true depth order based on these positions.

[0060] For example, if the number of neighboring points K is 20, and assuming they are sorted in ascending order of their true depth values, if the 3D point Pi is the 8th in its sorted order, then its true depth R_real = 8 / 20 = 0.4. The smaller the true depth value of a 3D point, the closer its actual distance.

[0061] In addition, please see Figure 5The step of obtaining the pseudo-depth order of 3D points after mapping to the pseudo-depth map can be achieved in the following way: S141, Project the LiDAR point cloud onto the image coordinate system to obtain the pixel coordinates of each 3D point in the image coordinate system; S142, obtain the pseudo-depth value of the pixel coordinates of the three-dimensional point and its neighboring points in the pseudo-depth map. S143, Sort the 3D point and its neighboring points based on the pseudo-depth values ​​of the 3D point and its neighboring points; S144, obtain the sorting position of the 3D points, and obtain the pseudo-depth sort of the 3D points based on the sorting position.

[0062] In this embodiment, for each 3D point Pi in the lidar point cloud, based on the 3D coordinates (X, Y, Z) of the 3D point, and by projecting the 3D point back onto the image coordinate system using camera intrinsic parameters, the pixel coordinates (u_i, v_i) corresponding to the 3D point are obtained, which can be represented as follows: u_i = (X×f x ) / Z + c x v_i = (Y×f ) / Z + c

[0063] The algorithm checks whether the projected pixel coordinates are within the valid range of the pseudo-depth map. If they are, it obtains the normalized pseudo-depth value D_norm for that pixel coordinate, which reflects the relative position of the 3D point in the scene. Furthermore, if the pixel coordinates exceed the valid range of the pseudo-depth map, it indicates that the 3D point has no relative position reference in the pseudo-depth map, and anomaly detection methods for isolated points can be used for subsequent anomaly assessment.

[0064] Furthermore, in this embodiment, the pseudo-depth values ​​of neighboring points within the neighborhood of a 3D point in the pseudo-depth map can also be obtained. The pseudo-depth values ​​of the 3D point and each neighboring point are sorted in ascending or descending order to obtain the sorted position of the 3D point, and then the pseudo-depth sort of the 3D point is obtained based on the sorted position.

[0065] For example, if the sorting is done in ascending order, and the 3D point is the 15th position in the sorting, then its pseudo-depth sorting R_fake = 15 / 20 = 0.75.

[0066] After obtaining the true depth sort R_real and pseudo depth sort R_fake of the 3D points using the above method, calculate the depth difference ΔR_i between the true depth sort and the pseudo depth sort: ΔR_i=|R_real - R_fake| The depth difference between the true depth sort and the pseudo depth sort can, to some extent, indicate whether a 3D point is an anomaly. For example, the depth difference corresponding to a multipath point is usually significantly greater than the depth difference corresponding to a true point. If the depth difference ΔR_i corresponding to a true point is usually less than 0.15, the depth difference corresponding to a multipath point is usually greater than 0.25.

[0067] Therefore, the following judgment rules can be set: If ΔR_i ≤ 0.15: the relative positions are consistent, denoted as C_i=1 (the probability that the three-dimensional point is the real point is high); If 0.15 < ΔR_i ≤ 0.25: a slight contradiction, denoted as C_i = 0.5 (the three-dimensional point may be a suspicious point and needs further confirmation); If ΔR_i > 0.25: This is a serious contradiction, and is denoted as C_i=0 (the probability that the three-dimensional point is a multipath point is high).

[0068] It should be noted that the values ​​in the above judgment rules can be dynamically adjusted based on the actual situation, and this embodiment does not limit this.

[0069] After obtaining the voting scores and depth differences for the 3D points using the methods described above, we can combine these two factors to determine whether a 3D point is an outlier. For example, if both the voting score and depth difference indicate that a 3D point is an outlier, then it can be determined that the 3D point is an outlier. If both the voting score and depth difference indicate that a 3D point is not an outlier, then it can be determined that the 3D point is not an outlier. If either the voting score or the depth difference indicates that a 3D point is an outlier, then the 3D point can be marked as an outlier, and further confirmation can be performed using other methods.

[0070] Please see Figure 6 To further improve the accuracy of anomaly identification, the anomaly data processing method provided in this embodiment may further include the following steps: S17, For 3D points that are determined to be abnormal, at least one of the set verification methods is used to verify the 3D points; S18. If the verification result shows that the three-dimensional point is abnormal, then the judgment result of the three-dimensional point being judged as an abnormal point shall be maintained. S19. If the verification result shows that there are no abnormalities in the three-dimensional points, then the three-dimensional points are corrected to normal points.

[0071] In this embodiment, for three-dimensional points that are initially identified as abnormal points through the above methods, one or more of the various verification methods can be used to verify the determination result of the three-dimensional points, thereby improving the accuracy of the final result.

[0072] In this embodiment, the various verification methods include isolation verification, intensity clustering verification, and spatial correlation verification.

[0073] The method of verifying 3D points using isolated verification can be implemented in the following ways: The number of neighboring points identified as anomalies within the neighborhood of the three-dimensional point is counted; the proportion of anomalies within the neighborhood is determined based on the number, and the nearest distance between the three-dimensional point and the nearest anomaly within the neighborhood is obtained; the three-dimensional point is verified as an anomaly by combining the proportion of anomalies and the nearest distance.

[0074] In this embodiment, for the three-dimensional points that are initially identified as anomalous points, the number of neighboring points identified as anomalous points within their neighborhood range, such as a circle with a radius of 1m, is counted. The percentage of anomalous points within the neighborhood range can be obtained by dividing this number by the total number of all neighboring points within the neighborhood range.

[0075] In addition, the nearest distance between a 3D point and anomalies within its neighborhood is obtained.

[0076] The 3D point is validated by combining the percentage of outliers and the nearest distance. For example, if the percentage of outliers is less than a preset percentage (e.g., 30%), and the nearest distance to the 3D point is greater than a preset distance (e.g., 0.5m), then the 3D point is considered an anomaly. However, in this context, to avoid mistakenly deleting truly isolated points, such as small object points or thin rod structure points, which are spatially discrete but not necessarily multipath points, the validation result indicates that the 3D point is anomaly. Therefore, it can be temporarily left unfiltered from the point cloud and further validated using certain methods later.

[0077] Furthermore, if the proportion of outliers is greater than or equal to a preset proportion, or the nearest distance to a 3D point is less than or equal to a preset distance, then the 3D point can be determined to be abnormal. This is typically a dense cluster of multipath points, as multipath points tend to form clusters near the reflecting surface, a situation consistent with the physical formation of multipath points. If the verification characterization of the 3D point shows an anomaly, the judgment result for the 3D point can be maintained, meaning the 3D point is determined to be an outlier.

[0078] In this embodiment, the intensity clustering verification method is used to verify the three-dimensional points, which can be achieved in the following way: The intensity data of the neighboring points of the three-dimensional point that are determined to be non-anomalies are obtained within the neighborhood of the three-dimensional point; clustering is performed based on the intensity data to obtain multiple intensity cluster centers; the distance between the intensity value of the three-dimensional point and each intensity cluster center is calculated, and the three-dimensional point is verified as an anomaly based on the distance.

[0079] For 3D points identified as outliers using the above methods, such as isolated suspicious points, intensity data (reflection intensity) of neighboring points that have been identified as non-outliers can be extracted. Clustering methods (such as K-means) are then used to cluster the intensity data, obtaining multiple intensity cluster centers. Assuming the number of clusters is 2, two intensity cluster centers, I_cluster1 and I_cluster2, can be obtained.

[0080] Obtain the intensity data I_suspect of the 3D points, and calculate the distance between the intensity data of the 3D points and each intensity cluster center. For example, the distances obtained are as follows: d1 = |I_suspect - I_cluster1| d2 = |I_suspect - I_cluster2| The anomalies of 3D points are verified by checking the distances obtained. For example, the smaller distance among several obtained distances can be determined and compared with a set distance, which is derived from the intensity standard deviation of non-anomaly points within the neighborhood. If the smaller distance is less than the set distance, it indicates that the intensity of the 3D point belongs to the intensity cluster of non-anomaly points, and the 3D point can be corrected to a non-anomaly point. If the smaller distance is greater than or equal to the set distance, it indicates that the intensity of the 3D point deviates from the intensity characteristics of non-anomaly points, and the determination that the 3D point is an anomaly point can be maintained.

[0081] In this embodiment, spatial correlation verification is used to verify the three-dimensional points, which can be achieved in the following way: The true depth information of the three-dimensional point and its neighboring points within its range in the lidar point cloud is obtained, as well as the pseudo depth information mapped to the pseudo depth map; based on the pseudo depth information, pseudo depth fluctuation value and pseudo depth trend are obtained, and based on the true depth information, true depth trend is obtained; combining the pseudo depth trend, true depth trend and pseudo depth fluctuation value, it is verified whether the three-dimensional point is an anomaly.

[0082] In this embodiment, the pseudo-depth fluctuation value can be obtained based on the pseudo-depth information of the three-dimensional point and its neighboring points in the pseudo-depth map, and the pseudo-depth trend can be obtained. For example, the pseudo-depth trend can be a certain gradual trend from near to far.

[0083] Similarly, the true depth trend can be based on the three-dimensional points and their neighboring points in the lidar point cloud.

[0084] If the pseudo-depth fluctuation value is less than the preset fluctuation value (e.g., 0.1), it indicates that the relative position in the neighborhood is stable and there is no sudden change. Furthermore, if the true depth trend is consistent with the pseudo-depth trend, the three-dimensional point can be corrected to a non-anomaly point.

[0085] If the pseudo-depth fluctuation value is greater than or equal to the preset fluctuation value, or if the true depth trend is opposite to the pseudo-depth trend, it indicates that the three-dimensional point is abnormal. In this case, the judgment result that the three-dimensional point is an abnormal point can be maintained.

[0086] The LiDAR point cloud anomaly data processing method provided in this embodiment uses a fusion of LiDAR point cloud data and pseudo-depth maps for anomaly point identification, improving the robustness of multimodal fusion. In scenarios such as severe weather or sparse LiDAR point cloud data, the supplementation with pseudo-depth maps ensures that the point cloud denoising effect is not limited by the data quality of the LiDAR point cloud itself. By fusing the depth continuity information from the pseudo-depth map, anomalies, especially multipath points, can be effectively distinguished.

[0087] Furthermore, the neighborhood voting mechanism combined with deep consensus allows for the preservation of geometric details of the point cloud, such as edges and corners, while removing outliers.

[0088] In this solution, an adaptive neighborhood range size and threshold design are adopted, which improves the retention rate of real points by more than 15% and the removal rate of abnormal points by 10%-20% in non-uniform density point clouds (such as indoor near-dense and far-sparse scenes).

[0089] In summary, this solution addresses the inaccurate anomaly identification issues in existing LiDAR point clouds caused by multipath effects and sensor noise, achieving efficient removal of multipath points and noisy points. It overcomes the limitations of denoising methods based on neighborhood information from a single LiDAR point cloud in non-uniform density point clouds and complex scenes, improving the adaptability of denoising methods to different densities and scenarios.

[0090] This scheme utilizes multimodal fusion of calibrated pseudo-depth maps and LiDAR point clouds to introduce depth continuity and semantic information, assisting in the identification of anomalous points in the LiDAR point cloud. It achieves robust removal of various types of anomalous points in complex indoor and outdoor scenes (such as scenes containing reflective surfaces and dynamic targets), while preserving the geometric details and structural integrity of the point cloud.

[0091] To execute the corresponding steps in the above-described embodiments and possible methods for processing abnormal lidar point cloud data, an implementation of a lidar point cloud abnormal data processing device is provided below. Optionally, this lidar point cloud abnormal data processing device can employ the above-described... Figure 1 The device structure of the electronic device shown.

[0092] Further, please refer to Figure 7 , Figure 7 This is a functional block diagram of a lidar point cloud anomaly data processing device provided in an embodiment of the present invention. It should be noted that the basic principle and technical effects of the lidar point cloud anomaly data processing device provided in this embodiment are the same as those in the corresponding method embodiments described above. For the sake of brevity, any parts not mentioned in this embodiment can be referred to the corresponding content in the aforementioned method embodiments. The lidar point cloud anomaly data processing device includes: The acquisition module is used to acquire the lidar point cloud of the target area and the image information of the target area, and convert the image information into a pseudo-depth map; The voting score acquisition module is used to obtain the difference in geometric features between each three-dimensional point and its neighboring points in the lidar point cloud, and to obtain the voting score of the three-dimensional point based on the difference. The depth difference acquisition module is used to obtain the true depth order of the three-dimensional points in the lidar point cloud, and to obtain the pseudo depth order of the three-dimensional points after mapping to the pseudo depth map, and to obtain the depth difference based on the true depth order and the pseudo depth order. The judgment module is used to determine whether the three-dimensional point is an anomaly point by combining the voting score and the depth difference.

[0093] Optionally, the above modules can be stored in the form of software or firmware. Figure 1 The memory shown is either stored in or embedded in the operating system (OS) of the electronic device, and can be... Figure 1 The processor executes the commands. Meanwhile, the data and program code required to execute these modules can be stored in memory.

[0094] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative; for example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of the invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0095] In addition, the functional modules in the various embodiments of the present invention can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0096] If the functionality is implemented as a software module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes: USB flash drive, portable hard drive, read-only memory (ROM). Various media that can store program code, such as only memory, random access memory (RAM), magnetic disks or optical disks.

[0097] The above are merely preferred embodiments of the present invention and are not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for processing anomalous point cloud data from a lidar system, characterized in that, The method includes: Acquire lidar point cloud and image information of the target area and convert the image information into a pseudo depth map; For each three-dimensional point in the lidar point cloud, the differences in geometric features between the three-dimensional point and its neighboring points are obtained, and the voting score of the three-dimensional point is obtained based on the differences. The true depth order of the three-dimensional points in the lidar point cloud is obtained, and the pseudo depth order of the three-dimensional points after being mapped to the pseudo depth map is obtained. The depth difference is obtained based on the true depth order and the pseudo depth order. The three-dimensional point is determined to be an outlier by combining the voting score and the depth difference. The step of obtaining the geometric differences between the three-dimensional point and its neighboring points includes: The geometric features of the three-dimensional point and its neighboring points are obtained, including the differences in dispersion, normal vector consistency, and intensity continuity. The comprehensive difference is obtained by combining the differences in dispersion, normal vector consistency, and intensity continuity. The dispersion difference is obtained in the following way: Calculate the distance between the 3D point and its neighboring points within its neighborhood; calculate the mean and standard deviation of all distances; and obtain the dispersion difference based on the distances, mean distance, and standard deviation. The normal vector consistency difference is obtained in the following way: Calculate the local principal normal vector of the neighboring points within the neighborhood of the three-dimensional point; calculate the angle between the normal vector of each neighboring point and the local principal normal vector; obtain the consistency difference of the normal vector based on the angle; The intensity continuity difference is obtained in the following way: The strength, mean strength, and standard deviation of the strength of the neighboring points within the neighborhood of the three-dimensional point are obtained; the strength deviation rate is calculated based on the strength, mean strength, and standard deviation of the strength; and the strength continuity difference is obtained based on the strength deviation rate.

2. The method for processing abnormal point cloud data of lidar according to claim 1, characterized in that, The method further includes: For 3D points identified as abnormal, at least one of the set verification methods is used to verify the 3D points; If the verification result indicates that the three-dimensional point is abnormal, then the determination that the three-dimensional point is an abnormal point is maintained. If the verification results show that the three-dimensional point is not abnormal, then the three-dimensional point will be corrected to a normal point.

3. The method for processing abnormal point cloud data of lidar according to claim 1, characterized in that, The step of obtaining the true depth order of the three-dimensional points in the lidar point cloud includes: Obtain the true depth value of the three-dimensional point and each neighboring point within its neighborhood range in the lidar point cloud; The three-dimensional point and its neighboring points are sorted based on their actual depth values. Obtain the sorted position of the three-dimensional points, and obtain the true depth sort of the three-dimensional points based on the sorted position.

4. The method for processing abnormal point cloud data of lidar according to claim 1, characterized in that, The step of obtaining the pseudo-depth sort of the 3D points after mapping to the pseudo-depth map includes: The laser radar point cloud is projected onto the image coordinate system to obtain the pixel coordinates of the corresponding pixel points of each three-dimensional point in the image coordinate system. Obtain the pseudo-depth value in the pseudo-depth map of the pixel coordinates corresponding to the three-dimensional point and its neighboring points within its neighborhood range. The three-dimensional point and its neighboring points are sorted based on their pseudo-depth values. Obtain the sorting position of the three-dimensional points, and obtain the pseudo-depth sorting of the three-dimensional points based on the sorting position.

5. The method for processing abnormal point cloud data of lidar according to claim 2, characterized in that, The various verification methods include isolated verification methods; The steps for verifying the three-dimensional points using an isolated verification method include: Count the number of neighboring points that are identified as abnormal points within the neighborhood of the three-dimensional point; Based on the quantity, determine the proportion of outliers within the neighborhood range, and obtain the nearest distance between the three-dimensional point and the nearest outlier within the neighborhood range; The three-dimensional point is verified as an anomaly by combining the percentage of outliers and the nearest distance.

6. The method for processing abnormal point cloud data of lidar according to claim 2, characterized in that, The various verification methods include intensity clustering verification; The steps for verifying the three-dimensional points using an intensity clustering verification method include: Obtain the intensity data of the neighboring points that are determined to be non-anomalies within the neighborhood of the three-dimensional point; Clustering is performed based on the intensity data to obtain multiple intensity cluster centers; Calculate the distance between the intensity value of the three-dimensional point and each intensity cluster center, and verify whether the three-dimensional point is an outlier based on the distance.

7. The method for processing abnormal point cloud data of lidar according to claim 2, characterized in that, The various verification methods include spatial correlation verification. The steps for verifying the three-dimensional points using a spatial correlation verification method include: The true depth information of the three-dimensional point and its neighboring points within its neighborhood range in the lidar point cloud is obtained, as well as the pseudo depth information mapped to the pseudo depth map. Based on the pseudo-depth information, pseudo-depth fluctuation values ​​and pseudo-depth trends are obtained, and based on the real depth information, real depth trends are obtained. By combining the pseudo-depth trend, the true depth trend, and the pseudo-depth fluctuation value, it is verified whether the three-dimensional point is an anomaly.

8. A lidar point cloud anomaly data processing device, characterized in that, The apparatus for implementing the lidar point cloud anomaly data processing method according to any one of claims 1-7 comprises: The acquisition module is used to acquire the lidar point cloud of the target area and the image information of the target area, and convert the image information into a pseudo-depth map; The voting score acquisition module is used to obtain the difference in geometric features between each three-dimensional point and its neighboring points in the lidar point cloud, and to obtain the voting score of the three-dimensional point based on the difference. The depth difference acquisition module is used to obtain the true depth order of the three-dimensional points in the lidar point cloud, and to obtain the pseudo depth order of the three-dimensional points after mapping to the pseudo depth map, and to obtain the depth difference based on the true depth order and the pseudo depth order. The judgment module is used to determine whether the three-dimensional point is an anomaly point by combining the voting score and the depth difference.

Images