Adaptive joint filtering method based on local feature and spatial distance relation

Abstract

The invention discloses a self-adaptive joint filtering method based on a local feature and spatial distance relationship, which belongs to the technical field of point cloud denoising, is used for point cloud denoising, and comprises the following steps of: after filtering out large-scale noise by using self-adaptive statistical filtering, carrying out noise suppression by using self-adaptive bilateral filtering; the adaptive bilateral filtering includes searching neighborhood points of the point cloud, calculating local curvature and local density, adjusting spatial domain and attribute domain parameters of the bilateral filtering based on the local curvature and the local density, introducing local features into weight parameters, performing normalization adjustment on the weight parameters, and updating geometric positions of the points. According to the method, noise of different densities and different distances is effectively identified and removed, so that the algorithm has higher robustness, and more detail information can be reserved while the denoising effect is enhanced.

Description

Technical Field

[0001] The present invention discloses an adaptive joint filtering method based on local features and spatial distance relationships, belonging to the technical field of point cloud denoising. Background Art

[0002] With the accelerating process of the intelligentization and networking of new energy vehicles, autonomous driving technology has become the core of industrial competition. LiDAR, with its advantages such as high precision and three-dimensional environmental perception, has gradually become one of the important sensors in autonomous driving technology. It generates three-dimensional point cloud data by scanning the surrounding environment to provide spatial information support for vehicles. However, in practical applications, point cloud acquisition is susceptible to various factors. Interferences such as bad weather like rain and snow, and vibrations during vehicle driving can all cause a large number of outlier noises and drift noises in the point cloud, directly affecting the accuracy of environmental perception and the safety and rationality of subsequent environmental planning. How to effectively filter point cloud noise in a real driving environment is an important task in current intelligent driving. The point cloud denoising methods in the prior art are often single denoising, and single denoising methods often have difficulty in balancing efficiency and accuracy, and filtering methods with fixed parameters also cannot adapt to multi-scene filtering. Summary of the Invention

[0003] The purpose of the present invention is to provide an adaptive joint filtering method based on local features and spatial distance relationships to solve the problems in the prior art that single denoising methods have difficulty in balancing efficiency and accuracy, and filtering methods with fixed parameters cannot adapt to multi-scene filtering.

[0004] The adaptive joint filtering method based on local features and spatial distance relationships includes filtering out large-scale noises using adaptive statistical filtering and then suppressing noises using adaptive bilateral filtering; The adaptive bilateral filtering includes searching for neighborhood points of the point cloud, calculating local curvature and local density, adjusting the parameters of the spatial domain and attribute domain of the bilateral filtering based on the local curvature and local density, introducing local features into the weight parameters, and performing normalization adjustment on the weight parameters to update the geometric positions of the points.

[0005] Searching for neighborhood points of the point cloud includes using a fixed neighborhood method in combination with a KD-Tree to search for the th point in the point cloud nearest neighborhood points, and the set of points in the neighborhood is , where is the th neighborhood point of , constructing the covariance matrix of the points in the neighborhood: ; ; In the formula, represents the mean value of the th neighborhood point, represents the th neighborhood point of

[0006] Calculating the local curvature includes performing eigenvalue decomposition on to obtain three eigenvalues , and the corresponding eigenvectors of the three eigenvalues are , , . is the estimated value of the local plane normal vector , and the local curvature is the ratio of the minimum eigenvalue to the sum of the eigenvalues: .

[0007] Calculating the local density includes calculating the Euclidean distance between and each point in the neighborhood: ; The local density is the reciprocal of the average Euclidean distance of each point in the neighborhood: .

[0008] Adjusting the spatial domain and attribute domain parameters of bilateral filtering includes that the spatial domain parameter and the attribute domain parameter are: ; ; In the formula, , respectively represent the initial parameters, , are adjustment factors.

[0009] Introducing local features into the weight parameter includes incorporating , , and the normal vector difference into the weight of bilateral filtering. The improved bilateral filtering weight is: ; In the formula, represents the normal vector difference between point and the neighborhood point .

[0010] Normalizing and adjusting the weight parameters includes: ; In the formula, represents the normalized weight value.

[0011] The geometric position of the updated point includes: .

[0012] Compared with the prior art, the present invention has the following beneficial effects: effectively identifying and removing noises with different densities and different distances, making the algorithm have stronger robustness, enhancing the denoising effect while retaining more detailed information. Description of the Drawings

[0013] Figure 1 is the original point cloud of the KITTI dataset; Figure 2 is the original point cloud of the KITTI dataset with added noise; Figure 3 is the visualization result of the statistical filtering of the KITTI dataset; Figure 4 is the visualization result of the radius filtering of the KITTI dataset; Figure 5 is the visualization result of the filtering of the present invention for the KITTI dataset; Figure 6 is the comparison of the precision rate indexes of each filtering algorithm under different scenarios; Figure 7 is the comparison of the recall rate indexes of each filtering algorithm under different scenarios; Figure 8 is the comparison of the original point retention rate indexes of each filtering algorithm under different scenarios; Figure 9 is the visualization result of the deviation of the statistical filtering; Figure 10 is the visualization result of the deviation of the radius filtering; Figure 11 is the visualization result of the deviation of the filtering of the present invention; Figure 12 is the original point cloud under the Boreas dataset; Figure 13 is the denoising effect of the statistical filtering under the Boreas dataset; Figure 14 is the denoising effect of the radius filtering under the Boreas dataset; Figure 15 is the denoising effect of the filtering of the present invention under the Boreas dataset. Detailed Embodiments

[0014] To make the objectives, technical solutions and advantages of the present invention clearer, the following provides a clear and complete description of the technical solutions in the present invention. Apparently, the described embodiments are some but not all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present invention without any creative efforts shall fall within the scope of protection of the present invention.

[0015] An adaptive joint filtering method based on local features and spatial distance relationships includes filtering out large-scale noise using adaptive statistical filtering and then suppressing noise using adaptive bilateral filtering. The adaptive bilateral filtering includes searching for neighborhood points of the point cloud, calculating local curvature and local density, adjusting the parameters of the spatial domain and attribute domain of the bilateral filtering based on the local curvature and local density, introducing local features into the weight parameters, and performing normalization adjustment on the weight parameters to update the geometric positions of the points.

[0016] Searching for neighborhood points of the point cloud includes using the fixed neighborhood method in combination with the KD-Tree to search for the th point in the point cloud and finding its nearest neighborhood points. The set of points within the neighborhood is , , where is the th neighborhood point of . The covariance matrix of the points within the neighborhood is constructed as follows: ; ; where represents the mean of , represents the th neighborhood point, and represents the th neighborhood point of .

[0017] Calculating the local curvature includes performing eigenvalue decomposition on . The three eigenvalues are , and the corresponding eigenvectors of the three eigenvalues are , , . is the estimated value of the local plane normal vector . The local curvature is the ratio of the minimum eigenvalue to the sum of the eigenvalues: .

[0018] Calculating the local density includes calculating the Euclidean distance from each point in the neighborhood : ; The local density is the reciprocal of the average Euclidean distance from each point in the neighborhood : .

[0019] Adjusting the spatial domain and attribute domain parameters of bilateral filtering includes that the spatial domain parameter and the attribute domain parameter are: ; ; In the formula, , respectively represent the initial parameters, , are adjustment factors.

[0020] Introducing local features into the weight parameter includes introducing , , and the normal vector difference into the weight of bilateral filtering. The improved bilateral filtering weight is: ; In the formula, represents the normal vector difference between point and the neighborhood point .

[0021] Normalizing and adjusting the weight parameter includes: ; In the formula, represents the normalized weight value.

[0022] Updating the geometric position of the point includes: .

[0023] The present invention first performs adaptive statistical filtering. The statistical filtering calculates the average distance and standard deviation between all points, and obtains a distance threshold accordingly. Subsequently, the average distance between each point and its K nearest neighbor points is calculated, and the distance threshold is compared with the average distance of the K points. Points not within the threshold range are regarded as noise points. The denoising effect of the algorithm is affected by the preset K value and the threshold standard deviation coefficient. An overly strict threshold will cause a large number of valid points to be erroneously deleted, while an overly loose threshold will not be able to effectively remove noise. Especially in scenarios such as road traffic scenes where the data density distribution is uneven, the global statistics cannot well describe the local situation, resulting in the inability to effectively retain data in low-density areas and the inability to fully denoise data in high-density areas. To solve the above defects, by introducing a local density and distance compensation mechanism into the statistical filtering algorithm, the adaptive adjustment of the algorithm threshold is realized, and the problem of erroneously deleting valid points due to low density is alleviated.

[0024] First, a KD-Tree point cloud index is constructed, and then the local density is calculated. To achieve the adaptive adjustment of the threshold according to the density during the denoising process, it is necessary to obtain the distribution of points in their local space. By calculating the local density, the sparsity degree of the point cloud in this area can be quantified, and it is introduced into the threshold standard deviation coefficient of the statistical filtering, so that the threshold can be adaptively enlarged or reduced according to the local density of the current point, thereby generating a differential filtering threshold for each point to ensure the effective retention of low-density points while removing the noise of high-density point clouds. The present invention adopts the method of a fixed-radius neighborhood to count the number of neighborhood points of each point within the specified radius and defines the number of points within the fixed radius neighborhood as the local density : ; ; According to the local density , the global average density of the point cloud is calculated : ; To enable the threshold standard deviation coefficient to better adapt to the local sampling conditions, based on the local density and the global average density , the threshold standard deviation adaptive coefficient is defined ; In the formula, represents the basic threshold coefficient, represents the density compensation exponent, which is used to adjust the compensation strength. When , it indicates a low-density area, and through Enlarge the ratio result to increase the corresponding rejection threshold and prevent low-density valid points from being erroneously deleted. Conversely, when , the ratio can be reduced to make the corresponding rejection condition more stringent and reduce the missed detection of high-density noise.

[0025] Based on the scanning characteristic of lidar that the points are denser near and sparser far away, that is, the area close to the sensor has a higher point cloud density and the point cloud is sparser in the area far from the sensor. In the traffic road scene, lidar can often scan a range of dozens of meters or even hundreds of meters. Nearby vehicles, road signs, etc. can be covered by dense point clouds, while distant objects have only a small number of point clouds. As the distance increases, the number of point clouds will decrease sharply. By designing a distance compensation mechanism, the problem of uneven distribution of point cloud data at different distances is improved. A more stringent filtering strategy is adopted for the high-density area at close range, and the filtering requirements are appropriately relaxed for the low-density area at long range to retain more effective information. To implement the above strategy, a compensation factor is introduced to dynamically adjust the point clouds at different distances, which is defined as follows: ; where represents the Euclidean distance from the point to the origin of the sensor, represents all medians, represents the adjustment factor. When at , it indicates that the point is far from the origin and the point cloud density in the area is low, and the filtering threshold needs to be appropriately increased; conversely, when at , it indicates that the point is close to the origin and the point cloud density in the area is high, and the filtering threshold needs to be appropriately decreased. Combining the local density and the distance compensation mechanism, the adaptive threshold formula is designed as follows: ; The specific process of adaptive statistical filtering is as follows: (1) Use KD-Tree to search for the nearest neighbor points within a fixed radius ; (2) Calculate the between the query point and its neighbor points : ; (3) Calculate the mean and standard deviation of the local average distances of all N points in the point cloud: ; ; (4) Calculate the local density of each point and the global average density , then calculate the distance from the point to the origin of the sensor, and construct a compensation factor based on the median distance , and design an adaptive filtering threshold ; (5) Compare with the filtering threshold. When , it is considered that is a valid point. If is not within this range, then the point is regarded as a noise point and removed.

[0026] After filtering out large-scale noise, the small-scale noise remaining in the point cloud may still affect the fine structure of the data and the accuracy of subsequent processing. Bilateral filtering designs weight functions in two domains, simultaneously considering the weights in the spatial domain and the attribute domain, assigns higher weights to the points in the local neighborhood that are similar to the target point in terms of position and attribute, and then moves the noise points along the normal vector direction according to the weights, so as to smooth the noise while retaining local geometric details. However, the traditional bilateral filtering algorithm uses fixed spatial and attribute scales, making it difficult to adapt to traffic road scenes with large density variations, resulting in unstable filtering effects. By introducing local feature information such as local curvature, density, and normal vector, the bilateral filtering algorithm can perform adaptive parameter adjustment and more precisely suppress noise and retain features.

[0027] The curvature index can reflect the flatness of the local surface of the point cloud. When the curvature index is large, it indicates that the point may be in a region with a sharp surface or obvious edge features. On the contrary, it indicates that the point may be in a smooth surface region.

[0028] Density can reflect the density of the local point cloud distribution. In the statistical filtering stage, a density calculation method based on the number of neighborhood points is used to detect outliers. In the bilateral filtering stage, the distribution of points will affect the filtering intensity. Therefore, the reciprocal of the average distance of neighborhood points is used as the density calculation method to adaptively adjust the filtering parameters.

[0029] To verify the effectiveness of the algorithm of the present invention, the KITTI and Boreas datasets are respectively selected for experiments. The KITTI dataset is collected from real traffic scenes and is one of the datasets for autonomous driving and computer vision evaluation. The Boreas dataset is a multi-season autonomous driving dataset that collects data by driving the same fixed route at different times, capturing real driving scenes in different seasons and adverse weather conditions. Therefore, it has rich environmental noise samples.

[0030] To evaluate the denoising effect of the algorithm, the present invention uses denoising precision , recall , and origin retention rate R o for quantitative evaluation and comparison. The calculation formulas for each evaluation index are as follows: ; ; ; TP represents the number of points correctly identified as noise and removed, FP represents the number of points that are actually non-noise but are wrongly removed, TN represents the number of points that are actually non-noise and are correctly retained, and FN represents the number of points that are actually noise but are wrongly retained.

[0031] To analyze the denoising effect of the algorithm in complex scenarios, based on the real-scene data of a certain intersection in the KITTI dataset, Gaussian noise with a mean of 0m, a standard deviation of 0.2m, and a quantity proportion of 30% is added. At the same time, to better simulate rainy and snowy weather, 10% uniformly distributed random spatial noise is superimposed. As Figure 1 , Figure 2 , Figure 3 , Figure 4 , Figure 5 shown, the Gaussian noise mainly forms high-density drift noise along the normal vector direction of the effective point cloud, and the random noise appears as outlier noise with uniform distribution across the whole area. Looking at the whole picture, it can be seen that each algorithm has a good effect on removing uniformly distributed random spatial noise. However, observing the details in the figure, it can be clearly seen that there are still a large number of noise points near the trees and vehicles in the middle and short distances after statistical filtering and radius filtering. The reason is that statistical filtering and radius filtering rely on preset parameters. When facing uneven point cloud distribution, in order to ensure that enough points can be retained in the relatively far area, the filtering parameters are often set relatively loosely, resulting in the inability to effectively remove noise in the high-density point cloud area in the short distance. In contrast, the algorithm of the present invention considers local features and distance compensation, can adapt to point clouds of different densities, flexibly change the filtering threshold, and effectively filter point clouds of different densities. Observing the rear of the vehicle nearby, it can be found that the application of adaptive bilateral filtering enables the edge features of the vehicle to be retained more effectively and also has an obvious effect on smoothing noise.

[0032] To achieve an objective evaluation of the algorithm performance, the present invention conducts a quantitative evaluation of the algorithm through denoising precision, recall rate, and origin retention rate. The precision reflects the proportion of actually noisy points among the removed points, the recall rate measures the proportion of all noisy points that are successfully removed, and the origin retention rate measures the proportion of original points that are retained during the denoising process. Analyzing Table 1, it can be seen that when the origin retention rate is around 90%, the precision and recall rate of the algorithm of the present invention are significantly higher than those of statistical filtering and radius filtering. The precision of statistical filtering is the lowest, indicating its weak ability to identify and remove noise. The recall rate of radius filtering is much lower than that of the other two algorithms, indicating that a large amount of noise is not removed due to the retention of valid points. In contrast, when the origin retention rates are similar, the precision of the algorithm of the present invention is increased by 11.08% and 10.48% respectively compared with statistical filtering and radius filtering, and the recall rate is increased by 13.90% and 22.49% respectively, indicating that the algorithm can reduce noise interference while ensuring higher complete data.

[0033] Table 1 Comparison of three indicators of each filtering algorithm; ; To verify the denoising effect of the algorithm in different scenarios, KITTI point cloud data in different scenarios is taken, Gaussian noise with a mean of 0m, a standard deviation of 0.2m, and a quantity ratio of 30% is added, and 10% uniformly distributed random spatial noise is also superimposed. Without changing the parameters of each algorithm, point cloud denoising is carried out. The precision, recall rate, and origin retention rate of each algorithm are compared as Figure 6 、 Figure 7 、 Figure 8 shown. Analyzing the figure, it can be found that the results are similar to those in Table 1. When the origin retention rates in different scenarios are similar, the precision and recall rate of the algorithm of the present invention are significantly higher than those of statistical filtering and radius filtering in each scenario. Compared with statistical filtering, the precision and recall rate are increased by about 5.8% and 15.9% respectively, and compared with radius filtering, they are increased by about 6.3% and 24.8% respectively. It shows that the algorithm can more accurately distinguish valid points from noise points and has higher robustness and adaptability in multiple scenarios.

[0034] To make a more objective evaluation of the algorithm, in the above traffic scenario, the point cloud data of a vehicle is selected, and the error analysis is carried out on the point cloud after the vehicle is extracted and the original point cloud. The results are as Figure 9 、 Figure 10 、 Figure 11 and Table 2 shown. Observing the figure, it can be seen that the number of high-deviation points of statistical filtering and radius filtering is much more than that of the algorithm of the present invention, indicating that the algorithm of the present invention can better filter out outliers, make the deviation within a smaller range by smoothing the noise, and improve the expression ability of edge features.

[0035] As can be seen from Table 2, the maximum distance deviation of the algorithm of the present invention is reduced by 6.48% and 9.10% respectively compared with the two traditional filtering algorithms. It is observed that the maximum distance error is significantly higher than the average distance error, indicating that there are still a very small number of high-deviation points in the three algorithms that have not been removed. In terms of the average distance error, the algorithm of the present invention is reduced by 54.56% and 55.90% respectively compared with the statistical filtering and radius filtering, indicating that the overall point cloud accuracy has been significantly improved; in terms of the standard deviation, the degree of error dispersion is reduced by 35.35% and 37.06% respectively, indicating that the algorithm of the present invention has obvious advantages in error stability.

[0036] Table 2 Deviation of each filtering algorithm model; ; To verify the robustness and denoising performance of the algorithm of the present invention under actual rain and snow conditions, the present invention additionally selects the Boreas dataset collected under real rain and snow weather as experimental data, and the results are as Figure 12 、 Figure 13 、 Figure 14 、 Figure 15 shown. It is not difficult to observe from the figure that all three filtering algorithms can filter out most of the rain and snow noise, but for the high-density noise point cloud area, the filtering effect of the algorithm of the present invention is significantly better than the traditional statistical filtering and radius filtering, and there is also significantly less drift noise near the vehicle, and the denoising effect is better.

[0037] The present invention proposes a point cloud denoising algorithm based on adaptive joint filtering. By considering the local density and spatial distance, the filtering threshold can be flexibly changed according to the neighborhood density and spatial distance, effectively retaining the sparse point cloud at a long distance while suppressing the high-density noise point cloud at a short distance; introducing local features to improve the weight parameters of the bilateral filtering algorithm, enhancing the edge feature expression ability while smoothing the noise. The experimental results show that the algorithm of the present invention has a better denoising effect compared with the traditional statistical filtering and radius filtering. In the multi-traffic scene experiment with uneven density, it can more effectively retain local information, and the precision and recall rate are increased by about 5.8% and 15.9% on average compared with the statistical filtering, and increased by about 6.3% and 24.8% compared with the radius filtering; the degree of error dispersion is reduced by 35.35% and 37.06% respectively. In a complex noise environment, the stable denoising performance of the algorithm of the present invention provides a solid and reliable basis for subsequent environmental perception and path planning.

[0038] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features, and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An adaptive joint filtering method based on local features and spatial distance relationships, characterized in that It includes using adaptive statistical filtering to filter out large-scale noise and then using adaptive bilateral filtering for noise suppression; The adaptive bilateral filtering includes searching for neighborhood points of the point cloud, calculating local curvature and local density, adjusting the parameters of the spatial domain and attribute domain of the bilateral filtering based on the local curvature and local density, introducing local features into the weight parameters, and performing normalization adjustment on the weight parameters to update the geometric position of the points.

2. The adaptive joint filtering method based on the local feature and spatial distance relationship according to claim 1, wherein Searching for the neighborhood points of the point cloud includes using the fixed neighborhood method in combination with the KD-Tree to search for the th point of the point cloud and its nearest neighborhood points. The set of points within the neighborhood is , . Wherein, is the th neighborhood point of . The covariance matrix of the points within the neighborhood is constructed as : ； ； In the formula, represents the mean value of represents the th neighborhood point, represents the th neighborhood point of 3. The adaptive joint filtering method based on the local feature and spatial distance relationship according to claim 2, characterized in that Calculating the local curvature includes performing eigenvalue decomposition, and the three eigenvalues , and the eigenvectors corresponding to the three eigenvalues are , , . is an estimated value of the local plane normal vector , and the local curvature is the ratio of the minimum eigenvalue to the sum of the eigenvalues: 。 4. The adaptive joint filtering method based on local features and spatial distance relationship according to claim 3, wherein, Calculating the local density includes calculating the Euclidean distance to each point in the neighborhood : ； Local density is the average Euclidean distance of each point in the neighborhood reciprocal of: 。 5. The adaptive joint filtering method based on the local feature and spatial distance relationship according to claim 4, wherein Adjusting the spatial domain and attribute domain parameters of bilateral filtering includes the spatial domain parameter and the attribute domain parameter as follows: ； ； In the formula, , respectively represent initial parameters, , are adjustment factors.

6. The adaptive joint filtering method based on local feature and spatial distance relationship according to claim 5, wherein Introducing local features into the weight parameters includes , , and integrating the normal vector difference into the weights of bilateral filtering. The improved bilateral filtering weights are as follows: ； In the formula, represents the point and the normal vector difference with the neighboring point ​ 7. The adaptive joint filtering method based on the local feature and spatial distance relationship according to claim 6, characterized in that Performing normalization adjustment on the weight parameters includes: ； In the formula, represents the normalized weight value.

8. The adaptive joint filtering method based on the relationship between local features and spatial distance according to claim 7, wherein Geometric position of the update point including: 。

Images