Dust filtering method and device, vehicle, readable storage medium and chip

Abstract

The embodiment of the invention provides a dust filtering method and device, a vehicle, a readable storage medium and a chip, and the method comprises the steps: determining original point cloud data corresponding to a laser radar; performing ground segmentation processing on the original point cloud data to determine a plurality of non-ground points; determining reflection intensity characteristics of non-ground points; determining candidate dust points according to the reflection intensity characteristics; determining a normal vector included angle variance and curvature characteristics corresponding to the candidate dust points; determining at least one dust point in the original point cloud data according to the normal vector included angle variance and curvature characteristics; and filtering the dust points, and determining dust filtering point cloud data corresponding to the original point cloud data. According to the scheme, the dust filtering accuracy of the laser radar is improved.

Description

Technical Field

[0001] This invention relates to the field of lidar technology, and more specifically, to a dust filtration method, apparatus, vehicle, readable storage medium, and chip. Background Technology

[0002] During unmanned operation in open-pit mines, the dusty environment can cause unmanned vehicles using lidar to misdetect point clouds caused by dust, leading to unexpected vehicle stops or missed detection of real targets in front of the vehicle due to dust obscuring them. This can result in safety accidents caused by obstacle encounters, stopping, or emergency obstacle avoidance under heavy loads.

[0003] In related technologies, point clouds are typically acquired using LiDAR. Features of clusters on the projection depth image of the point cloud distribution are then extracted. These extracted features are input into a trained classification model to obtain the dust probability for each cluster. However, due to the irregular shape of dust and the difficulty in preventing target objects and dust from being clustered together when close, the efficiency and accuracy of dust filtering are low in dusty environments like open-pit mines. Summary of the Invention

[0004] The purpose of this invention is to provide a dust filtration method, apparatus, vehicle, readable storage medium, and chip that can solve the problem of low efficiency and accuracy of dust filtration in environments with high dust levels, such as open-pit mines.

[0005] In view of this, an embodiment of the first aspect of the present invention provides a dust filtration method.

[0006] A second aspect of the present invention provides a dust filtration device.

[0007] An embodiment of the third aspect of the present invention provides a vehicle.

[0008] An embodiment of the fourth aspect of the present invention provides a readable storage medium.

[0009] An embodiment of the fifth aspect of the present invention provides a chip.

[0010] To achieve the above objectives, an embodiment of the first aspect of the present invention provides a dust removal method, comprising: determining original point cloud data corresponding to a lidar radar; performing ground segmentation processing on the original point cloud data to determine multiple non-ground points; determining the reflection intensity characteristics of the non-ground points; determining candidate dust points based on the reflection intensity characteristics; determining the variance of the normal vector angle and curvature characteristics corresponding to the candidate dust points; determining at least one dust point in the original point cloud data based on the variance of the normal vector angle and curvature characteristics; and filtering out the dust points to determine dust-filtered point cloud data corresponding to the original point cloud data.

[0011] A lidar system emits multiple laser pulses until the signal reaches an object, then reflects back to the receiver. Based on the known laser pulse velocity and reflection time, the distance between the detected object and the lidar receiver can be calculated. The lidar's laser emitter rotates at high speed internally, acquiring the positional information of objects around the vehicle relative to the lidar, thus determining the point cloud data of the surrounding environment, corresponding to the lidar's raw point cloud data. Ground segmentation processing is performed on the raw point cloud data to determine the point cloud data projected onto a two-dimensional grid within a corresponding spatial range. This two-dimensional grid consists of multiple grid cells, each containing multiple point cloud data points. Non-ground data is extracted by determining the height difference between the projected points of each point cloud data point within a grid cell and the horizontal distance between adjacent grid cells, identifying multiple non-ground points. Since there is a difference in reflection intensity between real targets and dust targets (real targets have a higher reflection intensity than dust), targets with reflection intensity less than the threshold are marked as candidate dust points by setting a threshold. Then, the normal vector and curvature features of the candidate dust points are calculated. The variance and curvature features of the normal vector of the real target relative to the dust within a certain neighborhood are smaller than those of the normal vector of the dust point. The dust points are extracted from the dust candidate points and filtered using the information corresponding to the normal vector and curvature features to obtain the dust-filtered point cloud data after dust removal.

[0012] In some technical solutions, optionally, ground segmentation processing is performed on the original point cloud data to determine multiple non-ground points, including: using a slope filtering algorithm to project the original point cloud data into a two-dimensional grid, the two-dimensional grid including multiple grid cells; determining the lowest point of each grid cell; determining the height difference and horizontal distance between adjacent lowest points; determining the slope corresponding to adjacent grid cells based on the height difference and horizontal distance; determining a slope angle threshold; determining non-ground points based on the slope and the slope angle threshold, wherein the slope corresponding to the non-ground points is greater than the slope angle threshold.

[0013] In this scheme, a slope filtering algorithm is used to project the original point cloud data into a two-dimensional grid, calculate the lowest point of each grid cell in the two-dimensional grid, compare the height difference and horizontal distance of the lowest points of adjacent grid cells, and determine the slope by the height difference and horizontal distance. If the slope exceeds the slope angle threshold, it is determined to be a non-ground point.

[0014] Understandably, by projecting raw 3D point cloud data onto a 2D mesh for processing, the complex global relationship calculations are simplified to 2D mesh cell relationship calculations, which can meet the stringent real-time requirements of autonomous driving. Furthermore, traditional heightmap methods often misclassify slopes as non-ground points; by analyzing the local rate of change in adjacent mesh cells, the accuracy of non-ground point identification can be improved.

[0015] In some technical solutions, candidate dust points can optionally be determined based on reflection intensity characteristics, including: determining the reflection intensity corresponding to non-ground points based on reflection intensity characteristics; determining a deep learning dust segmentation model; determining a dynamic threshold based on the deep learning dust segmentation model; and determining non-ground points with reflection intensity less than the dynamic threshold as candidate dust points.

[0016] In this scheme, the reflection intensity of non-ground points is analyzed. The reflection intensity of dust points is lower than that of normal targets, but there is overlap between the reflection intensity information of real targets and dust points. Directly using a set threshold for binary classification can lead to over-filtering or under-filtering. Therefore, this paper adopts a pre-selection approach, designating points with a reflection intensity threshold below the set threshold as candidate dust points. Initial screening of these candidate dust points quickly identifies a large number of points with high reflection intensity, reducing the amount of data processing required for subsequent verification of normal vector angle variance and curvature features.

[0017] In some technical solutions, optionally, a dynamic threshold is determined based on a deep learning dust segmentation model, including: determining a point cloud segmentation model and a corresponding startup period for the point cloud segmentation model; periodically starting the point cloud segmentation model to extract dust point clouds according to the startup period; determining the reflection intensity values ​​corresponding to multiple dust point clouds; determining the maximum reflection intensity value among the multiple reflection intensity values; and determining the dynamic threshold based on the maximum reflection intensity value.

[0018] In this scheme, the dynamic threshold determined by the model is updated periodically according to the deep learning point cloud dust target segmentation algorithm. Every time a startup cycle occurs, the deep learning algorithm is started, that is, the point cloud segmentation model statistically analyzes the reflection intensity information of the segmented dust point cloud and uses the maximum value as the reflection intensity segmentation threshold, i.e., the dynamic threshold, so as to realize the periodic update of the dynamic threshold.

[0019] In some technical solutions, optionally, the variance of the normal vector angle and curvature features corresponding to the candidate dust points are determined, including: establishing a set of neighboring points for each candidate dust point, the set of neighboring points including multiple neighboring points closest to the current candidate dust point; determining the covariance matrix corresponding to the set of neighboring points; determining the principal normal vector corresponding to the candidate dust point based on the covariance matrix; traversing each neighboring point in the set of neighboring points and calculating the normal vector corresponding to the neighboring point; determining the set of angles between the normal vector and the principal normal vector; determining the variance of the normal vector angle based on the set of angles; performing eigenvalue decomposition on the covariance matrix to determine multiple eigenvalues; and determining the curvature features based on the multiple eigenvalues.

[0020] In this scheme, the normal vector of the point cloud marked as a candidate dust point is calculated, and the variance of the angle between the normal vector of the current candidate dust point and the normal vector of the surrounding point cloud, as well as the curvature feature, are calculated.

[0021] Understandably, by quantifying the variance of the normal vector angle and curvature features through the fundamental differences in the physical form of dust and solid obstacles, the algorithm no longer relies on a single physical property that is easily affected by environmental interference, namely reflection intensity, but makes judgments based on more stable geometric morphological information, thereby improving the accuracy and reliability of dust recognition.

[0022] In some technical solutions, optionally, at least one dust point in the original point cloud data is determined based on the variance of the normal vector angle and the curvature feature, including: determining a preset variance threshold and a preset curvature threshold respectively; when the variance of the normal vector angle corresponding to the candidate dust point is greater than the preset variance threshold, and the curvature feature corresponding to the candidate dust point is greater than the preset curvature threshold, the candidate dust point is determined to be a dust point in the original point cloud data.

[0023] In this scheme, a dual-feature judgment mechanism is established by analyzing data on both the variance of the normal vector angle and the curvature feature. A candidate dust point is only identified as a true dust point when both the variance of the normal vector angle corresponding to the candidate dust point and the curvature feature corresponding to the candidate dust point are greater than a preset variance threshold. If either condition is not met, the candidate dust point is excluded.

[0024] A second aspect of the present invention provides a dust removal device, comprising: a point cloud determination module for determining original point cloud data corresponding to a lidar radar; a data segmentation module for performing ground segmentation processing on the original point cloud data to determine multiple non-ground points; a reflection intensity module for determining the reflection intensity characteristics of the non-ground points; a dust candidate module for determining candidate dust points based on the reflection intensity characteristics; a feature determination module for determining the variance of the normal vector angle and curvature characteristics corresponding to the candidate dust points; a dust determination module for determining at least one dust point in the original point cloud data based on the variance of the normal vector angle and curvature characteristics; and a dust removal module for removing the dust points to determine dust-filtered point cloud data corresponding to the original point cloud data.

[0025] An embodiment of the third aspect of this application provides a vehicle, the vehicle including: a body, and a dust removal device as described in the second aspect disposed within the body.

[0026] An embodiment of the fourth aspect of this application provides a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the dust filtration method as described in the first aspect.

[0027] An embodiment of the fifth aspect of this application provides a chip including a processor and a communication interface, the communication interface and the processor being coupled together, the processor being used to run a program or instructions to implement the steps of the dust filtration method as described in the first aspect.

[0028] Additional aspects and advantages of the technical solutions of the present invention will become apparent in the following description or may be learned by practice of the invention. Attached Figure Description

[0029] Figure 1 A schematic flowchart of a dust filtration method according to an embodiment of this application is shown;

[0030] Figure 2 A schematic flowchart of a dust filtration method according to an embodiment of this application is shown;

[0031] Figure 3 A schematic flowchart of a dust filtration method according to an embodiment of this application is shown;

[0032] Figure 4 A schematic flowchart of a dust filtration method according to an embodiment of this application is shown;

[0033] Figure 5 A schematic flowchart of a dust filtration method according to an embodiment of this application is shown;

[0034] Figure 6A schematic flowchart of a dust filtration method according to an embodiment of this application is shown;

[0035] Figure 7 A schematic block diagram of a dust filtration device according to an embodiment of this application is shown;

[0036] Figure 8 A schematic flowchart of a dust filtration method according to an embodiment of this application is shown;

[0037] Figure 9 A schematic diagram of dust filtration according to an embodiment of this application is shown.

[0038] in, Figure 7 The correspondence between the reference numerals and component names in the attached drawings is as follows:

[0039] 900: Dust filtration device; 902: Point cloud determination module; 904: Data segmentation module; 906: Reflection intensity module; 908: Dust candidate module; 910: Feature determination module; 912: Dust determination module; 914: Dust filtration module. Detailed Implementation

[0040] To better understand the above-described objectives, features, and advantages of the embodiments of the present invention, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0041] Many specific details are set forth in the following description in order to provide a full understanding of this application. However, embodiments of the invention may also be implemented in other ways different from those described herein. Therefore, the scope of protection of this application is not limited to the specific embodiments disclosed below.

[0042] The unmanned mining truck uses multiple sensors, including lidar, to detect obstacles, enabling it to avoid obstacles in time during unmanned driving.

[0043] However, in open-pit mining scenarios, the working environment is harsh and there is a lot of dust. The presence of dust can cause false detections and lead to shutdowns. In addition, dust can form dust point clouds on lidar or other multiple sensors, which can also cause real targets or obstacles to be obscured by dust during the detection process, resulting in missed detections.

[0044] In related technologies, LiDAR point cloud data is projected onto a depth image to obtain a projected depth image, overcoming the difficulty of point cloud density being affected by distance in the Cartesian coordinate system. After projection, obstacle clustering information obtained in the Cartesian coordinate system is combined to extract the features of clusters on the point cloud projection depth image. The extracted features are then input into a trained classification model to obtain the dust probability corresponding to each cluster. This allows obstacles labeled as dust to be removed from the obstacle sequence based on the obtained dust probability, achieving the effect of filtering out dust.

[0045] However, clustering algorithms struggle to avoid clustering target obstacles and dust into the same category when they are close together, leading to the filtering out of real targets or obstacles and the risk of missed detections. Furthermore, the morphology of dense dust in open-pit mining environments is inherently irregular, making it difficult to accurately cluster dust using clustering algorithms, resulting in low accuracy in dust clustering.

[0046] Based on this, embodiments of the present invention provide a dust filtration method.

[0047] The following is in conjunction with the appendix Figures 1 to 9 The dust filtration method, apparatus, vehicle, readable storage medium, and chip provided in this application will be described in detail through specific embodiments and application scenarios.

[0048] This embodiment provides a dust filtration method, such as Figure 1 As shown, dust filtration methods include:

[0049] Step S100: Determine the raw point cloud data corresponding to the lidar;

[0050] Step S102: Perform ground segmentation processing on the original point cloud data to identify multiple non-ground points;

[0051] Step S104: Determine the reflection intensity characteristics of non-ground points;

[0052] Step S106: Determine candidate dust points based on reflection intensity characteristics;

[0053] Step S108: Determine the variance of the normal vector angle and curvature characteristics corresponding to the candidate dust points;

[0054] Step S110: Determine at least one dust point in the original point cloud data based on the variance of the normal vector angle and curvature characteristics;

[0055] Step S112: Filter out the dust points and determine the dust-filtered point cloud data corresponding to the original point cloud data.

[0056] A lidar system emits multiple laser pulses until the signal reaches an object, then reflects back to the receiver. Based on the known laser pulse velocity and reflection time, the distance between the detected object and the lidar receiver can be calculated. The lidar's laser emitter rotates at high speed internally, acquiring the positional information of objects around the vehicle relative to the lidar, thus determining the point cloud data of the surrounding environment, corresponding to the lidar's raw point cloud data. Ground segmentation processing is performed on the raw point cloud data to determine the point cloud data projected onto a two-dimensional grid within a corresponding spatial range. This two-dimensional grid consists of multiple grid cells, each containing multiple point cloud data points. Non-ground data is extracted by determining the height difference between the projected points of each point cloud data point within a grid cell and the horizontal distance between adjacent grid cells, identifying multiple non-ground points. Since there is a difference in reflection intensity between real targets and dust targets (real targets have a higher reflection intensity than dust), targets with reflection intensity less than the threshold are marked as candidate dust points by setting a threshold. Then, the normal vector and curvature features of the candidate dust points are calculated. The variance and curvature features of the normal vector of the real target relative to the dust within a certain neighborhood are smaller than those of the normal vector of the dust point. The dust points are extracted from the dust candidate points and filtered using the information corresponding to the normal vector and curvature features to obtain the dust-filtered point cloud data after dust removal.

[0057] Understandably, dust morphology is not fixed compared to normal targets. By utilizing the distribution of surrounding point cloud normal vectors and local curvature features to distinguish dust points from surrounding targets, the dust filtering accuracy is improved. Furthermore, the current method of pre-labeling multiple point cloud data in the original point cloud data based on reflection intensity to determine candidate dust points reduces the amount of data processing, improves the algorithm's processing speed, and further meets the real-time requirements of autonomous driving.

[0058] Specifically, the lidar on the unmanned mining truck emits multiple laser pulses until the signal reaches an object, then reflects back to the receiver. Based on the known laser pulse velocity and reflection time, the distance between the detected object and the lidar receiver can be calculated. The lidar's laser emitter rotates at high speed internally, completing one revolution in 0.1 seconds. After one revolution, the positional information of objects around the vehicle relative to the lidar is obtained, and the point cloud data is abundant, providing a good depiction of the vehicle's surrounding environment. The lidar then transmits the data to a computer via network cable and protocols. The computer's lidar driver code then parses the transmitted data to obtain the point cloud data of the vehicle's surrounding environment. The raw point cloud data includes three-dimensional coordinates and reflection intensity (x, y, z, intensity).

[0059] During the ground segmentation process of the raw point cloud data, considering the unevenness of the mining area roads, the raw point cloud data is projected onto a two-dimensional grid, and ground points that account for a large proportion of the total number of points in the raw point cloud data are removed. Ground points are invalid information for obstacle detection and dust detection, and may even interfere with dust or obstacle detection. Non-ground points other than ground points are determined based on slope characteristics, which significantly reduces the amount of data that needs to be processed. Computational resources are concentrated on non-ground points that may represent obstacles or dust, improving the efficiency and accuracy of subsequent steps.

[0060] In some embodiments, optionally, the multiple point cloud data corresponding to the lidar include, in addition to location information, feature information corresponding to reflection intensity. The reflection intensity feature information represents the intensity of the laser beam reflected back from the object's surface. Different materials have different reflectivities to the laser. For example, metals reflect strongly, asphalt surfaces reflect weakly, and dust, due to its rough surface and small particles, has a lower reflection intensity than other object surfaces detected by the lidar in a mining environment. Low reflection intensity is identified as a key physical property feature for distinguishing dust.

[0061] In some embodiments, optionally, multiple candidate dust points are determined based on low reflectance intensity. A reflectance intensity threshold is set, and all non-ground points in the original point cloud data with reflectance intensity lower than this threshold are marked as candidate dust points for coarse screening. This coarse screening quickly eliminates most real obstacles with high reflectance intensity, narrowing down the scope for subsequent fine processing. The candidate dust points include at least one real object with low reflectance intensity compared to other real obstacles, such as a black plastic obstacle.

[0062] In some embodiments, optionally, geometric features are introduced to compensate for the shortcomings of the single physical feature of reflection intensity. Since dust clouds are diffuse and shapeless, their corresponding point cloud distribution is chaotic, with significant differences in the normal directions of each point cloud, resulting in a larger variance in the normal vector angle than that of the point cloud corresponding to the real object. Furthermore, dust point clouds have irregular shapes and high local curvature, meaning the curvature feature of the dust point cloud is greater than the curvature of the regular surface corresponding to the real object. By using both the variance in the normal vector angle and the curvature feature—two geometric features—dust points can be identified based on the physical characteristic of low reflection intensity, thereby increasing confidence, reducing the probability of false detection, and improving the reliability of dust filtering.

[0063] For example, dense dust will form a dust point cloud on the lidar. Dense dust refers to dust with a dust concentration greater than 10 mg / m².

[0064] In some embodiments, optionally, such as Figure 2 As shown, step S102: Perform ground segmentation processing on the original point cloud data to identify multiple non-ground points, including:

[0065] Step S1020: Using a slope filtering algorithm, the original point cloud data is projected onto a two-dimensional grid, which includes multiple grid cells;

[0066] Step S1022: Determine the lowest point of each grid cell;

[0067] Step S1024: Determine the height difference and horizontal distance between adjacent lowest points;

[0068] Step S1026: Determine the slope corresponding to adjacent grid cells based on the height difference and horizontal distance;

[0069] Step S1028: Determine the slope angle threshold;

[0070] Step S1030: Determine non-ground points based on slope and slope angle threshold. The slope corresponding to a non-ground point is greater than the slope angle threshold.

[0071] In this embodiment, a slope filtering algorithm is used to project the original point cloud data onto a two-dimensional grid, calculate the lowest point of each grid cell in the two-dimensional grid, compare the height difference and horizontal distance between the lowest points of adjacent grid cells, and determine the slope by the height difference and horizontal distance. If the slope exceeds the slope angle threshold, it is determined to be a non-ground point.

[0072] Understandably, by projecting raw 3D point cloud data onto a 2D mesh for processing, the complex global relationship calculations are simplified to 2D mesh cell relationship calculations, which can meet the stringent real-time requirements of autonomous driving. Furthermore, traditional heightmap methods often misclassify slopes as non-ground points; by analyzing the local rate of change in adjacent mesh cells, the accuracy of non-ground point identification can be improved.

[0073] In some embodiments, optionally, the projection of the original point cloud data onto a two-dimensional grid can obtain the point cloud projection depth image, and the point cloud data can be represented by (row, col), where row is determined by the pitch angle of each point in the point cloud data, and col is determined by the horizontal angle of the point cloud data relative to the horizontal plane. Here, the pitch angle θ and horizontal direction angle ψ are calculated point-by-point based on the mutual conversion between the Cartesian coordinate system and the laser coordinate system (reflection distance r, pitch angle θ, and direction angle ψ on the horizontal plane), and finally the rangeimage projection of the point cloud is obtained, which is the projection image of the original point cloud data onto the two-dimensional grid.

[0074] In some embodiments, the 3D point cloud in the original point cloud data is optionally assigned to the corresponding grid cell directly below it according to its x-axis and y-axis. A grid cell may contain multiple lidar points or no lidar points may fall into it.

[0075] In some embodiments, optionally, the spatial coordinates of each point cloud data are represented as (x, y, z, intensity), the projection size of the grid cell in the xy plane is set as (Sx, Sy), and the point cloud data is divided into several grid cells according to the xy plane with the origin of the xy plane as the starting point. The size of each grid cell is (Sx, Sy), wherein each grid cell contains point cloud data falling within the spatial range of the grid cell.

[0076] In some embodiments, optionally, within each non-empty grid cell, all falling LiDAR points are traversed to find the point with the smallest z-axis coordinate value, which is the lowest point within that grid cell. Since the ground is a continuous surface closest to the sensor, the lowest point within each grid cell represents the ground height of that grid cell.

[0077] In some embodiments, optionally, since the actual ground surface is usually continuously changing, its slope will be within a reasonable range; for example, there are design standards for the maximum slope angle of highways. The slope angle threshold corresponding to the slope variation of the actual ground surface is accurately determined based on the design schedule of the open-pit mine road.

[0078] In some embodiments, alternatively, non-ground points are determined based on the slope of the line connecting the lowest point of the current grid cell to the lowest point of the adjacent grid cell, since a steep slope change is formed between the obstacle and the ground.

[0079] In some embodiments, optionally, for the adjacency relationship of the current grid cell, if the calculated slope value is greater than a preset slope angle threshold, it indicates that the height change between the two grid cells is too drastic and does not conform to the continuous characteristics of the ground. In this case, the point with the higher height in the two grid cells is marked as a non-ground point, or the lowest point of the two grid cells with a slope higher than the preset slope angle threshold is marked as a non-ground point.

[0080] In some embodiments, optionally, such as Figure 3 As shown, step S106: Determine candidate dust points based on reflection intensity characteristics, including:

[0081] Step S1060: Determine the reflection intensity corresponding to non-ground points based on the reflection intensity characteristics;

[0082] Step S1062: Determine the deep learning dust segmentation model;

[0083] Step S1064: Determine the dynamic threshold based on the deep learning dust segmentation model;

[0084] Step S1066: Determine non-ground points with reflection intensity less than the dynamic threshold as candidate dust points.

[0085] In this embodiment, the reflection intensity of non-ground points is analyzed. The reflection intensity of dust points is lower than that of normal targets, but there is overlap between the reflection intensity information of real targets and dust points. Directly using a set threshold for binary classification can lead to over-filtering or under-filtering. Therefore, this paper adopts a pre-selection approach, setting points with a reflection intensity threshold below the set threshold as candidate dust points. Preliminary screening of candidate dust points quickly identifies a large number of points with high reflection intensity, reducing the amount of data processing required for subsequent verification of normal vector angle variance and curvature features.

[0086] Understandably, fixed thresholds cannot cope with environmental changes. For example, morning fog and strong afternoon sunlight can significantly alter the reflective characteristics of dust and object surfaces. By using a deep learning dust segmentation model to determine dynamic thresholds, the algorithm calculates dynamic thresholds based on the current real-time scene in the open-pit mine, automatically compensating for the impact of environmental changes. The dynamic threshold formula maintains an optimal range for distinguishing between dust and non-dust areas, ensuring stable high performance under different weather conditions, lighting, and road conditions, thus improving the confidence level in identifying candidate dust points.

[0087] In some embodiments, optionally, the reflection intensity features of each point are extracted from the set of non-ground points obtained after ground segmentation, and the corresponding numerical value of the reflection intensity is determined. The reflection intensity is directly measured by the lidar sensor and represents the intensity of the laser echo.

[0088] In some embodiments, the deep learning dust segmentation model optionally utilizes local or global point cloud data, including multi-dimensional features such as coordinates and reflection intensity. The model input is a probability prediction or binarized segmentation result of whether each point in the input point cloud is dust.

[0089] In some embodiments, the model type of the deep learning dust segmentation model is optionally a semantic segmentation model that includes at least one architecture.

[0090] In some embodiments, the training process of the deep learning dust segmentation model may optionally include a large amount of labeled data, which includes labeled dust points and non-dust points.

[0091] In some embodiments, the input to the deep learning dust segmentation model may optionally include, but is not limited to, reflection intensity, and may also include the three-dimensional coordinates of the point, the distance relative to the lidar, etc.

[0092] In some embodiments, optionally, such as Figure 4 As shown, step S1064: Determine the dynamic threshold based on the deep learning dust segmentation model, including:

[0093] Step S10640: Determine the point cloud segmentation model and the corresponding startup cycle of the point cloud segmentation model;

[0094] Step S10642: Start the point cloud segmentation model at regular intervals according to the startup cycle to extract dust point clouds;

[0095] Step S10644: Determine the reflection intensity values ​​corresponding to multiple dust point clouds;

[0096] Step S10646: Determine the maximum reflection intensity value among multiple reflection intensity values;

[0097] Step S10648: Determine the dynamic threshold based on the maximum reflection intensity value.

[0098] In this embodiment, the dynamic threshold determined by the model is updated periodically according to the deep learning point cloud dust target segmentation algorithm. Every time a startup cycle occurs, the deep learning algorithm is started, that is, the point cloud segmentation model statistically analyzes the reflection intensity information of the segmented dust point cloud and uses the maximum value as the reflection intensity segmentation threshold, that is, the dynamic threshold, so as to realize the periodic update of the dynamic threshold.

[0099] Understandably, complex point cloud segmentation models require a huge amount of computation, making it impossible to process point cloud data in real time at multiple frames per second on an in-vehicle computing platform. By periodically starting the process, the computational burden on the model can be reduced, significantly lowering computational overhead and improving the real-time performance of dynamic threshold updates while ensuring accuracy.

[0100] In one embodiment, the point cloud segmentation model is optionally a trained deep learning model whose function is to dynamically update the reflection intensity threshold according to the startup cycle.

[0101] In one embodiment, optionally, when the point cloud segmentation model starts extracting dust point clouds at regular intervals according to the startup cycle, it processes the original point cloud or non-ground point cloud of the current frame, determines the reflection intensity information of the segmented dust point cloud in the current environment, and determines the maximum value among multiple reflection intensity values ​​as the dynamic threshold within the current startup cycle.

[0102] In one embodiment, the dust point cloud extracted by the point cloud segmentation model may optionally include candidate dust points.

[0103] In one embodiment, the startup period may optionally be one hour or an integer multiple of one hour.

[0104] In some embodiments, optionally, such as Figure 5 As shown, step S108: Determine the variance of the normal vector angle and curvature features corresponding to the candidate dust points, including:

[0105] Step S1080: Establish a neighbor set for each candidate dust point, the neighbor set including the multiple neighboring points closest to the current candidate dust point;

[0106] Step S1082: Determine the covariance matrix corresponding to the neighboring point set;

[0107] Step S1084: Determine the principal normal vector corresponding to the candidate dust point based on the covariance matrix;

[0108] Step S1086: Traverse each neighboring point in the neighboring point set and calculate the normal vector corresponding to the neighboring point;

[0109] Step S1088: Determine the set of angles between the normal vector and the principal normal vector;

[0110] Step S1090: Determine the variance of the normal vector's included angle based on the included angle set;

[0111] Step S1092: Perform eigenvalue decomposition on the covariance matrix to determine multiple eigenvalues;

[0112] Step S1094: Determine the curvature feature based on multiple eigenvalues.

[0113] In this embodiment, the normal vector of the point cloud marked as candidate dust points is calculated, and the variance of the angle between the normal vector of the current candidate dust point and the normal vector of the surrounding point cloud, as well as the curvature feature, are calculated.

[0114] Understandably, by quantifying the variance of the normal vector angle and curvature features through the fundamental differences in the physical form of dust and solid obstacles, the algorithm no longer relies on a single physical property that is easily affected by environmental interference, namely reflection intensity, but makes judgments based on more stable geometric morphological information, thereby improving the accuracy and reliability of dust recognition.

[0115] For example, the reflectivity of a black rubber tire may be very low. When filtering based on reflectivity, the point cloud data corresponding to the black rubber tire is identified as a candidate dust point. However, because the surface of the black rubber tire is smooth, the variance of the corresponding normal vector angle is small, and it is a regular curved surface with a small value for the corresponding curvature feature, the candidate dust points can be further filtered based on both the variance of the normal vector angle and the curvature feature.

[0116] In some embodiments, optionally, a neighbor set is established for each candidate dust point, and a neighborhood query is performed. For each candidate dust point selected by reflection intensity, the neighbor set is determined by finding the K nearest points (K-Nearest Neighbors, KNN) within a small range centered on the candidate dust point.

[0117] In some embodiments, optionally, determining the covariance matrix corresponding to the neighboring point set includes: calculating the principal normal vector of the current candidate dust point using a principal component analysis algorithm; calculating the mean of the three-dimensional coordinates of all points in the neighboring point set as the centroid coordinates; and determining the covariance matrix by determining the mean of the vector products from the centroid to each point in the neighboring point set. The covariance matrix represents the distribution of all points in the neighboring point set in three dimensions (x, y, z) and the correlation between uncoated points.

[0118] In some embodiments, optionally, the covariance matrix is ​​decomposed into eigenvalues ​​to determine the eigenvector corresponding to the smallest eigenvalue. That is, the distribution of points changes least along the normal direction, so the eigenvector corresponding to the smallest eigenvalue is the normal direction of the local plane.

[0119] In some embodiments, optionally, local normal vectors are determined for all points within a neighboring point set, thereby determining the orientation information of each point in the neighborhood. If the surface of an object is smooth, the directions of the normal vectors of all points are basically consistent, i.e., the variance of the angle between the normal vectors is small; if the surface of an object is messy and irregular, the directions of the normal vectors of each point are different, i.e., the variance of the angle between the normal vectors is large. The variance of the angle between the normal vectors quantifies the surface roughness or irregularity of a local area. Dust clouds are diffuse and irregular, and the directions of the corresponding point cloud normal vectors are disordered, so the variance of the angle between the normal vectors is greater than that of the normal vectors of real objects.

[0120] In some embodiments, curvature can be optionally calculated using eigenvalues ​​of different dimensions in the covariance matrix to obtain curvature features. These curvature features quantify the degree of surface curvature in a local area. Dust clouds typically cluster in a small area, forming a high-curvature, uneven cluster, while the surface of a real object is smooth. Therefore, the curvature features corresponding to dust points are greater than those corresponding to real objects.

[0121] In some embodiments, optionally, in addition to the variance of the normal vector angle and curvature features, the feature parameters also include the density of the neighboring point set, and the dust points and non-dust points are distinguished by the fact that the density of the dust clump is greater than the density of the real object.

[0122] In some embodiments, optionally, such as Figure 6 As shown, step S110: Determine at least one dust point in the original point cloud data based on the variance of the normal vector angle and curvature features, including:

[0123] Step S1100: Determine the preset variance threshold and the preset curvature threshold respectively;

[0124] Step S1102: When the variance of the angle between the normal vectors corresponding to the candidate dust point is greater than the preset variance threshold, and the curvature feature corresponding to the candidate dust point is greater than the preset curvature threshold, the candidate dust point is determined to be a dust point in the original point cloud data.

[0125] In this embodiment, a dual-feature judgment mechanism is determined by analyzing data on both the variance of the normal vector angle and the curvature feature. A candidate dust point is only identified as a true dust point when both the variance of the normal vector angle corresponding to the candidate dust point and the curvature feature corresponding to the candidate dust point are greater than a preset variance threshold. If either condition is not met, the candidate dust point is excluded.

[0126] Understandably, the accuracy of a single geometric feature is not high in edge cases. By using a multiple verification mechanism of reflection intensity, variance of the normal vector angle, and curvature features, the confidence and robustness of the final result are improved, resulting in a significant improvement in the accuracy of dust recognition.

[0127] In some embodiments, optionally, a preset variance threshold is used as a benchmark value for judging the degree of irregularity, and is obtained statistically from labeled dust samples and non-dust samples. The preset variance threshold represents the minimum variance level at which a local surface of a point cloud is considered to be disordered.

[0128] In some embodiments, optionally, a preset curvature threshold is used as a benchmark value for determining the degree of surface curvature, and is statistically derived from labeled dust and dust samples and non-dust samples. The preset curvature threshold represents the minimum curvature level at which a local surface of a point cloud is considered to be uneven.

[0129] In one specific embodiment, optionally, such as Figure 8 As shown, the dust filtration methods include:

[0130] Step S200: Input point cloud;

[0131] Step S202: Filter out ground point clouds based on elevation differences;

[0132] Step S204: If the reflection intensity is greater than a preset threshold, a judgment is made;

[0133] If the judgment result of step S204 is negative, that is, the reflection intensity is less than or equal to the preset threshold, then step S2042 is executed: retain the point cloud;

[0134] When the judgment result of step S204 is yes, that is, the reflection intensity is greater than the preset threshold, step S206 is executed: confirm the candidate dust point;

[0135] Step S208: Calculate the normal vector and curvature;

[0136] The determination is made by step S210: whether the average angle between the normal vectors in the neighborhood is greater than the threshold.

[0137] If the judgment result of step S210 is negative, that is, if the average angle of the normal vectors within the neighborhood is less than or equal to the threshold, then step S2042 is executed: retain the point cloud.

[0138] If the judgment result of step S210 is yes, that is, within the neighborhood, the average included angle of the normal vector is greater than the threshold, then step S212 is executed: whether the curvature is greater than the set threshold.

[0139] If the judgment result of step S212 is yes, proceed to step S214: determine it as a dust spot.

[0140] In step S204: When the reflection intensity is greater than a preset threshold, during the process of judging the reflection intensity, every hour, dust points are segmented according to the semantic segmentation of the point cloud deep learning, and the maximum value of the reflection intensity of the dust points is counted as the threshold correction.

[0141] Specifically, the process begins with removing the ground surface. Considering the unevenness of roads in the mining area, a slope filtering algorithm is used. This involves projecting the point cloud onto a grid and calculating the lowest point of each cell. The elevation difference and horizontal distance between the lowest points of adjacent cells are compared; if the slope exceeds a threshold, the cell is considered non-ground.

[0142] Analyzing the reflection intensity of non-ground points: The reflection intensity of dust points is lower than that of normal targets, but there is overlap in the reflection intensity information between real targets and dust points. Directly using a set threshold for binary classification can lead to over-filtering or under-filtering. Therefore, this paper adopts a pre-selection approach, designating points with a reflection intensity threshold below the set threshold as candidate dust points. Furthermore, the threshold is periodically updated based on a deep learning point cloud dust target segmentation algorithm. Every hour, the deep learning algorithm is activated to statistically analyze the reflection intensity information of the segmented dust point cloud, and the maximum value is used as the reflection intensity segmentation threshold.

[0143] The normal vector of the point cloud marked as a candidate dust point is calculated. The variance of the angle between the normal vector of the current point and the normal vectors of the surrounding points, as well as the curvature, are calculated. If the variance is greater than a set threshold and the curvature is also greater than a set threshold, the current point is determined to be a dust point.

[0144] like Figure 9 As shown, multiple line segments represent multiple laser pulses emitted outward by the lidar, and multiple points on the line segments represent dust particles. During the continuous emission of multiple laser pulses by the lidar, the dust particles are marked and filtered out in real time.

[0145] Optionally, without considering real-time dust filtering, a deep learning point cloud segmentation algorithm can be used to segment the point cloud data.

[0146] Understandably, considering the unevenness of the road surface in the mining area, using height difference for ground extraction is more accurate than directly using plane fitting. Using a fixed threshold setting will lead to over- or under-filtering of dust. Using deep learning algorithms to use the reflection intensity value of the extracted dust points as a priori value can improve the dust filtering accuracy. Furthermore, by using reflection intensity for dust pre-labeling, the processing speed of the algorithm can be improved to meet the real-time requirements of autonomous driving.

[0147] like Figure 7 As shown in the illustration, this application embodiment also provides a dust removal device 900, which includes: a point cloud determination module 902 for determining the original point cloud data corresponding to a lidar; a data segmentation module 904 for performing ground segmentation processing on the original point cloud data to determine multiple non-ground points; a reflection intensity module 906 for determining the reflection intensity characteristics of the non-ground points; a dust candidate module 908 for determining candidate dust points based on the reflection intensity characteristics; a feature determination module 910 for determining the variance of the normal vector angle and curvature characteristics corresponding to the candidate dust points; a dust determination module 912 for determining at least one dust point in the original point cloud data based on the variance of the normal vector angle and curvature characteristics; and a dust removal module 914 for removing the dust points to determine the dust-filtered point cloud data corresponding to the original point cloud data.

[0148] This application also provides a vehicle, which includes a body and a dust removal device installed inside the body.

[0149] Optionally, the vehicle is an autonomous mining truck.

[0150] Optionally, the vehicle is equipped with at least one lidar.

[0151] This application also provides a readable storage medium storing a program or instructions. When executed by a processor, the program or instructions implement the various processes of the dust removal method embodiments described above, achieving the same technical effects. To avoid repetition, these will not be elaborated upon here. Furthermore, the readable storage medium improves the data storage capacity and data processing speed of the method in this application.

[0152] A computer-readable storage medium can be a tangible device that holds and stores instructions for use by an instruction execution device. A computer-readable storage medium can be an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing, but is not limited thereto. A non-exhaustive list of more specific examples of computer-readable storage media includes: portable computer floppy disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable optical disc read-only memory (CD-ROM), digital universal disk (DVD), memory cards, floppy disks, encoding mechanical devices (e.g., punched cards or grooves with raised structures for recording instructions), and any suitable combination of the foregoing. The computer-readable storage medium used herein should not be construed as the transmission of signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media, or electrical signals transmitted through wires.

[0153] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0154] This application also provides a chip, which includes a processor and a communication interface. The communication interface and the processor are coupled. The processor is used to run programs or instructions to implement the various processes of the above-described dust filtration method embodiments and achieve the same technical effects. To avoid repetition, it will not be described again here. Furthermore, the chip improves the data processing speed corresponding to the dust filtration method in this application.

[0155] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0156] In this invention, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance; the term "multiple" refers to two or more unless otherwise explicitly defined. The terms "install," "connect," "link," and "fix" should be interpreted broadly. For example, "connect" can be a fixed connection, a detachable connection, or an integral connection; "link" can be a direct connection or an indirect connection through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0157] In the description of this invention, it should be understood that the terms "upper," "lower," "left," "right," "front," "rear," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or unit referred to must have a specific orientation or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0158] In the description of this specification, the terms "one embodiment," "some embodiments," "specific embodiment," etc., refer to a specific feature, structure, material, or characteristic described in connection with that embodiment or example, which is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0159] The above are merely preferred embodiments of the present invention and are not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for dust filtration, characterized in that, include: Determine the original point cloud data corresponding to the lidar; The original point cloud data is subjected to ground segmentation processing to identify multiple non-ground points; Determine the reflection intensity characteristics of the non-ground points; Candidate dust points are determined based on the reflection intensity characteristics; Determine the variance of the normal vector angle and curvature characteristics corresponding to the candidate dust points; At least one dust point in the original point cloud data is determined based on the variance of the included angle of the normal vector and the curvature feature. The dust points are filtered out to determine the dust-filtered point cloud data corresponding to the original point cloud data.

2. The dust filtration method according to claim 1, characterized in that, The step of performing ground segmentation processing on the original point cloud data to identify multiple non-ground points includes: A slope filtering algorithm is used to project the original point cloud data into a two-dimensional grid, which includes multiple grid cells. Determine the lowest point of each grid cell; Determine the height difference and horizontal distance between adjacent lowest points; The slope corresponding to adjacent grid cells is determined based on the height difference and the horizontal distance. Determine the slope angle threshold; Non-ground points are determined based on the slope and the slope angle threshold, wherein the slope corresponding to the non-ground point is greater than the slope angle threshold.

3. The dust removal method according to claim 1, characterized in that, The step of determining candidate dust points based on the reflection intensity characteristics includes: The reflection intensity corresponding to the non-ground point is determined based on the reflection intensity characteristics; Determine the deep learning dust segmentation model; The dynamic threshold is determined based on the deep learning dust segmentation model. Non-ground points with a reflection intensity less than the dynamic threshold are identified as candidate dust points.

4. The dust filtration method according to claim 3, characterized in that, The step of determining the dynamic threshold based on the deep learning dust segmentation model includes: Determine the point cloud segmentation model and the corresponding startup cycle of the point cloud segmentation model; Dust point clouds are extracted by periodically starting the point cloud segmentation model according to the aforementioned startup cycle; Determine the reflection intensity values ​​corresponding to the plurality of dust point clouds; Determine the maximum reflection intensity value among the plurality of reflection intensity values; The dynamic threshold is determined based on the maximum reflection intensity value.

5. The dust filtration method according to claim 1, characterized in that, The determination of the variance of the normal vector angle and curvature features corresponding to the candidate dust points includes: For each candidate dust point, an adjacent point set is established, the adjacent point set including multiple adjacent points closest to the current candidate dust point; Determine the covariance matrix corresponding to the set of adjacent points; The principal normal vector corresponding to the candidate dust point is determined based on the covariance matrix. Traverse each neighboring point in the set of neighboring points and calculate the normal vector corresponding to the neighboring point. Determine the set of angles between the normal vector and the principal normal vector; The variance of the included angle of the normal vector is determined based on the set of included angles. The covariance matrix is ​​decomposed into eigenvalues ​​to determine multiple eigenvalues; Curvature features are determined based on multiple of the aforementioned feature values.

6. The dust removal method according to any one of claims 1 to 5, characterized in that, Determining at least one dust point in the original point cloud data based on the variance of the normal vector angle and the curvature feature includes: Determine the preset variance threshold and preset curvature threshold respectively; When the variance of the angle between the normal vectors corresponding to the candidate dust point is greater than the preset variance threshold, and the curvature feature corresponding to the candidate dust point is greater than the preset curvature threshold, the candidate dust point is determined to be a dust point in the original point cloud data.

7. A dust removal device, characterized in that, include: The point cloud determination module is used to determine the raw point cloud data corresponding to the lidar. The data segmentation module is used to perform ground segmentation processing on the original point cloud data to identify multiple non-ground points; A reflection intensity module is used to determine the reflection intensity characteristics of the non-ground points; A dust candidate module is used to determine candidate dust points based on the reflection intensity characteristics. The feature determination module is used to determine the variance of the normal vector angle and curvature features corresponding to the candidate dust points; A dust determination module is used to determine at least one dust point in the original point cloud data based on the variance of the normal vector angle and the curvature feature. The dust removal module is used to remove the dust points and determine the dust-filtered point cloud data corresponding to the original point cloud data.

8. A vehicle, characterized in that, Includes: a vehicle body, wherein the vehicle body is provided with the dust filtration device as described in claim 7.

9. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the dust filtration method as described in any one of claims 1 to 6.

10. A chip, characterized in that, The chip includes a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the steps of the dust filtration method as described in any one of claims 1 to 6.

Images