Method for removing dust points in point cloud of mining area environment

Abstract

The invention relates to a method for removing dust points in a point cloud of a mining area environment. The method comprises the following steps: acquiring an original point cloud of a surrounding environment of an unmanned mine card; determining a region of interest in the surrounding environment of the unmanned mine card, and removing points outside the region of interest from the original point cloud to obtain a point cloud of the region of interest; identifying and removing ground points in the point cloud of the region of interest; clustering the remaining point clouds of the region of interest after the ground points are removed so as to obtain a plurality of point cloud clusters which are mutually independent in space; and performing dust point identification on each point cloud cluster to obtain object points after dust points are removed.

Description

Technical Field

[0001] The invention belongs to the technical field of signal processing, and in particular relates to a method for removing dust points in a point cloud of a mining environment. Background Art

[0002] In modern mining operations, unmanned mining trucks are increasingly used to complete loading and transportation tasks. In order to operate normally, unmanned mining trucks usually have a perception system to detect the status of the unmanned mining truck itself and its surroundings. LiDAR is an important sensor in the perception system of unmanned mining trucks. Compared with cameras and millimeter-wave radars, LiDAR can provide more accurate three-dimensional information about the surrounding environment of unmanned mining trucks. However, the LiDAR beam will be blocked by dust particles and generate dust noise in the dusty mining environment. These dust noise points may be mistakenly identified as obstacles by the LiDAR perception algorithm. Therefore, removing dust noise points from the point cloud detected by the LiDAR is an important prerequisite for ensuring the accuracy of LiDAR perception.

[0003] Existing dust removal methods can be divided into deep learning-based methods and traditional methods based on whether neural networks are used. Deep learning-based methods usually require a large amount of point-by-point labeled training data. The collection of such data is very expensive and time-consuming. Moreover, when deep learning-based methods are used in scenarios not covered by the training data, the performance of the method will drop sharply.

[0004] The existing traditional methods include ROR, SOR, DROR, DSOR, LIOR and LIDROR. These methods mainly use the spatial sparsity or reflection intensity characteristics of dust particles to remove dust noise. Although some traditional methods can effectively remove most of the dust noise in the point cloud, these methods cannot meet the real-time application requirements of unmanned mining trucks due to their slow calculation speed. Summary of the invention

[0005] The object of the present invention is to provide a method for removing dust points in a point cloud of a mining environment, which can overcome at least one of the above-mentioned drawbacks of the prior art.

[0006] This object is achieved by a method and a computer-readable storage medium having the following features.

[0007] According to one aspect of the present invention, a method for removing dust points in a point cloud of a mining environment is proposed, the method comprising: obtaining an original point cloud of the surrounding environment of an unmanned mining truck, in particular, the original point cloud is obtained by laser radar scanning; determining an area of ​​interest in the surrounding environment of the unmanned mining truck, and removing points outside the area of ​​interest from the original point cloud to obtain a point cloud of the area of ​​interest; identifying and removing ground points in the point cloud of the area of ​​interest; clustering the point cloud of the area of ​​interest remaining after removing the ground points, thereby obtaining a plurality of spatially independent point cloud clusters, preferably, clustering is performed using a voxel-based clustering method; and identifying dust points for each point cloud cluster respectively to obtain object points after removing the dust points.

[0008] According to another aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored. The computer program includes executable instructions. When the executable instructions are executed by a processor, the method described above is implemented.

[0009] Therefore, the present invention proposes an efficient point cloud dust removal processing method based on the traditional method for the application scenario of unmanned mining trucks. The method has multiple advantages. First, the point cloud outside the road boundary and the point cloud in other unconcerned areas are filtered out by using the region of interest, thereby reducing the number of point clouds that need to be calculated for point cloud dust removal. Then, by using the feature that the dust noise points are higher than the ground, the ground point cloud is filtered out by ground segmentation, that is, identifying the ground points, further reducing the number of point clouds for point cloud dust removal calculation. Then, the non-ground point cloud is clustered, especially voxel clustering, to obtain a plurality of point cloud clusters separated from each other in space. Therefore, the point cloud filtering operations between each point cloud cluster can be independent of each other, and the speed of point cloud dust removal calculation can be improved by parallel operation. Finally, for each point cloud cluster, firstly, the dust points and object points are preliminarily distinguished based on the reflection intensity characteristics of each point, and then the spatial sparsity of each point and the shape and size of the dust point cloud cluster are preferably additionally tracked to restore the object points that are misjudged as dust points to prevent collision accidents. Therefore, the present invention proposes a more accurate and faster dust removal algorithm for the mine environment, which can not only meet the real-time requirements of unmanned mining trucks, but also has high accuracy and robustness. The object points after removing the dust points accurately indicate the obstacles on the mine road that may affect the driving of the unmanned mining truck, so that the driving route of the unmanned mining truck can be well planned based on these accurately detected obstacles.

[0010] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the scope of protection of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The present invention is described in detail below with the aid of the accompanying drawings. In the accompanying drawings,

[0012] Figure 1 A flow chart showing a method for removing dust points in a point cloud of a mining environment according to the present invention is shown. DETAILED DESCRIPTION

[0013] exist Figure 1 In the point cloud dust removal method according to the present invention shown, first, in step S1, the original point cloud of the surrounding environment of the unmanned mining truck is obtained. The original point cloud can be scanned by a laser radar installed on the unmanned mining truck, such as a front laser radar and / or a rear laser radar. Therefore, the original point cloud is in a vehicle body coordinate system fixed relative to the unmanned mining truck. The coordinate origin of the vehicle body coordinate system is, for example, located at the position of the laser radar that generates the original point cloud. Of course, other coordinate origin positions are also feasible, such as the center of gravity position of the unmanned mining truck. The x-axis of the vehicle body coordinate system is parallel to the longitudinal axis of the unmanned mining truck, the y-axis is parallel to the transverse axis of the unmanned mining truck that is perpendicular to the longitudinal axis, and the z-axis is perpendicular to the x-axis and the y-axis.

[0014] Alternatively, the original point cloud may also be generated by other sensor devices such as one or more of a millimeter wave radar, a depth camera, and a stereo camera, or by a combination of a laser radar and other sensor devices. The following is only described using a laser radar as an example, but the method steps described below are also applicable to dust removal of the original point cloud generated by one or more of the above sensor devices.

[0015] Next, in step S2, an area of ​​interest is determined in the surrounding environment of the unmanned mining truck, and points outside the area of ​​interest are removed from the original point cloud to obtain a point cloud of the area of ​​interest. The main reason why the existing traditional dust removal methods have slow calculation speed is that these methods directly use the nearest neighbor search method for all points of the original point cloud detected by the laser radar to identify dust points based on the spatial characteristics that dust points are sparser than object points. In the method according to the present invention, in order to improve the calculation speed and realize real-time point cloud dust removal, before identifying dust points, the area of ​​interest point cloud filter is first used to filter out points outside the area of ​​interest to reduce the number of point clouds that need to be dust-removed. That is, the area of ​​interest is determined in the surrounding environment of the unmanned mining truck, and only the point cloud within the area of ​​interest is dust-removed. As a result, the number of point clouds that need to be dust-removed is significantly reduced compared with the original point cloud. The area of ​​interest can be, for example, an area of ​​the surrounding environment of the unmanned mining truck that is located within the road boundary line of the road where the unmanned mining truck is located and is lower than a preset height threshold. Here, the road boundary line can be determined with the help of a digital map, especially a high-precision map, wherein, based on the posture information of the unmanned mining truck body, the coordinates of the road boundary line in the digital map are converted into the body coordinate system used by the original point cloud or the point cloud of the area of ​​interest. Then, the point cloud located outside the road boundary line is removed with the road boundary line converted into the body coordinate system as the filtering condition. Here, the posture information of the unmanned mining truck body may include the position and orientation of the unmanned mining truck on the road, which are originally required and detected by other vehicle systems. For example, these posture information can be detected by sensors on the unmanned mining truck or on the roadside, such as cameras and / or position sensors (such as GPS sensors) and / or speed sensors and / or acceleration sensors. The digital map can be stored in the unmanned mining truck, or retrieved by the unmanned mining truck from the background server.

[0016] The boundary of the region of interest in the height direction is divided by a preset height threshold. The height threshold is based on the coordinate origin of the vehicle body coordinate system. That is to say, the point cloud whose height coordinate in the vehicle body coordinate system is greater than the height threshold is filtered out. The height threshold can be set according to the external parameters of the unmanned mining truck, especially the vehicle body height and / or the position of the coordinate origin of the vehicle body coordinate system, such as the installation position of the laser radar on the unmanned mining truck. For example, the height threshold can be the height difference between the vehicle body height of the unmanned mining truck and the position height of the coordinate origin, or 1.1 times, 1.2 times or 1.5 times the height difference. Alternatively, the height threshold can also be the vehicle body height of the unmanned mining truck, or 0.8 times, 0.6 times or 0.5 times the vehicle body height. It should be noted here that the height boundary surface of the region of interest should be higher than the roof of the unmanned mining truck, or at least flush with the roof.

[0017] Then, in step S3, ground points are identified and removed from the point cloud of the area of ​​interest. During the driving and transportation of unmanned mining trucks in the mining area, more than half of the points in the point cloud are ground points, while dust and noise points are all non-ground points. Therefore, by identifying and removing ground points, that is, performing ground segmentation, the number of point clouds can be further reduced. Here, for example, ground segmentation can be performed by a plane fitting method. To this end, the point cloud of the area of ​​interest is divided into multiple sub-areas, that is, multiple grids (step S3.1), and plane fitting is performed on each grid respectively. This regional plane fitting has good adaptability to non-flat roads in mining areas. The grid can be constructed in the form of a polar coordinate grid, and the origin of the grid can coincide with the coordinate origin of the point cloud of the area of ​​interest, that is, the coordinate origin of the vehicle body coordinate system.

[0018] It should be noted here that although the point cloud is divided into multiple regions by the polar coordinate grid, the coordinates of each point can still be expressed in Cartesian coordinates. Here, it is possible to use both relative Cartesian coordinates relative to the grid origin and absolute Cartesian coordinates in the world coordinate system. In addition, in addition to the polar coordinate grid, a Cartesian grid can also be used to divide the point cloud of interest.

[0019] Then, in step S3.2, the best fit plane is determined for each grid. Here, for each grid, first select N p (For example, 20) lowest points are used as the initial point set for plane fitting, and the initial fitting plane is obtained based on the initial point set. The lowest point refers to the point at the bottom of a grid, that is, the point with the smallest height coordinate. Then the distance between each point in the grid and the initial fitting plane is calculated. If the distance is less than a preset distance threshold, the point is determined to belong to the initial fitting plane. After traversing all points in a grid, all points in the grid that belong to the initial fitting plane are re-plane fitted to obtain a new fitting plane. Then the distance between each point in the grid and the new fitting plane is recalculated, and based on the comparison of the distance with the distance threshold, it is determined whether the point belongs to the new fitting plane. And so on, iterate several times until the fitting plane converges, and the best fitting plane of the grid is obtained.

[0020] Next, in step S3.3, for each grid, the normal direction of the best fit plane is compared with the reference direction, and the point in the grid is determined to be a ground point or a non-ground point based on the comparison result. The reference direction can be, for example, the normal direction of the ground below the unmanned mining truck, or the height direction of the unmanned mining truck. For comparison, for example, the normal vector of the best fit plane can be calculated. And the normal vector The angle between the normal vector and the reference direction is compared with a preset angle threshold (e.g., 15°). If the angle with the reference direction is less than the angle threshold, the points in the grid that belong to the best fit plane are determined as ground points, and the remaining points in the grid are determined as non-ground points. If the angle is greater than the angle threshold, all points in the grid are determined as non-ground points.

[0021] In addition to the above-mentioned regional plane fitting method, any other known method suitable for ground segmentation may also be used.

[0022] After all ground points are removed, in step S4, the point cloud of the remaining area of ​​interest is clustered, that is, all non-ground points are clustered, thereby obtaining a plurality of spatially independent point cloud clusters. These point cloud clusters may contain only object points (i.e., obstacle clustering), or only dust points (i.e., dust clustering), or both object points and dust points (i.e., comprehensive clustering of obstacles and dust). Any suitable clustering method can be used here, preferably, a voxel-based clustering method can be used. The voxel-based clustering method clusters voxels including multiple points, so compared with the point-based clustering method, the voxel-based clustering method is more efficient.

[0023] In step S5, dust point recognition is performed on each point cloud cluster to obtain object points after dust points are removed. Here, dust point recognition of each point cloud cluster can be performed in parallel, because the dust filtering calculation between each point cloud cluster is independent of each other. Therefore, the multi-threaded parallel computing capability of the processor can be fully utilized to remove dust from each point cloud cluster at the same time.

[0024] To this end, in each point cloud cluster, dust points and object points are first preliminarily identified based on the reflection intensity characteristics of each point (step S5.1). Existing dust removal algorithms usually identify points with reflectivity lower than a preset threshold as dust points, because the reflection intensity of most dust particles is much smaller than the reflection intensity of other objects, such as obstacles on the road. In the method according to the present invention, in addition to the above characteristics, it is also considered that the dust close to the ground in the mining area will also show a great reflection intensity, such as the dust caused by the air vortex at the tail of the unmanned mining truck in the mining area and the dust about to fall to the ground. Therefore, in the method of the present invention, not only the points with reflection intensity lower than the preset lower reflection threshold, but also the points with reflection intensity higher than the preset upper reflection threshold are marked as dust points, wherein the upper reflection threshold is higher than the lower reflection threshold. Conversely, the points with reflection intensity between the lower reflection threshold and the upper reflection threshold are marked as object points.

[0025] However, the points with extremely low or high reflection intensity are not necessarily dust points. Some objects may also have extremely low or high reflection intensity due to their material properties or the different angles and distances at which the laser radar beam is incident on their surface. Therefore, in step S5.2, object points are recovered from the dust points initially identified according to the spatial sparsity of the points. Here, based on the spatial characteristics that dust points are sparser than object points, the nearest neighbor search method is used to identify and recover object points from the dust points determined in step S5.1. To this end, for each point initially identified as a dust point, the neighboring points within the search radius around the point are searched. If the number of neighboring points is greater than the preset neighboring point number threshold, the point and its neighboring points are re-marked as object points. For the neighboring points re-marked as object points, the neighboring points are no longer searched. The neighboring point number threshold is determined according to the angular resolution of the laser radar. For example, the neighboring point number threshold can be 5. The search radius is positively correlated with the scanning distance of the laser radar. That is, the larger the scanning distance, the larger the search radius is selected, and conversely, the smaller the scanning distance, the smaller the search radius is selected. In step S5.2, unlike the existing dust removal algorithm, in the existing dust removal algorithm, when a point with a large number of neighboring points is found, the point is marked as an object point, but the neighboring points of the point are searched again, thus traversing all points. In the method of the present invention, once a point is marked as an object point, its neighboring points are marked as object points, and there is no need to search for neighboring points of the neighboring point again. Therefore, compared with the existing dust removal algorithm, the amount of calculation is further reduced. Restoring object points from the initially identified dust points can prevent collision accidents caused by missed obstacle detection.

[0026] Optionally, in step S5.3, the dust points remaining after step S5.2 are further tracked in the form of point cloud clusters in multiple, preferably continuous, laser radar scanning frames, and the object point cloud clusters are identified and restored again based on the shape and size stability of the dust point cloud clusters. During the transportation of unmanned mining trucks, the shape and size of the dust on the road generally change dynamically. Therefore, if the shape and size of the target point cloud cluster are stable, it is considered that the target point cloud cluster is not dust but an obstacle, and the point cloud cluster is re-marked as an object point cloud cluster. In other words, the dust point cloud cluster that maintains a stable shape and size in multiple scanning frames is restored to an object point cloud cluster. The use of target tracking technology can further improve the accuracy and robustness of the dust removal results. Therefore, compared with the existing dust removal algorithms, the dust removal algorithm described in step S5 of the present invention is more accurate and faster.

[0027] In another variant of the present invention, after the dust points in each point cloud cluster are identified in step S5, these dust points can be removed from the original point cloud, thereby obtaining a more "clear" surrounding environment detection result with dust noise removed, which includes obstacles and ground point clouds. This clearer detection result can be used for subsequent processing or used by other vehicle devices of the unmanned mining truck for other vehicle functions. Therefore, in this variant, the method according to the present invention is not used to accurately identify obstacles in the area of ​​interest, but to obtain more accurate detection results of the sensor device.

[0028] In an exemplary embodiment of the present invention, a computer-readable storage medium is also provided, on which a computer program is stored, the program including executable instructions, which can implement the steps of the point cloud dust removal method described in the above embodiment when the executable instructions are executed by, for example, a processor. In some possible implementations, various aspects of the present invention can also be implemented in the form of a computer program product, which includes a program code, and when the computer program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps of various exemplary embodiments of the present invention described in the point cloud dust removal method in this specification.

Claims

1. A method for removing dust points in a point cloud of a mining environment, the method comprising the following steps: Obtaining the original point cloud of the surrounding environment of the unmanned mining truck, especially, the original point cloud is obtained by laser radar scanning; Determine an area of ​​interest in the surrounding environment of the unmanned mining truck, and remove points outside the area of ​​interest from the original point cloud to obtain a point cloud of the area of ​​interest; Identify and remove ground points in the point cloud in the area of ​​interest; Clustering the point clouds of the area of ​​interest remaining after removing the ground points, so as to obtain a plurality of spatially independent point cloud clusters, preferably, using a voxel-based clustering method to perform clustering; Dust points are identified for each point cloud cluster to obtain object points after removing dust points.

2. The method according to claim 1, characterized in that Dust points are identified for each point cloud cluster in parallel.

3. The method according to claim 1 or 2, characterized in that: Identifying dust points for each point cloud cluster includes: for each point cloud cluster, Preliminary identification of dust points and object points based on reflection intensity characteristics of the points; and Object points are recovered from the initially identified dust points according to the spatial sparsity of the points.

4. The method according to claim 3, characterized in that Dust point identification for each point cloud cluster also includes: The dust points remaining after the object points are restored according to the spatial sparsity are tracked in the form of point cloud clusters in multiple, preferably continuous, scanning frames, and the object point cloud clusters are restored according to the shape and size stability of the dust point cloud clusters.

5. The method according to claim 3 or 4, characterized in that: Preliminary identification of dust points and object points based on the reflection intensity characteristics of the points includes: Points with a reflectivity higher than a preset upper reflection threshold or lower than a preset lower reflection threshold are identified as dust points, wherein the upper reflection threshold is higher than the lower reflection threshold.

6. The method according to any one of claims 3 to 5, characterized in that Recovering object points from initially identified dust points based on the spatial sparsity of the points includes: For each point initially identified as a dust point, the nearest neighbor search method is used to search for neighboring points within the search radius around the point. If the number of neighboring points is greater than the preset threshold of the number of neighboring points, the point and its neighboring points are re-marked as object points. For the neighboring points re-marked as object points, the neighboring point search is no longer performed.

7. The method according to claim 6, characterized in that The threshold value of the number of neighboring points is set according to the angular resolution of the laser radar that generates the original point cloud, for example, the threshold value of the number of neighboring points is 5; and / or, the search radius is selected in a positive correlation with the scanning distance of the laser radar that generates the original point cloud.

8. The method according to any one of claims 4 to 7, characterized in that Restoring the object point cloud cluster based on the shape and size stability of the dust point cloud cluster includes: The dust point cloud clusters that maintain stable shapes and sizes in multiple scanning frames are restored as object point cloud clusters.

9. A computer-readable storage medium having a computer program stored thereon, the computer program comprising executable instructions, and when the executable instructions are executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images