A method for denoising point cloud of mine laser radar

Abstract

The present invention provides a mine laser radar point cloud denoising method, comprising: each laser transmitter in a mechanical rotating laser radar emits a laser beam into the mine to generate a frame of point cloud; according to the vertical angle of the laser transmitter, the serial number of the laser transmitter is sequentially reconstructed; a two-dimensional point cloud grid map based on a frame of point cloud is constructed; a multi-layer difference stability feature map is extracted from the two-dimensional point cloud grid map, and a difference stability fusion feature map is generated after fusion; point cloud pixels greater than a predetermined difference stability empirical threshold are removed from the difference stability fusion feature map to achieve denoising. The mine point cloud denoising method of the present invention effectively improves the point cloud denoising accuracy in a mine environment.

Description

Technical Field

[0001] The invention belongs to the field of measurement technology, and in particular relates to a mine laser radar point cloud denoising method. Background Art

[0002] Dust is generated in underground mines, which is mixed with toxic gases such as carbon monoxide and sulfur dioxide, as well as harmful gases such as methane and nitrogen. In order to reduce the concentration of dust in the air, many water spray devices are installed underground, which spray water mist in the air. When unmanned vehicles pass through these water mists, the water mist will cause dense point cloud noise in the on-board LiDAR. These noises will affect the effectiveness of unmanned tasks such as target detection and SLAM (simultaneous localization and mapping). At present, the methods for removing LiDAR point cloud noise are mainly divided into three categories:

[0003] The first method assumes that the intensity of point cloud noise is very small and removes point cloud noise by using an intensity threshold. This method is affected by the laser wavelength and the target reflectivity. The target reflectivity is mainly related to the target material and the distance between the target and the sensor. Therefore, the point cloud intensity on some targets is very small, and the noise intensity of a small number of point clouds near the sensor is very large.

[0004] The second method assumes that the point cloud noise density is relatively small and removes the point cloud noise by using the method of removing external points. However, rainfall, dust and water mist in underground mines around the sensor make the point cloud noise density generated by the lidar very large.

[0005] The last method uses spherical projection to obtain the point cloud depth map, and then uses convolutional neural network to identify point cloud noise. Since neural networks require a very large amount of data, this type of method loses a large number of points in the projection process in order to reduce the amount of calculation, and a large amount of effective training data is difficult to obtain, so this type of method has poor denoising effect.

[0006] In view of this, it is urgent to propose a mine lidar point cloud denoising method to improve the denoising accuracy. Summary of the invention

[0007] To this end, the present invention proposes a mine laser radar point cloud denoising method to improve the denoising accuracy.

[0008] The mine laser radar point cloud denoising method of the present invention comprises the following steps:

[0009] S1. Each laser transmitter in the mechanical rotating laser radar transmits a laser beam into the mine to generate a frame of point cloud; the frame of point cloud is a set of point clouds generated by the lasers emitted by each laser transmitter of the mechanical rotating radar rotating 360°;

[0010] S2, reconstructing the serial numbers of the laser emitters in order according to the vertical angles of the laser emitters;

[0011] S3, constructing a two-dimensional point cloud grid map based on a frame of point cloud; the specific process of constructing a two-dimensional point cloud grid map is:

[0012] S301, using the reconstructed laser transmitter serial number as the ordinate and the serial number of the laser beam emitted by the same laser transmitter as the abscissa;

[0013] S302, sequentially generate multi-layer grids using the three-dimensional coordinates x, y, z and intensity of the point cloud;

[0014] S303, obtaining a 4-layer 2D point cloud grid map;

[0015] S4, extracting multiple layers of difference stability feature maps from the two-dimensional point cloud grid map, and fusing them to generate a difference stability fusion feature map;

[0016] S5. Remove point cloud pixels greater than a predetermined difference stability empirical threshold from the difference stability fusion feature map to achieve denoising.

[0017] Further, in step S4, the multi-layer difference stability feature map includes a tangent angle difference stability feature map, an adjacent line distance difference stability feature map and an intensity difference stability feature map.

[0018] Furthermore, the extraction process of the tangent angle difference stability characteristic map is as follows:

[0019] S401a, f x and f y Use 1×3 convolution kernel [1 0 -1] to perform convolution operation and obtain feature map and

[0020] in,

[0021] f x is a two-dimensional point cloud grid map with a grid value of x;

[0022] f y is a two-dimensional point cloud grid map with grid value y;

[0023] x is the x-axis coordinate of the point cloud;

[0024] y is the y-axis coordinate of the point cloud;

[0025] S401b、 Each grid value in is divided by The grid value corresponding to the index position in the grid, and then the inverse tangent is calculated to obtain the tangent angle feature map

[0026] S401c, yes Use 1×3 convolution kernel [1 -1 0] to perform convolution operation to obtain the tangent angle difference feature map

[0027] S401d, yes The standard deviation is calculated using a 1×5 standard deviation kernel to obtain the tangent angle difference stability characteristic graph f 0 .

[0028] Furthermore, the extraction process of the adjacent line distance difference stability characteristic map is as follows:

[0029] S402a, f x 、f y and f z Use 3×1 convolution kernel [1 -1 0] respectively T The convolution operation can obtain the feature map f 1 0 、f 1 1 and f 1 2 ;

[0030] in,

[0031] f x is a two-dimensional point cloud grid map with a grid value of x;

[0032] f y is a two-dimensional point cloud grid map with grid value y;

[0033] f z is a two-dimensional point cloud grid map with a grid value of z;

[0034] x is the x-axis coordinate of the point cloud;

[0035] y is the y-axis coordinate of the point cloud;

[0036] z is the z-axis coordinate of the point cloud;

[0037] S402b, f 1 0 、f 1 1 and f 1 2 The grid values ​​with the same index position are first squared, then added, and finally squared to obtain the adjacent line distance difference feature map f 1 3 ;

[0038] S403c, f 1 3Use 1×3 convolution kernel [1 -1 0] to perform convolution operation and obtain the adjacent line distance difference feature map f 1 4 ;

[0039] S403d, f 1 4 The standard deviation is calculated using the 1×5 standard deviation kernel to obtain the stability characteristic graph f of the adjacent line distance difference. 1 .

[0040] Furthermore, the extraction process of the intensity difference stability characteristic map is as follows:

[0041] S403a, f intensity Use 1×3 convolution kernel [1 -1 0] to perform convolution operation to obtain the intensity difference feature map

[0042] in,

[0043] f intensity A two-dimensional point cloud grid map with a grid value of intensity;

[0044] intensity is the intensity value of the point cloud;

[0045] S403b, yes The standard deviation is calculated using a 1×5 standard deviation kernel to obtain the intensity difference stability characteristic map f 2 .

[0046] Furthermore, the following formula is used to fuse multiple layers of difference stability feature maps to generate a difference stability fusion feature map:

[0047]

[0048] in,

[0049] f i is the difference stability feature map of the i-th layer;

[0050] w i is the weight of the difference stability feature map of the i-th layer;

[0051] The weight is the experience value;

[0052] n is the number of layers of the difference stability feature map;

[0053] F is the difference stability fusion feature map.

[0054] Furthermore, in step S2, when reconstructing the serial numbers of the laser emitters, the laser emitters are sorted from small to large according to their vertical angles.

[0055] Furthermore, in step S1, when the laser transmitter emits a laser beam into the mine, the laser beam is emitted in a fixed rotation direction and at a fixed interval angle.

[0056] Furthermore, each of the laser emitters emits 1800 laser beams per rotation of 360°.

[0057] The above technical solution of the present invention has the following advantages compared with the prior art:

[0058] The present invention constructs a two-dimensional point cloud grid map based on the vertical angle size of the laser emitter and the order in which the same laser emitter emits laser beams. Then, a difference stability fusion feature map is extracted based on the two-dimensional point cloud grid map, and finally, an empirical threshold is used to remove noise based on the difference stability fusion feature map.

[0059] In the application process, the present invention does not need to use the assumption that the point cloud noise intensity value is very small and the point cloud noise density is relatively small, and is not affected by the size of the point cloud noise density, nor does it require a large amount of training data. After testing, the accuracy of this method in identifying the point cloud noise generated by water mist in the mine has been significantly improved compared with the existing technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 is a flow chart of a method provided by an embodiment of the present invention;

[0061] Figure 2 is a schematic diagram of a two-dimensional point cloud grid map provided by an embodiment of the present invention;

[0062] Figure 3 is a convolutional network schematic diagram of a tangent angle difference stability characteristic map provided by an embodiment of the present invention;

[0063] Figure 4 is a schematic diagram of a convolutional network of a stability feature map of adjacent line distance differences provided by an embodiment of the present invention;

[0064] Figure 5 Schematic diagram of a convolutional network of an intensity difference stability feature map provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0065] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0066] The mine laser radar point cloud denoising method of this embodiment is as follows: Figure 1 As shown, the following steps are included:

[0067] S1. Each laser transmitter in the mechanical rotating laser radar transmits a laser beam into the mine to generate a frame. The frame of point cloud is a set of point clouds generated by the lasers emitted by each laser transmitter of the mechanical rotating laser radar rotating 360°.

[0068] When the laser emitter emits a laser beam into the mine, it emits the laser beam in a fixed rotation direction and at a fixed interval angle. Each laser emitter emits 1800 laser beams per 360° rotation. The laser emitter completes the collection of one frame of point cloud data after rotating 360°. At the same time, the index of the point cloud is the order in which the laser beam is emitted.

[0069] S2. Reconstruct the serial numbers of the laser emitters in order from small to large according to the vertical angles of the laser emitters.

[0070] The present invention rearranges the serial numbers of laser emitters in the order of the vertical angle of the emitter from small to large, and reorganizes the point cloud based on the two-dimensional grid map. The method of constructing a two-dimensional point cloud grid map is different from the existing method of obtaining a two-dimensional point cloud depth map based on spherical projection. The spherical projection method calculates the pitch angle and azimuth angle of the point cloud based on the three-dimensional measurement value of the point cloud, and then obtains a depth map. Since the point cloud will be refracted when it hits the water mist and diffusely reflected when it hits the rough plane, the pitch angle and azimuth angle of the point cloud calculated based on the three-dimensional measurement value of the point cloud may be different from the vertical angle and the rotation angle of the corresponding laser emitter. The serial numbers of laser emitters are usually not arranged in the order of the vertical angle of the emitter from small to large. In order to facilitate the extraction of the difference stability feature, the serial numbers of laser emitters are rearranged in the order of the vertical angle of the emitter from small to large in this embodiment.

[0071] This embodiment takes a laser radar model HDL-32E as an example, and as shown in Table 1, the serial number of the laser transmitter is reconstructed.

[0072] Table 1

[0073]

[0074] S3. Construct a two-dimensional point cloud grid map based on one frame of point cloud.

[0075] The specific process of constructing a two-dimensional point cloud grid map is as follows:

[0076] S301, using the reconstructed laser transmitter serial number as the ordinate and the serial number of the laser beam emitted by the same laser transmitter as the abscissa.

[0077] S302 , generating multiple layers of grids in sequence using the three-dimensional coordinates x, y, z and intensity of the point cloud.

[0078] S303, obtaining a 4-layer two-dimensional point cloud grid map.

[0079] This embodiment takes the laser radar model HDL-32E as an example. Figure 2 As shown in the figure, a 4-layer 2D point cloud grid map is constructed using a frame of point cloud from HDL-32E. The coordinate origin of the grid map is in the lower left corner. The ordinate is the serial number of the rearranged laser emitter, and the abscissa is the serial number of the laser beam emitted by the same laser emitter during a 360° rotation. The 2D coordinates of the grid are the new index of the point cloud, and the values ​​of each layer of the grid are x, y, z and intensity. x, y, z are the 3D coordinates of the point cloud, and intensity is the intensity of the point cloud.

[0080] S4. Extract multiple layers of difference stability feature maps from the two-dimensional point cloud grid map, and generate a difference stability fusion feature map after fusion.

[0081] For mechanical rotating laser radar, whether it is the point cloud noise generated by hitting the water mist used for dust reduction in underground mines, rain, snow, fog, dust and other bad weather, or the point cloud noise generated by hitting the water mist used for dust reduction in underground mines, there is a very obvious characteristic that it is disorderly, while the point cloud hitting other regular objects is relatively orderly. The application scenario of the present invention is underground mines. In addition to water mist and dust, other objects in the mine (including vehicles, pedestrians, shaft walls and air doors, etc.) are regular. Therefore, if it is possible to find features that describe the degree of disorder of the point cloud, then it must be possible to effectively remove the point cloud noise generated by hitting the water mist in the mine.

[0082] In this embodiment, three layers of difference stability feature maps are extracted, the first layer is the tangent angle difference stability feature map, the second layer is the adjacent line distance difference stability feature map, and the third layer is the intensity difference stability feature map. Based on the idea proposed in the present invention, more layers of difference stability feature maps can be extracted later.

[0083] Each laser emitter of the mechanical rotating laser radar will emit a circle of laser beams when it rotates 360°. The shape of the point cloud hitting the flat ground is similar to a circular arc, the point cloud hitting regular targets such as vehicles, pedestrians, well walls and dampers is similar to a straight line, and the point cloud noise hitting the water mist is chaotic. A straight line can be regarded as a special case of an arc. Because the change in the size of the tangent angle of adjacent points on the arc is relatively uniform, the present invention describes the degree of disorder of the point cloud through the tangent angle difference stability characteristic diagram.

[0084] The extraction process of the tangent angle difference stability characteristic map is as follows: Figure 3 As shown, the specific process is as follows:

[0085] S401a, f x and fy Use 1×3 convolution kernel [1 0 -1] to perform convolution operation and obtain feature map and

[0086] in,

[0087] f x is a two-dimensional point cloud grid map with a grid value of x;

[0088] f y is a two-dimensional point cloud grid map with grid value y;

[0089] x is the x-axis coordinate of the point cloud;

[0090] y is the y-axis coordinate of the point cloud;

[0091] S401b、 Each grid value in is divided by The grid value corresponding to the index position in the grid, and then the inverse tangent is calculated to obtain the tangent angle feature map

[0092] S401c, yes Use 1×3 convolution kernel [1 -1 0] to perform convolution operation to obtain the tangent angle difference feature map

[0093] S401d, yes The standard deviation is calculated using a 1×5 standard deviation kernel to obtain the tangent angle difference stability characteristic graph f 0 .

[0094] When two adjacent laser emitters of a mechanical rotating laser radar rotate 360°, they will emit two circles of laser beams. No matter whether the point clouds in the two circles of laser beams are hitting the same target or different targets, as long as the targets are regular, according to the actual distribution of the point clouds, the distance changes between the two points in the two circles of point clouds that hit the regular targets and have the same horizontal coordinates in the two-dimensional point cloud grid map are relatively uniform. Therefore, the present invention describes the degree of disorder of the point cloud through the stability characteristic diagram of the adjacent line distance difference.

[0095] The extraction process of the adjacent line distance difference stability feature map is as follows: Figure 4 As shown, the specific process is as follows:

[0096] S402a, f x 、f y and f z Use 3×1 convolution kernel [1 -1 0] respectively T The convolution operation can obtain the feature map f 1 0 、f 11 and f 1 2 ;

[0097] in,

[0098] f x is a two-dimensional point cloud grid map with a grid value of x;

[0099] f y is a two-dimensional point cloud grid map with grid value y;

[0100] f z is a two-dimensional point cloud grid map with a grid value of z;

[0101] x is the x-axis coordinate of the point cloud;

[0102] y is the y-axis coordinate of the point cloud;

[0103] z is the z-axis coordinate of the point cloud;

[0104] S402b, f 1 0 、f 1 1 and f 1 2 The grid values ​​with the same index position are first squared, then added, and finally squared to obtain the adjacent line distance difference feature map f 1 3 ;

[0105] S403c, f 1 3 Use 1×3 convolution kernel [1 -1 0] to perform convolution operation and obtain the adjacent line distance difference feature map f 1 4 ;

[0106] S403d, f 1 4 The standard deviation is calculated using the 1×5 standard deviation kernel to obtain the stability characteristic graph f of the adjacent line distance difference. 1 .

[0107] In most cases, the material, color, distance from the lidar, etc. of the same target change relatively evenly, and the intensity of the point cloud is mainly related to these factors, so the intensity change of the point cloud hit by the same laser transmitter on the same target is also relatively even. However, there is refraction when the laser beam hits the water mist. At the same time, the distance between the point cloud hit by the same laser transmitter on the water mist and the lidar also changes unevenly, so the intensity change of the point cloud hit by the same laser transmitter on the water mist is also uneven.

[0108] The specific process of extracting the intensity difference stability characteristic map is as follows: Figure 5As shown, the specific process is as follows:

[0109] S403a, f intensity Use 1×3 convolution kernel [1 -1 0] to perform convolution operation to obtain the intensity difference feature map

[0110] in,

[0111] f intensity A two-dimensional point cloud grid map with a grid value of intensity;

[0112] intensity is the intensity value of the point cloud;

[0113] S403b, yes The standard deviation is calculated using a 1×5 standard deviation kernel to obtain the intensity difference stability characteristic map f 2 .

[0114] Furthermore, the following formula is used to fuse multiple layers of difference stability feature maps to generate a difference stability fusion feature map:

[0115]

[0116] in,

[0117] f i is the difference stability feature map of the i-th layer;

[0118] w i is the weight of the difference stability feature map of the i-th layer;

[0119] The weight is the experience value;

[0120] n is the number of layers of the difference stability feature map;

[0121] F is the difference stability fusion feature map.

[0122] S5. Remove point cloud pixels greater than a predetermined difference stability empirical threshold from the difference stability fusion feature map to achieve denoising.

[0123] Obviously, the above embodiments are merely examples for the purpose of clear explanation, and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the invention.

Claims

1. A mine laser radar point cloud denoising method, characterized in that: The steps include: S1. Each laser transmitter in the mechanical rotating laser radar transmits a laser beam into the mine to generate a frame of point cloud; the frame of point cloud is a set of point clouds generated by the lasers emitted by each laser transmitter of the mechanical rotating radar rotating 360°; S2, reconstructing the serial numbers of the laser emitters in order according to the vertical angles of the laser emitters; S3, constructing a two-dimensional point cloud grid map based on a frame of point cloud; the specific process of constructing a two-dimensional point cloud grid map is: S301, using the reconstructed laser transmitter serial number as the ordinate and the serial number of the laser beam emitted by the same laser transmitter as the abscissa; S302, sequentially generate multi-layer grids using the three-dimensional coordinates x, y, z and intensity of the point cloud; S303, obtaining a 4-layer 2D point cloud grid map; S4, extracting multiple layers of difference stability feature maps from the two-dimensional point cloud grid map, and fusing them to generate a difference stability fusion feature map; S5. Remove point cloud pixels greater than a predetermined difference stability empirical threshold from the difference stability fusion feature map to achieve denoising.

2. The mine laser radar point cloud denoising method according to claim 1, characterized in that: In step S4, the multi-layer difference stability characteristic map includes a tangent angle difference stability characteristic map, an adjacent line distance difference stability characteristic map and an intensity difference stability characteristic map.

3. The mine laser radar point cloud denoising method according to claim 2, characterized in that: The extraction process of the tangent angle difference stability characteristic map is as follows: S401a, f x and f y Use 1×3 convolution kernel [1 0 -1] to perform convolution operation and obtain feature map and in, f x is a two-dimensional point cloud grid map with a grid value of x; f y is a two-dimensional point cloud grid map with grid value y; x is the x-axis coordinate of the point cloud; y is the y-axis coordinate of the point cloud; S401b、 Each grid value in is divided by The grid value corresponding to the index position in the grid, and then the inverse tangent is calculated to obtain the tangent angle feature map S401c, yes Use 1×3 convolution kernel [1-10] to perform convolution operation to obtain the tangent angle difference feature map S401d, yes Using a 1×5 standard deviation kernel for standard deviation calculation can obtain the tangent angle difference stability characteristic map f0.

4. The mine laser radar point cloud denoising method according to claim 2, characterized in that: The extraction process of the adjacent line distance difference stability characteristic map is as follows: S402a, f x 、f y and f z Use 3×1 convolution kernel [1 -1 0] respectively T Convolution operation can obtain feature maps and in, f x is a two-dimensional point cloud grid map with a grid value of x; f y is a two-dimensional point cloud grid map with grid value y; f z is a two-dimensional point cloud grid map with a grid value of z; x is the x-axis coordinate of the point cloud; y is the y-axis coordinate of the point cloud; z is the z-axis coordinate of the point cloud; S402b, yes and The grid values ​​with the same index position are first squared, then added, and finally squared to obtain the adjacent line distance difference feature map. S403c, yes Use 1×3 convolution kernel [1 -1 0] to perform convolution operation to obtain the adjacent line distance difference feature map S403d, yes The standard deviation is calculated using a 1×5 standard deviation kernel to obtain the adjacent line distance difference stability feature map f1.

5. The mine laser radar point cloud denoising method according to claim 2, characterized in that: The extraction process of the intensity difference stability characteristic map is specifically as follows: S403a, f intensity Use 1×3 convolution kernel [1 -1 0] to perform convolution operation to obtain the intensity difference feature map in, f intensity A two-dimensional point cloud grid map with a grid value of intensity; intensity is the intensity value of the point cloud; S403b, yes The standard deviation is calculated using a 1×5 standard deviation kernel to obtain the intensity difference stability feature map f2.

6. The mine laser radar point cloud denoising method according to claim 1, characterized in that: The difference stability fusion feature map is generated by fusing multiple layers of difference stability feature maps through the following formula: in, f i is the difference stability feature map of the i-th layer; w i is the weight of the difference stability feature map of the i-th layer; The weight is the experience value; n is the number of layers of the difference stability feature map; F is the difference stability fusion feature map.

7. The mine laser radar point cloud denoising method according to claim 1, characterized in that: In step S2, when reconstructing the serial numbers of the laser emitters, the laser emitters are sorted from small to large according to their vertical angles.

8. The mine laser radar point cloud denoising method according to claim 1, characterized in that: In step S1, when the laser transmitter emits a laser beam into the mine, the laser beam is emitted in a fixed rotation direction and a fixed interval angle.

9. The mine laser radar point cloud denoising method according to claim 8, characterized in that: Each of the laser emitters emits 1800 laser beams per rotation of 360°.

Images