Method for obtaining mining area information based on laser radar and millimeter wave radar
By combining point cloud data processing technology of lidar and millimeter wave radar, dust interference is eliminated, and the accuracy of obstacle identification in mining areas is solved to ensure the normal operation of unmanned mine vehicles.
Patent Information
- Application Number
- CN202011307165.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-11-20
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2040-11-20
AI Technical Summary
In the mining area, dust interference causes the lidar to be unable to accurately identify obstacles, resulting in unmanned mine vehicles not being able to walk normally.
By obtaining point cloud data of lidar and millimeter wave radar, using data projection, filtering, clustering and comparison technologies, dust interference is eliminated and obstacles in the mining area are accurately identified.
Effectively removes dust interference, improves the accuracy of obstacle identification, and ensures the normal travel of unmanned mine vehicles.
Smart Images

Figure CN112666557B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of methods for obtaining information about obstacles in mining areas, and in particular to a method for obtaining information about mining areas based on laser radar and millimeter wave radar. Background Art
[0002] There is no green vegetation on the road in the mining area, resulting in large areas of dust surrounding the mining vehicles when the ore transport vehicles are driving on the road. The perception of unmanned mining vehicles mainly relies on multiple 32-line laser radars to perceive obstacles, but the dense dust raised around the mining vehicles will appear in the laser radar point cloud data. A large number of clustered and rapidly changing point clouds will be mistakenly identified as obstacles, causing the unmanned mining vehicles that rely on 32-line laser radars as perception sensors to be unable to move normally. Summary of the invention
[0003] In view of the above-mentioned prior art, the purpose of the present invention is to overcome the problem in the prior art that dust is extremely unfavorable to the perception of the environment in the mining area, causing obstacles to obstacle recognition and causing unmanned mining vehicles to be unable to move normally, thereby providing a method for obtaining mining area information based on laser radar and millimeter wave radar that can automatically remove dust, effectively and accurately identify obstacles in the mining area, and guide the effective movement of unmanned mining areas.
[0004] In order to achieve the above object, the present invention provides a method for obtaining mining area information based on laser radar and millimeter wave radar, the method comprising:
[0005] 1) Obtain point cloud data information of laser radar in the mining area;
[0006] 2) Obtain point cloud data information of millimeter wave radar in the mining area;
[0007] 3) The obtained point cloud data information of the millimeter wave radar is projected onto the point cloud data information of the laser radar, and the non-overlapping parts are eliminated to obtain the obstacle information in the mining area after removing the dust.
[0008] Preferably, the point cloud data information is calibrated using a mine car traveling in a mining area as an external parameter, and the point cloud data information obtained by the laser radar and the millimeter wave radar are both calibrated in a vehicle coordinate system.
[0009] Preferably, step 1) or step 2) includes:
[0010] 11) Obtain corresponding point cloud data information through laser radar or millimeter wave radar;
[0011] 12) Calibrate the point cloud data information obtained in step 11) in the vehicle coordinate system; wherein,
[0012] The point cloud data information at least includes three-dimensional coordinate information and reflection intensity information.
[0013] Preferably, step 3) specifically includes:
[0014] 31) Removal of point cloud information: The point cloud data information of the laser radar is obtained by using the elevation difference to remove the points located on the ground to obtain the processed point cloud information;
[0015] 32) Filtering of point cloud information: voxel filtering is applied to the processed point cloud information to obtain point cloud information with isolated noise points removed;
[0016] 33) Clustering of point cloud information: clustering the point cloud information after removing isolated noise points to form multiple point cloud clusters;
[0017] 34) Point cloud cluster processing: Project the obtained point cloud cluster into a grid in the XY plane;
[0018] 35) Fusion of millimeter-wave radar point cloud data information: Project the millimeter-wave radar point cloud data information onto the grid of the XY plane obtained in step 34), compare each grid, obtain obstacle information, and obtain corresponding mining area information.
[0019] Preferably, step 31) comprises:
[0020] 311) evenly dividing the XY plane into a plurality of grids, projecting the vehicle coordinate system calibrated with the point cloud data information of the laser radar onto the XY plane, wherein the point cloud data information is correspondingly distributed in one of the grids;
[0021] 312) taking the maximum difference in the heights of all transmission points in each grid in the Z direction as the height of the corresponding grid;
[0022] 313) Set a height threshold in each grid, and include corresponding point clouds in each grid whose height is greater than the height threshold, to obtain processed point cloud information.
[0023] Preferably, step 32) comprises:
[0024] 321) reconstructing each of the grids into a three-dimensional voxel grid, and using the processed point cloud information in each voxel grid as the voxels in each voxel grid;
[0025] 322) in each of the voxel grids, approximately replacing each voxel with the centroid of each voxel in the corresponding voxel grid, and reconstructing a filtered point cloud;
[0026] 323) The value of the preset radius is R, and a sphere is constructed with R as the radius for each filtered point cloud. If the number of point clouds contained in the sphere is less than the preset value, the point cloud formed as the center of the circle is eliminated, thereby completing the elimination of the point cloud of isolated noise points.
[0027] Preferably, step 33) includes: applying Euclidean clustering to the point cloud information obtained in step 32), if the distance between two point clouds is less than a set value, the two point clouds are formed into one category, and multiple point clouds are each formed into a multi-category point cloud cluster.
[0028] Preferably, the comparison method in step 35) specifically includes:
[0029] 351) If the grid contains point cloud data information of the millimeter wave radar, all point cloud data information in the point cloud cluster formed by the laser radar in the grid is retained;
[0030] 352) If there is no point cloud data information projected by the millimeter-wave radar in the grid, all point cloud clusters in the grid are removed.
[0031] Through the above technical scheme, the present invention, on the basis of obtaining the point cloud data information of the laser radar, further projects the point cloud data information of the millimeter-wave radar on the point cloud data information of the laser radar, further compares the obtained projection information, and specifically eliminates the dust point information, so as to accurately obtain the obstacle information and provide effective data support for the movement of the unmanned mining car. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present invention but do not constitute a limitation of the present invention. In the accompanying drawings:
[0033] Figure 1 It is a flow chart of the method for obtaining mining area information based on laser radar and millimeter wave radar provided by the present invention. DETAILED DESCRIPTION
[0034] The specific embodiments of the present invention are described in detail below. It should be understood that the specific embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.
[0035] like Figure 1 As shown, the present invention provides a method for obtaining mining area information based on laser radar and millimeter wave radar, and the specific operation method is as follows:
[0036] 1. Sensor calibration: In order to fuse the LiDAR and millimeter-wave radar data, we first calibrated the external parameters of the two sensors and calibrated them to the vehicle coordinate system. Only in a unified coordinate system can the sensor data be fused.
[0037] 2. Point cloud data information of LiDAR: LiDAR emits multiple laser pulses until the signal reaches the object, then reflects and returns to the receiver. Based on the known laser pulse speed and reflection time, the distance between the detected object and the LiDAR receiver can be calculated. The laser transmitter of the LiDAR rotates at high speed inside and can rotate one circle within 0.1s. After one circle, the position information of objects around the vehicle relative to the LiDAR is obtained, and there is a lot of point cloud data, which can well depict the environment around the vehicle. The LiDAR then transmits the data to the computer via the network cable using the UDP protocol. The computer's LiDAR driver code then parses the transmitted data to obtain point cloud data information of the vehicle's surrounding environment, including three-dimensional coordinates and reflection intensity (x, y, z, intensity).
[0038] 3. Millimeter wave radar data acquisition: The millimeter wave radar continuously sends millimeter wave signals to the target, and then uses the sensor to receive the millimeter wave returned from the object. By detecting the flight (round trip) time of the millimeter wave, TOF (time offlight) is used to obtain the distance of the target object. Finally, sparse obstacle points can be obtained. Because the millimeter wave radar has strong penetration ability, dust has no effect on it.
[0039] 4. Point cloud data information processing of laser radar:
[0040] a) Coordinate conversion: Convert the laser radar point cloud data information coordinate system to the vehicle coordinate system.
[0041] b) Remove ground points using elevation difference: Project all 3D points to the grid in the XY plane, and the grid to which they are projected is the occupied grid. Then calculate the maximum difference of the Z values of all points in each grid, which is used as the height of this grid. Finally, according to a height threshold, retain the point cloud in the grid with a height greater than this threshold, and finally obtain the point cloud without ground points.
[0042] c) Voxel filtering: Create a voxel grid in the three-dimensional space where the point cloud is located. For each voxel in the grid, the centroid of all points in the voxel is used to approximately replace the point in the voxel. Finally, the centroid points in all voxels form the filtered point cloud. The size of the voxel can control the degree of downsampling. The larger the voxel, the sparser the filtered point cloud, and the rate of subsequent clustering processing will be greatly improved.
[0043] d) Radius filtering: If the number of points in a sphere with a radius of R where a point cloud is located is less than a threshold, it means that this point is an outlier noise point and it is removed. Finally, a point cloud with isolated noise points removed can be obtained.
[0044] e) Point cloud clustering: Using the Euclidean clustering method, if the distance between two points is less than a threshold, the two points are considered to belong to the same class. In the end, multiple point cloud clusters belonging to different classes can be obtained, and each point cloud cluster contains multiple point clouds.
[0045] f) Projecting point cloud clusters to plane grids: All point cloud clusters obtained by clustering are projected to the grid of the XY plane, and the value of the corresponding occupied grid is the number of the point cloud cluster.
[0046] g) Fusion of millimeter-wave radar point cloud data information: Convert the millimeter-wave radar point cloud data information to the vehicle coordinate system. Project the millimeter-wave radar obstacle points to the grid obtained in step f). If the grid is occupied by the lidar point cloud, it means that all the point clouds in the point cloud cluster represented by the corresponding grid number are obstacle points, not dust points, and all the lidar point clouds in this cluster are retained. Finally, the point clouds in the point cloud cluster represented by the grid without projecting millimeter-wave radar obstacle point cloud data are discarded. Finally, the purpose of removing dust interference is achieved.
[0047] The preferred embodiments of the present invention are described in detail above. However, the present invention is not limited to the specific details in the above embodiments. Within the technical concept of the present invention, a variety of simple modifications can be made to the technical solution of the present invention, and these simple modifications all belong to the protection scope of the present invention.
[0048] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present invention will not further describe various possible combinations.
[0049] In addition, various embodiments of the present invention may be arbitrarily combined, and as long as they do not violate the concept of the present invention, they should also be regarded as the contents disclosed by the present invention.
Claims
1. A method for obtaining mining area information based on laser radar and millimeter wave radar, characterized in that: The method comprises: 1) Obtain point cloud data information of laser radar in the mining area; 2) Obtain point cloud data information of millimeter wave radar in the mining area; 3) Projecting the obtained millimeter wave radar point cloud data information to the laser radar point cloud data information, removing the non-overlapping parts, and obtaining the obstacle information in the mining area after removing the dust, which specifically includes: 31) Removal of point cloud information: The point cloud data information of the laser radar is obtained by using the elevation difference to remove the points located on the ground to obtain the processed point cloud information; 32) Filtering of point cloud information: voxel filtering is applied to the processed point cloud information to obtain point cloud information with isolated noise points removed; 33) Clustering of point cloud information: clustering the point cloud information after removing isolated noise points to form multiple point cloud clusters; 34) Point cloud cluster processing: Project the obtained point cloud cluster into a grid in the XY plane; 35) Fusion of millimeter wave radar point cloud data information: Project the millimeter wave radar point cloud data information onto the grid of the XY plane obtained in step 34), compare each grid, obtain obstacle information, and obtain corresponding mining area information; Among them, the comparison method in step 35) specifically includes: 351) If the grid contains point cloud data information of the millimeter wave radar, all point cloud data information in the point cloud cluster formed by the laser radar in the grid is retained; 352) If there is no point cloud data information projected by the millimeter-wave radar in the grid, all point cloud clusters in the grid are removed.
2. The method according to claim 1, characterized in that The point cloud data information is calibrated using the mine car traveling in the mining area as an external parameter, and the point cloud data information obtained by the laser radar and the millimeter wave radar are both calibrated in the vehicle coordinate system.
3. The method according to claim 2, characterized in that In step 1) or step 2) comprising: 11) Obtain corresponding point cloud data information through laser radar or millimeter wave radar; 12) Calibrate the point cloud data information obtained in step 11) in the vehicle coordinate system; wherein the point cloud data information at least includes three-dimensional coordinate information and reflection intensity information.
4. The method according to claim 3, characterized in that Step 31) comprises: 311) evenly dividing the XY plane into a plurality of grids, projecting the vehicle coordinate system calibrated with the point cloud data information of the laser radar onto the XY plane, wherein the point cloud data information is correspondingly distributed in one of the grids; 312) taking the maximum difference in the heights of all transmission points in each grid in the Z direction as the height of the corresponding grid; 313) Set a height threshold in each grid, and include corresponding point clouds in each grid whose height is greater than the height threshold, to obtain processed point cloud information.
5. The method according to claim 4, characterized in that Step 32) includes: 321) reconstructing each of the grids into a three-dimensional voxel grid, and using the processed point cloud information in each voxel grid as the voxels in each voxel grid; 322) in each of the voxel grids, approximately replacing each voxel with the centroid of each voxel in the corresponding voxel grid, and reconstructing a filtered point cloud; 323) The value of the preset radius is R, and a sphere is constructed with R as the radius for each filtered point cloud. If the number of point clouds contained in the sphere is less than the preset value, the point cloud formed as the center of the circle is eliminated, thereby completing the elimination of the point cloud of isolated noise points.
6. The method according to claim 5, characterized in that Step 33) includes: applying Euclidean clustering to the point cloud information obtained in step 32). If the distance between two point clouds is less than a set value, the two point clouds are formed into one category, and multiple point clouds are each formed into a multi-category point cloud cluster.
Citation Information
Patent Citations
Obstacle detection method based on information fusing of laser radar and CCD camera
CN106774296A
Ultrasonic radar and laser radar information fusion method and system
CN109814112A
Barrier detection and tracking method for port unmanned vehicle
CN110531376A
Obstacle detection method and device for automatic driving scene of port
CN110764108A
Laser point cloud ground segmentation method and device, computing equipment and storage medium
CN111445472A
