A road obstacle detection method for automatic driving of a mining truck

Abstract

The present application provides a kind of road surface obstacle detection for the automatic driving of mine truck, the point cloud scanned by laser radar is rasterized, and the features of each grid are calculated, including calculating the average inclination of point cloud in longitudinal and transverse directions, clustering based on the features of each grid, clustering the grids belonging to the same class of features into clusters, calculating the features of each point cloud cluster, inputting into classification model, obtaining the category of each point cloud cluster, assigning labels to each grid, extracting point cloud index, calculating obstacle position information, and providing basis for vehicle obstacle avoidance detour.The present application provides a road surface obstacle detection method for unstructured road in mining area, which has good robustness, does not require special hardware equipment, can effectively maintain and improve the safety and economy of intelligent mine truck during transportation operation, and has certain reference significance and engineering value.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of mine automatic driving, and particularly relates to a road obstacle detection method for mine truck automatic driving. BACKGROUND

[0002] In mine operation, due to the large load of mine trucks and high operation intensity, the repeated rolling of mine trucks and the influence of rain washing on mine area road surface, road surface diseases such as potholes, gullies and bumps are caused. In addition, due to the frequent full load or even overload driving of mine trucks, and the rear shift of the center of gravity of the vehicle body when climbing uphill, combined with the factors of bumpy and uneven road and large centrifugal force at turning, the mine truck often drops materials such as gravel during the process of transporting materials uphill under heavy load, which affects the passing of subsequent vehicles. The dropped gravel increases the risk of tire burst of the mine truck, not only affecting the entire road transportation operation, but also increasing the cost of replacing the tires of the mine truck, and the high strength of the tire pressure of the mine truck also has certain safety hazards.

[0003] In the traditional operation process of mine, road surface diseases such as potholes, gullies and bumps and road gravel need to rely on the driving experience and feeling of the driver to complete the avoidance, which puts high requirements on the driving level of the driver, and also reduces the driving experience and work efficiency of the driver. On the intelligent mine truck of automatic driving, it is necessary to rely on sensors and algorithms to solve this problem. Since the mine area road is a dirt road, the color contrast is not large, and the mine area needs to operate at night, therefore, the camera is not the best solution to solve this problem. Due to the inherent advantages of laser radar such as stability, accuracy and small influence of light, it becomes the first choice to solve this problem.

[0004] In the existing invention, flatness detection, flat ground point cloud ideal model, curvature calculation and point cloud height statistics are often used to distinguish ground point cloud and non-ground point cloud. This method is more suitable for structured roads, and is not suitable for mine area scenes with poor road flatness and large height fluctuations. In addition, the existing invention often uses deep learning technology, but this method needs to rely on a large amount of manual data labeling and model training, which has high cost and poor interpretability. SUMMARY

[0005] The present application provides a road obstacle detection method for mine truck automatic driving, which adopts a traditional non-deep learning method and can detect multiple road obstacles at the same time, thereby maintaining and improving the safety and economy of intelligent mine trucks during the transportation operation process.

[0006] The road obstacle detection method for mine truck automatic driving comprises the following steps:

[0007] Step 1: grid the point cloud scanned by the laser radar and project it into the grid, and calculate the features of the grid;

[0008] Step two: traverse the lateral and longitudinal inclination of all point clouds in the calculation grid, and calculate the average inclination of the entire grid in the lateral and longitudinal directions, which is taken as the grid feature;

[0009] Step three: clustering based on grid features, clustering grids belonging to the same class of features into clusters; calculate the features of each point cloud cluster and input them into the classification model for classification and recognition to obtain the class label, extract the obstacle point cloud and calculate the obstacle information.

[0010] Further need to explain is that in step one, the point cloud gridding processing is carried out by using the fan cutting method, and each grid is represented as follows:

[0011]

[0012] The grid C of the ith row and jth column ij , the minimum value z of the point cloud in the grid in the z axis direction min , the maximum value z of the point cloud in the grid in the z axis direction max , the depth Δz of the grid = z max -z min , the longitudinal average inclination λ of the grid point cloud average , the lateral average inclination θ average ;

[0013] The inclination feature is calculated and filled in step two, the number of grid point clouds, and the grid class label to be determined. The label has five categories: -2 represents a ditch, -1 represents a pit, 0 represents the ground, 1 represents road gravel, and 2 represents road bump.

[0014] Further need to explain is that in step two, let P g be the ground point cloud in the longitudinal direction at the current x value, that is, the point cloud hits the ground plane, and its coordinate value is (X g , Y g , Z g );

[0015] Let P i be the ith point cloud calculated point by point along the positive direction of the x axis at the current x value, and its coordinate value is (X i , Y i , Z i ). The inclination of the current point cloud P i to the ground point cloud P g is calculated As shown in formula (2):

[0016]

[0017]

[0018]

[0019]

[0020] where sign() is a sign function, and λ and λ are threshold constants related to the longitudinal feature of the obstacle, and λ <0, λ >0.

[0021] It is further explained that when the point cloud P i satisfies equation (4), it is considered to belong to the ground point cloud;

[0022] When equation (4) is not satisfied, it is considered that the current point cloud is an obstacle point cloud.

[0023] It is further explained that if P i is a ground point cloud, the ground point cloud is updated, that is, the current ground point cloud (P g =P i ) is used for the calculation of the inclination of the next point P i+1 ;

[0024] If P i belongs to the obstacle point cloud, the ground point cloud is not updated, and P g is still used and the calculation of the inclination of the point cloud with P i+1 , P i+2 , … is continued;

[0025] All point clouds are calculated according to equation (2), and after the calculation of the inclination of each point cloud is completed, it is judged whether the ground point cloud P g needs to be updated;

[0026] After the longitudinal inclination of all point clouds in the grid is calculated, the average longitudinal inclination λ average of the current grid is calculated according to equation (5).

[0027] It is further explained that the average transverse inclination θ average in the current grid is calculated by equations (6) to (8);

[0028]

[0029]

[0030]

[0031] is the ground point cloud in the transverse direction at the current y value, and the coordinate value is P j For the jth point cloud calculated point by point along the positive direction of the y-axis at the current y value, the coordinate value is (X j ,Y j ,Z j ), the inclination of the current point cloud P j to the ground point cloud is calculated.

[0032] Formula (7) is the standard for judging whether the current point cloud belongs to the obstacle point cloud, and alpha θ and beta θ are threshold constants related to the lateral characteristics of the obstacle, and alpha θ <0, beta θ >0;

[0033] According to formula (6), all point clouds are calculated, and after the inclination of each point cloud is calculated, it is judged whether the ground point cloud needs to be updated

[0034] When the lateral inclination of all point clouds in the grid is calculated, the average lateral inclination theta average of the current grid is calculated according to formula (8).

[0035] Further need to be explained is that in step three, based on the characteristics of each grid (z min ,z max ,Delta z, lambda average ,theta average ,number), clustering is carried out, and grids belonging to the same class of characteristics are clustered into clusters.

[0036] The z-axis minimum value, z-axis maximum value, z-axis average value and median value, point cloud quantity, and geometric size in the xyz direction of each point cloud cluster are calculated and input into a classification model or neural network for training.

[0037] Further need to be explained is that in the method, by training the classification model or neural network, the characteristics of each cluster after clustering are input, the categories of each cluster are output, each grid is assigned a label label, and the point cloud index is extracted, the obstacle position information is calculated, and the basis for vehicle obstacle avoidance is provided.

[0038] From the above technical solutions, it can be seen that the present application has the following advantages:

[0039] The road surface obstacle detection method for automatic driving of a mine truck provided by the present application includes the following steps: performing rasterization on point clouds scanned by a laser radar, calculating the features of each grid, clustering grids belonging to the same type of features based on the features of each grid, calculating the features of each point cloud cluster, inputting the features of each point cloud cluster into a classification model to obtain the types of the point cloud clusters, assigning labels to each grid, extracting point cloud indexes, and calculating obstacle position information. Thus, the road surface obstacle detection method not only provides a basis for vehicle obstacle avoidance and detouring, but also effectively maintains and improves the safety and economy of intelligent mine trucks during transportation operations. Moreover, the fan-shaped cutting rasterization method used in the present application can effectively solve the problem that the number of laser point clouds gradually decreases with the increase of the scanning distance.

[0040] The present application calculates the average inclination of point clouds in the longitudinal and transverse directions as an important feature of the grid, which can effectively utilize the sparse point cloud beam and limited data and information of the laser radar, accurately describe the characteristics of the obstacle, and quickly and effectively cluster the obstacle with the advantages of simplicity and robustness. BRIEF DESCRIPTION OF DRAWINGS

[0041] In order to more clearly illustrate the technical solutions of the present application, the drawings needed in the description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained according to these drawings without creative labor for those skilled in the art.

[0042] Fig. 1 The road surface obstacle detection method for automatic driving of a mine truck is shown in the schematic diagram.

[0043] Fig. 2 The rasterization method is shown in the schematic diagram.

[0044] Fig. 3 The inclination feature of the road surface obstacle is shown in the schematic diagram.

[0045] Fig. 4 The inclination calculation algorithm flowchart is shown in the schematic diagram.

[0046] Fig. 5 The geometric feature of the road surface obstacle is shown in the schematic diagram. DETAILED DESCRIPTION

[0047] The road surface obstacle detection method for automatic driving of a mine truck includes both hardware-level technology and software-level technology. The basic technology generally includes technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics, etc. The software technology of the road surface obstacle detection method mainly includes computer vision technology, etc.

[0048] The technical solutions of the present application will be described below in combination with the drawings. Figs. 1 to 5The road obstacle detection method for automatic driving of a mining truck is applied to detection and analysis of road obstacles in the automatic driving process of a mining area, adopts a traditional non-deep learning method, is simple and robust, can detect multiple road obstacles at the same time, and has a positive effect on improving obstacle detection of automatic driving.

[0049] The technical solutions in the embodiments of the present application will be clearly and completely described with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0050] Please refer to Figs. 1 to 5 The road obstacle detection method for automatic driving of a mining truck is applied to detection and analysis of road obstacles in the automatic driving process of a mining area, adopts a traditional non-deep learning method, is simple and robust, can detect multiple road obstacles at the same time, and has a positive effect on improving obstacle detection of automatic driving.

[0051] In step one, the core idea of laser point cloud gridding is to process the area scanned by the laser radar with a grid, and each grid point cloud represents a small area in space and contains a part of point cloud. The point cloud gridding processing is divided into two-dimensional gridding and three-dimensional gridding. Two-dimensional is actually a projection of three-dimensional point cloud. For example, do not consider the change of z value, and project the point cloud into the xy plane.

[0052] Project the three-dimensional point cloud into a multi-scale grid, and calculate the features of each grid. Each grid can be represented as:

[0053]

[0054] Where, C ij is the grid in the ith row and jth column, z min represents the minimum value of the point cloud in the grid in the z axis direction, z max represents the maximum value of the point cloud in the grid in the z axis direction, and Δz is the depth of the grid Δz=z max -z min , λ average represents the longitudinal average inclination of the grid point cloud, and θ average represents the transverse average inclination. This feature needs to be calculated and filled in step two. Number represents the number of point clouds in the current grid, label is the category to be determined, and there are five categories, -2 represents a ditch, -1 represents a pit, 0 represents the ground, 1 represents road gravel, and 2 represents road bump.

[0055] Rasterization is beneficial for calculating the distribution characteristics and statistical properties of local point clouds. However, due to the characteristics of lidar scanning, the number of lidar point clouds decreases with increasing distance, meaning that point clouds are sparse at greater distances. Therefore, if the same grid size is used, the number of point clouds contained in the grid at greater distances will be significantly unbalanced, and the grid size needs to be increased accordingly to more fairly reflect its true distribution and geometric characteristics.

[0056] Therefore, the present invention adopts the appendix as follows. Fig. 2 The grid map method shown divides the scanning area radially with the center of the lidar transmitter as the origin, with each grid having the same radial spacing. This method can better meet the requirement that the grid size increases accordingly with the scanning distance, which is beneficial for solving the problem of sparse point clouds at greater distances.

[0057] In step two, as shown in the attached document Fig. 3 As shown, assume that the point cloud P hit by the lidar g For ground point clouds, when encountering a raised obstacle in front, the point cloud hitting the raised obstacle is relative to the ground point cloud P. g The inclination gradually increases and then rapidly decreases.

[0058] When a negative obstacle is encountered, the tilt of the point cloud on the negative obstacle relative to the ground point cloud decreases rapidly and then gradually increases. Therefore, calculating the tilt can serve as an important basis for detecting and identifying road obstacles.

[0059] The calculation of tilt requires calculating the tilt in both the longitudinal and lateral directions simultaneously. This is primarily because of the characteristics of LiDAR point cloud scanning, where the horizontal resolution is generally higher than the vertical resolution. Therefore, the calculation of lateral tilt is more accurate.

[0060] Secondly, on unstructured roads such as those in mining areas, the weight of the mining trucks, their enormous loads, and repeated compaction are important causes of road surface defects such as potholes, gullies, and road bumps. The compaction of the mining trucks is generally along the longitudinal direction.

[0061] Therefore, these diseases differ in both longitudinal and transverse characteristics. In summary, it is necessary to calculate the point cloud tilt in both the longitudinal and transverse directions simultaneously.

[0062] Taking the vehicle coordinate system as an example, the forward direction of the vehicle is the positive x-axis, the left side is the positive y-axis, and the vertical upward direction is the positive z-axis. The tilt is calculated point by point along both the positive x-axis and y-axis. (See attached diagram) Fig. 4 As shown, taking the x-axis as an example, let's assume the point cloud P... g This refers to the ground point cloud at the current x-value in the vertical direction, i.e., the point cloud projected onto the ground plane, with coordinates (X...). g ,Yg ,Z g ). Let P i be the i-th point cloud calculated point by point along the positive direction of the x-axis at the current x value, with coordinate values (X i ,Y i ,Z i ), and the inclination of the current point cloud P i to the ground point cloud P g is calculated. As shown in equation (2):

[0063]

[0064]

[0065]

[0066]

[0067] wherein sign() is a sign function, as shown in equation (3), which returns 1 when the value in the parentheses is greater than 0, 0 when it is equal to 0, and -1 when it is less than 0.

[0068] α λ and β λ are threshold constants, and α λ <0, β λ >0. When the point cloud P i satisfies equation (4), it is considered to belong to the ground point cloud; when equation (4) is not satisfied, it is considered to be an obstacle point cloud: when the left inequality is not satisfied, it is considered to belong to the negative obstacle point cloud; when the right inequality is not satisfied, it is considered to belong to the convex obstacle point cloud.

[0069] Through the above equation, if P i is not an obstacle point cloud, it is considered to belong to the ground point cloud, and the ground point cloud needs to be updated, i.e., replacing the current ground point cloud (P g =P i ) for the calculation of the inclination of the next point P i+1 . If P i belongs to the obstacle point cloud, the ground point cloud is not updated, and P g is still used and the calculation of the inclination of the point cloud with P i+1 , P i+2 , … is continued. The ground point cloud is updated, and the purpose is to find the ground point cloud closest to the obstacle in order to accurately calculate the inclination of the obstacle point cloud.

[0070] All point clouds are calculated according to equation (2), and after the calculation of the inclination of each point cloud is completed, it is judged whether the ground point cloud P gWhen the longitudinal inclination of all point clouds in the current grid is calculated, the average longitudinal inclination λ of the current grid is calculated according to formula (5) average This feature.

[0071] Similarly, the average transverse inclination θ of the current grid is calculated according to formula (8) average As shown in formula (6) to formula (8). is the ground point cloud in the transverse direction at the current y value, and the coordinate value is P j is the jth point cloud calculated point by point along the positive direction of the y axis at the current y value, and the coordinate value is (X j ,Y j ,Z j ), the inclination of the current point cloud P j to the ground point cloud is calculated according to formula (6)

[0072] Similarly, formula (7) is a standard for judging whether the current point cloud belongs to the obstacle point cloud, and α θ and β θ are threshold constants related to the transverse characteristics of the obstacle, and α θ <0, β θ >0.

[0073] Similarly, all point clouds are calculated according to formula (6), and after the inclination of each point cloud is calculated, it is judged whether the ground point cloud

[0074] When the transverse inclination of all point clouds in the current grid is calculated, the average transverse inclination θ of the current grid is calculated according to formula (8) average This feature.

[0075]

[0076]

[0077]

[0078] In step three, the gridding of the point cloud and the calculation of the characteristics of each grid are completed in steps one and two. In this step, clustering is performed based on the characteristics of each grid (z min ,z max ,Δz,λ average ,θ average ,number).

[0079] The z-axis minimum value, z-axis maximum value, z-axis average value and median value, point cloud quantity, geometric size in the xyz direction, etc. of each point cloud cluster are calculated and input into a classification model or neural network for training. By training the classification model or neural network, the features of each cluster after clustering are input, and the categories of each cluster are output, so as to assign labels to each grid, extract point cloud indexes, and calculate obstacle position information, thereby providing a basis for vehicle obstacle avoidance and detouring.

[0080] In an embodiment of the present application, based on the road surface obstacle detection method for automatic driving of a mining truck, a possible embodiment will be given below to non-restrictively describe the specific implementation thereof.

[0081] As shown in the accompanying Fig. 5 As shown in the accompanying The general characteristics of the road surface gravel, bump, pothole and gully are described as follows: the road surface gravel is a convex obstacle with small size; the road surface bump is a convex long strip-shaped soil pile with large longitudinal size; as for small-size bumps, they are not easy to be distinguished from road surface gravel in data characteristics due to the sparseness of laser point cloud, but both of them need to be avoided and detoured, and thus can be classified into one category; the pothole is a negative obstacle with irregular convex polygon shape similar to a circle, and has small transverse and longitudinal size; and the gully is also a negative obstacle, but is generally long strip-shaped and has large longitudinal size. It can be seen that by using the sparse point cloud beam of the laser radar and limited data and information, the obstacles can be accurately described and effectively classified, and the obstacle avoidance processing is set based on different types to realize automatic driving.

[0082] In the road surface obstacle detection method, the computer program code for performing the operations of the present disclosure can be written in one or more programming languages or combinations thereof, including but not limited to object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" language or similar programming languages. The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to these embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for detecting road obstacles for automated driving of mining trucks, characterized in that, Includes the following steps: Step 1: Rasterize the point cloud scanned by the lidar and project it into a grid, then perform feature calculation on the grid. Point cloud rasterization is performed using a fan-shaped cutting method, and each raster is represented as follows: （1） No. Line 1 Column grid , Its features include the minimum value of the point cloud within the raster along the z-axis. The maximum value of the point cloud within the raster along the z-axis. Grid depth Longitudinal average tilt of grid point cloud Horizontal average tilt ; The tilt feature is calculated and filled in step two, which determines the number of point clouds in the raster. Raster category to be determined , There are five categories: -2 represents gullies, -1 represents potholes, 0 represents the ground, 1 represents road gravel, and 2 represents road bumps; Step 2: Traverse and calculate the horizontal and vertical tilt of all point clouds within the grid, and calculate the average horizontal and vertical tilt of the entire grid, using it as the grid feature. In step two, a point cloud is set. This is the ground point cloud at the current x-value in the vertical direction, i.e., the point cloud projected onto the ground plane, with coordinates of... ; set up The first point is calculated point by point along the positive x-axis at the current x-value. A point cloud, with coordinates value... Calculate the current point cloud Point clouds to the ground Inclination As shown in equation (2): （2） （3） （4） （5） in, For symbolic functions, and The threshold constant is related to the longitudinal features of the obstacle, and , ; When the cloud When equation (4) is satisfied, it is considered to belong to ground point cloud; When equation (4) is not satisfied, the current point cloud is considered to be an obstacle point cloud; if If it is a ground point cloud, then update the ground point cloud, that is, update the current ground point cloud. ) for the next point Calculation of inclination; if If the point cloud contains obstacle points, the ground point cloud will not be updated and will continue to be used. And continue to complete with , Calculation of the tilt of point clouds of ... Calculate all point clouds according to equation (2), and determine whether the ground point cloud needs to be updated after the tilt of each point cloud is calculated. ; After traversing and calculating the longitudinal tilt of all point clouds within the raster, the average longitudinal tilt of the current raster is calculated according to equation (5). ; The average lateral tilt within the current grid is calculated using equations (6) to (8). ; （6） （7） （8） It is the current The ground point cloud in the horizontal direction at the value, with coordinates of , For the present value along The first point is calculated point by point along the positive axis. A point cloud, with coordinates value... Calculate the current point cloud Point clouds to the ground Inclination ; Equation (7) is the criterion for determining whether the current point cloud belongs to the obstacle point cloud. and The threshold constant is related to the lateral features of the obstacle, and , ; Calculate all point clouds according to equation (6), and determine whether the ground point cloud needs to be updated after the tilt of each point cloud is calculated. ; After traversing and calculating the lateral tilt of all point clouds within the raster, the average lateral tilt of the current raster is calculated according to equation (8). ; Step 3: Cluster based on raster features, grouping rasteres belonging to the same feature class into clusters; calculate the features of each point cloud cluster, feed them into the classification model for classification and recognition, obtain category labels, extract obstacle point clouds, and calculate obstacle information.

2. The road obstacle detection method for automated driving of mining trucks according to claim 1, characterized in that, In step three, based on the features of each grid cell ( Clustering is performed to group grids that belong to the same type of feature into clusters; Calculate the minimum, maximum, average, median, number of points, and geometric dimensions in the x, y, and z directions for each point cloud cluster, and then feed them into a classification model or neural network for training.

3. The road obstacle detection method for automated driving of mining trucks according to claim 2, characterized in that, In this method, a classification model or neural network is trained, the features of each cluster are input as input, and the category of each cluster is output, assigning a label to each grid cell. It also extracts point cloud indexes and calculates obstacle location information to provide a basis for vehicle obstacle avoidance and detour.

Images