Roadway obstacle detection method based on three-dimensional point cloud

Abstract

The invention relates to the technical field of image processing, in particular to a roadway obstacle detection method based on three-dimensional point cloud, which comprises the following steps: acquiring a three-dimensional point cloud image, and obtaining a point cloud set in the three-dimensional point cloud image; processing the point cloud set by using an obstacle detection model, and identifying a pseudo obstacle point cloud set; and removing the point cloud of the pseudo obstacle point cloud set from the three-dimensional point cloud image. The obstacle detection model comprises a motion feature classification layer, and comprises the steps of obtaining a point cloud set and calculating the speed of a middle point; identifying whether the point cloud set belongs to a fast moving obstacle or a suspected obstacle based on the speed of the point; calculating the movement direction of the midpoint of the suspected obstacle; and identifying whether the point cloud set belongs to a pseudo obstacle or a slow moving obstacle based on whether the moving directions of the points are consistent. According to the motion feature classification layer, the false obstacles are identified through the speed threshold value and the motion direction, the false obstacles can be effectively screened, the false obstacles are prevented from being misjudged as real obstacles, and therefore the false detection rate of detection is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a roadway obstacle detection method based on three-dimensional point cloud. Background Art

[0002] Currently, the field of artificial intelligence has witnessed vigorous development. As one of its important application scenarios, unmanned driving technology is gradually moving from the laboratory to real life. Unmanned driving technology relies on the perception and detection of the environment by the unmanned vehicle, and obstacle detection is an important part of the environmental perception of the unmanned vehicle, and is also the key to the safe and efficient driving of the unmanned vehicle. Accurately identifying and distinguishing real obstacles from pseudo-obstacles plays a decisive role in the stability and safety of the unmanned driving system.

[0003] In practical applications, unmanned vehicles face complex and changeable environmental conditions. Especially when encountering natural phenomena such as rain, snow, and sandstorms, it is easy to misidentify pseudo-obstacles such as rain, snow, and sandstorms as obstacles. For example, in heavy rain weather, the splashing of raindrops interferes with the sensors of the unmanned vehicle, causing it to misidentify a dense rain curtain as an obstacle, and then trigger measures such as braking or automatic lane change to avoid danger. This not only reduces the driving efficiency, but may also lead to traffic chaos and even traffic accidents such as rear-end collisions; In the prior art, usually, the confidence values of each point in the three-dimensional point cloud image are initialized, and the three-dimensional point cloud image is divided into an obstacle area and a non-obstacle area according to the artificially set spatial spacing of imaging points and target size thresholds. Subsequently, the confidence is further adjusted according to the size and density information of the regional point cloud to eliminate pseudo-obstacles.

[0004] However, such methods have obvious limitations. Due to their reliance on fixed rules such as artificially preset size thresholds and density standards, it is difficult to adapt to the diversity of pseudo-obstacles in complex and changeable scenarios. In environments with different concentrations of sandstorms and various forms of rain and snow, the point cloud characteristics of pseudo-obstacles vary significantly. The existing methods cannot flexibly respond, and are prone to misjudgment or missed judgment, resulting in reduced safety of unmanned driving and insufficient reliability of the intelligent detection system. Summary of the Invention

[0005] To solve the above technical problem that the unmanned driving scenario is prone to misjudgment or missed judgment of pseudo-obstacles in a complex environment, the present invention provides a roadway obstacle detection method based on three-dimensional point cloud. The method includes: collecting a three-dimensional point cloud image, and obtaining a plurality of point cloud sets in the three-dimensional point cloud image; using a constructed obstacle detection model to process any point cloud set to identify the category information of the obstacle to which the point cloud set belongs; when the point cloud set belongs to a pseudo-obstacle, removing the point cloud of the pseudo-obstacle from the three-dimensional point cloud image; Among them, the obstacle detection model includes a motion feature classification layer, which is used to obtain the category of obstacles of the points in the point cloud set, and output the speed and motion direction of the point cloud set, including: obtaining the point cloud set, and calculating the speed of the points in the point cloud set; when the speed of the points in the point cloud set is above the speed threshold, it is determined that the point cloud set belongs to a fast-moving obstacle, otherwise it is a suspected obstacle; calculating the motion direction of the points in the suspected obstacle; when the motion direction of the points in the suspected obstacle is inconsistent with the motion directions of other points in the suspected obstacle, it is determined that the point cloud set belongs to a false obstacle.

[0006] In the motion feature classification layer of the present invention, the obstacle category is initially identified through the speed threshold, and different types of obstacles such as vehicles traveling at high speed, pedestrians moving slowly, wind, sand, rain, and snow are distinguished in a complex environment. This step greatly reduces the scope of subsequent further analysis, reduces unnecessary computational volume and processing time, and improves the efficiency of the entire detection. Subsequently, by analyzing the motion direction, the differences between obstacles such as wind, sand, rain, and snow and pedestrians are captured, and false obstacles are identified, effectively avoiding misjudging false obstacles as real obstacles and reducing the false detection rate. Finally, the false obstacles are excluded from the image, making the final detection result more reliable.

[0007] As a further improvement of the method of the present invention, calculating the speed of the points in the point cloud set includes: obtaining the position information of the points in the point cloud set at the current moment; predicting the position information of the points in the point cloud set at the next moment through Kalman filtering; calculating the speed of the points in the point cloud set based on the position information at the current moment and the position information at the next moment.

[0008] The Kalman filtering algorithm can effectively fuse historical data and current observations, and has good robustness to noise. In complex environments, such as poor light in the roadway and electromagnetic interference resulting in noisy measurement data, it can accurately predict the position, speed, etc. of points, providing a reliable basis for subsequent detection.

[0009] As a further improvement of the method of the present invention, calculating the motion direction of the points in the suspected obstacle includes: obtaining a velocity vector based on the position information of the points in the suspected obstacle; calculating the motion direction vector of the points according to the velocity vector; calculating the azimuth angle and elevation angle of the points in the suspected obstacle based on the motion direction vector.

[0010] By obtaining the velocity vector from the position information, then deriving the motion direction vector based on the velocity vector, and finally calculating the azimuth angle and elevation angle, the motion direction of the points in the suspected obstacle can be described comprehensively and with high precision. In complex roadway or unmanned driving scenarios, the motion direction characteristics of real obstacles and false obstacles are slightly different. This refined calculation method can effectively capture these differences, providing a key basis for subsequent false obstacle identification.

[0011] As a further improvement of the method of the present invention, the obstacle detection model further includes loss calculation, and the loss calculation includes: separately calculating the classification loss of the obstacle , the positioning loss of the obstacle , the motion vector loss of the obstacle ; calculating the total loss of the obstacle , where and are preset weighting coefficients.

[0012] By calculating the classification loss, positioning loss, and motion vector loss, the accuracy of classification is improved, the positioning accuracy is enhanced, and the motion vector estimation is optimized. The total loss function can flexibly adjust the proportions of the classification loss, positioning loss, and motion vector loss in the total loss through the preset weighting coefficients, improving the adaptability and performance of the model in various complex scenarios.

[0013] As a further improvement of the method of the present invention, the obstacle detection model further includes an input layer, and the input layer includes: obtaining the coordinate data of any point in the point cloud set; constructing its coordinate data matrix according to the coordinate data; constructing the point cloud data set of all points in the point cloud set according to the coordinate data matrix.

[0014] As another improvement of the method of the present invention, the obstacle detection model further includes a feature extraction layer, and the feature extraction layer includes: receiving the motion features of the point cloud set output by the motion feature classification layer, where the motion features include speed and motion direction; performing three-layer convolution on the motion features to obtain the feature maps of the three-layer convolution.

[0015] As another improvement of the method of the present invention, the obstacle detection model further includes a dense classification layer, and the dense classification layer includes: receiving the feature map output by the feature extraction layer; calculating the probability that the feature map belongs to the obstacle category through an activation function, where the obstacle categories include: fast-moving obstacles, slow-moving obstacles, and pseudo-obstacles; using a multi-layer perceptron MLP to extract the non-linear relationship between the motion features and the obstacle category, and calculating the motion state probability of the points in the suspected obstacle; performing weighted fusion on the motion state probability and the probability of the obstacle category to obtain the category information of the obstacle of the points in the suspected obstacle.

[0016] The dense classification layer can quickly convert the data in the feature map into the probability distribution of each obstacle category. At the same time, the multi-layer perceptron MLP deeply excavates the non-linear relationship between the motion features and the obstacle category, and calculates the motion state probability of the points in the suspected obstacle. The two work together to deeply analyze the feature map from multiple angles, fully utilize the hidden information in the data, enhance the expression ability of the obstacle features, and can more accurately capture the subtle differences of different types of obstacles, laying a foundation for accurate classification.

[0017] As another improvement of the method of the present invention, a three-dimensional point cloud image is collected, including: obtaining a three-dimensional point cloud image; performing denoising processing on the three-dimensional point cloud image.

[0018] As another improvement of the method of the present invention, obtaining a plurality of point cloud sets in the three-dimensional point cloud image includes: performing mean shift clustering on the point clouds in the three-dimensional point cloud image to obtain a plurality of point cloud sets in the three-dimensional point cloud image.

[0019] As another improvement of the method of the present invention, when the movement direction of the point in the suspected obstacle is inconsistent with the movement directions of other points in the suspected obstacle, it means that the angle difference between the point and any other point is greater than a preset difference threshold.

[0020] Advantages of the present invention: The motion feature classification layer of the present invention first preliminarily distinguishes different types of obstacles with a speed threshold, narrowing the subsequent analysis scope and improving the detection efficiency; then analyzes the motion direction to identify pseudo-obstacles, reducing the false detection rate, and excluding pseudo-obstacles makes the detection result more reliable. The Kalman filter algorithm fuses historical and current data to accurately predict the position and speed of points in a complex environment, providing a basis for detection. By calculating the classification, positioning, and motion vector losses, the classification accuracy, positioning accuracy, and motion vector estimation are improved. The total loss function can flexibly adjust the proportion of each loss through a preset weighting coefficient, enhancing the adaptability and performance of the model in complex scenarios. Description of the Drawings

[0021] Figure 1 is a flowchart of a roadway obstacle detection method based on three-dimensional point cloud provided by an embodiment of the present invention; Figure 2 is a flowchart of building an obstacle detection model in this embodiment; Figure 3 is a flowchart of motion detection in this embodiment; Figure 4 is a schematic diagram of the three-dimensional point cloud identified by building an obstacle detection model in this embodiment; Figure 5 is a schematic diagram of a three-dimensional point cloud image obtained after being processed by the method according to the embodiment of the present invention.

[0022] In the figure, 31, building; 32, blowing sand; 33, ground. Detailed Embodiment

[0023] This embodiment provides a roadway obstacle detection method based on three-dimensional point cloud, as Figure 1 shown, the method includes steps S100 - step S300: Step S100, collect a three-dimensional point cloud image, and obtain a plurality of point cloud sets in the three-dimensional point cloud image.

[0024] Specifically, 3D point cloud images can be collected by a 3D laser scanner or a depth camera.

[0025] After obtaining the 3D point cloud image, it is necessary to perform clustering processing on the point cloud therein. After the clustering processing, several clusters can be obtained. Each cluster is a point cloud set, and these points have a certain similarity in spatial position. Each cluster can be regarded as a point cloud representation of an obstacle or a region with similar characteristics. For example, in an autonomous driving scenario, one cluster may represent a vehicle, another cluster may represent a pedestrian, and some clusters represent sandstorms, rain, or snow.

[0026] The clustering processing of the point cloud can be achieved through clustering algorithms, such as the mean shift clustering algorithm, the K-means clustering algorithm, the Gaussian mixture model clustering algorithm, and so on.

[0027] In addition, it is also necessary to perform denoising processing on the 3D point cloud image. Gaussian filtering can be used for denoising to remove the noise points in the point cloud and perform data enhancement processing.

[0028] Step S200: Use the constructed obstacle detection model to process the point cloud set and identify the category information of the obstacle to which the point cloud set belongs.

[0029] Specifically, as Figure 2 shown, the obstacle detection model refers to the model used in the present invention to detect fast obstacles and pseudo-obstacles. This model is constructed based on the existing PointNet. The model contains a total of four layers, namely the input layer, the motion feature classification layer, the feature extraction layer, and the dense classification layer. This model mainly adds a motion feature classification layer, which is used to identify the category information of the obstacle. Next, how to construct these four layers will be introduced one by one.

[0030] Step S210: Construct the input layer.

[0031] As the entrance of the model, the role of the input layer is to receive the 3D point cloud set data and perform data conversion on it to facilitate the processing of the subsequent motion feature layer. It includes: receiving the point cloud set and obtaining the coordinate data of any point in the point cloud set; constructing its coordinate data matrix according to the coordinate data; and constructing the point cloud data set of all points in the point cloud set according to the coordinate data matrix.

[0032] Specifically, usually, each point in the 3D point cloud set has coordinates. Since the obstacle is moving, the coordinates of each point at different times are different. Therefore, each point in the point cloud set will form a point cloud coordinate matrix at each moment: ; where represents the th point at Point cloud coordinate data matrix at a moment, is the th coordinate corresponding to the moment.

[0033] After obtaining the point cloud coordinate matrices of all points in the point cloud set at different moments, the point cloud coordinate matrices of all points at the same moment are constructed into a point cloud data set: ; where ， represents the point cloud data set, represents the th point cloud coordinate data matrix of the moment.

[0034] For example, the point cloud set contains two points with coordinates , , and their k th moment corresponding point cloud coordinate matrices are respectively: ; ; The corresponding point cloud data set is: .

[0035] After the input layer constructs the point cloud data, it outputs it to the motion feature classification layer.

[0036] Step S220, construct the motion feature classification layer.

[0037] Specifically, the motion feature classification layer is used to obtain the category of obstacles of points in the point cloud set, including: classifying the point cloud set as a fast-moving obstacle or a slow-moving obstacle or a pseudo-obstacle according to motion detection, and outputting its motion features such as speed and motion direction to the dense classification layer.

[0038] As Figure 3 shown, the motion detection classification includes steps S221 - S224: Step S221, obtain the point cloud set and calculate the speed of points in the point cloud set.

[0039] Specifically, the motion feature classification layer first receives the point cloud data set output by the input layer. Then, it predicts the position at the next moment, i.e., the coordinate information, through the Kalman filter, and then calculates the velocity of the points in the point cloud set; when the velocity of the points in the point cloud set is above the velocity threshold, it is determined that the point cloud set belongs to a fast-moving obstacle, otherwise it is a suspected obstacle; the motion direction of the points in the suspected obstacle is calculated; when the motion direction of the points in the suspected obstacle is inconsistent with the motion direction of other points in the suspected obstacle, it is determined that the point cloud set belongs to a pseudo-obstacle.

[0040] It should be noted that in the point cloud data set output by the input layer, each point only has coordinate information and no velocity information. Therefore, it is necessary to reconstruct the point cloud coordinate matrix of each point and add the velocity coordinate. The velocity value can be initialized to 0 when there is only one moment. The reconstructed point cloud velocity coordinate matrix is as follows: ; where represents the point cloud coordinate data matrix of the th point at the th moment, is the coordinate corresponding to the th point at the th moment, represents the velocity of the th point in the coordinate direction at the th moment.

[0041] After the point cloud velocity coordinate matrix of each point is constructed, calculate the motion velocity of the points in the point cloud set.

[0042] Specifically, obtain the position information of the points in the point cloud set at the th moment; predict the position information of the points in the point cloud set at the th moment through the Kalman filter; calculate the motion velocity and velocity vector of the points in the point cloud set based on the position information at the th moment and the position information at the th moment.

[0043] Step S222, determine whether the point cloud set belongs to a fast-moving obstacle or a suspected obstacle.

[0044] Specifically, when the velocity of the points in the point cloud set is above the velocity threshold of 30 km / h, it is determined that the point cloud set belongs to a fast-moving obstacle, otherwise it is a suspected obstacle.

[0045] Preliminary identification of obstacle types through speed thresholds can quickly and efficiently distinguish different types of obstacles. The setting of the speed threshold is like an intelligent filter that can quickly distinguish fast-moving objects from slow or stationary objects. This preliminary screening greatly narrows the scope of subsequent further analysis, reduces unnecessary computational effort and processing time, and improves the operating efficiency of the entire detection system.

[0046] For the selection of the speed threshold, in the driverless scenario, generally speaking, when the speed of the obstacle relative to the autonomous vehicle reaches 30 km / h or more, it can be regarded as a fast-moving obstacle. For example, on a highway, if a vehicle approaches rapidly from behind at a speed significantly higher than the vehicle itself, or if a vehicle suddenly cuts into the front of the vehicle at high speed from the adjacent lane, these vehicles can be regarded as fast-moving obstacles.

[0047] Step S223: Calculate the movement direction of the midpoint of the suspected obstacle.

[0048] Specifically, if it is determined that the point cloud set belongs to a suspected obstacle, the next identification is required. This judgment is made through the movement direction. Specifically as follows: Obtain the velocity vector based on the position information of the midpoint of the suspected obstacle; calculate the movement direction vector of the point according to the velocity vector; calculate the azimuth angle and pitch angle of the midpoint of the suspected obstacle based on the movement direction vector as the movement direction of the midpoint of the suspected obstacle. Calculating the movement direction based on the position information is a prior art and will not be introduced in detail here.

[0049] Step S224: Determine whether the suspected obstacle point cloud set belongs to a false obstacle or a slow-moving obstacle.

[0050] The specific judgment logic is: When the movement direction of the point is inconsistent with the movement directions of other points in the suspected obstacle, it is determined that the point cloud set belongs to a false obstacle, otherwise it is determined to be a slow-moving obstacle. When the movement directions are inconsistent, it means that the angle difference between this point and any other point is greater than the preset difference threshold.

[0051] After initially screening out the suspected obstacles, further judging and identifying false obstacles based on the movement direction is the key highlight of this method. In the actual environment, false obstacles are often generated by various interference factors, such as rain, snow, sand and dust. The movement directions of these false obstacles are significantly irregular and random, which are completely different from the movement direction characteristics of real obstacles. By analyzing the movement direction, the system can accurately capture this difference. For example, the sundries that occasionally fly through in the roadway may be affected by various factors such as air flow, and their movement directions are significantly inconsistent with the movement directions of the surrounding real obstacles. At this time, the motion feature classification layer can accurately identify it as a false obstacle, avoiding misjudging it as a real obstacle, thus significantly reducing the false detection rate of the detection.

[0052] In addition, considering that when pedestrians are moving, the directions of their arms and legs are usually inconsistent. Therefore, the movement directions of all corresponding points cannot be completely the same. Thus, for the judgment of whether the movement directions are the same, a threshold can be set to prevent misjudgment.

[0053] For example: Set the threshold to 10. Assume that there are 15 points in the point cloud set whose movement directions are inconsistent with that of this point. Then it is judged that the movement directions of this point and other points are inconsistent, and the point cloud set to which this point belongs is a false obstacle. Assume that there are 5 points in the point cloud set whose movement directions are inconsistent with that of this point. Then it is judged that the movement directions of this point and other points are the same, and the point cloud set to which this point belongs is a slow-moving obstacle.

[0054] Finally, in order for the feature extraction layer to process data, the input layer needs to output the speed and movement direction of the point cloud set.

[0055] Step S230: Construct a feature extraction layer.

[0056] Specifically, the feature extraction layer needs to receive the motion features obtained by the motion feature classification layer, that is, speed and movement direction. Then, three-layer convolution is performed on the motion features. The specific operations of the three-layer convolution are the same as those of PointNet. For example, in the first layer of convolution, the features output by the feature extraction layer are extracted to obtain local features; in the second layer of convolution, the output of the first layer is enhanced in features to obtain medium-scale features; in the third layer of convolution, the output of the second layer is used for global feature extraction to obtain global features, and finally a feature map of the three-layer convolution is obtained.

[0057] In addition, since the PointNet model itself has the function of identifying static obstacles, static obstacles in the three-dimensional point cloud image can be obtained in this step.

[0058] Step S240: Construct a dense classification layer.

[0059] Specifically, it includes: receiving the feature map output by the feature extraction layer; calculating the probability that the feature map belongs to an obstacle category through an activation function. The obstacle categories include: fast-moving obstacles, slow-moving obstacles, and false obstacles; using a multi-layer perceptron MLP to extract the non-linear relationship between the motion features and the obstacle categories, and calculating the motion state probability of the point; performing weighted fusion on the motion state probability and the probability of the obstacle category to obtain the category information of the obstacle of the point.

[0060] Specifically, the dense classification layer usually applies an activation function, such as the softmax function. First, the feature map output by the feature extraction layer is converted into a probability distribution through the softmax function. In the present invention, it is the probability of the obstacle category. That is, the probabilities of categories such as fast-moving obstacles, slow-moving obstacles, static obstacles, and false obstacles.

[0061] The multi-layer perceptron (MLP) has a powerful non-linear fitting ability. Through the combination of multiple layers of neurons and the action of activation functions, it can automatically learn this complex non-linear relationship, thus more accurately classifying and understanding obstacles with different motion characteristics. Then, the MLP is used to extract the non-linear relationship between motion characteristics and obstacle categories, and calculate the motion state probability of points; Finally, the motion state probability and the probability of the obstacle category are weighted and fused to obtain the category information of the point's obstacle.

[0062] The category information of the obstacle refers to the comprehensive score including the obstacle classification result and the motion state result.

[0063] For example: the obstacle classification result is one of static obstacle, fast-moving obstacle, slow-moving obstacle, and pseudo-obstacle, and the corresponding motion state results can be speed 0, speed above 30 km / h, 5 m / s inconsistent with the motion direction, and 5 - 30 km / h consistent with the motion direction respectively.

[0064] In summary, the above is the construction process of the four layers of the obstacle detection model. Based on this, it is also necessary to perform loss judgment and optimization processing on the results detected by the model.

[0065] The loss function is used to measure the difference between the model prediction result and the true result, so as to guide the model to optimize parameters to achieve obstacle classification and 3D point cloud reconstruction. The loss function can be calculated using mean squared error loss, mean absolute error loss, and cross-entropy loss. The loss of this model includes three parts: classification loss, localization loss, and motion loss.

[0066] The classification loss measures the difference between the predicted probability of the obstacle category by the model and the true category. The localization loss measures the difference between the predicted position of the true position of the obstacle by the model. The motion loss measures the difference between the predicted probability of the obstacle motion direction and the true motion direction.

[0067] Finally, calculate the total loss of the obstacle : ; Among them, and are preset weighting coefficients, represents the classification loss of the obstacle, represents the localization loss of the obstacle, the motion vector loss of the obstacle; Regarding and For the settings of, accurately classifying the types of obstacles is crucial for autonomous driving decision-making. For example, differentiating between different types of obstacles such as pedestrians, vehicles, and traffic signs so that the vehicle can make different responses. Then, can be appropriately increased to strengthen the weight of the classification task. If during the vehicle's driving process, precise positioning of obstacles is more critical, such as accurately determining the position of obstacles to avoid collisions, then 's value may need to be relatively large. For example, on urban roads, the vehicle needs to frequently avoid pedestrians and other vehicles. At this time, the positioning task is of high importance, and can be set to 0.4, and can be set to 0.6. In some specific scenarios, such as in a parking lot, accurately identifying parking space signs and obstacle types may be more important.

[0068] In the PointNet model, optimization algorithms are also involved. Generally, stochastic gradient descent (SGD) and its variants, such as Adagrad, Adadelta, RMSProp, Adam, AdamW, etc. are used to update the model's parameters. These algorithms can adaptively adjust the learning rate based on the gradient information of the loss function to accelerate the model's convergence speed and avoid getting stuck in local optimal solutions. This part is prior art and will not be introduced in detail here.

[0069] Step S300: When the point cloud set belongs to a pseudo-obstacle, remove the point cloud of the pseudo-obstacle from the three-dimensional point cloud image.

[0070] As Figure 4 shown, through the operations of the above steps, obstacle category information such as pseudo-obstacles, fast-moving obstacles, slow-moving obstacles, and static obstacles in the three-dimensional point cloud image can be identified. As Figure 4 shown, it mainly shows static obstacles such as building 31, pseudo-obstacles such as blowing sand 32, and the ground 33. The point cloud set belonging to the pseudo-obstacle, i.e., the blowing sand 32, needs to be removed from the three-dimensional point cloud image. This can prevent the autonomous vehicle from taking measures such as braking or automatically changing lanes to avoid danger due to incorrect obstacle recognition. After removing the blowing sand, the three-dimensional point cloud image, as Figure 5 shown, there is no ground 33 in the figure. Generally, the ground 33 is removed after collecting the three-dimensional image to prevent it from affecting the subsequent detection of obstacles.

[0071] Although this specification has shown and described several embodiments of the present invention, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Many variations, changes and alternative ways will occur to those skilled in the art without departing from the spirit and scope of the present invention. It should be understood that various alternatives to the embodiments of the invention described herein may be employed in practicing the invention.

Claims

1. A roadway obstacle detection method based on three-dimensional point cloud, characterized in that Including: Collecting a three-dimensional point cloud image and obtaining a plurality of point cloud sets in the three-dimensional point cloud image; Processing any one of the point cloud sets using a constructed obstacle detection model to identify the category information of the obstacle to which the point cloud set belongs; When the point cloud set belongs to a pseudo-obstacle, removing the point cloud of the pseudo-obstacle from the three-dimensional point cloud image; Wherein, the obstacle detection model includes a motion feature classification layer for obtaining the category of the obstacle of the points in the point cloud set and outputting the speed and motion direction of the point cloud set, including: obtaining the point cloud set, calculating the speed of the points in the point cloud set; when the speed of the points in the point cloud set is above the speed threshold, determining that the point cloud set belongs to a fast-moving obstacle, otherwise it is a suspected obstacle; calculating the motion direction of the points in the suspected obstacle; when the motion direction of the points in the suspected obstacle is inconsistent with the motion direction of other points in the suspected obstacle, determining that the point cloud set belongs to a pseudo-obstacle.

2. The roadway obstacle detection method based on 3D point cloud according to claim 1, wherein The calculating the speed of the points in the point cloud set includes: Obtaining the position information of the points in the point cloud set at the current moment; Predicting the position information of the points in the point cloud set at the next moment through Kalman filtering; Calculating the speed of the points in the point cloud set based on the position information at the current moment and the position information at the next moment.

3. The roadway obstacle detection method based on three-dimensional point cloud according to claim 1, wherein, The calculating the motion direction of the points in the suspected obstacle includes: Obtaining a velocity vector based on the position information of the points in the suspected obstacle; Calculating the motion direction vector of the points according to the velocity vector; Calculating the azimuth angle and pitch angle of the points in the suspected obstacle based on the motion direction vector.

4. The roadway obstacle detection method based on 3D point cloud according to claim 1, characterized in that The obstacle detection model further includes loss calculation, and the loss calculation includes: Calculate the classification loss of the obstacle, respectively and the localization loss of the obstacle and the motion vector loss of the obstacle ; Calculate the total loss of the obstacle , where and are preset weighting coefficients.

5. The roadway obstacle detection method based on three-dimensional point cloud according to claim 1, characterized in that The obstacle detection model further includes an input layer, and the input layer includes: Obtaining the coordinate data of any point in the point cloud set; Constructing its coordinate data matrix according to the coordinate data; Constructing a point cloud data set of all points in the point cloud set according to the coordinate data matrix.

6. The roadway obstacle detection method based on three-dimensional point cloud according to claim 1, characterized in that, The obstacle detection model further includes a feature extraction layer, and the feature extraction layer includes: Receiving the motion features of the point cloud set output by the motion feature classification layer, where the motion features include speed and motion direction; Performing three-layer convolution on the motion features to obtain a feature map of the three-layer convolution.

7. The roadway obstacle detection method based on three-dimensional point cloud according to claim 6, wherein The obstacle detection model further includes a dense classification layer, and the dense classification layer includes: Receiving the feature map output by the feature extraction layer; Calculating the probability of the obstacle category to which the feature map belongs through an activation function, and the obstacle categories include: fast-moving obstacles, slow-moving obstacles, pseudo-obstacles; Using a multi-layer perceptron MLP to extract the non-linear relationship between the motion features and the obstacle categories, and calculating the motion state probability of the points in the suspected obstacle; Performing weighted fusion on the motion state probability and the probability of the obstacle category to obtain the category information of the obstacle of the points in the suspected obstacle.

8. The roadway obstacle detection method based on 3D point cloud according to claim 1, characterized in that The collecting the three-dimensional point cloud image includes: Obtaining a three-dimensional point cloud image; Performing denoising processing on the three-dimensional point cloud image.

9. The roadway obstacle detection method based on three-dimensional point cloud according to claim 1, characterized in that, The obtaining the plurality of point cloud sets in the three-dimensional point cloud image includes: Perform mean shift clustering on the point cloud in the three-dimensional point cloud image to obtain multiple point cloud sets in the three-dimensional point cloud image.

10. The roadway obstacle detection method based on three-dimensional point cloud according to claim 1, wherein When the motion direction of a point in the suspected obstacle is inconsistent with the motion directions of other points in the suspected obstacle, the angular difference between the point and any other point is greater than a preset difference threshold.

Images