Obstacle Recognition Method, Apparatus and Electronic Device

Abstract

The present invention provides an obstacle recognition method, apparatus and electronic device; wherein, the method includes: obtaining initial point cloud data of the vehicle surrounding environment, and filtering out ground noise points and edge noise points to obtain target point cloud data; and clustering the target point cloud data based on the run clustering algorithm to obtain initial obstacle information; and filtering the initial obstacle information according to a preset filtering rule to obtain obstacle information; the method first filters out ground noise points and edge noise points in the initial point cloud data, and clusters to obtain initial obstacle information; then filters the initial obstacle information again according to the preset filtering rule, so as to effectively alleviate the problem that the edge noise points and ground noise points affect the accuracy of the under-segmentation of the obstacle clustering result, improve the accuracy of obstacle recognition, and further improve the safety of vehicle autonomous driving, and has good practical value.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle environmental perception, and in particular to an obstacle recognition method, device and electronic device. Background Art

[0002] Autopilot is a technology that controls a vehicle to automatically drive on the road through a computer device. Due to the complex actual road conditions and the existence of a large number of obstacles such as pedestrians and vehicles, therefore, how to achieve obstacle recognition to plan a driving route for avoiding obstacles has become the key to autopilot.

[0003] Currently, obstacle recognition is mainly achieved through lidar sensing technology. Among them, obstacle clustering is a key step in the sensing technology. The method of point cloud clustering with the patent application number 202010570828.6 mainly calculates the distance between two points according to the radial distance and polar angle difference between two points for the ordered point cloud data of the lidar, and then judges whether they belong to the same target according to the distance and the first preset distance value. If so, they are classified into the same clustering set; and, calculate the core points of each clustering set, and fuse different clustering sets according to the distance between the core points and the second preset distance value. However, since there are often wire drawing points, that is, edge noise points, due to inaccurate ranging between the intersections of the detected obstacle edges by the lidar, although the above method can achieve clustering recognition of obstacles, it does not consider the influence of edge noise points on obstacle clustering, and the above distance value is a fixed value, which is easily affected by distance changes, thus reducing the clustering effect of obstacles and further affecting the recognition effect of obstacles. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide an obstacle recognition method, device and electronic device to alleviate the above problems, improve the recognition accuracy of obstacles, thereby improving the safety of vehicle autopilot, and having good practical value.

[0005] In a first aspect, an embodiment of the present invention provides an obstacle recognition method, which includes: obtaining initial point cloud data of the vehicle surrounding environment; where the initial point cloud data is a data set in a point cloud coordinate system; determining ground noise points in the initial point cloud data based on the grid method, and filtering the ground noise points to obtain first point cloud data; converting the first point cloud data to obtain second point cloud data in an image coordinate system; determining edge noise points in the second point cloud data according to a screening rule, and filtering the edge noise points to obtain target point cloud data; where the screening rule includes a dynamic search radius and an adjacent point included angle threshold; clustering the target point cloud data based on the run clustering algorithm to obtain initial obstacle information; and filtering the initial obstacle information according to a preset filtering rule to obtain obstacle information.

[0006] Optionally, the step of determining the edge noise points in the second point cloud data according to the screening rule includes: for the current point in the second point cloud data, determining the adjacent points of the current point according to the dynamic search radius; wherein, the distance between the adjacent points and the current point is not greater than the dynamic search radius; calculating the adjacent point angle between each adjacent point and the current point; and determining the edge noise points according to the adjacent point angle and the adjacent point angle threshold.

[0007] Optionally, the step of determining the edge noise points according to the adjacent point angle and the adjacent point angle threshold includes: judging whether the adjacent point angle is less than the adjacent point angle threshold; if so, determining the adjacent point corresponding to the adjacent point angle as the edge noise point.

[0008] Optionally, the step of determining the ground noise points in the initial point cloud data based on the grid method further includes: determining the grid map corresponding to the initial point cloud data based on the grid method; wherein, the grid map includes a plurality of grids; determining the target height value of each grid; wherein, the target height value is the minimum value of the heights of the multiple points within the grid; judging whether the target height value is not greater than the preset height threshold; if so, determining the grid corresponding to the target height value as the target grid, and taking the multiple points within the target grid as the ground noise points.

[0009] Optionally, the step of converting the first point cloud data to obtain the second point cloud data in the image coordinate system includes: converting the first point cloud data according to the first conversion rule to obtain the point cloud data in the spherical coordinate system; wherein, the first conversion rule is used to represent converting the data from the point cloud coordinate system to the spherical coordinate system; converting the point cloud data in the spherical coordinate system according to the second conversion rule to obtain the second point cloud data in the image coordinate system; wherein, the second conversion rule is used to represent converting the data from the spherical coordinate system to the image coordinate system.

[0010] Optionally, the step of setting a lidar on the vehicle to obtain the initial point cloud data of the vehicle surrounding environment includes: obtaining the initial point cloud data collected by the lidar; judging whether there is a pitch angle and / or a rotation angle and / or a yaw angle of the lidar relative to the vehicle body coordinate system; if so, performing attitude calibration on the initial point cloud data according to the preset attitude correction matrix to obtain the corrected initial point cloud data.

[0011] Optionally, the step of setting a depth camera on the vehicle to obtain the initial point cloud data of the vehicle surrounding environment further includes: obtaining the depth image of the vehicle surrounding environment collected by the depth camera; and converting the depth image to obtain the corresponding initial point cloud data.

[0012] Second aspect, an embodiment of the present invention further provides an obstacle recognition device, the device includes: an acquisition module, configured to acquire initial point cloud data of the vehicle surrounding environment; wherein, the initial point cloud data is a data set in a point cloud coordinate system; a first determination module, configured to determine ground noise points in the initial point cloud data based on a grid method, and perform filtering processing on the ground noise points to obtain first point cloud data; a conversion module, configured to convert the first point cloud data to obtain second point cloud data in an image coordinate system; a second determination module, configured to determine edge noise points in the second point cloud data according to a screening rule, and perform filtering processing on the edge noise points to obtain target point cloud data; wherein, the screening rule includes a dynamic search radius and an adjacent point angle threshold; a clustering module, configured to cluster the target point cloud data based on a run clustering algorithm to obtain initial obstacle information; a filtering module, configured to perform filtering processing on the initial obstacle information according to a preset filtering rule to obtain obstacle information.

[0013] Third aspect, an embodiment of the present invention further provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the steps of the method in the first aspect are implemented.

[0014] Fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, the steps of the method in the first aspect are executed.

[0015] The embodiments of the present invention bring the following beneficial effects:

[0016] The embodiments of the present invention provide an obstacle recognition method, device and electronic device. By determining edge noise points through a dynamic search radius and an adjacent point angle threshold, and filtering the edge noise points and ground noise points, so as to cluster the target point cloud data to obtain initial obstacle information; and, performing filtering processing on the initial obstacle information again according to a preset filtering rule, thereby effectively alleviating the problem that the edge noise points and ground noise points affect the accuracy of the under-segmentation of the obstacle clustering result, improving the recognition accuracy of the obstacle, and further improving the safety of vehicle autonomous driving, having good practical value.

[0017] Other features and advantages of the present invention will be described in the following specification, and, in part, will be obvious from the specification, or will be understood by implementing the present invention. The objectives and other advantages of the present invention are achieved and obtained by the structures specifically pointed out in the specification and the drawings.

[0018] To make the above objectives, features and advantages of the present invention more obvious and understandable, the following specific preferred embodiments are given below, and in conjunction with the accompanying drawings, detailed descriptions are as follows. Description of the Drawings

[0019] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for the description of the specific embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0020] Figure 1 It is a flowchart of an obstacle recognition method provided by an embodiment of the present invention;

[0021] Figure 2 It is a schematic diagram of calculating the included angle between adjacent points provided by an embodiment of the present invention;

[0022] Figure 3 It is a schematic diagram of the point cloud of an image structure provided by an embodiment of the present invention;

[0023] Figure 4 It is a schematic diagram of an obstacle recognition device provided by an embodiment of the present invention;

[0024] Figure 5 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. Specific Embodiments

[0025] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions of the present invention with reference to the drawings. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the scope of protection of the present invention.

[0026] To facilitate the understanding of this embodiment, the following will first introduce in detail an obstacle recognition method provided by an embodiment of the present invention.

[0027] An embodiment of the present invention provides an obstacle recognition method, as Figure 1 shown, the method includes the following steps:

[0028] Step S102, obtaining initial point cloud data of the vehicle surrounding environment;

[0029] Among them, the initial point cloud data is a data set in the point cloud coordinate system, including multiple points, and each point is usually represented by the point cloud coordinate system, namely the X-axis, Y-axis, and Z-axis; here, the point cloud coordinate system is a Cartesian coordinate system centered on the sensor for collecting point cloud data. In practical applications, common sensors include but are not limited to lidar and depth cameras, etc., and can be specifically set according to actual situations.

[0030] In one of the acquisition methods, a lidar is provided on the vehicle. For example, the lidar is provided on the top of the vehicle. In practical applications, the lidar is a sensor that can detect obstacles by sensing the surrounding environment, including but not limited to rotary lidars. The specific process is as follows: Obtain the initial point cloud data collected by the lidar; Determine whether there are pitch and / or rotation and / or yaw angles of the lidar relative to the vehicle body coordinate system; If so, perform attitude calibration on the initial point cloud data according to a preset attitude correction matrix to obtain the corrected initial point cloud data. Specifically, the electronic device directly obtains the initial point cloud data sent by the lidar. At this time, the point cloud coordinate system is a Cartesian coordinate system centered on the lidar. Since the lidar may have pitch and / or rotation and / or yaw angles relative to the vehicle body coordinate system, therefore, at this time, attitude correction needs to be performed on the initial point cloud data. For example, calibration can be performed according to the following formula:

[0031] P′ = R 校正 ·P (1)

[0032] where P′ represents the corrected initial point cloud data, R 校正 represents the attitude correction matrix, and P represents the initial point cloud data before correction. It should be noted that the above attitude correction matrix R 校正 can be set according to the actual situation.

[0033] In another acquisition method, a depth camera is also provided on the vehicle. Existing depth cameras can directly output point cloud data or directly output depth images. For the case of depth images, the specific acquisition process is as follows: Obtain the depth image of the surrounding environment of the vehicle collected by the depth camera; Convert the depth image to obtain the corresponding initial point cloud data. Specifically, after the electronic device obtains the depth image, it combines the built-in parameters of the depth camera (such as focal length and optical center, etc.) to convert the depth image to obtain the corresponding initial point cloud data. At this time, the point cloud coordinate system is a Cartesian coordinate system centered on the depth camera. If attitude correction needs to be performed on the initial point cloud data, the above formula can be specifically referred to. This embodiment of the present invention will not be elaborated in detail here.

[0034] It should be noted that whether the above sensor is a lidar or a depth camera, before obtaining the initial point cloud data, it is also necessary to calibrate the installation height H of the sensor relative to the ground in the electronic device, and some sensors may also need to calibrate other parameters, such as the built-in parameters of the depth camera, etc., so as to perform attitude correction or conversion processing on the initial point cloud data in the later stage, thereby improving the speed of data processing and further improving the obstacle recognition efficiency.

[0035] Step S104, determine the ground noise points in the initial point cloud data based on the grid method, and perform filtering processing on the ground noise points to obtain the first point cloud data;

[0036] Among them, the above-mentioned ground noise points refer to multiple points belonging to the ground area in the initial point cloud data; specifically, the determination process of the ground noise points is as follows: Based on the grid method, a grid map corresponding to the initial point cloud data is determined; among them, the grid map includes multiple grids; the target height value of each grid is determined; among them, the target height value is the minimum value of the heights of multiple points within the grid; it is judged whether the target height value is not greater than a preset height threshold; if so, the grid corresponding to the target height value is determined as the target grid, and the multiple points within the target grid are used as ground noise points.

[0037] Specifically, first, according to the grid method, the initial point cloud data is divided into multiple grids, and each grid includes at least one point; then, for each grid, according to the height of each of its points, that is, the z value, the minimum height within the grid is determined as the target height value of the grid, and then the grid with the target height value not greater than the preset height threshold is used as the target grid. Here, the preset height threshold is preferably the installation height H of the sensor relative to the ground. In addition, considering that since it is impossible to ensure 100% detection of ground noise points, some ground noise points may be missed due to slopes or uneven road surfaces, the preset height threshold can also be composed of the installation height H and the threshold T, that is, when the target height value is not greater than the preset height threshold (that is, -H + T), the grid corresponding to the target height value is the target grid, and all the multiple points within the target grid are ground noise points. Among them, the threshold T is preferably 10 cm to 30 cm, and the specific value can be set according to the actual situation. The installation height H of the above-mentioned sensor relative to the ground can, on the one hand, provide a prior height for ground noise point filtering, such as directly using H as the preset height threshold, and on the other hand, can also provide a prior range for the preset height threshold, such as using H and T as the preset height threshold.

[0038] In summary, for multiple points in the initial point cloud data, the smaller the height, that is, the z value, the higher the probability of belonging to ground noise points. Therefore, according to the target height value of each grid and the preset height threshold, ground noise points can be determined and filtered to obtain the first point cloud data without ground noise points, alleviating the problem that ground noise points connect obstacles together, resulting in under-segmentation of the obstacle clustering result, further improving the accuracy of the obstacle clustering result, that is, improving the recognition accuracy of obstacles, and ensuring the safety of vehicle autonomous driving.

[0039] In addition, for each grid, the detection of ground noise points can also be realized by methods such as the least squares method, the principal component analysis method, SVD (Singular Value Decomposition), and random sample consensus, and the specific setting can be made according to the actual situation.

[0040] Step S106: Convert the first point cloud data to obtain the second point cloud data in the image coordinate system;

[0041] Specifically, first convert the first point cloud data according to the first conversion rule to obtain the point cloud data in the spherical coordinate system. The first conversion rule is used to represent the conversion of data from the point cloud coordinate system to the spherical coordinate system. Then convert the point cloud data in the spherical coordinate system according to the second conversion rule to obtain the second point cloud data in the image coordinate system. The second conversion rule is used to represent the conversion of data from the spherical coordinate system to the image coordinate system. In practical applications, for the above first point cloud data, due to the sparse structure of the point cloud data, there is no obvious neighborhood relationship like an image, and considering that the horizontal and vertical laser emission lines of the sensor usually detect the environment at a fixed angle, first convert the first point cloud data according to the first conversion rule to obtain the point cloud data in the spherical coordinate system to determine the neighborhood relationship between points in the point cloud data.

[0042] Among them, the spherical coordinate system is a kind of three-dimensional coordinate system used to determine the positions of points, lines, planes, and solids in three-dimensional space. Usually, the origin is used as the reference point, and it is composed of the azimuth angle, elevation angle, and distance. Since each point in the first point cloud data is represented by (x, y, z) in the point cloud coordinate system and each point in the spherical coordinate system is represented by (α, β, γ), the above first conversion rule can be expressed by the following formula:

[0043]

[0044] Among them, α represents the elevation angle of the laser emission line, β represents the azimuth angle, and d represents the radius, that is, the distance. It should be noted that if β is negative, then β + 2π is used instead.

[0045] Since there are fixed horizontal and vertical angular resolutions for the laser emission lines emitted by the lidar, the point cloud data in the spherical coordinate system can be converted according to the second conversion rule to obtain the second point cloud data. Each image pixel in the image coordinate system consists of a four-channel point cloud coordinate containing the echo intensity. The conversion formula of the second conversion rule is as follows:

[0046]

[0047] Among them, row represents the row index of the image coordinate system, col represents the column index of the image coordinate system, α represents the elevation angle of the laser emission line, β represents the azimuth angle, ver_resolution represents the vertical angular resolution, hor_resolution represents the horizontal angular resolution, and int represents the integer operation.

[0048] In summary, through two conversion rules, the first point cloud data in the point cloud coordinate system can be converted to obtain the second point cloud data in the image coordinate system, that is, the second structured point cloud data of the image. Here, each image pixel can be characterized by (x, y, z, amp), where x, y, and z respectively represent the three-dimensional coordinates of the point cloud, and amp represents the echo intensity of the point cloud, so as to determine the edge noise points based on the structured second point cloud data. In addition, in practical applications, the first point cloud data with ground noise points filtered can also be directly searched for the octree result to obtain the second point cloud data in the image coordinate system.

[0049] Step S108: Determine the edge noise points in the second point cloud data according to the screening rules, and perform filtering processing on the edge noise points to obtain the target point cloud data;

[0050] In practical applications, for the obstacles detected by lidar, due to inaccurate ranging caused by the edge intersection between obstacles, that is, there are wire-drawing points. If they are not filtered out, it may cause the front and rear two obstacles to be misclassified as the same target during clustering, thus affecting the safety of vehicle autonomous driving. Therefore, the wire-drawing points between the above-mentioned obstacles are also called edge noise points.

[0051] For edge noise points, the screening rules include a dynamic search radius and an adjacent point angle threshold; among them, the dynamic search radius is determined by the horizontal angle resolution, vertical angle resolution, and neighborhood scaling factor of the lidar. The specific formula is as follows:

[0052]

[0053] Among them, R represents the dynamic search radius, a represents the neighborhood scaling factor, and a ≥ 1. Similar to the number of neighborhood pixels in an image, it can be set to 1 when screening edge filtering. Δθ represents the vertical angle resolution or horizontal angle resolution of the lidar, and d represents the radius, that is, the distance, that is, the distance between this point and the lidar (i.e., the origin of the point cloud coordinate system). Therefore, for multiple points in the second point cloud data, since the distance d of each point is different, the dynamic search radius of each point is also different. Compared with the conventional screening based on a fixed distance, the accuracy of screening edge point clouds (i.e., edge noise points) is improved, thereby improving the accuracy of the clustering result of the target point cloud data without edge noise points.

[0054] The process of determining the edge noise points in the second point cloud data according to the screening rules is as follows: For the current point in the second point cloud data, first determine the adjacent points of the current point according to the dynamic search radius; where the distance between the adjacent points and the current point is not greater than the dynamic search radius; calculate the adjacent point angle between each adjacent point and the current point; then, determine the edge noise points according to the adjacent point angle and the adjacent point angle threshold. That is, judge whether each adjacent point angle is less than the adjacent point angle threshold. If so, determine the adjacent point corresponding to the adjacent point angle as the edge noise point. Specifically, for each point in the second point cloud data, it is used as the current point, and the adjacent points of the current point are determined according to the dynamic search radius of the current point. For example, calculate the distances between the points in eight directions (such as up, down, left, right, upper left, upper right, lower left, and lower right) of the current point and the current point, and take the points with distances not greater than the dynamic search radius as adjacent points (that is, the points that meet the neighborhood relationship); then calculate the adjacent point angle between each adjacent point and the current point; if the adjacent point angle is less than the preset adjacent point angle threshold, the adjacent point corresponding to the adjacent point angle is the edge noise point.

[0055] For the sake of easy understanding, an example is given here. As Figure 2 shown, point A is the current point. First, calculate the distances between point A and multiple points in its surrounding directions, and determine the adjacent points as point B and point D according to the relationship between the distances and the dynamic search radius of point A; then, calculate the adjacent point angle between point D and point A and the adjacent point angle between point B and point A For example, project point B and point A onto the X-Y plane, and draw a perpendicular line from the shortest side of the radius on the X-Y plane to the other side. Here, draw a perpendicular line from OB to OA, that is, BC, and calculate the adjacent point angle between point B and point A according to the following formula:

[0056]

[0057] where, OA represents the projection radius of the current point A, OB represents the projection radius of the adjacent point B, AB represents the projection distance between point A and point B, represents the adjacent point angle between point B and point A, and Δθ represents the vertical angle resolution or horizontal angle resolution of the lidar. Therefore, by setting the same or different thresholds for the vertical and horizontal adjacent point angles, if the adjacent point angle calculated according to the vertical angle resolution or horizontal angle resolution is less than the corresponding adjacent point angle threshold, it is determined that the adjacent point and the current point do not belong to the same target, that is, the adjacent point is determined as the edge noise point of the current point, and the adjacent points belonging to the same target as the current point are determined as non-edge noise points. For example, the above adjacent point angle is greater than the adjacent point angle threshold, and, the adjacent point angle If it is less than the adjacent point angle threshold, then point D is determined as the edge noise point of point A. And, if the adjacent point angles of a certain point and all its adjacent points are all less than the corresponding adjacent point angle threshold, then this point belongs to the edge noise point.

[0058] It should be noted that in practical applications, the ratio of the projection radii of adjacent points can also be used to replace the adjacent point angle. For example, set a projection radius ratio threshold, and calculate the ratio of the projection radius of each adjacent point to the projection radius of the current point. If the ratio is greater than the projection radius threshold, then the adjacent point corresponding to this ratio is the edge noise point of the current point; conversely, if the ratio is not greater than the projection radius threshold, then the adjacent point corresponding to this ratio and the current point belong to the same target and are non-edge noise points of the current point; and for the remaining points in the second point cloud data, they can all be used as the current point, and the edge noise points of each current point are determined according to the above screening process, and the edge noise point set corresponding to the second point cloud data is filtered to obtain the target point cloud data without ground noise and edge noise.

[0059] Step S110, cluster the target point cloud data based on the run-based clustering algorithm to obtain the initial information of obstacles;

[0060] Specifically, the point set scanned by a single laser line in one cycle is called a run. In the image coordinate system, a row of pixels is a run. As Figure 3 shown, for the point cloud of the image structure (i.e., the target point cloud data in the image coordinate system), each point cloud is simplified and represented by one dimension. Among them, elements 1, 2, 2, and 3 in the first row form a run. Similarly, elements 1, 1, 3, and 4 in the second row also form a run. Therefore, the run-based clustering algorithm mainly includes three key parameters: the lookup table, the current row local clustering cluster set, and the previous row local clustering cluster set.

[0061] Among them, for each local clustering cluster set, it consists of the start and end column indexes and the cluster index number of this set; first, for the point cloud data of each row, cluster the points in the run horizontally. According to the horizontal angular resolution of the lidar, calculate the dynamic search radius of this point in the horizontal direction, and by setting the horizontal adjacent point angle threshold, search whether the adjacent point is a neighborhood point, and calculate the angle between the valid point clouds in the neighborhood. If the angle is greater than the horizontal adjacent point angle threshold, then this adjacent point and this point belong to the same target set. After searching all the row-wise local clustering cluster sets, initialize the size of the lookup table space to be the same as the number of clustering cluster sets, and initialize the information in the table to -1. The element index number of the lookup table represents the corresponding local clustering cluster set, and the element content represents the parent node of this local clustering cluster set. If the value is -1, it means that its parent node is itself, that is, after horizontal clustering, there is no other cluster set merged with this local clustering cluster set; the specific initialized lookup table is shown in the following table:

[0062] Table 1

[0063]

[0064] Among them, the structure of the local clustering clusters is shown in Table 2 below:

[0065] Table 2

[0066]

[0067] For example, for a certain local clustering cluster in the current row, the row index row: 1 indicates that the local clustering cluster is the traversal result of the first row; the starting index c1: 2 indicates that the starting point is the element at the position (1, 2); the ending index c2: 3 indicates that the ending point is the element at the position (1, 3); the number i: 0 indicates that for all local clustering clusters, the clustering number of this local clustering cluster is the first. Therefore, the above number is the id number of this local clustering cluster, that is, the cluster index number, and the starting index and the ending index are the starting and ending column indexes of this local clustering cluster. For the local clustering clusters of the remaining rows, the above local clustering cluster can be referred to, and the embodiments of the present invention will not be elaborated in detail here.

[0068] Secondly, perform a vertical traversal of the local clustering clusters. By setting the vertical adjacent point angle threshold, judge whether the point cloud in the upper local clustering cluster belongs to the neighborhood range of the elements in the current local clustering cluster along the three directions of upper left, up, and upper right. If the angle of the adjacent points in the vertical direction within the neighborhood range meets the threshold condition, that is, the angle is greater than the vertical adjacent point angle threshold, then merge the two local clustering clusters, and select the smallest index value from the local clustering clusters to be fused as the common parent node and assign it to the corresponding lookup table.

[0069] For the sake of easy understanding, an example is given here. For Figure 3 Among the multiple local clustering clusters in the lookup table, if they are divided into one class with the same numerical value, the element 1 in the first row is a local clustering cluster during the horizontal traversal, and the subsequent two elements 2 are merged into one local clustering cluster; the element 3 is a local clustering cluster; at this time, initialize three local clustering clusters to store the corresponding sets respectively. Taking the first local clustering cluster as an example, its row number is 1 (assuming the row number starts from 1), the starting point is 3, the ending point is 3, and the number id: 0.

[0070] During the vertical traversal of local clustering clusters, the local clustering clusters in the first row are skipped because there are no local clustering clusters in the previous row. For the second row, the local clustering clusters in the first row can be traversed. For example, for the element 1 in the second row, there is a set with the same element in its upper left among its eight neighborhoods, that is, there are local clustering clusters of the same type. The element 1 in the first row and the element 1 in the second row can be merged, and after the merger, the minimum value among the numbers in multiple local clustering clusters is used as the number of the corresponding parent node and assigned to the lookup table. Similarly, for the remaining local clustering clusters, the vertical traversal can be performed with reference to the element 1 in the second row as described above, and the embodiments of the present invention will not be elaborated in detail here.

[0071] In summary, by clustering the target point cloud data through the above-mentioned run-based clustering algorithm, the initial obstacle information can be obtained. Here, the initial obstacle information may include sub-information of multiple obstacles, which is specifically determined according to the actual obstacle situation. And, during the clustering process, the horizontal angular resolution and vertical angular resolution of the lidar, as well as the horizontal adjacent point included angle threshold, vertical adjacent point included angle threshold, and neighborhood space size, etc. need to be set. Preferably, the neighborhood scaling factor a during the clustering process can be set to 3. It should be noted that the above parameters such as the horizontal adjacent point included angle threshold, vertical adjacent point included angle threshold, and neighborhood space size can be the same or different during the clustering process and the process of screening edge noise points, and can be specifically set according to the actual situation.

[0072] Step S112, perform filtering processing on the initial obstacle information according to a preset filtering rule to obtain obstacle information.

[0073] Specifically, for the initial obstacle information obtained by the above clustering, there may be clustering noise points, etc. Therefore, in order to further improve the recognition accuracy of obstacles, it is also necessary to determine whether there are clustering noise points in the initial obstacle information and perform filtering processing on the determined clustering noise points.

[0074] Specifically, perform filtering processing on the initial obstacle information according to a preset filtering rule to obtain obstacle information. Among them, the preset filtering rule includes at least one of the following: the number of point clouds, the reflection intensity, the height threshold, and the shape feature. In practical applications, filtering is usually performed according to the number of clustering point clouds. However, since there are generally 5 to 6 points for obstacles at a long distance in reality, in order to avoid mis-filtering real obstacles, for the initial obstacle information after clustering, the possible clustering noise points in the initial obstacle information are also filtered through the preset filtering rule to obtain a more accurate obstacle recognition result, thereby ensuring the reliability and safety of vehicle autonomous driving.

[0075] For the above preset filtering rules, it is preferably a comprehensive filtering rule including the number of point clouds, reflection intensity, height threshold, and shape features. The specific process is as follows: First, set the point cloud number threshold to initially screen out a set of suspected noise points, such as a set with the number of point clouds less than the point cloud number threshold; considering that effective obstacles are usually above the ground, the height threshold Th of the obstacle from the ground can also be set. For example, if Th is set to 3 meters to 4 meters, then the points above Th in the initial obstacle information may be clustering noise points; in addition, noise points usually have weak reflection intensity, so the reflection intensity threshold can be set according to prior knowledge, and the points below this reflection intensity threshold in the initial obstacle information may be clustering noise points; and, in practical applications, since black vehicles are relatively common, and the point clouds are sparse and the reflection intensity is weak at a long distance, in order to avoid filtering out distant black targets, the geometric characteristics of the point clouds can also be calculated, and by calculating the covariance matrix of the set of suspected noise points, PCA (Principal Components Analysis) analysis is performed, and the relationship between the three eigenvalues is compared to judge the geometric characteristics of the set, that is, the clustering noise points are determined according to the shape features. The judgment rules for the shape features are as follows:

[0076]

[0077] Among them, λ1, λ2, and λ3 respectively represent the eigenvalues of the set of suspected noise points sorted from large to small, and T is the preset feature threshold. When the result is not less than the preset feature threshold, it indicates that the set of suspected noise points conforms to the linear or planar feature, that is, each point in the suspected set belongs to the same obstacle. Otherwise, it belongs to the clustering noise points.

[0078] In addition, if a certain set of suspected noise points in the initial obstacle information does not conform to all the judgment rules, it is determined as a noise set and filtered out. And for a certain set of suspected noise points in the initial obstacle information, it can be filtered out in sequence according to the number of point clouds, reflection intensity, height threshold, and shape features, or the same or different weight systems can be configured for the number of point clouds, reflection intensity, height threshold, and shape features respectively, and the respective results are weighted and summed to obtain the final comprehensive value, and it is judged whether it is a clustering noise point according to the comprehensive value and the preset comprehensive value. If so, it is filtered out. Therefore, the clustering noise points existing in the initial obstacle information are determined and filtered out through the comprehensive filtering rule including the number of point clouds, reflection intensity, height threshold, and shape features, avoiding the misdeletion of the clustering points of real obstacles, thereby improving the accuracy of the obstacle information after filtering.

[0079] In summary, for the obstacle recognition method provided by the embodiments of the present invention, first, ground noise points in the initial point cloud data are filtered based on the grid method, alleviating the problem that ground noise points connect the above-ground obstacles together, resulting in under-segmentation of the clustering results of the above-ground obstacles; then, edge noise points are determined through a dynamic search radius and an adjacent point angle threshold, solving the problem that due to wire-drawing points at the edges of obstacles, there is an adhesion relationship between different obstacles, resulting in under-segmentation of the clustering results; in addition, clustering is performed on the target point cloud data without edge noise points and ground noise points, further improving the accuracy of the initial information of the obstacles. Moreover, during the clustering process, clustering is performed through a dynamic search radius and different adjacent point angles (including a horizontal adjacent point angle threshold and a vertical adjacent point angle threshold), and it will not fail due to changes in distance, solving the problem that the fixed setting of the neighborhood search radius threshold leads to over-segmentation or under-segmentation of the obstacle clustering results. Also, compared with clustering that relies on distance, it has stronger robustness; and, the initial information of the obstacles is filtered again through a preset filtering rule. Thus, by filtering the noise points (including ground noise points and edge noise points) in the target point cloud data and filtering the clustering noise points in the initial information of the obstacles, the recognition accuracy of the obstacles is improved, thereby improving the safety and reliability of vehicle autonomous driving, and having good practical value.

[0080] Embodiment 2:

[0081] Corresponding to the above method embodiment, the embodiments of the present invention further provide an obstacle recognition device, as Figure 4 shown. The device sequentially includes: an acquisition module 41, a first determination module 42, a conversion module 43, a second determination module 44, a clustering module 45, and a filtering module 46; wherein, the functions of each module are as follows:

[0082] The acquisition module 41 is configured to acquire the initial point cloud data of the vehicle surrounding environment; wherein, the initial point cloud data is a data set in the point cloud coordinate system;

[0083] The first determination module 42 is configured to determine the ground noise points in the initial point cloud data based on the grid method, and perform filtering processing on the ground noise points to obtain the first point cloud data;

[0084] The conversion module 43 is configured to convert the first point cloud data to obtain the second point cloud data in the image coordinate system;

[0085] The second determination module 44 is configured to determine the edge noise points in the second point cloud data according to the screening rule, and perform filtering processing on the edge noise points to obtain the target point cloud data; wherein, the screening rule includes a dynamic search radius and an adjacent point angle threshold;

[0086] The clustering module 45 is configured to perform clustering on the target point cloud data based on the run clustering algorithm to obtain the initial information of the obstacles;

[0087] A filtering module 46 is configured to filter the initial obstacle information according to a preset filtering rule to obtain obstacle information.

[0088] An embodiment of the present invention provides an obstacle recognition device. First, ground noise points and edge noise points in the initial point cloud data are filtered, and obstacles' initial information is obtained by clustering; then, the initial obstacle information is filtered again according to a preset filtering rule, thereby effectively alleviating the problem that the under-segmentation of the obstacle clustering result caused by edge noise points and ground noise points affects the accuracy, improving the recognition accuracy of obstacles, and further improving the safety of vehicle automatic driving, which has good practical value.

[0089] In one possible implementation, the second determination module 44 is further configured to: for a current point in the second point cloud data, determine adjacent points of the current point according to a dynamic search radius; where the distance between an adjacent point and the current point is not greater than the dynamic search radius; calculate the adjacent point angle between each adjacent point and the current point; and determine an edge noise point according to the adjacent point angle and an adjacent point angle threshold.

[0090] In another possible implementation, the determining an edge noise point according to the adjacent point angle and the adjacent point angle threshold includes: determining whether the adjacent point angle is less than the adjacent point angle threshold; if so, determining the adjacent point corresponding to the adjacent point angle as an edge noise point.

[0091] In another possible implementation, the first determination module 42 is further configured to: based on a grid method, determine a grid map corresponding to the initial point cloud data; where the grid map includes a plurality of grids; determine a target height value for each grid; where the target height value is the minimum value of the heights of multiple points within the grid; determine whether the target height value is not greater than a preset height threshold; if so, determining the grid corresponding to the target height value as a target grid, and regarding multiple points within the target grid as ground noise points.

[0092] In another possible implementation, the conversion module 43 is further configured to: convert the first point cloud data according to a first conversion rule to obtain point cloud data in a spherical coordinate system; where the first conversion rule is used to represent the conversion of data from a point cloud coordinate system to a spherical coordinate system; convert the point cloud data in the spherical coordinate system according to a second conversion rule to obtain second point cloud data in an image coordinate system; where the second conversion rule is used to represent the conversion of data from a spherical coordinate system to an image coordinate system.

[0093] In another possible implementation, a lidar is provided on the vehicle, and the obtaining module 41 is further configured to: obtain the initial point cloud data collected by the lidar; determine whether there are pitch and / or rotation and / or yaw angles of the lidar relative to the vehicle body coordinate system; if so, perform attitude calibration on the initial point cloud data according to a preset attitude correction matrix to obtain the corrected initial point cloud data.

[0094] In another possible implementation, a depth camera is provided on the vehicle, and the obtaining module 41 is further configured to: obtain the depth image of the surrounding environment of the vehicle collected by the depth camera; perform conversion on the depth image to obtain the corresponding initial point cloud data.

[0095] The obstacle recognition device provided in the embodiment of the present invention has the same technical features as the obstacle recognition method provided in the above embodiment, so it can also solve the same technical problems and achieve the same technical effects.

[0096] The embodiment of the present invention further provides an electronic device, including a processor and a memory, where the memory stores machine-executable instructions that can be executed by the processor, and the processor executes the machine-executable instructions to implement the above obstacle recognition method.

[0097] See Figure 5 As shown, the electronic device includes a processor 100 and a memory 101. The memory 101 stores machine-executable instructions that can be executed by the processor 100, and the processor 100 executes the machine-executable instructions to implement the above obstacle recognition method.

[0098] Further, Figure 5 The electronic device shown further includes a bus 102 and a communication interface 103, and the processor 100, the communication interface 103, and the memory 101 are connected through the bus 102.

[0099] Among them, the memory 101 may include high-speed random access memory (RAM), and may also include non-volatile memory, such as at least one disk memory. The communication connection between this system network element and at least one other network element is realized through at least one communication interface 103 (which can be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. can be used. The bus 102 can be an ISA (Industrial Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, an EISA (Enhanced Industry Standard Architecture) bus, etc. The above buses can be divided into an address bus, a data bus, a control bus, etc. For the sake of easy representation, Figure 5 only a two-way arrow is used in Figure 5 , but it does not mean that there is only one bus or one type of bus.

[0100] The processor 100 may be an integrated circuit chip with the ability to process signals. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in the processor 100 or the instructions in the form of software. The above-mentioned processor 100 may be a general-purpose processor, including a central processing unit (CPU for short), a network processor (NP for short), etc.; it may also be a digital signal processor (DSP for short), an application specific integrated circuit (ASIC for short), a field-programmable gate array (FPGA for short), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of the present invention can be directly embodied as being executed and completed by a hardware decoding processor, or executed and completed by a combination of the hardware and software modules in the decoding processor. The software module may be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. This storage medium is located in the memory 101, and the processor 100 reads the information in the memory 101 and combines its hardware to complete the steps of the method in the foregoing embodiments.

[0101] This embodiment also provides a machine-readable storage medium, which stores machine-executable instructions. When the machine-executable instructions are called and executed by the processor, the machine-executable instructions cause the processor to implement the above obstacle recognition method.

[0102] The computer program product of the obstacle recognition method, device and electronic device provided by the embodiments of the present invention includes a computer-readable storage medium storing program code, and the instructions included in the program code can be used to execute the method described in the foregoing method embodiments. For the specific implementation, reference can be made to the method embodiments, which will not be elaborated here.

[0103] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems and devices described above can refer to the corresponding processes in the foregoing method embodiments, which will not be elaborated here.

[0104] In addition, in the description of the embodiments of the present invention, unless otherwise clearly specified and limited, the terms "installation", "connection", and "coupling" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the communication inside two components. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations.

[0105] If the above-mentioned function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs that can store program codes.

[0106] In the description of the present invention, it should be noted that the orientation or positional relationship indicated by the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. is based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation to the present invention. In addition, the terms "first", "second", and "third" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0107] Finally, it should be noted that the above-described embodiments are only specific embodiments of the present invention, used to illustrate the technical solutions of the present invention, rather than limiting it. The protection scope of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that any technician familiar with the technical field of the present invention can still modify the technical solutions recorded in the foregoing embodiments or easily think of changes, or perform equivalent replacements on some of the technical features; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. An obstacle recognition method, characterized in that, The method includes: Obtaining initial point cloud data of the vehicle surrounding environment; wherein, the initial point cloud data is a data set in a point cloud coordinate system; Determining ground noise points in the initial point cloud data based on a grid method, and performing filtering processing on the ground noise points to obtain first point cloud data; Converting the first point cloud data to obtain second point cloud data in an image coordinate system; Determining edge noise points in the second point cloud data according to a screening rule, and performing filtering processing on the edge noise points to obtain target point cloud data; wherein, the screening rule includes a dynamic search radius and an adjacent point included angle threshold; Clustering the target point cloud data based on a run-based clustering algorithm to obtain initial obstacle information; wherein, for the target point cloud data in the image coordinate system, a row of target point cloud data forms a run; the run-based clustering algorithm includes three key parameters: a lookup table, a current row local clustering cluster set, and an upper row local clustering cluster set; for each local clustering cluster set, it includes the start and end column indexes and the cluster index number of the set; first, for the target point cloud data of each row, cluster the point cloud within the run in the horizontal direction, calculate the dynamic search radius in the horizontal direction of this point according to the horizontal angular resolution of the lidar, and search whether the adjacent point is a neighborhood point by setting a horizontal adjacent point included angle threshold, calculate the included angle between valid point clouds within the neighborhood, if the included angle is greater than the horizontal adjacent point included angle threshold, then this adjacent point and this point belong to the same target set; after searching all row-wise local clustering cluster sets, initialize the size of the lookup table space to be the same as the number of clustering cluster sets, and initialize the information in the table to -1; the element index number of the lookup table represents the corresponding local clustering cluster set, and the element content represents the parent node of this local clustering cluster set, if the value is -1, it means its parent node is itself, that is, after horizontal clustering, there is no other cluster set merged with this local clustering cluster set; secondly, perform vertical traversal on the local clustering cluster sets, and judge whether the point cloud in the upper local clustering cluster set is within the neighborhood range of the elements in the current local clustering cluster along the three directions of upper left, up, and upper right by setting a vertical adjacent point included angle threshold, if the included angle in the vertical direction of the adjacent points within the neighborhood range meets the threshold condition, that is, the included angle is greater than the vertical adjacent point included angle threshold, then merge the two local clustering cluster sets, and select the smallest index value from the local clustering cluster sets to be fused as the common parent node, and assign it to the corresponding lookup table; Performing filtering processing on the initial obstacle information according to a preset filtering rule to obtain obstacle information.

2. The method according to claim 1, wherein The step of determining edge noise points in the second point cloud data according to the screening rule includes: For the current point in the second point cloud data, determining adjacent points of the current point according to the dynamic search radius; wherein, the distance between the adjacent point and the current point is not greater than the dynamic search radius; Calculating the adjacent point included angle between each adjacent point and the current point; Determining the edge noise points according to the adjacent point included angle and the adjacent point included angle threshold.

3. The method according to claim 2, characterized in that, The step of determining the edge noise points according to the adjacent point included angle and the adjacent point included angle threshold includes: Determine whether the included angle between the adjacent points is less than the adjacent point included angle threshold; If so, determine the adjacent point corresponding to the included angle between the adjacent points as the edge noise point.

4. The method according to claim 1, wherein The step of determining the ground noise points in the initial point cloud data based on the grid method further includes: Based on the grid method, determine the grid map corresponding to the initial point cloud data; wherein, the grid map includes a plurality of grids; Determine the target height value of each grid; wherein, the target height value is the minimum value of the heights of multiple points within the grid; Determine whether the target height value is not greater than the preset height threshold; If so, determine the grid corresponding to the target height value as the target grid, and use the multiple points within the target grid as the ground noise points.

5. The method according to claim 1, wherein The step of converting the first point cloud data to obtain the second point cloud data in the image coordinate system includes: Convert the first point cloud data according to the first conversion rule to obtain the point cloud data in the spherical coordinate system; wherein, the first conversion rule is used to represent the conversion of data from the point cloud coordinate system to the spherical coordinate system; Convert the point cloud data in the spherical coordinate system according to the second conversion rule to obtain the second point cloud data in the image coordinate system; wherein, the second conversion rule is used to represent the conversion of data from the spherical coordinate system to the image coordinate system.

6. The method according to claim 1, wherein A lidar is provided on the vehicle, and the step of obtaining the initial point cloud data of the vehicle surrounding environment includes: Obtain the initial point cloud data collected by the lidar; Determine whether there are pitch and / or rotation and / or yaw angles of the lidar relative to the vehicle body coordinate system; If so, perform attitude calibration on the initial point cloud data according to the preset attitude correction matrix to obtain the corrected initial point cloud data.

7. The method according to claim 1, characterized in that, A depth camera is provided on the vehicle, and the step of obtaining the initial point cloud data of the vehicle surrounding environment further includes: Obtain the depth image of the vehicle surrounding environment collected by the depth camera; Convert the depth image to obtain the corresponding initial point cloud data.

8. An obstacle recognition device, characterized in that, The device includes: An acquisition module, configured to acquire the initial point cloud data of the vehicle surrounding environment; wherein, the initial point cloud data is a data set in the point cloud coordinate system; A first determination module, configured to determine the ground noise points in the initial point cloud data based on the grid method, and perform filtering processing on the ground noise points to obtain the first point cloud data; A conversion module, configured to convert the first point cloud data to obtain the second point cloud data in the image coordinate system; A second determination module, configured to determine the edge noise points in the second point cloud data according to the screening rule, and perform filtering processing on the edge noise points to obtain the target point cloud data; wherein, the screening rule includes a dynamic search radius and an adjacent point included angle threshold; The clustering module is used to cluster the target point cloud data based on the run-based clustering algorithm to obtain the initial obstacle information. For the target point cloud data in the image coordinate system, a row of target point cloud data forms a run. The run-based clustering algorithm includes three key parameters: a lookup table, the current row's local clustering cluster set, and the previous row's local clustering cluster set. For each local clustering cluster set, it consists of the start and end column indices and the cluster index number of the set. First, for the target point cloud data of each row, the points in the run are clustered horizontally. According to the horizontal angular resolution of the lidar, the dynamic search radius in the horizontal direction of the point is calculated, and by setting the horizontal adjacent point angle threshold, it is searched whether the adjacent point is a neighborhood point, and the angle between the valid point clouds in the neighborhood is calculated. If the angle is greater than the horizontal adjacent point angle threshold, then the adjacent point and the point belong to the same target set. After searching all the local clustering cluster sets in the row direction, the size of the lookup table space is initialized to be the same as the number of clustering cluster sets, and the information in the table is initialized to -1. The element index number of the lookup table represents the corresponding local clustering cluster set, and the element content represents the parent node of the local clustering cluster set. If the value is -1, it means that its parent node is itself, that is, after horizontal clustering, there is no other cluster set merged with this local clustering cluster set. Secondly, the local clustering cluster sets are traversed vertically. By setting the vertical adjacent point angle threshold, it is judged whether the point clouds in the upper local clustering cluster set are within the neighborhood range of the elements in the current local clustering cluster along the three directions of upper left, up, and upper right. If the vertical angle of the adjacent points within the neighborhood range meets the threshold condition, that is, the angle is greater than the vertical adjacent point angle threshold, then the two local clustering cluster sets are merged, and the smallest index value is selected from the local clustering cluster sets to be fused as the common parent node and assigned to the corresponding lookup table. The filtering module is used to perform filtering processing on the initial obstacle information according to the preset filtering rules to obtain the obstacle information.

9. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method described in any one of claims 1-7 above.

10. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, and when the computer program is run by the processor, it executes the steps of the method described in any one of claims 1-7 above.

Images