Obstacle detection method and device, vehicle and storage medium

Abstract

The invention provides an obstacle detection method and device, a vehicle and a storage medium, and the method comprises the steps: obtaining first point cloud data and at least one bounding box; for any bounding box, determining at least one first point located in any bounding box from the first point cloud data; according to the speed of the at least one first point, the volume of any bounding box is expanded, and at least one second point located in any expanded bounding box is determined from the first point cloud data; and determining target point cloud data belonging to the same obstacle from the first point cloud data according to the second point in each expanded bounding box, thereby preventing the points in the first point cloud data from being outside the bounding box due to the position error and the bounding box time synchronization error, and further determining the target point cloud data belonging to the same obstacle according to the second point in each expanded bounding box. And the target point cloud data belonging to the same obstacle is detected from the first point cloud data, so that the accuracy of the detection result can be improved.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of automatic driving, and in particular to an obstacle detection method and device, a vehicle, and a storage medium. BACKGROUND

[0002] At present, with the development of automatic driving related technologies, automatic driving vehicles gradually enter people's field of vision. The automatic driving function of the automatic driving vehicle is of great significance for improving personal travel methods, promoting the upgrading of the automobile industry, and reducing traffic accidents.

[0003] In order to ensure the safety of the automatic driving vehicle during automatic driving, it is necessary to detect obstacles in a timely and effective manner. Therefore, how to accurately detect obstacles around the vehicle is very important. SUMMARY

[0004] The present disclosure aims to at least partially solve one of the technical problems in the related art.

[0005] To this end, the present disclosure provides an obstacle detection method and device, a vehicle, and a storage medium, which expand the volume of any bounding box by the speed of at least one first point in the first point cloud data located in the bounding box, and determine at least one second point in the expanded bounding box from the first point cloud data, so as to avoid the situation that the points in the first point cloud data are outside the bounding box due to position error and time synchronization error with the bounding box, thereby improving the accuracy of the detection result by detecting target point cloud data belonging to the same obstacle from the first point cloud data according to the second points in each expanded bounding box.

[0006] An embodiment of the present disclosure provides an obstacle detection method, which comprises: obtaining first point cloud data and at least one bounding box, wherein the bounding box is obtained by performing obstacle detection on second point cloud data, the first point cloud data is obtained by performing detection on a target region by a millimeter wave radar of a target vehicle at a target time, and the second point cloud data is obtained by performing detection on the target region by a laser radar of the target vehicle at the target time; determining at least one first point in any bounding box from the first point cloud data; expanding the volume of the any bounding box according to the speed of the at least one first point, and determining at least one second point in the expanded any bounding box from the first point cloud data; and determining target point cloud data belonging to the same obstacle from the first point cloud data according to the second points in each expanded bounding box.

[0007] Another aspect of the present disclosure provides an obstacle detection device, comprising: an obtaining module configured to obtain first point cloud data and at least one bounding box, wherein the bounding box is obtained by performing obstacle detection on second point cloud data, the first point cloud data is obtained by performing detection on a target region by a millimeter wave radar of a target vehicle at a target time, and the second point cloud data is obtained by performing detection on the target region by a laser radar of the target vehicle at the target time; a first determining module configured to determine at least one first point located in any bounding box from the first point cloud data; a processing module configured to expand the volume of the any bounding box according to the speed of the at least one first point, and determine at least one second point located in the expanded any bounding box from the first point cloud data; and a second determining module configured to determine target point cloud data belonging to the same obstacle from the first point cloud data according to the second points in each expanded bounding box.

[0008] Another aspect of the present disclosure provides a vehicle, comprising a processor and a memory for storing processor-executable instructions, wherein the processor is configured to implement the obstacle detection method according to any one of the preceding aspects.

[0009] Another aspect of the present disclosure provides a non-transitory computer-readable storage medium having stored thereon a computer program, wherein the computer program, when executed by a processor, implements the obstacle detection method according to any one of the preceding aspects.

[0010] Another aspect of the present disclosure provides a computer program product having stored thereon a computer program, wherein the computer program, when executed by a processor, implements the obstacle detection method according to any one of the preceding aspects.

[0011] The obstacle detection method provided by the present disclosure comprises the following steps: obtaining first point cloud data and at least one bounding box, wherein the bounding box is obtained by performing obstacle detection on second point cloud data, the first point cloud data is obtained by performing detection on a target region by a millimeter wave radar of a target vehicle at a target time, and the second point cloud data is obtained by performing detection on the target region by a laser radar of the target vehicle at the target time; for any bounding box, at least one first point located in the any bounding box is determined from the first point cloud data; the volume of the any bounding box is expanded according to the speed of the at least one first point, and at least one second point located in the expanded any bounding box is determined from the first point cloud data; and target point cloud data belonging to the same obstacle is determined from the first point cloud data according to the second points in each expanded bounding box. Thus, the volume of the any bounding box is expanded according to the speed of the at least one first point located in the any bounding box in the first point cloud data, and at least one second point located in the expanded any bounding box is determined from the first point cloud data, so that the points in the first point cloud data which are outside the bounding box due to position errors and time synchronization errors with the bounding box are avoided, and the accuracy of the detection result can be improved by detecting the target point cloud data belonging to the same obstacle from the first point cloud data according to the second points in each expanded bounding box.

[0012] Additional aspects and advantages of the present disclosure will be made apparent from the following description, which, taken together with the accompanying drawings, describes or illustrates such aspects and advantages by way of example as described below. BRIEF DESCRIPTION OF DRAWINGS

[0013] The above and / or additional aspects and advantages of the present disclosure will become apparent and be readily appreciated from the following description, taken in conjunction with the accompanying drawings, in which:

[0014] Figure 1 A flowchart of an obstacle detection method provided by an embodiment of the present disclosure;

[0015] Figure 2 A schematic diagram of point clouds in a primary bounding box and point clouds in an expanded bounding box provided by an embodiment of the present disclosure;

[0016] Figure 3 A flowchart of another obstacle detection method provided by an embodiment of the present disclosure;

[0017] Figure 4 A flowchart of another obstacle detection method provided by an embodiment of the present disclosure;

[0018] Figure 5 A flowchart of another obstacle detection method provided by an embodiment of the present disclosure;

[0019] Figure 6 A schematic diagram of an implementation principle of the obstacle detection method provided by the embodiments of the present disclosure is shown in the following figure.

[0020] Figure 7 A structural schematic diagram of an obstacle detection device provided by the embodiments of the present disclosure is shown in the following figure.

[0021] Figure 8 A block diagram of a vehicle provided by the embodiments of the present disclosure is shown in the following figure. DETAILED DESCRIPTION

[0022] The embodiments of the present disclosure are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar notations represent the same or similar elements or elements with the same or similar functions throughout. The embodiments described below by referring to the accompanying drawings are exemplary and are intended to explain the present disclosure, and cannot be understood as a limitation of the present disclosure.

[0023] At present, due to the existence of certain errors in the lateral position and height information of the millimeter wave point cloud, and the millimeter wave radar data and the target 3D bounding box (also known as the true value box) detected by the laser radar cannot be guaranteed to be time-synchronized (the frame rates may be different), so a part of the millimeter wave point cloud (such as 4D millimeter wave point cloud) may fall outside the bounding box; meanwhile, the millimeter wave point cloud also has some noise points, and these noise points may fall inside the bounding box, affecting the accuracy of the matching between the point cloud and the bounding box.

[0024] In view of the above problems, the present disclosure provides an obstacle detection method, device, vehicle and storage medium.

[0025] The obstacle detection method, device, vehicle and storage medium of the embodiments of the present disclosure are described below with reference to the accompanying drawings.

[0026] Figure 1 A flowchart of an obstacle detection method provided by the embodiments of the present disclosure is shown in the following figure.

[0027] The execution subject of the obstacle detection method of the embodiments of the present disclosure is an obstacle detection device, which can be arranged in a vehicle-mounted device, so that the vehicle-mounted device can perform the obstacle detection function.

[0028] As shown in the figure, Figure 1 the obstacle detection method can include the following steps:

[0029] Step 101, obtaining first point cloud data and at least one bounding box.

[0030] The bounding box is obtained by performing obstacle detection on the second point cloud data, the first point cloud data is obtained by detecting a target region by a millimeter wave radar of a target vehicle at a target time, and the second point cloud data is obtained by detecting the target region by a laser radar of the target vehicle at the target time.

[0031] As a possible implementation, the millimeter wave radar of the target vehicle detects the target region at the target time to obtain millimeter wave point cloud data, and the millimeter wave point cloud data is taken as the first point cloud data. The millimeter wave radar can be a four-dimensional (4D) millimeter wave radar or a three-dimensional (3D) millimeter wave radar. Meanwhile, the laser radar of the target vehicle detects the target region at the target time to obtain laser point cloud data, and the laser point cloud data is taken as the second point cloud data. The second point cloud data is subjected to obstacle detection to obtain at least one bounding box, and each bounding box corresponds to an obstacle.

[0032] In step 102, for any bounding box, at least one first point located in any bounding box is determined from the first point cloud data.

[0033] In order to accurately detect obstacles, the first point cloud data and the at least one bounding box can be comprehensively used for obstacle detection. As a possible implementation, the millimeter wave point cloud located in the bounding box can be determined first. As an example, for any bounding box, at least one first point located in any bounding box can be determined from the first point cloud data.

[0034] In step 103, the volume of any bounding box is expanded according to the speed of the at least one first point, and at least one second point located in the expanded any bounding box is determined from the first point cloud data.

[0035] It should be understood that, due to the possible errors in the lateral position and height information of the millimeter wave point cloud, and the time synchronization error between the millimeter wave point cloud and the bounding box, in order to avoid that the points in the first point cloud data are outside the bounding box due to the position error and the time synchronization error with the bounding box, the volume of any bounding box can be expanded, and at least one second point located in the expanded any bounding box can be determined from the first point cloud data. For example, as shown in Figure 2 , in the left original bounding box of Figure 2 , the right bounding box of Figure 2 is the expanded bounding box, the point cloud data in the original bounding box is the first point (such as Figure 2 the black point in the left original bounding box), and the point cloud in the expanded bounding box is the second point, which includes the black first point in the original bounding box and the gray point located in the expanded bounding box. Thus, by expanding the bounding box, the number of millimeter wave point clouds in the bounding box can be increased, and the enhancement of the millimeter wave point cloud data in the bounding box is realized.

[0036] In step 104, target point cloud data belonging to the same obstacle is determined from the first point cloud data according to the second points in each expanded bounding box.

[0037] To accurately achieve the detection of the obstacle, as an example, target point cloud data belonging to the same obstacle in the first point cloud data can be determined based on the enhanced millimeter wave point cloud data in the bounding box, i.e., according to the second points in each expanded bounding box, target point cloud data belonging to the same obstacle in the first point cloud data is determined.

[0038] To improve the accuracy of determining target point cloud data belonging to the same obstacle and avoid the influence of noise points, as an example, clustering can be performed based on the speed of each second point, and based on the clustering result, target point cloud data belonging to the same obstacle is determined.

[0039] Therefore, based on the target point cloud data belonging to the same obstacle, the attributes and positions of the obstacles can be more accurately identified.

[0040] In addition, it also needs to be explained that since the 4D millimeter wave radar has higher point cloud quantity and density and height features than the 3D millimeter wave radar, and has speed and intensity features compared with the laser radar, when the target point cloud data is 4D millimeter wave point cloud data, in response to the labeling operation, the first point cloud data can be labeled with obstacles (including labeling boxes and labeling categories) according to the target point cloud data, and the target detection model is trained according to the labeled first point cloud data, for example, the target detection model is used to perform regression prediction of obstacles on the labeled first point cloud data to obtain the position of at least one prediction box, the target detection model is used to perform category prediction of obstacles on the labeled first point cloud data to obtain the prediction category of the obstacle in the prediction box, and the target detection model is trained according to the difference between the position of the prediction box and the position of the labeling box, and according to the difference between the prediction category of the prediction box and the labeling category of the labeling box.

[0041] In summary, by obtaining first point cloud data and at least one bounding box, wherein the bounding box is obtained by performing obstacle detection on second point cloud data, the first point cloud data is obtained by performing detection on a target region by a millimeter wave radar of a target vehicle at a target time, and the second point cloud data is obtained by performing detection on the target region by a laser radar of the target vehicle at the target time; for any bounding box, at least one first point located in the any bounding box is determined from the first point cloud data; the volume of the any bounding box is expanded according to the speed of the at least one first point, and at least one second point located in the expanded any bounding box is determined from the first point cloud data; and target point cloud data belonging to the same obstacle is determined from the first point cloud data according to the second points in the expanded bounding boxes, thereby avoiding that the points in the first point cloud data are outside the bounding box due to position errors and time synchronization errors with the bounding box, so that the accuracy of the detection result can be improved by detecting the target point cloud data belonging to the same obstacle from the first point cloud data according to the second points in the expanded bounding boxes.

[0042] To clearly illustrate how the above embodiment determines the target point cloud data belonging to the same obstacle from the first point cloud data according to the second points in the expanded bounding boxes, the present disclosure proposes another obstacle detection method.

[0043] Figure 3 A flowchart of another obstacle detection method provided by the embodiment of the present disclosure.

[0044] As Figure 3 shown, the obstacle detection method can include the following steps:

[0045] Step 301, obtaining first point cloud data and at least one bounding box.

[0046] Wherein, the bounding box is obtained by performing obstacle detection on second point cloud data, the first point cloud data is obtained by performing detection on a target region by a millimeter wave radar of a target vehicle at a target time, and the second point cloud data is obtained by performing detection on the target region by a laser radar of the target vehicle at the target time.

[0047] Step 302, for any bounding box, at least one first point located in the any bounding box is determined from the first point cloud data.

[0048] Step 303, the volume of the any bounding box is expanded according to the speed of the at least one first point, and at least one second point located in the expanded any bounding box is determined from the first point cloud data.

[0049] Step 304, merging the second points in each expanded bounding box to obtain merged point cloud data.

[0050] In order to improve the accuracy and robustness of obstacle detection, as a possible implementation manner, the second points in each expanded bounding box can be merged to obtain merged point cloud data, that is, the second points in each expanded bounding box are integrated to obtain merged point cloud data.

[0051] Step 305, clustering the third points in the merged point cloud data according to the speed of the third points to obtain at least one cluster.

[0052] In order to avoid the interference of noise points in the millimeter wave point cloud data of the same obstacle and improve the processing efficiency of data, as a possible implementation manner, the third points in the merged point cloud data can be clustered according to the speed of the third points to obtain at least one cluster.

[0053] As an example, for any third point in the third points, the speed of the third point is feature extracted to obtain the speed feature of the third point; and the third points are clustered based on the distance between the speed features of the third points to obtain at least one cluster, wherein the distance between the speed feature of the third point in the same cluster and the cluster center point of the same cluster is less than a set threshold.

[0054] That is, the speed of each third point is feature extracted to obtain the speed feature (such as speed size, direction or acceleration, etc.) of each third point, and further, the third points can be clustered based on a clustering algorithm (such as K-means clustering algorithm, DBSCAN clustering algorithm (Density-Based Spatial Clustering of Applications with Noise), hierarchical clustering algorithm, etc.) and according to the distance between the speed features of the third points to obtain at least one cluster, and it should be noted that the distance between the speed feature of the third point in the same cluster and the cluster center point of the same cluster is less than a set threshold, and the cluster center point refers to the center of the cluster, that is, the most representative point in the cluster, which can make any object in the cluster closer to the cluster center point of the cluster than other cluster center points of the cluster. For example, the cluster center point of the cluster can be the average value or center point of all points in the cluster, etc.

[0055] Step 306, taking each point in any cluster as target point cloud data in the first point cloud data which is determined to belong to the same obstacle.

[0056] Further, each point in each cluster, i.e., the enhanced millimeter wave point cloud data matched with the corresponding bounding box and the target point cloud data of the obstacle framed by the corresponding bounding box, wherein the bounding box corresponding to each point in each cluster is the bounding box closest to each point.

[0057] It should be noted that the execution process of steps 301 to 303 can be implemented by any one of the embodiments of the present disclosure, and the present disclosure does not limit this, and will not be repeated here.

[0058] In summary, by merging the second points in each expanded bounding box to obtain a plurality of third points, clustering the plurality of third points according to the speed of the plurality of third points to obtain at least one cluster, and taking each point in any cluster as target point cloud data of the same obstacle in the first point cloud data, the different speed obstacles can be distinguished, the noise points in the bounding box can be removed, the target point cloud data of the same obstacle in the first point cloud data can be accurately determined, and the obstacle detection based on the target point cloud data of the same obstacle can be accurately performed.

[0059] In order to clearly illustrate how the above-mentioned embodiment expands the volume of any bounding box according to the speed of at least one first point, the present disclosure proposes another obstacle detection method.

[0060] Figure 4 The flowchart of another obstacle detection method provided by the embodiments of the present disclosure.

[0061] As shown in the figure, the obstacle detection method can include the following steps: Figure 4

[0062] Step 401, obtaining first point cloud data and at least one bounding box.

[0063] Wherein, the bounding box is obtained by performing obstacle detection on the second point cloud data, the first point cloud data is obtained by the target vehicle's millimeter wave radar detecting the target region at the target time, and the second point cloud data is obtained by the target vehicle's laser radar detecting the target region at the target time.

[0064] Step 402, for any bounding box, determining at least one first point located in any bounding box from the first point cloud data.

[0065] Step 403, in response to the speed of the at least one first point being greater than a set speed threshold, expanding the length and width of any bounding box according to a corresponding first expansion ratio, and expanding the height of any bounding box according to a first set length.

[0066] ​It should be understood that, since the length-width difference of different obstacles (such as vehicles) can be large, in order to avoid that the points in the first point cloud data are outside the bounding box due to position error and time synchronization error with the bounding box, the length, width and height of any bounding box can be respectively enlarged based on the speed of at least one first point in the bounding box.

[0067] As an example, in response to the speed of the at least one first point being greater than a set speed threshold, the length and width of any bounding box are enlarged according to a corresponding first enlargement ratio, and the height of any bounding box is enlarged according to a first set length.

[0068] For example, when the speed of the at least one first point in any bounding box is greater than 0, the length and width of the bounding box can be enlarged to 1.5 times the original length and width, and the height of the bounding box can be enlarged by 2 meters, with the center point of the bounding box as the reference.

[0069] Step 404, in response to the speed of the at least one first point being less than or equal to a set speed threshold, the length and width of any bounding box are enlarged according to a second set length, and the height of any bounding box is enlarged according to a third set length.

[0070] As another example, when the speed of the at least one first point is less than or equal to a set speed threshold, in order to avoid introducing too many millimeter wave point clouds unrelated to the obstacles in the bounding box, the length and width of any bounding box are enlarged according to a second set length, and the height of any bounding box is enlarged according to a third set length.

[0071] It should be noted that the enlargement range of the length, width and height of the bounding box when the speed of the at least one first point is greater than the set speed threshold is greater than the enlargement range of the length, width and height of the bounding box when the speed of the at least one first point is less than or equal to the set speed threshold.

[0072] For example, when the speed of the at least one first point is equal to 0, the length and width of any bounding box are enlarged by 0.1 meters, and the height of any bounding box is enlarged by 1 meter.

[0073] Step 405, according to the speed of the at least one first point, the volume of any bounding box is enlarged, and at least one second point located in the enlarged any bounding box is determined from the first point cloud data.

[0074] Step 406, according to the second points in each enlarged bounding box, target point cloud data belonging to the same obstacle is determined from the first point cloud data.

[0075] It should be noted that the execution processes of steps 401 to 402 and steps 405 to 406 can be respectively implemented in any of the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this and will not be repeated.

[0076] In summary, by responding to the speed of at least one first point being greater than a set speed threshold, the length and width of any bounding box are expanded according to a corresponding first expansion ratio, and the height of any bounding box is expanded to a first set length; in response to the speed of at least one first point being less than or equal to a set speed threshold, the length and width of any bounding box are expanded to a second set length, and the height of any bounding box is expanded to a third set length. Thus, based on the speed of at least one first point in any bounding box, any bounding box is expanded in a targeted manner, which can avoid points in the first point cloud data being outside the bounding box due to position errors and time synchronization errors with the bounding box, while also avoiding the introduction of too many irrelevant millimeter wave point clouds in each bounding box, reducing the amount of obstacle detection calculation while improving the accuracy of obstacle detection.

[0077] To clearly illustrate how the above-mentioned embodiments determine at least one first point located in any bounding box from the first point cloud data, the present disclosure proposes another obstacle detection method.

[0078] Figure 5 A flowchart of another obstacle detection method provided by the embodiments of the present disclosure.

[0079] As shown in Figure 5 , the obstacle detection method can include the following steps:

[0080] Step 501, obtaining first point cloud data and at least one bounding box.

[0081] Wherein, the bounding box is obtained by performing obstacle detection on the second point cloud data, the first point cloud data is obtained by detecting the target region at the target time by the millimeter wave radar of the target vehicle, and the second point cloud data is obtained by detecting the target region at the target time by the laser radar of the target vehicle.

[0082] Step 502, according to the mapping relationship between the laser radar coordinate system and the vehicle coordinate system in which the target vehicle is located, projecting any bounding box into the vehicle coordinate system to obtain a first mapping box corresponding to any bounding box.

[0083] In order to determine the points in the bounding box from the first point cloud data, the first point cloud data and at least one bounding box can be converted into the same coordinate for comparison.

[0084] As an example, according to the mapping relationship between the laser radar coordinate system and the vehicle coordinate system in which the target vehicle is located, any bounding box can be projected into the vehicle coordinate system to obtain a first mapping box corresponding to any bounding box in the vehicle coordinate system.

[0085] Step 503 : According to the mapping relationship between the polar coordinate system and the vehicle coordinate system, each point in the first point cloud data is converted to the vehicle coordinate system to obtain a first mapping point corresponding to each point in the first point cloud data.

[0086] Similarly, according to the mapping relationship between the polar coordinate system (millimeter wave radar coordinate system) and the vehicle coordinate system, each point in the first point cloud data can be converted to the vehicle coordinate system, so that the first mapping point corresponding to each point in the first point cloud data can be obtained.

[0087] Step 504: Determine at least one second mapping point located in the first mapping frame from each first mapping point.

[0088] As an example, at least one second mapping point located in the first mapping frame may be determined from the first mapping points according to the coordinate positions of the first mapping points and the coordinate position of any first mapping frame.

[0089] Step 505 : taking a point corresponding to at least one second mapping point in the first point cloud data as at least one first point located within any bounding box.

[0090] Furthermore, a point in the first point cloud data corresponding to at least one second mapping point is obtained. The point in the first point cloud data corresponding to at least one second mapping point is the millimeter wave point cloud data matched with the corresponding bounding box, that is, the point in the first point cloud data corresponding to at least one second mapping point is taken as at least one first point located in any bounding box.

[0091] Step 506 : Expand the volume of any bounding box according to the velocity of the at least one first point, and determine at least one second point located in any expanded bounding box from the first point cloud data.

[0092] The step of determining at least one second point within any enlarged bounding box from the first point cloud data may be as follows: projecting any enlarged bounding box onto the vehicle coordinate system based on a mapping relationship between the lidar coordinate system and the vehicle coordinate system to obtain a second mapping box corresponding to any enlarged bounding box; determining at least one third mapping point within the second mapping box from each first mapping point; and using the point corresponding to the at least one third mapping point in the first point cloud data as the at least one second point within any enlarged bounding box. The specific implementation is described in steps 502 to 505, which will not be further described in this disclosure.

[0093] Step 507 : determining target point cloud data belonging to the same obstacle from the first point cloud data according to the second points within each expanded bounding box.

[0094] It should be noted that the execution processes of steps 501 and 507 can be implemented in any of the embodiments of the present disclosure, and the present disclosure does not limit this, and will not be repeated here.

[0095] In summary, by projecting any bounding box into the vehicle coordinate system according to the mapping relationship between the laser radar coordinate system and the vehicle coordinate system in which the target vehicle is located, a first mapping box corresponding to any bounding box is obtained; each point in the first point cloud data is converted into the vehicle coordinate system according to the mapping relationship between the polar coordinate system and the vehicle coordinate system, to obtain a first mapping point corresponding to each point in the first point cloud data; at least one second mapping point located in the first mapping box is determined from the first mapping points; the point in the first point cloud data corresponding to the at least one second mapping point is taken as at least one first point located in any bounding box. Thus, by projecting the first point cloud data and any bounding box into the vehicle coordinate system, at least one first point located in any bounding box can be accurately determined from the first point cloud data.

[0096] Based on any of the above embodiments, taking the first point cloud data as a 4D millimeter wave point cloud and the bounding box as a 3D bounding box as an example, the implementation process of the present disclosure can be as shown in Figure 6 , mainly including the following steps:

[0097] 1. Match the 4D millimeter wave point cloud with the original ground truth box (bounding box), and the point cloud falling into the box is matched;

[0098] 2. Determine the dynamic and static of the current ground truth box according to the speed of the matched millimeter wave point cloud; if it is dynamic, the length and width of the ground truth box are expanded to 1.5 times the original, and the height is expanded by 2 meters; if it is static, the length and width of the ground truth box are expanded by 0.1 meters, and the height is expanded by 1 meter;

[0099] 3. Match the 4D millimeter wave point cloud with the expanded ground truth box, if a point falls into multiple expanded ground truth boxes at the same time, the point is assigned to the box closest to the center point of the ground truth box;

[0100] 4. Merge the points in the multiple expanded ground truth boxes to obtain merged point cloud data;

[0101] 5. Use the DBSCAN (Density-based spatial clustering of applications with noise) algorithm on the merged point cloud, and cluster according to the speed characteristics of the point cloud, and the obtained point cloud clustering result is the enhanced point cloud result matched with the ground truth box, that is, each point in any cluster is taken as target point cloud data belonging to the same obstacle.

[0102] In order to implement the above embodiment, the present disclosure further provides an obstacle detection device.

[0103] Figure 7 A schematic structural diagram of an obstacle detection device provided in an embodiment of the present disclosure.

[0104] like Figure 7 As shown, the obstacle detection device 700 includes: an acquisition module 710 , a first determination module 720 , a processing module 730 and a second determination module 740 .

[0105] Among them, the acquisition module 710 is used to obtain first point cloud data and at least one bounding box, wherein the bounding box is obtained by obstacle detection on the second point cloud data, the first point cloud data is obtained by the millimeter wave radar of the target vehicle detecting the target area at the target time, and the second point cloud data is obtained by the laser radar of the target vehicle detecting the target area at the target time; the first determination module 720 is used to determine, for any bounding box, at least one first point located in any bounding box from the first point cloud data; the processing module 730 is used to expand the volume of any bounding box according to the speed of at least one first point, and determine at least one second point located in any expanded bounding box from the first point cloud data; the second determination module 740 is used to determine the target point cloud data belonging to the same obstacle from the first point cloud data based on the second point in each expanded bounding box.

[0106] As a possible implementation method of an embodiment of the present disclosure, the second determination module 740 is used to merge the second points within each expanded bounding box to obtain merged point cloud data; cluster the multiple third points according to the speed of the multiple third points in the merged point cloud data to obtain at least one cluster; and use each point in any cluster as the target point cloud data determined to belong to the same obstacle in the first point cloud data.

[0107] As a possible implementation of an embodiment of the present disclosure, the second determination module 740 is configured to perform feature extraction on the velocity of any third point among multiple third points to obtain a velocity feature of any third point; and cluster the multiple third points based on the distance between the velocity features of the multiple third points to obtain at least one cluster, wherein the distance between the velocity feature of the third points in the same cluster and the cluster center point of the same cluster is less than a set threshold.

[0108] As a possible implementation manner of the embodiment of the present disclosure, the processing module 730 is configured to, in response to the speed of the at least one first point being greater than a set speed threshold, expand the length and width of any bounding box by a corresponding first expansion ratio, and expand the height of any bounding box by a first set length; and in response to the speed of the at least one first point being less than or equal to the set speed threshold, expand the length and width of any bounding box by a second set length, and expand the height of any bounding box by a third set length.

[0109] As a possible implementation manner of the embodiment of the present disclosure, the first determining module 720 is configured to project any bounding box into a vehicle coordinate system according to a mapping relationship between a laser radar coordinate system and the vehicle coordinate system in which the target vehicle is located, to obtain a first mapping box corresponding to any bounding box; convert each point in the first point cloud data into the vehicle coordinate system according to a mapping relationship between a polar coordinate system and the vehicle coordinate system, to obtain a first mapping point corresponding to each point in the first point cloud data; determine at least one second mapping point located in the first mapping box from the first mapping points; and take a point corresponding to the at least one second mapping point in the first point cloud data as at least one first point located in any bounding box.

[0110] As a possible implementation manner of the embodiment of the present disclosure, the processing module 730 is configured to project the expanded any bounding box into the vehicle coordinate system according to a mapping relationship between the laser radar coordinate system and the vehicle coordinate system, to obtain a second mapping box corresponding to the expanded any bounding box; determine at least one third mapping point located in the second mapping box from the first mapping points; and take a point corresponding to the at least one third mapping point in the first point cloud data as at least one second point located in the expanded any bounding box.

[0111] As a possible implementation manner of the embodiment of the present disclosure, the obstacle detection apparatus 700 further includes a labeling module and a training module.

[0112] The labeling module is configured to perform obstacle labeling on the first point cloud data according to the target point cloud data; and the training module is configured to train the target detection model according to the labeled first point cloud data.

[0113] The obstacle detection device of the embodiments of the present disclosure, by obtaining the first point cloud data and at least one bounding box, wherein the bounding box is obtained by performing obstacle detection on the second point cloud data, the first point cloud data is obtained by the millimeter wave radar of the target vehicle detecting the target region at the target time, and the second point cloud data is obtained by the laser radar of the target vehicle detecting the target region at the target time; for any bounding box, at least one first point located in any bounding box is determined from the first point cloud data; the volume of any bounding box is expanded according to the speed of the at least one first point, and at least one second point located in the expanded any bounding box is determined from the first point cloud data; and target point cloud data belonging to the same obstacle is determined from the first point cloud data according to the second points in each expanded bounding box, thereby, the volume of any bounding box is expanded according to the speed of the at least one first point located in any bounding box in the first point cloud data, and at least one second point located in the expanded any bounding box is determined from the first point cloud data, which avoids that the points in the first point cloud data are outside the bounding box due to position error and time synchronization error with the bounding box, so that the target point cloud data belonging to the same obstacle is detected from the first point cloud data according to the second points in each expanded bounding box, which can improve the accuracy of the detection result.

[0114] To achieve the above-mentioned embodiments, the present disclosure further proposes a vehicle, comprising a processor and a memory for storing processor-executable instructions, wherein the processor is configured to implement the obstacle detection method as described in the foregoing method embodiments.

[0115] To achieve the above-mentioned embodiments, the present disclosure further proposes a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the obstacle detection method as described in the foregoing method embodiments.

[0116] To achieve the above-mentioned embodiments, the present disclosure further proposes a computer program product having a computer program stored thereon, which, when executed by a processor, implements the obstacle detection method as described in the foregoing method embodiments.

[0117] Figure 8 is a block diagram of a vehicle 800 according to an example embodiment. For example, the vehicle 800 can be a hybrid vehicle, or a non-hybrid vehicle, an electric vehicle, a fuel cell vehicle, or other type of vehicle. The vehicle 800 can be an autonomous vehicle, a semi-autonomous vehicle, or a non-autonomous vehicle.

[0118] Referring to Figure 8In some embodiments, the vehicle 800 can include various subsystems, such as an infotainment system 810, a perception system 820, a decision control system 830, a drive system 840, and a computing platform 850. The vehicle 800 can include more or fewer subsystems, and each subsystem can include multiple components. In addition, each subsystem and each component of the vehicle 800 can be interconnected by wired or wireless means.

[0119] In some embodiments, the infotainment system 810 can include a communication system, an entertainment system, a navigation system, and the like.

[0120] The perception system 820 can include several sensors for sensing information of the environment surrounding the vehicle 800. For example, the perception system 820 can include a global positioning system (which can be a GPS system, a Beidou system, or other positioning system), an inertial measurement unit (IMU), a laser radar, a millimeter wave radar, an ultrasonic radar, and a camera.

[0121] The decision control system 830 can include a computing system, a vehicle controller, a steering system, a throttle, and a braking system.

[0122] The drive system 840 can include components that provide power motion for the vehicle 800. In one embodiment, the drive system 840 can include an engine, an energy source, a transmission system, and wheels. The engine can be one or a combination of an internal combustion engine, an electric motor, an air compression engine, or the like. The engine can convert energy provided by the energy source into mechanical energy.

[0123] Some or all functions of the vehicle 800 are controlled by the computing platform 850. The computing platform 850 can include at least one processor 851 and a memory 852, and the processor 851 can execute instructions 853 stored in the memory 852.

[0124] The processor 851 can be any conventional processor, such as commercially available CPUs. The processor can also include a Graphic Process Unit (GPU), a Field Programmable Gate Array (FPGA), a System on Chip (SOC), an Application Specific Integrated Circuit (ASIC), or a combination thereof.

[0125] Memory 852 can be implemented by any type of volatile or nonvolatile storage devices or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable programmable read only memory (EPROM), programmable read only memory (PROM), read only memory (ROM), magnetic storage, flash memory, a magnetic disk or a compact disk.

[0126] In addition to instructions 853, memory 852 can store data, such as road maps, route information, vehicle's position, direction, speed, etc. The data stored by memory 852 can be used by computing platform 850.

[0127] In embodiments of the present disclosure, processor 851 can execute instructions 853 to complete all or part of the steps in the above-described method embodiments.

[0128] In the description of the present specification, the description of the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present disclosure. In the present specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any appropriate manner in any one or more embodiments or examples. In addition, the person skilled in the art can combine and combine the different embodiments or examples described in the present specification and the features of the different embodiments or examples, without contradiction.

[0129] In addition, the terms "first", "second" are only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include at least one of the features. In the description of the present disclosure, the meaning of "a plurality of" is at least two, such as two, three, etc., unless otherwise specifically limited.

[0130] Any process or method descriptions or any other descriptions in the flow charts or otherwise described herein can be understood as representing code modules, segments, or portions of code that include one or more executable instructions for implementing specific logic functions (or steps) in the process, and the various embodiments of the present disclosure include additional implementations in which the order of steps can differ from those shown or discussed, including a step can occur at other times, including as recited in the description, and additional functions can be added or performed at a similar time. It is understood that the scope of the present disclosure encompasses all such embodiments.

[0131] The logic and / or steps represented in flow diagrams or otherwise described herein, for example, can be considered as a sequence of instructions to implement logic functions, and can be embodied in any computer-readable medium for use by an instruction execution system, apparatus, or device, such as a computer-based system, processor- containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. In the context of this specification, a "computer-readable medium" can be any means that can contain, store, communicate, propagate or transport the program for use by or in connection with the instruction execution system, apparatus, or device. The computer-readable medium can be a machine-readable storage device (e.g., magnetic, optical or other) a machine-readable storage diskette (e.g., floppy disk, optical disk, CD- ROM, etc.), a machine- readable storage card (e.g., PCMCIA card, etc.), a machine-readable storage tape (e.g., magnetic tape, optical tape, etc.), a machine-readable storage medium (e.g., RAM, ROM, etc.), a machine-readable signal (e.g., electrical, optical, etc.), a machine-readable medium (e.g., carrier wave, etc.) or any other suitable medium or means of embodying the program. More specific examples (a non-exhaustive list) of the computer-readable medium include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a RAM, a ROM, an EPROM, a FLASH memory card, an optical fiber, and a portable compact disc read-only memory (CD-ROM). Additionally, the computer-readable medium can be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, for example, via optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and stored in a computer memory.

[0132] It should be understood that portions of the present disclosure can be implemented in hardware, software, firmware or combinations thereof. In the above embodiments, the various steps or methods can be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. As such, if implemented in hardware, and in another embodiment, any of the following technologies, known in the art, or their combinations can be used: discrete logic circuitry having logic gates for implementing logic functions on data signals, application specific integrated circuits having appropriate combinational logic gates, programmable gate arrays (PGA), field programmable gate arrays (FPGA), and the like.

[0133] Those skilled in the art can understand that all or part of the steps carried out by the above-mentioned embodiment method can be completed by a program instructing the relevant hardware, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one of the steps of the method embodiment or a combination thereof.

[0134] In addition, each functional unit in each embodiment of the present disclosure can be integrated in one processing module, or each unit can exist physically separately, or two or more units can be integrated in one module. The integrated module can be realized in the form of hardware or in the form of a software functional module. When the integrated module is realized in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer readable storage medium.

[0135] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present disclosure have been shown and described above, it should be understood that the above embodiments are exemplary and should not be construed as limiting the present disclosure, and those skilled in the art can make changes, modifications, replacements and variations to the above embodiments within the scope of the present disclosure.

Claims

1. An obstacle detection method characterized by, The method comprises: acquiring first point cloud data and at least one bounding box, wherein the bounding box is obtained by performing obstacle detection on second point cloud data, the first point cloud data is obtained by performing detection on a target region by a millimeter wave radar of a target vehicle at a target time, and the second point cloud data is obtained by performing detection on the target region by a laser radar of the target vehicle at the target time; for any bounding box, determining at least one first point located in the any bounding box from the first point cloud data; enlarging a volume of the any bounding box according to a speed of the at least one first point, and determining at least one second point located in the enlarged any bounding box from the first point cloud data; determining target point cloud data belonging to the same obstacle from the first point cloud data according to the second points in each enlarged bounding box.

2. The method of claim 1, wherein, The method of determining target point cloud data belonging to the same obstacle from the first point cloud data according to the second points in each enlarged bounding box comprises: merging the second points in each enlarged bounding box to obtain merged point cloud data; performing clustering on a plurality of third points in the merged point cloud data according to speeds of the plurality of third points to obtain at least one cluster; regarding each point in any cluster as target point cloud data belonging to the same obstacle in the first point cloud data.

3. The method of claim 2, wherein, The method of performing clustering on a plurality of third points in the merged point cloud data according to speeds of the plurality of third points to obtain at least one cluster comprises: performing feature extraction on a speed of any third point in the plurality of third points to obtain a speed feature of the any third point; performing clustering on the plurality of third points based on distances between speed features of the plurality of third points to obtain at least one cluster, wherein a distance between a speed feature of a third point in a same cluster and a cluster center point of the same cluster is less than a set threshold.

4. The method of claim 1, wherein, The method of enlarging a volume of the any bounding box according to a speed of the at least one first point comprises: in response to the speed of the at least one first point being greater than a set speed threshold, enlarging a length and a width of the any bounding box according to a corresponding first enlargement ratio, and enlarging a height of the any bounding box according to a first set length; in response to the speed of the at least one first point being less than or equal to the set speed threshold, enlarging the length and the width of the any bounding box according to a second set length, and enlarging the height of the any bounding box according to a third set length.

5. The method of claim 1, wherein, The method of determining at least one first point located in the any bounding box from the first point cloud data for any bounding box comprises: projecting the any bounding box to a vehicle coordinate system of the target vehicle to obtain a first mapping box corresponding to the any bounding box according to a mapping relationship between a laser radar coordinate system and the vehicle coordinate system; converting each point in the first point cloud data to a first mapping point in the vehicle coordinate system according to a mapping relationship between a polar coordinate system and the vehicle coordinate system. determine, from each of the first mapping points, at least one second mapping point located in the first mapping box; take, as at least one first point located in the any surrounding box, a point in the first point cloud data corresponding to the at least one second mapping point.

6. The method according to claim 5, characterized in that The determining, from the first point cloud data, at least one second point located in the any surrounding box after expansion includes: projecting, according to a mapping relationship between the laser radar coordinate system and the vehicle coordinate system, the any surrounding box after expansion to the vehicle coordinate system to obtain a second mapping box corresponding to the any surrounding box after expansion; determining, from each of the first mapping points, at least one third mapping point located in the second mapping box; taking, as at least one second point located in the any surrounding box after expansion, a point in the first point cloud data corresponding to the at least one third mapping point.

7. The method according to any one of claims 1 to 6, characterized in that, The method further includes: performing obstacle labeling on the first point cloud data according to the target point cloud data; training a target detection model according to the labeled first point cloud data.

8. An obstacle detection device characterized by comprising: include: an acquisition module configured to acquire first point cloud data and at least one surrounding box, wherein the surrounding box is obtained by performing obstacle detection on second point cloud data, the first point cloud data is obtained by performing detection on a target region at a target time by a millimeter wave radar of a target vehicle, and the second point cloud data is obtained by performing detection on the target region at the target time by a laser radar of the target vehicle; a first determination module configured to determine, for any surrounding box, at least one first point located in the any surrounding box from the first point cloud data; a processing module configured to expand a volume of the any surrounding box according to a speed of the at least one first point, and determine at least one second point located in the any surrounding box after expansion from the first point cloud data; a second determination module configured to determine, from the first point cloud data, target point cloud data belonging to a same obstacle according to second points in each surrounding box after expansion.

9. A vehicle characterized by comprising: include: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to: when executing the processor-executable instructions, implement the method of any one of claims 1-7.

10. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by the processor, implements the method of any one of claims 1-7. The computer program, when executed by the processor, implements the method of any one of claims 1-7.

Images