Obstacle detection method and system based on ultrasonic point cloud

Abstract

The invention relates to an obstacle detection method and system based on ultrasonic point cloud. The method includes: acquiring an ultrasonic point cloud via a vehicle-mounted ultrasonic sensor, each ultrasonic point in the ultrasonic point cloud having at least one of an associated intensity of an ultrasonic echo and / or a width of the ultrasonic echo and associated two-dimensional coordinate information; rasterizing the ultrasonic point cloud; generating a dummy picture based on the rasterized ultrasound point cloud; and inputting the pseudo picture into a neural network model to identify at least one of the following: the type of the obstacle, the height of the obstacle, the contour line of the obstacle, and the contour line endpoint of the obstacle.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to obstacle detection, and in particular to an obstacle detection method and system based on ultrasonic point cloud. BACKGROUND

[0002] Obstacle detection is very important for various tasks, such as assisted driving, automatic parking, etc. Currently, multiple ultrasonic sensors are usually mounted on a vehicle to determine the position of an obstacle target by using triangulation.

[0003] However, after generating an ultrasonic point cloud by an ultrasonic sensor, a traditional clustering and line segment fitting method is generally used to generate an obstacle target for obstacle detection. However, due to the fact that an obstacle surface is usually uneven, the requirement for distance accuracy of triangulation is high, and the ultrasonic point cloud is often missing or deviated, the generated obstacle contour line usually has a relatively large error.

[0004] The present disclosure is improved in view of the above factors, but is not limited thereto. SUMMARY

[0005] To this end, the present disclosure proposes an obstacle detection method and system based on ultrasonic point cloud. The method of the present disclosure uses a deep learning algorithm to make a more accurate regression on the endpoints or contour lines of an obstacle, thereby improving the detection accuracy of an ultrasonic sensor for an obstacle in a parking task. In addition, in addition to using the coordinate information of the ultrasonic point cloud, the method of the present disclosure also uses the echo information (such as echo intensity, echo width, etc.) of the two detections forming the ultrasonic points as the input of the neural network, which can not only obtain more accurate endpoint or contour line output, but also be used to infer the height of the obstacle and the type of the obstacle, which is also a very important output in various tasks (such as parking tasks). In this way, the method and system of the present disclosure can use multiple dimensions of information of the ultrasonic point cloud as input, avoid or simplify manual feature extraction, learn more fine features from more bottom and richer data through end-to-end processing, thereby improving the accuracy and precision of obstacle recognition.

[0006] According to a first aspect of the present disclosure, an obstacle detection method based on ultrasonic point cloud is provided, comprising: collecting an ultrasonic point cloud via a vehicle-mounted ultrasonic sensor, each ultrasonic point in the ultrasonic point cloud having at least one of an intensity of an associated ultrasonic echo and a width of the ultrasonic echo and associated two-dimensional coordinate information; rasterizing the ultrasonic point cloud; generating a pseudo picture based on the rasterized ultrasonic point cloud; and inputting the pseudo picture into a neural network model to identify at least one of a type of an obstacle, a height of the obstacle, a contour line of the obstacle, and an endpoint of the contour line of the obstacle.

[0007] According to an embodiment, the size of the grid depends on the accuracy required for autonomous driving.

[0008] According to another embodiment, the pseudo picture comprises a plurality of channels, each channel being a respective different information of the ultrasound points in each grid.

[0009] According to yet another embodiment, a first channel of the plurality of channels is a number of the ultrasound points in each grid, a second channel is first dimension coordinate information of two-dimension coordinate information of the ultrasound points in each grid, a third channel is second dimension coordinate information of the two-dimension coordinate information of the ultrasound points in each grid, a fourth channel is intensity of ultrasound echoes of the ultrasound points in each grid, and a fifth channel is width of the ultrasound echoes of the ultrasound points in each grid.

[0010] According to yet another embodiment, in a case where a plurality of ultrasound points are included in a grid, coordinate information, intensity of ultrasound echoes, width of ultrasound echoes of the ultrasound points in the grid are average values or minimum values or maximum values of coordinate information, intensity of ultrasound echoes, width of ultrasound echoes of the plurality of ultrasound points.

[0011] According to yet another embodiment, the neural network model is trained using labeled ultrasound point cloud data, the labeled ultrasound point cloud data comprising labels of at least one of contour lines, contour line endpoints, heights, types of obstacles.

[0012] According to yet another embodiment, the labeled ultrasound point cloud data is further augmented by rotating, flipping or randomly deleting the ultrasound point cloud.

[0013] According to yet another embodiment, the method further comprises clustering the ultrasound point cloud before gridding the ultrasound point cloud, and performing the subsequent steps of the method respectively for ultrasound point cloud parts belonging to the same cluster.

[0014] According to yet another embodiment, the method further comprises performing piecewise line fitting respectively for ultrasound point cloud parts belonging to the same cluster to obtain initial contour lines of the respective obstacles, inputting respectively the ultrasound point cloud parts belonging to the same cluster into the neural network model to identify contour line endpoints of the respective obstacles, and using the identified contour line endpoints of the respective obstacles by the neural network model to correct the obtained initial contour lines of the respective obstacles to obtain corrected contour lines of the respective obstacles.

[0015] According to yet another embodiment, the method further comprises performing autonomous driving based on the identified types of obstacles, obstacle heights, contour lines of obstacles, contour line endpoints of obstacles.

[0016] According to a second aspect of the disclosure, there is provided an ultrasonic point cloud based obstacle detection system comprising: an ultrasonic sensor arranged on a vehicle, wherein the ultrasonic sensor is configured to collect an ultrasonic point cloud; and a computing device configured to receive the ultrasonic point cloud from the ultrasonic sensor, wherein the computing device is further configured to perform the method according to the first aspect of the disclosure.

[0017] According to an embodiment, the computing device comprises an on-board computer and / or a cloud, and wherein in case the computing device comprises a cloud, the system further comprises a communication device for communicating with the cloud.

[0018] According to a third aspect of the disclosure, there is provided a vehicle comprising a system according to the second aspect of the disclosure.

[0019] Aspects generally include methods, apparatus, systems, computer program products, and processing systems, as substantially described herein with reference to and as illustrated by the accompanying drawings.

[0020] The foregoing has outlined rather broadly the features and technical advantages of examples according to the disclosure in order that the detailed description can be better understood. Additional features and advantages will be described hereinafter. The disclosed conception and specific examples can be readily utilized as bases for modifying or designing other structures for carrying out the same purposes of the present disclosure. Such equivalent constructions do not depart from the scope of the appended claims. The characteristics of the concepts disclosed herein, both their organization and method of operation, together with associated advantages will be better understood from the following description when considered in connection with the accompanying drawings. Each of the figures is provided for the purpose of illustration and description, and is not intended as a definition of the limits of the claims. BRIEF DESCRIPTION OF DRAWINGS

[0021] So that the above-recited features and advantages of the above-stated aspects of the present disclosure can be understood in detail, a more particular description, briefly summarized above, can be had by reference to various aspects, some of which are illustrated in the appended drawings. It is to be noted, however, that the appended drawings illustrate only certain typical aspects of this disclosure and are therefore not to be considered limiting of its scope, for the description can admit to other equally effective aspects. The same reference numbers in different drawings can identify the same or similar elements.

[0022] Figure 1 A schematic illustration of an ultrasonic detection for a vehicle is shown;

[0023] Figure 2 A schematic illustration of a typical parking sweep a parking space procedure is shown;

[0024] Figure 3 A schematic illustration of a typical sweep a parking space width case is shown;

[0025] Figure 4A flowchart of an ultrasonic point cloud based obstacle detection method according to an embodiment is shown;

[0026] Figure 5 A schematic diagram of an ultrasonic point cloud and a first channel of a resulting pseudo picture after gridding according to an example embodiment is shown;

[0027] Figure 6 A comparison diagram of detection results of end points of a vehicle head contour line according to an example embodiment of the present disclosure and detection results of end points of a vehicle head contour line by a conventional multi-line segment fitting algorithm is shown;

[0028] Figure 7 A schematic block diagram of an ultrasonic point cloud based obstacle detection system according to an embodiment is shown; and

[0029] Figure 8 A schematic diagram of a vehicle according to an example embodiment of the present disclosure. DETAILED DESCRIPTION

[0030] The inventors have realized that, with the development of technology, the intelligent level of automobiles is continuously improved. Assistive driving and automatic parking technology are one of the important manifestations of the intelligent level of automobiles. Ultrasonic sensors have the advantages of low price and high ranging accuracy, and play an important role in the automatic parking scheme of automobiles. From the initial back-up radar alarm to the use of the triangulation principle and the traditional point cloud processing algorithm for obstacle detection, the tasks that ultrasonic sensors need to achieve are becoming more and more complex, and the required accuracy is also getting higher and higher. The limitations and bottlenecks of traditional algorithms are also exposed.

[0031] The inventors have also realized that a single ultrasonic sensor can usually only detect one-dimensional distance information within a certain FOV. When multiple ultrasonic sensors simultaneously detect the same object, or a single ultrasonic sensor detects the same object multiple times during movement, the position of the obstacle target can be determined using the triangulation method. Currently, multiple ultrasonic sensors are usually mounted on vehicles to determine the position of the obstacle target using the triangulation method. For example, a vehicle with automatic parking function generally mounts 12 ultrasonic sensors. As shown in FIG. 1, a schematic diagram of ultrasonic detection for a vehicle is shown. Figure 1

[0032] Figure 2 ​A typical parking space scanning process is shown. During the driving process, the ultrasonic sensors on the side of the front bumper detect the surface of the obstacle twice in succession, and a point on the surface of the obstacle can be determined using the principle of triangular positioning. Through a continuous scanning process, a point cloud representing the contour line of the obstacle can be generated. In the traditional scheme, these point clouds are clustered and line segment fitting methods are used to generate the contour line of the obstacle, i.e., the obstacle target, which is used to determine the parking space, vehicle obstacle avoidance and other functions, thereby ensuring the completion of the parking task.

[0033] The above process generates ultrasonic point clouds using the triangular positioning method, and generates obstacle targets using traditional clustering and line segment fitting methods. However, due to the unevenness of the obstacle surface and the high distance accuracy requirement of triangular positioning, the ultrasonic point cloud is often missing or drifting, and the contour line generated by the traditional algorithm usually has a large error. For example, Figure 3 A schematic diagram of a typical scanning parking space width is shown. In Figure 3 , each point represents an ultrasonic point in the obtained ultrasonic point cloud, the five-point star represents the true value of the obstacle vehicle head end point, and the solid line represents the obstacle contour line generated by the traditional algorithm. It can be seen that the obstacle contour line generated by the traditional algorithm is obviously wider than the true value of the obstacle vehicle head end point.

[0034] In addition, due to the sparsity and noise of the ultrasonic point cloud data itself, the traditional algorithm often performs poorly in complex scenes, mainly reflected in poor prediction of the position accuracy of the obstacle contour line, especially the end point position accuracy, and difficulty in completing more complex tasks, such as judging the height attribute of the obstacle (whether it is higher than the bumper) and the type of the obstacle.

[0035] Therefore, the present disclosure proposes an obstacle detection method and system based on ultrasonic point clouds, which uses a deep learning algorithm to more accurately regress the end points or contour lines of the obstacle, thereby improving the detection accuracy of the ultrasonic sensor in the parking task. In addition, in addition to using the coordinate information of the ultrasonic point cloud, the method of the present disclosure also uses the echo information (such as echo intensity, echo width, etc.) of the two detections that form the ultrasonic point as input to the neural network, which not only can obtain more accurate end point or contour line output, but also can be used to infer the height of the obstacle and the type of the obstacle, which is also very important output in various tasks (such as parking tasks). In this way, the method and system of the present disclosure can use multiple dimensions of information of the ultrasonic point cloud as input, avoid or simplify manual feature extraction, and through end-to-end processing, a large amount of data training, and learning of more fine features from more bottom and richer data, the accuracy and precision of obstacle recognition can be improved.

[0036] The detailed description set forth below, in connection with the appended drawings and embodiments described therewith, is intended as a description of various configurations and is not intended to represent the only configurations in which the concepts described herein can be practiced. The detailed description includes specific details for the purpose of providing a thorough understanding of various concepts. However, it will be apparent to those skilled in the art that these concepts can be practiced without these specific details.

[0037] Reference Figure 4 FIG. 4 shows a flowchart of an ultrasonic point cloud based obstacle detection method 400, according to an embodiment of the present disclosure.

[0038] As shown, the method 400 can include, at block 410, collecting an ultrasonic point cloud via an onboard ultrasonic sensor.

[0039] In an embodiment of the present disclosure, each ultrasonic point in the collected ultrasonic point cloud has at least one of an associated ultrasonic echo intensity, and / or an ultrasonic echo width, and associated two-dimensional coordinate information. For example, each ultrasonic point in the ultrasonic point cloud is obtained from two ultrasonic probes, each with a corresponding ultrasonic echo, whereby the echo intensity and / or width of the ultrasonic point can be the minimum, maximum, or average of the echo intensity and / or width of the corresponding echoes of the two ultrasonic probes. Alternatively, the echo intensity and / or width of the ultrasonic point can include the echo intensity and / or width corresponding to the two echoes, i.e., each ultrasonic point corresponds to two echo intensities and / or echo widths.

[0040] In an embodiment of the present disclosure, the onboard ultrasonic sensor can be positioned to emit ultrasonic waves to cover the area around the vehicle. For example, the mounting position and angle of the ultrasonic sensor relative to the vehicle can be set based on the field of view (FOV) of the ultrasonic sensor, the mounting position, the vehicle model (including the vehicle length, the vehicle width), and the like, to cover the area around the vehicle to the greatest extent.

[0041] In an embodiment of the present disclosure, the onboard ultrasonic sensor can include a plurality of ultrasonic sensors, and the fields of view of each two adjacent ultrasonic sensors of the plurality of ultrasonic sensors partially overlap to continuously cover the area around the vehicle (e.g., as shown in the schematic view of FIG. 1). Figure 1

[0042] Next, at block 420, the method 400 can include rasterizing the ultrasonic point cloud. In an example of the present disclosure, the size of the raster is dependent on the accuracy required for autonomous driving. For example, the raster size can be 4 cm x 4 cm.

[0043] Subsequently, the method 400 can include, at block 430, generating a pseudo picture based on the rasterized ultrasonic point cloud.

[0044] ​In an embodiment of the present disclosure, the pseudo picture can include a plurality of channels, and each channel can be a respective different information of the ultrasound points in each grid. For example, a first channel of the plurality of channels can be the number of ultrasound points in each grid, a second channel can be the first dimension coordinate information of the two-dimensional coordinate information of the ultrasound points in each grid, a third channel can be the second dimension coordinate information of the two-dimensional coordinate information of the ultrasound points in each grid, a fourth channel can be the intensity of the ultrasound echo of the ultrasound points in each grid, a fifth channel can be the width of the ultrasound echo of the ultrasound points in each grid, and so on. It will be appreciated that the pseudo picture can include any suitable number of channels, and each channel represents a specific information of the ultrasound points (for example, in the case that each ultrasound point corresponds to two echo intensities and / or echo widths of two ultrasound detections, the pseudo picture can also include two channels respectively corresponding to the two echo intensities and two channels respectively corresponding to the two echo widths), which will not be repeated here. The advantage of the pseudo picture is that it can use a common convolutional neural network to extract features, use different network heads, and complete different regression or classification tasks at the same time or separately.

[0045] Reference Figure 5 which shows an ultrasound point cloud and a first channel of a pseudo picture obtained after gridding according to an example embodiment of the present disclosure. As shown in (a), it shows an ultrasound point cloud of an obstacle being a vehicle head portion, and in (b), it shows a first channel of a pseudo picture obtained after gridding (also known as pixelization). It will be appreciated that the darker the color represents the more number of ultrasound points in the corresponding grid. From Figure 5 it can be seen that the point cloud density of the edges on both sides of the vehicle head is obviously smaller than other positions of the vehicle head. Figure 5

[0046] In a preferred embodiment of the present disclosure, in the case that a grid includes a plurality of ultrasound points, the coordinate information of the ultrasound points, the intensity of the ultrasound echo, and the width of the ultrasound echo in the grid are the average value or the minimum value or the maximum value or any other suitable value of the coordinate information, the intensity of the ultrasound echo, and the width of the ultrasound echo of the plurality of ultrasound points.

[0047] Continuing to refer to Figure 4 , the method 400 can include inputting the pseudo picture into a neural network model to identify at least one of: a type of the obstacle, a height of the obstacle, a contour line of the obstacle, an end point of the contour line of the obstacle.

[0048] In an embodiment of the present disclosure, the neural network model can be trained using labeled ultrasound point cloud data. In this embodiment, the labeled ultrasound point cloud data can include a label of at least one of: a contour line of the obstacle, an end point of the contour line, a height, a type of the obstacle.

[0049] ​In another embodiment of this disclosure, the labeled ultrasonic point cloud data is further augmented by rotating, flipping, and randomly deleting the ultrasonic point cloud, and simulated training is performed on different scenarios during the training process, thereby improving the generalization ability of the model in practical applications.

[0050] refer to Figure 6 The diagram shows a comparison between the detection results of the endpoints of the front profile of an obstacle vehicle according to an example embodiment of the present disclosure and the detection results of the endpoints of the front profile of an obstacle vehicle using a conventional multi-segment fitting algorithm. Figure 6 In the diagram, the "+" sign indicates the position of the detected contour endpoint. For example... Figure 6 As shown, the endpoints of the contour line fitted by the traditional algorithm ( Figure 6 Compared to (a) in the previous method, the method of this disclosure detects the endpoint positions of the contour lines more accurately. The position of the endpoints of the obstacle vehicle contour lines determines the size of the parking space between the obstacle vehicles, and the accuracy of the endpoint positions determines the accuracy of the parking space. In contrast, the accuracy of traditional algorithms for detecting parking space width is 80% with a probability deviation within 20cm, while using the method of this disclosure, the accuracy of parking space width detection can be improved to over 90% with a probability deviation within 20cm.

[0051] In a preferred embodiment of this disclosure, method 400 may optionally include clustering the ultrasonic point cloud before rasterizing it, thereby clustering different portions of the ultrasonic point cloud into different obstacles. For example, the ultrasonic point cloud may be a global point cloud around a vehicle, and multiple different obstacles may exist around the vehicle. Thus, the clustering algorithm can cluster different portions of the ultrasonic point cloud into different obstacles. Subsequently, method 400 may perform subsequent steps 420-440 of the method separately for ultrasonic point cloud portions belonging to the same cluster.

[0052] In another preferred embodiment of this disclosure, method 400 may optionally include performing multi-segment line fitting on ultrasonic point cloud portions belonging to the same cluster to obtain initial contour lines of corresponding obstacles; inputting ultrasonic point cloud portions belonging to the same cluster into a neural network model to identify the contour line endpoints of corresponding obstacles; and using the contour line endpoints of corresponding obstacles identified by the neural network model to correct the obtained initial contour lines of corresponding obstacles to obtain corrected contour lines of corresponding obstacles. In this embodiment, the neural network model may be used to identify contour line endpoints, and this identification result may be used to correct the obstacle contour lines obtained by the multi-segment line fitting method, thereby obtaining more accurate obstacle contour lines.

[0053] It will be appreciated that the results of the method 400 can be used for subsequent various tasks. Thus, in yet another embodiment of the present disclosure, the method 400 can further optionally comprise autonomous driving based on the type of the identified obstacle, the height of the obstacle, the contour line of the obstacle, the contour line endpoints of the obstacle.

[0054] In yet another preferred embodiment of the present disclosure, the method 400 can start to execute upon receiving an autonomous parking instruction. For example, the method 400 can start to execute after the driver presses an autonomous parking button. Thus, the method 400 can comprise first detecting whether an autonomous parking instruction is received, and executing steps 410-440 after receiving the autonomous parking instruction. In this embodiment, the method 400 can end after the vehicle is turned off. For example, after the vehicle is turned off, it can be considered that the vehicle has been parked, and thus the method 400 can end.

[0055] In yet another embodiment of the present disclosure, steps 420-440 of the method 400 can be executed in the cloud, and thus the method 400 can comprise transmitting the collected ultrasonic point cloud to the cloud for executing steps 420-440 in the cloud. Subsequently, the method 400 can comprise transmitting the recognition result to the vehicle. Further according to this embodiment, the method 400 can only notify the vehicle in the case that an obstacle is detected, so as to save network resources.

[0056] Reference is made below to Figure 7 which shows a schematic block diagram of an ultrasonic point cloud based obstacle detection system 700 according to an example embodiment of the present disclosure.

[0057] As shown, the system 700 can comprise an ultrasonic sensor 701 and a computing device 703 disposed on a vehicle. In an embodiment of the present disclosure, the ultrasonic sensor 701 can be configured to collect and transmit ultrasonic point clouds to the computing device 703, and the computing device 703 can be configured to receive the ultrasonic point clouds from the ultrasonic sensor 701 and execute methods described according to embodiments of the present disclosure, such as the method 400.

[0058] In a preferred embodiment of the present disclosure, the computing device 703 can comprise an on-board computer and / or a cloud, and thus methods of the present disclosure, such as the method 400, can be executed on the on-board computer and the cloud in any suitable manner. For example, steps 420-440 of the method 400 can be executed completely by the cloud or completely by the on-board computer or partially on the cloud and partially on the on-board computer. In the case that the computing device 703 comprises a cloud, the system 700 can further comprise a communication device 705 for communicating with the cloud, as shown by the dashed box 705 in Figure 7 .

[0059] Figure 8is a schematic diagram of a vehicle 800 according to an example embodiment of the present disclosure. In this embodiment, the vehicle 800 can include the components described above with reference to Figure 7 the example system 700 shown and described above.

[0060] As described above, the method and system of the present disclosure can collect ultrasonic point cloud data of different obstacles in various scenarios, which can include pillars, vehicles, water horses, and the like. The collected ultrasonic point cloud data can contain coordinate information of each ultrasonic point and corresponding echo information when each ultrasonic point is generated, including echo intensity, echo width, second echo, and the like. By annotating the collected data, a training data set is generated by annotating the contour lines or contour line endpoints, height, type, and the like of the obstacles through a true value system, such as lidar and image data. Subsequently, the generated training data set is used to train a neural network model to obtain a trained neural network model. In real vehicle operation, ultrasonic point clouds of obstacles can be collected, and the contour lines or contour line endpoints, height, type, and the like of the obstacles can be inferred in real time through the neural network model. The detection result can be directly used as the final output, or can be used as auxiliary information to correct the recognition of obstacles by traditional algorithms, and then be used for subsequent tasks.

[0061] In the method and system of the present disclosure, the ultrasonic point cloud is preferably first rasterized to generate a multi-channel pseudo picture as the input of the neural network model. There can be multiple ultrasonic points in each grid or no ultrasonic points. The first channel of the pseudo picture can be used to represent the number of ultrasonic points in each grid. In addition, each ultrasonic point has other multi-dimensional information, such as coordinate information (which itself has two dimensions), corresponding echo information (including but not limited to echo intensity, echo width) when the ultrasonic point is generated, and the like. Each dimension of information can be used as a channel of the pseudo picture. For the case where there are multiple ultrasonic points in a single grid, the information of this dimension can be max-pooled or averaged. In addition, the echo envelope can also be used. In this way, the method and system of the present disclosure improve the performance of ultrasonic sensor obstacle detection and recognition.

[0062] As described above, in the method and system of the present disclosure, the ultrasonic point cloud can be directly used as the input of the neural network model, reducing the dependence on the internal structure of the data, learning the original and rich feature information of the point cloud through a large amount of data training, and improving the accuracy and precision of obstacle recognition. In addition, the method and system of the present disclosure can also perform rotation, flipping, random deletion, and the like on the input ultrasonic point cloud through data augmentation technology, simulate different scenarios in the training process, and thus improve the generalization ability of the neural network model in actual application.

[0063] As mentioned above, in real scenarios, the ultrasonic system can collect and accumulate ultrasonic point clouds for a period of time during the parking process, which can include point clouds generated by more than one obstacle. For the point cloud of the whole image, it can either directly take it as the input of the neural network to detect and distinguish different obstacles, or first do a simple clustering on the point cloud to get the point cloud belonging to the same obstacle, and then take the local point cloud as the input of the neural network. In addition, the method and system of the present disclosure can also be combined with traditional algorithms, and the coordinates of the contour line endpoints predicted by the neural network model are used to correct the obstacle contour line detected by the traditional algorithm, so as to obtain more accurate obstacle contour line and contour line endpoint position.

[0064] Those skilled in the art will appreciate that although various embodiments are described in the present disclosure in connection with motor vehicles, the methods and systems of the present disclosure can also be applied to other suitable vehicles, such as airplanes, ships, etc., with corresponding adaptations.

[0065] The above DETAILED DESCRIPTION includes references to the accompanying drawings, which form a part of the DETAILED DESCRIPTION. The drawings provide illustration and discussion of the particular embodiments. These embodiments also are referred to in this document as “examples.” Such examples can include elements in addition to those illustrated or described. However, the examples set forth in this document are not exclusive examples. Rather, they are representative examples used to describe particular embodiments. Other examples can include, for example, other elements not expressly mentioned or described herein.

[0066] In the appended claims, the terms “including” and “comprising” are open-ended, that is, specify the presence of the stated features but do not preclude the presence of other features. In the claims, the term “first,” “second,” and “third,” and the like, merely identify the names of the elements and do not specifically limit the number or order of the elements. In the claims, means-plus-function clauses, if used, are intended to cover the structures described herein as performing the recited function and not only structural equivalents but also equivalent structures. In the claims, the term “consisting of’ is closed, that is, it excludes any element not specified in the claim.

[0067] In addition, the order of the operations illustrated in the specification is exemplary. In alternative embodiments, the operations can be performed in different orders than those illustrated, and the operations can be combined into single operations or split into more operations.

[0068] The above description is intended to be illustrative, and not restrictive. For example, the above-described examples (or one or more aspects thereof) can be used in combination with other examples. Other embodiments can be used, such as by one of ordinary skill in the art upon reviewing the above description. The Abstract allows a reader to quickly ascertain the nature of the technical disclosure. It is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. Also, in the above Detailed Description, various features can be grouped together or described in a single embodiment for the purpose of streamlining the disclosure. This disclosure is not to be interpreted in determining the scope of the claims. Rather, the claims are to take their broadest scope in accordance with the requirements of the law, and to this end, the claims are drafted using the broadest language possible. Furthermore, the use of the terms "a," "an," "the," etc. and "comprises," "comprising," "has," "having," "includes," "including," "contains," "containing," "consists," "consisting," "consists essentially," "consisting essentially of," "provides," "providing," "may," "might," "could," "would," "should," "can," "can," "like" and the like are to be understood to enable a claim limitation of any kind.

Claims

1. An obstacle detection method based on ultrasonic point clouds, comprising: Ultrasonic point clouds are acquired via an onboard ultrasonic sensor, wherein each ultrasonic point in the ultrasonic point cloud has at least one of the associated ultrasonic echo intensity and / or ultrasonic echo width, as well as associated two-dimensional coordinate information. Rasterize the ultrasonic point cloud; Generating pseudo-images based on rasterized ultrasonic point clouds; as well as The pseudo-image is input into a neural network model to identify at least one of the following: obstacle type, obstacle height, obstacle outline, and obstacle outline endpoints.

2. The method according to claim 1, characterized in that, The size of the grid depends on the precision required for autonomous driving.

3. The method according to claim 1, characterized in that, The pseudo-image includes multiple channels, each channel containing different information about the ultrasound points in each grid.

4. The method according to claim 3, characterized in that, The first channel of the multiple channels is the number of ultrasound points in each grid, the second channel is the first dimension coordinate information of the two-dimensional coordinate information of the ultrasound points in each grid, the third channel is the second dimension coordinate information of the two-dimensional coordinate information of the ultrasound points in each grid, the fourth channel is the intensity of the ultrasound echo of the ultrasound points in each grid, and the fifth channel is the width of the ultrasound echo of the ultrasound points in each grid.

5. The method according to claim 4, characterized in that, In the case where a grid includes multiple ultrasound points, the coordinate information, ultrasound echo intensity, and ultrasound echo width of the ultrasound points in the grid are the average, minimum, or maximum values ​​of the coordinate information, ultrasound echo intensity, and ultrasound echo width of the multiple ultrasound points.

6. The method according to claim 1, characterized in that, The neural network model is trained using labeled ultrasonic point cloud data, which includes annotations of at least one of the following: the outline of the obstacle, the endpoints of the outline, the height, and the type.

7. The method according to claim 6, characterized in that, The labeled ultrasonic point cloud data is further augmented by rotating, flipping, or randomly deleting the ultrasonic point cloud.

8. The method according to claim 1, characterized in that, It also includes clustering the ultrasonic point cloud before rasterizing it, and performing the subsequent steps of the method on the ultrasonic point cloud portions belonging to the same cluster.

9. The method according to claim 8, characterized in that, Also includes: Multi-segment line fitting is performed on the ultrasonic point cloud parts belonging to the same cluster to obtain the initial contour lines of the corresponding obstacles; The ultrasonic point cloud portions belonging to the same cluster are respectively input into the neural network model to identify the contour endpoints of the corresponding obstacles; as well as The initial contour of the corresponding obstacle is corrected using the endpoints of the corresponding obstacle's contour identified by the neural network model to obtain the corrected contour of the corresponding obstacle.

10. The method according to claim 1, characterized in that, It also includes autonomous driving based on the type of the identified obstacle, the obstacle height, the obstacle outline, and the endpoints of the obstacle outline.

11. An obstacle detection system based on ultrasonic point clouds, comprising: An ultrasonic sensor mounted on a vehicle, wherein the ultrasonic sensor is configured to acquire ultrasonic point clouds. as well as A computing device configured to receive an ultrasonic point cloud from the ultrasonic sensor. The computing device is further configured to perform the method according to any one of claims 1-10.

12. The system according to claim 11, characterized in that, The computing device includes an onboard computer and / or a cloud, and wherein, if the computing device includes a cloud, the system further includes a communication device for communicating with the cloud.

13. A vehicle comprising the system according to any one of claims 11-12.

Images