Obstacle detection method and device for assisting vehicle driving

Abstract

An obstacle detection method for assisting vehicle driving, the method including: a step S510 of acquiring ultrasonic echo data (202) captured while the vehicle is moving; a step S520 of acquiring information (204) associated with echo intersections based on the ultrasonic echo data (202); a step S540 of providing at least a portion of the ultrasonic echo data (202) and the information (204) associated with echo intersections as feature data to a machine learning model (208) to acquire obstacle detection information (210), wherein the machine learning model (208) uses at least one of a classification algorithm or a regression algorithm; and a step S560 of assisting vehicle driving based on the obstacle detection information (210).

Description

[Technical Field]

[0001] The present invention relates generally to obstacle detection, and more particularly to obstacle detection methods and devices for assisting vehicle driving. [Background technology]

[0002] With the continuous development of technology, existing obstacle detection methods and devices suitable for vehicle driving can detect the presence of obstacles around the vehicle while driving and give warnings to the driver to ensure safe driving of the vehicle. Such methods and devices are widely used in related applications such as autonomous driving systems, autonomous parking systems, and assisted driving systems.

[0003] Obstacle detection is currently typically performed using cameras, laser radar, millimeter-wave radar, and / or ultrasonic radar. Cameras have strong visual capabilities, but their imaging is highly affected by weather and they cannot directly detect distance. Laser radar has high accuracy and can better distinguish distance, but is easily affected by light or thermal radiation. Millimeter-wave radar has high anti-jamming capabilities, but its detection range and accuracy are limited. Furthermore, cameras and laser radar are expensive.

[0004] Due to the strong directivity of ultrasound, the ultrasonic radar range detection method is simple and low-cost, and has significant advantages in short-distance measurement, so that modern ultrasonic radars are currently widely applied in fields such as autonomous parking and stopping assistance. However, due to the low resolution of the data captured by ultrasonic radars, the prior art only supports simple calculation and ranging functions based on specific rules, and cannot be applied to complex scenarios. Summary of the Invention [Problem to be solved by the invention]

[0005] It is desirable to provide an obstacle detection method and device for assisting vehicle driving that can be accurate and robust in obstacle detection based on data captured by low-cost ultrasonic radar, thereby being applicable to complex road or parking lot scenarios to enable better path planning and obstacle avoidance during vehicle driving. [Means for solving the problem]

[0006] According to one aspect of the present invention, there is provided an obstacle detection method for assisting vehicle driving, the method including: acquiring ultrasonic echo data captured while a vehicle is moving; acquiring information associated with an echo intersection based on the ultrasonic echo data; providing at least a portion of the ultrasonic echo data and the information associated with the echo intersection as feature data to a machine learning model to acquire obstacle detection information, wherein the machine learning model uses at least one of a classification algorithm or a regression algorithm; and assisting vehicle driving based on the obstacle detection information.

[0007] According to another aspect of the present invention, there is provided a method for training a machine learning model for obstacle detection, the method including: acquiring ultrasonic echo data captured while a vehicle is moving; acquiring information associated with echo intersections based on the ultrasonic echo data; providing at least a portion of the ultrasonic echo data and the information associated with the echo intersections as feature data to a machine learning model to acquire obstacle detection information, wherein the machine learning model uses at least one of a classification algorithm or a regression algorithm; and calculating a loss function based on a difference between the obstacle detection information and the obstacle actual information.

[0008] According to yet another aspect of the present invention, there is provided an obstacle detection system for assisting vehicle driving, comprising a memory and a processor, the processor being coupled to the memory and configured to cause one or more units to perform a method according to any of the embodiments of the present invention.

[0009] According to yet another aspect of the present invention, there is provided a computer readable medium storing a computer program comprising instructions which, when executed by a control unit, cause one or more units to perform a method according to any of the various embodiments of the present invention.

[0010] According to yet another aspect of the present invention, there is provided a vehicle including one or more units for carrying out a method according to any of the various embodiments of the present invention.

[0011] Various embodiments of the protected subject matter will now be described, by way of example only, with reference to the accompanying drawings, in which the same reference numbers are used in different accompanying drawings to denote the same or similar elements. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a schematic diagram 100 of a vehicle that utilizes ultrasonic radar for obstacle detection. [Figure 2] FIG. 2 is a schematic diagram of a system 200 utilizing machine learning models for obstacle detection according to an embodiment of the present invention. [Figure 3A] 3 is a schematic diagram 300 of capturing ultrasonic echo data using ultrasonic radar, according to an embodiment of the present invention. [Figure 3B] 3 is a schematic diagram 301 of utilizing ultrasound echo data to obtain echo intersections according to an embodiment of the present invention. [Figure 3C] 3 is a schematic diagram 302 of utilizing ultrasound echo data to obtain drift angle information of echo intersections, according to an embodiment of the present invention. [Figure 4] 4 is a two-dimensional diagram 400 according to an embodiment of the present invention. [Figure 5] 5 is a flowchart of an obstacle detection method 500 for assisting vehicle driving according to an embodiment of the present invention. [Figure 6] FIG. 6 is a block diagram of a device 600 that may be used for obstacle detection according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0013] In the following description, numerous specific details are set forth to provide a thorough understanding of embodiments of the present invention. However, those skilled in the relevant art will recognize that the present invention can be practiced without one or more of the specific details or by using alternative methods, components, etc. In some instances, well-known structures and operations are not shown or described in detail to avoid unnecessarily obscuring the present invention.

[0014] It should be understood that the term "vehicle" or other similar terms used herein encompasses common motor vehicles such as passenger cars (including sport utility vehicles, buses, trucks, etc.), various commercial vehicles, and also encompasses hybrid vehicles, electric vehicles, etc. Additionally, while in most of the examples described below, the ultrasonic radar or sensor is shown as being mounted on a vehicle, it should be understood by those skilled in the art that the ultrasonic radar or sensor may be mounted in any suitable location, including, but not limited to, roadside units, parking lot infrastructure, etc.

[0015] Ultrasonic radar is a radar sensor device that uses ultrasonic waves to measure distance. It uses an ultrasonic emitting device to emit ultrasonic waves outward, then uses an ultrasonic receiving device to receive the echo reflected by an obstacle, and measures the distance from the obstacle based on the time difference. FIG. 1 shows a schematic diagram 100 of a vehicle that uses ultrasonic radar for obstacle detection. As shown in FIG. 1, an ultrasonic radar a mounted on the right side of vehicle A emits ultrasonic waves outward. When the ultrasonic waves encounter vehicle B, they are reflected and the echo is received by the ultrasonic radar a. The ultrasonic radar a or other processing unit of vehicle A can calculate the distance between vehicle A and vehicle B according to the time difference between the ultrasonic emission and the echo reception. However, apart from distance, more information about the obstacle, such as the location of the obstacle, cannot be obtained from the data captured by a single ultrasonic radar.

[0016] Generally, obstacle detection can be performed through a linkage of multiple ultrasonic radars. In one embodiment, ultrasonic echoes emitted by ultrasonic radar a can be received by ultrasonic radar a and ultrasonic radars b and c on two adjacent sides of ultrasonic radar a, from which the two-dimensional position of the obstacle can be calculated and the shape of the obstacle can be estimated according to rules known to those skilled in the art. In practical applications, the accuracy and robustness of obstacle detection based on ultrasonic data are difficult to guarantee, and existing rules only apply to simple scenarios and cannot address complex vehicle driving environments. Therefore, a new scheme is needed to achieve accurate and robust obstacle detection based on data captured by ultrasonic radars.

[0017] 2 shows a schematic diagram of a system 200 utilizing a machine learning model for obstacle detection according to an embodiment of the present invention. In the system 200, a machine learning model 208 processes input feature data such as 202, 204, and 208 using at least one of a classification algorithm 212 or a regression algorithm 214 to output obstacle detection information 210.

[0018] The classification algorithm 212 includes, but is not limited to, a Bayesian classification algorithm, a decision tree classification algorithm, a random forest classification algorithm, a support vector machine classification algorithm, a K-nearest neighbor classification algorithm, etc., or any applicable classification algorithm, particularly a classification algorithm suitable for performing multi-label classification tasks. The regression algorithm 214 includes, but is not limited to, a linear regression algorithm, a support vector machine regression algorithm, a decision tree regression algorithm, a random forest regression algorithm, an XGBoost regression algorithm, a Gaussian regression algorithm, a polynomial regression algorithm, etc., or any applicable regression algorithm, particularly a regression algorithm suitable for performing predictive frame regression tasks.

[0019] The input feature data includes a series of data acquired during vehicle movement. In the example shown in Figure 2, the feature data includes ultrasonic echo data 202 captured during vehicle movement, information 204 associated with echo intersections based on the ultrasonic echo data, and optionally, data 206 related to vehicle movement during vehicle movement. More details regarding the information are provided below.

[0020] FIG. 3A shows a schematic diagram 300 of capturing ultrasonic echo data using an ultrasonic radar according to an embodiment of the present invention. In one embodiment, as shown in FIG. 3A , an ultrasonic radar a mounted on the front right side of a vehicle A emits ultrasonic waves outward, and the ultrasonic radar a can monitor whether an echo reflected by an obstacle is received. Typically, an ultrasonic signal is reflected multiple times by an obstacle, thus generating multiple echoes. In one embodiment, because the quality of the echo signal reflected multiple times may be poor, the ultrasonic radar a may collect intensity information and peak information of only the echo signal received the first and second time. In other embodiments, a larger or smaller number of echo signals may be used based on signal quality not specified herein.

[0021] Referring to the description of FIG. 1, in response to the received echo signal, the ultrasonic radar a or other processing unit of vehicle A can calculate the echo distance and echo height based on the time difference between ultrasonic emission and echo reception. As shown in FIG. 3A, the actual echo point is E1', but the distance information acquired by the ultrasonic radar alone cannot determine the actual echo point location. In one embodiment of the present invention, the echo point location is determined based on the echo distance perpendicular to the vehicle movement direction, and as shown in FIG. 3A, the echo point location is determined to be E1 perpendicular to the vehicle movement direction. In other embodiments, the echo point location can be determined according to the echo distance at other angles based on statistical information. In still other embodiments, the echo point location can be determined according to the echo distance based on other general rules.

[0022] FIG. 3B shows a schematic diagram 301 of obtaining echo intersections using ultrasonic echo data according to an embodiment of the present invention. In one embodiment, as shown in FIG. 3B, an ultrasonic radar a mounted on the front right side of vehicle A continuously emits ultrasonic waves outward and continuously receives echo signals as vehicle A moves. Referring to the description of FIG. 3A, the position of the actual echo point cannot be determined from a single echo. Based on this, it is believed that the intersection position of two echoes may be closer to the position of the actual echo point, and the shape of the obstacle may be more accurately outlined by multiple echo intersections. FIG. 3B shows various echo points E1 to E6 determined based on echo distance information, and an intersection position of two echoes I determined based on at least the echo distance information. 12、 I 23 , I 34 , I 45 , I 56 3B shows six echoes, but other echoes may be used in other embodiments and are not specified here.

[0023] Furthermore, for the sake of simplicity, the echo intersections in the embodiment shown in FIG. 3B are calculated based on two adjacent echoes. In other embodiments, all actually generated echo intersections can be calculated without being limited to adjacent echoes. In still other embodiments, a window size can be determined according to computing resources, and the echo intersections generated by all echoes within the window size can be calculated. For example, the window size determined according to computing resources is 5, that is, the echo intersections between each echo and the two echoes adjacent to each echo in the vehicle driving direction are calculated.

[0024] 3C shows a schematic diagram 302 of utilizing ultrasound echo data to obtain drift angle information of echo intersections, according to an embodiment of the present invention. Referring to the above description of FIG. 3B, 12 The vehicle position can be obtained based on the two echo signals captured at x1 and x2. In addition to the position and distance information of the echo intersection, the drift angle information of the echo intersection also assists in obstacle detection. In obstacle detection applicable to vehicles, this is generally described in a two-dimensional plane by the positions of the two outer corners of the obstacle. Therefore, determining the drift angle information of the echo intersection helps to determine the positional relationship of the vehicle with respect to the obstacle and helps to determine the outer corner position of the obstacle.

[0025] As shown in Figure 3C, echo crossing I 12 The drift angle information can be determined as any one of 303-1 to 303-4, and in one embodiment, x12 may be the midpoint between two vehicle positions x1 and x2, and in another embodiment of the present invention, x12 may also be the line branching the angle of the angle formed. <x1-i12-x2>In one embodiment of the present invention, the echo crossing I 12 The drift angle is the distance from the vehicle's direction of travel 12 and the drift angle of the connecting line of x12, i.e., 303-1 or 303-2. In another embodiment of the present invention, the echo crossing I 12 The drift angle of I 12 The drift angle of the echo intersection can be determined as the drift angle of the ray perpendicular to the connecting line of x12 and x13 relative to the vehicle's moving direction, i.e., 303-3 or 303-4. In other embodiments, the drift angle of the echo intersection can be determined by any appropriate method. As the positional relationship between the vehicle and the obstacle changes, the drift angle of the echo intersection also changes. Considering the drift angle 303-4 as an example, as the vehicle gradually approaches the obstacle, the drift angle 303-4 gradually decreases. As the vehicle moves parallel to the obstacle, the drift angle 303-4 approaches zero. As the vehicle gradually approaches the obstacle, the drift angle 303-4 gradually increases again.

[0026] Additionally, data relating to vehicle movement may be acquired while the vehicle is moving, including, but not limited to, vehicle position information, vehicle angular velocity information, and vehicle distance traveled information. In one embodiment, the data points relating to vehicle movement correspond to ultrasound echo data points captured while the vehicle is moving.

[0027] 3A-3C, a series of data may be acquired during vehicle movement, and the series of data may be provided as feature data to a machine learning model 208, as shown in FIG. 2, and obstacle detection information 210 may be output using at least one of a classification algorithm 212 and a regression algorithm 214.

[0028] The ultrasound echo data 202 may include, but are not limited to, echo point position information, echo distance information, echo height information, echo signal intensity information, and echo signal peak information. In one embodiment of the present invention, the one or more ultrasound echo data 202 may be determined according to the method described in FIG. 3A.

[0029] The information 204 associated with the echo intersections may include, but is not limited to, position information of the echo intersections, distance information of the echo intersections, distance information of adjacent echo intersections, position information of adjacent echo intersections, and drift angle information of the echo intersections. In one embodiment of the present invention, the information 204 associated with one or more echo intersections may be determined according to the techniques described in Figures 3B-3C.

[0030] The data 206 regarding vehicle movement may include, but is not limited to, vehicle position information, vehicle angular velocity information, and vehicle distance traveled information.

[0031] It will be appreciated that the above data detailed in relation to Figures 3A to 3C may be detected by respective units (e.g., ultrasonic radar, ultrasonic sensor, steering wheel rotation angle sensor, speed sensor, etc.), and various parameters may be used in any combination.

[0032] Additionally, pre-processing may be performed on the data, which may involve various calculations and operations performed on the data described above. The data generated by the pre-processing is then input as feature data into the machine learning model 208.

[0033] In one embodiment of the present invention, the ultrasound echo data 202 may be preprocessed using a downsampling algorithm. To address the problem of imbalance in the classification of raw ultrasound echo data, a portion of the data may be selected from a majority set and recombined with a minority set to form a new data set; this technique is known as downsampling. For example, commonly used downsampling algorithms include, but are not limited to, random downsampling, EasyEnsemble, BalanceCascade, or NearMiss.

[0034] In one embodiment of the present invention, the ultrasound echo data 202 may be preprocessed using an upsampling algorithm. To address the problem of imbalance in the classification of raw ultrasound echo data, the data from the minority set may be expanded to an equal number of data from the majority set, a technique known as upsampling. For example, commonly used upsampling algorithms include, but are not limited to, interpolation, transpose convolution, and uppooling.

[0035] In another embodiment of the present invention, a gridding method can be used to preprocess the ultrasound echo data 202 and the information associated with echo intersections 204. Data gridding helps integrate high-dimensional feature data to facilitate input to a machine learning model for processing. For example, the gridding on 202 and 204 can be performed at least in part based on the echo point location information and the echo intersection location information included in the ultrasound echo data 202 and the information associated with echo intersections 204. To ensure accuracy and efficiency of the calculation, a compromise should be made between the particle size and the sparseness of the grid. In one embodiment, the grid size may be fixed at, for example, 3 cm. In another embodiment, the grid size may be flexibly adjustable based on a predetermined threshold condition. For example, when computing resources are sufficient, the grid size may be 1 cm, and when computing resources are expanded, the grid size may be 5 cm.

[0036] In further embodiments of the present invention, an aggregation algorithm may be used to process the gridded ultrasound echo data 202 and the information associated with the echo intersections 204. For example, commonly used aggregation algorithms include, but are not limited to, max pooling, min pooling, average pooling, overlap pooling, pyramid pooling, bilinear pooling, etc. The aggregation process can remove redundant information and further integrate feature data, thereby reducing the number of parameters and preventing overfitting.

[0037] In a further embodiment of the present invention, a two-dimensional map may be generated based on at least the aggregated ultrasound echo data 202 and information associated with echo intersections 204, and the feature data is provided to the machine learning model in the form of the two-dimensional map. Figure 4 illustrates a two-dimensional map 400 according to one embodiment of the present invention, where square dots represent ultrasound echo data 202, triangular dots represent information associated with echo intersections 204, and circular dots represent data 206 representing data related to vehicle movement. It will be understood that the data at each point in the two-dimensional map is necessarily a multidimensional array, including, for example, those described above in connection with Figures 2 and 3A-3C, respectively.

[0038] In a further embodiment of the present invention, a sparse matrix may be generated based on at least the aggregated ultrasound echo data 202 and the information associated with echo crossings 204, and the feature data is provided to the machine learning model in the form of a sparse matrix. It should be understood that the data in the sparse matrix is ​​necessarily a multidimensional tensor, for example, the data in a single grid may have the following array: [X coordinate, Y coordinate, echo distance, echo height, echo signal strength, ..., echo crossing distance, ..., vehicle angular velocity...], and that if a corresponding data item does not exist in the current grid, that item is set to zero.

[0039] The machine learning model 208 is acquired through training. The training process is based on a plurality of samples, each of which includes the aforementioned ultrasonic echo data and information associated with the echo intersection obtained based on the ultrasonic echo data. Each sample may also include data related to vehicle movement. Furthermore, the sample data also includes corresponding actual information of the obstacle as a label. In one embodiment, the label may be in the following array format: [obstacle outer angle coordinate 1, obstacle outer angle coordinate 2, obstacle height, obstacle type]. If there are multiple obstacles in one spatial region, there will be multiple corresponding labels. The obstacle detection information generated by the machine model 208 must meet certain accuracy requirements, i.e., the deviation between the obstacle detection information and the obstacle actual information must be less than a threshold. In one embodiment of the present invention, a plurality of sample data (such as ultrasound echo data 202, information associated with echo intersections 204, and optional data regarding vehicle movement 206) is input to a machine learning model 208 to be trained, and a loss function is calculated based on the obstacle detection information 210 and actual information of the obstacles output by the machine learning model 208, and the machine learning model 208 is trained by decreasing the value of the loss function until the value of the loss function converges.

[0040] In one embodiment of the present invention, when feature data is provided to the machine learning model in the form of a two-dimensional diagram, it may be processed using a machine learning model suitable for images, including, but not limited to, a convolutional neural network (CNN) model, a region-based convolutional neural network (R-CNN) model, or a YOLO model. In another embodiment of the present invention, when feature data is provided to the machine learning model in the form of a sparse matrix, it may be processed using a machine learning model suitable for sparse matrices, including, but not limited to, a multi-layer perceptron model, a pulsed neural network (SNN) model, a sparse convolutional neural network model, or a transformer neural network model.

[0041] The machine learning model 208 shown in FIG. 2 can output obstacle detection information 210. An obstacle detection task typically includes two parts, including obstacle identification and obstacle localization. In one embodiment of the present invention, obstacle identification may be achieved by a classification algorithm 212. In another embodiment of the present invention, obstacle localization may be achieved by a regression algorithm 214. In one embodiment of the present invention, the classification algorithm 212 and the regression algorithm 214 are executed simultaneously within the machine learning model 208. For example, the obstacle detection information may include one or more of the obstacle position, the obstacle height, and the obstacle type. In one embodiment of the present invention, the obstacle position may be represented by the spatial coordinates of its two exterior angles.

[0042] 2 may be configured within an autonomous driving system, an autonomous parking system, or an assisted parking system for a vehicle. System 200 may also be configured within a roadside unit, a road surveillance camera, parking lot infrastructure, etc. Leveraging the operational capabilities of machine learning models, the system overcomes the low resolution and information limitations of ultrasonic radar data, enabling accurate and robust obstacle detection while saving costs.

[0043] FIG. 5 shows a flow chart of an obstacle detection method 500 for assisting vehicle driving according to an embodiment of the present invention.

[0044] In step S510, a device (such as a vehicle, a roadside unit, a road surveillance camera, or parking lot infrastructure) utilizing the system 200 for detecting obstacles using a machine learning model is configured to acquire ultrasonic echo data captured during vehicle movement.

[0045] In one embodiment, the ultrasonic echo data includes some or all of the ultrasonic echo data 202 described above in connection with Figures 3A-3C, particularly Figure 3A. In one embodiment, the ultrasonic echo data may be captured by a single ultrasonic sensor or ultrasonic radar. In other embodiments, the ultrasonic echo data may be captured by multiple ultrasonic sensors or ultrasonic radars. In one embodiment, the ultrasonic echo data may be calculated based on data captured by a single ultrasonic sensor or ultrasonic radar. In other embodiments, the ultrasonic echo data may be calculated based on data captured by multiple ultrasonic sensors or ultrasonic radars.

[0046] In step S520, the device obtains information associated with echo intersections based on the ultrasound echo data. In one embodiment, the information associated with echo intersections includes some or all of the information associated with echo intersections 204 described above in connection with Figures 3A-3C, particularly Figures 3B and 3C. In one embodiment, the information associated with echo intersections may be calculated by an ultrasound radar capturing the ultrasound data. In other embodiments, the information associated with echo intersections may be calculated by any unit within the device having computing capabilities.

[0047] Optionally, in step S530, the device may acquire data related to vehicle movement while the vehicle is moving. In one embodiment, the data related to vehicle movement includes some or all of the data related to vehicle movement 206 described above in connection with Figures 3A to 3C. In one embodiment, the data related to vehicle movement may be captured by various sensors (e.g., a steering wheel rotation angle sensor, a speed sensor, etc.).

[0048] In step S540, the ultrasound echo data, information associated with the echo intersections, and (optional) data related to vehicle movement may be provided as feature data to the machine learning model 208 shown in Figure 2. Optionally, the ultrasound echo data, information associated with the echo intersections, and (optional) data related to vehicle movement may be pre-processed as described above.

[0049] In step S550, based on the feature data, the machine learning model 208 detects obstacles using at least one of the classification algorithm 212 or the regression algorithm 214.

[0050] In step S560, the control unit of the device assists vehicle driving based on the obstacle detection information, for example, the vehicle can plan a route or issue an alert based on the obstacle detection information, for another example, the roadside unit can notify or instruct the vehicle based on the obstacle detection information, and for yet another example, the parking infrastructure can identify a parking space or guide the vehicle to an available parking space based on the obstacle detection information.

[0051] 6 shows a block diagram of an apparatus 600 that may be used for obstacle detection according to an embodiment of the present invention. In one embodiment of the present invention, apparatus 600 may comprise a vehicle control unit or auxiliary system, such as an electronic control unit (ECU), an electronic management unit (EMU), a parking assistance system, an autonomous driving system, etc. In other embodiments, apparatus 600 may comprise any unit with data processing and computing capabilities, such as a vehicle, a roadside unit, a road surveillance camera, a parking lot infrastructure, etc.

[0052] The exemplary device 600 includes a processor 604 coupled to an internal communication bus 602, which is used to execute instructions in a memory 606 to implement the obstacle detection method for assisting vehicle operation detailed above. Examples of the processor 604 include a central processing unit (CPU), a microcontroller, etc. Suitable memory 606 for tangibly embodying computer program instructions and data includes various forms of memory, such as EPROM, EEPROM, and flash memory devices. The device 600 may further include an input interface 608 and an output interface 610. The input interface 608 is used to receive input signals and data, such as data from various sensors. The output interface 610 is used to transmit output signals and data, such as various command or warning signals for assisting vehicle operation.

[0053] A computer program may include computer-executable instructions that are used to cause the processor 604 of the device 600 to perform an obstacle detection method for assisting vehicle driving. The program may be recorded on any data storage medium, including a memory. For example, the program may be implemented in digital electronic circuitry, or using computer hardware, firmware, software, or a combination thereof. The process / method steps described in this disclosure may be performed by a programmable processor executing the program instructions, which operate on input data and generate output to perform the method steps, processes, or operations.

[0054] In addition to what is described in this document, various modifications can be made to the disclosed embodiments and implementations of the invention without departing from the scope of the disclosed embodiments and embodiments of the invention. Accordingly, the descriptions and examples herein should be construed as illustrative and not limiting. The scope of the invention should be determined solely by reference to the claims.

Claims

1. 1. An obstacle detection method for assisting vehicle driving, comprising: acquiring ultrasonic echo data captured while the vehicle is moving; obtaining information associated with echo intersections based on the ultrasound echo data; providing at least a portion of the ultrasonic echo data and information associated with the echo intersection as feature data to a machine learning model to obtain obstacle detection information, wherein the machine learning model uses at least one of a classification algorithm or a regression algorithm; assisting vehicle driving based on the obstacle detection information; A method comprising:

2. acquiring data relating to the vehicle movement while the vehicle is moving, the data relating to the vehicle movement including one or more of vehicle position information, angular velocity information of the vehicle, and travel distance information of the vehicle; providing the data regarding the vehicle movement as feature data to the machine learning model; The method of claim 1 further comprising:

3. The method of claim 1 , wherein the ultrasound echo data includes one or more of echo point position information, echo distance information, echo height information, echo signal intensity information, and echo signal peak information.

4. The method of claim 3 , wherein the ultrasonic echo data is based on data captured by a single ultrasonic sensor or ultrasonic radar.

5. The method of claim 3 , wherein the echo point location information is determined based on at least the echo distance information perpendicular to the vehicle movement direction.

6. Obtaining information associated with the echo intersection based on the ultrasound echo data includes: calculating echo crossings between various echoes based at least on said echo distance information; obtaining information associated with the echo crossings based on the calculated echo crossings; The method of claim 3, comprising:

7. 7. The method of claim 6, wherein the information associated with the echo crossing includes one or more of position information of the echo crossing, distance information of the echo crossing, distance information of adjacent echo crossings, position information of the adjacent echo crossings, and drift angle information of the echo crossings.

8. processing the ultrasound echo data using a downsampling or upsampling algorithm; Providing downsampled or upsampled ultrasound echo data as the feature data to the machine learning model; The method of claim 1 further comprising:

9. performing gridding on at least a portion of the ultrasound echo data and information associated with the echo intersections; processing the gridded ultrasound echo data and information associated with the echo intersections using an aggregation algorithm; The method of claim 1 further comprising:

10. The method of claim 9 , wherein the aggregation algorithm comprises max pooling, min pooling, average pooling, overlap pooling, pyramid pooling, or bilinear pooling.

11. generating a two-dimensional view based on the aggregated ultrasound echo data and information associated with the echo intersections; providing the feature data in the form of the two-dimensional diagram to the machine learning model; 10. The method of claim 9, further comprising:

12. 12. The method of claim 11, wherein the machine learning model comprises a convolutional neural network (CNN) model, a region-based convolutional neural network (R-CNN) model, or a YOLO model.

13. generating a sparse matrix based on the aggregated ultrasound echo data and information associated with the echo intersections; providing the feature data in the form of the sparse matrix to the machine learning model; 10. The method of claim 9, further comprising:

14. 14. The method of claim 13, wherein the machine learning model comprises a multi-layer perceptron model, a pulsed neural network (SNN) model, a sparse convolutional neural network model, or a transformer neural network model.

15. The method of claim 1 , wherein the machine learning model executes both the classification algorithm and the regression algorithm simultaneously.

16. The method of claim 1 , wherein the obstacle detection information includes one or more of a position of the obstacle, a height of the obstacle, and a type of the obstacle.

17. 1. A method for training a machine learning model for obstacle detection, comprising: acquiring ultrasonic echo data captured while the vehicle is moving; obtaining information associated with echo intersections based on the ultrasound echo data; providing at least a portion of the ultrasonic echo data and information associated with the echo intersection as feature data to a machine learning model to obtain obstacle detection information, wherein the machine learning model uses at least one of a classification algorithm or a regression algorithm; Calculating a loss function based on a difference between the detected information of the obstacle and the actual information of the obstacle; training the machine learning model by decreasing the value of the loss function; A method comprising:

18. 1. An obstacle detection system for assisting vehicle driving, comprising: Memory and a processor coupled to the memory; Equipped with 18. A system, wherein the processor is configured to cause one or more units to perform the method of any one of claims 1 to 17.

19. A computer readable medium storing a computer program comprising instructions that, when executed by a processor, cause one or more units to perform the method of any one of claims 1 to 17.

20. A vehicle comprising one or more units for carrying out the method according to any one of claims 1 to 17.

Images