Method and apparatus for determining objects around vehicle

Abstract

To provide a method for determining objects around a vehicle on the basis of ultrasonic signals.SOLUTION: A method includes the steps of: clustering on the basis of at least a part of ultrasonic data in an ultrasonic data set to obtain one or more sample clusters, each sample cluster corresponding to an object around a vehicle; determining an object type corresponding to each sample cluster on the basis of the ultrasonic data corresponding to the one or more sample clusters; and providing, to a corresponding first machine learning model in a first machine learning model set, the ultrasonic data corresponding to each sample cluster and the object type as feature data so as to output object position prediction information, the corresponding first machine learning model corresponding to each sample cluster being selected from the first machine learning model set on the basis of the object type corresponding to the sample cluster.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present application relates to automatic vehicle control, and more particularly to a method, a control system, a vehicle, a machine-readable storage medium, and a computer program product for determining objects around a vehicle based on ultrasonic signals. [Background technology]

[0002] Accurately identifying objects in a vehicle's surroundings is important for implementing various vehicle assistance functions in applications such as autonomous driving systems, automated parking systems, and driver assistance systems. For example, sensor signals can be used to detect objects within a vehicle's surroundings, enabling the vehicle to identify obstacles in real time and adjust its driving path accordingly. Similarly, in a parking lot environment, sensor signals can detect obstacles and accurately determine available parking spaces.

[0003] Ultrasonic sensors (USS) offer a cost-effective solution for object detection. However, the low resolution of data captured by ultrasonic radar hinders the effective application of traditional detection methods using ultrasonic signals in complex scenarios. Furthermore, the presence of various objects in different vehicle application scenarios necessitates improving the accuracy of object detection using ultrasonic signals. Increasing this accuracy is critical for optimizing assisted driving and parking performance. Summary of the Invention [Problem to be solved by the invention]

[0004] The following description is provided to present some concepts in a simplified form that are further described below in the detailed description. This description is not intended to highlight key or necessary features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter.

[0005] It is desirable to provide a method for determining objects around a vehicle based on ultrasonic signals. The method should be able to select an appropriate model to predict the location of an object based on its type across various driving scenarios. By ensuring the accuracy and robustness of object detection, this approach can be effectively applied to complex road and parking situations, thereby improving the performance of vehicle-assisted driving and parking. [Means for solving the problem]

[0006] In one aspect, an embodiment of the present disclosure provides a method for determining objects around a vehicle based on ultrasound signals, the method including: clustering based on at least a portion of ultrasound data in an ultrasound dataset to obtain one or more sample clusters, each sample cluster corresponding to an object around the vehicle; determining an object type corresponding to each sample cluster based on the ultrasound data corresponding to the one or more sample clusters; and providing the ultrasound data and the object type as feature data corresponding to each sample cluster to a corresponding first machine learning model in a first machine learning model set to output position prediction information for the object, wherein the corresponding first machine learning model corresponding to each sample cluster is selected from the first machine learning model set based on the object type corresponding to the sample cluster.

[0007] In another aspect, an embodiment of the present disclosure provides a control system for a vehicle, the system comprising: at least one processor; and a memory coupled to the at least one processor, the memory storing executable instructions that, when executed by the at least one processor, enable the at least one processor to perform a method according to any embodiment of the present disclosure.

[0008] In another aspect, an embodiment of the present disclosure provides a computer-readable medium storing a computer program including instructions that, when executed by the processor, enable one or more units to perform a method according to any embodiment of the present disclosure.

[0009] In another aspect, an embodiment of the present disclosure provides a computer program product including a computer program that, when executed by a processor, enables the processor to perform a method according to any embodiment of the present disclosure.

[0010] In another aspect, an embodiment of the present disclosure provides a vehicle, the vehicle comprising an ultrasonic sensor for transmitting and receiving ultrasonic signals, and one or more units for performing a method according to any embodiment of the present disclosure. [Brief explanation of the drawings]

[0011] The nature and advantages of the present disclosure may be further explained by reference to the accompanying drawings, in which similar components or structures may be designated with the same reference numerals, and in which: [Figure 1] 1 is a schematic diagram of an exemplary vehicle according to an embodiment of the present disclosure. [Figure 2] FIG. 1 is a schematic diagram of an exemplary control system in a vehicle, according to an embodiment of the present disclosure. [Figure 3A] FIG. 1 is a schematic diagram of an application scenario according to an embodiment of the present disclosure. [Figure 3B] FIG. 2 is a schematic diagram of ultrasound data according to an embodiment of the present disclosure. [Figure 4] 1 is a schematic diagram of a method and modules for determining objects around a vehicle according to an embodiment of the present disclosure; [Figure 5] 1 is a flowchart of a method for determining objects around a vehicle according to an embodiment of the present disclosure. [Figure 6]1 is a flowchart for training a machine learning model for determining objects around a vehicle, according to an embodiment of the present disclosure. [Figure 7] 1 is a flowchart of a method for assisting vehicle driving, according to an embodiment of the present disclosure. [Figure 8] FIG. 1 is a block diagram of a control system for a vehicle according to an example of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0012] The subject matter described in the present disclosure will now be discussed with reference to exemplary embodiments. It should be understood that the description of these embodiments is not intended to limit the scope, applicability, or implementation of the protection set forth in the claims, but is provided to aid those skilled in the art in better understanding and thereby implementing the subject matter described in the present disclosure. The function and arrangement of the described elements may be changed without departing from the scope of protection of the content of the present disclosure. Various processes or components may be omitted, substituted, or added in various embodiments as needed. For example, the described methods may be performed in an order different from that described, and various steps may be added, omitted, or combined. Furthermore, features described in connection with one embodiment may also be combined in other embodiments.

[0013] As used in this disclosure, the terms "comprising" and variations thereof are open terms and mean "including, but not limited to." The term "based" means "based at least in part on." The terms "one embodiment" and "an embodiment" mean "at least one example." The term "another embodiment" means "at least one other embodiment." Terms such as "first," "second," etc. may refer to different or the same object. Unless explicitly stated otherwise in context, the definition of a term will be consistent throughout the description.

[0014] Ultrasonic sensors have attracted considerable attention and research in the automotive field due to their advantages of low cost, fast response, and ease of integration. However, effectively utilizing ultrasonic sensors for object detection in the automotive field remains a current challenge. To address this issue, embodiments of the present disclosure provide technical solutions for determining objects around a vehicle based on ultrasonic signals. A detailed description is provided below with reference to specific embodiments.

[0015] 1 is a schematic diagram of an exemplary vehicle according to an embodiment of the present disclosure. It will be understood that the following examples are provided solely to enhance the understanding of the present disclosure and are not intended to impose any limitations on the scope of the present disclosure.

[0016] In the example of FIG. 1 , at least one ultrasonic sensor 110 (represented simply as a black dot in FIG. 1 ) may be mounted on the vehicle 100. For example, as shown in FIG. 1 , the ultrasonic sensors 110 may be mounted on the front, rear, left, and right sides of the vehicle 100, i.e., corresponding to the front, rear, left, and right sides of the vehicle 100, respectively. FIG. 1 shows 16 ultrasonic sensors. However, the number and locations of the ultrasonic sensors 110 are not limited to those shown in FIG. 1 . In various implementations, the vehicle 100 may include more or fewer ultrasonic sensors, and the locations of the ultrasonic sensors may be changed as well.

[0017] The ultrasonic sensors 110 may be configured to emit ultrasonic signals around the vehicle 100. For example, two ultrasonic sensors 110 located on the right side of the vehicle 100 may be configured to emit ultrasonic signals 120. The ultrasonic signals 120 are described below as an example. The ultrasonic signals 120 may encounter various objects during transmission, such as obstacles (e.g., other vehicles, ground locking mechanisms, trees, rails, fences, etc.), objects related to the vehicle's driving operations (e.g., curves, etc.), pedestrians, etc. When encountering an object, the ultrasonic signals 120 may be reflected. After the ultrasonic signals 120 are reflected, the ultrasonic sensors 110 may receive a reflected signal (commonly referred to as an echo signal).

[0018] Various ultrasonic data may be obtained based on the transmitted ultrasonic signals and the received echo signals. These operations may be implemented in various ways. For example, in some implementations, the vehicle 100 may include an ultrasonic sensor module. The ultrasonic sensor module may include an ultrasonic sensor 110 and a processing unit. The ultrasonic sensor 110 may include a transmitter for transmitting ultrasonic signals and a receiver for receiving echo signals. The processing unit may be configured to obtain various ultrasonic data based on the transmitted ultrasonic signals and the received echo signals. Furthermore, the processing unit may be configured to control the transmission and reception of signals by each ultrasonic sensor.

[0019] For example, in FIG. 1 , two ultrasonic sensors 110 provided on the front side of the right side of the vehicle 100 constitute an ultrasonic module 105, which includes two ultrasonic sensors for transmitting and receiving ultrasonic signals and a processing unit (not shown in FIG. 1 ). The processing unit can acquire various ultrasonic data based on the ultrasonic signals transmitted by the two ultrasonic sensors and the echo signals received by them. Similarly, two ultrasonic sensors 110 can be provided on the rear side of the right side of the vehicle 110, two on the front side of the left side, two on the rear side of the left side, four on the front, and four on the rear, and all of these ultrasonic sensors 110 can constitute an ultrasonic sensor module with corresponding processing units.

[0020] Alternatively, in another embodiment, a single ultrasonic sensor 110 of the vehicle 100 may be configured as an ultrasonic sensor module having a corresponding processing unit, and the processing unit may be configured to acquire various ultrasonic data based on the ultrasonic signals transmitted by the single sensor and the echo signals received. In another embodiment, four ultrasonic sensors 110 and corresponding processing units located on the right side of the vehicle 100 in FIG. 1 may be configured as an ultrasonic sensor module, and the processing unit may be configured to acquire various ultrasonic data based on the ultrasonic signals transmitted by the four sensors and the received ultrasonic signals. Similarly, four ultrasonic sensors 110 located on the left side of the vehicle 100 may be configured as an ultrasonic sensor module, and each of the four ultrasonic sensors 110 may be configured as an ultrasonic sensor module, and the processing unit may be configured to acquire various ultrasonic data based on the ultrasonic signals transmitted by the four sensors and the echo signals received. In another embodiment, all ultrasonic sensors 110 shown in FIG. 1 may be configured as an ultrasonic sensor module having a corresponding processing unit, and the processing unit may be configured to acquire various ultrasonic data based on the ultrasonic signals transmitted by the all ultrasonic sensors and the echo signals received.

[0021] Although the ultrasonic sensor and the ultrasonic sensor module are described separately above depending on the context, the ultrasonic sensor may refer to a sensing unit for transmitting ultrasonic signals and receiving echo signals in a narrow sense, or may refer to an ultrasonic sensor module in a broad sense. Those skilled in the art will be able to distinguish the meaning of the ultrasonic sensor in a specific context.

[0022] As described above, the processing unit may obtain various ultrasound data based on the ultrasound signals transmitted and the echo signals received by the ultrasound sensor 110. The ultrasound data may include various associated data.

[0023] In some embodiments, the ultrasonic data may include echo data. The echo data may include information related to the echo signal. For example, the echo data may include an echo timestamp, an echo amplitude, an echo significance, an echo distance, an echo height, an echo coordinate (e.g., having two coordinates), a sensor coordinate (e.g., having two coordinates), or the like of the echo signal. In some embodiments, the echo data may be acquired using a centerline method. For example, while the vehicle 100 is traveling along the direction 140, the ultrasonic sensor 110 located on the front right side of the vehicle 100 may transmit an ultrasonic signal and receive an echo signal. In the centerline method, a reflection point on a detected object is assumed to lie on the centerline 150 of the ultrasonic arc. Based on this, the processing unit may detect the reflection point of the corresponding ultrasonic signal and use it as the location of the echo signal. This location may be represented by, for example, the echo coordinate.

[0024] In some embodiments, the ultrasonic data may include echo intersection data. The echo intersection data may include information related to intersections between different echo signals. For example, the echo intersection data may include an echo intersection coordinate (e.g., two coordinates), an echo intersection distance, an adjacent echo intersection coordinate (e.g., two coordinates), an adjacent echo intersection height, an adjacent echo intersection distance, an adjacent echo intersection deflection, a sensor coordinate, etc. For example, while the vehicle 100 is traveling along the direction 140, an ultrasonic sensor 110 located on the front right side of the vehicle 100 may be configured to transmit an ultrasonic signal and receive an echo signal. The two sensors 110 may be configured to transmit two ultrasonic signals 120 and receive corresponding echo signals. Based on the known positional relationship between the two ultrasonic sensors 110, the transmission times of the two ultrasonic signals 120, the reception times of the corresponding echo signals, and other information, the processing unit may be configured to calculate an echo intersection 160 of the two ultrasonic signals 120, where the echo intersection 160 may represent a reflection point on a detected object. Therefore, the echo intersection data may include information about the intersection 160. In some embodiments, the echo intersection is not limited to the intersection of two echoes of ultrasonic signals transmitted by the two ultrasonic sensors. For example, the echo intersection may be the intersection of two echoes of an ultrasonic signal transmitted by the same ultrasonic sensor at different times during movement. Furthermore, the echo intersection data may be calculated based on the echo data. For example, the corresponding echo intersection data may be obtained based on the ultrasonic arcs of any two echo data within a specific range.

[0025] In some embodiments, the ultrasound data may comprise the echo data and / or echo intersection data described above, and optionally, the ultrasound data may also comprise other data related to the ultrasound signal, such as data related to the ultrasound signal acquired according to methods known or possible in the art.

[0026] In some embodiments, the processing unit of the ultrasonic sensor module may be configured to provide the acquired ultrasonic data to a control unit 130 of the vehicle 100. For example, the control unit 130 may be an Electronic Control Unit (ECU) of the vehicle. In some embodiments, some or all of the operations performed by the processing unit of the ultrasonic sensor module may also be performed by the control unit 130. For example, the control unit 130 may be configured to acquire echo data and / or echo intersection data based on the ultrasonic signals transmitted by the ultrasonic sensor 110 and the received echo signals. In some embodiments, the vehicle 100 may also include other processing units for performing these operations.

[0027] In some implementations, the sensors, processing unit, and control unit may be included in a vehicle control system. The vehicle control system may be configured to provide various controls for the vehicle. For ease of understanding, Figure 2 is a schematic diagram of an exemplary control system in a vehicle according to an embodiment of the present disclosure.

[0028] In the embodiment of Figure 2, the same reference numerals are used for the same components as those in Figure 1. Furthermore, it should be understood that Figure 2 shows only some components related to the technical solution of the present disclosure. In actual implementation, the vehicle control system may be configured with various other components that are not limited by the present disclosure.

[0029] In the example of FIG. 2, vehicle control system 200 may include ultrasonic sensor modules 105-1 through 105-N, each including one or more ultrasonic sensors 110 and corresponding processing units 115-1 through 115-N. As discussed above in conjunction with FIG. 1, in some implementations, a vehicle may include only one ultrasonic sensor module. For example, while an ultrasonic sensor module includes multiple ultrasonic sensors and processing units attached to vehicle 100, in different embodiments, a vehicle may include one or more ultrasonic sensor modules, each including one or more ultrasonic sensors and corresponding processing units.

[0030] The control system 200 may further include a control unit 130. The control unit 130 may be configured to control the operation of any one of the ultrasonic sensor modules 105-1 to 105-N. For example, the control unit 130 may be configured to control the ultrasonic sensor 110 of any one of the ultrasonic sensor modules 105-1 to 105-N to transmit an ultrasonic signal and receive an echo signal. The control unit 130 may be configured to receive ultrasonic data from any one of the ultrasonic sensor modules 105-1 to 105-N and perform further operation or vehicle control based on the ultrasonic data. As described above, some or all of the operations performed by the processing units of the ultrasonic sensor modules 105-1 to 105-N may be performed by the control unit 130 or another processing unit.

[0031] The control system 200 may further include a human-machine interface 180. The control unit 130 may be configured to output information that can be understood by a user (e.g., a driver) via the human-machine interface 180, or may be configured to receive information input by a user from the human-machine interface. In some embodiments, a user may be configured to input a selection regarding entering an assisted parking mode or an autonomous driving mode via the human-machine interface 180. Upon receiving a user input to enter the assisted parking mode or the autonomous driving mode, the control unit 130 may be configured to control the vehicle to operate in the assisted parking mode or the autonomous driving mode. For example, in the assisted parking mode, the vehicle is controlled to automatically find a parking space and park automatically. In another example, in the autonomous driving mode, the vehicle is controlled to plan a route and avoid obstacles. In some embodiments, the control unit 130 may be configured to automatically control the vehicle to enter the assisted / autonomous parking mode or the assisted / autonomous driving mode without user input. Regardless of which mode and how it is entered, control unit 130 may be configured to detect objects around the vehicle by processing the ultrasound data, such as determining object information such as object location, type, and height. Based on the object information, control unit 130 may be configured to perform various operations related to vehicle parking planning or route planning, such as detecting available parking spaces for the vehicle, planning a parking path for the vehicle based on the detected parking spaces, and planning a driving path for the vehicle based on detected obstacles.

[0032] Generally, to detect objects around a vehicle, it is necessary to use acquired ultrasound data to determine relevant information such as the object's location. In a real-world application scenario, many different types of objects may exist around the vehicle, such as other vehicles, ground lock signs, trees, rails, walls, curves, and pedestrians. In prior art, the same model is used to detect various different types of objects and predict relevant information such as the object's location. However, the model cannot distinguish between object types, and using the same model for multiple different object types may result in potential inaccuracies in location prediction. This limitation significantly affects the performance of vehicle assisted driving and parking and the user experience.

[0033] In the technical solution of the present disclosure, ultrasound data may be preprocessed using a clustering technique. For example, the ultrasound data may be clustered to obtain sample clusters corresponding to various objects, and the object type of each object may be determined based on the sample clusters. Then, a corresponding model may be selected based on the object type of each object, and the ultrasound data may be processed using a prediction technique to obtain position prediction information for each object. Compared with using the same model to detect multiple objects of different object types, selecting a corresponding model based on object type to detect objects of a similar object type may provide more accurate and reliable position prediction information. As a result, vehicle-assisted driving / parking performance and user experience may be significantly improved.

[0034] FIG. 3A is a schematic diagram of an application scenario according to an embodiment of the present disclosure.

[0035] 3A , for example, in a parking lot, an ultrasonic sensor 310 in an ultrasonic sensor module transmits ultrasonic signals 320 to acquire ultrasonic data while a vehicle 330 is moving in the direction of the arrow, thereby detecting objects around the vehicle based on the ultrasonic data. As shown in FIG. 3A , various types of objects may be present around the vehicle 330, such as other vehicles 340-1 and 340-2, ground locking mechanisms 350-1 and 350-2, and cylindrical objects 360-1 to 360-3. Other types of objects (not shown), such as rails and curves, may also be present around the vehicle 330. The acquired ultrasonic data may include characteristic information about the various objects, which may be used to predict the object's location, type, etc.

[0036] FIG. 3B is a schematic diagram of ultrasound data according to an embodiment of the present disclosure.

[0037] In the example shown in FIG. 3B, the collected ultrasound data is shown within a spatial range that may include echo data and echo intersection data. The spatial range shown in FIG. 3B may correspond to the spatial range shown in FIG. 3A. Each ultrasound data sample point represents an echo data sample point or an echo intersection data sample point, with the circular sample points representing echo data sample points and the triangular sample points representing echo intersection data sample points. Each echo sample point may include an echo timestamp, an echo amplitude, an echo significance, an echo distance, an echo height, an echo coordinate (e.g., having two coordinates), a sensor coordinate (e.g., having two coordinates), etc. of the echo signal. Each echo intersection data sample point may include an echo intersection coordinate (e.g., having two coordinates), an echo intersection distance, adjacent echo intersection coordinates (e.g., having two coordinates), adjacent echo intersection heights, adjacent echo intersection distances, adjacent echo intersection deflections, a sensor coordinate, etc. Those skilled in the art will understand that in addition to the characteristics of the echo data and echo intersection data exemplified above, the echo data and echo intersection data may also include other characteristics related to ultrasound signals, and all characteristics of the echo data and echo intersection data, such as those known in the art or that may be adopted in the future, may be applied to the technical solutions disclosed herein. Those skilled in the art will understand that in addition to the exemplary echo data and echo intersection data, the ultrasound data may also include other data related to ultrasound signals, such as data related to ultrasound signals acquired based on methods known in the art or possible methods in the future. Those skilled in the art will understand that FIG. 3B is merely a schematic diagram of ultrasound data sample points for illustrative purposes. In various practical applications, the number of ultrasound data sample points within a specific spatial range will be greater, and the distribution will be more complex.

[0038] In one embodiment, the ultrasonic data shown in FIG. 3B may be captured by an ultrasonic sensor during a period of vehicle movement. In another embodiment, the ultrasonic data shown in FIG. 3B may be captured by an ultrasonic sensor during a distance of vehicle movement. For example, the ultrasonic data shown in FIG. 3B may be obtained by fusing data captured by the ultrasonic sensor during multiple sliding windows based on a sliding window mechanism. The size of the sliding window may be fixed or variable, and the size of the sliding window corresponds to the actual spatial extent covered by the ultrasonic signal 320.

[0039] 3B may be cached in a buffer or memory of a vehicle's processing system, and the control unit 130 or other vehicle processing unit may be configured to perform subsequent processing on the cached ultrasound data. In one embodiment, the control unit 130 or other vehicle processing unit may be configured to perform subsequent processing on the cached ultrasound data to determine the location and / or type of the object. In one embodiment, the control unit 130 or other vehicle processing unit may be configured to plan a vehicle driving path based on the determined location and / or type of the object, such as to assist in parking or driving the vehicle.

[0040] 4 is a schematic diagram of a method and modules for determining surrounding objects of a vehicle according to an embodiment of the present disclosure, which may include a clustering module 410, a feature extraction module 420, a classification module 430, a model selection module 440, and a location prediction module 450.

[0041] 4, the ultrasound data shown in FIG. 3B may be used as input to the clustering module 410, which performs clustering based on at least a portion of the ultrasound data in the ultrasound dataset to obtain sample clusters C1 to C7. In one embodiment, the clustering module 410 may be configured to cluster only one of the ultrasound echo data and the ultrasound echo intersection data. In another embodiment, the clustering module 410 may be configured to cluster both the ultrasound echo data and the ultrasound echo intersection data. Because the density of the ultrasound echo intersection data is high and the distance between different ultrasound echo intersection data is larger, clustering based on only the ultrasound echo intersection data may have certain advantages over clustering based on both the ultrasound echo data and the ultrasound echo intersection data, which helps to obtain better clustering results and facilitate noise removal.

[0042] In one embodiment, the clustering module 410 may be configured to cluster at least a portion of the ultrasound data using Density-Based Spatial Clustering of Applications with Noise (DBSCAN). DBSCAN is a density-based clustering algorithm that defines clusters as maximal sets of density-connected points, divides sufficiently dense regions into clusters, and can find clusters of any shape in a noisy spatial database. DBSCAN does not require prior knowledge of the number of clusters to be formed, can identify clusters of any shape, and can detect noise points. Considering the complex environment that the technical solution of the present disclosure may encounter in a specific application scenario, for example, a parking lot may contain obstacle areas of various shapes and sizes, the collected ultrasound data may contain various noises, and the number of objects in a spatial range may be zero or more obstacles. Therefore, the above-mentioned features of DBSCAN are particularly well suited to the technical solution of the present disclosure. DBSCAN is a clustering method known in the art. Therefore, specific details thereof will not be described in detail. It should be understood that other suitable clustering methods may also be used in the clustering module 410, and the technical solutions of the present disclosure are not limited to DBSCAN.

[0043] A cluster ultrasound data set may include zero or more sample clusters. Zero sample clusters in a spatial range indicates no objects in the vehicle's current surroundings. If one or more sample clusters are included in a spatial range, each sample cluster may correspond to an object in the vehicle's surroundings. As shown in FIG. 4, sample clusters C1 and C7 may correspond to vehicles 340-1 and 340-2 shown in FIG. 3A, sample clusters C2, C4, and C6 may correspond to cylindrical objects 360-1 through 360-3 in FIG. 3A, and sample clusters C3 and C5 may correspond to ground locking mechanisms 350-1 and 350-2 in FIG. 3A.

[0044] In the example shown in Figure 4, ultrasound data corresponding to one or more sample clusters may be used as input to classification module 430, which may be configured to determine an object type corresponding to each sample cluster based on the ultrasound data corresponding to each sample cluster and output a label capable of representing the object type corresponding to each sample cluster. As shown in Figure 4, sample clusters C1 and C7 may have a corresponding label L1 representing the object type "vehicle," sample clusters C2, C4, and C6 may have a corresponding label L2 representing the object type "cylindrical," and sample clusters C3 and C5 may have a corresponding label L3 representing the object type "ground lock mechanism." In other embodiments, various object types and labels are possible.

[0045] In one embodiment, the classification module 430 may be configured to use an eXtreme Gradient Boosting (XG Boost) model to determine the object type corresponding to each sample cluster based on a classification algorithm. The XG Boost model is a gradient boosted tree algorithm that repeatedly trains multiple decision trees and gradually improves model performance through gradient boosting. XG Boost obtains sample classification results by training a decision tree for each category separately and combining the outputs of each subtree to effectively handle multi-classification tasks. Considering that the technical solution of the present disclosure may include multiple types of objects in a specific application scenario, the above-mentioned characteristics of the XG Boost model are particularly suitable for the technical solution of the present disclosure. Since the XG Boost model is a classification method known in the art, its specific details will not be described in detail. It can be understood that other suitable classification methods may also be used in the classification module 430, and the technical solution of the present disclosure is not limited to the XG Boost model.

[0046] Optionally, the ultrasound data corresponding to one or more sample clusters may be input to the feature extraction module 420 before being input to the classification module 430, and the extracted features associated with the ultrasound signals of each sample cluster may also be used as input to the classification module 430.

[0047] In one embodiment, the feature extraction module 420 may be configured to determine features associated with the ultrasound signals of each sample cluster based on the entire ultrasound data corresponding to the sample cluster.

[0048] In other embodiments, the feature extraction module 420 may be configured to determine a first subset of ultrasound data corresponding to a first edge of the object and a second subset of ultrasound data corresponding to a second edge of the object based on the entire ultrasound data corresponding to each sample cluster, and to determine features associated with the ultrasound signals of the sample cluster based on the first subset of ultrasound data and the second subset of ultrasound data.

[0049] Taking vehicle 340-2 shown in FIG. 3A as an example, its position may be represented by a bounding box (shown by a dotted line). To determine the position of vehicle 340-2 relative to vehicle 330, the positions of two corner points A and B closest to both the bounding box and vehicle 330 are particularly important. Therefore, compared to all ultrasound data corresponding to vehicle 340-2, ultrasound data at the two ends of sample cluster C7 corresponding to corner points A and B of vehicle 340-2 are more important for accurately determining the positions of these corner points. For example, for the sample cluster corresponding to vehicle 340-2, a first ultrasound data subset may include ultrasound data associated with a left first end of sample cluster C7 corresponding to corner point A, and a second ultrasound data subset may include ultrasound data associated with a right second end of sample cluster C7 corresponding to corner point B.

[0050] The first and second ultrasound data subsets may be determined in various ways. For example, different data sampling ratios may be determined based on different bounding box sizes. For example, a first ratio value of 100% may be used for a bounding box size of 1.5 meters in the drive direction, and a second ratio value of 50% may be used for a bounding box size of 3 meters in the drive direction. Data sampling ratios may be applied to different dimensions, such as the number of samples, distance from the endpoint, etc.

[0051] In one embodiment, for each sample cluster, statistics of the ultrasound data features in the sample cluster extracted by feature extraction module 420 may be used as features of the sample cluster. In another embodiment, statistics of the ultrasound data features in a first ultrasound data subset extracted by feature extraction module 420 may be used as a first feature subset of the sample cluster, and statistics of the ultrasound data features in a second ultrasound data subset extracted by feature extraction module 420 may be used as a second feature subset of the sample cluster. Then, both the first feature subset and the second feature subset may be used as features of the sample cluster.

[0052] For example, the extracted features may include one or more of echo amplitude, echo intensity, echo distance, echo intersection coordinate, distance from echo intersection to sensor, echo height, echo intersection height, sensor coordinate, angle to sensor, etc. To determine coordinates of the corner point locations, such as planar coordinates, the same or different features may be extracted for each horizontal and vertical coordinate of each corner point location, which may depend on the implementation.

[0053] For example, the statistics used may include one or more of the mean, minimum, maximum, variance, quantile, and kurtosis. For example, the same or different statistics may be used based on features extracted from the horizontal and vertical coordinates of each corner point location, respectively. This may depend on the implementation.

[0054] Returning to FIG. 4, in the example shown in FIG. 4, the ultrasound data corresponding to one or more sample clusters along with the object type determined by the classification module 430 may be used as input to the model selection module 440, which determines the predictive model to use for each sample cluster based on the object type corresponding to each sample cluster.

[0055] In one embodiment, a prediction model set includes multiple prediction models, each pre-trained for a particular type of object. For example, each prediction model may have the same or different structure and parameters and / or may be based on the same or different algorithms. Using separate prediction models for specific types of objects may enhance location prediction by taking advantage of unique characteristics of each object type, resulting in improved accuracy compared to a unified prediction model that predicts locations for multiple object types.

[0056] 4, ultrasound data corresponding to one or more sample clusters may be used as input to a location prediction module 450, which outputs location prediction information for one or more objects. For each sample cluster, the location prediction module 450 is implemented based on the prediction model to be used for each sample cluster determined by the model selection module 440.

[0057] In one embodiment, for a single sample cluster, the location prediction module 450 may be configured to use an eXtreme Gradient Boost (XG Boost) model based on a regression algorithm to determine a location prediction for an object corresponding to the sample cluster. The XG Boost model is a gradient boosted tree algorithm that repeatedly trains multiple decision trees, each of which acts as a weak learner. The model's performance is gradually improved through gradient boosting, and these weak learners are eventually combined into a strong learner. The XG Boost model's features of preventing overfitting and improving generalization are particularly suitable for the technical solution presented in this disclosure. The XG Boost model is a regression method known in the art, and therefore, its specific details will not be described in detail. It can be understood that other suitable prediction methods may also be used in the location prediction module 450, and the technical solution of the present disclosure is not limited to the XG Boost model.

[0058] Optionally, the features of one or more sample clusters extracted by feature extraction module 420 may also be used as input to location prediction module 450 to output location prediction information for one or more objects.

[0059] In one embodiment, the output position prediction information of the object may be characterized by the positions of two corner points of the object, such as the positions of corner points A and B of vehicle 340-2 shown in FIG. 3A . For example, the position prediction information of the object may include the coordinates of a first corner point and a second corner point of the object. Furthermore, the position prediction information of the object may also include height information of the object. For example, ultrasonic signals may detect curves around the vehicle. However, if the curvature is small, the impact on parking may be minimal. Therefore, the height information of the object may also be useful for parking or driving assistance. Furthermore, the output may include other information, such as the distance from the object to the vehicle and the object type determined by the classification module 430.

[0060] In the technical solutions described in conjunction with specific embodiments, the object type of each object is determined by the sample cluster obtained based on clustering, and a corresponding model is selected based on the object type of each object to predict the location information of the object. Compared with using the same model to detect objects of multiple different object types, selecting a corresponding model based on object type to detect objects of a similar object type may provide more accurate and reliable location prediction information, thereby improving vehicle-assisted driving / parking performance and user experience.

[0061] FIG. 5 is a flowchart of a method for determining objects around a vehicle according to an embodiment of the present disclosure.

[0062] At step 510, clustering may be performed based on at least a portion of the ultrasound data in the ultrasound dataset to obtain one or more sample clusters, each sample cluster corresponding to an object in the vehicle's surroundings.

[0063] In one example, the ultrasonic data set may be captured by an ultrasonic sensor while the vehicle is moving for a period of time, and / or the ultrasonic data set may be captured by an ultrasonic sensor while the vehicle is moving for a period of time, and the number of ultrasonic sensors may vary from one sensor to multiple sensors.

[0064] In one embodiment, the ultrasound data set may include ultrasound echo data and ultrasound echo intersection data. The ultrasound echo data may include one or more of an echo coordinate, an echo height, an echo amplitude, an echo intensity, or an echo distance for each ultrasound signal. The ultrasound echo intersection data may include one or more of an echo intersection coordinate, an echo intersection height, an echo intersection distance, an echo intersection deflection, an adjacent echo intersection coordinate, an adjacent echo intersection height, an adjacent echo intersection distance, an adjacent echo intersection deflection, and a sensor coordinate.

[0065] In one embodiment, one or more of the ultrasound echo data and ultrasound echo intersection data in the ultrasound dataset may be clustered.

[0066] In one embodiment, clustering at least a portion of the ultrasound data in the ultrasound dataset may be implemented by density-based spatial clustering of noisy applications (DBSCAN).

[0067] In step 520, a feature associated with the ultrasound signal of each sample cluster may optionally be extracted based on the ultrasound data corresponding to one or more sample clusters.

[0068] In one embodiment, features associated with the ultrasound signals of the sample clusters may be determined based on the entire ultrasound data corresponding to each sample cluster, while in other embodiments, a first subset of ultrasound data corresponding to a first end and a second subset of ultrasound data corresponding to a second end may be determined based on the entire ultrasound data corresponding to each sample cluster, and features associated with the ultrasound signals of the sample clusters may be determined based on the first subset of ultrasound data and the second subset of ultrasound data.

[0069] For example, the first ultrasonic data subset and the second ultrasonic data subset may correspond to two corner points of a bounding box along the horizontal driving direction of the vehicle. The first ultrasonic data subset and the second ultrasonic data subset may be determined in various ways. For example, different data extraction ratios may be determined based on different sizes of the bounding boxes. For example, a first ratio value of 100% may be used for a bounding box size of 1.5 meters in the driving direction, and a second ratio value of 50% may be used for a bounding box size of 3 meters in the driving direction. The data extraction ratio may be applied to different dimensions, such as the number of samples, the distance from the endpoint, etc.

[0070] For example, the extracted features may include one or more of echo amplitude, echo intensity, echo distance, echo intersection coordinate, distance from echo intersection to sensor, echo height, echo intersection height, sensor coordinate, angle to sensor, etc. To determine the coordinates of the corner point locations, the same or different features may be extracted for the horizontal and vertical coordinates of each corner point location.

[0071] For example, statistics may be calculated based on the ultrasound data corresponding to each sample cluster, and features associated with the ultrasound signals of the sample cluster may be determined based on the calculated statistics. The statistics used may include one or more of the mean, minimum, maximum, variance, quantile, and kurtosis. For example, the same or different statistics may be used based on features extracted from the horizontal and vertical coordinates of each corner point location, respectively.

[0072] In step 530, an object type corresponding to each sample cluster may be determined based on ultrasound data corresponding to one or more sample clusters.

[0073] In one embodiment, features associated with the ultrasound signals of each sample cluster extracted in step 520 may be provided as feature data to a second machine learning model to output an object type corresponding to each sample cluster. For example, the second machine learning model may be implemented based on a classification algorithm using an eXtreme Gradient Boosting (XG Boost) model. Furthermore, the second machine learning model may be pre-trained in a supervised manner on multiple sample clusters corresponding to multiple object types.

[0074] In step 540, the ultrasound data and the object type corresponding to each sample cluster may be provided as feature data to a corresponding first machine learning model in a first machine learning model set selected from the first machine learning model set based on the object type corresponding to the sample cluster, to output object location prediction information.

[0075] In one embodiment, the features associated with the ultrasound signals of each sample cluster extracted in step 520 may be provided as feature data to a corresponding first machine learning model to output position prediction information for the object corresponding to each sample cluster.

[0076] In one embodiment, the first machine learning model may be implemented using an eXtreme Gradient Boosting (XGBoost) model based on a regression algorithm. For example, the first machine learning model may be trained in a supervised manner based on ultrasound data corresponding to a sample cluster corresponding to a single object type.

[0077] In one embodiment, the output position prediction information of the object may include one or more of the coordinates of the first and second corner points of the object, height information of the object, and the distance from the object to the vehicle. In addition, object type information may also be output.

[0078] FIG. 6 is a flowchart for training a machine learning model for determining objects around a vehicle, according to an embodiment of the present disclosure.

[0079] In step 610, the training data may be configured to be input into the machine learning model to be trained.

[0080] In step 620, the machine learning model being trained may be configured to be iteratively updated based on a loss function until an end-of-training condition is reached.

[0081] In one embodiment, the machine learning model to be trained may be a classification model, such as the specific embodiment of classification module 430 described in FIG. 4 and / or the second machine learning model described in FIG. 5. The machine learning model may be trained in a supervised manner for multiple object types. For example, the training data set may include {ultrasound data corresponding to sample clusters, object types corresponding to the sample clusters}. Additionally or alternatively, the training data set may include {features associated with ultrasound signals of ultrasound data corresponding to the sample clusters, object types corresponding to the sample clusters}. For example, the machine learning model may be implemented based on a classification algorithm using an eXtreme Gradient Boosting (XG Boost) model.

[0082] In one embodiment, the machine learning model to be trained may be a regression model, such as the specific embodiment of the location prediction module 450 described in FIG. 4 and / or the first machine learning model described in FIG. 5. The machine learning model may be trained in a supervised manner for a single object type, or a set of multiple machine learning models may be trained for multiple object types, and the multiple machine learning models may have the same or different structures and parameters and / or use the same or different algorithms. For example, the training data set may include {ultrasound data corresponding to the sample clusters and location information of corresponding objects in the sample clusters} corresponding to sample clusters. Additionally or alternatively, the training data set may include {features associated with the ultrasound signals of the ultrasound data corresponding to the sample clusters and location information of corresponding objects in the sample clusters} or {statistics of features associated with the ultrasound signals of the ultrasound data corresponding to the sample clusters and location information of corresponding objects in the sample clusters}. For example, the machine learning model may be implemented based on a regression algorithm using an eXtreme Gradient Boosting (XG Boost) model.

[0083] FIG. 7 is a flowchart of a method for assisting vehicle driving according to an embodiment of the present disclosure.

[0084] In step 710, the method may include determining, based on the ultrasonic signals, location prediction information for objects around the vehicle. In particular, the method of each embodiment of the present disclosure may be used to perform step 710.

[0085] In step 720, the method may include assisting the vehicle in planning a driving route based on the output position prediction information of the object.

[0086] For example, available parking spaces for a vehicle may be detected based on predicted object location information, and a route for the vehicle to enter the parking space may be planned. In other embodiments, obstacles may be detected based on predicted object location information, and a vehicle route may be planned to avoid the obstacles.

[0087] FIG. 8 is a block diagram of a control system for a vehicle according to an example of the present disclosure.

[0088] Control system 800 may include one or more processors 810 and memory 820. Memory 820 may store executable instructions. Processor 810 may execute the executable instructions stored or coded in memory 820, thereby performing the various operations and / or functions described above in conjunction with Figures 1-7. Although not shown in Figure 8, those skilled in the art will appreciate that control system 800 may include various other components, such as various communication modules, bus modules, and possibly user interface modules.

[0089] In some embodiments, the control system 800 may include the control unit 130 and / or the processing unit 115 shown in FIGS.

[0090] Additionally, embodiments of the present disclosure provide a vehicle. The vehicle may be configured with an ultrasonic sensor, such as that shown in Figure 1, used to transmit and receive ultrasonic signals. The vehicle may also be configured with a control system 800, as shown in Figure 8.

[0091] Embodiments of the present disclosure also provide a machine-readable storage medium that may be configured to store executable instructions that, when executed by a processor, perform the various operations and / or functions described above in conjunction with Figures 1-5. For example, the computer-readable storage medium may include, but is not limited to, random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), static random access memory (SRAM), a hard disk, flash memory, etc.

[0092] Embodiments of the present disclosure also provide a computer program product, which may comprise a computer program that, when executed by a processor, may perform various operations and / or functions described above in conjunction with FIGS.

[0093] The foregoing embodiments of the present disclosure have been described above. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the examples and still achieve desirable results. Moreover, the processes depicted in the figures do not necessarily require a particular or sequential order to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or advantageous.

[0094] Not all steps and units shown in the above flowcharts and system diagrams are required. Certain steps or units may be omitted based on actual needs. The device structure described in the above embodiments may be a physical or logical structure. That is, some units may be realized by the same physical entity, while other units may be realized by multiple physical entities, or may be realized together by specific components in multiple separate devices.

[0095] Throughout this description, the term "exemplary" means "serving as an embodiment, example, or illustration," and does not imply "preferred" or "advantageous" over other embodiments. Particular embodiments include specific details to facilitate an understanding of the described technology. However, these technologies may be practiced without these specific details. In some instances, well-known structures and devices are shown in block diagram form in order to avoid obscuring the concepts of the described embodiments.

[0096] The preceding description of the present disclosure is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to the present disclosure will be apparent to those skilled in the art, and the general principles defined in the present disclosure may be applied to other modifications without departing from the scope of protection of the present disclosure. Therefore, the present disclosure is not limited to the exemplary embodiments and designs described in the present disclosure, but is accorded the widest scope defined by the principles and novel configurations disclosed in the present disclosure.

Claims

1. 1. A method for determining objects around a vehicle based on ultrasonic signals, comprising: clustering based on at least a portion of the ultrasound data in the ultrasound dataset to obtain one or more sample clusters, each sample cluster corresponding to an object in the vehicle's surroundings; determining an object type corresponding to each sample cluster based on the ultrasound data corresponding to the one or more sample clusters; providing the ultrasound data and the object type as feature data corresponding to each sample cluster to a corresponding first machine learning model in a first machine learning model set to output position prediction information for the object, wherein the corresponding first machine learning model corresponding to each sample cluster is selected from the first machine learning model set based on the object type corresponding to the sample cluster; A method comprising:

2. 2. The method of claim 1, wherein the ultrasonic data set is captured by an ultrasonic sensor as the vehicle moves over a period of time and / or the ultrasonic data set is captured by an ultrasonic sensor as the vehicle moves over a distance.

3. The method of claim 1 , wherein the ultrasound data sets include ultrasound echo data and ultrasound echo intersection data.

4. The method of claim 3 , wherein clustering based on at least a portion of the ultrasound data in the ultrasound dataset comprises clustering based solely on the ultrasound echo intersection data.

5. The ultrasound echo data includes one or more of echo coordinates, echo height, echo amplitude, echo intensity, or echo distance for each ultrasound signal; and / or 4. The method of claim 3, wherein the ultrasound echo intersection data includes one or more of an echo intersection coordinate, an echo intersection height, an echo intersection distance, an echo intersection deflection, an adjacent echo intersection coordinate, an adjacent echo intersection height, an adjacent echo intersection distance, an adjacent echo intersection deflection, and a sensor coordinate.

6. 2. The method of claim 1, wherein clustering based on at least a portion of the ultrasound data in the ultrasound dataset comprises clustering at least a portion of the ultrasound data in the ultrasound dataset by Density-Based Spatial Clustering of Applications with Noise (DBSCAN).

7. extracting features associated with the ultrasound signal of each sample cluster based on the ultrasound data corresponding to the one or more sample clusters; determining a feature associated with the ultrasound signal of each sample cluster based on the entirety of the ultrasound data corresponding to the sample cluster; or determining a first subset of ultrasound data corresponding to a first end and a second subset of ultrasound data corresponding to a second end based on the entire ultrasound data corresponding to each sample cluster; and determining features associated with the ultrasound signals of the sample cluster based on the first subset of ultrasound data and the second subset of ultrasound data. The method of claim 1 further comprising:

8. Extracting the features associated with the ultrasound signal of each sample cluster based on the ultrasound data corresponding to the one or more sample clusters includes: calculating statistics based on the ultrasound data corresponding to each sample cluster; determining the features associated with the ultrasound signals of the sample clusters based on the calculated statistics; and The method of claim 7, comprising:

9. Determining the object type corresponding to each sample cluster based on the ultrasound data corresponding to the one or more sample clusters includes:

2. The method of claim 1, comprising providing the features associated with the ultrasound signal of each sample cluster as feature data to a second machine learning model to output the object type corresponding to each sample cluster.

10. 10. The method of claim 9, wherein the second machine learning model is implemented based on a classification algorithm using an eXtreme Gradient Boosting (XG Boost) model.

11. The method of claim 10 , wherein the second machine learning model is trained in a supervised manner based on ultrasound data corresponding to sample clusters corresponding to multiple object types.

12. 2. The method of claim 1, wherein the first machine learning model of the first set of machine learning models is implemented based on a regression algorithm using an eXtreme Gradient Boosting (XG Boost) model.

13. 13. The method of claim 12, wherein the first machine learning model of the first set of machine learning models is trained in a supervised manner based on ultrasound data corresponding to a sample cluster corresponding to a single object type.

14. The output of the position prediction information of the object is outputting the coordinates of a first corner point of the object and the coordinates of a second corner point of the object; outputting height information of the object; and / or outputting the distance from the object to the vehicle; The method of claim 1 , comprising:

15. The method of claim 1 , further comprising outputting the object type prediction information.

16. The method of claim 1 , further comprising assisting the vehicle in planning a driving route based on the output position prediction information of the object.

17. 1. A control system for a vehicle, comprising: at least one processor; a memory coupled to the at least one processor, the memory storing executable instructions that, when executed by the at least one processor, enable the at least one processor to perform the method of any one of claims 1 to 16; and A control system comprising:

18. 17. A computer readable medium storing a computer program comprising instructions which, when executed by a processor, enable one or more units to perform the method of any one of claims 1 to 16.

19. A computer program product comprising a computer program which, when executed by a processor, performs the method of any one of claims 1 to 16.

20. A vehicle, an ultrasonic sensor for transmitting and receiving ultrasonic signals; one or more units for carrying out the method according to any one of claims 1 to 16; A vehicle equipped with:

Images