Method and device for determining an object in the environment of a vehicle

Abstract

The present application relates to a method for determining an object in the surroundings of a vehicle based on an ultrasonic signal, comprising: clustering based on at least part of ultrasonic data in a set of ultrasonic data to obtain one or more sample clusters, each sample cluster corresponding to an object in the surroundings of the vehicle; determining an object type corresponding to each sample cluster based on the ultrasonic data corresponding to the one or more sample clusters; and providing the ultrasonic data corresponding to each sample cluster and the object type as feature data to a corresponding first machine learning model in a first machine learning model set so as to output position prediction information of the object, the respective first machine learning model for each sample cluster is selected from the first set of machine learning models based on the object type corresponding to that sample cluster.

Description

Technical Field

[0001] This application relates to vehicle automatic control, and more specifically, to methods, control systems, vehicles, machine-readable storage media, and computer program products for determining objects in the surrounding environment of a vehicle based on ultrasonic signals. Background Technology

[0002] Accurately identifying objects in the vehicle's surrounding environment is a crucial step in realizing various vehicle assistance functions in applications such as autonomous driving systems, automated parking systems, and driver assistance systems. For example, sensor signals can be used to detect objects in the vehicle's surrounding environment to detect obstacles and adjust the driving route in a timely manner, or to detect obstacles and accurately determine parking spaces in a parking environment.

[0003] Ultrasonic sensors (USS) offer a low-cost method for object detection; however, due to the low resolution of data captured by ultrasonic radar, conventional methods using ultrasonic signals for detection are not effectively applicable to complex scenarios. Furthermore, various vehicle applications involve a variety of objects, necessitating improved accuracy in ultrasonic signal-based object detection to enhance the performance of vehicle driver assistance and parking functions. Summary of the Invention

[0004] The following brief introduction is provided to present some of the selected concepts in a simplified manner, which will be further described in the detailed description that follows. This brief introduction is not intended to highlight the key or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter.

[0005] The aim is to provide a method for determining objects in the surrounding environment of a vehicle based on ultrasonic signals. This method can select an appropriate model to predict the location of objects based on the object type in various vehicle driving scenarios, so as to have accuracy and robustness in object detection. This method can be applied to complex road or parking scenarios and help improve the performance of vehicle assisted driving and parking functions.

[0006] On one hand, embodiments of this disclosure provide a method for determining objects in the surrounding environment of a vehicle based on ultrasonic signals, comprising: clustering at least a portion of ultrasonic data in an ultrasonic dataset to obtain one or more sample clusters, wherein each sample cluster corresponds to an object in the surrounding environment of the vehicle; determining an object type corresponding to each sample cluster based on ultrasonic data corresponding to the one or more sample clusters; and providing the ultrasonic data corresponding to each sample cluster and the object type as feature data to a corresponding first machine learning model in a first machine learning model set to output location prediction information for the object, wherein the corresponding first machine learning model for each sample cluster is selected from the first machine learning model set based on the object type corresponding to that sample cluster.

[0007] On the other hand, embodiments of this disclosure provide a control system for a vehicle, including: at least one processor; and a memory coupled to the at least one processor, thereon storing executable instructions that, when executed by the at least one processor, cause the at least one processor to implement the method according to any embodiment of this disclosure.

[0008] On the other hand, embodiments of this disclosure provide a computer-readable medium storing a computer program including instructions that, when executed by a processor, cause one or more units to perform the method described according to any embodiment of this disclosure.

[0009] On the other hand, embodiments of this disclosure provide a computer program product including a computer program that, when executed by a processor, implements the method described according to any embodiment of this disclosure.

[0010] On the other hand, embodiments of this disclosure provide a vehicle comprising: an ultrasonic sensor for emitting and receiving ultrasonic signals; and one or more units for performing the method according to any embodiment of this disclosure. Attached Figure Description

[0011] A further understanding of the nature and advantages of this disclosure can be achieved by referring to the accompanying drawings. In the drawings, similar components or features may have the same reference numerals.

[0012] Figure 1 A schematic diagram of an exemplary vehicle according to an embodiment of the present disclosure is shown.

[0013] Figure 2 A schematic diagram of an exemplary control system in a vehicle according to an embodiment of the present disclosure is shown.

[0014] Figure 3AA schematic diagram illustrating an application scenario based on an embodiment of this disclosure is shown.

[0015] Figure 3B A schematic diagram of ultrasound data according to an embodiment of the present disclosure is shown.

[0016] Figure 4 A schematic diagram of a method and modules for determining objects in the environment surrounding a vehicle, according to an embodiment of the present disclosure, is shown.

[0017] Figure 5 A flowchart illustrating a method for determining objects in the environment surrounding a vehicle, according to an embodiment of this disclosure, is shown.

[0018] Figure 6 A flowchart illustrating an embodiment of the present disclosure is shown for training a machine learning model for determining objects in the environment surrounding a vehicle.

[0019] Figure 7 A flowchart of a method for assisting vehicle driving according to an embodiment of the present disclosure is shown.

[0020] Figure 8 A block diagram of a control system for a vehicle according to an embodiment of the present disclosure is shown. Detailed Implementation

[0021] The subject matter described herein will now be discussed with reference to exemplary embodiments. It should be understood that these embodiments are discussed merely to enable those skilled in the art to better understand and implement the subject matter described herein, and are not intended to limit the scope, applicability, or examples set forth in the claims. The function and arrangement of the elements discussed may be changed without departing from the scope of this disclosure. Various processes or components may be omitted, substituted, or added as needed in the various examples. For example, the described methods may be performed in a different order than described, and steps may be added, omitted, or combined. Furthermore, features described in some examples may be combined in other examples.

[0022] As used herein, the term "comprising" and its variations are open terms meaning "including but not limited to". The term "based on" means "at least partially based on". The terms "one embodiment" and "an embodiment" mean "at least one embodiment". The term "another embodiment" means "at least one other embodiment". The terms "first", "second", etc., may refer to different or the same objects. Unless explicitly indicated by the context, the definition of a term remains consistent throughout the specification.

[0023] Due to their advantages such as low cost, rapid response, and ease of integration, ultrasonic sensors have received widespread attention and research in the vehicle field. However, how to effectively utilize ultrasonic sensors for object detection in the vehicle field remains one of the challenges. Therefore, embodiments of this disclosure provide a technical solution for determining objects in the surrounding environment of a vehicle based on ultrasonic signals. These will be described in detail below with reference to specific embodiments.

[0024] Figure 1 A schematic diagram of an exemplary vehicle according to an embodiment of this disclosure is shown. It should be understood that the following examples are only intended to help better understand this disclosure and do not constitute any limitation on the scope of this disclosure.

[0025] exist Figure 1 In the example, at least one ultrasonic sensor 110 can be installed on vehicle 100 (in Figure 1 (Simply represented as black dots in Chinese). For example, as shown below. Figure 1 As shown, the ultrasonic sensor 110 can be installed on the front, rear, left, and right sides of the vehicle 100, corresponding to the front, rear, left, and right sides of the vehicle 100, respectively. Figure 1 Sixteen ultrasonic sensors are shown. However, the number and installation location of the ultrasonic sensors 110 are not limited to... Figure 1 As shown, in different implementations, vehicle 100 may include more or fewer ultrasonic sensors, and the mounting locations of the ultrasonic sensors may also vary.

[0026] The ultrasonic sensor 110 can emit ultrasonic signals toward the surrounding environment of the vehicle 100. For example, two ultrasonic sensors 110 located on the right front side of the vehicle 100 can emit ultrasonic signals 120. The following explanation uses ultrasonic signal 120 as an example. During propagation, the ultrasonic signal 120 may encounter various objects, such as obstacles (e.g., other vehicles, parking locks, trees, railings, walls, etc.), objects related to the vehicle's driving operation (e.g., curbs), pedestrians, etc. When encountering an object, the ultrasonic signal 120 may be reflected. After the ultrasonic signal 120 is reflected, the ultrasonic sensor 110 can receive the reflected signal (often referred to as the echo signal).

[0027] Various ultrasonic data can be obtained based on the transmitted ultrasonic signals and the received echo signals. These operations can be implemented in various ways. For example, in some implementations, vehicle 100 may include an ultrasonic sensor module. The ultrasonic sensor module may include an ultrasonic sensor 110 and a processing unit. The ultrasonic sensor 110 may include a transmitter that transmits ultrasonic signals and a receiver that receives echo signals. The processing unit can obtain various ultrasonic data based on the transmitted ultrasonic signals and the received echo signals. Furthermore, the processing unit can also control the transmission and reception of signals by each ultrasonic sensor.

[0028] For example, in Figure 1 In the middle, the two ultrasonic sensors 110 located on the right front side of the vehicle 100 can constitute an ultrasonic sensor module 105, which includes two ultrasonic sensors for transmitting and receiving ultrasonic signals and a processing unit (in Figure 1 (Not explicitly shown in the text). This processing unit can obtain various ultrasonic data based on the ultrasonic signals emitted and the echo signals received by the two ultrasonic sensors. Similarly, the two ultrasonic sensors 110 located on the right rear side, the two ultrasonic sensors 110 on the left front side, the two ultrasonic sensors 110 on the left rear side, the four ultrasonic sensors 110 on the front side, and the four ultrasonic sensors 110 on the rear side of the vehicle 100 can all form an ultrasonic sensor module with the corresponding processing unit.

[0029] Alternatively, for example, a single ultrasonic sensor 110 of vehicle 100 can be combined with a corresponding processing unit to form an ultrasonic sensor module, which can obtain various ultrasonic data based on the ultrasonic signals emitted and received by the single sensor. Another example is in... Figure 1 In this system, the four ultrasonic sensors 110 located on the right side of the vehicle 100 and their corresponding processing units can constitute an ultrasonic sensor module. This processing unit can obtain various ultrasonic data based on the ultrasonic signals emitted and the echo signals received by these four sensors. Similarly, the four ultrasonic sensors 110 located on the left side of the vehicle 100 can each form an ultrasonic sensor module with their corresponding processing units. For example, Figure 1 All the ultrasonic sensors 110 shown can be combined with corresponding processing units to form an ultrasonic sensor module, which can obtain various ultrasonic data based on the ultrasonic signals emitted and the echo signals received by all these ultrasonic sensors.

[0030] Although ultrasonic sensors and ultrasonic sensor modules have been described separately above, in this document, depending on the context, an ultrasonic sensor can be narrowly defined as a sensing unit used to transmit ultrasonic signals and receive echo signals, or broadly defined as an ultrasonic sensor module. Those skilled in the art will be able to distinguish the meaning of an ultrasonic sensor in a specific context.

[0031] As mentioned earlier, the processing unit can obtain various ultrasonic data based on the ultrasonic signals emitted and received by the ultrasonic sensor 110. The ultrasonic data can include a wide variety of related data.

[0032] In some embodiments, ultrasonic data may include echo data. Echo data may include information related to the echo signal. For example, echo data may include the echo timestamp, echo amplitude, echo significance, echo distance, echo height, echo coordinates (e.g., may include two coordinates), sensor coordinates (e.g., may include two coordinates), etc. In some embodiments, the centerline method can be used to obtain echo data. For example, as vehicle 100 travels along direction 140, ultrasonic sensor 110 located on the right front side of vehicle 100 can emit ultrasonic signals and receive echo signals. In the centerline method, it is assumed that the reflection point on the detected object is on the centerline 150 of the ultrasonic arc. Based on this, the processing unit can detect the reflection point of the corresponding ultrasonic signal as the position of the echo signal, which can be represented, for example, by echo coordinates.

[0033] In some embodiments, ultrasonic data may include echo intersection data. Echo intersection data may include information related to the intersection points between different echo signals. For example, echo intersection data may include echo intersection coordinates (e.g., may include two coordinates), echo intersection distance, adjacent echo intersection coordinates (e.g., may include two coordinates), adjacent echo intersection height, adjacent echo intersection distance, adjacent echo intersection deflection angle, sensor coordinates, etc. For example, during the travel of vehicle 100 along direction 140, an ultrasonic sensor 110 located on the right front side of vehicle 100 can emit ultrasonic signals and receive echo signals. Two sensors 110 can emit two ultrasonic signals 120 and receive corresponding echo signals. Based on the known positional relationship of the two ultrasonic sensors 110, the emission time of the two ultrasonic signals 120, the reception time of the corresponding echo signals, etc., the processing unit can calculate the echo intersection point 160 of the two ultrasonic signals 120, which may represent a reflection point on the detected object. Accordingly, the echo intersection data may include information about this intersection point 160. In some embodiments, the echo intersection is not limited to the intersection of two echoes from ultrasonic signals emitted by two ultrasonic sensors. For example, the echo intersection can also be the intersection of two echoes from ultrasonic signals emitted at different times during the movement of the same ultrasonic sensor. Furthermore, echo intersection data can be calculated based on echo data. For example, the corresponding echo intersection data can be obtained based on the ultrasonic arc of any two echo data within a certain range.

[0034] In some embodiments, ultrasound data may include the echo data and / or echo intersection data described above. Optionally, ultrasound data may also include other data related to the ultrasound signal, such as ultrasound signal-related data obtained based on methods known in the art or possible future methods.

[0035] In some embodiments, the processing unit of the ultrasonic sensor module can provide the obtained ultrasonic data to the control unit 130 of the vehicle 100. For example, the control unit 130 may be the vehicle's Electronic Control Unit (ECU). In some embodiments, some or all of the operations performed by the processing unit of the ultrasonic sensor module may also be performed by the control unit 130. For example, the control unit 130 may obtain echo data and / or echo intersection data based on the ultrasonic signals emitted and received by the ultrasonic sensor 110. In some embodiments, the vehicle 100 may also include other processing units to perform such operations.

[0036] In some implementations, the sensors, processing units, and control units mentioned above can all be included in the vehicle's control system. The vehicle's control system can perform various controls on the vehicle. For ease of understanding, Figure 2 A schematic diagram of an exemplary control system in a vehicle according to an embodiment of the present disclosure is shown.

[0037] exist Figure 2 In the example, with Figure 1 The same components use the same reference numerals. Furthermore, it should be understood that... Figure 2 This disclosure only shows some components related to the technical solutions of this disclosure. In actual implementation, the vehicle control system may also include various other components, which are not limited in this disclosure.

[0038] exist Figure 2 In the example, the vehicle control system 200 may include ultrasonic sensor modules 105-1 to 105-N. Ultrasonic sensor modules 105-1 to 105-N each include one or more ultrasonic sensors 110 and corresponding processing units 115-1 to 115-N. (As described above...) Figure 1 As described, in some implementations, the vehicle may include only one ultrasonic sensor module. For example, the ultrasonic sensor module includes multiple ultrasonic sensors and a processing unit mounted on the vehicle 100; in different implementations, the vehicle may include one or more ultrasonic sensor modules, each of which may include one or more ultrasonic sensors and a corresponding processing unit.

[0039] The control system 200 may further include a control unit 130. The control unit 130 can control the operation of any of the ultrasonic sensor modules 105-1 to 105-N. For example, the control unit 130 can control the ultrasonic sensor 110 in any of the ultrasonic sensor modules 105-1 to 105-N to emit ultrasonic signals and receive echo signals; the control unit 130 can receive ultrasonic data from any of the ultrasonic sensor modules 105-1 to 105-N, and perform further calculations or control the vehicle based on the ultrasonic data, etc. As described above, some or all of the calculations performed by the processing units in the ultrasonic sensor modules 105-1 to 105-N may also be performed by the control unit 130 or other processing units.

[0040] The control system 200 may also include a human-machine interface 180. The control unit 130 may output information understandable to a user (e.g., a driver) via the human-machine interface 180, and may receive information input by the user from the human-machine interface. In some embodiments, the user may input a selection regarding entering an assisted parking mode or an autonomous driving mode via the human-machine interface 180. After receiving user input regarding entering an assisted parking mode or an autonomous driving mode, the control unit 130 may control the vehicle to operate in the assisted parking mode or the autonomous driving mode. For example, in assisted parking mode, the control unit may control the vehicle to automatically find a parking space and park automatically. Another example is controlling the vehicle to plan a route and avoid obstacles in autonomous driving mode. In some embodiments, the control unit 130 may automatically control the vehicle to enter an assisted / automatic parking mode or an assisted / autonomous driving mode without user input. Regardless of how the vehicle enters any of these modes, the control unit 130 may process ultrasonic data to detect objects in the vehicle's surrounding environment, such as determining object information like location, type, and height. Based on object information, the control unit 130 can perform various operations related to vehicle parking or route planning, such as detecting available parking spaces for the vehicle, planning parking routes for the vehicle based on the detected parking spaces, and planning driving routes for the vehicle based on detected obstacles, etc.

[0041] Typically, to detect objects in the environment surrounding a vehicle, ultrasonic data is used to determine the object's location and other relevant information. In real-world applications, the environment around a vehicle may contain various types of objects, such as other vehicles, parking locks, trees, railings, walls, curbs, and pedestrians. Current technologies use the same model to detect multiple object types and predict their locations. However, this model cannot identify the object type, and using the same model for multiple object types can lead to inaccurate location predictions. This significantly impacts the performance of vehicle driver assistance or parking functions, and also affects the user experience.

[0042] In the technical solution disclosed herein, clustering technology can be used to preprocess ultrasonic data. For example, ultrasonic data can be clustered to obtain sample clusters corresponding to each object. Based on the sample clusters, the object type of each object can be determined. Subsequently, based on the object type of each object, an appropriate model can be selected to process the ultrasonic data using prediction technology to obtain the location prediction information of each object. Compared to using the same model to detect multiple different object types, selecting the appropriate model based on the object type to detect objects of the same object type can provide more accurate and reliable location prediction information. This can significantly improve vehicle assisted driving / parking performance and user experience.

[0043] Figure 3A A schematic diagram illustrating an application scenario based on an embodiment of this disclosure is shown.

[0044] exist Figure 3A In the illustrated embodiment, for example in a parking lot, while the vehicle 330 is traveling in the direction of the arrow, the ultrasonic sensor 310 in the ultrasonic sensor module emits an ultrasonic signal 320 to obtain ultrasonic data, thereby detecting objects in the vehicle's surrounding environment based on the ultrasonic data. Figure 3A As shown, the surrounding environment of vehicle 330 may contain various types of objects, such as other vehicles 340-1 and 340-2, parking locks 350-1 and 350-2, and cylinders 360-1 to 360-3. Other types of objects (not shown), such as railings and curbs, may also exist in the surrounding environment of vehicle 330. The acquired ultrasonic data can contain characteristic information about various objects, and this characteristic information can be used to predict the location, type, etc., of the objects.

[0045] Figure 3B A schematic diagram of ultrasound data according to an embodiment of the present disclosure is shown.

[0046] exist Figure 3B In the illustrated embodiment, the collected ultrasound data is shown in a spatial range, which may include echo data and echo intersection data. Figure 3B The spatial range shown can correspond to Figure 3AThe spatial range is shown. Each ultrasound data sample point represents an echo data sample point or an echo intersection data sample point, with circular sample points representing echo data sample points and triangular sample points representing echo intersection data sample points. Each echo sample point may include the echo timestamp, echo amplitude, echo significance, echo distance, echo height, echo coordinates (e.g., may include two coordinates), sensor coordinates (e.g., may include two coordinates), etc. Each echo intersection data sample point may include echo intersection coordinates (e.g., may include two coordinates), echo intersection distance, adjacent echo intersection coordinates (e.g., may include two coordinates), adjacent echo intersection height, adjacent echo intersection distance, adjacent echo intersection deflection angle, sensor coordinates, etc. Those skilled in the art will understand that, in addition to the echo data and echo intersection data characteristics exemplified above, echo data and echo intersection data may also include other ultrasound signal-related characteristics. Features of echo data and echo intersection data known in the art and those that may be adopted in the future can all be applied to the technical solutions of this disclosure. Those skilled in the art will understand that, in addition to the echo data and echo intersection data exemplified, ultrasound data may also include other data related to the ultrasound signal, such as ultrasound signal-related data obtained based on methods known in the art or possible future methods. Figure 3B This diagram of ultrasound data sample points is shown for illustrative purposes only. In various practical applications, the number of ultrasound data sample points within a specific spatial range will be greater and the distribution will be more complex.

[0047] In one embodiment, for example Figure 3B The ultrasonic data shown may have been captured by an ultrasonic sensor over a period of time during vehicle movement. In another embodiment, for example... Figure 3B The ultrasonic data shown can be captured by ultrasonic sensors over a distance the vehicle has traveled. For example, Figure 3B The ultrasound data shown can be obtained by fusing data captured by an ultrasound sensor during multiple sliding windows, based on a sliding window mechanism. The size of the sliding window can be fixed or variable, and the size of the sliding window corresponds to the actual spatial range covered by the ultrasound signal 320.

[0048] In one embodiment, for example Figure 3BThe ultrasonic data shown can be cached in a buffer or memory of the vehicle's processing system, and the vehicle's control unit 130 or other processing unit can perform subsequent processing on the cached ultrasonic data. In one embodiment, the vehicle's control unit 130 or other processing unit can perform subsequent processing on the cached ultrasonic data to determine the location and / or type of an object. In one embodiment, the vehicle's control unit 130 or other processing unit can plan a vehicle driving route based on the determined location and / or type of the object, for example, to assist the vehicle in parking or driving.

[0049] Figure 4 A schematic diagram of a method and modules for determining objects in the surrounding environment of a vehicle, according to embodiments of this disclosure, is shown. Figure 4 As shown, it may include a clustering module 410, a feature extraction module 420, a classification module 430, a model selection module 440, and a location prediction module 450.

[0050] exist Figure 4 In the illustrated embodiment, it is possible to Figure 3B The ultrasound dataset shown is used as input to clustering module 410, which clusters data from at least a portion of the ultrasound data in the dataset to obtain sample clusters C1 to C7. In one embodiment, clustering module 410 may cluster only the ultrasound echo data and the ultrasound echo intersection data. In another embodiment, clustering module 410 may cluster both the ultrasound echo data and the ultrasound echo intersection data. Because the ultrasound echo intersection data has a higher density and a greater distance between different ultrasound echo intersection data, clustering based solely on the ultrasound echo intersection data may have certain advantages over clustering based on both ultrasound echo data and ultrasound echo intersection data, which is beneficial for obtaining better clustering results and making it easier to eliminate noise.

[0051] In one embodiment, the clustering module 410 may employ Density-Based Spatial Clustering of Applications with Noise (DBSCAN) to cluster at least a portion of the ultrasound data. DBSCAN is a density-based clustering algorithm that defines a cluster as the largest set of density-connected points. It can divide regions with sufficiently high density into clusters and can discover clusters of arbitrary shapes in noisy spatial databases. DBSCAN does not require prior knowledge of the number of clusters to be formed, can discover clusters of arbitrary shapes, and can identify noise points. Considering the complex environments that the technical solutions of this disclosure may face in specific application scenarios—for example, parking lots may include obstacle areas of various shapes and sizes, the collected ultrasound data may include various noises, and the number of objects appearing in the spatial range is not fixed, possibly zero or more obstacles—the above-mentioned characteristics of DBSCAN are particularly suitable for the technical solutions of this disclosure. DBSCAN is a clustering method known in the art, therefore its specific details will not be described in detail. It is understood that other suitable clustering methods may also be used in the clustering module 410, and the technical solutions of this disclosure are not limited to DBSCAN.

[0052] Clustered ultrasound datasets can include zero or more sample clusters. Including zero sample clusters in the spatial domain indicates that no object exists in the vehicle's current surrounding environment. When one or more sample clusters are included in the spatial domain, each sample cluster can correspond to an object in the vehicle's surrounding environment. Figure 4 As shown, sample clusters C1 and C7 can correspond to Figure 3A Vehicles 340-1 and 340-2, and sample clusters C2, C4, and C6 can correspond to Figure 3A Cylinders 360-1 to 360-3, and sample clusters C3 and C5, can correspond to Figure 3A The ground locks are 350-1 and 350-2.

[0053] exist Figure 4 In the illustrated embodiment, ultrasound data corresponding to one or more sample clusters can be used as input to the classification module 430. The classification module 430 determines the object type corresponding to each sample cluster based on the ultrasound data corresponding to each sample cluster, and can output a label that represents the object type corresponding to each sample cluster. Figure 4As shown, sample clusters C1 and C7 can have a corresponding label L1, which represents the object type "vehicle"; sample clusters C2, C4, and C6 can have a corresponding label L2, which represents the object type "cylinder"; and sample clusters C3 and C5 can have a corresponding label L3, which represents the object type "ground lock". In other embodiments, various types of objects and labels are possible.

[0054] In one embodiment, the classification module 430 may employ an Extended Gradient Boosting (XG Boost) model based on a classification algorithm to determine the object type corresponding to each sample cluster. The XG Boost model is a gradient boosting tree algorithm that iteratively trains multiple decision trees and gradually improves model performance through gradient boosting. XG Boost trains decision trees separately for each category and combines the outputs of each subtree to obtain the classification result for the samples, enabling it to effectively handle multi-class classification tasks. Considering that the technical solutions of this disclosure may include various types of objects in specific application scenarios, the above-mentioned characteristics of the XG Boost model are particularly suitable for the technical solutions of this disclosure. The XG Boost model is a known classification method in the art, therefore its specific details will not be described in detail. It is understood that other appropriate classification methods may also be used in the classification module 430, and the technical solutions of this disclosure are not limited to the XG Boost model.

[0055] Optionally, before inputting the ultrasound data corresponding to one or more sample clusters into the classification module 430, the ultrasound data corresponding to one or more sample clusters can be input into the feature extraction module 420, and the extracted features of each sample cluster associated with the ultrasound signal can also be used as input to the classification module 430.

[0056] In one embodiment, the feature extraction module 420 may determine the features associated with the ultrasound signal for each sample cluster based on all ultrasound data corresponding to each sample cluster.

[0057] In another embodiment, the feature extraction module 420 may determine a first subset of ultrasound data corresponding to the first end of the object and a second subset of ultrasound data corresponding to the second end of the object based on all ultrasound data corresponding to each sample cluster, and determine the features of the sample cluster associated with the ultrasound signal based on the first subset of ultrasound data and the second subset of ultrasound data.

[0058] by Figure 3ATaking vehicle 340-2 as an example, its position can be described by a bounding box (shown by a dotted line). For vehicle 330, the positions of the two corner points A and B closest to vehicle 330 are more critical for determining the position of vehicle 340-2. Therefore, compared to all the ultrasonic data corresponding to vehicle 340-2, the ultrasonic data at the two ends of sample cluster C7 corresponding to the two corner points A and B of vehicle 340-2 are more important for determining the accurate corner point positions. For example, for the sample cluster corresponding to vehicle 340-2, the first subset of ultrasonic data may include the ultrasonic data associated with the first left end of sample cluster C7 corresponding to corner point A, and the second subset of ultrasonic data may include the ultrasonic data associated with the second right end of sample cluster C7 corresponding to corner point B.

[0059] The first and second subsets of ultrasound data can be determined in various ways. For example, different data extraction ratios can be determined based on different bounding box sizes. For example, a first ratio of 100% can be used for a bounding box size of 1.5 meters in the direction of travel, and a second ratio of 50% can be used for a bounding box size of 3 meters in the direction of travel. This data extraction ratio can be applied to different dimensions, such as the number of samples, the distance from the endpoints, etc.

[0060] In one embodiment, for each sample cluster, the statistical measures of the features of multiple ultrasound data within the sample cluster extracted by the feature extraction module 420 can be used as the features of the sample cluster. In another embodiment, for each sample cluster, the statistical measures of the features of multiple ultrasound data in a first subset of ultrasound data extracted by the feature extraction module 420 can be used as a first feature subset of the sample cluster, and the statistical measures of the features of multiple ultrasound data in a second subset of ultrasound data extracted by the feature extraction module 420 can be used as a second feature subset of the sample cluster. Subsequently, both the first feature subset and the second feature subset are used as the features of the sample cluster.

[0061] For example, the extracted features may include one or more of the following: echo amplitude, echo intensity, echo distance, echo intersection coordinates, distance from the echo intersection to the sensor, echo height, echo intersection height, sensor coordinates, and angle with the sensor. To determine the coordinates of corner positions, such as planar coordinates, the same or different features can be extracted for the x and y coordinates of each corner position, depending on the implementation.

[0062] For example, the statistics used may include one or more of the following: mean, minimum, maximum, variance, quantile, and kurtosis. For example, the same or different statistics may be used based on features extracted separately for the x and y coordinates of each corner location, depending on the implementation.

[0063] return Figure 4 ,exist Figure 4 In the illustrated embodiment, ultrasound data corresponding to one or more sample clusters, along with the object type determined by the classification module 430, can be used as input to the model selection module 440. The model selection module 440 determines the prediction model to be used for each sample cluster based on the object type corresponding to each sample cluster.

[0064] In one embodiment, there exists a set of prediction models, comprising multiple prediction models, each pre-trained for a specific type of object. For example, the prediction models may have the same or different structures and parameters, and / or may be based on the same or different algorithms. Compared to using a uniform prediction model to predict the location of multiple object types, using separate prediction models for specific object types allows for better location prediction based on the features of that specific object type, thereby improving prediction accuracy.

[0065] exist Figure 4 In the illustrated embodiment, ultrasound data corresponding to one or more sample clusters can be used as input to the location prediction module 450 to output location prediction information for one or more objects. For each sample cluster, the location prediction module 450 is implemented based on a prediction model determined by the model selection module 440 to be used for each sample cluster.

[0066] In one embodiment, for a single sample cluster, the location prediction module 450 can employ an Extended Gradient Boosting (XGBoost) model based on a regression algorithm to determine the predicted location of the object corresponding to that sample cluster. The XGBoost model is a gradient boosting tree algorithm that iteratively trains multiple decision trees, each acting as a weak learner, and gradually improves model performance through gradient boosting, ultimately combining multiple weak learners into a strong learner. The XGBoost model's characteristics in preventing overfitting and improving generalization ability are particularly suitable for the technical solutions of this disclosure. The XGBoost model is a known regression method in the art, therefore its specific details will not be described in detail. It is understood that the location prediction module 450 can also employ other suitable prediction methods, and the technical solutions of this disclosure are not limited to the XGBoost model.

[0067] Optionally, features of one or more sample clusters extracted by the feature extraction module 420 can also be used as input to the location prediction module 450 to output location prediction information for one or more objects.

[0068] In one embodiment, the output object position prediction information can be characterized by the positions of the object's two corner points, such as... Figure 3A The location of corner points A and B of vehicle 340-2 is specified. For example, the object's location prediction information may include the coordinates of the object's first and second corner points. Additionally, the object's location prediction information may also include the object's height information. For instance, ultrasonic signals may detect a curb around the vehicle, but since the curb is low and has little impact on parking, the object's height information is still valuable for assisted parking or driving functions. Furthermore, the output may include the distance from the object to the vehicle, as well as other information such as the object type determined by the classification module 430.

[0069] In the technical solution described in conjunction with specific embodiments, the object type of each object is determined by obtaining sample clusters based on clustering, and an appropriate model is selected based on the object type of each object to predict the object's location information. Compared to using the same model to detect multiple different object types, selecting the appropriate model based on the object type to detect objects of the same object type can provide more accurate and reliable location prediction information, thereby improving vehicle assisted driving / parking performance and user experience.

[0070] Figure 5 A flowchart illustrating a method for determining objects in the environment surrounding a vehicle, according to an embodiment of this disclosure, is shown.

[0071] At step 510, clustering can be performed based on at least a portion of the ultrasound data in the ultrasound dataset to obtain one or more sample clusters, wherein each sample cluster corresponds to an object in the environment surrounding the vehicle.

[0072] In one embodiment, the ultrasonic data set may be captured by an ultrasonic sensor during a period of vehicle movement, and / or the ultrasonic data set may be captured by an ultrasonic sensor during a period of vehicle movement. The ultrasonic sensors may be of varying numbers, such as single or multiple.

[0073] In one embodiment, the ultrasound data set may include ultrasound echo data and ultrasound echo intersection data. The ultrasound echo data may include one or more of the following: echo coordinates, echo height, echo amplitude, echo intensity, or echo distance of each ultrasound signal echo. The ultrasound echo intersection data may include one or more of the following: echo intersection coordinates, echo intersection height, echo intersection distance, echo intersection deflection angle, coordinates of adjacent echo intersections, heights of adjacent echo intersections, distances of adjacent echo intersections, deflection angle of adjacent echo intersections, and sensor coordinates.

[0074] In one embodiment, one or more of the ultrasound echo data and ultrasound echo intersection data in the ultrasound dataset can be clustered.

[0075] In one embodiment, clustering at least a portion of the ultrasound data in the ultrasound dataset can be achieved using density-based spatial clustering (DBSCAN) with noisy applications.

[0076] At step 520, optionally, features associated with ultrasound signals for each sample cluster can be extracted based on ultrasound data corresponding to one or more sample clusters.

[0077] In one embodiment, the ultrasound signal-associated features of a sample cluster can be determined based on all ultrasound data corresponding to each sample cluster. In another embodiment, a first subset of ultrasound data corresponding to a first end and a second subset of ultrasound data corresponding to a second end can be determined based on all ultrasound data corresponding to each sample cluster, and the ultrasound signal-associated features of the sample cluster can be determined based on the first and second subsets of ultrasound data.

[0078] For example, the first and second subsets of ultrasonic data can correspond to the two corner points of a bounding box along the lateral direction of vehicle travel, respectively. The first and second subsets of ultrasonic data can be determined in various ways. For instance, different data extraction ratios can be determined based on different bounding box sizes; for example, a first ratio of 100% can be used for a bounding box size of 1.5 meters in the direction of travel, and a second ratio of 50% can be used for a bounding box size of 3 meters in the direction of travel. This data extraction ratio can be applied to different dimensions, such as the number of samples, the distance from the endpoints, etc.

[0079] For example, the extracted features may include one or more of the following: echo amplitude, echo intensity, echo distance, echo intersection coordinates, distance from echo intersection to sensor, echo height, echo intersection height, sensor coordinates, and angle with the sensor. To determine the coordinates of corner points, the same or different features can be extracted for the x-coordinate and y-coordinate of each corner point.

[0080] For example, statistics can be calculated based on ultrasound data corresponding to each sample cluster, and the characteristics of that sample cluster associated with the ultrasound signal can be determined based on the calculated statistics. The statistics used can include one or more of the following: mean, minimum, maximum, variance, quantile, and kurtosis. For example, the same or different statistics can be used based on features extracted separately for the x and y coordinates of each corner location.

[0081] At step 530, the object type corresponding to each sample cluster can be determined based on the ultrasound data corresponding to one or more sample clusters.

[0082] In one embodiment, the features associated with ultrasound signals for each sample cluster extracted in step 520 can be provided as feature data to a second machine learning model to output the object type corresponding to each sample cluster. For example, the second machine learning model can be implemented using an extended gradient boosting (XG Boost) model based on a classification algorithm. Furthermore, the second machine learning model can be pre-trained in a supervised manner for multiple sample clusters corresponding to multiple object types.

[0083] At step 540, the ultrasound data corresponding to each sample cluster and the object type can be provided as feature data to the corresponding first machine learning model in the first machine learning model set to output the object location prediction information. The corresponding first machine learning model for each sample cluster is selected from the first machine learning model set based on the object type corresponding to that sample cluster.

[0084] In one embodiment, the features associated with the ultrasound signals of each sample cluster extracted in step 520 can be provided as feature data to the corresponding first machine learning model to output the location prediction information of the object corresponding to each sample cluster.

[0085] In one embodiment, the first machine learning model may be implemented using an extended gradient boosting (XG Boost) model based on a regression algorithm. For example, the first machine learning model may be trained in a supervised manner based on ultrasound data corresponding to sample clusters corresponding to a single object type.

[0086] In one embodiment, the output object location prediction information may include one or more of the following: the coordinates of the object's first and second corner points, the object's height information, and the distance from the object to the vehicle. Additionally, object type information may also be output.

[0087] Figure 6A flowchart illustrating an embodiment of the present disclosure is shown for training a machine learning model for determining objects in the environment surrounding a vehicle.

[0088] At step 610, the training data may be input into the machine learning model to be trained.

[0089] At step 620, the machine learning model to be trained can be iteratively updated based on the loss function until the training termination condition is met.

[0090] In one embodiment, the machine learning model to be trained can be a classification model, for example... Figure 4 The specific implementation of the classification module 430 described in the document, and / or Figure 5 The second machine learning model described herein. This machine learning model can be trained in a supervised manner for multiple object types. For example, the training dataset may include {ultrasound data corresponding to a sample cluster, the corresponding object type of the sample cluster}, or alternatively, the training dataset may include {features of ultrasound signals associated with ultrasound data corresponding to a sample cluster, the corresponding object type of the sample cluster}. For example, this machine learning model may be implemented using an extended gradient boosting (XG Boost) model based on a classification algorithm.

[0091] In one embodiment, the machine learning model to be trained can be a regression model, for example... Figure 4 The specific implementation of the location prediction module 450 described in the document, and / or Figure 5 The first machine learning model described herein. This machine learning model can be trained in a supervised manner for a single object type, and can be a collection of multiple machine learning models trained for multiple object types. These multiple machine learning models can have the same or different structures and parameters, and / or employ the same or different algorithms. For example, the training dataset may include {ultrasound data corresponding to a sample cluster, and location information of the corresponding object in that sample cluster}; alternatively, the training dataset may include {features of ultrasound data associated with ultrasound signals corresponding to a sample cluster, and location information of the corresponding object in that sample cluster} or {statistics of features of ultrasound data associated with ultrasound signals corresponding to a sample cluster, and location information of the corresponding object in that sample cluster}. For example, the machine learning model may be implemented using an Extended Gradient Boosting (XG Boost) model based on a regression algorithm.

[0092] Figure 7 A flowchart of a method for assisting vehicle driving according to an embodiment of the present disclosure is shown.

[0093] At step 710, the system may include predictive information about the location of objects in the vehicle's surrounding environment based on ultrasonic signals. Step 710 may be performed using methods described in various embodiments of this disclosure.

[0094] At step 720, the vehicle may be able to plan its route based on the output location prediction information of the object.

[0095] For example, available parking spaces can be detected based on object location prediction information, and routes for vehicles to enter these spaces can be planned. As another example, obstacles can be detected based on object location prediction information, and vehicle routes can be planned to avoid these obstacles.

[0096] Figure 8 A block diagram of a control system for a vehicle according to an embodiment of the present disclosure is shown.

[0097] The control system 800 may include one or more processors 810 and a memory 820. The memory 820 may store executable instructions. The processor 810 may execute the executable instructions stored or encoded in the memory 820, thereby achieving the above-described combination. Figures 1 to 7 The various operations and / or functions described. Although not in Figure 8 As shown in the diagram, but those skilled in the art will understand that the control system 800 may include various other components, such as various communication modules, bus modules, and possibly user interface modules.

[0098] In some embodiments, the control system 800 may include Figure 1 and 2 The control unit 130 and / or processing unit 115 shown.

[0099] Embodiments of this disclosure also provide a vehicle. This vehicle may have features such as... Figure 1 The ultrasonic sensor shown is used to transmit and receive ultrasonic signals. The vehicle may also have... Figure 8 The control system 800.

[0100] Embodiments of this disclosure also provide a machine-readable storage medium. The machine-readable storage medium may store executable instructions, which, when executed by a processor, can achieve the above-described combination. Figures 1 to 5The various operations and / or functions described. For example, machine-readable storage media may include, but are not limited to, random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), static random access memory (SRAM), hard disk, flash memory, etc.

[0101] Embodiments of this disclosure also provide a computer program product. The computer program product may include a computer program. When executed by a processor, the computer program can achieve the above-described combinations. Figures 1 to 7 The various operations and / or functions described.

[0102] Specific embodiments of this disclosure have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0103] Not all steps and units in the above process and system structure diagrams are necessary; some steps or units can be omitted according to actual needs. The device structure described in the above embodiments can be a physical structure or a logical structure. That is, some units may be implemented by the same physical entity, some units may be implemented by multiple physical entities, or they may be jointly implemented by certain components in multiple independent devices.

[0104] The term "exemplary" as used throughout this specification means "serving as an example, instance, or illustration" and does not imply "preferred" or "advantageous" compared to other embodiments. Detailed descriptions are included for the purpose of providing an understanding of the described techniques. However, these techniques can be practiced without these detailed descriptions. In some instances, well-known structures and apparatuses are shown in block diagram form to avoid obscuring the concepts of the described embodiments.

[0105] The foregoing description of this disclosure is provided to enable any person skilled in the art to implement or use this disclosure. Various modifications to this disclosure will be apparent to those skilled in the art, and the general principles defined herein can be applied to other variations without departing from the scope of this disclosure. Therefore, this disclosure is not limited to the examples and designs described herein, but is consistent with the widest scope of the principles and novel features disclosed herein.

Claims

1. A method for determining objects in the surrounding environment of a vehicle based on ultrasonic signals, comprising: Clustering is performed based on at least a portion of the ultrasound data in the ultrasound dataset to obtain one or more sample clusters, wherein each sample cluster corresponds to an object in the environment surrounding the vehicle; The object type corresponding to each sample cluster is determined based on the ultrasound data corresponding to the one or more sample clusters; The ultrasound data corresponding to each sample cluster and the object type are provided as feature data to the corresponding first machine learning model in the first machine learning model set to output the object location prediction information. The corresponding first machine learning model for each sample cluster is selected from the first machine learning model set based on the object type corresponding to that sample cluster.

2. The method according to claim 1, wherein, The ultrasonic data set is captured by an ultrasonic sensor during a period of vehicle movement, and / or the ultrasonic data set is captured by an ultrasonic sensor during a period of vehicle movement.

3. The method according to claim 1, wherein, The ultrasound data set includes ultrasound echo data and ultrasound echo intersection data.

4. The method according to claim 3, wherein, The clustering based on at least a portion of the ultrasound data in the ultrasound dataset includes: clustering based solely on the ultrasound echo intersection data.

5. The method according to claim 3, wherein, The ultrasonic echo data includes one or more of the following: echo coordinates, echo height, echo amplitude, echo intensity, or echo distance of each ultrasonic signal echo; and / or, The ultrasonic echo intersection data includes one or more of the following: echo intersection coordinates, echo intersection height, echo intersection distance, echo intersection deflection angle, coordinates of adjacent echo intersections, height of adjacent echo intersections, distance of adjacent echo intersections, deflection angle of adjacent echo intersections, and sensor coordinates.

6. The method according to claim 1, wherein, The clustering based on at least a portion of the ultrasound data in the ultrasound dataset includes: clustering at least a portion of the ultrasound data in the ultrasound dataset using density-based spatial clustering (DBSCAN) with noise application.

7. The method according to claim 1, further comprising: Extracting features associated with ultrasound signals for each sample cluster based on ultrasound data corresponding to the one or more sample clusters, including: The characteristics of a sample cluster associated with the ultrasound signal are determined based on all the ultrasound data corresponding to each sample cluster; or Based on all the ultrasound data corresponding to each sample cluster, a first subset of ultrasound data corresponding to the first end and a second subset of ultrasound data corresponding to the second end are determined, and the features of the sample cluster associated with the ultrasound signal are determined based on the first subset of ultrasound data and the second subset of ultrasound data.

8. The method according to claim 7, wherein, The extraction of features associated with ultrasound signals for each sample cluster based on ultrasound data corresponding to the one or more sample clusters includes: Statistics are calculated based on the ultrasound data corresponding to each sample cluster, and the characteristics of the sample cluster associated with the ultrasound signal are determined based on the calculated statistics.

9. The method according to claim 1, wherein, Determining the object type corresponding to each sample cluster based on ultrasound data corresponding to the one or more sample clusters includes: The features associated with the ultrasound signals of each sample cluster are provided as feature data to a second machine learning model to output the object type corresponding to each sample cluster.

10. The method according to claim 9, wherein, The second machine learning model is implemented using the Extended Gradient Boosting (XGBoost) model based on a classification algorithm.

11. The method according to claim 10, wherein, The second machine learning model is trained in a supervised manner based on ultrasound data corresponding to sample clusters of multiple object types.

12. The method according to claim 1, wherein, The first machine learning model in the first set of machine learning models is implemented using the Extended Gradient Boosting (XG Boost) model based on a regression algorithm.

13. The method according to claim 12, wherein, One of the first machine learning models in the first set of machine learning models is trained in a supervised manner based on ultrasound data corresponding to sample clusters corresponding to a single object type.

14. The method according to claim 1, wherein, The output includes the object's predicted location information: Output the coordinates of the first corner point of the object, and the coordinates of the second corner point of the object; Output the height information of the object; and / or Output the distance from the object to the vehicle.

15. The method according to claim 1, further comprising: Output type prediction information for the object.

16. The method according to claim 1, further comprising: The output of the object's location prediction information is used to assist the vehicle in planning its driving route.

17. A control system for a vehicle, comprising: At least one processor; A memory coupled to the at least one processor stores executable instructions that, when executed by the at least one processor, cause the at least one processor to implement the method according to any one of claims 1 to 16.

18. A computer-readable medium storing a computer program comprising instructions that, when executed by a processor, cause one or more units to perform the method according to any one of claims 1-16.

19. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1 to 16.

20. A vehicle, the vehicle comprising: An ultrasonic sensor is used to emit and receive ultrasonic signals; as well as One or more units for performing the method according to any one of claims 1-16.

Images