Method for performing object classification

Abstract

The present invention relates to a method for classifying objects into object classes based on information of at least one ultrasonic sensor of a vehicle.

Description

[Technical field]

[0001] The present invention relates to a method for classifying objects into object classes. [Background technology]

[0002] It is known to perform height estimation of an object using geometric methods based on information captured by ultrasonic sensors, which makes it possible to distinguish, for example, between linear objects representing curbs and those in the form of walls.

[0003] A problem with height estimation is that known methods for estimating height give erroneous results when used on vehicle shaped objects. Summary of the Invention [Problem to be solved by the invention]

[0004] Starting from this, the object of the invention is to provide a method which allows a reliable classification of objects into predefined object classes. [Means for solving the problem]

[0005] This object is achieved by a method having the features of independent claim 1. Preferred embodiments are the subject of the dependent claims. A system for classifying objects into object classes based on information of at least one ultrasonic sensor of a vehicle is the subject of parallel claim 14, as well as a vehicle equipped with such a system.

[0006] According to a first aspect, a method for classifying objects into object classes based on information from at least one ultrasonic sensor is disclosed, the method comprising the steps of:

[0007] First, a number of detections of at least one ultrasonic sensor of the vehicle are received, which may of course also be obtained by cooperation of a number of ultrasonic sensors. Each detection is assigned position information and directional information. The position information is indicative of a reflection position from which an ultrasonic signal of the at least one ultrasonic sensor is reflected. The position information particularly preferably comprises coordinates indicative of the reflection position. The directional information is indicative of a direction along which the ultrasonic signal propagates between the reflection position and the at least one ultrasonic sensor. The directional information can, for example, be given as an angle, particularly preferably as an azimuth angle.

[0008] The received detections are then assigned to clusters, where a cluster contains multiple detections and thus constitutes a group of detections. One to multiple clusters may relate to an object, i.e. a detection originating from an object may be assigned to one or multiple clusters.

[0009] After the clusters are formed, information regarding the statistical distribution of position information and information regarding the statistical distribution of directional information are calculated for each cluster, based on the position information and directional information of the detections assigned to each cluster.

[0010] Finally, classification of the objects into object classes is performed based on information about the statistical distribution of the position information of the clusters and information about the statistical distribution of the orientation information.

[0011] A technical advantage of the disclosed method is that information about the statistical distribution of position information of a cluster and information about the statistical distribution of orientation information can be used as a decision aid for assigning the cluster, and therefore the objects assigned to the cluster, to which object class, thereby achieving accurate and efficient object class classification, particularly for the purpose of excluding detections in the cluster assigned to the object class "vehicle" from height estimation.

[0012] According to an embodiment, the position information comprises at least a first and a second coordinate. The information on the statistical distribution of the position information comprises information based on a covariance matrix of the first and second coordinates of the position information. By taking into account the covariance matrix of the first and second coordinates of the position information, in particular the distribution of detections in the longitudinal direction and in the transverse direction running perpendicular thereto can be used for classification into object classes. In particular the determinant of the covariance matrix of the first and second coordinates of the position information can be used as a criterion. This determinant is larger for two-dimensionally formed objects (in particular objects with rounded edges and / or corners), such as vehicles, than for linear objects (for example curbs or wall objects).

[0013] According to one embodiment, the information on the statistical distribution of the position information comprises eigenvalues ​​of a covariance matrix of the first and second coordinates of the position information, which provide a measure of the spread of the cluster along its major and minor axes, from which the shape of the object can be estimated.

[0014] According to one embodiment, the information on the statistical distribution of the position information comprises a ratio of eigenvalues ​​of the covariance matrix of the first and second coordinates of the position information, which can be frequently adopted as a criterion since it is very different for linear objects and objects formed in two dimensions.

[0015] According to an embodiment, the information on the statistical distribution of the directional information of the detection includes the variance of the directional information, which is very small for linear objects but very high for rounded objects such as pillars, and the variance of the directional information for a vehicle is between that of linear objects and rounded objects, since the vehicle contour has flat and rounded surfaces.

[0016] According to one embodiment, the information on the statistical distribution of the directional information of the detections comprises the time derivative of the directional information, so that the change in the directional information over time can be used as a criterion for object class classification.

[0017] According to an embodiment, the information on the statistical distribution of the directional information of the detections comprises a time derivative of the directional information filtered by a filter function, which can particularly preferably be a filter for filtering statistical outliers, so that the change in the filtered directional information over time can be used as a criterion for object class classification.

[0018] According to an embodiment, the classification is performed based on at least a first and a second threshold, the first threshold being a threshold for information on the statistical distribution of the position information and the second threshold being a threshold for information on the statistical distribution of the direction information of the detection, where it is clear that two or more decision rules based on these thresholds may be used as decision criteria for the object class classification.

[0019] According to one embodiment, the first and second thresholds are determined by training data, the training data comprising object information and label information assigned to the object information, the label information indicating the object class to which each object belongs, and using the training data, a decision rule can be created regarding which thresholds of information on the statistical distribution of position information and information on the statistical distribution of direction information are used to classify into each object class.

[0020] According to one embodiment, the classification is performed using decision trees or random decision forests, which allows object class classification with little computational effort.

[0021] According to some other embodiments, a neural network is used for classification, which is pre-trained with training data that has label information for each object class.

[0022] According to one further embodiment, a classification into object classes "vehicle" and "non-vehicle" is performed, so that objects that are vehicles can be distinguished from objects of other object classes, for example "wall" or "curb".

[0023] According to another embodiment, depending on the classification result of the object, height estimation of the object is or is not performed, in particular for detections related to the object class "vehicle", height estimation is not performed, thereby minimizing the computational load and error rate during height estimation.

[0024] According to a further aspect, a system for classifying objects into object classes based on information of at least one ultrasonic sensor of a vehicle is disclosed, the system comprising a computing unit configured to perform the following steps: receiving a plurality of detections of at least one ultrasonic sensor of the vehicle, each detection being assigned location information and direction information, the location information indicating a reflection point from which an ultrasonic signal of the at least one ultrasonic sensor is reflected, and the direction information indicating a direction along which the ultrasonic signal propagates between the reflection point and the at least one ultrasonic sensor; forming clusters of detections based on the received detections, where a cluster encompasses a plurality of detections; calculating information about the statistical distribution of location information and information about the statistical distribution of directional information of the detections assigned to each cluster; Classifying the objects into object classes based on information about the statistical distribution of the location information of the clusters and information about the statistical distribution of the orientation information.

[0025] According to another further aspect, a vehicle is disclosed that includes a system for classifying objects into object classes.

[0026] In the present invention, expressions such as "approximately," "substantially," or "approximately" are to be construed as meaning an error within + / - 10%, preferably + / - 5%, from the exact value, and / or an error that is insignificant to the function.

[0027] Developments, advantages and scope of application of the invention are shown in the following description with reference to examples and drawings, in which all described and / or shown features are the subject of the invention, both individually and in any combination, independently of the respective claims or their references, the contents of which are also incorporated into the description.

[0028] The present invention will now be described in detail with reference to the drawings and examples. [Brief description of the drawings]

[0029] [Figure 1] FIG. 1 illustrates diagrammatically a vehicle equipped with ultrasonic sensors including ultrasonic sensors distributed around the vehicle and a computing unit for evaluating the information provided by the ultrasonic sensors; [Diagram 2] FIG. 2 illustrates a graph showing the position information of detection in a longitudinal parking situation between two vehicles; [Diagram 3] FIG. 3 illustrates a graph showing the position information and partly the directional information of the detection in a longitudinal parking situation according to FIG. 2; [Figure 4] FIG. 4 illustrates a number of clusters determined based on the detections shown in FIGS. 2 and 3; [Diagram 5] FIG. 5 shows an example of a decision tree for classifying objects into object classes; and [Figure 6] FIG. 6 illustrates a block diagram illustrating method steps for classifying objects into object classes. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0030] 1 illustrates very diagrammatically a vehicle 1. The vehicle 1 is equipped with a number of ultrasonic sensors 2 with which perimeter acquisition is performed.

[0031] The ultrasonic sensor 2 is connected to at least one computing unit 3 with which the method for classifying an object O into an object class, described below, is implemented.

[0032] 2 shows a graph with a number of detections captured by ultrasonic sensors 2 of a vehicle 1 and assigned to objects O. The detections relate, for example, to a longitudinal parking situation in which a longitudinal parking space is formed between two vehicles. The detections arranged linearly between the two vehicles relate, for example, to a curb delimiting the side of the parking space.

[0033] A detection is then represented by a point. The reflection position at which a detection is formed by reflection on a surrounding object is represented by position information. The position information can comprise two coordinates by which the reflection position is defined in a horizontal level plane. These are particularly preferably x- and y-coordinates which are plotted on the respective graph axes in the graph of FIG. 2. In other words, the position information is represented in a Cartesian coordinate system. Alternatively, the position information can also be represented in a cylindrical or spherical coordinate system.

[0034] The statistical distribution of the position information is used to classify the objects.

[0035] If available, the position information may also include information about the height of the object region where the reflection is formed. In other words, the position information may indicate the reflection position in three-dimensional space. Information about the third dimension (i.e., height) may also be used to classify the object.

[0036] For each detection, respective directional information is also determined, which indicates from which direction the detection was received. The directional information can be, for example, an angle subtended on a horizontal level plane indicating the direction of a connection line connecting the reflection position and the sensor position of the ultrasonic sensor from which the reflected ultrasonic signal was emitted and / or received. The angle can be, for example, measured relative to a coordinate axis, for example the x-axis. Thus, the directional information can, for example, include the angle between the x-axis and the connection line connecting the reflection position and the sensor position of the ultrasonic sensor.

[0037] Figure 3 illustrates a graph similar to Figure 2, but with each detection assigned to the vehicle on the left additionally assigned directional information, respectively, which is indicated by a line. These lines indicate the direction in which the detection was determined. In other words, the lines indicate the direction in which the ultrasonic sensor 2 was located when transmitting and / or receiving the ultrasonic signal. It should be noted that due to the high propagation speed and the resulting short runtime of the ultrasonic signal between transmission and reception, it can be approximately assumed that the ultrasonic sensor 2 was in the same position when it transmitted the ultrasonic signal and when it received the reflected part of the ultrasonic signal.

[0038] To assign the detections to an object, at least one cluster C is formed from the captured detections. To form the clusters, known clustering algorithms can be used, for example density-based clustering methods, partitioned clustering methods, but particularly preferably the k-means clustering algorithm or the DBSCAN algorithm can be used to form the clusters.

[0039] Figure 4 illustrates a number of clusters C determined based on the detections shown in Figures 2 and 3. The ellipses shown in Figure 4 respectively illustrate the covariance matrix of the xy-coordinates of each detection assigned to each cluster C. A cluster C relates to a group of positionally related detections, preferably showing how each group of detections is arranged in a horizontal level plane (i.e., the xy-plane).

[0040] It should be noted that a threshold for the number of detections is preferably predefined, above which the number of detections forming a cluster must be, thus avoiding the formation of clusters with a small number of detections.

[0041] After the clusters are formed, it is possible to determine, for each cluster, information regarding the statistical distribution of the position information of the detections and information regarding the statistical distribution of the directional information.

[0042] It is particularly preferred that for each cluster C the mean value of the position information is calculated, which means the center of the cluster. This can be determined, for example, by averaging the x- and y-coordinates of the detections of a cluster. In addition, the covariance of the x- and y-coordinates of the detections assigned to each cluster can also be calculated. In other words, a two-dimensional Gaussian distribution of the x- and y-coordinates of the detections is calculated.

[0043] The information on the statistical distribution of the direction information of the detection of a cluster may include the average value and variance of the direction information of the detection of a cluster, where the average value and variance may be calculated directly, or the direction information may be filtered, for example, using a smoothing filter or filtered to remove statistical outliers, before the calculation of the average value and variance.

[0044] From the information about the statistical distribution of the position information and the information about the statistical distribution of the orientation information, further information or values ​​can be calculated that can be used to classify the objects.

[0045] For example, the determinant of the covariance matrix of the first and second coordinates of the position information can be calculated, which is larger for multi-dimensionally shaped objects than for linear objects, i.e., the determinant of the covariance matrix can be used to distinguish between multi-dimensionally shaped objects and linear objects.

[0046] Furthermore, the eigenvalues ​​of the covariance matrix of the first and second coordinates of the position information can be calculated. In particular, the first and second eigenvalues ​​can be obtained. These eigenvalues ​​indicate the spread along the major axis of cluster C and along a minor axis that runs perpendicular thereto. It is particularly preferred that these eigenvalues ​​give the lengths of the major and minor axes of an ellipse with which the position and orientation of cluster C can be reproduced. The ellipse assigned to cluster C is depicted in FIG. 4.

[0047] Furthermore, a quotient can be calculated from the first and second eigenvalues ​​of the covariance matrix of the first and second coordinates of the position information. The ratio of the eigenvalues ​​provides indirect evidence of whether it is a linear object or not, because the ratio of the eigenvalues ​​of a linear object is significantly different from that of a two-dimensionally formed object.

[0048] The variance of the directional information of the detections of a cluster also provides indirect evidence as to whether it is a linear object or a two-dimensional object: for example, the variance of the directional information is very small for linear objects, but very high for rounded objects such as pillars. Since a vehicle has both linear vehicle areas and diffusely reflective areas such as rearview mirrors and door handles, the variance of the directional information for a vehicle is between that of linear and round objects.

[0049] Additionally, the statistical characteristics of the directional information of the detections or of the time derivatives of the filtered directional information of the detections, i.e., the changes in the mean, variance, etc., can be determined.

[0050] It should be noted that the information on the statistical distribution of the location information and the information on the statistical distribution of the direction information are preferably determined recursively, i.e. the formation of the cluster C is updated each time one or more detections are received, thereby obtaining updated clusters. The information on the statistical distribution of the location information and the information on the statistical distribution of the direction information of the detections assigned to a cluster are also updated after the update of the cluster, i.e. are calculated anew on the basis of the detections newly added to the cluster.

[0051] The information about the statistical distribution of the position information of the detections and the information about the statistical distribution of the directional information can then be used to create decision rules, based on which classification of the objects into object classes is performed.

[0052] To create the decision rules, training data can be used. The training data includes information about objects and label information assigned to the objects. The label information defines to which object class each object should be assigned. The decision rules are particularly preferably threshold values. The threshold magnitude can be determined based on training data.

[0053] The threshold can be assigned to specific information regarding the statistical distribution of position information or specific information regarding the statistical distribution of directional information, and can indicate in particular that information below the threshold is classified into a first object class and information above the threshold is classified into a second object class.

[0054] To classify objects, one can use decision trees or random forests.

[0055] The structure and decision rules of a decision tree or randomized decision forest can be determined by training data.

[0056] 5 illustrates a decision tree as an example of a method for implementing classification into object classes "curb", "wall" and "vehicle". Particularly preferably, the decision tree is used for classification into object classes "vehicle" and "non-vehicle".

[0057] The decision rules of the decision tree are based on information about the statistical distribution of the position information and information about the statistical distribution of the directional information. As information about the statistical distribution of the position information, the quotient of the first and second eigenvalues ​​of the covariance matrix of the first and second coordinates of the position information, i.e. the ratio of both eigenvalues ​​of the covariance matrix, is used. As information about the statistical distribution of the directional information, the variance of the directional information is used. Based on the information about the statistical distribution of the position information, the information about the statistical distribution of the directional information and the thresholds assigned to these pieces of information, it is possible to assign the detections to a cluster C of one object class each. In particular, this makes it possible to determine whether the recognized detections of cluster C relate to the object class "vehicle" or not.

[0058] According to the decision tree of FIG. 5, if the variance of the directional information is greater than 0.00665 and the eigenvalue ratio is greater than 0.0012, it can be assumed with 99% certainty that the detection of the cluster is a vehicle.

[0059] After classifying the objects or the clusters to which they are assigned into object classes, it is possible to selectively implement a height estimation algorithm based on the information of the ultrasonic sensors, in particular the detections assigned to the cluster of object class "vehicle" are excluded from the height estimation in order to avoid erroneous estimations.

[0060] FIG. 6 shows, in its schematic representation, the steps of the method according to the invention for classifying objects into object classes using ultrasonic sensor means of a vehicle.

[0061] First, a plurality of detections of at least one ultrasonic sensor of the vehicle are received (S10), each detection having associated therewith location information and direction information, the location information indicating a reflection location from which an ultrasonic signal of the at least one ultrasonic sensor is reflected, and the direction information indicating a direction along which the ultrasonic signal propagated between the reflection location and the at least one ultrasonic sensor.

[0062] Subsequently, clusters of detections are formed based on the received detections, with one cluster containing multiple detections (S11).

[0063] For the clusters, information on the statistical distribution of the position information and information on the statistical distribution of the direction information of the detections assigned to each cluster are then calculated (S12).

[0064] Finally, the clusters assigned to an object are classified into object classes based on information about the statistical distribution of the position information and information about the statistical distribution of the direction information (S13).

[0065] The invention has been described by way of the above examples, however it will be appreciated that numerous modifications and variations are possible without departing from the scope of the invention as defined in the appended claims. [Explanation of symbols]

[0066] 1 vehicle 2. Ultrasonic Sensor 3 Calculation Unit C Cluster E Decision Tree O Object s1 First threshold s2 Second threshold

Claims

1. 1. A method for classifying objects (O) into object classes based on information from at least one ultrasonic sensor (2) of a vehicle (1), comprising: The method comprises the steps of: a) receiving (S10) a plurality of detections of at least one ultrasonic sensor (2) of the vehicle (1), each detection being assigned position information and direction information, the position information indicating a reflection point from which the ultrasonic signal of the at least one ultrasonic sensor (2) is reflected, and the direction information indicating a direction along which the ultrasonic signal propagates between the reflection point and the at least one ultrasonic sensor (2); b) forming clusters (C) of detections based on the received detections (S11), where one cluster (C) includes a plurality of detections; c) calculating information on the statistical distribution of the position information and the statistical distribution of the direction information of the detections assigned to each cluster (C) (S12); d) a step (S13) of classifying the objects (O) into object classes based on information about the statistical distribution of the position information of the clusters (C) and information about the statistical distribution of the direction information; A method comprising:

2. 2. The method of claim 1, wherein the location information includes at least one first and one second coordinate, and the information about the statistical distribution of the location information includes information based on a covariance matrix of the first and second coordinates of the location information.

3. The method described in claim 2, characterized in that the information regarding the statistical distribution of the location information includes eigenvalues ​​of the covariance matrix of the first and second coordinates of the location information.

4. The method described in claim 3, characterized in that the information regarding the statistical distribution of the location information includes a ratio of the eigenvalues ​​of the covariance matrix of the first and second coordinates of the location information.

5. 5. The method according to claim 1, wherein the information about the statistical distribution of the direction information of the detections comprises the variance of the direction information.

6. 5. The method according to claim 1, wherein the information about the statistical distribution of the direction information of the detections comprises the time derivative of the direction information.

7. 5. The method according to claim 1, wherein the information about the statistical distribution of the direction information of the detections comprises the time derivative of the direction information filtered by a filter function.

8. A method according to any one of claims 1 to 4, characterized in that the classification is performed based on at least first and second thresholds (s1, s2), the first threshold (s1) being a threshold for information relating to the statistical distribution of position information, and the second threshold (s2) being a threshold for information relating to the statistical distribution of detection direction information.

9. The method of claim 8, wherein the first and second thresholds (s1, s2) are determined by training data having label information for each object class.

10. A method according to any one of claims 1 to 4, characterized in that the classification is performed using a decision tree (E) or a random decision forest.

11. A method according to any one of claims 1 to 4, characterized in that a neural network is used for the classification, and the neural network is trained using training data having label information for each object class.

12. 5. The method according to claim 1, wherein a classification into object classes "vehicle" and "non-vehicle" is performed.

13. 5. The method according to claim 1, wherein depending on the classification result of the object (O), a height estimation of the object (O) is or is not performed.

14. A system for classifying objects (O) into object classes based on information from at least one ultrasonic sensor (2) of a vehicle (1), comprising: The system comprises a calculation unit (3), which performs the following steps: e) receiving a plurality of detections of at least one ultrasonic sensor (2) of the vehicle (1), each detection being assigned position information and direction information, the position information indicating a reflection point from which the ultrasonic signal of said at least one ultrasonic sensor (2) is reflected, and the direction information indicating a direction along which the ultrasonic signal propagates between the reflection point and said at least one ultrasonic sensor (2); f) forming clusters (C) of detections based on the received detections, where one cluster (C) includes a plurality of detections; g) calculating information about the statistical distribution of the position information and the statistical distribution of the direction information of the detections assigned to each cluster (C); h) classifying the objects (O) into object classes based on information about the statistical distribution of the location information and information about the statistical distribution of the direction information of the clusters (C); A system configured to perform the

15. A vehicle including the system of claim 14.

Images