Method for classifying objects

Abstract

The invention relates to a method for classifying objects into object classes on the basis of information of at least one ultrasonic sensor of a vehicle.

Description

[0001] Methods for object classification

[0002] The invention relates to a method for classifying objects into object classes

[0003] It is known to estimate the height of objects using geometric methods based on information acquired by ultrasonic sensors. For example, height estimation can distinguish a linear object, such as a curb, from a linear object in the shape of a wall.

[0004] The problem with height estimation is that the known methods for height estimation lead to incorrect results when applied to objects in the form of vehicles.

[0005] Based on this, it is the object of the invention to provide a method for object classification which enables a reliable classification of objects into predetermined object classes.

[0006] This object is achieved by a method having the features of independent patent claim 1. Preferred embodiments are the subject of the dependent claims. A system for classifying objects into object classes based on information from at least one ultrasonic sensor of a vehicle is the subject of independent patent claim 14, and a vehicle with such a system is the subject of independent patent claim 15.

[0007] According to a first aspect, a method for classifying a

[0008] The method is disclosed in the following: Classification of an object into an object class based on information from at least one ultrasonic sensor.

[0009] First, multiple detections from at least one ultrasonic sensor of a vehicle are received. It is understood that the detections can be obtained through the interaction of multiple ultrasonic sensors. Each detection is assigned position information and direction information. The position information indicates the reflection location at which the reflection of an ultrasonic signal from the at least one ultrasonic sensor occurred. The position information can, in particular, comprise coordinates that indicate the reflection location. The direction information indicates the direction along which the ultrasonic signal propagates between the reflection location and the at least one ultrasonic sensor. The direction information can, for example, be specified by an angle, in particular an azimuth angle.

[0010] The received detections are then assigned to clusters, with each cluster comprising multiple detections. The clusters thus form groups of detections. One or more clusters can refer to an object, meaning that the detections originating from an object can be assigned to one or more clusters.

[0011] After the clusters are formed, information on the statistical distribution of the position information and the statistical distribution of the direction information are calculated for the respective clusters, based on the position information and the direction information of the detections assigned to the respective cluster. Finally, the object is classified into an object class based on the information on the statistical distribution of the position information and the statistical distribution of the direction information of the clusters.

[0012] The technical advantage of the disclosed method is that the information on the statistical distribution of the position information and the information on the statistical distribution of the directional information of the clusters can be used to aid in determining which object class a cluster, and thus an object assigned to the cluster, should be assigned to. This allows for accurate and efficient object classification, particularly with the goal of excluding detections in clusters assigned to the "vehicle" object class from the height estimation.

[0013] According to one embodiment, the position information comprises at least a first and a second coordinate. The information on the statistical distribution of the position information comprises information based on the covariance matrix of the first and second coordinates of the position information. By taking into account the covariance matrix of the first and second coordinates of the position information, the distribution of detections in a longitudinal direction and a transverse direction perpendicular thereto can be used to classify the object classes. In particular, the determinant of the covariance matrix of the first and second coordinates of the position information can be used as a decision criterion. This determinant is greater for two-dimensionally shaped objects (in particular objects with curves and / or corners), such as vehicles, than for linear objects (e.g., curbs or wall objects).According to one embodiment, the information regarding the statistical distribution of the position information includes the eigenvalues ​​of the covariance matrix of the first and second coordinates of the position information. The eigenvalues ​​of the covariance matrix of the first and second coordinates of the position information provide a measure of the extent of a cluster along its major and minor axes, so that the shape of the object can be inferred from the eigenvalues.

[0014] According to one embodiment, the information regarding the statistical distribution of the position information includes the ratio of the eigenvalues ​​of the covariance matrix of the first and second coordinates of the position information. The ratio of the eigenvalues ​​differs greatly for linear objects and two-dimensionally shaped objects and can therefore advantageously be used as a basis for decision-making.

[0015] According to one embodiment, the information regarding the statistical distribution of the directional information of the detections includes the variance of the directional information. The variance of the directional information is very low for linear objects and very high for curved objects such as poles. For vehicles, the variance of the directional information lies between the variance of linear objects and curved objects, since the vehicle contour has both flat and curved surfaces.

[0016] According to one embodiment, the information regarding the statistical distribution of the directional information from the detections includes the temporal derivative of the directional information. This allows the change in the directional information over time to be used as a basis for decision-making regarding object classification.

[0017] According to one embodiment, the information regarding the statistical distribution of the directional information of the detections comprises the temporal derivative of directional information filtered using a filter function. The filter function can, in particular, be a filter for filtering outliers. This allows the change in the filtered directional information over time to be used as a basis for decision-making for object classification.

[0018] According to one embodiment, the classification is performed based at least on a first and a second threshold, wherein the first threshold specifies a threshold for the information on the statistical distribution of the position information, and the second threshold specifies a threshold for the information on the statistical distribution of the directional information of the detections. It is understood that more than two decision rules based on thresholds are used as a decision basis for object classification.

[0019] According to one embodiment, the first and second threshold values ​​are determined using training data. The training data includes object information and label information associated with this object information. The label information indicates which object class the respective object belongs to. Using the training data, decision rules can be created that determine the thresholds of the information regarding the statistical distribution of the position information and the information regarding the statistical distribution of the direction information at which classification into the respective object classes can occur. According to one embodiment, the classification is performed using a decision tree or a random forest. This allows object classification with low computational effort.

[0020] According to another embodiment, a neural network is used for classification, wherein the neural network was trained using training data that includes label information on the respective object classes.

[0021] According to another embodiment, classification is performed into the object classes "vehicle" and "no vehicle." This allows objects representing vehicles to be distinguished from other objects, for example, objects belonging to the object classes "wall" or "curb."

[0022] According to another embodiment, a height estimate of the object is performed or not performed depending on the result of the object's classification. In particular, no height estimate can be performed for detections relating to the object class "vehicle." This minimizes the computational effort and the error rate in the height estimation.

[0023] According to a further aspect, a system for classifying objects into object classes based on information from at least one ultrasonic sensor of a vehicle is disclosed. The system comprises a computing unit configured to perform the following steps: a) receiving a plurality of detections of at least one

[0024] Ultrasonic sensor of a vehicle, with each detection being assigned a

[0025] Position information and direction information are assigned to the object, wherein the position information indicates the reflection location at which the reflection of an ultrasonic signal from the at least one ultrasonic sensor occurred, and wherein the direction information indicates the direction along which the ultrasonic signal propagates between the reflection location and the at least one ultrasonic sensor; b) forming clusters of detections based on the received detections, wherein a cluster comprises a plurality of detections; c) calculating information on the statistical distribution of the position information and information on the statistical distribution of the direction information of the detections that are assigned to the respective cluster; d) classifying the object into an object class based on the information on the statistical distribution of the position information and the statistical distribution of the direction information of the clusters.

[0026] According to yet another aspect, a vehicle having a system for classifying objects into object classes is disclosed.

[0027] The terms “approximately”, “essentially” or “about” mean, in the sense of the invention, deviations from the exact value by + / - 10%, preferably by + / - 5% and / or deviations in the form of changes that are insignificant for the function.

[0028] Further developments, advantages, and possible applications of the invention will become apparent from the following description of exemplary embodiments and from the figures. All described and / or illustrated features, individually or in any combination, are fundamentally part of the invention, regardless of their summary in the claims or their reference back to them. The content of the claims is also incorporated into the description.

[0029] The invention is explained in more detail below with reference to exemplary embodiments and the figures. They show:

[0030] Fig. 1 shows, by way of example and schematically, a vehicle with an ultrasonic sensor system comprising a plurality of ultrasonic sensors distributed circumferentially around the vehicle and a computer unit for evaluating the information provided by the ultrasonic sensor system;

[0031] Fig. 2 shows an example of a diagram showing the position information of detections relating to a parallel parking situation between two vehicles;

[0032] Fig. 3 shows, by way of example, a diagram showing the position information and partly the direction information of detections relating to the parallel parking situation according to Fig. 2;

[0033] Fig. 4 shows, by way of example, several clusters that were determined based on the detections shown in Fig. 2 and Fig. 3;

[0034] Fig. 5, for example, shows a decision tree for classifying objects into object classes; and

[0035] Fig. 6 shows an example of a block diagram illustrating the steps of the method for classifying objects into object classes. Figure 1 shows an example and a rough schematic of a vehicle 1. The vehicle 1 has a plurality of ultrasonic sensors 2, which are used to detect the surroundings.

[0036] The ultrasonic sensors 2 are coupled to at least one computing unit 3, by means of which the method described below for classifying an object 0 into an object class is carried out.

[0037] Figure 2 shows an example diagram in which a plurality of detections are plotted, recorded by the ultrasonic sensors 2 of the vehicle 1 and assigned to several objects 0. The detections relate, for example, to a parallel parking situation in which a parallel parking space is formed between two vehicles. The linearly arranged detections between the two vehicles refer, for example, to a curb that borders the parking space at the sides.

[0038] A detection is represented by a point. The reflection location at which the detection occurred due to a reflection from an object in the surroundings is specified by position information. The position information can include at least two coordinates by which the reflection location is defined in a horizontal plane. These are, in particular, x- and y-coordinates, which are plotted on the respective diagram axes in the diagram in Fig. 2. In other words, the position information is specified in Cartesian coordinates.

[0039] Alternatively, it is possible to specify the position information in a cylindrical coordinate system or a spherical coordinate system. The statistical distribution of the position information is used to classify an object.

[0040] If available, the position information can also contain information about the height of the object area where the reflection occurred. In other words, the position information can indicate the location of the reflection in three-dimensional space. Information about the third dimension (i.e., height) can also be used to classify the object.

[0041] Directional information is also determined for each detection. The directional information indicates the direction from which the detection was received. The directional information can, for example, be an angle spanned in the horizontal plane that indicates the direction of a connecting line that connects the reflection location with the sensor position of the ultrasonic sensor through which the reflected ultrasonic signal was emitted and / or through which the reflected ultrasonic signal was received. The angle can, for example, be measured relative to a coordinate axis, for example relative to the x-axis. The directional information can thus, for example, include the angle that the connecting line that connects the reflection location with the sensor position of the ultrasonic sensor encloses with the x-axis.

[0042] Fig. 3 shows, by way of example, a diagram similar to that of Fig. 2, wherein each detection assigned to the left-hand vehicle is additionally assigned directional information. The directional information is indicated in each case by a line. The line indicates the direction in which the detection was determined. In other words, the line indicates the direction in which the ultrasonic sensor 2 was located when transmitting and / or receiving the ultrasonic signal. It should be noted that due to the high propagation speed and the associated short propagation time of the ultrasonic signal between transmission and reception, it can be approximately assumed that the ultrasonic sensor 2 is located at the same location when transmitting an ultrasonic signal and when receiving the reflected components of this ultrasonic signal.

[0043] To assign multiple detections to an object, at least one cluster C is formed from the detected detections. Known clustering algorithms can be used to form a cluster, such as density-based clustering methods, partitioning clustering methods, etc. In particular, a K-means clustering algorithm or a DBSCAN algorithm can be used to form the clusters.

[0044] Fig. 4 illustrates several clusters C determined based on the detections shown in Figs. 2 and 3. The ellipses shown in Fig. 4 each represent the covariance matrix of the xy coordinates of those detections assigned to the respective cluster C. The clusters C refer to spatially connected groups of detections and preferably indicate how the respective group of detections is spatially arranged in a horizontal plane (i.e., the xy plane).

[0045] Preferably, a threshold for the number of detections is specified. The number of detections that are to form a cluster must exceed this threshold. This prevents a cluster from being formed based on a small number of detections.

[0046] After clustering, information on the statistical distribution of the position information and information on the statistical distribution of the direction information of the detections can be determined for the respective clusters.

[0047] In particular, the mean of the position information is calculated for each cluster C. This mean thus indicates the center of the cluster. It can be determined, for example, by averaging the x- and y-coordinates of the detections in a cluster. Furthermore, the covariance to the x- and y-coordinates of the detections assigned to the respective cluster can be calculated. In other words, the two-dimensional Gaussian distribution of the x- and y-coordinates of the detections is calculated.

[0048] The information on the statistical distribution of the directional information of the detections in a cluster can include the mean and variance of the directional information of the detections in a cluster. The mean and variance can either be calculated directly, or the directional information can be filtered before calculating the mean and variance, for example, using a smoothing filter or a filter to filter out statistical outliers.

[0049] From the information on the statistical distribution of the position information and the information on the statistical distribution of the direction information, further information or quantities can be calculated that can be used to classify the objects:

[0050] For example, the determinant of the covariance matrix of the first and second coordinates of the position information can be calculated. This is larger for multidimensional objects than for linear objects. Thus, the determinant of the covariance matrix can be used to distinguish between multidimensional objects and linear objects.

[0051] Furthermore, the eigenvalues ​​of the covariance matrix of the first and second coordinates of the position information can be calculated. In particular, a first and a second eigenvalue are obtained. The eigenvalues ​​indicate the extension of cluster C along its major axis and its perpendicular minor axis. In particular, the eigenvalues ​​indicate the length of the major and minor axes of an ellipse, which can be used to model the position and orientation of cluster C. These ellipses assigned to cluster C are shown in Fig. 4.

[0052] Furthermore, the quotient of the first and second eigenvalues ​​of the covariance matrix of the first and second coordinates of the position information can be calculated. The ratio of the eigenvalues ​​provides an indication of whether the object is linear or not, since the ratio of the eigenvalues ​​differs significantly for linear objects and two-dimensional objects.

[0053] The variance of the directional information of the detections in a cluster also provides an indication of whether the object is linear or two-dimensional. For example, the variance of the directional information is very small for linear objects, but very high for round objects such as posts or similar objects. The variance of the directional information for vehicles lies midway between the variance of the directional information for linear and round objects, since vehicles have both straight vehicle areas and diffusely reflective areas, such as mirrors, door handles, etc. Furthermore, the temporal derivative of statistical properties of the directional information of the detections or the filtered directional information of the detections can be determined, i.e., for example, the change in the mean, the variance, etc.

[0054] Preferably, the information on the statistical distribution of the position information and the information on the statistical distribution of the direction information are determined iteratively, such that the formation of clusters C is updated when one or more detections are received. This results in updated clusters. The information on the statistical distribution of the position information and the information on the statistical distribution of the direction information of the detections assigned to a cluster are also updated after a cluster is updated, i.e., recalculated based on the new detections added to the cluster.

[0055] The information on the statistical distribution of the position information and the information on the statistical distribution of the direction information of the detections can then be used to form decision rules, whereby the objects are classified into object classes based on the decision rules.

[0056] Training data can be used to generate the decision rules. This training data contains information about objects and label information assigned to the objects. The label information indicates which object class the respective object should be assigned to. The decision rules can, in particular, be thresholds. The level of the thresholds can be determined based on the training data. The thresholds can be assigned to specific information about the statistical distribution of the position information or specific information about the statistical distribution of the direction information. The thresholds can, in particular, indicate that information below the threshold indicates classification into a first object class and information above the threshold indicates classification into a second object class.

[0057] A decision tree or a random forest can be used to classify objects.

[0058] The structure and decision rules of the decision tree or random forest can be determined by the training data.

[0059] Fig. 5 shows an example of a decision tree. The decision tree serves as an example for classifying objects into the classes "curb," "wall," and "vehicle." In particular, the decision tree serves to classify objects into the object classes "vehicle" and "non-vehicle."

[0060] The decision rules of the decision tree refer to information on the statistical distribution of the position information and information on the statistical distribution of the directional information. The quotient of the first and second eigenvalues ​​of the covariance matrix of the first and second coordinates of the position information is used as the information on the statistical distribution of the position information, i.e., the ratio of the two eigenvalues ​​of the covariance matrix. The variance of the directional information is used as the information on the statistical distribution of the directional information. Based on the information on the statistical distribution of the position information, the information on the statistical distribution of the directional information, and the thresholds associated with this information, the detections of the respective cluster C can be assigned to an object class.This makes it possible to decide in particular whether the detections of the recognized cluster C refer to the object class “vehicle” or not.

[0061] According to the decision tree of Fig. 5, if the variance of the directional information is greater than 0.00665 and the ratio of the eigenvalues ​​is greater than 0.0012, it can be assumed with a probability of 99% that the detections of the cluster refer to a vehicle.

[0062] After classifying the objects or clusters assigned to objects into object classes, a height estimation algorithm can be selectively performed based on the information from the ultrasonic sensors. In particular, detections assigned to clusters of the "vehicle" object class can be excluded from the height estimation to avoid incorrect estimates.

[0063] Fig. 6 shows a schematic representation of the steps of a method according to the invention for classifying objects into object classes by means of an ultrasonic sensor system of a vehicle.

[0064] First, several detections from at least one ultrasonic sensor of a vehicle are received (S10). Each detection is assigned position information and direction information. The position information indicates the reflection location at which the reflection of an ultrasonic signal from the at least one ultrasonic sensor occurred. The direction information indicates the direction along which the ultrasonic signal propagates between the reflection location and the at least one ultrasonic sensor.

[0065] Clusters of detections are then formed based on the received detections. A cluster comprises multiple detections (S11).

[0066] For the clusters, information on the statistical distribution of the position information and information on the statistical distribution of the direction information of the detections assigned to the respective cluster are then calculated (S12).

[0067] Finally, a cluster assigned to an object is classified into an object class based on the information on the statistical distribution of the position information and the statistical distribution of the direction information (S13).

[0068] The invention has been described above using exemplary embodiments. It is understood that numerous changes and modifications are possible without departing from the scope of protection defined by the patent claims.

[0069] List of reference symbols 1 vehicle

[0070] 2 ultrasonic sensors

[0071] 3 Computing unit

[0072] C Cluster E Decision tree

[0073] 0 Object s1 first threshold s2 second threshold

Claims

Patent claims 1) Method for classifying objects (0) into object classes based on information from at least one ultrasonic sensor (2) of a vehicle (1), comprising the following steps: a) Receiving multiple detections from at least one ultrasonic sensor (2) of a vehicle (1), wherein each detection is associated with position information and direction information, the position information indicating the reflection location where the reflection of an ultrasonic signal from the at least one ultrasonic sensor (2) occurred, and the direction information indicating the direction along which the ultrasonic signal propagates between the reflection location and the at least one ultrasonic sensor (2) (S10); b) Forming clusters (C) of detections based on the received detections, wherein a cluster (C) comprises multiple detections (S11);c) Calculating information on the statistical distribution of position information and information on the statistical distribution of direction information of the detections assigned to the respective cluster (C) (S12); d) Classifying the object (0) into an object class based on the information on the statistical distribution of position information and the statistical distribution of direction information of the clusters (C) (S13).; 2) Method according to claim 1, characterized in that the position information comprises at least a first and a second coordinate, wherein the information on the statistical distribution of the position information comprises information based on the Covariance matrix of the first and second coordinates of the Position information is used. 3) Method according to claim 2, characterized in that the information on the statistical distribution of the position information includes the eigenvalues ​​of the covariance matrix of the first and second coordinates of the position information. 4) Method according to claim 3, characterized in that the information on the statistical distribution of the position information includes the ratio of the eigenvalues ​​of the covariance matrix of the first and second coordinate of the position information. 5) Method according to one of the preceding claims, characterized in that the information on the statistical distribution of the directional information of the detections includes the variance of the directional information. 6) Method according to one of the preceding claims, characterized in that the information on the statistical distribution of the directional information of the detections includes the temporal derivative of the directional information. 7) Method according to one of the preceding claims, characterized in that the information on the statistical distribution of the directional information of the detections comprises the temporal derivative of directional information filtered by means of a filter function. 8) Method according to one of the preceding claims, characterized in that the classification is based at least on A first and a second threshold (s1, s2) are applied, wherein the first threshold (s1) specifies a threshold for the information on the statistical distribution of the position information and the second threshold (s2) specifies a threshold for the information on the statistical distribution of the direction information of the detections. ) Method according to claim 8, characterized in that the first and second thresholds (s1, s2) are determined using training data that includes label information for the respective object classes. 0) Method according to any of the preceding claims, characterized in that the classification is performed using a decision tree (E) or a random forest.1) A method according to any one of the preceding claims, characterized in that a neural network is used for classification, wherein the neural network was trained using training data containing label information for the respective object classes. 2) A method according to any one of the preceding claims, characterized in that classification into the object classes "vehicle" and "no vehicle" is performed. 3) A method according to any one of the preceding claims, characterized in that, depending on the result of the classification of object (0), a height estimate of object (O) is performed or not. ) System for classifying objects (0) into object classes based on information from at least one ultrasonic sensor (2) of a vehicle (1), wherein the system has a computing unit (3) configured to perform the following steps: e) Receiving multiple detections from at least one Ultrasonic sensor (2) of a vehicle (1), wherein each detection is associated with position information and direction information, the position information indicating the reflection location where the reflection of an ultrasonic signal from the at least one ultrasonic sensor (2) occurred, and the direction information indicating the direction along which the ultrasonic signal propagates between the reflection location and the at least one ultrasonic sensor (2); f) forming clusters (C) of detections based on the received detections, wherein a cluster (C) comprises several detections; g) calculating information on the statistical distribution of the position information and information on the statistical distribution of the direction information of the detections that are assigned to the respective cluster (C);h) Classifying the object (0) into an object class based on the information on the statistical distribution of the position information and the statistical distribution of the direction information of the clusters (C). ) Vehicle comprising a system according to claim 14.;

Images