Method and apparatus for verifying a vehicle recognition algorithm

Abstract

Methods for verifying a vehicle recognition algorithm, comprising: - Generating a point cloud (100) representing a target vehicle (10), wherein the point cloud (100) comprises a normal distribution (110, 120, 130), wherein the point cloud (100) comprises a first multidimensional normal distribution (110) running along a first edge (11) of the target vehicle (10); - Capturing the point cloud (100) with a vehicle detection algorithm; - Comparing the output of the vehicle recognition algorithm with data based on the target vehicle (10); and - Verifying or falsifying the vehicle recognition algorithm based on comparison.

Description

[0001] The present invention relates to a method and a device for verifying a vehicle recognition algorithm.

[0002] Verifying the functionality of vehicle recognition algorithms is regularly a complex and time-consuming process.

[0003] A method for the synthetic generation of radar and lidar point clouds is known from publication DE 10 2021 201 331 A1.

[0004] It is an object of the present invention to provide an improved method and a device for verifying a vehicle recognition algorithm.

[0005] This problem is solved by the subject matter of the independent claims. Advantageous further developments are specified in the dependent claims.

[0006] According to a first aspect, a procedure for verifying a vehicle recognition algorithm is specified.

[0007] The method according to the invention comprises the steps of generating a point cloud representing a target vehicle, wherein the point cloud comprises a normal distribution, capturing the point cloud with a vehicle recognition algorithm, comparing the output of the vehicle recognition algorithm with data based on the target vehicle, and verifying or falsifying the vehicle recognition algorithm based on the comparison.

[0008] The method serves, in particular, to verify the functionality and especially the accuracy of one or more vehicle recognition algorithms. Specifically, one or more vehicle recognition algorithms can be classified, and their results or outputs can be verified or falsified. The method is specifically used to estimate the position, length, width, and / or orientation of a target vehicle, or to verify or falsify such estimates.

[0009] Vehicle recognition algorithms are specifically those used in motor vehicles, generating point clouds from sensor data, particularly from one or more radar sensors, during the scanning of the vehicle's surroundings. These radar sensors are used to detect one or more objects in the vehicle's vicinity, especially other vehicles such as passenger cars (also referred to as target vehicles), and to control the vehicle's steering accordingly.

[0010] For example, one or more driver assistance programs, such as a collision warning assistant, adaptive cruise control, a traffic jam assistant and / or a partially or fully autonomous driving mode, use the sensor data and, in particular, the vehicle recognition algorithm(s) as an input source for their control.

[0011] Since verifying such vehicle recognition algorithms in real-world driving conditions is complex and requires particularly high-resolution sensors, the method according to the invention provides a solution for quickly and easily generating highly realistic point clouds of a target vehicle. This allows these vehicle recognition algorithms to be tested, and their function, and especially their accuracy, to be verified or falsified. The point cloud or point clouds generated by the method can also be referred to as the "ground truth," i.e., the actual ground reality against which the vehicle recognition algorithms are then measured.

[0012] As a first step in the process, a point cloud representing a target vehicle is generated. It is understood that multiple point clouds can also be generated, particularly those relating to or representing several target vehicles and / or the movement of one or more such vehicles. For the sake of simplicity, the following discussion will primarily refer to a single point cloud representing a target vehicle at a given point in time and location relative to the vehicle.

[0013] The point cloud follows a normal distribution. A normal distribution can also be called a Gaussian distribution, Gaussian function, Gaussian normal distribution, Gaussian distribution curve, Gaussian curve, Gaussian bell curve, Gaussian bell function, Gaussian bell, or bell curve. This normal distribution represents a probability distribution, in this case of points that relate to or characterize the target vehicle. In particular, several normal distributions can be superimposed or combined into a point cloud, as will be described later.

[0014] In a further step, this generated point cloud is then captured by a vehicle recognition algorithm. This capture can also be described as evaluation or scanning, whereby the point cloud takes the place of the actually captured sensor data, for example from a radar sensor, and is processed by the vehicle recognition algorithm.

[0015] In a subsequent step, the result, i.e., the output of the vehicle recognition algorithm, is compared with data based on the target vehicle. This data, derived from the target vehicle, can also be a point cloud, particularly one that actually represents or corresponds to the vehicle. This data can specifically be so-called ground truth, meaning data that reflects reality and has been previously provided.

[0016] This checks whether and to what extent the vehicle recognition algorithm has recognized the point cloud, or in particular the normal distribution it contains that represents the motor vehicle, and to what extent this corresponds to actual reality.

[0017] The vehicle recognition algorithm is then verified or falsified based on the comparison. Specifically, the vehicle recognition algorithm is quantified and / or qualified, or its performance is determined. For example, if the target vehicle is detected above a threshold, the vehicle recognition algorithm is verified. Alternatively, if the target vehicle is detected below a threshold, the vehicle recognition algorithm is falsified. Furthermore, additional data can be provided that includes performance, qualification, and / or quantification of the vehicle recognition algorithm. The result and / or the additional data can be displayed on a user interface, such as a monitor, and / or stored as data for further processing.

[0018] The method according to the invention makes it possible to verify the functionality and, in particular, the accuracy of one or more vehicle recognition algorithms. Specifically, the solution according to the invention allows one or more vehicle recognition algorithms to be classified and, in particular, their results or outputs to be verified or falsified.

[0019] According to the invention, the point cloud comprises a multidimensional normal distribution.

[0020] The normal distribution encompassed by the point cloud extends in several dimensions, in particular at least two, and more specifically exactly two dimensions, and can also be described as a multivariate, and in particular a bivariate, normal distribution. In particular, the at least two dimensions are perpendicular to each other, one of which coincides with an edge of the target vehicle.

[0021] This makes it possible to check vehicle recognition algorithms with particular precision.

[0022] The first, second, and third normal distributions described below are assigned ordinal numbers arbitrarily, and it is understood that they can exist or be used independently and are not necessarily interdependent. In particular, only the third normal distribution and / or a combination of the second and third normal distributions can be used without considering the first normal distribution.

[0023] According to the invention, the point cloud comprises a first multidimensional normal distribution that runs along a first edge of the target vehicle.

[0024] The first multidimensional normal distribution is specifically designed as a two-dimensional or bivariate normal distribution. This extends along or over a first edge of the target vehicle. This edge can be either a longitudinal or a transverse edge of the target vehicle. For the sake of simplicity, it is assumed in the following that the first edge of the target vehicle, along which the first point cloud runs or which it covers, is a longitudinal edge of the target vehicle.

[0025] The edge, which can also be called a side, is essentially a straight line, ideally assumed to be the vehicle's boundary. Specifically, the first edge and the second edge, described later, define a so-called bounding box of the target vehicle, i.e., a rectangle within which the target vehicle and all its components are located.

[0026] This also makes it possible to check vehicle recognition algorithms with particular precision.

[0027] According to further training, the first multidimensional normal distribution spans a standard deviation ellipse, such that twice the standard deviation extends over half the length of the target vehicle.

[0028] According to this further development, the first multidimensional normal distribution is thus selected or dimensioned such that the two-dimensional standard deviation, which is typically given in the form of an ellipse, is as long in the second standard deviation as half the length of the target vehicle.

[0029] In particular, the major axis of the ellipse coincides with the longitudinal edge of the target vehicle and / or a center point of the ellipse coincides with the center point of the longitudinal edge of the target vehicle and / or extends along it. Specifically, the standard deviation ellipse of three times the standard deviation of the first normal distribution extends along one and a half times half the length of the longitudinal edge of the target vehicle and thus coincides with it.

[0030] This further training also makes it possible to check vehicle recognition algorithms particularly precisely.

[0031] According to a further development, the point cloud includes a second multidimensional normal distribution that runs along a second edge of the target vehicle, with the second edge being perpendicular to the first edge.

[0032] The second multidimensional normal distribution is specifically designed as a two-dimensional or bivariate normal distribution. This extends along or over a second edge of the target vehicle. This edge can be either a longitudinal or a transverse edge of the target vehicle. For the sake of simplicity, it is assumed in the following that the second edge of the target vehicle, along which the second point cloud runs or which it covers, is a transverse edge of the target vehicle that is perpendicular to the longitudinal edge along which the first point cloud runs.

[0033] This second edge, the transverse edge, together with the first edge, the longitudinal edge, defines the so-called bounding box of the target vehicle.

[0034] This further training also makes it possible to check vehicle recognition algorithms particularly precisely.

[0035] According to further training, the second multidimensional normal distribution spans a standard deviation ellipse, such that the twice standard deviation extends over half the width of the target vehicle.

[0036] According to this further development, the second multidimensional normal distribution is thus selected or dimensioned such that the two-dimensional standard deviation, which is also typically given in the form of an ellipse, is as long in the second standard deviation as half the width of the target vehicle.

[0037] In particular, the major axis of the ellipse coincides with the transverse edge of the target vehicle and / or a center point of the ellipse coincides with the center point of the transverse edge of the target vehicle and / or extends along it. Specifically, the standard deviation ellipse of three times the standard deviation of the second normal distribution extends along one and a half times half the width of the transverse edge of the target vehicle and thus coincides with it.

[0038] This further training also makes it possible to check vehicle recognition algorithms particularly precisely.

[0039] According to further training, the point cloud includes a third multidimensional normal distribution that runs along the entire target vehicle.

[0040] The third point cloud covers, in particular, the entire target vehicle or the associated bounding box, or encloses it.

[0041] This advanced training also makes it possible to verify vehicle recognition algorithms with particular precision. In particular, the third point cloud serves to take into account reflections, especially those that form under the target vehicle.

[0042] According to further training, the third multidimensional normal distribution spans a standard deviation ellipse, such that the twice standard deviation extends over half the width and half the length of the target vehicle.

[0043] In particular, a center point of the ellipse of the third multidimensional normal distribution coincides with a center point of the target vehicle or the bounding box. Likewise, the major axis of the third ellipse runs parallel to one or both longitudinal edges, and the minor axis runs parallel to one or both transverse edges of the target vehicle or the bounding box.

[0044] In particular, the standard deviation ellipse of the three standard deviations of the third normal distribution extends along one and a half times half the length of the longitudinal edge and along one and a half times half the width of the transverse edge of the target vehicle and thus coincides.

[0045] This further training also makes it possible to check vehicle recognition algorithms particularly precisely.

[0046] According to further training, the point density of the point cloud is varied depending on the distance and / or speed of the target vehicle.

[0047] In particular, the point density is reduced at a greater distance compared to a smaller distance and / or the point density is increased at a greater speed compared to a lesser speed.

[0048] This further training also makes it possible to check vehicle recognition algorithms particularly precisely.

[0049] For use cases or application situations that may arise during the procedure and are not explicitly described here, it may be provided that, according to the procedure, an error message and / or a request for user feedback is issued and / or a default setting and / or a predetermined initial state is set.

[0050] According to another aspect, a device for verifying a vehicle recognition algorithm is specified, wherein the device is configured to perform a method according to one of the embodiments described above.

[0051] The device may include a data processing device or a processor unit configured to carry out an embodiment of the method according to the invention.

[0052] The processor device can comprise at least one microprocessor and / or at least one microcontroller and / or at least one FPGA (Field Programmable Gate Array) and / or at least one DSP (Digital Signal Processor). In particular, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or an NPU (Neural Processing Unit) can be used as the microprocessor. Furthermore, the processor device can comprise program code configured to execute the embodiment of the method according to the invention when executed by the processor device. The program code can be stored in a data memory of the processor device. The processor device can, for example, be based on at least one circuit board and / or on at least one SoC (System on Chip).

[0053] As a further solution, the invention also includes a computer-readable storage medium comprising program code which, when executed by a computer or a computer network, causes it to execute an embodiment of the method according to the invention. The storage medium can be provided at least partially as a non-volatile data storage medium (e.g., as flash memory and / or as an SSD - solid state drive) and / or at least partially as a volatile data storage medium (e.g., as RAM - random access memory). The storage medium can be located within the computer or computer network. However, the storage medium can also be operated, for example, as an app store server and / or cloud server on the internet. The computer or computer network can provide a processor circuit with, for example, at least one microprocessor.The program code can be provided as binary code and / or as assembly code and / or as source code of a programming language (e.g. C) and / or as a program script (e.g. Python).

[0054] The invention also includes combinations of the features of the described embodiments. The invention therefore also includes realizations that each exhibit a combination of the features of several of the described embodiments, provided that the embodiments have not been described as mutually exclusive.

[0055] The following are exemplary embodiments of the invention described. This is illustrated by: Fig. 1 a schematic view of an embodiment of a method for verifying a vehicle recognition algorithm; Fig. 2 a further schematic view of an embodiment of a method for verifying a vehicle recognition algorithm; Fig. 3 a further schematic view of an embodiment of a method for verifying a vehicle recognition algorithm; and Fig. 4 another schematic view of an embodiment of a method for verifying a vehicle recognition algorithm.

[0056] The exemplary embodiments described below are preferred embodiments of the invention. In these exemplary embodiments, the described components each represent individual features of the invention, which can be considered independently of one another and each further develops the invention independently. Therefore, the disclosure is intended to include combinations of features of the embodiments other than those shown. Furthermore, the described embodiments can also be supplemented by further features of the invention already described.

[0057] In the figures, identical reference symbols denote functionally equivalent elements.

[0058] Fig. Figure 1 shows a schematic view of an embodiment of a method for verifying a vehicle recognition algorithm.

[0059] An example of a target vehicle 10 is shown, idealized by a rectangular bounding box. The target vehicle is located in the field of view 21 of a radar sensor 20, which is positioned at the origin of the coordinate system. The target vehicle extends slightly inclined along the x and y directions of the coordinate system within the field of view 21. This is, in particular, a random positioning of the target vehicle 10, the detection of which is to be checked, verified, or falsified by a vehicle detection algorithm.

[0060] The method described below is carried out by a device that is not shown in detail.

[0061] First, a point cloud 100 is generated, representing the target vehicle 10.

[0062] The point cloud 100 comprises a total of three normal distributions 110, 120, 130, each of which is two-dimensional.

[0063] The first multidimensional normal distribution 110 runs along a first edge 11 of the target vehicle 10, which is here referred to as the left longitudinal edge of the target vehicle 10 by way of example.

[0064] The first multidimensional normal distribution 110 spans a standard deviation ellipse 111 such that twice the standard deviation extends over half the length of the target vehicle 10, as will be explained in more detail in connection with Fig. 3 will be explained.

[0065] The second multidimensional normal distribution 120 runs correspondingly along a second edge 12 of the target vehicle 10, which runs perpendicular to the first edge 11 and is here referred to by way of example as the rear transverse edge of the target vehicle 10.

[0066] The second multidimensional normal distribution 120 also spans a standard deviation ellipse 121, corresponding to the first multidimensional normal distribution 110, so that the twice standard deviation extends over half the width of the target vehicle 10.

[0067] The third multidimensional normal distribution 130 runs along the entire target vehicle 10 or covers it completely.

[0068] The third multidimensional normal distribution 130 spans a standard deviation ellipse 131 such that the twice standard deviation extends over half the width and half the length of the target vehicle 10, as will be explained in more detail in connection with Fig. 4 will be explained.

[0069] In particular, only at most two edges of the target vehicle are considered, or a point cloud is generated for them, specifically a transverse edge and a longitudinal edge, each facing the radar sensor 20 or being closest to it. It is understood that in other random arrangements, a different combination of two edges, or in the special case where the target vehicle 10 is positioned exactly perpendicular or centrally in front of the radar sensor 20, even a single edge, can be considered, as described in connection with Fig. 2 will be explained.

[0070] Subsequently, the generated point cloud 100 is captured and evaluated using a vehicle recognition algorithm, the output of the vehicle recognition algorithm is compared with the target vehicle 10, and the vehicle recognition algorithm is verified or falsified based on the comparison.

[0071] The point density of the point cloud 100 can be varied, in particular depending on the distance and / or speed of the target vehicle 10.

[0072] Fig. Figure 2 shows another schematic view of an embodiment of a method for verifying a vehicle recognition algorithm.

[0073] In particular, the special case is described in which the target vehicle 10 is arranged exactly perpendicular to the radar sensor 20, especially such that a center line of the target vehicle 10 coincides with a y-axis of the coordinate system, i.e. the target vehicle is located exactly in the middle in front of the radar sensor 20 in the field of view 21.

[0074] This means that only the rear transverse edge 12 is visible, and only the second multidimensional normal distribution 120 and the third multidimensional normal distribution 130 are considered, but not the first multidimensional normal distribution, as in Fig. 1 shown.

[0075] Fig. Figure 3 shows another schematic view of an embodiment of a method for verifying a vehicle recognition algorithm.

[0076] In particular, it is shown how the standard deviation ellipse 131 of the first two-dimensional normal distribution 110 behaves in its dimension to the first edge 11 and to the second edge 12.

[0077] The first two-dimensional point distribution 110 is arranged such that a midpoint M of the standard deviation ellipse 111 coincides with the midpoint of the first edge 11. The dimensions are chosen such that the second standard deviation 2SY1 extends in the y-direction over half the length of the first edge 11, and the third standard deviation 3SY1 extends in the y-direction over one and a half times the half-length, or half the length, of the first edge 11. Similarly, the third standard deviation 3SX1 extends in the x-direction over half the width of the second edge 12.

[0078] The same applies correspondingly to the second standard deviation ellipse, which is not shown in detail.

[0079] Fig. Figure 4 shows another schematic view of an embodiment of a method for verifying a vehicle recognition algorithm.

[0080] In particular, it is shown how the standard deviation ellipse 131 of the third two-dimensional normal distribution 130 relates in its dimension to the first edge 11 and to the second edge 12.

[0081] The third two-dimensional point distribution 110 is arranged such that a center point M' of the standard deviation ellipse 131 coincides with the center point of the target vehicle 20 and thus with both the first edge 11 and the second edge 12. The dimensions are chosen such that the second standard deviation 2SY3 extends in the y-direction over half the length of the first edge 11, and the second standard deviation 2SX3 extends in the x-direction over half the length of the second edge 12. Likewise, the third standard deviation 3SY3 extends in the y-direction over one and a half times half the length of the first edge 11, and the third standard deviation 3SX3 extends in the x-direction over one and a half times half the width of the second edge 12.

[0082] Overall, the examples demonstrate how a stochastic approach can be provided to generate realistic radar point clouds for motor vehicles.

Claims

[1] Method for verifying a vehicle recognition algorithm, comprising: - Generating a point cloud (100) representing a target vehicle (10), wherein the point cloud (100) comprises a normal distribution (110, 120, 130), wherein the point cloud (100) comprises a first multidimensional normal distribution (110) running along a first edge (11) of the target vehicle (10); - Capturing the point cloud (100) with a vehicle detection algorithm; - Comparing the output of the vehicle recognition algorithm with data based on the target vehicle (10); and - Verifying or falsifying the vehicle recognition algorithm based on comparison. [2] Method according to claim 1, wherein the first multidimensional normal distribution (110) spans a standard deviation ellipse (111) such that the twofold standard deviation (2SY1) extends over half the length of the target vehicle (10). [3] Method according to claim 1 or 2, wherein the point cloud (100) comprises a second multidimensional normal distribution (120) extending along a second edge (12) of the target vehicle (10), the second edge (12) being perpendicular to the first edge (11). [4] Method according to claim 3, wherein the second multidimensional normal distribution (120) spans a standard deviation ellipse (121) such that the twice standard deviation extends over half the width of the target vehicle (10). [5] Method according to claims 1 to 4, wherein the point cloud (100) comprises a third multidimensional normal distribution (130) extending along the entire target vehicle (10). [6] Method according to claim 5, wherein the third multidimensional normal distribution (130) spans a standard deviation ellipse (131) such that the two standard deviations (2SX3, 2SY3) extend over half the width and half the length of the target vehicle (10). [7] Method according to one of the preceding claims, wherein the point density of the point cloud (100) varies depending on the distance and / or speed of the target vehicle (10). [8] Device for verifying a vehicle recognition algorithm, wherein the device is configured to perform a method according to one of the preceding claims.

Images