Method and device for testing a vehicle recognition algorithm

Abstract

The invention relates to a method for testing a vehicle detection algorithm, comprising generating a point cloud (100) representing a target vehicle (10), wherein the point cloud (100) comprises a normal distribution (110, 120, 130), detecting the point cloud (100) with a vehicle detection algorithm, comparing the output of the vehicle detection algorithm with data based on the target vehicle (10), and verifying or falsifying the vehicle detection algorithm based on the comparison.

Description

[0001] The present invention relates to a method and a device for checking a vehicle recognition algorithm.

[0002] Testing the functionality of vehicle detection algorithms is often time-consuming.

[0003] It is an object of the present invention to provide an improved method and apparatus for checking a vehicle recognition algorithm.

[0004] This problem is solved by the subject matter of the independent claims. Advantageous further developments are specified in the dependent claims.

[0005] According to a first aspect, a method for checking a vehicle recognition algorithm is provided.

[0006] The method according to the invention comprises the steps of generating a point cloud representing a target vehicle, wherein the point cloud comprises a normal distribution, detecting the point cloud with a vehicle detection algorithm, comparing the output of the vehicle detection algorithm with data based on the target vehicle, and verifying or falsifying the vehicle detection algorithm based on the comparison.

[0007] The method serves, in particular, to verify the functionality and, in particular, the accuracy of one or more vehicle detection algorithms. In particular, one or more vehicle detection algorithms can be classified and, in particular, their results or outputs can be verified or falsified. In particular, the method serves to estimate a position, length, width, and / or orientation of a target vehicle, or to verify or falsify such an estimate.

[0008] Vehicle detection algorithms are, in particular, algorithms used in motor vehicles, which are point clouds generated from sensor data, in particular from one or more radar sensors, when scanning the surroundings of the motor vehicle. These radar sensors are used to detect one or more objects in the surroundings of the motor vehicle, in particular other vehicles, such as passenger cars, which are also referred to herein as target vehicles, and to control the motor vehicle based on this data.

[0009] For example, one or more driving assistance programs, such as a collision warning assistant, adaptive cruise control, a traffic jam assistant and / or a partially or fully autonomous driving mode, use the sensor data and in particular the vehicle detection algorithm(s) as an input source for their control.

[0010] Since testing such vehicle detection algorithms in real-world operation in a motor vehicle is complex and, in particular, requires particularly high-resolution sensors, the method according to the invention provides a solution that quickly and easily generates point clouds of a target vehicle that are as realistic as possible. These vehicle detection algorithms can thus be tested and their functionality and, in particular, their accuracy verified or even falsified. The point cloud or clouds generated using the method can also be referred to as ground truth, i.e., the ground reality, the actual situation against which the vehicle detection algorithms are then measured.

[0011] According to a first step of the method, a point cloud is generated that represents a target vehicle. It is understood that multiple point clouds can also be generated, in particular those that relate to or represent multiple target vehicles and / or a movement of one or more such target vehicles. For the sake of simplicity, reference will primarily be made below to a point cloud that represents a target vehicle at a given point in time or location relative to the motor vehicle.

[0012] The point cloud comprises a normal distribution. A normal distribution can also be referred to as a Gaussian distribution, Gaussian function, Gaussian normal distribution, Gaussian distribution curve, Gaussian curve, Gaussian bell curve, Gaussian bell function, Gaussian bell, or bell curve. This normal distribution represents a probability distribution, in this case of points that concern or characterize the target vehicle. In particular, several normal distributions can be superimposed or combined into a point cloud, as will be described later.

[0013] In a further step, this generated point cloud is then captured by a vehicle detection algorithm. This capture can also be referred to as analysis or sampling, whereby the point cloud replaces the actual captured sensor data, for example, from a radar sensor, and is processed by the vehicle detection algorithm.

[0014] In a subsequent step, the result—i.e., the output of the vehicle detection algorithm—is compared with data based on the target vehicle. The data based on the target vehicle can also be a point cloud, particularly one that actually represents or corresponds to the vehicle. In particular, the data can be so-called ground truth, i.e., data that corresponds to reality, and which was previously provided.

[0015] It is thus checked whether and to what extent the vehicle recognition algorithm has recognized the point cloud or, in particular, the normal distribution it contains, which represents the motor vehicle, and to what extent this corresponds to actual reality.

[0016] The vehicle detection algorithm is then verified or falsified based on the comparison. In particular, the vehicle detection algorithm is quantified and / or qualified, or its quality is determined. For example, the detection of the target vehicle may be above a threshold, whereupon the vehicle detection algorithm is verified. Alternatively, the detection of the target vehicle may be below a threshold, whereupon the vehicle detection algorithm is falsified. In addition, further data may be provided that include a quality, a qualification, and / or quantification of the vehicle detection algorithm. The result and / or the further data can, in particular, be output to a user interface, such as a monitor, and / or stored as data for further processing.

[0017] The method according to the invention makes it possible to check the functionality and, in particular, the accuracy of one or more vehicle detection algorithms. In particular, the solution according to the invention can classify one or more vehicle detection algorithms and, in particular, verify or falsify their results or output.

[0018] According to a further development, the point cloud includes a multidimensional normal distribution.

[0019] The normal distribution encompassed by the point cloud thus extends in multiple dimensions, in particular at least two, and more particularly exactly two dimensions, and can also be referred to as a multivariate, in particular bivariate, normal distribution. In particular, the at least two dimensions are perpendicular to one another, with one dimension, in particular, coinciding with an edge of the target vehicle.

[0020] This training makes it possible to test vehicle recognition algorithms particularly precisely.

[0021] The first, second, and third normal distributions described below are, in particular, arbitrarily assigned ordinal numbers. It is understood that these can also exist alone or be intended solely and, in particular, are not necessarily based on one another and / or are not interdependent. In particular, the third normal distribution and / or a combination of the second and third normal distributions can be used exclusively, without taking the first normal distribution into account.

[0022] According to a further development, the point cloud comprises a first multidimensional normal distribution which runs along a first edge of the target vehicle.

[0023] The first multidimensional normal distribution is designed, in particular, as a two-dimensional or bivariate normal distribution. This extends along or across a first edge of the target vehicle. This can refer to either a longitudinal edge or a transverse edge of the target vehicle. For simplicity, it is assumed below that the first edge of the target vehicle, along which the first point cloud runs or which it covers, is a longitudinal edge of the target vehicle.

[0024] The edge, which can also be referred to as a side, is essentially a straight line that is ideally assumed to be the boundary of the vehicle. In particular, the first edge and the second edge described below define a so-called bounding box of the target vehicle, i.e., a rectangle within which the target vehicle or all of its components are located.

[0025] This training also makes it possible to test vehicle recognition algorithms particularly precisely.

[0026] According to a further development, the first multidimensional normal distribution spans a standard deviation ellipse such that twice the standard deviation extends over half the length of the target vehicle.

[0027] According to this further development, the first multidimensional normal distribution is thus selected or dimensioned in such a way that the two-dimensional standard deviation, which is typically given in the form of an ellipse, is as long as half the length of the target vehicle in the second standard deviation.

[0028] In particular, the major axis of the ellipse coincides with the longitudinal edge of the target vehicle and / or a center point of the ellipse coincides with the center point of the longitudinal edge of the target vehicle and / or extends along it. In particular, the standard deviation ellipse of three times the standard deviation of the first normal distribution extends along one and a half times half the length of the longitudinal edge of the target vehicle and coincides therewith.

[0029] This training also makes it possible to test vehicle recognition algorithms particularly precisely.

[0030] According to a further development, the point cloud comprises a second multidimensional normal distribution which runs along a second edge of the target vehicle, wherein the second edge runs perpendicular to the first edge.

[0031] The second multidimensional normal distribution is designed, in particular, as a two-dimensional or bivariate normal distribution. This extends along or across a second edge of the target vehicle. This can refer to either a longitudinal edge or a transverse edge of the target vehicle. For the sake of simplicity, it is assumed below that the second edge of the target vehicle, along which the second point cloud runs or which it covers, is a transverse edge of the target vehicle that is perpendicular to the longitudinal edge along which the first point cloud runs.

[0032] This second edge, the transverse edge, together with the first edge, the longitudinal edge, forms the so-called bounding box of the target vehicle.

[0033] This training also makes it possible to test vehicle recognition algorithms particularly precisely.

[0034] According to a further development, the second multidimensional normal distribution spans a standard deviation ellipse such that twice the standard deviation extends over half the width of the target vehicle.

[0035] According to this further development, the second multidimensional normal distribution is thus selected or dimensioned in such a way that the two-dimensional standard deviation, which is also typically given in the form of an ellipse, is as long as half the width of the target vehicle in the second standard deviation.

[0036] In particular, the major axis of the ellipse coincides with the transverse edge of the target vehicle and / or a center point of the ellipse coincides with the center point of the transverse edge of the target vehicle and / or extends along it. In particular, the standard deviation ellipse of three times the standard deviation of the second normal distribution extends along one and a half times half the width of the transverse edge of the target vehicle and coincides therewith.

[0037] This training also makes it possible to test vehicle recognition algorithms particularly precisely.

[0038] According to a further development, the point cloud includes a third multidimensional normal distribution that runs along the entire target vehicle.

[0039] The third point cloud covers or envelops the entire target vehicle or the associated bounding box.

[0040] This refinement also makes it possible to test vehicle detection algorithms with particular precision. In particular, the third point cloud serves to account for reflections that form, particularly beneath the target vehicle.

[0041] According to a further development, the third multidimensional normal distribution spans a standard deviation ellipse such that twice the standard deviation extends over half the width and half the length of the target vehicle.

[0042] In particular, a center point of the ellipse of the third multidimensional normal distribution coincides with a center point of the target vehicle or the bounding box. Likewise, in particular, the major axis of the third ellipse runs parallel to one or both longitudinal edges, and the minor axis runs parallel to one or both transverse edges of the target vehicle or the bounding box.

[0043] In particular, the standard deviation ellipse of three times the standard deviation of the third normal distribution extends along one and a half times half the length of the longitudinal edge and along one and a half times half the width of the transverse edge of the target vehicle and coincides therewith.

[0044] This training also makes it possible to test vehicle recognition algorithms particularly precisely.

[0045] According to a further development, the point density of the point cloud is varied depending on the distance and / or speed of the target vehicle.

[0046] In particular, at a greater distance the point density is reduced compared to a smaller distance and / or at a greater speed the point density is increased compared to a lower speed.

[0047] This training also makes it possible to test vehicle recognition algorithms particularly precisely.

[0048] For use cases or application situations that may arise during the method and which are not explicitly described here, it may be provided that, in accordance with the method, an error message and / or a request to enter user feedback is issued and / or a default setting and / or a predetermined initial state is set.

[0049] According to a further aspect, a device for checking a vehicle recognition algorithm is provided, wherein the device is designed to carry out a method according to one of the previously described embodiments.

[0050] For this purpose, the device may comprise a data processing device or a processor device which is configured to carry out an embodiment of the method according to the invention.

[0051] For this purpose, the processor device can have at least one microprocessor and / or at least one microcontroller and / or at least one FPGA (Field Programmable Gate Array) and / or at least one DSP (Digital Signal Processor). In particular, a CPU (Central Processing Unit), a GPU (Graphical Processing Unit) or an NPU (Neural Processing Unit) can be used as the microprocessor. Furthermore, the processor device can have program code which, when executed by the processor device, is configured to carry out the embodiment of the method according to the invention. The program code can be stored in a data memory of the processor device. The processor device can, for example, be based on at least one circuit board and / or on at least one SoC (System on Chip).

[0052] As a further solution, the invention also encompasses a computer-readable storage medium comprising program code which, when executed by a computer or computer network, causes the computer to carry out an embodiment of the method according to the invention. The storage medium can be provided at least partially as a non-volatile data memory (e.g. as a flash memory and / or as an SSD - solid state drive) and / or at least partially as a volatile data memory (e.g. as a RAM - random access memory). The storage medium can be arranged in the computer or computer network. However, the storage medium can also be operated on the Internet, for example, as a so-called app store server and / or cloud server. The computer or computer network can provide a processor circuit with, for example, at least one microprocessor.The program code may be provided as binary code and / or as assembly code and / or as source code of a programming language (e.g. C) and / or as a program script (e.g. Python).

[0053] The invention also encompasses combinations of the features of the described embodiments. The invention therefore also encompasses implementations that each have a combination of the features of several of the described embodiments, unless the embodiments are described as mutually exclusive.

[0054] Exemplary embodiments of the invention are described below. Shown are: Fig. 1 is a schematic view of an embodiment of a method for verifying a vehicle detection algorithm; Fig. 2 shows a further schematic view of an embodiment of a method for checking a vehicle recognition algorithm; Fig. 3 shows a further schematic view of an embodiment of a method for testing a vehicle recognition algorithm; and Fig. 4 shows a further schematic view of an embodiment of a method for testing a vehicle detection algorithm.

[0055] The exemplary embodiments explained below are preferred embodiments of the invention. In the exemplary embodiments, the described components of the embodiments each represent individual features of the invention that can be considered independently of one another, each of which also develops the invention independently of one another. Therefore, the disclosure is intended to encompass combinations of the features of the embodiments other than those shown. Furthermore, the described embodiments can also be supplemented by further features of the invention already described.

[0056] In the figures, the same reference symbols denote elements with the same function.

[0057] Fig. 1 shows a schematic view of an embodiment of a method for testing a vehicle detection algorithm.

[0058] A target vehicle 10 is shown as an example, idealized by a rectangular bounding box. The target vehicle is located in a field of view 21 of a radar sensor 20, which is arranged at the origin of the coordinate system. The target vehicle extends in the field of view 21, for example, slightly inclined along the x- and y-directions of the coordinate system. This is, in particular, a random positioning of the target vehicle 10, the detection of which is to be checked, verified, or falsified by a vehicle detection algorithm.

[0059] The method described below is carried out by a device not shown in detail.

[0060] First, a point cloud 100 is created, which represents the target vehicle 10.

[0061] The point cloud 100 comprises a total of three normal distributions 110, 120, 130, each of which is two-dimensional.

[0062] The first multidimensional normal distribution 110 runs along a first edge 11 of the target vehicle 10, which is referred to here as the left longitudinal edge of the target vehicle 10 by way of example.

[0063] The first multidimensional normal distribution 110 spans a standard deviation ellipse 111, so that twice the standard deviation extends over half the length of the target vehicle 10, as will be explained in more detail in connection with Fig. 3 will be explained.

[0064] The second multidimensional normal distribution 120 runs correspondingly along a second edge 12 of the target vehicle 10, which runs perpendicular to the first edge 11 and is referred to here as the rear transverse edge of the target vehicle 10 by way of example.

[0065] The second multidimensional normal distribution 120 also spans a standard deviation ellipse 121 corresponding to the first multidimensional normal distribution 110, so that twice the standard deviation extends over half the width of the target vehicle 10.

[0066] The third multidimensional normal distribution 130 runs along the entire target vehicle 10 or covers it completely.

[0067] The third multidimensional normal distribution 130 spans a standard deviation ellipse 131, so that twice the standard deviation extends over half the width and half the length of the target vehicle 10, as will be explained in more detail in connection with Fig. 4 will be explained.

[0068] In particular, only a maximum of two edges of the target vehicle are considered or a point cloud is generated for this purpose, in particular a transverse edge and a longitudinal edge, each of which faces or is closest to the radar sensor 20. It is understood that with other random arrangements, a different combination of two edges or, in the special case that the target vehicle 10 is arranged exactly vertically or centrally in front of the radar sensor 20, only a single edge can be considered or taken into account, as is the case in connection with Fig. 2 will be explained.

[0069] Subsequently, the generated point cloud 100 is captured and evaluated with a vehicle detection algorithm, the output of the vehicle detection algorithm is compared with the target vehicle 10, and the vehicle detection algorithm is verified or falsified based on the comparison.

[0070] The point density of the point cloud 100 can be varied in particular depending on the distance and / or the speed of the target vehicle 10.

[0071] Fig. 2 shows a further schematic view of an embodiment of a method for testing a vehicle detection algorithm.

[0072] In particular, the special case is described in which the target vehicle 10 is arranged exactly vertically in front of the radar sensor 20, in particular such that a center line of the target vehicle 10 coincides with a y-axis of the coordinate system, i.e. the target vehicle is located exactly centrally in front of the radar sensor 20 in the field of view 21.

[0073] As a result, only the rear transverse edge 12 is visible and only the second multidimensional normal distribution 120 and the third multidimensional normal distribution 130 are taken into account, but not the first multidimensional normal distribution, as in Fig. 1 shown.

[0074] Fig. 3 shows a further schematic view of an embodiment of a method for testing a vehicle detection algorithm.

[0075] In particular, it is shown how the standard deviation ellipse 131 of the first two-dimensional normal distribution 110 behaves in its dimension to the first edge 11 and the second edge 12.

[0076] The first two-dimensional point distribution 110 is arranged such that a center point M of the standard deviation ellipse 111 coincides with the center point of the first edge 11. The dimensions are selected such that the second standard deviation 2SY1 extends in the y-direction over half the length of the first edge 11, and the third standard deviation 3SY1 extends in the y-direction over one and a half times half the length, or half the length, of the first edge 11. Likewise, the third standard deviation 3SX1 extends in the x-direction over half the width of the second edge 12.

[0077] The same applies correspondingly to the second standard deviation ellipse, which is not shown in detail.

[0078] Fig. 4 shows a further schematic view of an embodiment of a method for testing a vehicle detection algorithm.

[0079] In particular, it is shown how the standard deviation ellipse 131 of the third two-dimensional normal distribution 130 behaves in its dimension to the first edge 11 and the second edge 12.

[0080] The third two-dimensional point distribution 110 is arranged such that a center point M' of the standard deviation ellipse 131 coincides with the center point of the target vehicle 20 and thus with both the first edge 11 and the second edge 12. The dimensions are selected such that the second standard deviation 2SY3 extends in the y-direction over half the length of the first edge 11, and the second standard deviation 2SX3 extends in the x-direction over half the length of the second edge 12. Likewise, the third standard deviation 3SY3 extends in the y-direction over one and a half times the half-length of the first edge 11, and the third standard deviation 3SX3 extends in the x-direction over one and a half times the half-width of the second edge 12.

[0081] Overall, the examples demonstrate how a stochastic approach can be provided to generate realistic radar point clouds for motor vehicles.

Claims

[1] A method for verifying a vehicle detection algorithm, comprising: - generating a point cloud (100) representing a target vehicle (10), wherein the point cloud (100) comprises a normal distribution (110, 120, 130); - capturing the point cloud (100) with a vehicle detection algorithm; - comparing the output of the vehicle detection algorithm with data based on the target vehicle (10); and - Verify or falsify the vehicle detection algorithm based on the comparison. [2] The method of claim 1, wherein the point cloud (100) comprises a multidimensional normal distribution (110, 120, 130). [3] The method of claim 2, wherein the point cloud (100) comprises a first multidimensional normal distribution (110) extending along a first edge (11) of the target vehicle (10). [4] Method according to claim 3, wherein the first multidimensional normal distribution (110) spans a standard deviation ellipse (111) such that twice the standard deviation (2SY1) extends over half the length of the target vehicle (10). [5] Method according to claim 2 to 4, wherein the point cloud (100) comprises a second multidimensional normal distribution (120) which runs along a second edge (12) of the target vehicle (10), the second edge (12) being perpendicular to the first edge (11). [6] Method according to claim 5, wherein the second multidimensional normal distribution (120) spans a standard deviation ellipse (121) such that twice the standard deviation extends over half the width of the target vehicle (10). [7] Method according to claim 2 to 6, wherein the point cloud (100) comprises a third multidimensional normal distribution (130) which runs along the entire target vehicle (10). [8] Method according to claim 7, wherein the third multidimensional normal distribution (130) spans a standard deviation ellipse (131) such that twice the standard deviation (2SX3, 2SY3) extends over half the width and half the length of the target vehicle (10). [9] Method according to one of the preceding claims, wherein the point density of the point cloud (100) varies depending on the distance and / or the speed of the target vehicle (10). [10] Device for checking a vehicle recognition algorithm, wherein the device is designed to carry out a method according to one of the preceding claims.

Images