Method for checking an artificial neural network for the purpose of analyzing echoes, detected by means of a sensor, for a motor vehicle

Abstract

The invention relates to a method for checking an artificial neural network (9) for the purpose of analyzing echoes (8), detected by means of a sensor (2), for a motor vehicle (1). The method comprises: emitting (S1) at least two pulses (7) which differ from one another by at least one feature, by means of the sensor (2); detecting (S2) an echo (8) for each emitted pulse (7) by means of the sensor (2); and, for each of the detected echoes (8), determining (S3) an item of result information (11) by applying the artificial neural network (9) to the detected echo (8), wherein the item of result information (11) describes a result of a specified problem (10) which the artificial neural network (9) has been trained to solve. Subsequently checking (S4) whether the determined result information (11) describe the same result and, only if this is the case, treating (S5) an analysis carried out by the artificial neural network (9) as reliable.

Description

[0001] Method for testing an artificial neural network for evaluating echoes detected by a sensor for a motor vehicle

[0002] The invention relates to a method for testing an artificial neural network for evaluating echoes detected by a sensor for a motor vehicle. The invention also relates to a motor vehicle having a sensor, a control device for a motor vehicle, and a computer program product for implementing such a method.

[0003] A motor vehicle can have at least one sensor whose measuring principle is based on the sensor emitting at least one pulse or impulse into the sensor's surroundings and then detecting an echo that is created, for example, when the pulse is reflected off an object in the surroundings. An ultrasonic sensor, a radar device, and / or a LIDAR device, for example, are based on such a pulse-emitting measuring principle. The echo detected and provided by the sensor can be evaluated, for example to classify the object in the surroundings. The evaluation can be based on machine learning methods, in particular on an artificial neural network. The result of the evaluation can then be provided, for example, to a function of the motor vehicle, such as a driver assistance system for at least partially, in particular fully automatically, controlling the motor vehicle.

[0004] DE 10 2019 218 349 A1 discloses a method for classifying at least one ultrasonic echo from echo signals. A classified echo image is generated based on received echo signals using a trained neural encoder-decoder network and an input matrix formed from a plurality of detected echo signals.

[0005] For the evaluation of at least one detected echo using the artificial neural network, it is currently not possible to reliably determine whether the classification performed by the artificial neural network is actually correct or whether it produced an erroneous result due to noise, for example. The artificial neural network should therefore be tested to reliably rule out negative influences, such as noise, on the results. The object of the invention is to provide a solution by means of which an echo evaluation performed using an artificial neural network can be reliably tested.

[0006] The problem is solved by the subject matter of the independent patent claims.

[0007] A first aspect of the invention relates to a method for testing an artificial neural network for evaluating echoes detected by a sensor for a motor vehicle. The sensor is, for example, an ultrasonic sensor. Alternatively or additionally, the sensor can be a radar device and / or a LIDAR device. The artificial neural network can be any neural network that has been trained at least to classify the data provided to it in connection with a motor vehicle-related task. The data provided to it describes at least one echo detected by the sensor, which was received by the sensor from an environment of the sensor and thus detected. The environment of the sensor preferably corresponds to at least part of an environment of the motor vehicle in which the sensor is arranged.Known methods for training and designing the structure of the artificial neural network can be used.

[0008] The method comprises emitting at least two pulses, which differ from one another in at least one characteristic, by means of the sensor. In the case of the ultrasonic sensor as the sensor, for example, at least two ultrasonic pulses are emitted by the ultrasonic sensor. Preferably, numerous pulses are emitted one after the other, i.e. more than two pulses. Alternatively or additionally, one pulse can be emitted by the sensor. In the sense of the invention, a pulse can therefore alternatively or additionally be understood as a pulse. The pulse is generated by the sensor. The emitted pulses can differ from one another, for example, with regard to a frequency as a characteristic. It is therefore possible, for example, for a first pulse to be emitted at 50 kilohertz and then a second pulse at, for example, 52 kilohertz. These frequencies mentioned are to be understood purely as examples.Different changes in the feature or even several features of the at least two pulses are possible.

[0009] In a further process step, the sensor detects a respective echo for each emitted pulse. For example, if the respective pulse is reflected by an object in the sensor's vicinity, a corresponding echo signal is sent back, referred to here as the detected echo. If exactly two pulses are emitted, for example, two echoes are detected: one for the first pulse and one for the second pulse. The more pulses are emitted, the more respective echoes are detected.

[0010] For each of the detected echoes, a result information is determined. This is done by applying the artificial neural network to the detected echo. If exactly two echoes are detected, both echoes are evaluated using the artificial neural network. The determined result information describes the outcome of a predefined task that the artificial neural network was trained to solve. The task could, for example, be to classify the object. In this example, the task of the artificial neural network is to determine which class the object in the environment belongs to. The network then determines one of the classes it knows as the class to which the object belongs.It is assumed that for the multiple pulses that differ from one another in at least one characteristic, the same result and thus consistent result information are determined when their echoes are evaluated by the artificial neural network. The result information is considered consistent if it describes the same result, at least in terms of content. It is therefore expected that, if the artificial neural network can reliably evaluate the echoes, all the determined results will be consistent with each other in terms of content.

[0011] The method also includes checking whether the determined result information describes the same result. Only if this is the case, i.e., only if the determined result information describes the same result, is an evaluation of the echo detected by the sensor using the artificial neural network considered reliable. Therefore, to check the reliability of the evaluation using the artificial neural network, it is ultimately proposed to artificially vary the emitted pulses in order to be able to compare the results of the respective echoes. Since noise occurs statistically, but the detected echo is always a function of the emitted pulse, noise can be easily identified as such.If the determined result information agrees with each other despite variations in the emitted pulses, this can be considered proof that the results of the artificial neural network are reliable, regardless of, for example, ambient noise and / or another noise source. It is assumed that the ambient noise or the other noise source can be located in a specific frequency range, for example, so that by varying the characteristics of the pulse when evaluating the echoes, it can be determined whether a received echo is primarily or exclusively due to noise or whether it contains information about the sensor's environment. Consequently, an evaluation of the respective echo performed using the artificial neural network is reliably verified.

[0012] An advantageous embodiment provides that the determined result information is provided to at least one function of the motor vehicle only if the evaluation by the artificial neural network is assessed as reliable. Preferably, it is provided as a function to a driver assistance system. For example, taking the result information into account, a driver assistance system of the motor vehicle designed for at least partially automatic, in particular fully automatic, driving, which can, for example, control at least a longitudinal and / or lateral guidance of the motor vehicle, can then be operated. The function can, for example, rely on the classification provided by the artificial neural network and regard this as reliably correct information that it can, for example, use for its application.This ensures that only truly reliable evaluations can influence other functions of the vehicle.

[0013] An additional embodiment provides that the at least one feature by which the emitted pulses differ from one another is one of the following features: a pulse duration, a frequency, a shape, and / or a code imprinted on the pulse. The pulse duration can be varied, for example, such that the pulse duration of the first pulse, which is, for example, 50 microseconds, is doubled, so that the second pulse has, for example, a pulse duration of 100 microseconds. For further second pulses, a doubling of the pulse duration or the same pulse duration interval as between the first two pulses can be selected, i.e., in this example, a third pulse could have a pulse duration of 200 microseconds or 150 microseconds. If the frequency is varied, a frequency change can be specified, for example, in steps of one to two kilohertz between the individual pulses.In terms of shape, for example, the width of the pulse can be changed, in particular deformed. Furthermore, predetermined information in the form of a code can be imprinted on the pulse. Alternatively or additionally, a sweep can be varied, i.e. a sampling rate which, for example, specifies frequency and / or pulse duration differences in which pulses varied over a frequency range or pulse duration range are emitted. For example, the sweep can specify that a certain number of further pulses are emitted, with an individual deviation from an initial pulse being specified for each of the further pulses. The deviations between any two of the further pulses are preferably always the same. Changes to the features other than those mentioned are possible. Known methods for varying the features of pulses can be used.Ultimately, various features can be used to characterize a pulse emitted by a sensor. What's relevant here is that the two emitted pulses differ from each other in at least some way. Influencing these features is therefore particularly versatile.

[0014] A preferred embodiment provides that an F-measure is determined for the respective result information and a check is carried out to determine whether the determined F-measure is less than a minimum value. If this is the case, at least one further pulse is transmitted which differs in at least one feature from the at least two pulses transmitted so far. The result information for the echo detected for the further pulse is also determined and checked. During the check, a check is carried out to determine whether the result information determined for the further detected echo describes the same result as the result information determined for the echoes previously detected after the at least two pulses were transmitted. The F-measure is in particular an F1 measure. The F-measure provides a possibility for evaluating a measured value.The F-measure, for example, is the harmonic mean of accuracy and hit rate, whereby in the F1 measure, accuracy and hit rate are equally weighted. The F-measure is typically dependent on the signal-to-noise ratio of the echo. The F-measure is therefore a measure of the reliability of the artificial neural network. If a classification, i.e. the result information determined using the artificial neural network, is correct, the F-measure and in particular the F1 measure is exactly 1. However, if the F-measure, in particular the F1 measure, is not equal to 1, for example, less than 1, this indicates that the result information is not completely trustworthy. If it is now determined that the F-measure, in particular the F1 measure, is not sufficiently high, new pulses are emitted and consequently the number of evaluated echoes is increased. The minimum value specified for this can be 0.9, for example.A larger or smaller minimum value is possible. The fact that at least one additional pulse is transmitted is based on the observation that a measurement result, and in particular a signal-to-noise ratio of a measurement, can improve by a factor corresponding to the square root of the number of measurements performed. Increasing the number of measurements reduces the error and thus the noise. For this reason, depending on the F-measure, it can be determined whether additional echoes should be evaluated or whether sufficient echoes have already been evaluated. The F-measure is typically determined by the artificial neural network, so this measure can be determined and evaluated with little effort.

[0015] It can be provided that if the F-measure is smaller than the minimum value, the entire measurement is always repeated. This means that the previously acquired echoes are discarded and the pulses are re-emitted, with the number of pulses increased compared to the previous value. The described method for determining the reliability of the evaluation is then performed with the newly acquired echoes for the newly transmitted pulses. In other words, the testing of the artificial neural network can be repeated during a new measurement with the increased number of transmitted pulses.

[0016] Furthermore, an embodiment may provide for the artificial neural network to be trained to solve multiple tasks. For example, it may have been trained on two, three, or even more different tasks. The result information is determined and verified for each of the multiple tasks. This means that for each task, it is verified whether the respective result information for the multiple echoes all describe the same result or not. If, for at least one of the multiple tasks, it is determined during the verification that the determined result information does not describe the same result, the verification is performed again, with a larger number of pulses being emitted for the renewed verification than before.If the results for at least one of the tasks indicate that the relevant evaluation using the artificial neural network was not reliable enough, the entire process is preferably carried out again, i.e. the test of the artificial neural network is repeated for all tasks, even for those that produced consistent results. During the renewed and thus repeated test, the number of pulses emitted is also increased in order to obtain even more precise results. It is then checked again, for example, whether the evaluation using the artificial neural network is not sufficiently reliable after all and whether, for example, there was only a source of noise beforehand. Before the renewed test, the artificial neural network can be retrained, for example, the artificial neural network whose task could not be solved reliably.Easily implemented measures are therefore provided to re-test an artificial neural network that was previously assessed as unreliable, for example.

[0017] A further embodiment provides that the testing of the artificial neural network is repeated at predetermined time intervals while the sensor is activated. The predetermined time intervals are, for example, 1 minute, 2 minutes, 3 minutes, 5 minutes, or in particular 10 minutes. Larger or smaller time intervals, and in particular time intervals between the aforementioned, are possible. It can therefore be provided that during operation of the motor vehicle, in which echoes are received and evaluated by means of the sensor and in which, for example, the function of the motor vehicle is active based on the provided echoes, a variation of at least two pulses is repeatedly generated, for example once per minute, and evaluated according to the method described above in order to check whether the artificial neural network continues to deliver reliable results.This makes it possible to provide an additional safety check, particularly for the vehicle's function designed for at least partially automated driving.

[0018] Furthermore, according to one exemplary embodiment, it is provided that the predetermined time interval between two checks depends on the current speed of the motor vehicle at which the sensor is arranged. For example, it can be provided that the artificial neural network is checked at shorter intervals when driving on a highway compared to a journey at a lower speed compared to a journey on the highway, for example when driving in an inner-city area or on a country road. It is assumed here that an even greater reliability of the artificial neural network is required when the motor vehicle is traveling at high speed compared to a lower speed, in particular when the motor vehicle is controlled at least partially automatically on the basis of the echoes detected by the sensor.For this reason, the frequency of the check is predetermined depending on the current journey and thus the speed. This also saves resources, as the checks are only performed at short intervals in situations where it is particularly relevant, and at longer intervals otherwise. The speed of the vehicle can be recorded and provided, for example, using a speed sensor on the vehicle.

[0019] In addition, one embodiment provides that the respective result information at least describes whether the detected echo is real or noise. Thus, a first task of the artificial neural network can be to classify whether an echo or noise is present. The artificial neural network can therefore be trained to distinguish between detected data typical of noise and detected data describing an echo of a transmitted pulse. It is assumed that the noise, like the echo, is received by the sensor and thus detected. The artificial neural network can therefore analyze the data provided to it by the sensor to determine whether it is typical of a noise spectrum or an echo.This is a particularly useful task for an artificial neural network, for example to decide whether further evaluation is useful because an echo was actually detected, or not because only noise was detected.

[0020] An additional embodiment provides that the respective result information at least describes whether the detected echo describes an object that is relevant for a given target application or not. A second task of the artificial neural network can therefore be to classify whether the echo, in particular already classified as such, describes an object that is relevant for a given target application or whether the object described by the echo is irrelevant for the given target application. It can therefore be determined whether the object is relevant for further analysis in the motor vehicle or not. It can be assumed here that it has already been determined that the echo is not noise, but an actual echo that describes an object actually located in the environment.The target application could be, for example, the function of the motor vehicle and / or the determination of an object class of the object, so that, for example, another neural network can then determine whether the object is another road user or not. This is a useful task for an artificial neural network, for example, to decide whether or not further analysis of the echo should be performed.

[0021] In addition, one embodiment can provide for the respective result information to at least describe whether the detected echo describes an object that has a height and / or width that is smaller than a respective predefined minimum value. A third task of the artificial neural network can therefore be to classify whether the height and / or width of the object described by the echo is smaller than the minimum value on the one hand, or greater than or equal to the minimum value on the other. Different minimum values ​​can be specified for the height and width. The minimum value for the height and / or width can be 20 centimeters, for example. Larger or smaller minimum values ​​than this example are possible. This task is particularly suitable, for example, for determining whether an object in the vicinity of the sensor is relevant for the function of the motor vehicle, in particular for a driving function.For example, an object smaller than the minimum value can be assumed to be a speed bump, a curb, and / or another traversable object. It may therefore be particularly useful to classify the object based on its height and / or width.

[0022] In addition, one embodiment provides that the respective result information also includes whether the detected echo describes an object that has a height and / or width that is greater than a respective predetermined maximum value. The maximum value is greater than the minimum value. The maximum value can be 60 centimeters, for example. Alternatively, the maximum value can be any value greater or less than 60 centimeters. Different minimum values ​​can be specified for the height and width. The evaluation of the object with regard to the maximum value preferably leads to a further class being provided in the third task. It can therefore now be determined whether the object is smaller than the minimum value, whether it is greater than or equal to the minimum value but less than or equal to the maximum value, or whether it is larger than the maximum value.This serves to accurately assess whether the object could represent an obstacle for the motor vehicle and should therefore be taken into account, for example, by the function.

[0023] Furthermore, an embodiment can be provided which comprises that the respective result information also includes whether the detected echo describes an object that has a height and / or width that lies between the predetermined minimum value and the predetermined maximum value. In the above-mentioned example, it can therefore be checked whether the object is between 20 centimeters and 60 centimeters high and / or wide or whether the height and / or width lies outside this value range. In particular, it is provided that an intermediate value is specified between the minimum value and the maximum value. The intermediate value can be 40 centimeters, for example. The result information can then describe whether the received echo describes an object that has a height and / or width that lies between the minimum value and the intermediate value or whether it lies between the intermediate value and the maximum value.Ultimately, this adds a fourth category to the third task, so that it can be determined whether the height and / or width of the object is less than the minimum value, whether it is greater than or equal to the minimum value but less than or equal to the intermediate value, whether it is greater than the intermediate value but less than or equal to the maximum value, or whether it is greater than the maximum value. A different intermediate value can be specified for the width than for the height. Ultimately, this enables a particularly clear and specific subdivision of the object's height into different classes, which can, for example, be linked to different object classes or types of objects in further evaluation steps. For example, this can make it possible for the ultrasonic sensor as a sensor to describe the sensor's environment based on its measurement data, in which the detected object is evaluated at least according to its spatial extent.

[0024] A further aspect of the invention relates to a motor vehicle with a sensor. The motor vehicle is designed to emit at least two pulses, which differ from one another in at least one feature, by means of the sensor, and to detect a respective echo for the respective emitted pulse, by means of the sensor. Furthermore, it is designed to determine result information for each of the detected echoes by applying an artificial neural network to the detected echo, wherein the result information describes a result of a predetermined task, which the artificial neural network was trained to solve, and to check whether the determined result information describes the same result. If this is the case, the motor vehicle is designed to evaluate an evaluation by means of the artificial neural network as reliable.

[0025] The motor vehicle is, for example, a passenger car, a truck, a bus, a motorcycle and / or a moped. The sensor is preferably an ultrasonic sensor. Alternatively or additionally, the sensor is a radar device and / or LIDAR device. The sensor is preferably arranged in a front region and / or in a rear region of the motor vehicle, for example in a bumper arranged there. Alternatively or additionally, the sensor can be arranged, for example, on a windshield, a rear window, in doors of the motor vehicle and / or in side mirrors of the motor vehicle. Alternatively or additionally, the sensor is arranged in the interior of the motor vehicle. Furthermore, the invention relates to a control device for a motor vehicle. This control device is designed to carry out the method described above.It is assumed here that the control device can provide the sensor with commands based on which the sensor, for example, emits the at least two pulses and provides the echoes detected by it to the control device. The artificial neural network is preferably stored in a memory unit of the control device and can be applied by the latter to the echoes provided by the sensor. The control device carries out the described method, in particular an exemplary embodiment or a combination of exemplary embodiments of the described method. The control device has a processor device. The processor device can have at least one microprocessor and / or at least one microcontroller and / or at least one FPGA (Field Programmable Gate Array) and / or at least one DSP (Digital Signal Processor).Furthermore, the processor device may comprise program code, which may alternatively be referred to as a computer program product. The program code may be stored in a data memory of the processor device.

[0026] The invention also relates to a computer program product. The computer program product is at least designed to perform the check using the artificial neural network. For this purpose, the computer program product preferably receives the detected echoes from the sensor. The computer program product thus relates at least to the steps according to the invention that are not explicitly performed by the sensor.

[0027] The exemplary embodiments described in connection with the method according to the invention, both individually and in combination with one another, apply accordingly, to the motor vehicle according to the invention, the control device according to the invention, and the computer program product according to the invention, as applicable. The invention encompasses combinations of the described exemplary embodiments.

[0028] Showing:

[0029] Fig- 1 is a schematic representation of a motor vehicle with several

[0030] Sensors; and Fig. 2 shows a schematic representation of a signal flow graph of a method for testing an artificial neural network for evaluating echoes detected by a sensor for a motor vehicle.

[0031] Fig. 1 shows a motor vehicle 1 which has a plurality of sensors 2 in a front area and in a rear area. The sensors 2 are arranged in a bumper of the motor vehicle 1. Here, for example, six sensors 2 are sketched in the front area and six sensors 2 in the rear area. The motor vehicle 1 can have more or fewer sensors 2 than the sensors 2 sketched here. The respective sensor 2 is preferably an ultrasonic sensor. Alternatively or additionally, the sensor 2 is a radar device, i.e. a radar sensor, and / or a LIDAR device. Ultimately, the sensor 2 is a pulse-emitting sensor 2 whose measuring principle is based on receiving echoes 8 (see reference numeral 8 in Fig. 2) as, for example, reflections of the emitted pulses 7 (see reference numeral 7 in Fig. 2) and / or impulses.

[0032] The motor vehicle 1 has a control device 3, which is, for example, a central computing device of the motor vehicle 1. The control device 3 is preferably designed to control the respective sensor 2 of the motor vehicle 1, i.e., to provide it with a control command, for example, and then to receive from the sensor 2, for example, the echoes 8 detected by the sensor 2. The control device 3 can further be designed to provide a function 4 for the motor vehicle 1, such as a driver assistance system for at least partially automatic, in particular fully automatic, driving of the motor vehicle 1.

[0033] The motor vehicle 1 can comprise an output device 5, by means of which, for example, a result of a check of an artificial neural network 9 (see reference numeral 9 in Fig. 2) is output. The output device 5 can be, for example, a screen, in particular a touch-sensitive screen, and / or a loudspeaker of the motor vehicle 1. The output device 5 can, for example, indicate whether or not the function 4 can currently be activated.

[0034] Fig. 2 shows steps of a method for checking an artificial neural network 9 for evaluating echoes 8 detected by the sensor 2 for the motor vehicle 1. In a method step S1, at least two pulses 7, which differ from one another in at least one feature, are emitted by the sensor 2. The feature in which the at least two pulses 7 differ from one another is, for example, a pulse duration, a frequency, a shape and / or a code impressed on the pulse 7.

[0035] It is further outlined here that the two pulses 7 are emitted in the direction of an object 6 located in the vicinity of the sensor 2. The vicinity of the sensor 2 here corresponds to at least part of the surroundings of the motor vehicle 1 in which the sensor 2 is arranged.

[0036] In a method step S2, a respective echo 8 is detected for the respective emitted pulse 7. Here, for example, due to the two emitted pulses 7, exactly two echoes 8 are sketched, which move from the object 6 in the direction of the sensor 2 and are detected by the sensor 2.

[0037] In a method step S3, an artificial neural network 9 is applied to each of the detected echoes 8. In this process, result information 11 is determined. The result information 11 describes a result of a predetermined task 10, which the artificial neural network 9 was trained to solve. The artificial neural network 9 specified here can, for example, have three tasks 10, i.e., can have been trained to solve three tasks 10. A first task 10 provides, for example, that the determined result information 11 at least describes whether the detected echo 8 is real or whether it is noise. The second task 10 can, for example, provide that the respective result information 11 describes whether the detected echo 8 describes the object 6 that is relevant for a predetermined target application of the motor vehicle 1 or not.The object 6 can, for example, be relevant for the target application if it is another motor vehicle 1, a pedestrian, a cyclist and / or an obstacle for the motor vehicle 1. A further possible task 10 can provide that the respective result information 11 at least describes whether the received echo 8 describes an object 6 that has a height and / or width that is less than a respective predetermined minimum value, or that is greater than or equal to the minimum value but less than or equal to an intermediate value, or that is greater than the intermediate value but less than or equal to a maximum value, or that is greater than the maximum value. A different minimum value, maximum value and / or intermediate value can be specified for the width than for the height. The minimum value is less than the intermediate value and this is less than the maximum value.The third task 10 can, for example, distinguish four different classes, so that by applying the artificial neural network 9, it is checked which of these four height-dependent and / or width-dependent classes the object 6 belongs to. For the first two tasks 10 mentioned, only two possible classes were distinguished, namely, for example, the class "real echo" and the class "noise" or the class "object relevant for the given target application" and the class "object not relevant for the given target application." The individual tasks 10 can be solved by a common artificial neural network 9, or there can be a separate artificial neural network 9 for each task 10. In this case, the result information 11 can be determined for several artificial neural networks 9 and subsequently compared with each other, for example.It is possible that the artificial neural network 9 has even more tasks 10 than those mentioned as examples.

[0038] In a method step S4, a check is performed to determine whether the determined result information 11 describes the same result. This check determines whether, for example, the two echoes 8 each deliver the result "true echo" for the first task 10 or not. If this is the case, i.e., if all result information 11 describes the same result, the evaluation by the artificial neural network 9 is assessed as reliable in a method step S5. It is thus determined that the artificial neural network 9 is reliably suitable for evaluating and classifying the echoes 8.

[0039] If this was assessed in method step S5, the determined result information 11 can be provided to function 4 of the motor vehicle 1 in a method step S6. Thus, for example, the driver assistance system can then be operated as function 4, taking into account the result information 11 and / or the echoes 8 that were determined by the sensor 2 and, if applicable, evaluated by the artificial neural network 9.

[0040] It can be provided that an F-measure, in particular an F1 measure, is determined for the result information 11. A check is then carried out to determine whether the determined F-measure is less than a minimum value. If this is the case, at least one further pulse 7 is transmitted, which differs in at least one feature from the at least two pulses 7 transmitted so far. The result information 11 is also determined for the echo 8 detected for the further pulse 7 and this is checked. If, for example, a higher F-measure is then achieved for the further pulse 7 by evaluating the corresponding echo 8, this can possibly lead to the evaluation by the artificial neural network 9 being assessed as reliable.

[0041] Furthermore, a method step S7 can be provided which assumes that the artificial neural network has been trained to solve a plurality of tasks 10. The result information 11 for the plurality of tasks 10 is then determined. A check is then carried out to determine whether the determined result information 11 does not describe the same result for at least one of the plurality of tasks 10 upon checking. In this case, the checking is carried out again, with a larger number of pulses 7 being emitted for this purpose than before. In a method step S8, therefore, purely by way of example, five pulses 7 are emitted and consequently five echoes 8 are received, which are then fed to the artificial neural network 9 to solve the respective tasks 10, so that, for example, five pieces of result information 11 are determined for each task 10, which can then be compared with one another with regard to their results.If this larger number of pulses 7 ultimately results in all tasks 10 being reliably solved and thus the artificial neural network 9 being evaluated as reliable for each task 10, the method can be terminated and the evaluation updated in method step 5. Otherwise, it can be provided that, for example, the entire neural network 9 is evaluated as unreliable because it was determined that at least one of the tasks 10 of the artificial neural network 9 could not be reliably solved.

[0042] In general, it is possible for the described check, i.e., at least method steps S1 to S9, to be repeated continuously while, for example, sensor 2 is in an activated state. The check is preferably repeated at predetermined time intervals of, for example, 1 minute, 2 minutes, 3 minutes, 5 minutes, or in particular 10 minutes. The time interval between two checks can depend on the current speed of motor vehicle 1. The higher the speed, the more frequently the check can be performed, for example.

[0043] Overall, the examples demonstrate a quality control approach for pulse-ranging sensors. An additional and novel solution involves working with different echo shapes (8 echoes). The principle proposed here involves performing measurements with different echo shapes. For example, the frequency, duration, shape, sweep, and so on are varied. The noise is statistical, but the received echo (8) is a function of the transmitted pulse (pulse (7). This allows for a clear classification (F1-measure = 1) after multiple measurements. The number of measurements and the variation of the transmitted pulse are performed by a parallel measurement of the signal-to-noise ratio. This proposed solution can be applied to any type of pulse-echo detection sensor.

Claims

Patent claims 1. A method for testing an artificial neural network (9) for evaluating echoes (8) detected by a sensor (2) for a motor vehicle (1), comprising: - emitting (S1) at least two pulses (7) which differ from one another in at least one feature by means of the sensor (2); - detecting (S2) a respective echo (8) for the respective emitted pulse (7) by means of the sensor (2); and - for each of the detected echoes (8), determining (S3) a piece of result information (11) by applying the artificial neural network (9) to the detected echo (8), wherein the result information (11) describes a result of a predetermined task (10) for the solution of which the artificial neural network (9) was trained; characterized in that it is checked (S4) whether the determined result information (11) describes the same result, and only if this is the case, an evaluation by means of the artificial neural network (9) is assessed as reliable (S5).

2. Method according to claim 1, characterized in that only if the evaluation by means of the artificial neural network (9) is assessed as reliable, the determined result information (11) is provided to at least one function (4) of the motor vehicle (1), in particular a driver assistance system of the motor vehicle (1) (S6).

3. Method according to one of the preceding claims, characterized in that the at least one feature in which the emitted pulses (7) differ from one another is one of the following features: - a pulse duration; - a frequency; - a form; and / or - a code imprinted on the pulse (7).

4. Method according to one of the preceding claims, characterized in that an F-measure, in particular an F1-measure, is determined for the respective result information (11) and it is checked whether the determined F-measure is smaller than a minimum value, wherein if this is the case, at least one further pulse (7) is emitted which differs in at least one feature from the at least two pulses (7) emitted so far and the result information (11) is determined and checked for the echo (8) detected for the further pulse (7).

5. Method according to one of the preceding claims, wherein the artificial neural network (9) has been trained to solve a plurality of tasks (10) and the result information (11) for the plurality of tasks (10) is determined and checked, wherein if for at least one of the plurality of tasks (10) the determined result information (11) does not describe the same result during the checking, the checking is carried out again, wherein for the renewed checking a larger number of pulses (7) is emitted than before.

6. Method according to one of the preceding claims, characterized in that the checking of the artificial neural network (9) is repeated at predetermined time intervals during an activated state of the sensor (2).

7. Method according to claim 6, characterized in that the predetermined time interval between two checks is dependent on a current speed of the motor vehicle (1) in which the sensor (2) is arranged.

8. Method according to one of the preceding claims, characterized in that the respective result information (11) at least describes whether the detected echo (8) is real or noise.

9. Method according to one of the preceding claims, characterized in that the respective result information (11) describes at least whether the detected echo (8) describes an object (6) that is relevant for a predetermined target application or not.

10. Method according to one of the preceding claims, characterized in that the respective result information (11) at least describes whether the detected echo (8) describes an object (6) which has a height and / or width which is smaller than a respective predetermined minimum value.

11. Method according to claim 10, characterized in that the respective result information (11) also includes whether the detected echo (8) describes an object (6) which has a height and / or width which is greater than a respective predetermined maximum value, wherein the maximum value is greater than the minimum value.

12. Method according to claim 10 and 11, characterized in that the respective result information (11) also includes whether the detected echo (8) describes an object (6) which has a height and / or width which lies between the minimum value and the maximum value, in particular which lies between the minimum value and an intermediate value which lies between the minimum value and the maximum value, or between the intermediate value and the maximum value.

13. Motor vehicle (1) with a sensor (2), characterized in that the motor vehicle (1) is designed to: - to emit at least two pulses (7) by means of the sensor (2) which differ from one another in at least one feature; - to detect a respective echo (8) for the respective emitted pulse (7) by means of the sensor (2), - to determine result information (11) for each of the detected echoes (8) by applying an artificial neural network (9) to the detected echo (8), wherein the result information (11) describes a result of a predetermined task (10) for the solution of which the artificial neural network (9) was trained, - to check whether the result information obtained (11) describes the same result, and - if this is the case, to consider an evaluation using the artificial neural network (9) as reliable.

14. Control device (3) for a motor vehicle (1), wherein the control device (3) is designed to carry out the steps of a method according to one of claims 1 to 12 provided for a control device (3).

15. Computer program product comprising instructions which, when executing the Program by a control device (3) of a motor vehicle (1) causing it to carry out the steps of a method according to one of claims 1 to 12 provided for a control device (3).

Images