Method for operating a driver assistance system of a vehicle and driver assistance system of a vehicle

By storing interfering data in the database and comparing real-time, combining neural network analysis and supplementing sensor data inspection, the problem of deep neural network interference in driver assistance system is solved, improving the safety and stability of the system.

CN114008683BActive Publication Date: 2024-11-05VOLKSWAGEN AG
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202080043498.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2019-06-14
Filing Date
2020-06-12
Publication Date
2024-11-05
Estimated Expiration
2040-06-12

AI Technical Summary

Technical Problem

Deep neural networks are susceptible to harmful interference in driver assistance systems, resulting in unstable output data and the inability to effectively identify and verify the authenticity of sensor data. Especially under natural changes and human interference, it affects the reliability and safety of the system.

Method used

By storing natural and harmful interference data in the database, comparing sensor data in real time, analyzing proven data using neural networks, generating trusted control data, and examining differences by supplementing sensor data, distinguishing natural changes from harmful interference, improving nerves Network parameters to enhance anti-interference ability.

Benefits of technology

Reliable verification and interference identification of sensor data is achieved, the safety and stability of the driver assistance system is improved, and the reliable operation of the system is ensured under different environmental conditions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114008683B_ABST
    Figure CN114008683B_ABST
Patent Text Reader

Abstract

The invention relates to a method for operating a driver assistance system (20) for a vehicle, wherein sensor data of the vehicle's surroundings are continuously recorded, the recorded sensor data are verified, the verified sensor data are analyzed by means of a neural network (11), and analyzed sensor data are generated therefrom, and control data for partially or fully automatically controlling the vehicle are generated based on the analyzed sensor data. The method according to the invention is characterized in that, when verifying the sensor data, at least first sensor data recorded at a first earlier point in time are compared with second sensor data recorded at a second later point in time, the result of the comparison is compared with a database (8) in which data about disturbances of input data (9) of the neural network (11) are stored, wherein it is checked whether the second sensor data is generated at least partially by disturbances of the first sensor data stored in the database (8). The invention also relates to a driver assistance system (20) using the method.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to a method for operating a driver assistance system of a vehicle, wherein sensor data from the surroundings of the vehicle are continuously recorded. The recorded sensor data are verified. The verified sensor data are then analyzed with the aid of a neural network. This generates analyzed sensor data. Based on the analyzed sensor data, control data for partially or fully automatic control of the vehicle are generated. The invention also relates to a driver assistance system for a vehicle, having a sensor unit arranged in the vehicle for recording sensor data from the surroundings of the vehicle. Background Art

[0002] Modern vehicles include driver assistance systems, which support the driver in controlling the vehicle or partially or completely take over the driving task. By using such driver assistance systems, different degrees of automation of vehicle control can be achieved. In the case of a low degree of automation, only information and warnings are output to the driver. In the case of a high degree of automation, the driver assistance system actively intervenes in the control of the vehicle. For example, there is intervention in steering the vehicle or in positive or negative acceleration. In the case of a higher degree of automation, the intervention in the vehicle's equipment is to the extent that certain types of vehicle movements, such as straight-line driving, can be performed automatically. In the case of the highest degree of automation, the vehicle can be driven automatically.

[0003] In such driver assistance systems, the analysis of digital images recorded from the vehicle's surroundings during driving is of vital importance. Only if the digital images are analyzed correctly can the driver assistance system control the vehicle reliably. Machine learning has great potential in the analysis of digital images for driver assistance systems. Raw sensor data, for example, generated by the vehicle's cameras, radar sensors or lidar sensors, are processed with the aid of deep neural networks. The neural network generates output data from which the driver assistance system derives relevant information about partially or fully automated driving. For example, the type and position of objects in the vehicle's environment and their behavior are determined. In addition, the road geometry and road topology can be determined with the aid of neural networks. In particular, special convolutional networks (English: convolutional neuronalnetworks) are suitable for processing digital images.

[0004] For use in driver assistance systems, such deep neural networks are trained. In this case, the parameters of the neural network can be appropriately adjusted by the input data without the intervention of a human expert. For a given parameterization, the difference between the output of the neural network and the ground truth is measured. This difference is also called the "loss". In this case, the so-called loss function is selected in such a way that the parameters can be related to it in a differentiable way. In the process of gradient descent, the parameters of the neural network are then adjusted in each training step according to the derivative of the difference determined based on more examples. These training steps are repeated as often as necessary until the difference, i.e. the loss, no longer decreases.

[0005] In this approach, parameters are determined without the need for evaluation by human experts or semantically driven modeling. The consequence for neural networks is that they are often largely opaque to humans and their computations cannot be explained. As a result, deep neural networks in particular are often not amenable to systematic testing or formal verification.

[0006] There is also the problem that deep neural networks are susceptible to harmful interference (adversial perturbations). Small tampering of the input data that is barely or not noticeable to a person, or that does not change the assessment of the situation, can lead to output data that is significantly different from the output data that would have been produced without the tampering. Such tampering can be both intentional changes to the sensor data and random changes to the image due to sensor noise, weather effects, or certain colors and contrasts.

[0007] It is not possible to foresee here which input features the neural network will react so sensitively that the output data will also change significantly in the case of a slight change in the input data. This has the consequence that synthetic data cannot be successfully used to train neural networks used in such driver assistance systems. It has been found that neural networks trained in simulations or with other synthetic data have very poor performance when used in driver assistance systems that use real sensor data. It has also been found that the implementation of driver assistance systems with neural networks in other domains also significantly reduces the functional quality. For example, it can happen that a driver assistance system with a neural network trained in the summer may not be suitable for implementation in the winter. Therefore, the development and approval of neural networks for driver assistance systems based on simulations is problematic.

[0008] Therefore, it is necessary to develop robust neural networks for driver assistance systems. The neural network should generate usable output data for the driver assistance system even when the input data are disturbed.

[0009] When neural networks are used in driver assistance systems, it should also be possible to detect whether the recorded sensor data is genuine. It should be possible to reliably verify the sensor data. In particular, it should be possible to detect whether the sensor data has been changed due to harmful interference. Summary of the invention

[0010] It is therefore an object of the present invention to specify a method and a driver assistance system of the type mentioned at the outset, in which the recorded sensor data are reliably verified.

[0011] This object is achieved according to the invention by a method having the features of claim 1 and a driver assistance system having the features of claim 10. Advantageous refinements and improvements are revealed in the dependent claims.

[0012] In the method according to the invention for operating a driver assistance system for a vehicle, sensor data from the surroundings of the vehicle are continuously recorded. The recorded sensor data are verified. When verifying the sensor data, at least first sensor data recorded at a first earlier point in time are compared with second sensor data recorded at a second later point in time. The result of the comparison is compared with data from a database in which data about disturbances of the input data of the neural network are stored, wherein it is checked whether the second sensor data is at least partially generated by disturbances of the first sensor data stored in the database. The verified sensor data are then analyzed with the aid of the neural network. This generates analyzed sensor data. Based on the analyzed sensor data, control data for partially or fully automatically controlling the vehicle are then generated.

[0013] In the method according to the invention, sensor data recorded from the vehicle's surroundings are reliably verified using disturbances previously stored in a database. In this way, it is possible to reliably identify known harmful disturbances for a neural network used in a driver assistance system when the neural network is used in the driver assistance system. In particular, disturbances can be identified in real time. In this way, it is also possible to distinguish temporal changes in the sensor data caused by natural changes in the vehicle's surroundings from harmful disturbances.

[0014] The sensor data are in particular digital images of a camera of the vehicle. The digital images record the surroundings of the vehicle. The sensor data are in particular raw sensor data, in particular raw sensor data of a camera.

[0015] The sensor data may also be data generated by a radar sensor or a lidar sensor. In this case, the sensor data are in particular also sensor raw data.

[0016] Data about disturbances in the input data of the neural network are stored in a database, which is arranged in particular in the vehicle. When comparing sensor data recorded at different points in time with the database, the database can therefore be accessed directly, i.e. without a wireless interface. This ensures that balancing can be carried out in the vehicle at any time, even if the vehicle has no data connection to external devices.

[0017] The data stored in the database describe in particular naturally occurring interferences. These are, for example, weather effects such as fog or snow, sensor noise, camera dirt or changes caused by texture. In addition, naturally occurring interferences are objects that occur naturally in the surroundings of the vehicle, such as printed posters or stickers on objects. Alternatively, other, for example man-made interferences can also be stored in the database, but a classification of the interferences is additionally stored therein, so that when comparing the sensor data recorded at different points in time with the database, it can be checked whether changes have occurred in the sensor data from naturally occurring interferences or from other, for example man-made, in particular harmful interferences.

[0018] If, in the case of harmful interference, for example a pattern or a grid is placed on the sensor data, this can be identified by the verification through a plausibility check, since such a pattern or a grid is not a naturally occurring interference.

[0019] If the balance shows that the change in the sensor data is caused by a naturally occurring disturbance, the sensor data can be verified. If this is not the case, the change in the sensor data is most likely caused by a harmful disturbance. In this case, the sensor data are not verified and are then not used to generate analyzed sensor data by means of a neural network.

[0020] According to an improvement of the method according to the invention, the plausibility of the verified sensor data is checked in the following manner: supplementary sensor data are obtained from the surroundings of the vehicle, the supplementary sensor data are analyzed, the difference between the second sensor data and the first sensor data is determined, and the difference is checked whether it is consistent with the analysis of the supplementary sensor data. The reason for the difference in the sensor data recorded at different times is, on the one hand, the movement of the vehicle relative to the surroundings. However, in addition to the differences related to the movement of the vehicle, differences may also arise from changes in the surroundings. The supplementary sensor data can be used to check whether these other differences are caused by such environmental changes. By the plausibility check, harmful interference can be identified, which itself corresponds to or is similar to a naturally occurring interference. If it is known from the sensor data, for example, that the weather conditions have changed in such a way that fog appears in the surroundings of the vehicle, its plausibility can be checked by the supplementary sensor data. If, for example, the optical sensor shows that there is no fog, that is, there is clear visibility, it can be recognized that the sensor data changes that may be caused by fog are actually caused by harmful interference. By checking the plausibility of the sensor data, the check of the obtained sensor data can be further improved. This improves the safety when using the method in a driver assistance system.

[0021] When evaluating the supplementary sensor data, in particular the visibility in the vehicle's surroundings is determined. It is then checked whether the difference between the second sensor data and the first sensor data is at least partially due to the visibility in the vehicle's surroundings. The visibility in the vehicle's surroundings can be checked particularly easily by sensors that are usually already provided in the vehicle, independently of the sensors of the driver assistance system. In this way, harmful interference with the driver assistance system can be detected by sensors that are not directly used by the driver assistance system to generate control data. This also increases safety when the method is run in a driver assistance system.

[0022] According to an improvement of the method according to the invention, an improved set of parameters of the neural network used in the method is generated by the following steps:

[0023] a. providing a neural network with a set of associated parameters;

[0024] b. generating training data with the aid of an example sensor data set;

[0025] c. generating a first analysis of the example-data set based on the training data by means of a neural network;

[0026] d. generating disturbed input data by means of the above method, generating disturbed input data as training data of an example sensor data set for a neural network;

[0027] e. generating a second analysis of the example-sensor data set based on the disturbed input data by means of a neural network;

[0028] f. Comparing the first analysis with the second analysis; and

[0029] g. generating an improved set of parameters for the neural network based on the comparison results of the first analysis and the second analysis.

[0030] The example sensor data set is in particular a digital example image. The sensor data used to improve the parameter set are previously acquired sensor data, which are used to train the neural network. However, they can represent similar sensor data, such as similar digital images, which are generated on site, for example, when using the method in a driver assistance system.

[0031] Thus, an improved set of parameters is obtained by training the neural network. The training is carried out on disturbed and undisturbed sensor data, ie, in particular digital images. The improved set of parameters is then obtained, for example, by gradient descent (adversarial training).

[0032] In this development of the method according to the invention, the disturbed input data can be generated in particular by defining a first metric that indicates how the degree of change of the digital image is measured and defining a second metric that indicates for what purpose the input data of the digital image is disturbed. An optimization problem is generated from the combination of the first metric and the second metric. The optimization problem is solved by means of at least one solution algorithm, wherein the solution indicates a target disturbance of the input data, and the disturbed input data are generated for the neural network from the sensor data by means of the target disturbance.

[0033] The sensor data are in particular digital images. In this case, the target disturbance thus generates a disturbed (ie, altered) digital image, which forms the input data for a neural network that analyzes the digital image.

[0034] In the method according to the invention, possible harmful interferences of a neural network for analyzing sensor data are considered at a structural level. The interference is considered as a combination of different elements for which different metrics are defined. Surprisingly, it is possible to achieve that, instead of using only randomly combined interferences, a large number of new harmful target interferences can be generated based on known interferences by analyzing the structure of the known interferences with regard to the metrics.

[0035] In the method according to the invention, an optimization problem is advantageously generated from two metrics, which measure changes in sensor data, in particular digital images. For such optimization problems, there are a large number of known solution algorithms. These solution algorithms can therefore be used to solve the optimization problem. This generates a target disturbance of the input data. With the help of this target disturbance, disturbed input data can then be generated for the neural network from the sensor data. The neural network can then be tested and trained based on these disturbed input data. Advantageously, the method according to the invention makes it possible to generate new disturbances very quickly and in a simple manner.

[0036] The first metric used in the method according to the invention indicates how to measure the degree of change of the sensor data. If the sensor data is a digital image of a camera, the disturbance used to test the neural network should generally be as small as possible. The first metric indicates how to quantify the degree of change of the digital image. For example, a digital image can be changed by moving, rotating, or mirroring the pixels of the image. The first metric indicates the degree of change of such a conversion. According to the first metric, the rotation or translation of the digital image can be defined by a fixed point and a rotation angle or translation distance in the horizontal and vertical directions. In addition, the first metric can determine the image distance for each pixel of the image in the following manner: determine the sum of the differences of all pixel values. The pixel value can be, for example, a grayscale value or a color value. For each pixel, the difference in the pixel value of the original image and the disturbed image is formed. The difference is determined for each pixel, and then the differences are accumulated. As a result, an image distance is generated, which indicates the difference between the two images according to the first metric.

[0037] Furthermore, modified image regions can be observed according to the first metric. The image regions can be defined by a starting point and an extent in the horizontal and vertical directions, or by a list of pixels. Image distances can be determined for these image regions according to the first metric.

[0038] Furthermore, the first metric may indicate a degree of variation of the digital image associated with an image characteristic, such as brightness, contrast and / or structure values, or any combination thereof.

[0039] When defining the first metric, restrictions may also be included, for example, the changes observed in the first metric only take into account image regions in which, for example, certain image features are present. For example, only regions may be observed in which the contrast exceeds a certain threshold.

[0040] According to the design of the method according to the invention, the second metric targets the change in the class of the object. The second metric in particular measures the difference between the true model output and the expected erroneous model output, i.e. the goal of the countermeasure interference. For example, in the case of digital images, small image areas or a small number of pixels may be interfered with so that the objects of the digital image are no longer recognized as traffic participants (e.g. pedestrians), but as objects of another category, such as road areas. In addition, the interference can be intended to ensure that whenever an area is identified as a street, this street is always identified as an empty street without other traffic participants.

[0041] According to another design of the method according to the invention, the second metric can be aimed at the disappearance of the object. The purpose of the interference is, for example, that the identified object undergoes a change that causes it to disappear. The second metric can also be related to only certain image regions. For example, the purpose of the interference described by the second metric can be aimed at the following situation: objects of a certain class cannot appear in a certain image region.

[0042] According to another embodiment of the method according to the invention, the second measure is directed to the change of a certain class of objects. For example, objects can be identified and classified. For example, image areas can be assigned to traffic participants. The second measure is then aimed at, for example, displaying this object larger or smaller or at another position. For example, an object classified as a pedestrian can be displayed smaller or larger. In this case, the enlargement is defined, for example, by the absolute amount of pixels by which the object is enlarged or reduced to the left, right, above and below due to interference.

[0043] There are a large number of possible disturbances that can be described by the second measure. Any changes can be made to the sensor data in order to change the sensor data in such a way that, when analyzing the sensor data in a driver assistance system, in particular safety-relevant results can no longer be correctly obtained. For example, a pattern or a grid can be applied to the sensor data so that objects of a certain class, such as pedestrians, disappear in the digital image, but other objects continue to be correctly classified. For the application of the method according to the invention in driver assistance systems, those second measures which measure naturally occurring disturbances are particularly relevant here: the model output looks plausible, but deviates from the truth in certain safety-relevant details.

[0044] According to the design of the method according to the invention, the disturbances described by the first and / or second metric are naturally occurring disturbances. For use in a driver assistance system, possible disturbances described by the first and / or second metric are thus selected which are particularly relevant for checking and improving the neural network used in the driver assistance system.

[0045] According to another design of the method according to the invention, the first and / or second metric is stored in a database. A data set about naturally occurring disturbances, which are measured using the first and / or second metric using the naturally occurring disturbances, is then loaded from the database. For example, a measure of possible disturbances at the input data (first metric) and a measure of possible changes in the model output (second metric) can be stored in the database. According to a design of the method according to the invention, a data set of naturally occurring disturbances (measured using the first metric) and possible targets (adversarial changes in the model output - for example, ignoring all pedestrians - measured using the second metric) is then loaded from the database.

[0046] In another design of the method according to the invention, disturbed input data can be generated in particular in the following manner: a first group comprising first metrics is defined, which first metrics respectively indicate in different ways how the degree of change of the sensor data is measured; a second group comprising second metrics is defined, which second metrics respectively indicate in different ways what the disturbance of the sensor data is directed at, an arbitrary combination of the first metric of the first group and the second metric of the second group is selected, an optimization problem is generated from the selected combination of the first and second metrics, the optimization problem is solved by means of at least one solution algorithm, wherein the solution indicates a target disturbance of the input data, and disturbed input data is generated for the neural network from the sensor data by means of the target disturbance.

[0047] The advantage of this method is that any first metric of the first group and any second metric of the second group can be used in order to reach the target interference by solving the optimization problem. The more metrics the first group and the second group contain, the more different target interferences can be generated by this method. Therefore, a very large number of target interferences can be generated.

[0048] According to another embodiment of the method according to the invention, the first group includes at least two, in particular at least five different first metrics. However, the first metric may also include more than 10, 20 or more than 100 metrics.

[0049] According to another embodiment of the method according to the invention, the second group includes at least two, in particular at least five different second metrics. However, the second metrics may also include more than 10, 20 or more than 100 metrics.

[0050] The first and / or second measure of the first or second group can in particular have the features described above, either individually or in combination.

[0051] According to another embodiment of the method according to the invention, a third metric is defined, which indicates to which sensor data the third disturbance is applied. For example, the disturbance is applied to all data, only one data point or data with a certain condition, for example all data with a multi-lane road. An optimization problem is then generated from a combination of at least two of the first, second and third metrics. The optimization problem is generated in particular from a combination of the first, second and third metrics. The sensor data are in particular digital images. These digital images are analyzed in particular by a neural network in the driver assistance system.

[0052] The third metric may specifically relate to all sensor data, such as all digital images. For example, in all digital images, interference may cause objects of a certain class to disappear.

[0053] Furthermore, the third metric can only affect a subset of the sensor data, in particular the digital images. The disturbance can, for example, describe only those digital images which contain objects of a certain class, for example objects classified as pedestrians. Furthermore, the third metric can describe digital images recorded on snowy or rainy days. Thus, when used in a driver assistance system, disturbed input data of a neural network can, for example, lead to a different assessment of a particular traffic situation or environmental situation.

[0054] According to another embodiment of the method according to the invention, the third metric describes only sensor data containing a specific object. Alternatively or additionally, the third metric can select only specific digital images.

[0055] According to another embodiment of the method according to the invention, any combination of the first metric of the first group, the second metric of the second group and the third metric is selected. An optimization problem is then generated from the selected combination of the first, second and third metrics.

[0056] The third metric can in particular have the features described above, alone or in combination.

[0057] An optimization problem based on metric generation can be formulated, for example, as follows: for a given maximum change of a digital image, for example by rotating a certain image region, the number of pixels classified as persons should be minimized, and more precisely, as many images as possible should contain persons.

[0058] In another example, in areas of high contrast where changes in the input image are minimal, the number of pixels classified as persons should be minimal, and more specifically, images in which persons appear as much as possible.

[0059] In the method according to the invention, a solution algorithm is specified for this optimization problem. According to one design of the method according to the invention, the solution algorithm includes an iterative method that uses the gradient of the neural network to determine the direction of change. In addition, iterative methods using sampling, evaluation and combinations thereof can be used.

[0060] According to another embodiment of the method according to the invention, a Monte Carlo method is used as a solution algorithm, wherein noise is generated, for example, for a digital image and the result is checked. According to another embodiment, a genetic algorithm can be used to solve the optimization problem.

[0061] The solution to the optimization problem can be, for example, a perturbed digital image, or a perturbation that can be used to perturb the sensor data in order to generate perturbed input data for the neural network. The perturbed sensor data or the perturbed digital image then represent the input data for the neural network to be examined. Perturbations can also be applied to a set of input data by combining at the pixel level, for example by summing.

[0062] If the sensor data is a digital image, the first analysis and the second analysis may include a semantic segmentation of the digital image, the identification of digital image objects, the classification of digital image objects or the identification of the position of an object in the digital image. In addition, the analysis may be used to identify how objects in the digital image change. These analyses are particularly relevant when neural networks are used in driver assistance systems, so that it is important that the neural network is robust to disturbances that occur in such an analysis, so that slight changes occur when using disturbed input data during the analysis.

[0063] According to another embodiment of the method according to the invention, a set of solution algorithms is defined, which contains a plurality of solution algorithms, each of which solves the optimization problem differently in order to generate different target disturbances of the input data. Then, any one of the solution algorithms in the set of solution algorithms is selected in order to generate disturbed input data for the neural network from the sensor data. In this way, an even greater number of target disturbances can be generated, since the solution algorithms can also be varied, wherein each solution algorithm achieves a different target disturbance.

[0064] The solution algorithms of the solution algorithm set may include an iterative method that uses the gradient of a neural network to determine the direction of change, as well as a sampling-based method, a gradient-based method, a gradient-based method with momentum, and / or an agent-model-based method.

[0065] The invention also relates to a driver assistance system for a vehicle. The driver assistance system has a sensor unit arranged in the vehicle for recording sensor data of the surroundings of the vehicle. In addition, it has a verification unit for verifying the recorded sensor data, wherein the verification unit is designed to compare at least first sensor data recorded at a first earlier time point with second sensor data recorded at a second later time point when verifying the sensor data, compare the comparison result with data of a database, in which data about interference of neural network input data are stored, wherein it is checked whether the second sensor data is at least partially generated by the interference of the first sensor data stored in the database. The driver assistance system also includes an evaluation unit connected to the sensor unit, in which a neural network is stored, and the evaluation unit is designed to analyze the verified sensor data with the help of the neural network and thereby generate analyzed sensor data. In addition, the driver assistance system has a control unit, which is connected to the evaluation unit and is designed to generate control data for partially automatically or fully automatically controlling the vehicle based on the analyzed sensor data.

[0066] The driver assistance system according to the invention is particularly designed to carry out the method according to the invention. Therefore, it also has the same advantages.

[0067] The data about disturbances in the input data stored in the database have in particular disturbed input data for such a neural network, which are also stored in the evaluation unit, for example. Alternatively, however, data about disturbances in the input data of any neural network can also be stored in the database, so that the database can be designed for the use of different neural networks in the evaluation unit.

[0068] The sensor unit is in particular a camera for recording digital images. The digital images then record the surroundings of the vehicle. The camera is in particular arranged in the vehicle. Likewise, the database is preferably also arranged in the vehicle and is connected to the verification unit via a wired connection.

[0069] According to a development of the driver assistance system according to the invention, the verification unit is designed to determine a difference between the second sensor data and the first sensor data. In addition, according to this design, the driver assistance system has a supplementary sensor unit, which is designed to acquire supplementary sensor data from the surroundings of the vehicle. The driver assistance system also has a credibility unit connected to the supplementary sensor unit and the verification unit, which is designed to analyze the supplementary sensor data and check whether the difference is consistent with the evaluation of the supplementary sensor data. In particular, it can be checked whether the difference may be caused by a disturbance determined by the evaluation of the supplementary sensor data. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] The invention will now be explained by means of embodiments with reference to the accompanying drawings.

[0071] Figure 1 The structure of an embodiment of a driver assistance system according to the present invention is schematically shown;

[0072] Figure 2 A flow chart of an exemplary embodiment of a method for operating a driver assistance system in a vehicle according to the present invention is schematically shown;

[0073] Figure 3 schematically shows the structure of a generator for generating disturbed input data;

[0074] Figure 4 A flow chart schematically illustrates a method of generating disturbed input data;

[0075] Figure 5 shows an example of interference;

[0076] Figure 6 The structure of an apparatus for generating a parameter set is schematically shown; and

[0077] Figure 7 A flow chart of a method for improving a set of parameters of a neural network is shown. DETAILED DESCRIPTION

[0078] Will refer to Figure 1 An embodiment of a driver assistance system 20 according to the present invention is explained.

[0079] The driver assistance system 20 comprises a sensor unit 21 arranged in the vehicle. This embodiment involves a camera which records digital images of the vehicle's surroundings. These digital images may be generated continuously during use of the vehicle.

[0080] The sensor unit 21 is connected to a verification unit 22. The verification unit 22 can verify the sensor data recorded by the sensor unit 21, i.e. determine whether undesired changes, in particular harmful interference, have occurred in the sensor data. In this way, the sensor data are also verified, i.e. the unchanged origin of the sensor data from the sensor unit 21 is checked. For this purpose, the verification unit 22 compares sensor data recorded at different times, in particular consecutive sensor data sets. For example, two consecutive sensor data sets can be compared. However, it is also possible to compare a large number of consecutive sensor data sets in order to determine changes. The verification unit 22 determines the difference between the sensor data recorded later and the sensor data recorded earlier.

[0081] The verification unit 22 is connected to the database 16. Data about disturbances in the neural network input data are stored in the database 16. In this case, a distinction is made in the database 16 whether the disturbance is a natural disturbance or whether this is not the case. A natural disturbance is understood to be a disturbance that influences the sensor data of the vehicle's surroundings, in the present case a digital image, as these sensor data can also be caused by naturally occurring phenomena in the vehicle's surroundings.

[0082] Changes in the digital image due to natural disturbances correspond, for example, to changes in the digital image when weather phenomena occur, such as when fog, snow or rain occur. Furthermore, natural disturbances are understood to be changes in the image in which objects are inserted into the image or disappear from the image, which can also occur in the vehicle's surroundings. For example, posters or stickers can appear on objects in the vehicle's surroundings.

[0083] Other non-naturally occurring disturbances include, in particular, harmful disturbances to the neural network, by which the safe operation of the driver assistance system is disturbed. Such disturbances are marked in the database 16. In this way, it can be determined with the aid of the verification unit 22 whether a change in the sensor data originates from a natural disturbance or whether the change may be caused by a harmful disturbance. The verification unit 22 is designed to compare the comparison results of the sensor data recorded at different points in time with the data of the database 16. Here, it can be checked whether the sensor data recorded later is at least partially caused by the disturbance of the first sensor data stored in the database. If the result shows that the change is not caused by a natural disturbance, the verification unit 22 verifies the recorded sensor data and transmits it to the credibility unit 23.

[0084] The credibility unit 23 is connected to a supplementary sensor unit 24. Alternatively or additionally, the credibility unit 23 can also be connected to an interface 25, via which supplementary sensor data of the credibility unit 23 can be transmitted. For example, the supplementary sensor unit 24 collects the field of view in the surroundings of the vehicle and is, to be precise, independent of the sensor unit 21 designed as a camera. Alternatively or additionally, weather data can also be transmitted to the credibility unit 23 via the interface 25. As a result, the credibility unit 23 can determine the field of view in the surroundings of the vehicle.

[0085] The plausibility unit 23 can now analyze the supplementary sensor data and check whether the differences in the recorded sensor data determined by the verification unit 22 are consistent with the analysis of the supplementary sensor data. For example, the plausibility check in the plausibility unit 23 is negative if the verification unit 22 shows that the field of view in the surroundings of the vehicle has been greatly reduced because the digital image recorded by the sensor unit 21 shows the presence of fog, and the supplementary sensor data transmitted to the plausibility unit 23 by the supplementary sensor unit 24 or via the interface 25 show that there is clear visibility. Although changes in the recorded sensor data of the sensor unit 21 may result from natural disturbances, it turns out that such changes are most likely caused by harmful disturbances. In this case, the sensor data that have not been plausibly checked are not processed further.

[0086] The plausibility unit 23 is connected to an evaluation unit 26. The neural network 11 is stored in the evaluation unit 26. The evaluation unit 26 is designed to analyze the verified and plausibility-checked sensor data with the aid of the neural network 11 and to generate analyzed sensor data therefrom. The analysis is performed in a manner known per se. For example, a semantic segmentation is performed on the digital image and the objects recognized in the digital image are assigned to different classes. In this way, it can be determined, for example, whether a pedestrian is located on the road and how the pedestrian moves relative to the vehicle and relative to the road.

[0087] The evaluation unit 26 is connected to a control unit 27. The control unit 27 is designed to generate control data for partially or fully automatic control of the vehicle based on the analyzed sensor data. These control data are transmitted by the driver assistance system 20 to actuators 28, which control, for example, the steering and propulsion or braking of the vehicle. In addition, the actuators 28 can control signals emitted by the vehicle. Such partially or fully automatic control of a vehicle by means of a driver assistance system 20 is known per se.

[0088] refer to Figure 2 , an exemplary embodiment of a method according to the invention for operating a driver assistance system 20 is explained below:

[0089] In step T1 , sensor data from the surroundings of the vehicle are continuously recorded by means of sensor unit 21 .

[0090] In step T2, the first sensor data recorded at a first earlier time point is compared with the second sensor data recorded at a second later time point, and a comparison result is generated.

[0091] In step T3, the comparison result is compared with the data of the database 16. It is considered here whether the data of the database 16 belongs to interference that may have a natural cause or to harmful interference.

[0092] In step T4, it is checked whether the second sensor data are at least partially generated by a disturbance of the first sensor data stored in the database. If the stored disturbance is a natural disturbance, the recorded sensor data are verified in step T5.

[0093] In step T6, the plausibility of the verified sensor data is checked. To this end, additional sensor data is acquired from the vehicle's surroundings and the additional sensor data is analyzed. Furthermore, differences between the second sensor data and the first sensor data are determined and it is checked whether these differences are consistent with the analysis of the additional sensor data.

[0094] In step T7, the verified and plausibility-checked sensor data are analyzed by means of the neural network 11. In step T8, analyzed sensor data are generated from this.

[0095] Finally, in step T9 , control data for partially automatic or fully automatic control of the vehicle are generated and output based on the analyzed sensor data.

[0096] In a refinement of an embodiment of the method according to the invention and in a refinement of an embodiment of the driver assistance system 20 according to the invention, a parameter set is obtained in a special manner for the neural network 11 used. In this case, in order to train the neural network 11, before the actual use in the driver assistance system 20, disturbed input data obtained in a special manner are generated.

[0097] For this purpose, refer to the following Figure 3 An embodiment of a generator 10 is described for generating perturbed input data for a neural network for analyzing digital images of an embodiment of a driver assistance system.

[0098] In this embodiment, the sensor data are analyzed by the neural network or disturbed input data are generated for the neural network from these sensor data. In these embodiments, the sensor data are raw data from vehicle sensors, which are obtained before the actual use of the neural network in the driver assistance system. The sensor can be a camera, a radar sensor, a lidar sensor or any other sensor that generates sensor data, which is further processed in the driver assistance system. In the following, it is assumed, for example, that the sensor data are digital images taken by a camera of the vehicle. However, other sensor data can also be used in the same way.

[0099] The generator 10 comprises a first metric unit 1, a second metric unit 2 and a third metric unit 3. The first metric unit 1 comprises a first metric indicating how to measure the degree of change of a digital image. The first metric unit 1 defines how to measure the degree of change of a digital image. The definition of the first metric can be input into the first metric unit 1. However, the first metric unit 1 can also access a database 16 via an interface, in which data with a plurality of possible definitions of metrics are stored, which measure the degree of change of a digital image. For example, the first metric can compare the image distance between two digital images and output a value for the image distance. The image distance can be defined, for example, by the sum of the difference values ​​of all pixel values ​​of the digital images to be compared.

[0100] In this embodiment, the first measurement unit 1 selects disturbances that are as natural as possible from the database 16. Natural disturbances are understood to be disturbances that affect the digital image of the vehicle's surroundings, just as they would also occur due to naturally occurring phenomena in the vehicle's surroundings. Changes in the digital image due to natural disturbances correspond, for example, to changes in the digital image when weather phenomena occur, such as changes that occur when fog, snowfall or rain occur. In addition, natural disturbances are understood to mean changes in the image in which objects are inserted into the image or disappear from the image, which can also occur in the vehicle's surroundings. For example, posters or stickers may be embedded in objects in the vehicle's surroundings. The second measurement unit 1 does not take into account other, non-natural disturbances that may also be contained in the database 16, since these disturbances are less relevant for testing the neural network used in the driver assistance system.

[0101] The second metric unit 2 includes a second metric, which indicates what the disturbance of the input data of the digital image is directed to, i.e. the second metric defines what the disturbance of the digital image is directed to. The definition of the second metric can be transmitted to the second metric unit 2 by input. Similarly, the second metric unit 2 can also be connected to a database 16, in which data about a large number of disturbances are stored, which are directed to specific changes of the digital image. It can be a collection of these disturbances here.

[0102] In the exemplary embodiment, the second measurement unit 2 selects disturbances from the database 16 that are as plausible as possible. Plausible disturbances are understood to be disturbances that clearly lead to a real model output, but differ from it in relevant details. In the case of plausible disturbances, for example, a correct segmentation occurs, but in which the lane markings are constantly moving. Other implausible disturbances, such as those that may also be contained in the database 16, are not taken into account by the second measurement unit 2, since they are less relevant for testing the neural network used in the driver assistance system. Severely implausible model outputs can thus be easily detected.

[0103] The second measure can, for example, be intended to increase the size of all objects assigned to a certain class, such as the class of pedestrians. The perturbation thus generates a digital image in which objects of the initial image classified as pedestrians are iteratively enlarged in all four directions, wherein the resulting segmentations of the perturbed digital image are again combined with one another. The result is a digital image in which all objects that do not belong to the class of pedestrians remain unchanged, but objects that belong to the class of pedestrians are displayed enlarged. The other objects are changed only to the extent that they are changed by enlarging the objects of the class of pedestrians.

[0104] The third metric unit 3 includes a third metric, which indicates to which digital images the interference applies. For example, it can be defined by this metric that the interference applies only to digital images that show other traffic participants, ie, for example, pedestrians, cyclists and other vehicles.

[0105] The three measurement units 1 to 3 are connected to a processing unit 4. The processing unit 4 is designed to generate an optimization problem from the three metrics of the first to third measurement units 1 to 3. For example, the optimization problem includes a loss function of a neural network, which contains as parameters a disturbance parameter and an image generated by the disturbance (second metric). For the optimization problem, the minimum value of the disturbance parameter should be found, and more precisely, for a digital image defined according to the third metric, and the condition is that the degree of change of the generated image relative to the output image according to the first metric is below a certain value.

[0106] The processing unit 4 transmits the optimization problem as a data set to the solution unit 5. The solution unit 5 is connected to a database 6, in which at least one solution algorithm for the optimization problem, preferably a plurality of solution algorithms, is stored. Such solution algorithms are known per se. For example, Monte Carlo methods, genetic algorithms and / or gradient-based methods can be stored in the database 6, which the solution unit 5 can call. With the aid of these solution algorithms, the solution unit 5 can generate a target disturbance of the input data of the digital image as a solution to the optimization problem. The target disturbance thus generates a disturbed digital image, which can be used as input data for a neural network for analyzing the digital image. The neural network is particularly designed for analyzing digital images of a driver assistance system.

[0107] The solver unit 5 transmits the target perturbations to the generator unit 7. The generator unit 7 is also connected to a database 8 in which a plurality of digital images are stored. With the aid of the target perturbations, the generator unit 7 can perturb the digital images in the database 8 so that perturbed input data 9 of the digital images are generated for the neural network. The perturbed input data 9 are then output by the generator unit 7. With these perturbed input data 9, the neural network can then be tested, trained, or the parameter set of the neural network can be improved.

[0108] refer to Figure 4 An embodiment of the method according to the invention will be explained for generating disturbed input data 9:

[0109] In step S1 , a first metric is defined which indicates how to measure the degree of change of a digital image. The first metric or a data set describing the first metric is stored in a first metric unit 1 .

[0110] In step S2 , a second metric is defined which indicates what the digital image is disturbed against. This second metric or a data set describing the second metric is also stored in the second metric unit 2 .

[0111] Finally, in step S3 a third metric is defined which indicates to which digital images the disturbance applies. The third metric or a data set describing the third metric is stored in the third metric unit 3 .

[0112] In step S4 , the data set describing the three metrics is transmitted to the processing unit 4 .

[0113] In step S5 , the processing unit 4 generates an optimization problem according to the combination of the three metrics. In step S6 , the processing unit 4 transmits a data set describing the generated optimization problem to the solving unit 5 .

[0114] In step S7, the solution unit 5 solves the optimization problem by means of at least one solution algorithm, which has been transmitted to the solution unit 5, for example, by accessing the database 6. The solution is a target disturbance of the digital image.

[0115] In step S8 , the data set of the target disturbance is transmitted to the generation unit 7 .

[0116] In step S9, the generating unit 7 generates the disturbed digital image as input data 9 of the neural network by accessing the database 8. In step S10, the disturbed input data 9 are output.

[0117] The following describes the method according to the present invention in detail by taking the enlargement of pedestrian objects as an example:

[0118] Refer to the following FIG. 5A to FIG. 5C , taking the enlargement of pedestrian objects as an example, the method according to the present invention is described in detail:

[0119] Given a model M. For this model there is an input x. This input x is a digital image, such as Figure 5A As shown. In addition, the output M(x)=y is defined. The disturbance is represented by Δ, so the changed input x'=x+Δ is obtained. The changed output is then y'=M(x+Δ). The target output is represented by y''.

[0120] The output y of model M is Figure 5BThe digital image x has been segmented, that is, the pixels of the digital image x have been classified, such as Figure 5B This results in the following classification:

[0121] K1: sky;

[0122] K2: Natural;

[0123] K3: Buildings;

[0124] K4: Pedestrian;

[0125] K5: Traffic signs;

[0126] K6: Street;

[0127] K7: Mark.

[0128] The target output y'' to be generated by the disturbance Δ is Figure 5C The target interference Δ is to magnify the pedestrian. The target interference is defined as that each pixel is allowed to have a maximum displacement value of 3. The target data consists of a specific image x.

[0129] The first metric is then defined as follows:

[0130]

[0131] The size of the disturbance is therefore measured as the maximum pixel value between 0 and 255 in the disturbance Δ.

[0132] The second metric is defined as follows:

[0133]

[0134] It defines the sum of pixel differences from the target output.

[0135] The third metric is defined as follows:

[0136]

[0137] According to this third metric, only the input image x has a smaller size. Therefore, if d3(x') < 1 is required, the attack only concerns the input image x. If d3(x') < 2 is required, the focus on the data to be attacked changes significantly: the attack then concerns all images.

[0138] The optimization problem then formed from these three metrics is as follows:

[0139]

[0140] According to the optimization problem, we should find Δ so that d2(Δ) is minimized, where d1(Δ) on x<3.

[0141] The optimization problem can be solved by means of known solution algorithms. Thus, new countermeasures are obtained from the known (d1, d3) and new (d2) metrics. New countermeasures are also generated by recombining the known metrics (d1, .., d3) in a new way or by combining them with another solution algorithm. Therefore, the method according to the invention allows to construct almost any number of new countermeasures in a simple way.

[0142] According to a variation of this example, the first metric can only allow pixel changes in image regions classified as "trees". The following optimization problem is then obtained: Δ should be found in the image region "trees" of the digital image x so that d2(Δ) is minimized, where d1(Δ)<3.

[0143] According to another variation of this example, the interference of all images can be searched for the third metric, where the first metric d1 and the second metric d2 remain unchanged. The optimization problem can then be expressed as follows: to find Δ such that d2(Δ) is minimized for all images, where d1(Δ)<3. In other words: to find Δ with d1(Δ)<3, so that the model output for all input images x looks like y".

[0144] refer to Figure 6 An embodiment of an apparatus for generating a set of parameters for a neural network will be described:

[0145] The device comprises a database 8 with digital images. Figure 3 The generator 10 depicted is connected to this database 8. A neural network 11 is connected to the database 8 and to the generator 10. The output of the neural network 11 is connected to a first analysis unit 12 and a second analysis unit 13. The first analysis unit 12 generates a first analysis with the aid of the neural network 11 based on the digital images supplied to the neural network 11 as input data from the database 8. The second analysis unit 13 generates a second analysis based on the disturbed input data 9 supplied to the neural network 11 by the generator 10. To generate the disturbed input data 9, the generator 10 accesses the database 8 with digital images and applies the target disturbance generated by the generator 10 to the database.

[0146] The first analysis unit 12 and the second analysis unit 13 are connected to a comparison unit 14. The comparison unit is designed to compare the first analysis and the second analysis with one another.

[0147] The comparison unit 14 is connected to a parameter set generation unit 15. The parameter set generation unit 15 is designed to generate an improved parameter set for the neural network 11 based on the comparison results of the first analysis and the second analysis transmitted from the comparison unit 14. The parameter set of the neural network 11 is generated by the parameter set generation unit 15 so that the disturbed input data 9 of the digital image generated by the generator 10 have little influence on the analysis of these input data by means of the neural network 11. In particular, the improved parameter set is generated in such a way that the influence of the disturbed input data 9 on the semantic segmentation of the digital image for the disturbed input data by means of the neural network 11 does not lead to an incorrect classification of safety-related objects for the driver assistance system, which objects disappear or are displayed in a modified manner. The neural network 11 can thus be trained with the help of the disturbed input data 9 generated by the generator 10.

[0148] refer to Figure 7 A method for improving the set of parameters of a neural network will be described:

[0149] In step R1, a neural network with an associated set of parameters is provided. This neural network will be checked.

[0150] In step R2, training data is generated with the aid of a large number of digital images.

[0151] In step R3 , the neural network is trained in a manner known per se using training data, and a first analysis of the digital image is generated with the aid of the neural network on the basis of the training data.

[0152] In step R4, with the help of reference Figure 4 The explained method generates perturbed input data as training data of digital images.

[0153] In step R5 , a second analysis of the digital image is generated by means of a neural network based on the disturbed input data, ie based on the digital image to which the target disturbance has been applied.

[0154] In step R6 , the first analysis and the second analysis are compared with each other.

[0155] In step R8, based on the results of the comparison and the second analysis, an improved set of parameters for the neural network is generated.

[0156] Another embodiment of a generator 10 according to the invention and a method according to the invention for generating disturbed input data 9 is explained below:

[0157] As in the first embodiment, the generator 10 of another embodiment includes a first metric unit 1 and a second metric unit 2. However, in this case, the first metric unit 1 includes a first group of a plurality of first metrics, each of which indicates how to measure the degree of change of the sensor data. In this case, the second metric unit 2 includes a second group of a plurality of second metrics, each of which indicates what the interference of the input data 9 of the sensor data is directed to. In this case, the processing unit 4 connected to the first metric unit 1 and the second metric unit 2 is designed to generate an optimization problem by any combination of the first metric of the first group and the second metric of the second group.

[0158] The solution unit 5 connected to the processing unit 4 is then designed to solve the optimization problem by means of at least one solution algorithm, wherein the solution indicates a target disturbance of the input data 9 of the sensor data. Similarly to the first exemplary embodiment, the generation unit 7 is also designed to generate disturbed input data 9 for the neural network 11 from the sensor data by means of the target disturbance.

[0159] The method of another embodiment is performed similarly to the method of the first embodiment. However, in this case, a first group is defined that includes first metrics that each indicate how the degree of change of the sensor data is measured. In addition, a second group is defined that includes second metrics that each indicate what the disturbance of the sensor data is directed to. Any combination of the first metric of the first group and the second metric of the second group is then selected, and an optimization problem is generated from the selected combination of the first and second metrics. Then, as in the method of the first embodiment, the optimization problem is solved with the aid of at least one solution algorithm, wherein the solution indicates a target disturbance of the input data 9. With the aid of the target disturbance, disturbed input data 9 are generated for the neural network 11 from the sensor data.

[0160] List of Reference Numerals

[0161] 1 First unit of measure

[0162] 2 Second unit of measure

[0163] 3 Third unit of measure

[0164] 4 Processing Units

[0165] 5 Solution Unit

[0166] 6 Database with solution algorithm

[0167] 7 Generation Unit

[0168] 8 Databases with digital image data

[0169] 9 Disturbed Input Data

[0170] 10 Generator

[0171] 11 Neural Networks

[0172] 12 First Analysis Unit

[0173] 13 Second analysis unit

[0174] 14 Comparison unit

[0175] 15 Parameter group generation unit

[0176] 16 Database with interference

[0177] 20 Driver Assistance Systems

[0178] 21 Sensor unit

[0179] 22 Verification Unit

[0180] 23 Credibility Units

[0181] 24 Supplementary sensor units

[0182] 25 Interface for additional sensor data

[0183] 26 evaluation units

[0184] 27 Control Unit

[0185] 28 Actuator.

Claims

1. A method for operating a driver assistance system (20) of a vehicle, wherein: continuously recording sensor data of the vehicle's surroundings, and generating control data for partially or fully automatic control of the vehicle based on the analyzed sensor data, It is characterized in that Verify the recorded sensor data, analyzing the verified sensor data by means of a neural network (11), thereby generating the analyzed sensor data, When verifying the sensor data, at least first sensor data recorded at a first earlier point in time are compared with second sensor data recorded at a second later point in time, The result of the comparison is compared with data of a database (16) in which data about disturbances of the input data of the neural network are stored, wherein it is checked whether the second sensor data is generated at least partially by disturbances of the first sensor data stored in the database (16).

2. The method according to claim 1, It is characterized in that The sensor data are digital images of a camera of the vehicle, and the digital images record the surroundings of the vehicle.

3. The method according to claim 1, It is characterized in that The data stored in the database (16) describe naturally occurring disturbances.

4. The method according to claim 1, It is characterized in that The plausibility of the verified sensor data is checked by obtaining supplementary sensor data from the surroundings of the vehicle, analyzing the supplementary sensor data, determining a difference between the second sensor data and the first sensor data, and checking whether the difference is consistent with the analysis of the supplementary sensor data.

5. The method according to claim 4, It is characterized in that When analyzing the supplementary sensor data, visibility in the surroundings of the vehicle is determined and a check is performed to determine whether the difference between the second sensor data and the first sensor data is at least partially due to the visibility in the surroundings of the vehicle.

6. The method according to any one of the preceding claims 1 to 5, wherein: The improved parameter set of the neural network (11) is generated by the following steps: providing a neural network (11) having an associated set of parameters, generating training data with the aid of example sensor data sets, generating a first analysis of the example sensor data set based on the training data by means of the neural network (11), generating disturbed input data (9) as training data for the example sensor data set, generating a second analysis of the example sensor data set based on the disturbed input data (9) by means of the neural network (11), Compare the first and second analyses, and Based on the comparison results of the first analysis and the second analysis, the improved parameter set is generated for the neural network (11), wherein the disturbed input data (9) is generated in the following manner: defining a first metric indicating how to measure a degree of variation in sensor data, defining a second metric indicating what the sensor data is disturbed for, generating an optimization problem from a combination of the first metric and the second metric, The optimization problem is solved by means of at least one solution algorithm, wherein the solution indicates a target disturbance of the input data, and With the aid of the target disturbance, disturbed input data are generated for the neural network (11) from the sensor data.

7. The method according to claim 6, It is characterized in that In each of the first and second analyses, a semantic segmentation of a digital image of a camera of the vehicle recording the vehicle surroundings is performed.

8. The method according to any one of claims 1 to 5, wherein: The improved parameter set of the neural network (11) is generated by the following steps: providing a neural network (11) having an associated set of parameters, generating training data with the aid of example sensor data sets, generating a first analysis of the example sensor data set based on the training data by means of the neural network (11), generating disturbed input data (9) as training data for the example sensor data set, generating a second analysis of the example sensor data set based on the disturbed input data (9) by means of the neural network (11), Compare the first and second analyses, and generating an improved set of parameters for the neural network (11) based on the comparison of the first analysis and the second analysis, The disturbed input data (9) is generated in the following manner: defining a first group including first metrics, each of which differently indicates how to measure a degree of variation of sensor data, defining a second group comprising second metrics, which respectively indicate in different ways what the disturbance of the sensor data is directed to, selecting any combination of the first metric of the first group and the second metric of the second group, generating an optimization problem from the selected combination of the first metric and the second metric, The optimization problem is solved by means of at least one solution algorithm, wherein the solution indicates a target disturbance of the input data, and With the aid of the target disturbance, disturbed input data are generated for the neural network (11) from the sensor data.

9. The method according to claim 8, It is characterized in that In each of the first and second analyses, a semantic segmentation of a digital image of a camera of the vehicle recording the vehicle surroundings is performed.

10. The method according to claim 8, It is characterized in that defining a third metric indicating to which sensor data the disturbance applies, and The optimization problem is generated by a combination of at least two of the first metric, the second metric, and the third metric.

11. A driver assistance system (20) for a vehicle, comprising: a sensor unit (21) arranged in the vehicle and used to record sensor data of the surrounding environment of the vehicle; A verification unit (22) for verifying the recorded sensor data, wherein The verification unit (22) is designed to, when verifying the sensor data, compare at least first sensor data recorded at a first earlier point in time with second sensor data recorded at a second later point in time, compare the result of the comparison with data of a database (8), in which data about disturbances of input data (9) of the neural network (11) are stored, wherein it is checked whether the second sensor data is generated at least partially by disturbances of the first sensor data stored in the database (8); an evaluation unit (26) connected to the sensor unit (21), in which the neural network (11) is stored and which is designed to analyze verified sensor data with the aid of the neural network (11) and thereby generate analyzed sensor data; A control unit (27) is connected to the evaluation unit (26) and is designed to generate control data for semi-automatic or fully automatic control of the vehicle based on the analyzed sensor data.

12. The driver assistance system (20) according to claim 11, It is characterized in that The sensor unit (21) is a camera for recording digital images, wherein the digital images record the surroundings of the vehicle.

13. The driver assistance system (20) according to claim 11 or 12, It is characterized in that The verification unit (22) is designed to determine differences between the second sensor data and the first sensor data, The driver assistance system (20) has a supplementary sensor unit (24) which is designed to acquire supplementary sensor data from the surroundings of the vehicle. The driver assistance system (20) also has a plausibility unit (23) connected to the supplementary sensor unit (24) and the verification unit (22), the plausibility unit (23) being designed to analyze the supplementary sensor data and to check whether the discrepancy is consistent with the analysis of the supplementary sensor data.