Method, device and computer program for authorizing the use of a sensor system for detecting objects in the environment of a vehicle

By analyzing the deviation combination and joint probability distribution of the sensor system, the risks of autonomous vehicles are evaluated, and the problem of ensuring a high number of kilometers for autonomous vehicles is solved, and the accurate risk assessment and safety improvement of autonomous vehicle sensor systems are achieved.

CN114651190BActive Publication Date: 2024-11-08ZF FRIEDRICHSHAFEN AG
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202080077767.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2019-11-08
Filing Date
2020-11-06
Publication Date
2024-11-08
Estimated Expiration
2040-11-06

AI Technical Summary

Technical Problem

The prior art is difficult to effectively reduce the number of guaranteed kilometers of autonomous vehicles, which leads to challenges in the approval and use of autonomous vehicles, especially the lack of effective sensor models in the sensing environment, and the inability to accurately calculate the risk of accidents caused by errors.

Method used

By analyzing the deviation combination and joint probability distribution of each sensor in the sensor system, the failure probability is calculated, and combining functional tests, error response tests and hardware tests, the residual risk of autonomous vehicles is evaluated, thereby reducing the number of guaranteed kilometers.

Benefits of technology

It realizes an accurate risk assessment of the autonomous vehicle sensor system, reduces the guaranteed number of kilometers, provides the reliability of approved use of autonomous vehicles, reduces test distance and time, and improves the safety and reliability of the system.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114651190B_ABST
    Figure CN114651190B_ABST
Patent Text Reader

Abstract

The present invention relates to a method for authorizing the use of a sensor system (10) for detecting objects in the environment of a vehicle (1), the method comprising the following steps: providing a joint probability distribution (V1) for deviations (a) between output data of the sensor system (10) and reference data at the level of a program part of the sensor system (10) for detecting objects, at the level of a sensor (11) of the sensor system (10) and / or at the level of a fusion of the sensor system (10), sampling deviation combinations and calculating the occurrence probability (p) of the sampled deviation combinations with the help of the joint probability distribution (V2), loading the sampled deviation combinations into reference data, processing the loaded reference data with the help of a fusion unit (12) of the sensor system (10) and obtaining a fusion result (V3), removing the occurrence probabilities (p) of those deviation combinations that lead to a fusion result that satisfies a predetermined condition from the joint probability distribution and obtaining a residual probability distribution (V4), integrating the residual probability distribution and obtaining an absolute error probability (P) (V5), and authorizing the use of the sensor system (10) in dependence on the absolute error probability (P) (V6).
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to a method, a device and a computer program for authorizing the use of a sensor system for detecting objects in the environment of a vehicle. The invention relates in particular to authorizing the use in relation to systematically and statistically occurring SOTIF risks. SOTIF stands for safety of the intended functionality. Furthermore, the invention relates to a sensor system for detecting objects in the environment of a vehicle. Background Art

[0002] Approval for use of the system is necessary for the transition of the technical system from the development phase to series production. Approval for use is only achieved if the technical system meets previously defined requirements, in particular the requirements for the safety of the technical system. The concept of approval for use currently used in the automotive industry provides for separate approvals for different automation levels, in which controllability by a human driver is always granted, either through direct control by the driver or through a feedback level to the driver.

[0003] The special function of autonomous driving is that starting from a certain level of automation (e.g. SAE level 3+), the return level to the human driver is eliminated. Safety then comes entirely from the technical system of the driving robot and the vehicle. In the following document "Autonomes Fahren–Technische, rechtliche und gesellschaftliche Aspekte (Autonomous Driving - Technical, Legal and Social Aspects)", Springer Vieweg, 2015, Wachenfeld and Winner disclosed in their article "Die Freigabe des autonomen Fahrens (Approved Use of Autonomous Driving)" that, based on 3,375 fatal accidents recorded by the German police in 2012 and 709 billion kilometers of driving in Germany, this figure means that on average there are 210 million kilometers between two fatal accidents. Based on the Poisson distribution of these numbers and the requirement that the performance of autonomous vehicles is twice as good as that of vehicles currently driven by human drivers, autonomous vehicles need to travel a test distance of at least 2.1 billion kilometers. In this case, a proof with a probability of 50% is provided. However, five accidents occurred at this probability. Another statistical model that addresses the question of how many test kilometers an autonomous vehicle must travel to demonstrate that it is reliable in terms of safety and risk is the RAND model, see http: / / www.jstor.org / stable / 10.7249 / j.ctt1btc0xw. The RAND model provides a figure of 8.8 billion test kilometers. Another model is the Minimal Endogenous Morality (MEM), which is a risk analysis model for safety-related systems such as railway facilities or power plants. For highway driver assistance systems (Autobahnpiloten), the MEM model provides a figure of 10 -9 / hour mortality threshold.

[0004] Such a high number of guaranteed kilometers poses a challenge for approval for use. In addition, there are no legal requirements for the approval of autonomous vehicles. Therefore, in the foreseeable future, it will not be possible to approve the use of autonomous vehicles with existing technology. Although such a method is required in the SOTIF standard ISO / PAS 21448:2019 "Road vehicles-Safety of intended functionality", there is a lack of usable methods in the prior art for approval of use and calculation of the expected residual risk of the system.

[0005] The statistical models known from the prior art relate to the entire system of the autonomous vehicle and cannot be easily applied specifically to perception, i.e. the detection or awareness of objects in the vehicle environment. Accidents caused by errors in detection cannot be defined with the help of statistics. It cannot be assumed that errors in detection do not lead to accidents. The PEGASUS research project (i.e. a project to establish generally accepted commercial standards, tools and methods as well as scenarios and situations for the approval of the use of highly automated driving functions) does provide a comprehensive approach for ensuring SAE J 3016 Level 3 systems. However, due to the lack of effective sensor models, scenario-based approaches cannot be used to ensure perception, but only to ensure the decision-making and planning algorithms of the driving strategy. Summary of the invention

[0006] The object of the present invention is to reduce the number of guaranteed kilometres required for approval of a sensor system for detecting objects in the vehicle environment.

[0007] The subject matter of the invention achieves this task by taking into account the deviation combinations of the individual sensors of the sensor system and calculating the probability of failure. This allows the analysis of the residual risk of the autonomous vehicle. This also includes further aspects of the approval for use, such as functional tests, error response tests or hardware tests.

[0008] According to one aspect, the present invention provides a method for authorizing the use of a sensor system for detecting objects in a vehicle environment. The method comprises the following method steps:

[0009] providing a joint probability distribution for deviations between output data of the sensor system and reference data at the level of a program part of the sensor system for detecting an object, at the level of sensors of the sensor system and / or at the level of a fusion of the sensor system,

[0010] · sampling the deviation combinations and calculating the occurrence probability of the sampled deviation combinations with the help of the joint probability distribution,

[0011] Loading the sampled deviation combination to the reference data, processing the loaded reference data with the help of the fusion unit of the sensor system and obtaining a fusion result,

[0012] removing the occurrence probabilities of those deviation combinations that lead to fusion results that meet predetermined conditions from the joint probability distribution, and obtaining the residual probability distribution,

[0013] Integrate the residual probability distribution and obtain the absolute error probability, and

[0014] Reliance on absolute error probabilities to approve the use of sensor systems.

[0015] According to one aspect of the invention, the method according to the invention is implemented by a computer.

[0016] According to another aspect, the invention provides an apparatus for authorizing the use of a sensor system for detecting objects in a vehicle environment. The apparatus comprises at least one processing unit, which is adapted to implement the steps of the method according to the invention. According to one aspect of the invention, the processing unit is hardware of a control device of the vehicle, such as a computer or a part thereof. The processing unit is also adapted to fuse the output data of the individual sensors of the sensor system.

[0017] According to another aspect, the invention provides a sensor system for detecting objects in a vehicle environment. The sensor system comprises a plurality of sensors of one sensor technology or at least one sensor of each sensor technology, a device according to the invention and a fusion unit for fusing the data.

[0018] According to another aspect, the invention provides a computer program for authorizing the use of a sensor system for detecting objects in a vehicle environment. The computer program comprises instructions which, when the computer program is executed by a processing unit, cause the processing unit to carry out the steps of the method according to the invention. The instructions are formulated in a programming language, such as an object-oriented programming language, such as C++. According to one aspect of the invention, the computer program is implemented by a processing unit of a device according to the invention.

[0019] According to another aspect, the invention provides a computer-readable data carrier on which a computer program according to the invention is stored. The data carrier is, for example, a semiconductor memory in the form of a memory module, such as a memory card. For example, the semiconductor memory is a flash memory. For example, the data carrier is connected to an input interface of a control device of a vehicle. Thus, the control device of the vehicle can implement the method.

[0020] According to another aspect, the invention provides a vehicle. The vehicle comprises at least one sensor system according to the invention, at least one control device and an actuator, wherein the sensor system, the control device and the actuator are connected in a signal transmission manner so that the vehicle is at least conditionally autonomously driven. In addition, the vehicle comprises an interface to at least one reference sensor system. In addition, the vehicle comprises an apparatus according to the invention. During the driving of the vehicle, an absolute error probability is obtained according to the steps of the method of the invention and is fed back into the driving strategy of the vehicle. According to one aspect of the invention, a statement about the absolute error probability that depends on the system state of the sensor system, the environmental conditions and the objects near the vehicle is fed back into the driving strategy. Conditionally autonomous driving means that the autonomous vehicle performs lateral and longitudinal guidance and monitors its surroundings, but the human driver must react as a return level when intervention is required. For example, the vehicle is a road vehicle. According to one aspect of the invention, the vehicle is an autonomous passenger vehicle.

[0021] According to another aspect of the invention, the invention provides a data carrier signal, which transmits a computer program according to the invention. For example, the computer program is stored in a database, such as a cloud storage. With the aid of the data carrier signal, the entire fleet can be reached in order to verify the sensor systems of the individual vehicles. To this end, the computer program according to the invention is transmitted to the control devices of the vehicles in the fleet with the aid of the data carrier signal, for example from a cloud storage, and the method according to the invention is implemented. According to one aspect of the invention, the control device includes means for network operation, such as a transmission control protocol (TCP for short), or an Internet protocol (IP for short), a network stack. This results in a larger test base, because data collected from each vehicle in the fleet can be used.

[0022] Further embodiments of the invention are apparent from the dependent claims, the drawings and the description of preferred exemplary embodiments.

[0023] A sensor system, or also known as an environmental awareness system, is a system for sensing the environment and / or vehicle state variables. The sensor system is an ADAS or AD sensor system. ADAS means Advanced Driver Assistance System, i.e. a driver assistance system, which is an additional electronic device used to support the driver in specific driver situations. AD means Autonomous Driving. The sensor system includes a plurality of sensors of one sensor technology or at least one sensor of each sensor technology. With the aid of a plurality of sensors, the sensor system is a redundant system. The sensor technology is, for example, optical technology. Sensors in the field of optical technology include, for example: image sensors (which are used in the visible light range or the infrared range); camera systems with image sensors (such as monocular cameras, stereo cameras, 3D cameras or time-of-flight sensors, known in English as time off light (time of flight), for example lidar sensors or lidar systems. Another sensor technology is radar technology with radar sensors and radar systems. Another sensor technology is based on airborne and / or structure-borne sound. The sensors of this sensor technology are, for example, ultrasonic sensors or microphones. For example, the sensor measures the xyz position of an object, the elevation angle, azimuth angle and distance of the object to a reference point of the sensor, or the velocity vector of the object. The measurements of the individual sensors are output as an object list by a program part, which is processed, for example, by the hardware of the sensor, in this case a system on a chip, SoC (System-on-a-Chip) for short. For example, the program The part is part of the feature extraction algorithm. The sensor system also includes a fusion unit, for example, a computer platform with a processor, such as a CPU and / or multiple GPUs. The computer platform implements a program, such as a fusion algorithm, which generates an object list fused from the object lists of the individual sensors. The sensor system is a real sensor system for road traffic. The output data of the sensor system are real test data, i.e., data collected in field experiments. The sensor model and / or the model of the sensor system are also verified by means of the method according to the present invention. The sensor model describes the sensor in the virtual test environment. The sensor model describes and / or simulates the interaction between the sensor and the virtual test environment.

[0024] The reference data are obtained with the aid of a reference sensor system. The reference sensor system verifies or validates the output data of the sensor system. The reference sensor system is also a sensor system arranged, for example, on a vehicle. Alternatively, the reference data are simulated within a defined parameter space based on a synthetic sensor situation. The reference data are output data from individual sensors of the reference sensor system or fusion results from the reference sensor system. The sensors of the reference sensor system are in particular high-resolution sensors, for example the reference sensor system comprises a Velodyne HDL-64E lidar sensor.

[0025] Deviations include existential errors, such as recognition errors. For example, the sensor system recognizes an object despite the absence of the object in the reference data. Such errors are false positive errors. Or the sensor system does not recognize the object despite the presence of the object in the reference data. Such errors are false negative errors. In addition, deviations include classification errors. For example, the sensor system recognizes a pedestrian, although the object is actually a vehicle. In addition, deviations include measurement errors in continuous and / or discrete measurement variables. For example, the sensor system incorrectly identifies the position of the recognized object by x meters along the direction of travel, or incorrectly measures the width of the road by y meters. These types of deviations may trigger serious errors in the sensor system individually or cumulatively, such as errors that become risks for SOTIF. There is an expected functional safety in the absence of unreasonable risks caused by functional defects of the expected function or by harm caused by human rationally foreseeable misoperation. For example, when the object recognition algorithm recognizes a person riding a skateboard as a person but rejects this recognition as an error due to the incredible speed of the person, there is a SOTIF risk. According to the present invention, these deviations are verified so as to indicate that the approved use of the sensor system is justified by means of an absolute error probability. The method according to the invention therefore also provides a solution for verifying SOTIF. Deviations are evaluated with an evaluation logic. The evaluation logic comprises, for example, an IC, ASIC or FPGA component or a program implemented in the form of a software code segment. The program is implemented, for example, by a data reading device comprising the evaluation logic. The program receives measurement and reference data of the sensors of the sensor system as data input. Deviations are provided as data output.

[0026] The joint probability distribution, also called joint probability, describes the statistical correlation of the deviations of the individual sensors with respect to one another. The joint probability distribution makes it possible to evaluate sensor systems whose deviations are statistically independent. For example, the joint probability distribution is provided with the aid of a modeled probability distribution of the deviations of the individual sensors in the sensor system. For example, the joint probability distribution is composed of The modeled probability distribution approximates the deviations of the individual sensors. The mapping quality of the model in relation to the observed deviations is determined, for example, by means of a so-called goodness of fit (GoF for short) indicator, which is implemented, for example, by computer operation. Examples of GoF indicators are χ2 , Anderson-Darling test, negative log-likelihood, Bayesian information criterion, Akaike information criterion, normalized root mean square error or root mean square error. For example, the normalized root mean square error GoF is defined as follows:

[0027]

[0028] where x(i) is a value of the probability distribution, and x ref (i) is the model value.

[0029] The GoF value tells how well the model represents the data. If the value is bad, the model's statement is not reliable. If the value is good, the model's statement can be trusted. In the GoF, a statement about the model's accuracy is therefore mapped with respect to the choice of model parameters.

[0030] In field experiments, it was found that the probability of a deviation of a single sensor (single sensor error) is higher than the probability of finding errors in multiple sensors at the same time. For this reason, the deviations of the individual sensors are detected in the field and combined with each other to form deviation combinations. For example, when detecting distances, the radar has a deviation with a probability of p%, the camera has a deviation with a probability of q%, and the lidar has a deviation with a probability of r%, and a combination is performed. According to the invention, the deviation types of the individual sensors are also combined, for example, the camera recognizes a false positive event, in which the vehicle is recognized as a pedestrian and the position of the pedestrian is incorrectly recognized by x meters in the direction of travel. By sampling deviation combinations and calculating their probability of occurrence, critical areas in the error space of the combination are evaluated, although these deviation combinations have only been observed individually but never simultaneously in the field experiment. Compared with the data from the field experiment, the sampling of deviation combinations can produce combinations that occur very rarely in the field experiment, sometimes up to one error per billions of operating hours.

[0031] The deviation combinations are sampled, for example, in such a way that the deviations of the individual sensors are sampled stepwise. According to one aspect of the invention, each sensor is sampled at a more dense interval near the mean of the respective deviations than at positions further away from the mean. According to another aspect of the invention, a uniform or Markov chain Monte Carlo method / algorithm is used in order to extract random samples from the deviations. For example, such sampling is selected in such a way that the evaluation is optimal in terms of the cumulative error range.

[0032] The sampled deviation combination is loaded onto the reference data, for example, by processing the reference data in the form of an object list as a vector. The deviation combination is represented as an error vector. The vector belonging to the reference data and the error vector are added.

[0033] The fusion unit, also called high-level object fusion (abbreviated HOF), implements a fusion algorithm. For example, the fusion unit receives low-level features from the individual sensors. For example, the low-level features are points or edges. The fusion unit generates high-level features from these low-level features, for example indicating semantically relevant areas of an object. Each individual sensor can include its own fusion unit, which is used to generate sensor-specific high-level features. The fusion unit of the sensor system then fuses the high-level features of the individual sensors. The rationality of the individual sensors is thereby checked. The fusion algorithm includes, for example, instructions that cause the fusion unit to fuse the average value of the reference data of the individual sensors. According to one aspect of the invention, the fusion unit also evaluates the temporal correlation of the fused input.

[0034] The remaining probability distribution is obtained in such a way that a description is generated according to the requirements (also called necessary conditions) for the fusion result. For example, the requirements are "the mislocalization of the perceived object can only be ±1% of the distance" or "the object cannot be recognized for more than x cycles". The description is "the requirement is met" or "the requirement is not met". If the method according to the invention is computer-run, these requirements and descriptions are implemented, for example, in instructions for conditional statements and branches, such as if / else structures. The joint probability distribution is filtered with the help of these requirements and conditions.

[0035] The remaining probability density is integrated using known numerical methods executed by a computer, for example by trapezoidal numerical integration:

[0036]

[0037] Confidence intervals are determined for the absolute error probabilities obtained in this way. According to one aspect of the invention, the confidence intervals are also limited to the distribution parameters of the individual sensor characteristics. The confidence level is then propagated into the overall statement. When the confidence level is high, the model parameters can be reliably identified from the individual deviations. If the model parameters cannot be clearly identified from the deviations, the confidence level is low. The convergence of the error probability is indicated by the confidence interval. The convergence of the error probability indicates how many more test kilometers are required so that the error probability lies within a certain confidence interval. The use of the sensor system is authorized based on the convergence of the confidence interval and / or the error probability. Confidence levels are necessary for statements about the performance and residual risks of the sensor system, which otherwise are unreliable.

[0038] The residual probability density corresponds to the accumulation of the error range of the sensor system. Using the absolute error probability, a statement about the residual risk attached to the system is generated.

[0039] In the offline case, this statement is then used to legitimize the test base, for example regarding the data and / or scenarios used. A statement is, for example, that the absolute error probability is low. Since the 2 billion kilometers or hours of equivalent required in the current literature do not necessarily have to be driven due to the observation of a single error pattern, the legitimacy of a smaller test base can make the approved use of autonomous vehicle technology technically possible.

[0040] In the online situation, i.e. during actual driving of the vehicle, the assessment of the residual risk can be used to minimize the risk of the current system state. This is done by feeding the risk statement into the driving strategy or safety strategy.

[0041] According to one aspect of the invention, a small amount of data (i.e. output data of the sensor, reference data (real or synthetic, respectively)) is introduced into the method at the beginning of the method and a first statement is created. In addition, a statistically valid extrapolation is performed. The experimental design can therefore be estimated by defining a target confidence level. Therefore, a large amount of cost is saved in the development of autonomous vehicles. Statements about the performance of the sensor system can only be statistically guaranteed after a sufficient distance or time has been actually driven. The method provides a target confidence level. Extrapolation provides an estimate in advance. If the method is fed with new data from field experiments in an incremental (or continuous) manner, matching calculations can be performed separately. On the one hand, this makes planning more accurate because the extrapolation is matched on a larger data basis; on the other hand, due to the current confidence level, the statements about the system become more statistically convincing. If convergence of the model cannot be observed, it means that the modeling is insufficient or the choice of approximation is insufficient. In such cases, the method is adjusted and the calculation is repeated.

[0042] The evaluation of the data is influenced by the inaccuracy of the reference used for the analysis. In order to take this fact into account, according to one aspect of the invention, this inaccuracy is calculated or estimated for all reference data and assigned to the reference data. The maximum accuracy to be achieved for the overall mapping of the sensor system is determined with the aid of this inaccuracy. For example, the reference sensor system is inaccurate, but the reference data of the reference sensor system correspond exactly to the reference data of the sensor system. This gives rise to an impression of accuracy that does not exist in this case. The maximum accuracy is limited by determining the inaccuracy of the reference sensor system, and in the example no high accuracy is calculated.

[0043] According to one design of the present invention, the deviation between the output data of the sensor system and the reference data includes a deviation in distance measurement, a deviation related to the weather or a deviation related to the vehicle state. For example, the deviation of each sensor of the sensor system with respect to the distance is taken into account. These deviations are taken into account in different scenarios, such as "braking of a target object" or "stopping and continuing of the autonomous vehicle and / or other vehicles". The residual risk that depends on the presence and intensity of precipitation, such as rain or snow, is also obtained through the deviation related to the weather. The deviation related to the vehicle state means, for example, that the deviation is evaluated in dependence on the vehicle speed in order to obtain a residual risk related to the respective vehicle speed.

[0044] According to another design solution of the present invention, a probability distribution of the deviation of each sensor of the sensor system is provided, and the deviation combination is sampled with the help of the respective probability distributions of the sensors. The probability distribution of the deviation of each sensor (also called the deviation probability function) is obtained, for example, by clustering the distance measurement deviation between the sensor and the reference sensor in an interval with an interval width of 1m. In other words, a two-dimensional histogram is obtained. The histogram is based on the fact that there are clusters of 1m interval length in the distance from the object to the autonomous vehicle. In each of these clusters, there is a histogram of the deviation between the reference and the sensor. As a result, a histogram is obtained. This is used to model the dependence of the distance. These data in the cluster are used to fit the probability distribution of the deviations of these sensors. Parametric or non-parametric distributions are used for fitting.

[0045] For example, let’s use the normal distribution. The normal distribution is a parametric distribution with two parameters, the mean μ and the standard deviation σ:

[0046]

[0047] According to another aspect of the invention, a non-parametric kernel distribution is used with a kernel function K and a bandwidth parameter h:

[0048]

[0049] The kernel function K is a non-negative, normalizable, symmetric or asymmetric function. Asymmetric functions enable systematic shifts in the error space in the direction of positive deviations. For example, the kernel function K is a Gaussian distribution or a Beta distribution. For example, the optimal bandwidth parameter h is determined by an AMISE algorithm or by a cross-validation method. Multidimensional kernel functions are used for multidimensional applications.

[0050] For example, the fitting of the distribution is performed by means of a computer-run statistical bootstrapping according to an aspect of the invention. The bootstrapping selects a finite number of random samples from a data set and analyzes the random samples at each draw, for example, the mean μ and standard deviation σ.

[0051] According to one aspect of the invention, artificial data about the deviations between output data of sensors of the sensor system and reference data are generated with the aid of a probability distribution of the deviations of the individual sensors. According to one aspect of the invention, the artificially generated data are added to the real data of the deviations and / or the deviation combinations in predefined steps. According to one aspect of the invention, the method is performed incrementally with respect to the amount of data available, wherein the artificial data are incrementally added to the existing data. Thus, the trends of the method are analyzed, such as how the convergence develops. Furthermore, it is analyzed how the fit changes depending on the increase in the amount of data. According to one aspect of the invention, the artificial data are used for extrapolation considerations.

[0052] According to another aspect of the invention, the probability distribution of the deviations of the individual sensors, the resulting joint probability distribution and their influence, such as the absolute error probability, are calculated offline on a computer or online in the vehicle. For the online calculation, the method according to the invention is implemented, for example, in a software code segment implemented by the hardware of the vehicle control device.

[0053] According to another aspect of the invention, the joint probability distribution is provided as a multidimensional normal distribution of the deviations of the individual sensors of the sensor system. The multidimensional normal distribution is:

[0054]

[0055] Wherein, ∑ is the covariance matrix of the output data of the sensors of the sensor system. The covariance matrix is ​​a generalization of the variance of a one-dimensional random variable.

[0056] According to another embodiment of the present invention, output data of the sensor system and reference data are obtained during vehicle driving. The reference data includes reference data collected by a reference sensor system and / or reference data generated by means of a sensor model. For example, the vehicle driving is performed on a specially provided test track.

[0057] According to another aspect of the present invention, the loaded reference data is processed in a model-in-loop, software-in-loop or hardware-in-loop simulation or by means of a substitute model. Thus, the critical state in terms of identification error is evaluated, and the system response of the sensor system is mapped to the deviation combination. In the model-in-loop simulation, the embedded system is simulated in a loop together with the environment model. For example, the model-in-loop simulation is performed in the MATLAB development environment. In the hardware-in-loop, the embedded system (such as a sensor system or a fusion unit) is connected to a matching counterpart via its input and output, i.e., the so-called HiL simulator. The HiL simulator simulates the real environment of the system. Compared with hardware-in-loop, no special hardware is used in the software-in-loop method. The created model of the software is only converted into a code that can be understood by the target hardware. For example, this code is implemented on the development machine together with the simulated model, rather than on the target hardware as in the case of hardware-in-loop. The upper concept of model-in-loop, software-in-loop or hardware-in-loop is the XiL method. Therefore, the system response of the sensor system to the deviation combination is determined. In order to determine the system response, a mathematical substitute model (i.e., the so-called proxy model) is also used to realize online real-time calculation. In order to be able to calculate the influence of the loaded reference data in real time, a learning surrogate model of the system response can be created from the data. The learning surrogate model can then be used instead of XiL to check the influence of errors. In addition, it is possible to perform system optimization in a simplified manner based on this surrogate model, because the response becomes reversible.

[0058] According to another aspect of the invention, Bayesian reasoning is used to determine the system response, which enables online and real-time reasoning. Alternatively, the loaded reference data is processed by means of a special control device with GPU support for calculating the system response. In addition, extremely simplified models (such as N-out-of-K models) that only describe the system response in a very rudimentary way are possible. Such models are particularly useful in concept pre-development and system optimization in the early stages of a product.

[0059] According to another aspect of the invention, the fusion result is obtained by using the redundancy of the sensor system. The redundancy leads to a further reduction in the number of kilometers of protection. If the sensor system includes three sensors, the fusion result is formed by the remaining sensors when one sensor fails. If two sensors fail, no fusion result is generated.

[0060] According to another design of the present invention, a limit state function is used to evaluate the deviation between the fusion result and the reference data in order to remove the occurrence probability from the probability distribution. The limit state function (which is also called the limit state function) provides a first value when the requirement is met and a second value different from the first value when the requirement is not met. For example, the limit state function provides a value of 0 when the requirement is met and a value of 1 when it is not met. According to another aspect of the present invention, the limit state function is multidimensional. Therefore, the entire statistical error range of the sensor system is evaluated in terms of residual risk with the help of the limit state function.

[0061] According to another aspect of the present invention, the amount of initial data and reference data increases incrementally, whereby the calculations can be matched step by step.

[0062] The method and apparatus for carrying out the described method are summarized for clarity:

[0063] The method and the device are used for an autonomous vehicle with data recording. The data recording is performed by a sensor system on the one hand; on the other hand, sufficiently accurate reference data must be recorded in parallel with a reference sensor system for recording the data. The reference data are correlated in time and space. This reference can be recorded with the aid of a system that is also fixed to the vehicle or with the aid of an external reference system.

[0064] The data, i.e. the output data of the sensor system, are associated with a reference in a storage unit, such as a database, a cloud storage or a memory card. A cloud / computing cluster-based embodiment of the invention enables the processing of very large amounts of data. According to one aspect of the invention, the invention is implemented on a computing cluster. Deviations between the output of the autonomous system and the reference are evaluated with the aid of an evaluation logic, more precisely at the level of the entire system, i.e. with the aid of a fused object list, and at the level of a single sensor, i.e. at the level of an object list, detection, feature, or algorithm sub-element for generating an object list, detection, feature, from one or more sensors of the same technology type.

[0065] The statistical distribution of the various aspects of the environment recognition system is then analyzed and mapped mathematically, numerically and technically with the aid of curve fitting. As a result, a probability distribution of the deviations of the individual sensors of the sensor system is obtained. The mapping also includes the occurrence correlations between the individual phenomena. The individual deviations are then combined with each other in space and / or time to form an error graph. This analysis leads to a combination of deviations.

[0066] The error graph is evaluated for critical states with respect to the recognition error. For this purpose, methods and systems of different technical aspects are provided to map the system response onto the error graph. For this purpose, perfect input data are loaded with error vectors in time and space, and the system response is determined with the help of a suitable remaining system, i.e. the perception software or the control device. According to another aspect of the invention, critical states with respect to the system reaction are evaluated, for example when the trajectory becomes incorrect, for example when the trajectory crosses a pedestrian. In this case, the remaining system is, for example, a perception system or a behavior software. The selection of the method depends on the field of application, for example, offline or online. The invention comprises, for example:

[0067] • Software-in-the-loop or hardware-in-the-loop, where the actual environment is identified with the software in order to map the true impact of the error to the system. This is preferably applied offline.

[0068] Mapping models, also called proxy models, in which the system response is numerically approximated mathematically. This is preferably applied online. Bayesian reasoning is also conceivable here, with which reasoning can be performed online and in real time. The invention also includes processing by a special control device with GPU support, on which the system response is calculated.

[0069] The present invention also includes extremely simplified models, such as N-out-of-K models that only describe the system response in a very rudimentary way. These are particularly helpful for concept pre-development and system optimization in the early stages of a product.

[0070] The method calculates the support costs initially and iteratively in a coordinated manner. This improves the planning of ongoing operation and data recording.

[0071] By analyzing the error graph and its effect on the sensor system, the impact of individual error sources is analyzed more effectively. Therefore, the system can be designed to meet the requirements in a more targeted and cost-effective manner.

[0072] When replacing a sensor in a sensor system, the method and device are used to evaluate the impact on system reliability. If the data is statistically similar or better and the impact of the individual deviations of the sensor is observed not to lead to sensor system degradation and residual risk, the sensor can be replaced.

[0073] By inversely analyzing the error patterns (i.e. the deviation combinations), the maximum error contribution of the sensor system can be determined in a targeted manner with the aid of the method and the device, and the sensor system can be improved accordingly. The development work is thus more effective and potentially more cost-effective. Compared to a complete numerical analysis of the sensitivity of the improvement potential, the elements of the system that have the greatest influence on the improvement can be pointed out here.

[0074] An estimate of the current reliability is made possible by online calculations. This requires very efficient calculations, since statements about the reliability must be available in real time. For example, using methods of Bayesian reasoning. This leads to an increase in technical safety.

[0075] According to one aspect of the invention, the method according to the invention is used to verify and evaluate the subsystems of an autonomous vehicle in order to model the characterization of the identification system. Thus, the impact of possible errors and their probabilities are used to evaluate the residual risk of the autonomous vehicle. This use is very similar to the sensor model used for virtual assurance.

[0076] In order to improve the efficiency of data transmission during calculation, the method is already executed when new data are introduced into the database. It is thus possible that at least the first access to the data is no longer possible by re-accessing the data. When recalculating the data, a normal read access must be performed again.

[0077] The subject matter of the invention makes it possible to approve the use of a sensor system of SAE J3016 level 3+ by reducing the number of guaranteed kilometers, by taking into account the combination of deviations when calculating the probability of failure and, according to one aspect of the invention, by using redundancy of at least parts of the sensor system. Furthermore, the subject matter of the invention makes it possible to efficiently optimize the sensor system during the development process. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] The exemplary embodiments shown in the drawings illustrate the invention.

[0079] Figure 1 : shows the camera data cluster with respect to distance,

[0080] Figure 2 : Show Figure 1 Histogram of the distribution of deviations of the camera data in ,

[0081] Figure 3 : Show Figure 2 The bootstrap distribution of the mean of the distribution of camera data in ,

[0082] Figure 4 : Show Figure 2 The bootstrap distribution of the standard deviation of the distribution of camera data in ,

[0083] Figure 5 : Show Figure 2 The fitted probability distribution of the deviations of the camera data in has the form Figure 3 The confidence interval limits for the mean and the Figure 4 The standard deviation of

[0084] Figure 6: a histogram showing the distribution of deviations of the additional camera data and the fitted probability distribution of these deviations,

[0085] Figure 7 : a histogram showing the distribution of deviations of the radar data and the fitted probability distribution of these deviations,

[0086] Figure 8 : a histogram showing the distribution of deviations of the lidar data and the fitted probability distribution of these deviations,

[0087] Fig. 9 : shows the joint probability distribution of the deviations of the camera data and the radar data for a predetermined deviation of the lidar data,

[0088] Fig.10 : Show Fig. 9 The remaining probability distribution of the joint probability distribution in is,

[0089] Fig.11 : shows the residual probability distribution integrated by means of trapezoidal numerical integration, where the width of the confidence interval is indicated by a box,

[0090] Fig.12 : shows the absolute error probability of the fit with lower and upper confidence levels,

[0091] Fig.13 : shows an off-line embodiment of the method according to the present invention,

[0092] Fig.14 : shows an off-line embodiment of the method according to the present invention,

[0093] Fig.15 : shows the technical structure of an embodiment of the device according to the present invention, and

[0094] Fig.16 : shows an embodiment of a vehicle according to the present invention.

[0095] In the drawings, the same reference numerals denote the same or functionally similar parts. For the sake of clarity, reference parts that are important for the respective understanding are described in the corresponding drawings. The measurement units of distances and deviations in all drawings are meters. DETAILED DESCRIPTION

[0096] Figure 1 The diagram in shows the deviation a of the camera sensor over the distance d. The camera is a sensor 11 of the sensor system 10, see also Fig.16The deviation a results from a comparison of the camera data of the collected measured values ​​with reference data of a reference sensor system 20. The reference sensor system comprises a high-resolution sensor 21, for example a high-resolution camera. The reference data comprises, for example, measurement data of the high-resolution camera. The camera data and the reference data are stored in a database, for example in a cloud storage or in a data farm. An evaluation logic 13 - for example a computer, a processor core or a chip with logic components - forms, for example, a difference between the camera data and the reference data and outputs it as a deviation a, see also Fig.13 The camera's errors are therefore recorded.

[0097] According to the invention, the deviation a is similarly obtained for other sensors 11 of the sensor system 10, such as radar and / or lidar. For example, the deviation a for lidar data is synthetically generated. For this purpose, the data table of the lidar sensor is used as reference data.

[0098] According to the invention, deviation a between sensor system 10 and reference system 20 is also taken into account for different precipitation intensities or different vehicle speeds.

[0099] The data of the sensor system 10 include, for example, 50 collected scenarios, such as “stop and go”, which have approximately 3 million data points.

[0100] according to Figure 1 The deviation a shown produces Figure 2 The histogram shown in . The histogram represents the occurrence probability p of each deviation a of the camera. Such a histogram is obtained similarly for the other sensors 11 of the sensor system.

[0101] From Figure 3 and Figure 4 The bootstrapping of the deviations a of the camera data illustrated in yields the mean and standard deviation of these deviations a. A similar bootstrapping is performed for the other sensors 11 of the sensor system 10. In the bootstrapping, n random samples are selected and the mean and standard deviation of each random sample are calculated. This results in Figure 3 The distribution of the mean values ​​shown in Figure 4 The standard deviation of the distribution is shown in .

[0102] With the confidence interval bounds for the mean obtained in this way (see Figure 3 ) and the confidence interval bounds for the standard deviation (see Figure 4 ) is calculated using the probability distribution approximation Figure 2 The resulting probability distribution for the deviation a of the camera data is Figure 5 Shown in.

[0103] Figure 6shows another probability distribution of the deviation a for another camera data, Figure 7 shows the probability distribution of the deviation a of the radar data, and Figure 8 The probability distribution of the deviation a of the lidar data is shown. Figure 8 Synthetic bias based on lidar data a, the basis of the synthetic bias is to use the data sheet of the lidar sensor as a reference.

[0104] exist Fig. 9 The joint probability distribution shown in is a multidimensional probability distribution obtained from the individual probability distributions of the deviations a of the camera, radar, and lidar data. Fig. 9 In , the x-axis represents the deviation a of the radar data, the y-axis represents the deviation a of the camera data, and the z-axis represents the joint probability distribution. The joint probability distribution is shown for a fixed deviation a of the lidar data. For each deviation a of the lidar data, according to Fig. 9 According to the invention, the joint probability distribution is provided in method step V1.

[0105] In method step V2, deviation combinations are sampled and the occurrence probability p of the sampled deviation combinations is calculated with the aid of the joint probability distribution. Thus, critical error combinations are generated. For example, the error combinations are simulated in a simulation and / or post-processing environment. The error influence is then evaluated with the aid of the joint probability distribution.

[0106] In a third method step, the sampled deviation combination is loaded onto the reference data. The loaded reference data is processed by the fusion unit 12 of the sensor system 10 and a fusion result is obtained.

[0107] In the fourth method step V4, the fusion result is evaluated using a limit state function in order to remove the occurrence probability p of those deviation combinations that lead to fusion results that meet the predetermined conditions from the joint probability distribution. Fig.10 The residual probability distribution shown in .

[0108] In a fifth method step V5, the remaining probability distribution is integrated and the absolute error probability P is obtained, see Fig.11 and Fig.12 For example, the absolute error probability P can be approximately calculated by using trapezoidal numerical integration.

[0109] In a sixth method step V6, the sensor system 10 is approved for use in a test and approval process, depending on the absolute error probability P. The absolute error probability P represents the legal basis for the test basis, i.e. the legal basis for the output data, reference data and scenarios used by the sensor system 10. Since the observation of individual error graphs does not require the actual travel of the equivalent of 2 billion kilometers or hours required in the literature, for example, of the vehicle 1 and the sensor system 10 and the reference sensor system 20 installed thereon, the approval for use of the autonomous vehicle technology can be achieved with the aid of a small test basis with an absolute error probability P.

[0110] Fig.13 The overview diagram illustrates the subject matter of the invention in an offline variant. An autonomous vehicle 1 is, for example, Fig.16 The mathematical stochastic model comprises the probability distributions of the individual sensors 11 of the sensor system 10 and the resulting multidimensional joint probability distribution. The method according to the invention is carried out with the aid of a processing unit 30. Method steps V2 and V3 are carried out in a simulation and post-processing environment which can be part of the processing unit. The simulation environment is, for example, software such as CarMaker. A statement about the residual risk is obtained with the aid of method steps V4 and V6.

[0111] Fig.14 The overview map illustrates the subject matter of the invention in an online variant. Fig.13 In contrast, the absolute error probability P is continuously evaluated and fed back into the driving strategy of the vehicle 1 .

[0112] Fig.15 The figure shows the technical structure of the processing unit 30 of the device according to the present invention. The core of the method performed by the processing unit 30 is to obtain the probability distribution of the deviations of the various sensors 11 (see Figure 6 , Figure 7 and Figure 8 ), that is, obtaining the joint probability distribution derived therefrom, sampling the deviation combination, and obtaining the absolute error probability P. The data input to the processing unit 30 includes the output data of the sensor system 10 and the reference data of the reference sensor system 20. The data output of the processing unit includes the absolute error probability P.

[0113] Fig.16 The passenger vehicle in FIG. 1 is an example of a vehicle 1 according to the present invention. The vehicle 1 comprises a sensor system 10 having a plurality of sensors 11. Fig.16 Only one of the sensors 11 is shown in FIG. 1 . The output data of the individual sensors 11 are fused in a fusion unit 12 of the sensor system 10 . The vehicle 1 further comprises a reference sensor system 20 having a plurality of reference sensors 21 . Fig.16Only one of the reference sensors 21 is shown in FIG. The output data of the individual sensors 21 are fused in a fusion unit 22 of the reference sensor system 20 .

[0114] As a fully electric minibus, the passenger vehicle provides comfortable space for the passengers. At the same time, its floor space is no larger than that of a large luxury car. It is therefore agile and suitable for use in urban environments. A passenger vehicle that travels in a locally emission-free manner offers various connectivity services and the possibility of highly automated driving. According to one aspect of the invention, a vehicle 1, such as a passenger vehicle, comprises a connectivity interface. The user can flexibly request the passenger vehicle as a shuttle via the connectivity interface via an application and is therefore no longer bound by rigid driving schedules.

[0115] List of reference numbers

[0116] 1 Vehicle

[0117] 10 Sensor System

[0118] 11 Sensors

[0119] 12 Fusion Unit

[0120] 13 Evaluation Logic

[0121] 14 Database

[0122] 20 Reference System

[0123] 21 Sensors

[0124] 22 Fusion Unit

[0125] 30 processing units

[0126] V1-V6 Method Steps

[0127] p probability of occurrence

[0128] P absolute error probability

[0129] d Distance

[0130] a deviation

Claims

1. A method for authorizing the use of a sensor system (10) for detecting objects in the environment of a vehicle (1), the method comprising the following steps: providing a joint probability distribution (V1) for deviations (a) between output data of the sensor system (10) and reference data at the level of a program part of the sensor system (10) for detecting an object, at the level of a sensor (11) of the sensor system (10) and / or at the level of a fusion of the sensor system (10), · sampling a deviation combination and calculating the occurrence probability (p) of the sampled deviation combination with the aid of the joint probability distribution (V2), Loading the sampled deviation combination to reference data, processing the loaded reference data by means of a fusion unit (12) of the sensor system (10) and obtaining a fusion result (V3), removing the occurrence probabilities (p) of those deviation combinations that lead to fusion results that meet predetermined conditions from the joint probability distribution, and obtaining a residual probability distribution (V4), Integrate the residual probability distribution and obtain the absolute error probability (P) (V5), and Authorizing the use of the sensor system (10) in dependence on the absolute error probability (P) (V6).

2. The method according to claim 1, wherein: The deviation between the output data of the sensor system (10) and the reference data includes a distance measurement deviation, a weather-related deviation or a vehicle status-related deviation.

3. The method according to claim 1 or 2, wherein: A probability distribution of the deviation of the sensor (11) is provided for each sensor (11) of the sensor system (10), and the deviation combinations are sampled with the aid of the respective probability distribution of the sensor (11).

4. The method according to any one of claims 1 to 2, wherein: The joint probability distribution is provided as a multidimensional normal distribution of the deviations of the individual sensors (11) of the sensor system (10).

5. The method according to any one of claims 1 to 2, wherein: Output data of the sensor system (10) and the reference data are acquired during driving of the vehicle (1), wherein the reference data include reference data collected by a reference sensor system (20) and / or reference data generated with the aid of a sensor model.

6. The method according to any one of claims 1 to 2, wherein: The loaded reference data are processed in a model-in-the-loop, software-in-the-loop or hardware-in-the-loop simulation or by means of a surrogate model.

7. The method according to any one of claims 1 to 2, wherein: A fusion result is obtained by using the redundancy of the sensor system (10).

8. The method according to any one of claims 1 to 2, wherein: The deviation between the fusion result and the reference data is evaluated using a limit state function to remove the occurrence probability (p) from the probability distribution.

9. The method according to any one of claims 1 to 2, wherein: The amounts of the output data and the reference data increase incrementally.

10. Device for authorizing the use of a sensor system (10) for detecting objects in the environment of a vehicle (1), the device comprising at least one processing unit (30) adapted to implement the steps of the method according to any one of claims 1 to 9.

11. A sensor system (10) for detecting objects in the environment of a vehicle (1), comprising a plurality of sensors (11) of one sensor technology or at least one sensor (11) of each sensor technology and a device according to claim 10.

12. Computer program for authorizing the use of a sensor system (10) for detecting objects in the environment of a vehicle (1), the computer program comprising instructions which, when the computer program is executed by a processing unit (30), lead to the execution of the steps of the method according to any one of claims 1 to 9.

13. A vehicle (1), comprising: at least one sensor system (10) according to claim 11, at least one control device and an actuator, wherein the sensor system (10), the control device and the actuator are connected in a signal-transmitting manner such that the vehicle (1) is at least conditionally autonomously driven, an interface to at least one reference sensor system (20), and · The device according to claim 10, wherein, during the driving of the vehicle (1), an absolute error probability (P) is obtained according to the steps of the method according to any one of claims 1 to 9, and the absolute error probability is fed back into the driving strategy of the vehicle (1).