Method for validating an application of a navigation module

Abstract

The invention relates to a method for clearing an application of a navigation module (1) for a security level, wherein the navigation module (1) has at least one GNSS module (2) for determining navigation data as input data for a navigation process in a motor vehicle (3), and the method has the following steps: a) receiving application data relating to the application; b) receiving test data relating to the application, said test data having been determined during tests using the vehicles (3) which comprise the applied navigation module (1); c) creating a large number of test scenarios (4) for the application at least on the basis of the application data and the test data using a Monte Carlo method; d) carrying out tests on the application on the basis of the test scenarios (4) created in step c); and e) giving clearance to the application at least on the basis of the results of the tests carried out according to step e).

Description

[0001] Procedure for validating a navigation module application

[0002] The invention particularly relates to a method for verifying the specified desired integrity risk of a localization created with the navigation module in an application of a navigation module.

[0003] The invention relates to the validation and verification of the quality of high-precision navigation modules for vehicles (also called vehicle localization systems), which are intended, for example, to determine the position of vehicles with an accuracy in the range of centimeters.

[0004] To operate such navigation modules, the fusion of data from various sensors is required. This is the only way to achieve the desired high precision of localization (also known as position determination). Sensor fusion algorithms must be regularly redeveloped and adapted for the specific application of navigation modules in specific vehicle types, because even minor differences in the sensors whose data are to be fused, as well as the precise environment of the individual sensors on the respective vehicle type, can have a significant impact on localization and its accuracy.

[0005] Such navigation modules are intended, for example, for automated driving applications or advanced driver assistance systems, such as the activation / deactivation of automated driving functions, safe stopping maneuvers, or lane keeping for SAE Level 3 and beyond. High-precision navigation modules typically include a GNSS (Global Navigation Satellite System) receiver, a high-performance inertial measurement unit (IMU), and wheel speed sensors (WSS) to meet performance, availability, and safety requirements. Furthermore, such navigation modules process GNSS correction data and integrity information from a global GNSS reference station network. Such high-precision navigation modules can typically only determine high-precision localizations in conjunction with other connected sensors in the vehicle.Such high-precision navigation modules are therefore often not clearly distinguishable from other components in the vehicle in their applications.

[0006] Such navigation modules regularly require verification for their application in accordance with ISO 21448. ISO 21448 is a standard for the assurance of intended functionality (SOTIF) of road vehicles. According to this standard, it must be demonstrated how secure the navigation module is against undetectedly performing an incorrect localization. In this context, a localization is classified as incorrect if the position error exceeds a specified limit.

[0007] Key SOTIF-relevant signals are the fused position and a corresponding protection level (PL). This protection level can be represented in the form of an ellipse around the specific position. The protection level indicates the probability that an actual position lies outside the described ellipse. The higher the protection level, the lower this probability. The elliptical shape of the protection level is usually due to the fact that a somewhat higher localization uncertainty occurs in a direction of movement of a vehicle (usually along a route), and is usually permissible. The protection level therefore defines a statistical upper error limit for the fused localization solution and thus reflects the localization uncertainty.The aim of the method described here is to demonstrate the desired integrity risk, which can then be used by the user of the localization solution to carry out a risk analysis of systems that use the localization solution.

[0008] The goal of a SOTIF classification is to enable users of the localization solution (such as autonomous or highly automated driving systems in motor vehicles) to easily identify and classify the security of the localization solution based on the SOTIF classification, and to enable risk analyses to be created for systems using the localization solution using the SOTIF classification. The following parameters, among others, are used for the specification: • Alert Limit (AL = Alert Limit): The "Alert Limit" defines the position error tolerance that must not be exceeded without signaling a loss of integrity at the navigation module output.

[0009] • Time to alert (TTA): The “Time to Alert” defines the maximum permissible time that may elapse between the occurrence of a loss of integrity and its signaling, e.g. via the integrity validity indicator.

[0010] • Target Integrity Risk (TIR): The "Target Integrity Risk" is expressed as a probability (dimensionless) or as a probability per unit of time (usually per hour) and indicates the probability that the localization error of the localization solution will exceed the alert limit without signaling the loss of integrity within the "Time to Alert." The "Target Integrity Risk" describes the target rate and is usually specified by the customer. A corresponding probability is also referred to as the HMI rate (Hazardously Misleading Information). The HMI rate indicates the probability that an incorrect localization solution will be issued.

[0011] • Safety or integrity availability: Safety availability is the percentage of time during which the system's services are available to the user, meaning, in the navigation module context, that safe and reliable localization is possible. It is a function of both the physical properties of the environment and the technical capabilities of the navigation module system.

[0012] The specification of these parameters depends on many SOTIF-related conditions, constraints, and limitations, such as the defined operational design domain (ODD), the customer's use case, and the vehicle functionality for which the localization solution is to be used. For highway applications, an example of a parameter set to be defined for AD / ADAS functions could be a TIR at the 10' level. 4 / h to 10- 6 / h, AL of 3-5m and TTA of 5-1 Os. Such a parameter set defines the verification to be provided. Such a parameter set must be strictly adhered to if a navigation module is to be classified as "safe" for a specific application. This parameter set forms the basis for all further activities, such as the analysis of the SOTIF trigger conditions and the development of a SOTIF concept and a SOTIF strategy to demonstrate the safety of the intended functionality. Whether such safety of the provided localization solutions exists must be demonstrated for each application (e.g., for each vehicle type).

[0013] Building on this, the object of the invention presented here is to present an efficient validation strategy with which appropriate proof can be provided efficiently so that a security level can be demonstrated for a specific application.

[0014] Described here is a method for validating an application of a navigation module for a safety level, wherein the navigation module has at least one GNSS module for determining navigation data as input data for navigation in a motor vehicle, and the method has the following steps: a) receiving application data relating to the application; b) receiving test data relating to the application that was determined during tests with the motor vehicle samples comprising the applied navigation module; c) creating a large number of test scenarios for the application at least based on the application data and the test data using a Monte Carlo method; d) performing tests of the application based on the test scenarios created in step c); and e) granting approval for the application at least based on results of the tests performed according to step e).

[0015] The procedure is particularly advantageous if the approval granted in accordance with step e) is a SOTIF approval.

[0016] The release is specifically a SOTIF release, the verification of which must be provided according to ISO 21488. However, the procedure can also be used for other releases and / or validations of navigation modules according to other standards. The term "validation" here refers specifically to the process used to determine the release. The procedure serves, in particular, to provide evidence of the defined desired integrity risk to the protected component (the localization / position to be determined with the navigation module) for a specific application of a navigation module.

[0017] It is particularly preferred if the release granted in step e) specifies an upper limit for the probability of at least one undetected error occurring in navigation data determined with the navigation module in the respective release.

[0018] Based on customer requirements, the specific use case, the system design, experience, or even historical events (i.e., incidents observed in the past, for example, with other navigation systems or comparable products), hazardous events and scenarios are identified and compiled in a validation catalog. The validation catalog lists all tests that must be performed to obtain the SOTIF classification.

[0019] The validation catalog is the starting point for SOTIF validation, which forms the basis for the SOTIF classification. The validation catalog is preferably not static and not fixed for a specific navigation module. Rather, such a validation catalog is a living document that continues to be adapted and improved even after its initial use to classify specific applications of a particular navigation module.

[0020] Here, it is proposed to create the validation catalog based on application data determined for a specific application of the navigation module in a specific vehicle type, vehicle model series, etc. (Step a)). The term "application" here refers to the configuration of the navigation module for a specific vehicle type or vehicle model series.

[0021] Each case or situation that must be tested according to the validation catalog for the assignment of a specific SOTIF classification is preferentially classified in the validation catalog based on the probability of occurrence in the field. The validation catalog generally includes more probable and exceptional cases or situations. In step b), test data relating to the respective application, obtained from tests with vehicle samples, is received. Vehicle samples here are vehicles configured according to the respective application—in particular, the first examples of the respective vehicle type or vehicle model series.

[0022] Nominal events / scenarios: All effects and behaviors expected during a sufficiently large number of test driving hours are specified as nominal, i.e. systematic effects or periodic errors with an occurrence rate of 10'2 / h or higher, assuming that more than 1,000 hours of statistically representative test drives were evaluated. Such events / scenarios or effects and behaviors are preferentially covered by the test data received in step b). In the context of test drives, the term "representative" means that the driving situations tested during the test drives are sufficiently representative of the normal use of the respective vehicle. 1,000 hours of test drives in which a vehicle drives on a test track are therefore, for example, less representative than test drives conducted in normal road traffic. In general, all events that can be evaluated using test drives are preferably also tested using test drives, since real-life behavior, including interfaces and communication paths, vehicle design, chassis and installation problems, etc., is reflected in the data.

[0023] Which tests were performed during the test drives, which form the basis for the test data received in step b), are preferably already defined in the validation catalog. Therefore, the test drives are preferably not completely random; rather, specific scenarios / effects are deliberately tested during the test drives.

[0024] Extraordinary events / scenarios are, in particular, effects and behaviors that have a probability of occurrence <10' 2 / h for the specified ODD and application. Exceptional events are typically not covered by the given empirical test drive database, or test drives cannot provide sufficient assurance that such exceptional events / scenarios are properly accounted for. In 1,000 hours of test drives, an event or scenario occurs with a probability of <10' 2 / h statistically occurs less than ten times in total. Therefore, only relatively few occurrences of such scenarios can be investigated with test drives if the total number of test drives carried out is limited.

[0025] To test such exceptional events / scenarios, a large number of test scenarios are preferably created for the application in step c), this is done using the Monte Carlo method.

[0026] A suitable validation schema should be defined for each case in the validation catalog. The validation schema preferably includes a validation setup that describes a structure (e.g., also called a test setup) with which the validation is performed. The validation schema can have a suitable structure for each case in the validation catalog and does not have to be uniform. The validation setup can include an embedded model (possibly components of the navigation module) that is tested (MIL = Model-in-the-Loop).

[0027] It is particularly advantageous if, when creating the test scenarios in step c), input data for the navigation module is taken into account, the occurrence of which was determined on the basis of test data received in step b).

[0028] It is also advantageous if input data for the navigation module that was determined in previous tests of the navigation module in other applications and / or in test models is used to create the test scenarios in step c).

[0029] Furthermore, it is advantageous if, in step c), a large number of test scenarios with modified input data are generated using the Monte Carlo method by randomly modifying input data.

[0030] It is also advantageous if the number of test scenarios created in step c) and the number of tests performed in step d) is sufficiently large to ensure a statistical probability that particularly problematic input data for navigation will be tested. Furthermore, it is advantageous if the tests in step d) are carried out at least partially with embedded models (MIL = Model-in-the-Loop) that simulate the applied navigation module.

[0031] Preferably, further tests of the application based on the test scenarios created in step c) are conducted using such an embedded model (MIL = Model-in-the-Loop). The embedded model does not have to be a vehicle with which test drives can be performed. In certain implementation variants, this can also be a model specifically configured for the respective test, which contains software components and, if applicable, hardware components of the navigation module, which are embedded in an environment adapted to the respective application or with which the respective application can be tested.

[0032] For the granting of the respective approval according to step e), preference is given to the test data from test drives (step b)) and to the results of the further tests carried out according to step d).

[0033] As explained above, a validation scheme is defined for the release / validation of the respective application. This scheme serves as the basis for the testing and specifies what is to be checked during the tests. This preferably applies to the test data obtained from test drives (step b)) as well as to the tests conducted according to step d). The validation scheme can also include analytical methods, such as the fault tree analysis (FTA) described here.

[0034] The SOTIF validation strategy described here is based primarily on three so-called validation pillars:

[0035] • A validation via test drives, with which nominal events can be detected and investigated (data received in step b)).

[0036] • A simulation-based validation, which can be carried out, for example, using the so-called Monte Carlo method, and which can support or supplement the validation of both nominal and exceptional events (step d)).

[0037] • A SOTIF fault tree analysis focuses on the analysis of exceptional events.

[0038] SOTIF fault tree analysis differs fundamentally from the other two pillars. Validation with test data and validation using the Monte Carlo method involve real-world investigations, whereas SOTIF fault tree analysis is an analytical approach that relies on correctly identifying the causes of the errors that occur, allowing the probabilities of subsequent errors to be analytically calculated based on this.

[0039] It should also be noted that the three validation pillars are all used (to varying degrees) both to assess known hazardous events and to assess unknown hazardous events and scenarios. A clear separation between the investigation of known hazardous events and unknown hazardous events is not desired and is not implemented. In this context, hazardous events are events that, if undetected, can cause incorrect localization. Known hazardous events are events whose occurrence is known to cause undetected incorrect localization. Unknown hazardous events are events that are not known to cause undetected incorrect localization.

[0040] Some aspects covered in each validation pillar are:

[0041] • Statistical validation of large data sets: In all of the described validation pillars, large amounts of data are ultimately examined to identify possible scenarios / events in which errors may occur.

[0042] • Scenario- or event-oriented validation: All validation pillars aim to determine how frequently certain scenarios or events occur and lead to localization errors that may not be detected. • Validation of sensor performance fluctuations: Sensor performance fluctuations are also considered, at least implicitly, in all of the validation pillars mentioned.

[0043] • Review of the system design: The basic design / structure of the respective navigation module in the respective application is also checked with each validation pillar

[0044] • Sensitivity test: The sensitivity of the navigation module in the respective application or the sensitivity of the individual components is also tested with each validation column

[0045] In principle, the findings obtained with the individual validation pillars overlap, so that the application of all three validation pillars for validation creates a comprehensive overall picture

[0046] In a final validation step, the results are aggregated to an overall HMI rate and safety availability, which is considered in the argumentation for the SOTIF validation of the navigation module in the respective application.

[0047] The next sections describe the three SOTIF validation pillars, including their respective activities, in more detail.

[0048] The test-drive-based component or pillar of SOTIF validation aims to validate and release integrity requirements based on real test drives. The specified target integrity rate (TIR) ​​would theoretically require a much larger amount of data than can be collected during a product development phase in test drives. For example, to achieve a target integrity rate of 10' 5 To empirically validate the sensor performance of 100,000 hours per hour, significantly more than 100,000 hours of test drives would be required to apply common description approaches, where an event should statistically occur at least 10 times during the test period. To overcome this problem, statistical prediction methods can be used to predict sensor behavior based on a limited data set. Three crucial points must be considered:

[0049] • The limited test drive dataset is representative of the sensor's statistical behavior. • The statistical prediction methods are chosen to be compatible with the system behavior.

[0050] • Any remaining subcritical event, i.e. a localization error that appears as a statistical outlier although it does not exceed the alert limit, is thoroughly analyzed and can be classified as safe with regard to the specified integrity requirements.

[0051] A test vehicle equipped with the navigation module under test and a significantly more precise reference system is used to record the data. The driving scenarios and the data volume are defined in advance to ensure diverse and representative test coverage. The data is evaluated by comparing the fused localization solution determined with the navigation module under test with the position determined with the significantly more precise system (also called the "reference truth position"). To ensure the representativeness of the dataset, various aspects must be considered. For example, the integrity-relevant behavior of the test vehicle, and in particular the behavior and accuracy of the fused position, must be close to the target system. This includes components that are not directly part of the navigation module but have an impact on the navigation module's SOTIF approval.Examples include the type and design of the GNSS antenna, the mounting position of the antenna and sensor, the quality of the GNSS correction data, and the software. Furthermore, the ground truth information provided by a reference system (also called "ground truth position") must, as a rule of thumb, be at least ten times more accurate than the accuracy of the system under test.

[0052] In the test drives, priority is given to testing driving scenarios that are known to be particularly critical with regard to the accuracy of the positions determined by the navigation module.

[0053] A database of special scenarios is particularly preferred for defining the driving scenarios for the test drives: Scenarios that include both known problem cases (e.g., aquaplaning, emergency braking) and special vehicle configurations (e.g., driving with a roof box or trailer) are defined and driven in order to take into account conditions that influence various SOTIF-relevant elements of the navigation module, such as the GNSS signal reception behavior.

[0054] Statistical representativeness of the database: To define a statistically representative database, the dataset must contain a representative variation of scenarios, including but not limited to weather, traffic, and environmental conditions, as well as road types and trajectories. Furthermore, depending on the level of the specified target integrity risk and the expected nominal performance of the system, a minimum amount of data is required to ensure sufficient forecast reliability.

[0055] Validation is supplemented by test drives, preferably using statistical forecasts.

[0056] Preferably, the probabilities of certain events determined during test drives are enhanced with statistical forecasts and evaluated. After collecting data from test drives, for example, a test drive structure can be compared with real-world operating scenarios and converted to calculate the frequencies of certain events occurring in the field or in real-world operating scenarios. Extreme value theory, for example, can be used for this purpose. Furthermore, probability density functions (PDFs) can be determined, adjusted, and applied. Adjusted probability density functions can be determined.

[0057] Using this model, so-called subcritical events are identified as statistical outliers that can significantly contribute to increasing the HMI probability. In a final validation step, these events are analyzed in detail to derive a reliable argument for the respective probabilities for the occurrence of undetected incorrect positioning.

[0058] Examples of typical and very dominant error sources that can be identified during test drives and then better evaluated using statistical predictions are the effects of multipath signal propagation, i.e., GNSS signal reflections from environmental obstacles such as buildings, vegetation, or traffic, which lead to signal delays. Large localization errors caused by multipath can be analyzed with regard to the error envelopes of multipath. For example, a GNSS observation error caused by multipath effects is limited by the conditions of the scenario, which are influenced by the existing signal reflectors and the vehicle dynamics. On the other hand, the potential for a measurement error due to multipath to affect the fused solution can be analyzed using simulations.Supported by simulations of comparable scenarios, it can be argued that the error in this or a comparable scenario cannot exceed the safety-critical localization error threshold.

[0059] Priority is given to ensuring that all known relevant and important scenarios are covered during test drives. In particular, the statistical behavior of the recorded data with respect to the TIR is analyzed. Based on this, the expected probability of HMI occurrence can be predicted. Subcritical events that emerge as outliers from this statistical analysis are preferably analyzed in detail to derive a case that these risks do not occur in the field (during real-life use of the navigation module) or only occur to an acceptable extent.

[0060] Model-in-the-Loop (MiL) simulations based on the Monte Carlo (MC) method are preferred for testing the behavior of the navigation module without test drives. The Monte Carlo method allows for extensive parameter variations that can be tested using a model navigation module.

[0061] Applying the Monte Carlo (MC) method to obtain test data for analyzing the navigation module in Model-in-the-Loop (MiL) is particularly useful for certain events / scenarios. Undetected incorrect localizations can occur, for example, in connection with the following events / scenarios:

[0062] • Loss of GNSS and / or correction data

[0063] • Difficult GNSS environment, e.g., poor visibility and multipath effects; or correcting for data quality variations.

[0064] Such events / scenarios are regularly tested during test drives. Various input parameters used by the navigation module to generate the localization are subject to certain variations in such events / scenarios.

[0065] Using Model-in-the-Loop (MiL), localization can be tested using the navigation module based on a specific set of parameters.

[0066] By varying parameters using the Monte Carlo method, Model-in-the-Loop (MiL) can then be used to create and test extensive variations of parameter combinations that occurred in test drives.

[0067] In particular, worst-case scenarios (particularly unfavorable parameter combinations) can be identified, which can be investigated using Model-in-the-Loop (MiL).

[0068] Such worst-case driving scenarios can either be derived from test drives (through parameter variations) or specifically constructed based on experience and expectations. Subsequently, using the Monte Carlo method, a multitude of individual test scenarios (parameter combinations as input values ​​for localization) can be determined around a worst-case scenario based on experience and expectations. These can then be systematically investigated using Model-in-the-Loop (MiL).

[0069] Typical worst-case driving scenarios are typically characterized by high driving dynamics and challenging environmental conditions, combined with parameter variation based on the Monte Carlo method. This approach is now described in more detail.

[0070] For the use of Model-in-the-Loop (MiL) and the Monte Carlo method, a simulation environment is preferably set up. Worst-case driving scenarios are simulated using a vehicle dynamics simulation environment. To simulate the difficult environmental conditions related to GNSS, configurable synthetic observations are modeled using a so-called GNSS rover simulator. Monte Carlo-based imperfections can be injected into IMU signals, correction data, and GNSS signals based on the results of the vehicle dynamics simulation environment and the GNSS rover simulator. The simulated data is processed using a software-in-the-loop environment. The outputs of the navigation module software under test are finally evaluated against the requirements, analyzing, for example, the variation of the localization error and the sensitivity of the integrity parameters.

[0071] The Monte Carlo method is used to inject errors into the parameters being processed. The output data of the navigation module software under test is influenced by the parameters used in localization in a very complex way, making it impossible to understand in advance which combination of imperfections in the measured inputs will most challenge the algorithm. For this reason, a simulation approach based on the Monte Carlo method is chosen to apply the various imperfections to the algorithm's input parameters. The idea of ​​ensuring the integrity of a complex system with this method is to choose a sufficiently large number of simulation runs, each with a different set of modified input parameters, to identify and cover the worst-case combination of input imperfections.In this way, the system's response can be investigated under the most challenging conditions without knowing all the details in advance. The ranges of various deficiencies in the input parameters can be determined, for example, using statistical distributions measured during the characterization of the individual input systems.

[0072] It is particularly advantageous if the navigation module has at least one inertial sensor module that provides input data for navigation.

[0073] It is also advantageous if the navigation module has at least one correction data interface through which GNSS correction data can be received as input data for navigation. The three components with the greatest influence on localization as the output of the navigation module software under test are the input parameters provided by the inertial sensors, the GNSS correction data, and the raw GNSS observations as the output of the GNSS receiver, all in combination with corresponding quality information.

[0074] The inertial sensor system includes acceleration and yaw rate sensors. The sensors are characterized in the laboratory to obtain the various possible values ​​of errors such as noise or offset. The results are preferably fitted with the appropriate distribution function to obtain the characteristic values, e.g., mean or variance. For each simulation run, a set of imperfections of the inertial sensor system is determined by calculating a possible realization of the imperfections according to the probability given by the corresponding statistics. These imperfections are added to the signals simulated with the dynamic vehicle simulation.

[0075] Based on the specification and characterization of the correction data by the correction data provider, a suitable distribution of imperfections is selected. Similar to inertial sensing, the signal imperfections are determined in each run according to statistical probability. The various imperfections are then added to the correction data.

[0076] GNSS measurement accuracy and quality can be degraded by the environment, e.g., due to multipath effects. The measurement and the corresponding quality information are modified using a Monte Carlo-based simulation approach to account for these influences on the navigation module software. The statistics of the measured GNSS inaccuracies are investigated, and characteristic values ​​for the distributions for a configurable number of altitude-dependent classes are derived. The inaccuracies of the GNSS observations are preferably modeled using a stationary Gaussian-Markov process, including the influence on the signal strength and the quality information. As in the case of the inertial sensors and the correction data, a set of input parameters for the Gaussian-Markov process is preferably calculated for each test using the Model-in-the-Loop (MiL).These imperfections are added to the previously calculated synthetic observation.

[0077] This Monte Carlo method combines various worst-case driving scenarios and challenging environmental conditions with numerous imperfection variants. For each set of simulations, the system behavior and its sensitivity to integrity-related requirements are verified using the Model-in-the-Loop (MiL) approach. Finally, a conclusion is drawn regarding integrity risk and safety availability.

[0078] Monte Carlo simulations via MiL and single-fault injections via HiL provide simulation-based support for the verification process. The Monte Carlo method complements the test-drive-based verification, while single-fault injection complements the fault tree analysis.

[0079] The fault tree analysis (FTA) method is used to analyze signal error propagation through the navigation module or through the individual data processing stages within the navigation module or in the navigation module's software. Fault tree analysis supports functional development and can be used, in particular, to validate the navigation module to demonstrate the safety of the intended functionality with regard to unusual events.

[0080] The fault tree analysis is preferably carried out starting from a so-called “normal case” of fault-free operation, which can never be achieved in reality.

[0081] Potential exceptional events or conditions (also referred to as hazardous events) are derived and evaluated as part of a SOTIF hazard analysis and risk assessment. An important input source is the validation catalog described above, which lists the scenarios / events that are also analyzed during test drives.

[0082] As part of the fault tree analysis, possible subsequent errors are listed for each potentially occurring error. The probabilities with which an error leads to a possible subsequent error are determined. This creates a complex fault tree, which can then be used to estimate the actual probability of an undetected faulty localization occurring as a result of certain input parameters.

[0083] Additionally, a confidence rating is preferably introduced to demonstrate how trustworthy the information is for an exceptional event. For example, ionospheric irregularities are highly dependent on solar activity, and a frequency of occurrence was derived by analyzing time series of geomagnetic indices and historical events. Due to the complex geophysical processes and phenomena behind this triggering condition, a reliable exposure rating is almost impossible, so the confidence rating can be set low, increasing the need for fallback injection tests, as explained later. Rather, the result is the probability of a top event, in our case, an HMI.The result is also: To see which paths / follow-up errors / cutsets make the largest contributions to the top event, in order to then be able to counteract them through increased testing and / or development of countermeasures (=monitors) in the software.

[0084] Preferably, the fault tree analysis includes the formation of clusters of scenarios / events.

[0085] Since fault tree analysis is used to analyze exceptional events, the probability values ​​used in the fault tree analysis for the occurrence of certain events and for the relationship between certain events and certain subsequent events are either derived from expert assessments based on considerations in other GNSS-related areas such as avionics, or historical assessments from previous analyses of the navigation module or comparable systems are used. For many of the identified triggering events, a purely analytical assessment is deemed insufficient. For this reason, a cut-set analysis may also be carried out to identify the most critical paths, i.e. those events that have a particularly high probability of causing undetected incorrect localizations. On this basis, further measures may then be taken if necessary.Further tests are carried out – for example, during test drives or, if necessary, using the Monte Carlo method and the fault injection described above. The goal of tests with fault injection is to stimulate dangerous positioning behavior by injecting individual or combined triggering events or conditions. The behavioral response of the navigation module is recorded and subsequently evaluated to confirm or change the risk assessments obtained with the fault tree analysis. The testing process is divided into a system qualification test and a software qualification test, with the focus on black-box testing of the system with pass / fail criteria derived from customer or system requirements, and on white-box testing with pass / fail criteria based on the system's expected detection and error prevention capabilities.

[0086] When constructing the fault tree for the fault tree analysis, the following questions are preferably answered:

[0087] • Does fault injection stimulate functional behavior?

[0088] • What is the order of error occurrence, detection and response time?

[0089] • Do subsequent investigations confirm the original assumptions on which the fault tree was constructed?

[0090] • To what extent is the error symptomatic and characteristic? For example, does the error increase over time?

[0091] • Which mechanisms (e.g. monitors) lead to a mitigation of the influence of errors?

[0092] The answer to this question is preferably evaluated and discussed by experts with a view to providing potential feedback for the quantitative fault tree analysis and can lead to confirmation and / or adjustment of the assumptions regarding the probability of exposure, severity, or prevention. All steps are performed iteratively and lead to a stable fault tree analysis with a quantitative probability estimate that describes the probability of undetected occurrence of an incorrect localization. The validation results from the three validation pillars—test drive-based approval, Monte Carlo simulations, and fault tree analysis—are preferably aggregated or combined into an overall validation to ultimately demonstrate the safety of the intended functionality in accordance with step e), if necessary.

[0093] The type of aggregation in this context can be very diverse. A first approach could be the summation of the individual probabilities for the undetected occurrence of a localization error. This is preferably done under the assumption that all validation pillars complement each other. A more sophisticated approach could be the weighting of the result probabilities of the individual validation pillars, depending on the use case and validation type.

[0094] The invention and the technical context of the invention are explained in more detail below with reference to the figures. The figures show preferred embodiments to which the invention is not limited. It should be noted in particular that the figures, and in particular the proportions depicted in the figures, are only schematic. They show:

[0095] Fig. 1: a schematic representation of the operation of a navigation module whose application can be validated and classified using the method described here;

[0096] Fig. 2: a schematic representation of the protection level;

[0097] Fig. 3: an embedded model for performing tests based on test data generated using the Monte Carlo method; and

[0098] Fig. 4: an overview of possible influences that can cause a distortion of the localizations.

[0099] Fig. 1 shows a vehicle 3 in a traffic situation. Navigation is carried out using a navigation module 1 arranged in the vehicle 3. This essentially relies on signals from GNSS satellites as well as information from a correction data service 19, inertial sensors and wheel speeds. Information from the correction data service 19 is preferably also distributed via satellites. For the localizations determined with the navigation module 1, a tolerance range 16 exists. If the deviation of a determined position 14 from the actual position 15 of the vehicle 3 lies with sufficient certainty within the tolerance range 16, the localization is OK and the integrity of the position output is met. The tolerance range 16 is also referred to as the protection level. The protection level is continuously provided. Above the alarm limit, an alarm must be issued within a predetermined alarm time.

[0100] The minimum safety specified by the protection level and the SOTIF approval is explained in more detail in Fig. 2. The top of Fig. 2 shows the true position, the output (or determined) position 14 and the protection level as an ellipse 13 in an example situation. The lower part of Fig. 2 shows the tolerance range 16 defined at the top of the plane as an ellipse 13, plotted here on a line for illustration. The actual position 15 and an example of a determined position 14 that deviates from the actual position 15 can be seen. Plotted on the line is a probability distribution 20 of determined positions 14. It can be seen that almost all determined positions 14 lie within the tolerance range 16 around the actual position 15. The alarm limit 17 is also plotted on the line according to Fig. 2.If determined positions 14 cannot be determined with sufficient accuracy so that they lie within the tolerance range 16 but beyond the alarm limit 17, an alarm must be issued.

[0101] According to Fig. 3, an embedded model 21 (MIL = Model in the Loop) is shown, with which the described method can be carried out at least partially (in particular according to step d).

[0102] The navigation module 1 is emulated here in the embedded model 21 with its input data sources. The input data sources are, in particular, the GNSS module 2, the inertial sensor module 5, the GNSS correction data interface 6, and the wheel speed sensors 10. Further data sources may be located upstream of the input data sources, simulating input data for the input data sources. These include, for example, a dynamic simulation 7, which simulates the driving dynamics of a vehicle, and a GNSS data simulator 8, which simulates GNSS input data. According to the embedded model 21 described here, modifications or errors are injected into the input data in a targeted manner using a Monte Carlo simulation 9. The tolerance of the navigation module 1 to these injected modifications and errors can be simulated in a simulation 11 and evaluated in an evaluation 12 in order to perform further tests according to step d).

[0103] Fig. 4 schematically illustrates the various input data and influencing variables that can be investigated using a fault tree analysis. Vehicle 3 with navigation module 1 (not shown separately here) is visible. Input data for performing localization with navigation module 1 are represented as arrows pointing toward vehicle 3.

[0104] Input data initially comes from GNSS satellites 18. Errors in the input data are generated, for example, by atmospheric layers, such as the plasmasphere 22, the ionosphere 23, and the troposphere 24. In the individual layers, ions or the weather, for example, can influence the input data / signals of the GNSS satellites 18. Correction data is provided by a correction data service 19 and, if necessary, via geostationary satellites 25 as input data to the vehicle 3 or the navigation module 1 arranged therein. Plants 26 and buildings 27 can also disrupt the reception of input data by the vehicle 3 or the navigation module 1, for example by causing reflected signals, interference, etc. All of these influences, or the potential for these influences to distort localizations created with the navigation module 1, can be analyzed using a fault tree analysis to identify particularly critical situations.

Claims

patent claims 1. A method for releasing an application of a navigation module (1) for a security level, wherein the navigation module (1) has at least one GNSS module (2) for determining navigation data as input data for navigation in a motor vehicle (3), and the method has the following steps: a) receiving application data relating to the application; b) receiving test data relating to the application which were determined during tests with the vehicles (3) comprising the applied navigation module (1); c) creating a large number of test scenarios (4) for the application at least based on the application data and the test data using a Monte Carlo method; d) carrying out tests of the application based on the test scenarios (4) created in step c); and e) issuing a release for the application at least based on results of the tests carried out according to step d).

2. The method according to claim 1, wherein the release granted according to step e) is a SOTIF release 3. Method according to claim 1 or 2, wherein according to the release granted in step e) an upper limit is specified for the probability of at least one undetected error occurring in navigation data determined with the navigation module (1).

4. Method according to one of the preceding claims, wherein the navigation module (1) has at least one inertial sensor module (5) which provides input data for the navigation.

5. Method according to one of the preceding claims, wherein the navigation module (1) has at least one correction data interface (6) via which GNSS correction data can be received as input data for the navigation.

6. Method according to one of the preceding claims, wherein, in the creation of the test scenarios (4) in step c), input data for the navigation module (1) are taken into account, the occurrence of which was determined on the basis of test data received in step b).

7. Method according to one of the preceding claims, wherein for the creation of the test scenarios (4) in step c) input data for the navigation module (1) are used which were determined in previous tests of the navigation module (1) in other applications and / or in test models.

8. Method according to one of the preceding claims, wherein in step c) a plurality of test scenarios (4) with modified input data are generated by a random modification of input data using the Monte Carlo method.

9. Method according to one of the preceding claims, wherein the number of test scenarios (4) created in step c) and the number of tests carried out in step d) is chosen to be so large that there is a statistical probability that particularly problematic input data for navigation are tested.

10. The method according to any one of the preceding claims, wherein the tests in step d) are carried out at least partially with embedded models (MIL = Model-in-the-Loop) which simulate the applied navigation module (1).

Images