DEVICE FOR ENVIRONMENT DETECTION OF AN AUTOMATED DRIVING SYSTEM

Description

[0001] The invention relates to a device for environmental detection of an automated driving system.

[0002] The automotive industry is currently grappling with the question of how much validation effort is required to ensure the safety of an automated driving system (AD system). One acceptance criterion for an AD system is, for example, that no more than one fatal accident occurs in 10⁹ operating hours. However, statistically proving the required acceptance criterion at the overall system level is difficult. Conversely, reliability at certain subsystem levels can only be demonstrated statistically. In particular, the causes of intermittent, or sporadic, errors in environmental perception are unpredictable and, moreover, extremely sensitive to even the smallest software and / or parameter changes. Therefore, it is impossible to define logical scenarios with a clearly defined error space from which the completeness of the verification could even begin to be argued.The applicant is also unaware of any standard that, in addition to the qualitative approach, also quantitatively describes the required scope of assurance, i.e., a standardized quantitative proof of the required acceptance criterion. Quantity represents the main challenge here. Furthermore, a differentiation between the proof of electrical / electronic robustness and the reliability of the intended function is currently lacking.

[0003] US patent 2019 / 0113916 A1 discloses the systems of an autonomous vehicle and their operation. Autonomous vehicles can rely on data inputs, processes, and output commands. Errors due to translation errors, failed or faulty equipment, connections, and other components can cause a dynamic vehicle path to approach a dynamic safety zone. In this case, a warning message can be sent and processed by the motion controller, depending on whether the vehicle is responding correctly to commands or not. If the vehicle's path approaches the safety zone, a containment error message is sent to the appropriate system.

[0004] US Patent 2020 / 218913 A1 discloses techniques for determining the state of motion of a target object. In one aspect, an onboard computer of an ego vehicle recognizes the target object in one or more images, determines one or more first attributes of the target object based on measurements of one or more images, determines one or more second attributes of the target object based on measurements of a map of a roadway on which the target object is located; and determines the state of motion of the target object based on the one or more first attributes and the one or more second attributes of the target object.

[0005] US Patent 2018 / 267550 A1 discloses a method for operating a controlled object embedded in a changing environment, wherein the controlled object and its environment are periodically observed using sensors, and in each period at least three independent data processing paths are executed based on the data obtained from the observation of the controlled object and its environment.

[0006] US Patent 2016 / 368491 A1 discloses a method for the automated driving operation of a motor vehicle. A standard trajectory and a safety trajectory are determined, the safety trajectory ensuring a safe stop of the vehicle in an emergency, depending on the current vehicle environment. The safety trajectory is fed to a second control unit, which transmits signals to vehicle actuators to control the vehicle based on the safety trajectory in the event of a safety situation.

[0007] The white paper “Safety First For Automated Driving”, see https: / / apollo.auto / docment / Safety First for Automated Driving handover to PR.pdf summarizes Safety by Design and Verification and Validation (V&V) methods of SAE L3 and L4 for automated driving.

[0008] Starting from this prior art, the invention was based on the objective of providing an improved device and an improved method for validating an AD system with regard to the required scope of safeguards and SOTIF risks. In particular, it was an objective of the invention to provide a holistic method for demonstrating the safety of the intended functionality of an L4 AD system, especially with regard to unforeseeable, sporadic malfunctions and malfunctions that are volatile with respect to software / parameter changes.

[0009] The device according to claim 1 solves this problem.

[0010] The invention assumes that the overall AD system can be serially decomposed and thus validated into the subsystems environment perception (also called See), trajectory planning (also called Think), and driving action (also called Act). With an error rate of 10⁻⁹ h at the overall system level, this results in an error rate of 3.33 x 10⁻¹⁰ h for each subsystem, provided that a single critical error in one of the subsystems leads to a catastrophic failure of the overall AD system. Environment perception includes environmental sensors. Environment perception can also include one or more domain ECUs on which the processing of raw sensor data takes place. Within the scope of the invention, it is possible, in principle, for the processing of the raw sensor data to also take place directly on the hardware of the environment sensors.Trajectory planning includes an AD-domain ECU that calculates environmental models from signals from the environmental sensors and determines control and / or regulation signals to follow a specific trajectory. The driving action subsystem includes actuators for longitudinal and / or lateral guidance, chassis components, and drive components.

[0011] The invention provides a device for environmental detection of an automated driving system according to claim 1.

[0012] The device includes hardware and software components.

[0013] In an unclaimed method for validating an automated driving system, the overall system of the driving system, comprising the subsystems environment perception (where the environment perception subsystem includes multiple environment sensors), trajectory planning, and driving action, is decomposed and validated. In a further method step, the environment perception subsystem is decomposed into individual processing channels belonging to the respective environment sensors by a device according to the invention. In a further method step, the environment perception subsystem is validated against objects based on previously recorded raw data, based on the decomposition into the individual processing channels. Validation against raw data means demonstrating the required robustness of the individual processing channels by reprocessing previously recorded raw data to a statistically significant extent.

[0014] This disclosure can therefore provide a holistic procedure for demonstrating the safety of the intended functionality of an L4 AD system.

[0015] Advantageous embodiments of the invention are shown in the drawing and the description of preferred embodiments. Automated driving systems execute the dynamic driving action automatically. The driving action includes driving functions for longitudinal and / or lateral control, for example, acceleration such as braking or steering, and driving functions for communication with the environment, for example, activation of a lighting system such as headlights or turn signals, or activation of sound-generating devices, for example, a horn. Automated driving systems comprise hardware components, for example, environmental sensors, control units, actuators, drive and chassis components, and software components, for example, software for evaluating raw data from the environmental sensors, for determining environmental models, or for planning a trajectory or mission.The SAE J3016 standard describes the classification and definition of terms for road-bound motor vehicles with autonomous driving systems. For example, the SAE J3016 standard defines an automation level for automated driving systems. The invention relates in particular to SAE J3016 Level 4 AD systems with correspondingly defined operating functions, also called Operational Design Domain (ODD), defined monitoring of the driving environment, also called Object and Event Detection and Response (OEDR), defined trajectory planning, also called maneuver behavior, and defined system errors, also called failure mode behaviors. The AD system is designed to function for these ODDs, OEDRs, maneuver behaviors, and failure mode behaviors.In one embodiment of the invention, the AD system is a bus line with a fixed route and lane assignment, which is to be operated by a bus, for example a people mover, in SAE J3016 Level 4 operation.

[0016] A signal processing path comprises signal lines and signal evaluation. The signal processing paths according to the invention comprise hardware and / or software channels, for example, sensor channels.

[0017] The first signal processing path, also called the performance path, of the device governs predictive control within the comfort range, for example, determining accelerations and decelerations below 1.5 m / s², based on information that can only be partially and intermittently captured by the sensors and high-performance and / or knowledge-based algorithms, such as detecting a pedestrian's intention to cross the street. To achieve this high functional performance, the environmental sensors in the first signal processing path operate far outside their safe detection range and beyond the safety zone. This leads to statistical and scene-systematic fluctuations of measurement points and thus to errors in environmental perception. Furthermore, learned algorithms, especially those from the field of machine learning, make interpretation errors under these conditions.The behavior of the evaluation algorithm is not necessarily unambiguously predictable. This means that reliable control cannot be guaranteed and the performance path may provide incorrect trajectory specifications. The invention does not initially specify any requirements for the software architecture of the performance path. With regard to artificial intelligence, comprehensively trainable or trained artificial neural networks for object recognition, semantic image segmentation or perception, for example, convolutional or recurrent networks, or networks comprising long-short-term memories for creating temporal relationships, such as tracking, fusion, and / or functional dependencies, the software architecture of the performance path is freely selectable.The invention involves the use of artificial intelligence within the individual processing channels of the safety path, provided that the fundamental requirements for the architecture of the safety path are met and suitable alternative solutions for conventional SWE4 / SWE5 test activities are found.

[0018] The second signal processing path, also called the safety path, of the device—that is, the signal processing path according to the invention—evaluates the sensor data for each required measurement task only within a safe detection area, the safety zone. The safe detection area, also called the safety zone, is the area immediately surrounding a vehicle, where the goal is to avoid accidents by fully utilizing the vehicle's deceleration and / or evasive maneuver potential. Within this safety zone, the AD system must guarantee maximum reliability. The safety zone is defined, among other things, based on the ODD (Off-Road Traffic) route segments, the vehicle's speed, and / or recorded reaction and deceleration values ​​of other road users. The evaluation is performed using traceable algorithms, with the processing channels independently processing their raw data and outputting evaluable objects.The objects to be evaluated are then logically combined to determine whether they are detected or not. The second signal processing path is a fallback to the first. It checks the environment detection and / or trajectory planning of the first path and corrects it, if necessary, by using an inconvenient method, for example.

[0019] 8 m / s 2 < maximum dynamic maneuver to prevent a fault in the first signal processing path from becoming a critical fault. The safety path thus represents a fallback level of the performance path. The maximum dynamic maneuver is an example of a safe trajectory. This determines a safe trajectory in the second signal processing path. Another example of a safe trajectory is an emergency stop maneuver.

[0020] Through the individual, sensor-specific processing channels of the signal processing path according to the invention, i.e., the safety path, the raw data are processed independently of one another, i.e., in particular without feedback loops or fusion, in order to determine environmental models from the respective raw data. Through the sensor-specific and thus independent processing channels, which include sensor-specific software channels, the invention achieves extensive functional redundancy on the safety path. The detected objects, including lanes, open spaces, vehicles, people, elements of infrastructure and / or vegetation, and other environmental artifacts, are combined by applying explicit logical connections between the individual processing channels.

[0021] Regardless of functional redundancy, the individual processing channels of the safety path are also subject to sporadic, i.e., non-continuous, and unpredictable detection errors that vary depending on the software version. To demonstrate robustness against these errors, a continuous run can be performed.

[0022] The architecture for normal driving situations, i.e., for the performance path, is designed to impose no restrictions in order to achieve the best possible performance. According to the invention, the safety requirements are offloaded to the safety path. In this context, the performance path is called Doer and the safety path is called Checker.

[0023] Redundancy with respect to function (software / SOTIF) is fully achieved on the second processing path through independent processing channels, and with respect to hardware failure by copying the safe trajectory to the hardware of the first processing path or another independent hardware unit. Within the scope of the invention, in addition to environment perception, trajectory planning and driving action are also performed in the first and second signal processing paths; that is, the device comprises a first and a second signal processing path for the overall AD system.

[0024] Environmental sensors detect the environment of the AD system and are based on various sensor technologies, such as optics (e.g., camera sensors or lidar sensors), radar, or acoustics (e.g., microphones). Within the scope of the invention, the environmental sensors of the AD system also include accelerometers. The environmental sensors are each connected to the first and second paths for signal transmission.

[0025] The first and second control units are, for example, each an AD Domain ECU for controlling automated driving operation, such as driverless, fully automated driving. ECU stands for Electronic Control Unit. Domain refers to a domain within a vehicle, such as the powertrain, chassis, body, or multimedia system. Many functions interact within a domain, but also across domain boundaries, in a functional network. The control units comprise components such as microcontrollers, ASICs, FGPAs, or a processor (e.g., a multi-core processor), and memory modules. The processor is configured for signal / data exchange with memory modules. For example, the processor includes a bus system. The memory modules constitute the main memory. Examples of memory modules are RAM, DRAM, SDRAM, or SRAM.In a multi-core processor, several cores are arranged on a single chip, that is, a single semiconductor component. Multi-core processors achieve higher computing power and are more cost-effective to implement on a single chip compared to multi-processor systems where each individual core is located in a processor socket, and the individual processor sockets are arranged on a motherboard.

[0026] For the safety path according to the invention, determining environmental models in the respective sensor channels means that for each sensor of the AD system, the raw sensor data, for example, pixel-wise brightness values ​​of a camera sensor, are processed independently of other raw sensor data in the control units in such a way that the environmental model is described in such a way that it can be unambiguously evaluated by a human, particularly with regard to its criticality for the chain of effects. A sensor channel, which is a processing channel, refers to the detection or recognition chain of effects. For example, objects, free space, and / or lanes are determined in the respective environmental model. The requirement that the environmental model be described in such a way that it can be unambiguously evaluated by a human does not initially apply to the performance path.

[0027] Due to the independent evaluation, the sensor channels are largely functionally redundant, even though they are processed on the same hardware, namely on the safety path, for example, on the control unit of the safety path. According to the invention, the decomposition of the functional error rates, i.e., the SOTIF errors, takes place on the safety path to the individual sensor channels. This leaves the performance path completely free with regard to raw data processing. In particular, this demonstrates robustness against non-continuous errors in the individual sensor channels. This ensures the required reliability, for example, 10⁻⁹ h, which is a "SOTIF-Level-D" This would correspond to the individual sensor channels, as described in more detail below. A SOTIF Level D would be analogous to the ASIL level, enabling a differentiation between E / E and software errors and thus also differentiated safety concepts.

[0028] Compared to the prior art, the invention thus provides a distinction between SOTIF levels and ASIL levels. Previously, the prior art did not differentiate between hardware and software redundancy. The architecture proposed according to the invention distinguishes between SOTIF and E / E. The test strategy proposed according to the invention is based on this distinction.

[0029] By generating a copy of the control signals, the invention ensures not only a safety objective regarding SOTIF failures but also a safety objective regarding electrical / electronic (E / E) fail-safe operation in accordance with ISO 2626-2. According to one aspect of the invention, the control signals are copied continuously, for example, with each measurement cycle. This means that in the event of a hardware failure of the control unit, a copy of the control signals exists for a currently calculated safe trajectory, which can be accessed, for example, from a performance path.

[0030] According to the invention, the control unit is configured to copy the control and / or regulation signals to a processing unit external to the signal processing path, in particular to an E / E-independent external processing unit. This allows the external processing unit to execute the control and / or regulation signals if the control unit's hardware fails. For example, the control unit includes a wired or wireless interface to the external processing unit. The control and / or regulation signals are copied to a memory of the control unit and continuously transferred to a memory of the external processing unit via the interface. The external processing unit is the control unit of the performance path.

[0031] SOTIF errors, such as sporadic detection errors, are unrelated to hardware failures. The hardware and basic services function flawlessly, yet serious errors occur due to misinterpretations of measurement ambiguities, model errors, and / or association errors in the evaluation software. Conversely, hardware failures are unrelated to software functionality. The crucial factor is that components fail due to electrical / electronic malfunctions or harmful external influences. The invention achieves robustness against SOTIF errors while simultaneously ensuring the safety objective regarding E / E errors.

[0032] For example, the safe trajectory calculated in the safety path is copied to the ECU of the performance path. This measure results in hardware-related redundancy with respect to the safe trajectory. If one of the two ECUs fails, the safe trajectory present on the other ECU is used to implement, for example, an emergency stop maneuver. This effectively decomposes an E / E-related ASIL-D requirement for the overall AD system into two separate ASIL-B requirements for the two redundant ECUs: the first and the second. These requirements are based on the hardware included in the first path, or other independent hardware to which the safe trajectory has been copied, and the hardware included in the second path. However, the SOTIF robustness requirement depends solely on the evaluation software running on the second ECU in the second path ("SOTIF-D").

[0033] According to the invention, the safety path comprises program instructions that cause the control unit to combine the objects detected by the individual processing channels in an OR decision, a majority decision, or an N-out-of-X decision, where X is the number of environmental sensing sensors and N is a predetermined number with N ≤ X, when the program is running on the control unit. These decisions are merely examples of a logical operation proposed according to the invention.

[0034] For example, the safety path, and also the performance path, are based on three different sensor technologies, such as radar, camera, and lidar. For instance, X=3 and N=2. This means the logical operation is a 2 / 3 majority decision, which corresponds to a 2 out of 3 majority redundancy. A 2 / 3 decision-maker is a rule-based decision-maker that always considers an object to exist if two of the three sensor channels confirm its presence. Such a decision has the advantage of being equally sensitive to false positives and false negatives. Due to the 2 out of 3 redundancy, the error rate at the sensor channel level is then less than 10⁻⁵ h. To validate this error rate, 284,600 operating hours are necessary to demonstrate the absence of catastrophic errors, which corresponds to 14.2 x 10 6< test kilometers, assuming that a test vehicle travels at an average speed of 50 km / h.without a catastrophic error. Without the inventive decomposition of the environment detection onto three redundant sensor channels with a rule-based decision-maker, 450.0 x 10⁹ test kilometers would have to be driven for each of the subsystems without a catastrophic error occurring.

[0035] Another example is X=3 and N=1. This is a 1 / 3 decision point, which considers an object to exist only if one of the three sensors detects it. The 1 / 3 decision point is advantageous for minimizing false negatives. In a scenario, for example, the radar detects an object. The camera and lidar do not. A 2 / 3 decision point, combining the camera, lidar, and radar data, would not output an object. This could result in a false negative if the object actually exists. However, the 1 / 3 decision point outputs an object as soon as it is detected by even one environmental sensor.

[0036] Another aspect of the invention relates to a one-third decision-maker in a first detection stage and a false-positive filter in a second detection stage. The false-positive filter executes a deterministic algorithm to filter out the false positive detections accepted by the one-third decision-maker. The false-positive filter is also called a hypothesis falsifier. This architecture has the advantage that, compared to the two-thirds decision-maker, the validation targets are an order of magnitude lower. For example, with a one-third decision-maker in the second path, 272,700 test kilometers must be demonstrated without a catastrophic failure occurring (functional correlation is neglected for simplicity).

[0037] It is also within the scope of the invention to evaluate raw data from X environmental sensing sensors of the same sensor technology, for example, 3, 5 or more radar sensors, camera sensors, or lidar sensors. However, the use of different sensor technologies has the advantage that functional correlation is also lower than with sensors of the same sensor technology due to different measurement principles.

[0038] According to another aspect of the invention, the control unit is designed to merge the detected objects by performing logical operations over processing cycles. The objects are merged over several processing cycles, in particular over a defined number of processing cycles, or per processing cycle. This increases the robustness of the system (especially with regard to false detections).

[0039] In a further embodiment, the safety path to each of the environmental sensors comprises a first processing channel and a second processing channel. The control unit is designed to detect and combine static objects in the individual first processing channels, to detect and combine dynamic objects in the individual second processing channels, and to determine the control and / or regulation signals based on these combinations.

[0040] Static objects describe the static environment and are part of the ODD / OEDER description. For example, static obstacles such as guardrails, curbs, buildings, or lane information are static objects.

[0041] Dynamic objects include, for example, pedestrians, cyclists, and / or vehicles. Dynamic objects trigger sporadic detection errors and / or continuous filing errors.

[0042] Due to underlying physical principles, continuous error patterns occur in such a way that the probability of occurrence of the frequency of occurrence follows a continuous distribution function. For example, the position measurement of objects is subject to deviation according to a continuous distribution; e.g., the Gaussian normal distribution. Non-continuous errors, also called sporadic errors, lie in deviation ranges beyond the standard deviation.

[0043] Non-continuous errors include false positives, for example, when a real object is incorrectly located. This is also referred to as a sporadic storage error. Another example is ghost detection, where a non-existent object is detected by the environmental awareness system. Non-continuous errors also include false negatives, where a real object is not detected by the environmental awareness system.

[0044] Non-continuous errors are observed less frequently than continuous ones. Errors associated with general sensor errors are related to distance, resolution, and noise. These errors can be observed in every processing cycle because the sensor practically never measures with 100% accuracy, and are therefore continuous errors. In contrast, sporadic errors occur less and less frequently with increasing development maturity, which does not make them less significant, but merely more unpredictable. By considering the detection performance over a long continuous run according to the invention, a statement about sporadic errors is generated.

[0045] For continuous errors, a distribution of the error amplitude with a good confidence interval can be measured relatively quickly. This requires comparatively little effort, and therefore, for example, DGPS reference systems (differential GPS reference systems) can be used, which also require equipping the target objects. To measure continuous errors, a highly accurate reference, for example in the centimeter range, is generally used to generate statistically significant results. Typically, such measurement systems, for example DGPS, require measuring devices in each object being examined, such as a Hunter target setup. In the case of sporadic errors, not every target object encountered during the endurance test can be equipped with DGPS. This means that only one reference system is available, which is installed on the endurance test vehicle.This is naturally less precise, but that's not significant, because the sporadic malfunctions cause much larger deviations than those resulting from continuous sensor errors. Alternative methods can also be used, such as drones that analyze the relative movement of surrounding objects using images. This effort is feasible precisely because the distributions were learned relatively quickly.

[0046] The invention makes it possible to distinguish between continuous and non-continuous errors. According to the invention, a non-continuous error is distinguished by a sudden and, compared to continuous errors, very large deviation from the reference. According to one aspect of the invention, non-continuous errors are distinguished from continuous errors by an accuracy limit of the reference, for example, a 3 sigma deviation.

[0047] Acceptance criteria relate to the a priori unknown error modes, initially at the overall AD system level. The functional architecture of the device proposed according to the invention, comprising the serial decomposition of the overall system into the subsystems environment perception, trajectory planning and driving action, and the decomposition into individual sensor channels of the environment perception, enables the verification of the robustness of the AD system against errors whose parameter space is initially unknown.

[0048] To demonstrate recognition robustness in real-world traffic, raw data can be collected in an initial endurance run, and the recognition performance of individual processing channels can be determined via reprocessing of the raw data in development stages and / or updates of the SEE environment recognition subsystem software, especially with each major software release.

[0049] The trigger conditions for detection errors in real-world traffic, especially for non-continuous errors, are not known or predictable a priori due to measurement ambiguities and / or inadequate object models, and are volatile with respect to even the slightest software adjustments. Robustness can be demonstrated through the continuous reprocessing of a quantitatively and qualitatively significant dataset. The required dataset is derived directly from the acceptance criterion (similar to a safety target) required for the overall AD system and its decomposition into the robustness required for the individual sensor channels. Logical operations make the decomposition of the acceptance criterion for the overall system into validation targets for the individual sensor channels mathematically possible, thereby making the scope of the validation practically manageable.

[0050] As described, the trigger conditions for detection errors in real-world traffic are unpredictable a priori due to measurement ambiguities combined with inadequate object models. Furthermore, these trigger conditions are volatile with even the slightest software adjustments. This unpredictability and volatility necessitate a statistically significant test with continuous stimuli, thus requiring an endurance-based evaluation of the robustness of the environmental perception system. Based on this, raw data from individual sensors can be collected in an initial endurance run, for example, using multiple vehicles. This constitutes an open-loop process. For instance, the raw data is reprocessed for each software release. This quantitatively demonstrates the robustness of the environmental perception system and / or the individual sensor channels against false detections.Despite the volatility, the robustness of the system against sporadic errors based on unknown and volatile trigger conditions can thus be confirmed even after a change in the software and / or other parameters.

[0051] To demonstrate the detection capability, the environmental sensors can be arranged on a rail-mounted test platform, taking into account their actual installation position within the driving system. Test runs for individual scenarios are repeated. This initially proves that the system reliably detects all relevant objects within the safety zone.

[0052] The basis for all further testing activities is proof that all relevant object classes, including static obstacles, are fundamentally detected under all ODD / OEDER-specific boundary conditions within the safety domain. For an AD system, the detection of vulnerable road users is the primary safety objective. In contrast, ADAS merely assists the driver in their driving actions, and there is no guarantee that a vulnerable road user will actually be detected under all relevant circumstances. For example, detection performance is not a safety objective for an emergency braking system. Demonstrating fundamental detection on a small number of dummies under clinical laboratory conditions, as in the case of ADAS, is insufficient for AD systems.Since vulnerable road users must be detected in as many poses and environmental conditions as possible, individual dummies with severely limited movement capabilities are insufficient for realistic testing. Validation requires testing with real people under realistic and, where possible, reproducible environmental conditions. The invention proposes mounting the sensor set on a rail-supported test platform, taking into account a realistic installation position. This ensures, via mechanical safeguards, that the apparatus does not actually collide with human test subjects, for example, in a crash test facility. The rail-supported test platform thus allows human test subjects to be present in the safety-critical area of ​​AD environmental perception without being endangered by the test itself.

[0053] By repeating test runs for individual scenarios, especially multiple repetitions for specific scenarios, it is ensured that individual observed test results do not reflect chance rather than actual robustness. Because a scenario occurs infrequently, the system can fail more often than if a scenario occurred frequently in order to achieve the same overall failure rate. Based on the number of negative test results and the total number of repetitions, a confidence level for the required failure rate can be calculated using a Bernoulli trial. According to one aspect of the invention, the frequency with which a specific scenario occurs in reality, i.e., the so-called exposure, is taken into account. The magnitude of the failure rate therefore relates to the combined probability that, firstly, a specific scenario occurs at all, and secondly, that the system simultaneously fails.With an exposure time on the order of 10⁻³ < h, the system failure rate for a sensor channel in the corresponding specific scenario should not exceed 1 / 10. Assuming that the occurrence of the scenario is statistically independent of the system failure rate in that scenario, that the scenario occurs only once every 1000 hours, and that the required rate is 10⁻⁴ < h, the system only needs a failure rate less than or equal to 1 / 10. For a specific scenario, this results in 15 (21, 28, 43) repetitions at a required confidence level of 80% (90%, 95%, 99%). To achieve the required confidence level, not a single test repetition may fail.

[0054] To demonstrate robustness against false positives, especially ghost detection, a second, route-dependent endurance run can be performed. It is expected that the scope of this endurance run will be significantly shorter than that of the first, depending on the chosen decision logic and a potentially less stringent acceptance criterion regarding false positives.

[0055] In the first endurance test, the robustness of the AD system against serious placement errors is demonstrated. These are false positive and false negative detections where the measured position of the object detected by the sensor deviates from the position of the actual object. In the second endurance test, the robustness of the AD system against ghost detections is demonstrated. Ghost detections usually do not occur in connection with real, dynamic objects. Therefore, the static environment is crucial for this error category, meaning, in particular, the specific path. According to a further aspect of the invention, when the AD system is applied to a specific path, continuously recorded data is reprocessed with every software / system change. This demonstrates robustness against location-specific, safety-critical ghost detections.

[0056] The first and / or second endurance run can be conducted under varying light, weather, and / or vegetation conditions. Since differing light and weather conditions, as well as the seasonally varying state of vegetation, can be decisive factors in false detections, especially ghost detections, taking these factors into account improves the robustness against such errors.

[0057] The scope of the first endurance run can be determined using a confidence level to confirm an error rate for a given maximum error rate using conditional probability. P [ λ < FR | n k , where P as confidence indicates a probability that the error rate λ to be confirmed is smaller than the specified maximum error rate FR, given that k false positives were observed out of n trials.

[0058] The derivation for the required scope of the first endurance run is based on a Bernoulli trial, in which a sample is taken in discrete observation steps and the result, i.e., false / error detection: yes or no, is evaluated using k out of n. The probability for the Bernoulli trial follows the binomial distribution, which can be approximated by the Poisson distribution for small error rates. Due to the inventive architecture for the second path and the decomposition of the robustness required for the AD system to the individual sensor channels, the first endurance run becomes realistic even in a statistically significant scope. The trial scope n is expressed, for example, in n hours or as n kilometers. Using Bayes' theorem, the required trial scope n can be calculated for a given maximum error rate FR, confidence P, and an error rate k.Similarly, the total number of repetitions of test runs for individual scenarios, especially multiple repetitions for specific individual scenarios, can be calculated.

[0059] A test phase with a safety driver on a target route can be conducted. This test phase represents a closed-loop procedure and ensures that specific scenarios not covered by the first and / or second endurance runs do not occur frequently. In particular, this test phase minimizes the residual risk of malfunctions in unknown, unsafe scenarios. Furthermore, certain situations in an open-loop recorded data catalog (compare the input of raw data and its reprocessing) do not even arise because, unlike the AD system, the test driver will avoid traffic-critical scenarios from the outset. The resulting uncertainty regarding undiscovered error scenarios is addressed by this test phase and the driving tests with the rail-mounted test platform.

[0060] Pitch and roll behavior can be simulated, for example, with high temporal resolution. This allows the influence of the real AD vehicle's driving dynamics to be investigated, for example, during emergency braking. Pitch and roll behavior can be simulated, for instance, using a rail-mounted test rig.

[0061] The permissible number of errors in the trials can be set to zero. This means that not a single error may be observed in the corresponding test set for the required error rate to be confirmed with the necessary confidence. One goal is to provide proof with as little effort as possible. In principle, more errors can be permitted, but then a larger sample size is required for proof; that is, more endurance kilometers and therefore higher costs.

[0062] For a 1 / 3 decision-maker with a deterministic false-positive filter, the robustness requirement, i.e., the error rate, per sensor channel is 8 × 10⁻⁴ < 1 / h. This requires a circumference of 5800 km for the first endurance run to achieve 99% confidence. If, in comparison, a 2 / 3 decision-maker is used, the error rate per sensor channel is 1.05 × 10⁻⁵ < 1 / h. This requires a circumference of 439,000 km for the first endurance run to achieve 99% confidence (functional correlation neglected in this example).

[0063] When decomposing the required false detection rates, a functional correlation of sporadic errors occurring across sensor channels can be computationally considered. The robustness required of the individual sensor channels increases with increasing correlation. Consequently, the scope of the first endurance run also increases with the functional correlation of the sporadic detection errors of the individual sensor channels.

[0064] The higher the functional correlation, the lower the functional redundancy, and therefore the greater the robustness required for the individual sensor channels, and consequently, the longer the test duration. However, the lower the correlation, the more test kilometers are needed to demonstrate the correlation. This means there is a minimum test duration required to statistically prove the correlation or the required error rate. In the example of a 1 / 3 decision-maker with a false-positive filter, this is 2 × 10⁶ km.

[0065] A functional correlation is, for example, that the placement accuracy of each sensor decreases with distance. Functional dependency also exists, for instance, if at least similar methods for environmental modeling are used in the individual sensor channels. The endurance data thus serve to demonstrate the required single-channel robustness and to determine the functional correlation of the current software version with regard to sporadic detection errors. For a 1 / 3 decision-maker in the second path with a deterministic false-positive filter, the required error rate results in a minimum endurance range of 2 × 10⁶ km with a correlation coefficient of 10⁻³.

[0066] The invention is explained by way of example in the following figures. They show: Fig. 1 an embodiment of a device according to the invention, Fig. 2 an example of a safety zone and performance zone, Fig. 3 An embodiment of a method for false positive detection errors and false negative detection errors comprising a 2 / 3 decision-maker, Fig. 4 Examples of qualitative continuous load distributions, Fig. 5 Examples of errors and Fig. 6 An example of sporadic false positives.

[0067] In the figures, identical reference symbols denote identical or functionally similar parts. For clarity, the relevant reference parts for understanding the respective figure are indicated within each figure.

[0068] Fig. 1 Figure 10 shows a system architecture of a device according to the invention. A sensor set at the sensor level comprises different sensor technologies, namely, for example, radar technology (RAD), camera technology (CAM), and lidar technology (LID). The invention is not limited to a specific number of sensor technologies or environmental sensing sensors. An embodiment or application example of the invention is an L4 driving system comprising 5 radar sensors, 4 camera sensors, and 2 lidar sensors. An automated driving system (AD) comprises at least one environmental sensing sensor, preferably several environmental sensing sensors, for each sensor technology. The sensors used each cover the safety zone. The raw data from the environmental sensing sensors based on radar technology (RAD) are point or raw detections. The raw data from the environmental sensing sensors based on camera technology (CAM) are raw images.The raw data from the environmental sensing sensors based on lidar technology (LID) are point clouds. Environmental features U detected by the environmental sensing sensors include, for example, weather, road users, vehicle types, buildings, and landscape features. Environmental models describe the environment in such a way that the relevant environmental features U can be clearly evaluated by a person. The raw data is provided to the device 10 via interface INT.

[0069] The Environmental Detection Subsystem (EDS) of the overall Driving Dynamics (AD) system comprises a first path P1, also called the Performance Path, and a second path P2, also called the Safety Path. Both the first path P1 and the second path P2 contain signal lines to the environmental detection sensors.

[0070] The first path, P1, is designed for comfortable driving behavior, such as slow acceleration or deceleration, for example, from 1.5 m / s², or starting off through anticipatory analysis of the surroundings. This places significantly greater demands on the sensors' detection ranges, pushing them to their performance limits. Naturally, this results in a higher susceptibility to errors. The specific performance requirements for the first path are defined according to one aspect of the invention in consultation with a customer and / or depending on the requirements of the driving function, as it primarily concerns comfort. Consequently, the error rates for this path are significantly higher than those required for the second path, P2. For example, corresponding requirements could be derived from comparable ADAS functions.

[0071] The first path P1 includes a first control unit ECU1. The first control unit ECU1 is an electronic control unit, also called an Electronic Control Unit. The first control unit ECU1 comprises hardware, such as a microcontroller, and software. When executed on the first control unit ECU1, the software instructs the microcontroller to evaluate signals from the environmental sensors, determine initial control signals for trajectory planning (THINK), and execute a corresponding driving action (ACT). The evaluation includes the fusion of detected static and / or dynamic environmental features (U) and the identification of static and / or dynamic objects. Based on these static and / or dynamic objects, a trajectory planning subsystem (THINK) of the overall driving system (AD) calculates a trajectory. This trajectory is then executed by an ACT subsystem of the overall driving system (AD).The first control unit ECU1 determines the environment recognition SEE and trajectory planning THINK. The driving action ACT is executed by actuators of the driving system AD for drive, chassis and / or longitudinal and / or lateral control, which are connected to the first control unit ECU1 via corresponding interfaces that transmit signals.

[0072] The second path, P2, is the fallback level of the first path, P1, with the purpose of minimizing risks to the life and limb of occupants and other road users in a quantitatively calculable way. Within the so-called safety zone S surrounding the AD driving system, the aim is to avoid an accident by fully utilizing the deceleration and evasive maneuver potential of the AD driving system, for example, an acceleration or deceleration of less than 8 m / s². Within this safety zone S, the AD driving system must guarantee maximum reliability. Therefore, demonstrating the required robustness in this area significantly defines the necessary safety measures.

[0073] The second path, P2, includes a second control unit, ECU2. ECU2 is an electronic control unit, also known as an Electronic Control Unit. It comprises hardware, such as a microcontroller, and software. When executed on ECU2, the software instructs the microcontroller to evaluate signals from the environmental sensors, determine secondary control signals for trajectory planning (THINK), and execute a corresponding driving action (ACT). Environmental detection (SEE), trajectory planning (THINK), and driving action (ACT) are thus processed in the first path, P1, and the second path, P2. The evaluation includes the detection of static and / or dynamic objects in the raw data from the environmental sensors, depending on sensor channel-specific environmental models.

[0074] The hardware and / or software design of the first path P1, in particular of the first control unit ECU1, is arbitrary. The safety path P2 according to the invention runs on the second control unit ECU2. Functional redundancy is achieved on the second control unit ECU2 through independent software / sensor channels. The safe trajectory is calculated on the second control unit ECU2, i.e., control and / or regulation signals are generated, and, for example, an emergency brake is triggered if an object is located in the safety zone S. The first path P1 plays no role in this. As long as the first path P1 does not malfunction, no obstacle appears in the safety zone S; however, if it does, the second path P2 takes over.

[0075] According to the invention, the device 10 provides the actuators of the AD driving system with the second control and / or regulation signals, with the second path P2 independently monitoring the safety zone S. If an object appears in the safety zone S, the safety path P2 triggers, for example, braking.

[0076] Error modes of dynamic objects include sporadic recognition errors and continuous storage errors. Error modes of static objects are part of the ODD / O-EDER description. The invention makes it possible to validate an acceptance criterion of the AD driving system for these error modes, particularly with regard to dynamic objects, and especially sporadic recognition errors.

[0077] By copying a safe trajectory calculated on the second path P2 to the first control unit ECU1, device 10 also ensures a safety objective with regard to E / E faults. This is because copying the safe trajectory results in hardware-related redundancy. For example, if the second control unit ECU2 fails due to a hardware fault, device 10 uses the safe trajectory present on the first control unit ECU1 to implement an emergency stop maneuver. Thus, device 10, and consequently the AD driving system, achieves an ASIL-D rating with regard to E / E faults if both the first control unit ECU1 and the second control unit ECU2 have an ASIL-B rating.

[0078] Furthermore, it shows Fig. 1 Safety path P2 has a first processing channel and a second processing channel, in which the raw data is evaluated independently in each sensor channel. Static objects are evaluated in the first processing channel, and dynamic objects in the second. The static objects from the individual sensor channels are combined in the first processing channel using a logical operation, for example, by a rule-based decision-maker. Similarly, the dynamic objects from the individual sensor channels are combined in the second processing channel using a logical operation, for example, by a rule-based decision-maker. Based on the fused static and dynamic objects, safety path P2 calculates a safe trajectory.

[0079] Fig. 2 The diagram shows a safety zone S and a performance zone P, each corresponding to the field of view of environmental sensors. The field of view corresponding to performance zone P is evaluated in the first path P1. The field of view corresponding to safety zone S is evaluated in the second path P2. The AD driving system, for example, is an L4 people mover.

[0080] The high demands on the system's robustness relate to safety zone S, for example, an emergency braking zone. To demonstrate this robustness, the invention proposes individual, independent sensor channels, observable interfaces, and rule-based decision-makers.

[0081] To be able to see beyond the safety zone S, artificial intelligence and complex machine learning architectures are used in the performance zone P. Even experts cannot predict the behavior of these architectures, and, more importantly, their overall error rate cannot be decomposed. Based on this premise, the invention employs these methods for the comfort zone, methods for which high robustness is difficult to confirm, but which typically allow for much greater visibility. In normal driving situations, all driving decisions are based on the environmental model.

[0082] For monitoring the safety area, the invention employs simpler methods for environmental detection, which can be clearly evaluated in terms of robustness, but cannot see as far ahead. These methods function as a fallback in an emergency situation and are calculated to be reliable (quantitative confidence for the required robustness).

[0083] Fig. 3 Figure 1 shows a fault tree for false positive detections (FP) and false negative detections (FN) with a 2 / 3 decision point. The numerical values ​​given for FR are the respective error rates (FR). An acceptance criterion for the overall driving system (AD) is distributed across the subsystem levels of environment detection (SEE), trajectory planning (THINK), and driving action (ACT) by the decomposition according to the invention, in particular serial decomposition. The decomposition to the subsystem levels corresponds to process step V1 of the method according to the invention. In process step V2, the environment detection (SEE) is decomposed into the individual sensor channels CAM, RAD, and LID. The sensor channels correspond to detection chains. Redundancy arises from the independent processing of the raw data from different sensors using correspondingly different methods. This achieves the SOTIF (Safety of Transmittance in the Event of Failure) safety objective.This decision algorithm is equally sensitive to false positives (FP) and false negatives (FN). This architecture results in error rates (FR) of FR = 10⁻⁵ < h at the sensor channel level. The hours required to demonstrate the absence of catastrophic errors, based on this error rate (FR), amount to 284,600 h. This test time is achieved, for example, when 32 test vehicles continuously collect test data over a year.

[0084] According to the invention, endurance tests are performed. Naturally, the endurance test should be distributed in such a way that it corresponds to the normal driving situation of the AD driving system in real-world use. Fig. 4 This illustrates the problem of the continuous run distribution. The x-axis represents all relevant scene vector combinations, meaning the entirety of all theoretically possible concrete scenarios, including their relevant parameters. A single concrete scenario represents a unique combination of relevant environmental influences as well as static and dynamic object scenery. The y-axis represents the frequency with which a specific concrete scenario occurs. The task, therefore, is to estimate the actual frequency distribution of the concrete scenarios as accurately as possible during subsequent regular operation and to design the continuous run accordingly.

[0085] If the distribution of the endurance run differs from that of subsequent regular operation, the endurance run will not cover all relevant concrete scenarios. This creates the possibility that certain false detections will be missed during the reprocessing of the endurance run recordings. This is particularly problematic if the frequency of the concrete scenarios not included in the endurance run is high in real-world operation. One way to counteract this would be to increase the statistically required endurance run volume again. This would randomly collect more scenarios, and the range of uncovered concrete scenarios would automatically tend to decrease. The question, however, is how much more data would be required to make the uncovered range sufficiently small.

[0086] For this purpose, the invention proposes a phase in which the finished AD driving system is initially operated on a target route with a safety driver.

[0087] Another measure is to take local operating conditions into account, for example, significant traffic simulation or relevant traffic statistics or traffic observations, so that all relevant vehicle constellations are known and the scenarios seen in a continuous run can be compared with them.

[0088] The same applies to weather conditions or weather statistics. The content of the endurance run can essentially be defined using very similar methods to those used to justify the completeness of the scenes for validating the Think driving function: Everything that is observed can be defined. The operating conditions are observable and therefore definable.

[0089] This allows for a complete description of the distribution of operating conditions. However, the operating conditions are only a subset of the scene vector combinations that can lead to a sensor malfunction. For this reason, the invention proposes a reprocessing of the data continuously recorded during a run. This statistically collects all other unknown parameters, including the trigger conditions, without requiring their knowledge. However, the operating conditions alone are sufficient to describe the content of the run.

[0090] Any remaining uncertainty, combined with the fact that a test driver (endurance run = open loop) will avoid critical situations in their development, is covered by the invention via the closed-loop phase with a safety driver.

[0091] Provided no malfunctions occur that are not also observed during the endurance run, the latter can be considered sufficiently representative of the actual operating route, and the AD driving system can subsequently be operated without a safety driver. An important aspect for minimizing the area not covered by the endurance run is that all relevant constellations with dynamic objects were observed to a sufficient extent and in all environmental conditions, especially weather and lighting conditions, during the endurance run recording. Which object constellations are relevant can be determined from nanoscopic traffic simulations and / or from representative traffic observations for identified ODD route sections.

[0092] According to the invention, the endurance test is not performed on a specific section of the route to be opened. Even if, for example, there is usually little traffic on the section of the route to be opened, the endurance test is selected according to the invention in such a way that high traffic densities on the relevant ODD route sections are also taken into account. On the one hand, one does not want to stop operating the AD system if the traffic density increases; on the other hand, one might also want to be able to use the system in comparable ODDs with denser traffic without having to start from scratch.

[0093] Fig. 5 The invention comprehensively identifies potential errors of the AD driving system, including errors at the sensor level, environmental perception, and driving function. Potential errors considered by the invention include, for example: Lateral localization error: Due to faulty lateral localization, vehicles may cross lane markings. Collisions with laterally positioned or oncoming traffic, as well as stationary objects, cannot be ruled out. A system for reliable lane positioning can remedy this. Such a system can be secured, among other things, through additional infrastructure monitoring. Longitudinal localization error: Longitudinal localization errors can also have critical consequences. This is particularly true when, for example, a lateral lane marking must not be crossed at a specific point, such as an exit or bus stop. If longitudinal positioning is faulty, a critical lane departure can occur.Using additional landmarks, such as lines or QR codes, can enable reliable positioning. Object detection – Accuracy: Measurement errors in object detection can lead to inaccurate object properties. These can be potentially dangerous if, for example, objects are placed in the wrong lane, or trajectories are planned too close to or through parts of a real object. Object detection – Existence: Sporadic errors can occur when determining the existence of objects. These can be critical because other road users are either not detected or assigned to the wrong lane. Detections for which no real object exists, or detections with seriously inconsistent object properties, also pose a high risk.In the context of SAE Level 4 vehicles, false negative detection is considered the most critical error, as collisions with real-world road users are significantly more likely to be fatal than incorrect braking. Driving function: Regardless of correct environmental perception, errors can also occur in trajectory generation during driving function calculations. This happens when a trajectory is calculated that is inadequate with regard to the driving action and safety. Such errors lead to incorrect lateral movements, as well as incorrect accelerations or decelerations.

[0094] Fig. 6 This demonstrates how sporadic false detections are identified. Using reference sensors from a reference system, statistical misdistributions are generated for individual sensors regarding true positive detection (TP), true negative detection (TN), false positive detection (TP), and false negative detection (FN). In the recording endurance vehicles, additional comparison sensors are carried as reference sensors alongside the sensor set being tested. To test the environment model under test, a reference environment model is determined from the raw data of the reference sensors. Since raw data processing in the reference system does not need to occur in real time, the reference environment model can advantageously be determined in a post-processing step. An automatic comparison of the environment models under test with the reference environment model reduces the effort required to determine reliable information about the objects actually existing in the scene under consideration at their respective positions.Instead of manually annotating all relevant objects in a recorded video stream, corresponding annotations are only made when there is a discrepancy between the environment model under test and the reference model. The characteristics of the reference sensors are used to define thresholds between storage errors and sporadic errors. Events that exceed the threshold for reference accuracy with respect to sporadic errors are annotated manually. Bezugszeichen

[0095] 10 Device AD ​​Automated Operating Driving System SEE Environment Detection THINK Trajectory Planning ACT Driving Action P1 First Path ECU1 First Control Unit P2 Second Path ECU2 Second Control Unit E1 First Processing HF Second Processing S Safety Zone P Performance Zone INT Interface RAD Environment Detection Sensors Based on Radar Technology CAM Environment Detection Sensors Based on Camera Technology LID Environment Detection Sensors Based on Lidar Technology FR Error Rate SF Non-Continuous Detection Errors FP False Positive Detection FN False Negative Detection TP True Positive Detection TN True Negative Detection U Environment Features V1-V3 Process Steps

Claims

1. Device (10) for environment recognition (SEE) of a driving system (AD) that is able to be operated in an automated manner, comprising a first signal processing path (P1) comprising a first control unit (ECU1) designed to determine first closed-loop and / or open-loop control signals for trajectory planning (THINK) of the driving system (AD) on the basis of raw data from environment detection sensors (RAD, CAM, LID) from a performance range (P) of the driving system (AD), wherein in the performance range (P) slow acceleration or deceleration or start-up is subject to closed-loop control by predictive analysis of the environment as a result of a required greater detection range of the environment detection sensors (RAD, CAM, LID), a second signal processing path (P2) comprising a second control unit (ECU2) comprising one processing channel for each of the environment detection sensors (RAD, CAM, LID) and designed to determine second closed-loop and / or open-loop control signals for trajectory planning (THINK) of the driving system (AD) on the basis of raw data from a safety range (S) of the driving system (AD), wherein • the second control unit (ECU2) is designed to identify in the individual processing channels static and / or dynamic objects in the individual raw data of the environment detection sensors (RAD, CAM, LID) on the basis of processing-channel-specific environment models, wherein the respective raw data are processed in an environment model independently of other raw data, wherein the second signal processing path (P2) comprises program commands that cause the second control unit (ECU2) to combine the identified objects in an OR decision, a majority decision or an N-out-of-X decision, wherein X is the number of environment detection sensors (RAD, CAM, LID) and N is a predetermined number where N ≤ X, when the program is run on the second control unit (ECU2), and to determine closed-loop and / or open-loop control signals for trajectory planning (THINK) of the driving system (AD) on the basis of the combination in order to follow a particular trajectory, and • the safety range (S) is the safe detection range which is situated immediately around a vehicle and which is smaller than the performance range (P), wherein the second signal processing path (P2) further comprises: • signal lines and signal evaluation for environment recognition (SEE), • first interfaces (INT) to environment detection sensors (RAD, CAM, LID) in order to obtain raw data from the safety range (S) of the driving system (AD), wherein the second control unit (ECU2) copies the closed-loop and / or open-loop control signals to a processing unit which is situated external to the second signal processing path (P2) and the device (10) is designed to provide the second closed-loop and / or open-loop control signals to actuators of the driving system (AD) if at least one object has been identified in the second signal processing path (P2) and the second closed-loop and / or open-loop control signals are copied to the first control unit (ECU1).