System and procedure for verifying sub-functions of ADAS / ADS systems

Abstract

System (100) for verifying partial functions of advanced driver assistance systems (ADAS) and / or automated driving systems (ADS) at synchronization points (750, 755), wherein a synchronization point (750, 755) is a predefined point in time or phase in the development process of an ADAS / ADS system at which all necessary changes to the system architecture, software or hardware are integrated, and wherein specific test cases (350) are executed at this synchronization point (750, 755) to ensure that all relevant functional groups (450) function properly, comprising an input module (200) for inputting available functional groups (450) and synchronization points (750), in particular with specifications relating to the system architecture or software updates;a test case generation module (300) configured to generate a set of test cases (350) for testing and verifying an ADAS / ADS system; a function grouping module (400) configured to divide the set of test cases (350) into function groups (450), each function group (450) representing a sub-function of the ADAS / ADS system; a prioritization module (500) configured to assign priority classes (550) to the function groups (450) based on the probability of errors occurring and the effort required to correct them; a synchronization module (700) configured to assign the corresponding function groups (450) and the test cases (350) associated with the function groups (450) to the synchronization points (750, 755);a test management module (800) configured to consider the maximum number of test cases (350) per synchronization point (750) and, if this number (350) is exceeded, to move test cases (350) with a low priority class (550) to subsequent synchronization points (755); and an output module (900) configured to output the selected test cases (950) for a synchronization point (750) to increase the efficiency of verification tests and reduce resource requirements.

Description

[0001] The invention relates to a method, a system and a computer program product for verifying partial functions of advanced driver assistance systems (ADAS) and / or automated driving systems (ADS) at synchronization points.

[0002] Modern advanced driver assistance systems (ADAS) and automated driving systems (ADS) encompass a range of technologies designed to improve vehicle safety and driving comfort. A typical feature is adaptive cruise control (ACC), which uses radar or laser sensors to maintain a safe distance from the vehicle ahead. It automatically adjusts the vehicle's speed to the flow of traffic, eliminating the need for the driver to constantly change speed manually. Another feature is lane departure warning (LDW), which uses cameras or sensors to detect unintentional lane departures. The driver is warned by visual, audible, or haptic signals and prompted to correct the vehicle's position.Another feature is Automatic Emergency Braking (AEB), in which the brakes are automatically applied to avoid a collision with a vehicle or pedestrian.

[0003] Advanced Driver Assistance Systems (ADAS) and Automated Driving Systems (ADS) are integrated into vehicles and are a central component of the vehicle architecture. ADAS / ADS systems work together with various electronic control units (ECUs), sensors such as cameras, radar, and lidar, and algorithms to support or completely take over driving tasks, depending on the level of automation. The goal is to increase safety, comfort, and the automation of driving. To this end, the ADAS / ADS system includes a wide range of functions that are constantly being developed to further improve vehicle safety and driving comfort.

[0004] Integrating an ADAS / ADS system into a vehicle requires an extensive application, verification and validation process to ensure optimal functionality, safety and adaptation to different vehicle platforms.

[0005] In the automotive industry, the application process involves the targeted adjustment of parameters within a data set to optimize the behavior of an ADAS / ADS system without altering the underlying software code. Application is a crucial step in adapting ADAS / ADS systems to different vehicle variants, platforms, markets, and specific customer requirements, while simultaneously ensuring manufacturer-specific behavior.

[0006] The data set loaded into the control units contains machine-readable parameters that the functional software accesses during operation. This allows changes to be made even in complex ADAS / ADS systems without requiring modifications to the software itself, minimizing effort and increasing flexibility. Some application parameters can be easily derived from the vehicle configuration or system specification, such as sensor positions or the availability of functions within the vehicle.

[0007] Other parameters, however, must be refined in an iterative testing and adjustment process, as their impact on the ADAS / ADS system behavior is not directly predictable. This iterative process includes numerous real-world test drives and simulations to ensure safe, comfortable, and natural system behavior. An example of this is the application of Adaptive Cruise Control (ACC), where the approach behavior to a newly detected target vehicle within the ego vehicle's driving corridor is optimized.

[0008] Optimally implemented ADAS / ADS systems offer two key advantages: First, they increase customer acceptance of ADAS / ADS systems, as predictable and safe behavior of the system builds driver confidence and makes them more willing to delegate all or part of the driving task to the system. Second, flawless implementation represents a competitive advantage. In independent tests comparing ADAS / ADS systems from different manufacturers in terms of their driving behavior and performance, ADAS / ADS systems with optimally implemented parameters demonstrate superior performance. This is especially true for systems with SAE Level 3 and higher.

[0009] SAE Level 3 refers to the classification of automated driving according to the Society of Automotive Engineers (SAE) standard. This standard defines six levels of autonomous driving, from Level 0 to Level 5. Level 3 (conditional automated driving) means that the vehicle can independently perform certain driving tasks, but the driver must be able to resume control at any time if prompted by the ADAS / ADS system. The vehicle can drive autonomously for limited periods and in specific scenarios, such as on highways, without requiring active driver intervention. However, the driver must be ready to take over control.

[0010] In contrast to Level 2 (partially automated driving), where the driver must remain constantly attentive, Level 3 allows for greater autonomy, but still requires the driver's willingness to take control in complex or unforeseen situations.

[0011] The verification of an ADAS / ADS system serves to demonstrate its conformity with the system specification. To this end, test cases are derived from the specific requirements, verifying the behavior and functionality of the ADAS / ADS system in various driving situations. A test case comprises a concrete scenario and criteria for passing or failing. When verifying ADAS / ADS systems, such as adaptive cruise control (ACC), scenarios such as approaching other vehicles, human-machine interaction, and communication with partner systems are tested.

[0012] Each test case evaluates whether the ADAS / ADS system meets a specific requirement. If a test case fails, the deviation is analyzed to determine whether there is a faulty implementation or an insufficient requirement. Errors are fixed and retested in the next development iteration. This iterative process improves the system's quality and ensures that the ADAS / ADS system functions reliably.

[0013] The verification process is carried out specifically for different vehicle derivatives, configurations, and markets to ensure that the ADAS / ADS system meets the requirements in various environments. A key component is the pre-approval test, which identifies potential performance deficiencies early on to minimize future approval risks.

[0014] Validation serves as a specification-independent verification that an ADAS / ADS system functions correctly and provides the required safety. Customer and stakeholder requirements are taken into account. During development, the ADAS / ADS system is tested in real-world traffic as part of the validation process to ensure that it functions safely and reliably in everyday use. Unlike verification, these tests are not based on fixed test cases, but rather on evaluable events and acceptance criteria.

[0015] An example of validating an Adaptive Cruise Control (ACC) system is its use in a prototype vehicle, where it is tested whether safety-critical or unpleasant situations occur, such as delayed vehicle detection or delayed braking. Performance parameters such as the minimum distance to other vehicles or the maximum deceleration are evaluated to assess the performance of the ADAS / ADS system.

[0016] As with application and verification, validation is carried out in various vehicle derivatives, configurations, and markets. If malfunctions or performance deficiencies occur during validation, these are analyzed and corrected in subsequent development cycles to ensure improved system performance of the ADAS / ADS system.

[0017] ADAS / ADS systems require a significantly higher number of verification test cases compared to conventional systems. This is due to the multitude of relevant scenarios, the high system complexity, and the networking with partner systems in the vehicle. To meet these challenges, the electrical and electronic components and systems are continuously developed in an iterative process. At defined points in time, known as "combined releases," all E / E components in the vehicle must have the latest hardware and software version. This ensures that the partner systems can always be tested with the latest hardware and software of the other systems.

[0018] System-level verification tests are typically, but not exclusively, performed in relation to a specific release version of the system. They serve to demonstrate that the system requirements are met with the respective release version and also help to identify potential performance deficiencies and malfunctions.

[0019] A key challenge in conducting these verification tests is that resources such as time, personnel, prototype vehicles, and hardware-in-the-loop capacity are limited for a given release stage. Due to the large number of required test cases, it is often impossible to perform all tests for every release stage, which directly impacts the test strategy and test prioritization.

[0020] German patent DE 10 2016 220 913 A1 discloses a method for generating test cases for autonomous vehicles. The test cases are automatically generated by analyzing data collected from vehicles participating in public road traffic. Additionally, simulated test cases based on real-world scenarios are generated. A comparative analysis is then performed between the real-world and simulated test cases, grouping comparable cases into a single case.

[0021] German patent DE 10 2022 132 922 A1 relates to a method for the virtual verification of automated driving functions in ADAS or ADS systems of a motor vehicle. For this purpose, a traffic scenario is modeled as a temporal sequence. A test agent creates verification-relevant test cases, which are executed using a simulation module. The simulation results are then analyzed by an evaluation module, which uses KPIs and evaluation parameters to determine the performance and safety of the driving function. The verification analysis is based on the evaluation results and is provided by an output module, particularly in the form of diagrams and histograms for verification reports.

[0022] German patent DE 10 2022 132 917 A1 concerns a method for determining the criticality and controllability of traffic scenarios within the framework of the virtual validation of automated driving functions in ADAS or ADS systems. A scenario describes an unfolding traffic event. A test agent generates targeted test cases based on a test strategy, each of which is assigned a relevant scenario. These are executed in a simulation module that models the driving environment, the vehicle, and the driving behavior. Subsequently, the simulation results are analyzed with regard to criticality and controllability. Based on this evaluation, a dataset of scenarios classified as critical is created and output in order to specifically identify safety-relevant situations.

[0023] DE 10 2022 107 338 B3 concerns a method for testing automated driving functions, which identifies critical scenario-specific parameter combinations using a high-performance, scalable test environment and selectively feeds these into a complex, trustworthy test environment in order to be able to decide on the release of the automated driving function based on the risk identified.

[0024] The invention is based on the objective of developing a system and a method that improves the efficiency of verification tests for sub-functions of ADAS / ADS systems in order to minimize the resource expenditure for testing combined releases, particularly with regard to time, personnel, prototype vehicles and hardware-in-the-loop capacities.

[0025] This problem is solved according to the invention with respect to a method by the features of claim 1, with respect to a system by the features of claim 9, with respect to an ADAS system by the features of claim 14, and with respect to a computer program product by the features of claim 15. The further claims relate to preferred embodiments of the invention.

[0026] The system according to the invention increases the efficiency and reliability of verification tests in the development of complex ADAS / ADS systems with a multitude of sub-functions. A significant advantage is that it is not necessary to perform tests with all test cases for every synchronization point. Instead, a specific set of test cases to be tested is identified for each synchronization point, leading to a significant increase in efficiency and reduced resource requirements. Additionally, the number of test cases per synchronization point is limited, and excess test cases are deferred to the next synchronization point.

[0027] However, if there is a significant shift in test cases, especially when high-priority functional groups are affected, this can be seen as an indicator that more test resources need to be provided for the respective synchronization point to ensure that critical functions are tested in a timely manner.

[0028] According to a first aspect, the invention provides a system for verifying partial functions of advanced driver assistance systems (ADAS) and / or automated driving systems (ADS) at synchronization points. A synchronization point is a predefined time or phase in the development process of an ADAS / ADS system at which all necessary changes to the system architecture, software, or hardware are integrated. At this synchronization point, specific test cases are executed to ensure that all relevant functional groups function correctly. The system includes an input module for entering available functional groups and synchronization points, in particular with specifications relating to the system architecture or software updates.a test case generation module configured to generate a set of test cases for testing and verifying an ADAS / ADS system; a function grouping module configured to divide the set of test cases into function groups, each function group representing a sub-function of the ADAS / ADS system; a prioritization module configured to assign priority classes to the function groups based on the probability of errors occurring and the effort required to correct them; a synchronization module configured to assign the corresponding function groups and the test cases associated with the function groups to the synchronization points; a test management module configured to consider the maximum number of test cases per synchronization point and, if this number is exceeded, to postpone test cases with a low priority class to subsequent synchronization points;and an output module configured to output the selected test cases for a synchronization point to increase the efficiency of verification tests and reduce resource requirements.

[0029] In a training course, the test case generation module is configured to automatically generate test cases based on parameterized scenarios with predefined scenario parameters and scenario parameter values.

[0030] In an advantageous embodiment, the functional groups include, among other things, a display concept, settings and activations, warning messages and error handling, sensors and perception including cameras, lidar, radar and ultrasound, signal processing between control units and actuators.

[0031] In another embodiment, the prioritization module is configured to use a weighted evaluation algorithm to weight the probability of errors occurring and the effort required to correct them differently in order to classify the functional groups into priority classes.

[0032] Advantageously, the synchronization module is configured to decide, based on dependencies between functional groups and software or hardware changes, which functional groups or test cases should be tested at a synchronization point.

[0033] In particular, the test management module is configured to determine, based on the available number of test resources such as hardware-in-the-loop test benches, test vehicles, and personnel, which test cases can be executed per synchronization point.

[0034] In another embodiment, the test management module is configured to adjust the priority class of test cases when these test cases are moved to a subsequent synchronization point due to limited resources.

[0035] In particular, the output module is configured to output the test cases in a machine-readable form in order to integrate them directly into an automated test system running on either Model-in-the-Loop (MiL), Software-in-the-Loop (SiL) or Hardware-in-the-Loop (HiL) test benches.

[0036] Advantageously, the output module is configured to generate electronic reports on the executed test cases and their results for later analysis and troubleshooting.

[0037] According to a second aspect, the invention provides a method for verifying partial functions of advanced driver assistance systems (ADAS) and / or automated driving systems (ADS) at synchronization points. A synchronization point is a predefined time or phase in the development process of an ADAS / ADS system at which all necessary changes to the system architecture, software, or hardware are integrated. At this synchronization point, specific test cases are executed to ensure that all relevant functional groups function correctly. The method comprises the following steps: - Entering available function groups and synchronization points, especially with specifications for system architecture or software updates; - Generating a set of test cases for testing and verifying an ADAS / ADS system; - Assigning test cases to functional groups that are used to verify specific sub-functions of the ADAS / ADS system; - Assigning priority classes to functional groups based on the probability of errors occurring and the effort required to correct them; - Assigning the function groups and the test cases linked to the respective function groups to synchronization points; - Managing the maximum number of test cases per synchronization point, with lower priority test cases being moved to later synchronization points if the maximum number is exceeded; - Adjusting the priority class of the moved function groups or test cases if these test cases are executed in a subsequent synchronization point; - Outputting the selected test cases to a synchronization point to increase the efficiency of verification tests and reduce resource requirements.

[0038] In a training course, it is planned that the test cases will be automatically generated based on parameterized scenarios with predefined scenario parameters and scenario parameter values.

[0039] In an advantageous embodiment, the functional groups include, among other things, a display concept, settings and activations, warning messages and error handling, sensors and perception including cameras, lidar, radar and ultrasound, signal processing between control units and actuators.

[0040] In another embodiment, it is provided that, based on the available number of test resources such as hardware-in-the-loop test benches, test vehicles, and personnel, it is determined which test cases can be executed per synchronization point.

[0041] Advantageously, the test cases are output in a machine-readable form in order to integrate them directly into an automated test system that runs on either Model-in-the-Loop (MiL), Software-in-the-Loop (SiL) or Hardware-in-the-Loop (HiL) test benches.

[0042] In particular, electronic reports are generated on the executed test cases and their results for later analysis and troubleshooting.

[0043] The third aspect of the invention provides an ADAS / ADS system, wherein the ADAS / ADS system uses the method according to the second aspect to verify at least one function of the ADAS / ADS system.

[0044] According to a fourth aspect, the invention provides a computer program product comprising an executable program code configured to perform the method according to the second aspect when executed.

[0045] The invention will now be explained in more detail with reference to an embodiment shown in the drawing.

[0046] This shows: Fig. 1 a block diagram to illustrate an embodiment of a system according to the invention; Fig. 2 a flowchart to explain the individual process steps of a process according to the invention; Fig. 3 a block diagram of a computer program product according to an embodiment of the third aspect of the invention.

[0047] Further features, aspects and advantages of the invention or its embodiments are explained in the following description in conjunction with the patent claims.

[0048] Fig. Figure 1 shows a system 100 according to the invention for verifying partial functions of advanced driver assistance systems (ADAS) and / or automated driving systems (ADS). The system 100 according to the invention comprises an input module 200 with a user interface, a database 370, a test case generation module 300, a function grouping module 400, a prioritization module 500, a synchronization module 700, and a test management module 800.

[0049] The input module 200, the test case generation module 300, the database 370, the function grouping module 400, the prioritization module 500, the synchronization module 700, and the test management module 800 can each be equipped with a processor and / or a memory unit. They can be at least partially integrated into a common processing unit.

[0050] In the context of the invention, a "module" is defined as a self-contained, specialized unit of software and / or hardware components. A module is designed to perform a specific function or task and is independent and self-contained; that is, it accepts specific inputs, performs internal processing, and then delivers specific outputs or results. A module can communicate with other modules or components via interfaces. These interfaces determine how data or commands are input into the module and how results or information are output. In particular, a module can be integrated into a cloud computing infrastructure.

[0051] In the context of the invention, a "processor" can be, for example, a machine or an electronic circuit. A processor can, in particular, be a central processing unit (CPU), a microprocessor, or a microcontroller, e.g., an application-specific integrated circuit or a digital signal processor, possibly in combination with a memory unit for storing program instructions, etc. A processor can also be a virtualized processor, a virtual machine, or a soft CPU. The processor is specifically configured to execute the program instructions in order to perform or implement the method according to the invention or a step thereof. It can also be provided that a processor comprises several modules.

[0052] In the context of the invention, a "storage unit" or "storage module" can refer, for example, to volatile memory in the form of random access memory (RAM), permanent storage such as a hard drive or data carrier, or, for example, a replaceable storage module. The storage module can also be a cloud storage solution.

[0053] In the context of the invention, "data" includes both raw data and already processed data from measurement results from sensors such as cameras, as well as from other data sources.

[0054] In particular, the test case generation module 300, the database 370, the function grouping module 400, the prioritization module 500, the synchronization module 700 and the test management module 800 can be integrated into a cloud computing infrastructure.

[0055] The term cloud computing infrastructure refers to the entirety of hardware, software, network, and storage resources used to support cloud computing services. Cloud computing infrastructure provides a platform for delivering a wide range of services, including data storage and management, application development and deployment, and virtual computing resources. It encompasses services for providing computing resources such as processing power and storage to support cloud-based applications. To protect data and applications from unauthorized access, security technologies such as firewalls, encryption, and identity and access management tools are employed. A cloud computing infrastructure can significantly increase the speed of data processing through high computing power, enabling faster computational results.

[0056] Furthermore, communication links are provided for the exchange and transmission of data between the individual modules, in particular as wireless communication links, e.g. as mobile communication links such as 5G or 6G, and / or as near-field communication links, e.g. Bluetooth. ® , Ethernet, NFC (Near Field Communication) or Wi-Fi ® , are trained. Additionally, cryptographic encryption methods can be provided for data exchange.

[0057] The test case generation module 300 generates a relevant number of test cases 350. For this purpose, the test case generation module 300 includes a software application for creating test cases 350, whereby the generated test cases 350 can be arranged in a test case space.

[0058] The creation of test cases 350 for testing and verifying a specific ADAS / ADS system is based on parameterized scenarios SZp i, which are also referred to as logical scenarios, and concrete scenarios SZc i A parameterized scenario SZp is defined within the scope of this invention as a scenario characterized by various scenario parameters P1, P2, ..., P n from a set of possible scenario parameters P i and associated scenario parameter values ​​PV1, PV2, ..., PV n from a set of possible scenario parameter values ​​PV i is defined, where both the scenario parameters P i as well as the associated scenario parameter values ​​PV i Not all of them are fixed. A concrete scenario SZc is a scenario in which the specific scenario parameters Pc are defined. i and the associated specific scenario parameter values ​​PVc i or value ranges of the specific scenario parameter values ​​PVc iare defined. Both scenarios, the parameterized scenario SZp and the concrete scenario SZc, are scenarios that are specifically written in machine-readable code. Scenario parameter P i within the scope of the present invention, characterize properties and features of a parameterized scenario SZp i .

[0059] An example of a scenario parameter P i a parameterized scenario SZp i This is the speed of an ego vehicle. For this scenario parameter "speed", the value range of the associated scenario parameter value PV can be set. i The range from 100 km / h to 180 km / h. For another parameterized scenario SZp k However, the range of values ​​for the scenario parameter "speed" can extend from 40 km / h to 70 km / h.

[0060] Scenario parameter P iFor a test case 350, a variety of physical, chemical, vehicle-related, and environmental parameters can be included. Examples of scenario parameters P i are: - Physical quantities such as torque, rotational speed, voltage, current, acceleration, velocity, braking value, direction, angle, radius, position and number. - Moving objects such as a vehicle, a person, or a cyclist. - Immovable objects such as buildings, trees, or other static obstacles. - Road configurations such as highways, traffic signs, traffic lights, tunnels, roundabouts, turning lanes or intersections. - Topographical structures such as inclines, declines or uneven road surfaces. - Traffic volume and associated traffic rules such as speed limits, right-of-way rules or stop signs. - Time parameters such as time of day, season and traffic volume. - Weather conditions such as precipitation, snow, black ice, fog or rain intensity. - Lighting conditions such as daylight, twilight, darkness or blinding sunlight. - Road surface conditions such as wet, dry, icy or unpaved road surface. - Road markings such as solid or dashed lines, pedestrian crossings. - Road surfaces such as asphalt, concrete or cobblestones. - Vehicle types such as cars, trucks, motorcycles or buses, and their condition, such as tire pressure or brake wear. - Communication signals such as vehicle-to-vehicle (V2V) or vehicle-to-infrastructure (V2I) communication. - Environments such as urban, rural, or industrial environments. - Driver conditions, e.g. attentive or distracted drivers, as well as fatigue.

[0061] These scenario parameters P iThey contribute to creating realistic and diverse test conditions for the verification of advanced driver assistance systems (ADAS) and automated driving systems (ADS). They make it possible to consider a wide range of influences to ensure that an ADAS / ADS system functions reliably even in complex and demanding environments.

[0062] For the creation of parameterized scenarios SZp k Various sources can be used, such as requirement specifications, expert knowledge, and / or measurements in public road traffic or on test tracks using sensors. The sensors used can include, in particular, radar systems with at least one radar sensor, LiDAR systems for optical distance and speed measurement, 2D / 3D image-capturing cameras in the visible, IR, and / or UV range, ultrasound systems, and / or GPS systems.

[0063] These parameterized scenarios SZp are generated from various data sources k They are stored in database 370. Scenario parameters P are also stored in database 370. i , Scenario parameter values ​​PV i concrete scenarios SZc i , specific scenario parameters PC i , specific scenario parameter values ​​PVc i and possibly other information is stored.

[0064] The software application of the test case generation module 300 includes, in particular, a test agent that uses a test strategy to generate suitable parameterized scenarios SZp. i , specific scenario parameters PC i as well as specific scenario parameter values ​​PVc i and, if necessary, selects further information and creates 350 test cases from it to test, apply, verify and validate the ADAS / ADS system.

[0065] The test strategy used is determined by the purpose of the simulation, i.e., what insights into the behavior of the ADAS / ADS system or a specific driving function are to be gained during the simulation. An example of a simulation task is the simulation of an overtaking maneuver on a highway (cut-in scenario).

[0066] The respective test strategy can be entered by a user via input module 200, e.g., in the form of text or voice messages. Input module 200 contains an artificial intelligence-based language model that converts these text messages into a format suitable for test case generation module 300.

[0067] For example, in test case 350, the specific scenario parameters to be varied are Pc i selected depending on the specific use case. Other specific scenario parameters PC iare kept constant, for example scenario parameters PC i , which describe the vehicle when only one vehicle derivative is considered for the test.

[0068] The test cases 350 determined in this way are stored in database 370. The data for a test case 350 can include a test identification number (test ID), a username, a creation time, and the name of the selected parameterized scenario Szp. i Included. Additionally, the specific scenario parameters PC are included. i saved, which contains the parameterized scenario Szp i into a specific scenario Szc i and thus transform into a test case 350.

[0069] The Function Grouping Module 400 serves to systematically structure a set of test cases 350 by subdividing them into specific function groups 450. The goal of the Function Grouping Module 400 is to ensure that each sub-function of a complex ADAS / ADS system can be verified separately and in detail by the test cases 350 of the specific function groups 450. The test cases 350 are organized according to specific functions and characteristics, with each function group 450 covering a specific sub-function of the ADAS / ADS system. In this way, the Function Grouping Module 400 ensures that all relevant functions of the ADAS / ADS system are tested accurately and comprehensively. The function groups 450 include, in particular, the following aspects: - Display concept: Verification of the presentation of information on displays, screens or other interfaces. - Setting and Activation: Test cases 350 examine the correct function of settings and activations of sub-functions. This group includes, for example, test cases that ensure the brake function can be correctly set and activated. These may include tests that verify whether the braking process is initiated correctly, either manually or automatically, whether different braking modes such as an emergency braking function work, and whether all relevant settings, such as brake force adjustment, are correctly implemented. - Warning messages and error handling: Checking system responses to error messages and handling error conditions. - Sensors and Perception: Verification of correct sensor data processing and its impact on the ADAS / ADS system. Sensors collect information from the environment and provide it to the ADAS / ADS system, which makes decisions based on this data. The verification in this functional group 450 ensures that sensors such as cameras, lidar, radar, and ultrasound function correctly, that sensor data is accurately acquired and processed, and that the ADAS / ADS system operates reliably based on this sensor data. Examples of sensors and their verification are given below: a) Camera sensors: A camera detects traffic signs or road markings. Test cases 350 verify whether the camera correctly recognizes the signs under various conditions (e.g., rain, darkness, glare from the sun) and correctly transmits the data to the control unit. It is also checked whether the software correctly interprets the image data and initiates appropriate measures (e.g., speed adjustment). b) Lidar (Light Detection and Ranging): A lidar sensor is used for distance measurement or 3D environmental perception. Test cases 350 verify the accuracy of distance measurement in various scenarios, such as with moving objects, obstacles, or adverse weather conditions. Verification checks whether the ADAS / ADS system correctly processes the acquired distance data and triggers appropriate control commands (e.g., braking or lane changes). c) Radar sensors: A radar sensor is used to detect vehicles in the blind spot or for adaptive cruise control. Test cases 350 ensure that radar sensors reliably detect static and moving objects even in poor visibility (e.g., fog, rain). Verification checks whether the received data leads to safe control of the ADAS / ADS system, such as initiating a braking maneuver when the distance is too short. d) Ultrasonic sensors: Ultrasonic sensors are used to detect obstacles during parking. Verification checks whether the ultrasonic sensor reliably detects obstacles near the vehicle and provides the driver (or an autonomous parking system) with the correct data to ensure safe parking. Test cases 350 verify the sensor's performance with different object types and distances. - Signal processing (between control units): Verification that communication between different control units is correct and error-free. Signal processing (between control units) refers to the communication and data exchange between different electronic control units (ECUs) in a vehicle. Each control unit performs specific tasks, such as controlling the engine, brakes, or sensors. The verification in this functional group 450 ensures that the various control units of an ADAS / ADS system communicate with each other flawlessly and in a timely manner. This ensures that critical functions such as brakes, power steering, or sensor processing operate reliably, which is crucial for the safety and performance of the vehicle in which the ADAS / ADS system is used.The following aspects are particularly important for verifying signal processing: a) Correct signal transmission: In modern ADAS / ADS systems, signals between control units are often transmitted via various bus systems (e.g., CAN bus, FlexRay, Ethernet). Test cases 350 in this functional group 450 verify whether the signals are correctly sent from a control unit and correctly interpreted and processed by the receiving control units. For example, the camera detects a traffic sign, and the camera control unit sends the information about the detected speed limit to the powertrain control unit, which adjusts the speed. The verification checks whether the signal was transmitted without errors and processed correctly. b) Avoiding signal loss or delays: Test cases 450 verify that communication between the control units occurs without delays, signal loss, or interference. Particularly in safety-critical systems such as brakes or the control of automated driving maneuvers, it is important that signals are transmitted in real time and without errors. For example, in an emergency braking assistant, sensor data must be transmitted to the braking system immediately to initiate braking in a timely manner. c) Timing of control units: Control units often operate simultaneously or in dependency on each other. Verification ensures that control units are correctly time-synchronized and process data in the correct chronological order. For example, if the control unit for the turn signal indicator and the control unit for the ADAS / ADS system are active at the same time, it must be ensured that communication is correctly synchronized and no conflicting signals are sent. d) Fault detection and handling: Another key aspect of verification is ensuring that the ADAS / ADS system reacts appropriately to communication errors between the control units. This may include activating fallback systems or issuing warnings. For example, if the communication network between the control units is interrupted, safety measures such as activating an emergency module or triggering a warning for the driver must be initiated. Actuator technology: Verification of the correct functioning of actuators and their responses to control commands. Actuator technology deals with the testing of actuators, i.e., the components that perform physical actions based on control commands. Actuators convert electrical signals received from control units into mechanical movements, forces, or other physical actions. In an ADAS / ADS system, they play a key role in performing functions such as steering, braking, or acceleration. The following aspects are particularly important when verifying actuators: a) Correct execution of control commands: Test cases 450 verify whether the actuators execute the commands from the control units correctly and within the expected time. This applies particularly to safety-relevant functions such as braking or steering. For example, if the control unit issues a command to initiate emergency braking, the brake actuator must immediately and completely implement the signal so that the braking force is built up correctly. b) Accuracy and response speed: A key component of the verification process is checking the accuracy and speed with which the actuators respond to control commands. This involves verifying whether the actuator reacts quickly enough to the signal and executes the required movement or action precisely. For example, in a lane keeping assist system, the steering actuator must keep the vehicle precisely and promptly in its lane, without delay or deviation from the desired movement. c) Overload and long-term behavior: In test cases 450, it is verified whether the actuators function flawlessly even under load (e.g., in extreme heat, cold, or intensive use) and show no failures or deviations. This type of verification is important to ensure that the actuators function reliably even under difficult operating conditions. For example, a brake actuator must function flawlessly even during repeated hard braking (e.g., when driving downhill) and must not overheat or lose braking power. d) Self-diagnosis and fault messages: Modern actuators are often equipped with sensors that monitor their condition and report problems to the control unit. Test case 450 verifies whether the actuator sends correct fault messages to the control unit in the event of malfunctions such as mechanical defects or sensor problems. For example, if a steering actuator has a mechanical defect, the ADAS / ADS system should be able to detect this defect and issue a warning to the driver. e) Redundancy and Fallback Mechanism: In safety-critical systems, it is important that a fallback mechanism is activated in the event of an actuator failure. Test Case 450 verifies that the ADAS / ADS system is capable of taking emergency measures in the event of an actuator failure, such as activating a secondary actuator or shutting down the ADAS / ADS system to a safe state. For example, if an actuator critical for autonomous driving fails, the ADAS / ADS system should either switch to a redundant actuator or ensure that the vehicle comes to a safe stop.

[0070] This clear structuring of test cases 350 into functional groups 450 enables targeted verification of each individual sub-function of the ADAS / ADS system and ensures that no important aspects are overlooked. It also facilitates the traceability and maintenance of the test structure, as changes to individual components can be specifically tested and validated.

[0071] Various algorithms can be used to group the 350 test cases in the 400 function grouping module, based on both machine learning and rule-based methods. The choice of the appropriate algorithm depends on the structure of the test data for the 350 test cases and the available resources. Possible algorithms include: - K-Means Clustering: A clustering method in which test cases are grouped based on similarities in input parameters or expected results. K-Means is particularly useful when it comes to clustering test cases based on numerical or vector features. - Rule-based algorithms: Here, test cases 350 are assigned to function groups 450 based on predefined rules. For example, test cases containing specific keywords such as "signal processing" or "actuator" can be directly assigned to the corresponding function group 450. - Language models: For test cases 350 that use text-based descriptions, artificial intelligence algorithms such as Natural Language Processing (NLP) techniques can be used to extract relevant keywords and context information and assign the test cases 350 to the functional groups 450.

[0072] Prioritization Module 500 evaluates and ranks the various functional groups 450 according to two main criteria: the probability of errors occurring and the cost or effort of error correction for the different functions of an ADAS / ADS system. The goal of Prioritization Module 500 is to identify those functional groups 450 where errors are most likely to occur or whose correction is particularly costly, in order to assign them a higher priority in the testing and verification process.

[0073] Function groups 450, which have a higher probability of errors occurring, are assigned to a higher priority class 550. This probability can be based on historical data, known weaknesses in the technology, or the complexity of the function in question. For example, a function group 450 that monitors communication between many control units with regard to signal processing might be classified as more error-prone because timing and signal processing are complex processes.

[0074] Function groups 450, whose troubleshooting is associated with high costs or effort, are also given higher priority. This can refer to the financial costs, the time required, or the potential damage to the ADAS / ADS system and other vehicle systems. For example, brake function faults that are safety-relevant and incur high development costs are given high priority due to their serious consequences and the cost of their correction.

[0075] The prioritization module 500 can use various algorithms for prioritizing function groups 450: - K-Means Clustering: The K-means clustering algorithm can be used to group Function Groups 450 into clusters based on their properties. These properties can include, for example, the probability of failure or the cost of troubleshooting. The algorithm groups Function Groups 450 with similar values ​​into different clusters, allowing Function Groups 450 with comparable risks and costs to be considered together.

[0076] After applying K-means clustering, clusters are assigned a priority class of 550 based on the specific characteristics of the contained functional groups (450). Functional groups (450) that fall into a cluster with a high probability of failure and / or high cost of error correction are classified as high priority. These functional groups (450) receive a higher priority during testing.

[0077] The algorithm forms clusters that are subsequently assigned to a priority class of 550. To do this, the cluster properties can be analyzed and classified into high, medium, or low priority according to predefined rules. - Weighted Assessment: Each Function Group 450 is assessed using a points system based on various criteria. In this case, the two main criteria are the probability of a fault occurring and the cost of correcting the fault. Each of these criteria is assigned a weight that reflects its relative importance. For example, the probability of a fault occurring may be weighted more heavily than the cost of correcting the fault if the frequency of the fault is assumed to be more severe than the cost of correcting it.

[0078] Points are then awarded on a scale (e.g., 1 to 10) for each criterion. The score for each functional group (450) is then calculated by multiplying the score by the corresponding weighting. The functional groups (450) with the highest total score (based on the weighted score) receive the highest priority and are selected for the verification process.

[0079] Synchronization module 700 monitors the assignment of function groups 450 to the respective synchronization points 750, which arise during the integration of hardware or software changes into the ADAS / ADS system. A synchronization point 750 represents a common, combined release in which new functions, updates, or bug fixes are implemented. Synchronization module 700 ensures that only the test cases 350 relevant to the respective synchronization point 750 are scheduled, specifically those affected by the changes in the function groups 450 assigned to that synchronization point 750. This eliminates irrelevant test cases 350 and optimizes test resources.

[0080] If changes are made to hardware or software components at a synchronization point 750 that affect a specific functional group 450, the synchronization module 700 automatically schedules the corresponding test cases 350 for this synchronization point 750. This ensures that every change is properly verified. For example, if a new braking function is implemented, the synchronization module 700 assigns the test cases 350 for the brake control (setting and activation, actuators) to this synchronization point 750.

[0081] For function groups 450 that are not affected by changes at a synchronization point 750, the synchronization module 700 does not provide any test cases 350. These function groups 450 are only tested if direct or indirect dependencies exist. For example, if changes are made at a synchronization point 750 only to the steering software, and these changes have no impact on the sensors, no test cases 350 for the sensors are assigned to this synchronization point 750.

[0082] The synchronization module 700 uses a rule-based algorithm to assign the function groups 450 to the respective synchronization points 750. Predefined rules are used to determine which function groups 450 are affected by a specific change (e.g., a software update or bug fix). These rules take into account dependencies and interfaces between the function groups 450 and the system changes.

[0083] Based on these rules, the synchronization module 700 decides which test cases 350 must be assigned to a synchronization point 750. For example, a rule could specify that changes to the braking function also affect the actuators. In this case, the test cases 350 of both function groups 450 are automatically assigned to the corresponding synchronization point 750.

[0084] Furthermore, artificial intelligence algorithms such as neural networks can also be used. The neural network is trained with suitable training datasets to learn the dependencies and interfaces between the functional groups 450 and the system changes.

[0085] At a synchronization point 750, only a limited number of tests with test cases 350 can be performed. This limit depends on available resources such as test vehicles, test personnel, hardware-in-the-loop (HiL) time, or test benches. The test management module 800 takes these resource limitations into account and controls the test planning accordingly. In this way, the test management module 800 ensures that test resources are used optimally in a development process by limiting the number of test cases to a maximum of 350 for each synchronization point 750.

[0086] When the maximum number of test cases (350) for a synchronization point (750) is reached, the test management module (800) moves the test cases (350) with the lowest priority class (550) to the next synchronization point (755). There, these functional groups (450) are rescheduled to ensure that all essential sub-functions of the ADAS / ADS system are tested.

[0087] By monitoring the number of test cases 350 for each synchronization point 750 and postponing test cases 350 with a lower priority class 550 to the next synchronization point 755 when the maximum permissible number of test cases is exceeded, the test management module 800 increases the efficiency of verification tests and reduces resource requirements. For example, if test resources are scarce and the test cases 350 for the braking function have a higher priority, the test cases 350 for less critical functions, such as the comfort control, are postponed to the next synchronization point 755.

[0088] Additionally, the test management module 700 dynamically adjusts the priority classes 550. Since postponed test cases 350 are tested too late at a synchronization point 750, the test management module 800 increases their priority class 550 for the next synchronization point 755. This ensures that these test cases 350 are tested with higher urgency and that no critical tests are postponed indefinitely. For example, a test case 350 for a diagnostic function that was postponed due to resource constraints receives a higher priority for the next synchronization point 755 to ensure that it is executed.

[0089] The test management module 800 can use a dynamic programming algorithm to plan the most efficient distribution of test cases 350 across multiple synchronization points 750 and 755. This develops a long-term strategy to ensure that the test cases 350 are evenly distributed and that all sub-functions of the ADAS / ADS system are tested within an acceptable timeframe, thus enabling efficient and resource-saving test planning.

[0090] Output module 900 is responsible for outputting the test cases 950 to be executed at a specific synchronization point 750. It ensures that the relevant test cases 950 are provided via an interface in a suitable format for manual or automated processing. Output is provided in various formats, depending on the requirements of the test system or test environment.

[0091] In particular, output module 900 can output the selected test cases 950 in list form. These lists contain the description of the selected test cases 950, including the preconditions, the actions to be performed, and the expected results. These lists can be provided either electronically or in printed form, depending on which format is most suitable for the test engineer.

[0092] The output module 900 can also output the selected test cases 950 in a machine-readable format that can be processed directly by automated test systems. Such test systems include, for example, Model-in-the-Loop (MiL), Software-in-the-Loop (SiL), or Hardware-in-the-Loop (HiL) test benches. This output format enables the direct integration of the selected test cases 950 into automated test sequences and significantly accelerates the testing process, as no manual input or configuration is required.

[0093] Each selected test case 950 typically contains a structured description of the preconditions, such as the state of the ADAS / ADS system or driving environment, the actions to be performed during the test, and the expected results to verify that the ADAS / ADS system is responding correctly. This test case description ensures that the tests are traceable and reproducible.

[0094] Model-in-the-Loop (MiL) is a testing method in which a mathematical model of the system under test is tested before the software is developed or implemented on the target hardware. It is performed in a simulation environment and enables early testing and verification of the system logic. MiL test benches simulate the behavior of software components or control units using a model, allowing for the early detection of errors.

[0095] Software-in-the-Loop (SiL) testing uses the actual software (without hardware) in a simulated environment. This allows for the verification of algorithms and software logic before the software is transferred to the target hardware. Test cases 350 are performed directly on the software to detect errors or inconsistencies early on.

[0096] Hardware-in-the-Loop (HiL) testing extends the testing process by integrating the actual hardware (e.g., control units) into the test loop. The ADAS / ADS system is tested on a test bench with real or simulated sensor signals and actuators to verify its behavior in a realistic environment. This type of testing is particularly important to ensure that the software interacts correctly with the hardware and that the ADAS / ADS system functions correctly in real-world use.

[0097] In addition to the aforementioned test benches, other automated test systems can also be used to execute the selected test cases 950 directly and without human intervention. Such automated test systems can support continuous integration and enable tests to be performed regularly and automatically after each code change or software version, significantly improving quality assurance in complex projects.

[0098] In Fig. Figure 2 shows the procedure steps for verifying sub-functions of ADAS / ADS systems at synchronization points 750 and 755.

[0099] In step S10, available function groups 450 and synchronization points 750, especially with specifications for system architecture or software updates, are entered.

[0100] In step S20, a set of test cases 350 is generated for testing and verifying an ADAS / ADS system.

[0101] In step S30, the test cases 350 are assigned to function groups 450, which are used to verify specific sub-functions of the ADAS / ADS system.

[0102] In step S40, priority classes 550 are assigned to the functional groups 450 based on the probability of errors occurring and the effort required to correct them.

[0103] In step S50, the synchronization points 75, 755 are assigned the function groups 450 and the test cases 350 linked to the respective function groups 450.

[0104] In step S60, the maximum number of test cases 350 per synchronization point 750 is managed, with test cases 350 with low priority being moved to later synchronization points 755 if the maximum number is exceeded.

[0105] In step S70, the priority class 550 of the moved function groups 450 or test cases 350 is adjusted if these test cases 350 are used in a subsequent synchronization point 755.

[0106] In step S80, the selected test cases 950 are output for a synchronization point 750 to increase the efficiency of the verification tests and reduce the resource requirements.

[0107] Fig. Figure 3 schematically represents a computer program product 1000 comprising an executable program code 1050 configured to execute the method according to the first aspect of the present invention.

[0108] The system 100 according to the invention reduces the effort and costs for verifying sub-functions of an ADAS / ADS system through optimized planning and prioritization of test cases 350. Only tests with the test cases 350 necessary for verifying a sub-function are performed, thereby enabling more efficient use of test resources and a more structured testing process. This leads to an acceleration of verification, since not all test cases 350 are tested for every synchronization point 750, but rather only the relevant, selected test cases 950 are used in a targeted and prioritized manner. This allows for a dynamic adaptation of test resources to the respective different requirements of the various sub-functions of an ADAS / ADS system. Reference sign 100 System 200 Input module 300 Test Case Generation Module 350 test cases 370 database 400 Function grouping module 450 Functional group 500 Prioritization Module 550 priority class 700 Synchronization module 750 Synchronization point 755 next synchronization point 800 Test Management Module 900 Output module 950 selected test cases 1000 computer program product 1050 program code

Claims

System (100) for verifying partial functions of advanced driver assistance systems (ADAS) and / or automated driving systems (ADS) at synchronization points (750, 755), wherein a synchronization point (750, 755) is a predefined point in time or phase in the development process of an ADAS / ADS system at which all necessary changes to the system architecture, software or hardware are integrated, and wherein specific test cases (350) are executed at this synchronization point (750, 755) to ensure that all relevant functional groups (450) function properly, comprising an input module (200) for inputting available functional groups (450) and synchronization points (750), in particular with specifications relating to the system architecture or software updates;a test case generation module (300) configured to generate a set of test cases (350) for testing and verifying an ADAS / ADS system; a function grouping module (400) configured to divide the set of test cases (350) into function groups (450), each function group (450) representing a sub-function of the ADAS / ADS system; a prioritization module (500) configured to assign priority classes (550) to the function groups (450) based on the probability of errors occurring and the effort required to correct them; a synchronization module (700) configured to assign the corresponding function groups (450) and the test cases (350) associated with the function groups (450) to the synchronization points (750, 755);a test management module (800) configured to consider the maximum number of test cases (350) per synchronization point (750) and, if this number (350) is exceeded, to move test cases (350) with a low priority class (550) to subsequent synchronization points (755); and an output module (900) configured to output the selected test cases (950) for a synchronization point (750) to increase the efficiency of verification tests and reduce resource requirements. System (100) according to claim 1, wherein the test case generation module (300) is configured to automatically generate test cases (350) based on parameterized scenarios (SZpi) with predefined scenario parameters (Pi) and scenario parameter values ​​(PVi). System according to claim 1 or 2, wherein the functional groups (450) include, among other things, a display concept, settings and activations, warning messages and error handling, sensors and perception including cameras, lidar, radar and ultrasound, signal processing between control units and actuators. System (100) according to one of claims 1 to 3, wherein the prioritization module (500) is configured to weight the probability of errors occurring and the effort required to correct them differently using a weighted evaluation algorithm in order to classify the functional groups (450) into priority classes (550). System (100) according to one of claims 1 to 4, wherein the synchronization module (700) is configured to decide, based on dependencies between functional groups (450) and software or hardware changes, which functional groups (450) or test cases (350) are to be tested at a synchronization point (750, 755). System (100) according to one of claims 1 to 5, wherein the test management module (800) is configured to determine, based on the available number of test resources such as hardware-in-the-loop test benches, test vehicles, personnel, which test cases (350) can be executed per synchronization point (750, 755). System (100) according to one of claims 1 to 6, wherein the test management module (800) is configured to adjust the priority class (550) of test cases (350) when these test cases (350) are moved to a subsequent synchronization point (755) due to limited resources. System (100) according to any one of claims 1 to 7, wherein the output module (900) is configured to output the selected test cases (950) in a machine-readable form in order to integrate them directly into an automated test system running on either Model-in-the-Loop (MiL), Software-in-the-Loop (SiL) or Hardware-in-the-Loop (HiL) test benches; and wherein the output module (900) is configured to generate electronic reports on the executed test cases (950) and their results for subsequent analysis and troubleshooting. Method for verifying sub-functions of advanced driver assistance systems (ADAS) and / or automated driving systems (ADS) at synchronization points (750, 755), wherein a synchronization point (750, 755) is a predefined point in time or phase in the development process of an ADAS / ADS system at which all necessary changes to the system architecture, software, or hardware are integrated, and wherein specific test cases (350) are executed at this synchronization point (750, 755) to ensure that all relevant functional groups (450) function correctly, comprising the procedure steps of: - Inputting (S10) available functional groups (450) and synchronization points (750), in particular with specifications relating to the system architecture or software updates; - Generating (S20) a set of test cases (350) for testing and verifying an ADAS / ADS system;- Assigning (S30) the test cases (350) to function groups (450) used to verify specific sub-functions of the ADAS / ADS system; - Assigning (S40) priority classes (550) to the function groups (450) based on the probability of errors occurring and the effort required to correct them; - Assigning (S50) the function groups (450) and the test cases (350) associated with the respective function groups (450) to synchronization points (750, 755); - Managing (S60) the maximum number of test cases (350) per synchronization point (750), whereby lower-priority test cases (350) are moved to later synchronization points (755) if the maximum number is exceeded; - Adjusting (S70) the priority class (550) of the moved function groups (450) or test cases (350) if these test cases (350) will be executed at a subsequent synchronization point (755);- Output (S80) of the selected test cases (950) to a synchronization point (750) to increase the efficiency of verification tests and reduce resource requirements.; Method according to claim 9, wherein the test cases (350) are automatically generated based on parameterized scenarios (SZpi) with predefined scenario parameters (Pi) and scenario parameter values ​​(PVi). Method according to claim 9 or 10, wherein the functional groups (450) include, among other things, a display concept, settings and activations, warning messages and error handling, sensor technology and perception including cameras, lidar, radar and ultrasound, signal processing between control units and actuator technology. Method according to one of claims 9 to 11, wherein, based on the available number of test resources such as hardware-in-the-loop test benches, test vehicles, personnel, it is determined which test cases (350) can be executed per synchronization point (750, 755). A method according to any one of claims 9 to 12, wherein the selected test cases (950) are output in a machine-readable form for direct integration into an automated test system running on either Model-in-the-Loop (MiL), Software-in-the-Loop (SiL) or Hardware-in-the-Loop (HiL) test benches; and wherein electronic reports on the executed test cases (950) and their results are generated for subsequent analysis and troubleshooting. ADAS / ADS system, wherein the method according to one of claims 9 to 13 is used to verify at least one function of the ADAS / ADS system. Computer program product (1000) comprising an executable program code (1050) configured to perform the method according to any one of claims 9 to 13 when executed.

Images