AI Safety Function Testing With Dynamic Vehicle Scenario Inputs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for testing AI-based safety functions in vehicle control systems, such as obstacle detection, face challenges in generating universally usable input data sets to ensure the functions' reliability across various operational situations, especially when the AI has not been trained for specific scenarios.
Innovation Solution
A device and method that include a test data generator to create dynamic test input data based on current environmental and operational conditions, using AI techniques like deep learning, to assess the AI-based safety function's performance in object recognition, ensuring it can recognize objects accurately even in untrained situations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional safety monitoring functions use predefined test input data, then the testing process is simple and universally applicable, but the testing cannot adequately assess AI-based safety functions in specific application situations and untrained scenarios
Solution Approach 1:
The patent creates a copy of the AI-based safety function that is functionally equivalent to the original. This copy is used for testing purposes, allowing the generation of application-specific test data without affecting the actual safety function. The copy receives test inputs and produces test outputs that can be evaluated against expected results, enabling tailored testing for different scenarios while maintaining the integrity of the original system.
Solution Approach 2:
The test data generator dynamically modifies parameters of test input data based on specific application situations and environmental conditions. Instead of using fixed predefined test data, the system adjusts parameters such as object characteristics, environmental factors, and sensor conditions to match the current operational context, thereby achieving adaptability to different scenarios.
2Reliability
If AI-based safety functions are tested with dynamically generated test data based on current usage situations, then the reliability assessment in untrained scenarios is improved, but the complexity of the testing system increases
Solution Approach 1:
The patent introduces a test data generator as an intermediary component that bridges the gap between the AI-based safety function and the testing requirements. This generator creates application-specific test inputs based on current usage situations and environmental data, enabling reliable assessment of the safety function in untrained scenarios without requiring direct modification of the AI model or manual test case creation.
Solution Approach 2:
The testing system implements feedback mechanisms where test outputs from the AI-based safety function (or its copy) are evaluated against expected results. This feedback loop allows the system to assess reliability in real-time, identify deficiencies in untrained scenarios, and potentially adjust future test data generation to cover previously untested situations, thereby continuously improving reliability assessment.
3Reliability
If comprehensive test coverage for all possible situations is achieved, then the safety function is sufficiently tested, but the amount of test data and testing time required becomes prohibitively large
Solution Approach 1:
The patent employs dynamic test data generation that adapts to current usage situations rather than exhaustively testing all possible scenarios. The test data generator creates relevant test cases based on the current operational context, environmental conditions, and identified risk factors, achieving effective test coverage with significantly reduced testing time compared to comprehensive exhaustive testing.
Data Source
Figure 1
Figure 2
AI summary
The invention relates to a device (100) for testing a safety function of a vehicle control system, wherein the safety function (202) is based on artificial intelligence (AI) and is suitable for object recognition of an object in the vehicle's environment depending on environmental data of the vehicle.The device (100) comprises: - an interface (101) configured to read environmental data (D1) of the vehicle acquired by means of vehicle sensors, - a test data generator (102) configured to generate test input data (D2) for a copy of the safety function (103) depending on the environmental data (D1), - the copy of the safety function (103) which is functionally identical to the safety function (202) of the vehicle, - an analysis unit (104) configured to provide the test input data (D2) of the copy of the safety function of the control system and to execute the copy of the safety function (103) depending on the test input data (D2) to generate test output data (D3), and - a test unit (105) configured to test the test output data (D3) of the copy of the safety function based on a predefined test criterion and to output a corresponding test result (PE).