Camera-Based Control Training With Projected Virtual Traffic
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Training AI algorithms for automated driving functions is challenging due to the risks in public roads and the inability of closed tracks to simulate complex traffic scenarios realistically, while pure simulation lacks realism.
Innovation Solution
A method and system that combine real-world testing with traffic simulation by reflecting the vehicle's position in a simulation environment, projecting virtual objects into the camera's field of view, and using reinforcement learning to evaluate the control system's response.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If training is conducted in public road spaces, then real-world traffic scenarios can be tested, but safety risks to drivers and other road users increase
Solution Approach 1:
The patent uses projection systems to create visual copies of virtual traffic objects (pedestrians, vehicles, signs) that appear in the camera's field of view during real-world driving. This allows the AI to train on realistic-appearing scenarios without actual physical risks, as the projected images are merely visual representations rather than real objects
Solution Approach 2:
The projection system acts as an intermediary between the virtual simulation environment and the real-world camera input. It translates virtual objects into visual projections that the camera can capture, creating a bridge that combines virtual scenario generation with real-world camera-based training while maintaining safety
2Object-affected harmful factors
If training is conducted on closed test tracks, then safety is ensured, but complex traffic scenarios cannot be adequately tested
Solution Approach 1:
The system dynamically generates diverse traffic scenarios by combining the vehicle's real-time position data with a simulation environment that can create various traffic situations. This allows the training to adapt to different scenario types (pedestrians, vehicles, signs) while maintaining the safety of closed track testing
Solution Approach 2:
The patent adds a virtual dimension to the physical test track environment by projecting visual representations of virtual objects onto the real-world scene. This creates a hybrid training environment that combines the safety of physical tracks with the scenario diversity of virtual simulations
3Object-affected harmful factors
If pure simulation methods are used, then training can be conducted safely, but sufficient realism is lacking due to absence of real-world driving physics
Solution Approach 1:
The patent merges pure simulation methods with real-world camera-based testing by combining virtual scenario generation with actual camera footage captured during real driving on test tracks. This hybrid approach integrates the safety and scenario diversity of simulation with the realism and physical accuracy of real-world camera input
Data Source
Figure 1~2
Figure 3~4
AI summary
The invention relates to a method and a training system for training a camera-based control system (10) of a motor vehicle (1). The training system comprises a motor vehicle (1) with a camera-based control system (10) to be trained and with a paired camera (11); a simulated environment (2) for providing a traffic simulation (20); and at least one projection system (3). The simulated environment (20) is connected to the camera-based control system (10) and the projection system (3) for the purpose of data transmission such that a temporal and geographical position of the motor vehicle (1) driving on an area (4) is reproduced in the traffic simulation (20). The traffic simulation (20) is designed to generate training scenarios comprising virtual object on the basis of the respective reproduced position of the motor vehicle and additionally a graphical display for each of the simulated virtual objects. The at least one projection system (3) is designed to project the generated graphical displays into the field of view of the camera (11). The control system (10) to be trained is designed to process an analysis result (25) which analyzes the reaction of the control system (10) to each training scenario.