Vehicle AR Overlay Failover Using Hypervisor Virtual Machines
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing in-vehicle augmented reality systems experience delays in providing graphical overlays, which can increase safety risks during vehicle travel.
Innovation Solution
A signal processing device that includes a memory and processor configured to execute multiple virtual machines on a hypervisor, detect objects from camera data, and generate graphical overlays using a shared memory to rapidly and stably provide augmented reality-based information, including vehicle speed, heading direction, and preceding vehicle data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If traditional single-system architecture is used for processing camera data and generating graphical overlays, then device complexity is low, but processing speed and responsiveness are insufficient leading to safety delays
Solution Approach 1:
The system divides the processing architecture into multiple independent virtual machines (first VM for object detection, second VM for graphical overlay generation, third VM for guide information generation) that operate in parallel. Each VM handles specific processing tasks independently, enabling concurrent execution and improving overall processing speed without requiring a complete system redesign.
Solution Approach 2:
The patent introduces a temporal dimension to the processing architecture by implementing multi-frame processing capabilities. The system can process multiple camera frames simultaneously across different virtual machines, allowing predictive processing of future frames and compensating for processing delays through temporal redundancy and prediction algorithms.
2Reliability
If multiple virtual machines are executed on a hypervisor to process camera data rapidly, then processing speed and stability improve, but device complexity increases
Solution Approach 1:
The hypervisor serves as an intermediary layer that manages resource allocation and communication between multiple virtual machines. It provides standardized interfaces for inter-VM data transfer, handles memory management, and ensures isolated execution environments, thereby improving system stability and reliability while abstracting the complexity from the application layer.
Solution Approach 2:
The system creates virtual copies of processing environments through virtual machines, allowing the same processing logic to be replicated across multiple isolated instances. This enables parallel processing of camera data while maintaining consistent and reliable execution patterns, improving both speed and stability through redundancy.
3Loss of time
If real-time object detection and graphical overlay generation are performed concurrently, then responsiveness improves, but processing stability may deteriorate due to resource contention
Solution Approach 1:
The system segments concurrent processing tasks into separate virtual machines with dedicated resource allocations. The first VM handles object detection, the second VM generates graphical overlays, and the third VM produces guide information. This segmentation prevents resource contention by ensuring each task has guaranteed computational resources, maintaining stability while achieving real-time responsiveness.
Solution Approach 2:
The system implements continuous processing pipelines within each virtual machine that maintain steady state operation. Camera data flows continuously through detection, overlay generation, and guide information creation without interruption or batch processing, ensuring consistent response times and stable processing performance under varying load conditions.
Data Source
AI summary
A signal processing device and an augmented reality apparatus for a vehicle including the same are disclosed. The signal processing device according to an embodiment of the present disclosure includes: a memory to store map data; and a processor to generate a graphical overlay based on camera data from a camera, the map data, or sensor data from a sensor device, wherein the processor is configured to execute a plurality of virtual machines on an hypervisor, and generate a first graphical overlay through a first augmented reality engine or generate a second graphical overlay, including less data than the first graphical overlay, through a second augmented reality engine, in response to the first augmented reality engine being stopped, output the second graphical overlay instead of the first graphical overlay. Accordingly, an augmented reality-based overlay may be stably provided.