Autonomous Vehicle VR Display Obstruction Removal
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Autonomous vehicle virtual reality displays can cause stress and distraction due to the presence of object obstructions like other vehicles, pedestrians, and bicycles, which diminish the immersive experience and situational awareness for the operator.
Innovation Solution
A neural network-based system that identifies environmental objects and removes obstructions from camera feeds to generate a rendered window for display, using a generative adversarial network (GAN) to create a more relaxing and realistic scenery by superimposing background objects into the rendered window.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If full virtual reality displays with obstructions are provided in real-time, then the operator can view outside scenery through the displays, but unnecessary stress and distraction are caused to the operator due to obstructions such as other vehicles
Solution Approach 1:
The system extracts and removes obstructive objects (vehicles, pedestrians, bicycles) from the virtual reality display feed while preserving the background scenery. This is achieved through object detection algorithms that identify harmful objects and selectively eliminate them from the rendered display, allowing the operator to view the outside environment without stress-inducing obstructions.
Solution Approach 2:
The system applies different processing qualities to different regions of the display. Obstructive objects are completely removed or blurred, while the surrounding background scenery is preserved with high fidelity. This local differentiation allows the operator to maintain situational awareness of the environment without being distracted by harmful objects.
2Object-affected harmful factors
If obstructions are removed from the virtual reality display, then a more relaxing and immersive experience is provided, but the complexity of the system increases due to neural network processing
Solution Approach 1:
The system introduces a neural network intermediary that automatically performs the complex task of identifying and removing obstructive objects. This intermediary layer handles the computationally intensive object detection and removal processes, shielding the operator from the complexity while delivering the desired stress-free viewing experience.
Solution Approach 2:
The patent replaces traditional mechanical or manual methods of obstruction removal with neural network-based computer vision algorithms. This substitution enables automated, real-time processing of video feeds to identify and remove obstructions without requiring manual intervention, despite the increased computational complexity.
3Loss of information
If real-time processing of camera feeds is performed to remove obstructions, then situational awareness is enhanced, but the processing time and computational resources increase
Solution Approach 1:
The system performs preliminary processing by pre-training neural networks to recognize common obstructive objects and pre-establishing removal algorithms. During real-time operation, these pre-trained models enable rapid identification and removal of obstructions without requiring complex on-the-fly decision-making, thus reducing processing time while maintaining situational awareness.
Data Source
AI summary
This disclosure describes a vehicle system for identifying environmental objects within an operator's surroundings in a first window and removing object obstructions through a neural network in a second window to generate a rendered window, or scene, for display. As more windows are processed, the neural network may increase its understanding of the surroundings. The neural network may be a generative adversarial network (GAN) that may be trained to identify environmental objects by introducing noise data into those environmental objects. After receiving a first window from at least one camera on the vehicle, a rendered window may be created from a second window received from the at least one camera based on environmental objects identified within the first window through the GAN and removing object obstructions. The rendered window, without the object obstructions, and having superimposed environmental objects based on the GAN, may be displayed on an output device.