HMD Obstruction Removal Using Line-of-Sight View Synthesis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The challenge of maintaining a clear field of view while operating a vehicle is hindered by objects obstructing the line of sight, leading to reduced situational awareness and potential disorientation due to missed information in the scenery.
Innovation Solution
Employing line of sight tracking and mapping of vehicle objects, combined with additional sensor capabilities to generate an unobstructed view by extracting and overlaying unobstructed regions of interest from sensors positioned to avoid obstructions, using various tracking technologies and processors to calculate and display an unobstructed scene.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If the user views the outside scene through the HMD, then the user can obtain information from the scenery, but the field of view may be obstructed by objects within the vehicle (such as canopy frame)
Solution Approach 1:
The system introduces an intermediary computational process that detects obstructions in the user's field of view and replaces obstructed portions with image data from alternative camera viewpoints. This mediator (the processing system) transfers the unobstructed view from cameras to the user's display, eliminating the harmful obstruction effect while preserving the useful information flow.
Solution Approach 2:
The system transitions from a single fixed viewpoint (the user's direct line of sight) to multiple dimensional viewpoints by utilizing cameras positioned at different locations within the vehicle. When an obstruction blocks the direct view, the system retrieves image data from alternative spatial dimensions (different camera positions) to provide the unobstructed view, effectively adding spatial dimensionality to the viewing solution.
2Loss of information
If additional sensors are mounted on the vehicle to provide alternative viewpoints, then obstructions can be bypassed, but the device complexity increases
Solution Approach 1:
The system employs multi-functional components that serve multiple purposes. The cameras are not only used for obstruction removal but also for providing the user with a first-person view of the external environment. The processing system handles both obstruction detection and image synthesis tasks. This multi-functionality reduces the need for separate dedicated components, thereby managing complexity while achieving multiple goals.
Solution Approach 2:
The system uses the vehicle's existing sensor infrastructure and computational resources to perform obstruction removal, rather than requiring entirely separate dedicated systems. The processing leverages available camera data and onboard computing capabilities, allowing the system to serve itself using existing assets, thus minimizing the complexity increase from additional specialized components.
Data Source
Figure 1
Figure 2A~2B
Figure 3A~3C
AI summary
A method and a system for displaying a scene to a user wearing a head mounted display (HMD) while removing obstructions in a field of view (FOV) of the user are provided herein. The method may include: capturing by a sensor a vehicle image of the scene wherein the first sensor is mounted on the vehicle; tracking a position and orientation of the HMD in a specified coordinate system, to yield a line of sight (LOS) of the user wearing the HMD; obtaining a database containing obstacles data indicating at least one physical object located within the vehicle and affecting the FOV of the user; calculating an obstructed portion in the FOV of the user, based on the LOS and the database; generating from the vehicle image, an un-ob structed view which includes a portion of the scene overlapping the obstructed portion; and displaying in the HMD the un-ob structed view.