VR Image Generation Using Past Frame Substitution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Head-mounted displays experience latency issues when generating and displaying virtual reality videos, leading to deviations in the user's perceived direction and potential discomfort, exacerbated by network delays in server-client systems.
Innovation Solution
An image generation system that includes an image generation section, storage for past frames, a selection mechanism to choose substitute images from these frames, and an image providing section to output images before the current viewpoint position, reducing latency by using past frames for immediate display and reprojecting them as necessary.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If real-time image generation is performed for virtual reality video, then image quality and viewpoint accuracy are improved, but latency increases causing deviation between head direction and displayed image
Solution Approach 1:
The system pre-generates multiple images from different viewpoint positions before they are actually needed. When the user's head moves to a new direction, the system can immediately display a pre-generated image from the nearest viewpoint position, eliminating the latency of real-time generation while maintaining viewpoint accuracy.
Solution Approach 2:
The system dynamically selects which pre-generated image to display based on the user's current head direction. The selection section chooses the image from the viewpoint position closest to the current line-of-sight direction, and the displayed viewpoint is dynamically adjusted to match the user's actual head orientation, resolving the contradiction between pre-generation and real-time accuracy.
2Extent of automation
If images are transmitted via network from server to client, then centralized image generation is achieved, but network delay increases the deviation between generation time and display time
Solution Approach 1:
The server pre-generates images from multiple viewpoint positions and transmits them to the client in advance. This allows the client to have a buffer of pre-generated images available locally, reducing the impact of network delay when the user moves their head during VR viewing.
Solution Approach 2:
The system dynamically selects and displays the most appropriate pre-generated image from the transmitted buffer based on the user's current head direction, rather than waiting for real-time generation or transmission. This dynamic selection minimizes the effective delay experienced by the user while maintaining centralized image generation on the server.
3Loss of time
If multiple viewpoint images are pre-generated and stored, then latency is reduced for display, but storage requirements and system complexity increase
Solution Approach 1:
The system segments the viewpoint space into discrete positions and pre-generates images only for these specific segmented positions rather than continuously. This reduces the total number of images that need to be stored while still providing adequate coverage for head movement, balancing latency reduction with storage and complexity constraints.
Solution Approach 2:
The system generates slightly more viewpoint images than strictly necessary (excessive action) to ensure coverage for all possible head directions, but uses a finite set of discrete positions rather than continuous generation. This partial pre-generation approach reduces latency while keeping storage requirements manageable through selective sampling of viewpoint positions.
Data Source
AI summary
An image processing section 750 draws an image viewed in a line-of-sight direction from a viewpoint position at a certain time. An image storage section 760 holds the image drawn by the image processing section 750 as past frames. A past frame selection section 790 selects, from among the past frames held in the image storage section 760, a substitute image to be viewed in the line-of-sight direction from a current viewpoint position. An image providing section 770 outputs the substitute image before providing the image viewed in the line-of-sight direction from the current viewpoint position, drawn by the image processing section 750.