VR View Image Generation via Route-Based Video Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current virtual reality technologies are limited by high computational and communication resource requirements, suboptimal user experience, and restricted freedom of movement in three-dimensional environments, particularly when generating images from real-world captures, leading to disjointed and inconsistent experiences.
Innovation Solution
An apparatus and method that utilize video items linked to route data in an N-dimensional space, allowing efficient generation of view images by selecting routes based on distance metrics and biasing towards currently selected routes, reducing latency and resource usage, and enabling dynamic user experiences with reduced storage and communication needs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple video streams are loaded for switching positions, then the virtual reality experience allows position changes, but the data rate and storage requirements increase significantly
Solution Approach 1:
The patent divides the virtual environment into multiple pre-defined routes, each represented by a video stream. Instead of loading all possible video streams for all positions, the system segments the content into route-specific streams that are loaded on-demand based on the user's current position and movement direction, significantly reducing the amount of data that needs to be stored and transmitted.
Solution Approach 2:
The patent performs preliminary actions by pre-processing and encoding video content along predetermined routes before transmission. The video streams are prepared in advance with metadata indicating their spatial relationships and valid position ranges, allowing the client to efficiently select and load only the necessary streams without real-time processing overhead.
2Quantity of substance
If video streams are loaded on demand for position switching, then storage requirements are reduced, but latency increases due to loading interruptions
Solution Approach 1:
The patent performs preliminary actions by pre-loading video streams for adjacent positions before they are actually needed. When the user is at a given position, the system proactively loads the video streams for neighboring positions in the background, so that when position switching is required, the data is already available in memory, eliminating loading interruptions and reducing latency.
Solution Approach 2:
The patent ensures continuous useful action by maintaining a buffer of pre-loaded video streams that are ready for immediate playback. This creates a continuous supply of content without interruptions, as the system continuously manages the loading and unloading of video streams in the background while the user experiences seamless position transitions.
3Productivity
If a predetermined model of the scene is used, then the virtual reality experience can be generated efficiently, but the freedom of movement is restricted
Solution Approach 1:
The patent implements dynamics by allowing the virtual environment to adapt to user movements in real-time. Instead of a fixed predetermined model, the system dynamically selects and switches between multiple video streams based on the user's current position, viewing direction, and movement trajectory, providing flexible movement freedom while maintaining efficient image generation through pre-encoded streams.
Solution Approach 2:
The patent utilizes parameter changes by varying the selected video stream based on user position parameters and viewing direction parameters. The system monitors these parameters continuously and switches between different pre-encoded video streams according to the current parameter values, enabling efficient rendering without requiring a complete dynamic model evaluation.
4Reliability
If real world captures are used instead of virtual models, then the authenticity is improved, but the computational resources and data rates increase
Solution Approach 1:
The patent applies segmentation by dividing the real-world captured environment into multiple route-specific video streams rather than transmitting the entire environment at full resolution. Each stream contains only the relevant portion of the real-world capture for a specific route, reducing the overall data rate while maintaining the authenticity of the real-world scenes.
Solution Approach 2:
The patent extracts only the necessary portions of real-world captures for specific routes and positions. Instead of transmitting all captured data, the system extracts and transmits only the video streams relevant to the user's current position and movement, reducing data rate while preserving the realism of the extracted portions.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An apparatus comprising a store (101) for storing route data for a set of routes in an N-dimensional space where each route of the set of routes is associated with a video item including frames comprising both image and depth information. An input (105) receives a viewer position indication and a selector (107) selects a first route of the set of routes in response to a selection criterion dependent on a distance metric dependent on the viewer position indication and positions of the routes of the set of routes. A retriever (103, 109) retrieves a first video item associated with the first route from a video source (203). An image generator (111) generates at least one view image for the viewer position indication from a first set of frames from the first video item. In the system, the selection criterion is biased towards a currently selected route relative to other routes of the set of routes.