VR Video Rendering via Attention Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing VR devices face challenges in rendering video images with high quality and low latency due to limited computing resources, leading to delayed presentation and potential user dizziness.
Innovation Solution
A method and system that dynamically render video images in VR scenes by considering user viewpoint trajectory information, dividing images into sub-regions, and using attention models and user eyes trajectory prediction models to optimize rendering quality and reduce computational load.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If the VR device renders the video image with high quality, then the rendering quality is improved, but the rendering rate decreases and causes delay in presentation
Solution Approach 1:
The video image is divided into multiple sub-regions, and the system selectively renders only those sub-regions where the user's viewpoint is likely to fall (based on attention coefficients), rather than rendering the entire image. This segmentation approach reduces the total rendering workload while maintaining quality in critical areas.
Solution Approach 2:
The system applies different rendering strategies to different sub-regions of the video image. Sub-regions with high attention coefficients (where user viewpoint is likely to fall) are rendered with high quality, while other sub-regions may be rendered with lower quality or skipped entirely. This local differentiation optimizes the balance between rendering quality and rendering rate.
2Loss of time
If the VR device reduces computing resources usage, then the delay problem is alleviated, but the rendering quality decreases causing user dizziness
Solution Approach 1:
The system performs preliminary calculations to determine attention coefficients and predict user viewpoint distribution before rendering. By pre-identifying which sub-regions require high-quality rendering based on user behavior patterns, the system can allocate computing resources more efficiently and avoid last-minute rendering delays while maintaining quality where needed.
Solution Approach 2:
The system uses user viewpoint information as feedback to dynamically adjust rendering quality. By continuously monitoring where users are likely to look and adjusting rendering efforts accordingly, the system ensures high quality is maintained in relevant areas while reducing overall computational load to prevent delay.
3Ease of operation
If the VR device renders all sub-regions with high quality, then the user experience is improved, but the computing resources are exhausted
Solution Approach 1:
The system dynamically changes rendering parameters (such as resolution, detail level, or rendering algorithm complexity) based on attention coefficients for different sub-regions. Sub-regions with high attention coefficients receive high-quality rendering parameters, while others receive reduced parameters, optimizing the balance between user experience and computing resource consumption.
Data Source
AI summary
A method and a system for rendering video images in virtual reality (VR) scenes are provided. The method includes providing a video image at a current time point, dividing the video image at the current time point into a plurality of sub-regions, inputting image feature information of the sub-regions and acquired user viewpoint feature information into a trained attention model for processing to obtain attention coefficients of the sub-regions indicating probability values at which user viewpoints at a next time point fall into the sub-regions, rendering the sub-regions based on the attention coefficients of the sub-regions to obtain a rendered video image at the current time point, inputting the attention coefficients of the sub-regions and the image feature information of the sub-regions into a trained user eyes trajectory prediction model for processing, obtaining user eyes trajectory information in a current time period, dividing, for video images at subsequent time points within the current time period, the video images at the subsequent time points into a plurality of sub-regions, calculating attention coefficients of the sub-regions in a video image at each of the subsequent time points within the current time period respectively based on the user eyes trajectory information in the current time period, and rendering the corresponding sub-regions based on the attention coefficients of the sub-regions to obtain a rendered video image at each of the subsequent time points.