Server-Client Image Processing Pipeline for HMD Latency Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image display systems face challenges in reducing delay times during real-time image processing and transmission, particularly in head-mounted displays, which can lead to degraded realistic sensations and motion sickness.
Innovation Solution
An image display system that includes a server and a client terminal, where the server performs partial image processing and transmission, and the client terminal corrects and displays the images in real-time, using pipeline-processing to minimize delay.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If image data is transmitted via network from server to client terminal for real-time display, then image quality can be improved through abundant server resources, but delay time period increases due to communication transmission
Solution Approach 1:
The patent divides the image processing workflow into two segments: the server performs a partial forming process (first conversion) on image data before transmission, and the client terminal completes the forming process (second conversion) after reception. This segmentation allows the server to reduce transmission data size and improve image quality while the client performs final formatting to minimize delay, resolving the contradiction between image quality and transmission delay.
2Manufacturing precision
If complete forming process is performed on server before transmission, then image quality is improved, but transmission data size and processing delay increase
Solution Approach 1:
The server performs only a partial forming process (first conversion) on the image data before transmission, rather than completing the entire forming process. This partial action reduces the amount of data transmitted while still improving image quality to some extent. The remaining forming process is completed by the client terminal, balancing image quality improvement with reduced transmission burden.
3Speed
If image data is processed and transmitted in real-time, then responsiveness is improved, but image quality may be compromised due to processing constraints
Solution Approach 1:
The server performs preliminary processing (first conversion) on the image data before transmission, preparing the data in advance for faster client-side processing. This preliminary action improves responsiveness by reducing the client's processing burden while maintaining image quality through the two-stage conversion approach, allowing real-time display with acceptable quality.
Data Source
AI summary
A server performs a part of a forming process necessary for conversion into formats corresponding to display modes of a head mounted display and a flat-plate display connected to an image processing apparatus, and transmits a processing result to the image processing apparatus. At this time, the server switches the part of the process to transmit any one of a pair of a left-eye image and a right-eye image, an image suited for the flat-plate display, and an image constituted by a left-eye image and a right-eye image to each of which distortion for an ocular lens has been given.