Partial-Image Streaming for Synchronized Low-Latency Displays
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image display technologies experience delays due to communication latency between client terminals and servers, leading to degraded realism and potential motion sickness, especially in head-mounted displays, as user interactions are not promptly reflected in the displayed image.
Innovation Solution
An image processing apparatus and system that processes and displays moving images in units of partial images, adjusting output timing based on formation times to minimize delay and synchronize with user movements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If image data is transmitted via network communication from server to client terminal, then image quality can be improved through streaming transfer, but delay time period increases due to communication latency
Solution Approach 1:
The patent divides image data into multiple blocks, each containing multiple macroblocks, and transmits them separately with individual timing information. This segmentation allows the receiver to process and display blocks in the correct temporal order, reducing overall delay while maintaining image quality through selective rendering of available blocks
Solution Approach 2:
The patent attaches formation time information to each image block before transmission, enabling the receiver to pre-organize and buffer blocks according to their intended display timing. This preliminary timing organization allows for smoother playback with reduced latency by eliminating the need to wait for complete frame reception before displaying available content
2Manufacturing precision
If complete frame data is transmitted before display, then image quality is maintained, but display delay increases significantly
Solution Approach 1:
The patent segments complete frame data into multiple transmittable blocks with hierarchical structure (blocks containing macroblocks). This allows progressive transmission and rendering of image portions without waiting for complete frame assembly, reducing display delay while maintaining quality through systematic block processing and timing synchronization
3Ease of operation
If user operation data is transmitted to server and processed, then interactivity is improved, but round-trip communication delay increases
Solution Approach 1:
The patent transmits formation time information with each image block in advance, enabling the client to pre-synchronize display timing with user operation feedback. This preliminary timing information allows the system to maintain interactivity by predicting when updated image blocks should be displayed, reducing the perceived round-trip delay for interactive applications
Data Source
AI summary
A compression coding section compression-codes an image formed by an image forming section of a server in units of a partial image smaller than one frame, and transmits the compression-coded image to an image processing apparatus via a communication section together with its formation time. An image data acquisition section of the image processing apparatus acquires the transmitted image data, and a decoding decompression section decoding-decompresses the image data for each partial image. An image processing section performs necessary processing for each of the partial images. A display control section outputs data of the partial images to a display device while changing an output target or controlling an output order according to an acquisition status of the data of the partial images.