Ink Rendering Latency Reduction via Digitizer-to-Display Bypass
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current ink rendering techniques in tablet computing devices introduce significant latency, making the user experience slow and awkward, especially in co-located input and output scenarios where the rendering of user input from pens or fingers lags by 50 to 100 milliseconds or more.
Innovation Solution
A reduced-latency ink rendering system and method that bypasses at least some layers of the operating system and traditional rendering process by sending user input directly from the digitizer to the display controller, using software and hardware embodiments to synchronize vertical and raster refresh rates with a master clock, and employing techniques like light-based, pressure-based, or mechanical deformation to instantly update display pixels.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional ink rendering process through operating system and graphics card is used, then processing and manipulation of ink is achieved, but latency of 50 to 100 milliseconds or greater is introduced
Solution Approach 1:
The patent segments the ink rendering process into two distinct paths: a fast path for immediate visual feedback that bypasses the operating system and graphics card, and a traditional path for processing and manipulation that maintains accuracy. The fast path sends ink data directly from the digitizer to the display controller, while the traditional path continues through the operating system for applications that require processing. This segmentation allows the system to achieve low latency for visual feedback while maintaining rendering accuracy through the traditional path.
2Loss of time
If direct path from digitizer to display controller is used, then latency is reduced, but processing and manipulation capabilities are lost
Solution Approach 1:
The patent introduces an intermediary mechanism in the form of a buffer or memory structure that receives ink data from the digitizer and distributes it to multiple destinations simultaneously. This intermediary allows the fast path to send data directly to the display controller for immediate rendering while also forwarding the same data to the operating system and applications that require processing and manipulation. The intermediary ensures that both the fast path and traditional path receive the same ink data, maintaining processing capability while enabling low-latency display.
3Productivity
If bypassing operating system layers is implemented, then rendering speed is improved, but system integration and coordination are compromised
Solution Approach 1:
The patent implements preliminary action by pre-synchronizing the fast path rendering with the vertical refresh rate of the display and by pre-establishing the data flow paths between the digitizer, buffer, and display controller. This preliminary configuration allows the system to bypass the operating system during actual ink rendering without creating coordination problems, as the timing and data routing are already established. The synchronization with vertical refresh ensures that fast-path rendering integrates seamlessly with the display refresh cycle, eliminating the need for complex runtime coordination with the operating system.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A reduced-latency ink rendering system and method that reduces latency in rendering ink on a display by bypassing at least some layers of the operating system. "Ink" is any input from a user through a touchscreen device using the user's finger or a pen. Moreover, some embodiments of the system and method avoid the operating system and each central-processing unit (CPU) on a computing device when initially rendering the ink by going directly from the digitizer to the display controller. Any correction or additional processing of the rendered ink is performed after the initial rendering of the ink. Embodiments of the system and method address ink-rendering latency in software embodiments, which include techniques to bypass the typical rendering pipeline and quickly render ink on the display, and hardware embodiments, which use hardware and techniques that locally change display pixels. These embodiments can be mixed and matched in any manner.