Touch Display Rendering With Predicted Coordinates to Cut Output Lag
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing touch input devices experience significant delays in display output, particularly when using a stylus pen, which disrupts the perceived reality of direct manipulation on touch screens due to latency between user actions and system responses.
Innovation Solution
A method and device that predict future touch events based on input coordinates and characteristics, generating an output image on a buffer being scanned by the display, using techniques such as recurrent neural networks and direct rendering to reduce latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If a touch sensor display device is used, then user interaction capability is improved, but display output lag increases due to sequential processing of touch and display frames
Solution Approach 1:
The patent applies preliminary action by predicting future touch input coordinates based on historical touch data before the actual touch input occurs. This allows the display system to prepare and pre-render display frames at predicted touch locations, so when the actual touch input arrives, the display output is already ready or close to ready, significantly reducing the display output lag while maintaining seamless user interaction.
2Measurement precision
If touch input processing is prioritized over display output, then touch response accuracy is improved, but display frame rendering is delayed
Solution Approach 1:
The system performs preliminary touch coordinate prediction using machine learning models that analyze historical touch data patterns. This prediction occurs in advance, allowing the display processing unit to start rendering display frames at predicted touch locations before the actual touch input is fully processed, thereby maintaining both touch response accuracy and display frame timing.
Solution Approach 2:
The patent implements dynamic adjustment of processing priorities based on predicted touch coordinates. The display processing unit dynamically switches between processing actual touch inputs and using predicted coordinates, optimizing the balance between touch response accuracy and display frame rendering timing according to the specific operational context and user interaction patterns.
3Speed
If display frames are generated in advance, then display output speed is improved, but display accuracy at actual touch location deteriorates
Solution Approach 1:
The system employs feedback mechanisms where the actual touch input coordinates are compared with the predicted coordinates. This feedback loop allows the machine learning model to continuously learn and improve its prediction accuracy, ensuring that pre-generated display frames are increasingly accurate at the actual touch locations, thereby maintaining both display output speed and precision.
Solution Approach 2:
Display frames are generated in advance at predicted touch coordinates through preliminary action. The machine learning model continuously refines these predictions based on feedback from actual touch inputs, ensuring that the pre-generated frames become progressively more accurate, thus achieving both high display output speed and high precision at actual touch locations.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Provided are a device and method for reducing a display output delay of a touch input. The method includes receiving a touch event as a touch input, obtaining input coordinates corresponding to position coordinates where the touch event occurs and touch characteristics corresponding to touch information of the touch event, based on the touch input, obtaining predicted coordinates of a touch event after at least one frame, based on the input coordinates and the touch characteristics, obtaining output coordinates of an image to be displayed on the display based on the input coordinates and the predicted coordinates, obtaining an output image based on the output coordinates and the touch characteristics, generating a rendering layer in a buffer being scanned by the display, and outputting the output image onto the display by rendering the output image on the rendering layer.