Motion Prediction Model Reduces Touch Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computing devices experience significant touch latency due to the large processing stack required to process touch events, which delays the response to user input on touch-sensitive screens.
Innovation Solution
A computer-implemented method using a machine-learned motion prediction model, specifically a recurrent neural network, to predict the future locations of user input objects, allowing devices to respond promptly without waiting for the entire input action to be processed through the processing stack.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a large processing stack is used to process touch events, then the processing is thorough and reliable, but the touch latency increases significantly
Solution Approach 1:
The system performs preliminary actions by predicting future touch locations before the actual touch events occur. The machine learning model analyzes current and historical touch data to forecast where the user will touch next, allowing the system to pre-load or pre-process content at those predicted locations. This eliminates the need to wait for the complete processing stack to handle each touch event in real-time, significantly reducing perceived latency while maintaining processing reliability.
2Productivity
If software scheduling is optimized, then processing efficiency improves, but there is still a minimum processing time required
Solution Approach 1:
By predicting future touch locations in advance, the system performs useful work before it is strictly needed. Content at predicted touch locations can be pre-rendered, pre-processed, or pre-positioned, so when the actual touch occurs, the minimum processing time is already minimized. This approach pushes the effective processing time backward, making the system appear to respond instantaneously even though processing is occurring in advance.
3Loss of time
If the processing stack is reduced, then touch latency decreases, but the processing completeness may be compromised
Solution Approach 1:
The prediction model allows the system to perform processing in advance when there is no time pressure. By forecasting future touch events, the system can allocate sufficient processing resources and time for each event before it occurs, ensuring complete and reliable processing. When the actual touch event happens, the heavy processing has already been done or is already in progress, so the perceived latency is low while processing completeness is maintained.
Data Source
Figure 1
Figure 2~3
Figure 4
AI summary
The present disclosure provides systems and methods that leverage machine learning to perform user input motion prediction. In particular, the systems and methods of the present disclosure can include and use a machine-learned motion prediction model that is trained to receive motion data indicative of motion of a user input object and, in response to receipt of the motion data, output predicted future locations of the user input object. The user input object can be a finger of a user or a stylus operated by the user. The motion prediction model can include a deep recurrent neural network.