Neural Network Embedding Transfer for Predictive Keyboards
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural network-based predictive keyboards on resource-constrained devices face challenges due to high memory and processing resource requirements, leading to reduced accuracy and efficiency in data entry, especially when network latency is involved.
Innovation Solution
Implementing a client-server deployment where a neural network is trained on a server and a subset of item embeddings is stored on the device, allowing for local predictions and dynamic transfer of missing embeddings from the server, optimizing resource usage and improving prediction quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a neural network is deployed locally on a resource-constrained device to enable predictive keyboard functionality, then prediction accuracy is improved, but memory and processing resource consumption increases significantly
Solution Approach 1:
The system segments the neural network components by separating the model architecture from the embeddings. The device stores only the model architecture locally while keeping embeddings on a server, allowing the model to run locally with minimal memory footprint while maintaining access to comprehensive embedding data through dynamic retrieval.
Solution Approach 2:
The embeddings are extracted from the local device and placed on a remote server. This extraction allows the device to operate with a lightweight model architecture while still utilizing the full embedding vocabulary through dynamic transfer of only the necessary embeddings during operation.
2Measurement precision
If a neural network is deployed locally on a resource-constrained device to enable predictive keyboard functionality, then prediction accuracy is improved, but processing resource consumption increases significantly
Solution Approach 1:
The model architecture is pre-configured and trained on the server, with all learning and complex computations performed in advance. During device operation, the pre-trained model performs only lightweight inference operations, significantly reducing processing resource consumption while maintaining prediction accuracy.
3Measurement precision
If item embeddings are transferred dynamically from server to device during operation, then prediction quality is maintained, but network latency impact increases
Solution Approach 1:
The system implements asynchronous embedding transfer that allows the model to continue operating with existing embeddings while new embeddings are being transferred in the background. This approach rushes through the network transfer operation without blocking the predictive keyboard functionality, minimizing the perceived latency impact.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An electronic device is described which has at least one input interface to receive at least one item of a sequence of items. The electronic device is able to communicate with a server, the server storing a neural network and a process which generates item embeddings of the neural network. The electronic device has a memory storing a copy of the neural network and a plurality of item embeddings of the neural network. In the case when there is unavailability at the electronic device of a corresponding item embedding corresponding to the received at least one item, the electronic device triggers transfer of the corresponding item embedding from the server to the electronic device. A processor at the electronic device predicts at least one candidate next item in the sequence by processing the corresponding item embedding with the copy of the neural network and the plurality of item embeddings.