Adaptive Language Model Unlearning for Text Entry

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing graphical keyboards for computing devices often require users to perform additional actions to correct erroneous predictions, reducing the efficiency of gesture-based text input due to the inability to adaptively unlearn new character strings that were incorrectly learned.

Innovation Solution

A computing device is enabled to adaptively unlearn character strings by decreasing their likelihood value in the language model when rejected by the user, allowing for gradual removal from the model without complete deletion, thus improving the accuracy of predicting new character strings.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If the computing device learns new character strings aggressively, then the accuracy of predicting new character strings improves, but the risk of learning incorrect words increases

Engineering Contradiction:

Improveaccuracy of predicting new character stringsVSAvoidrisk of learning incorrect words

Core Design Contradiction:

Measurement precisionVSReliability

Solution Approach 1:

The system dynamically adjusts the likelihood values of character strings based on user feedback. When users correct predictions, the system decreases the likelihood of the corrected string and increases the likelihood of the correct string, allowing the language model to adapt and learn from user interactions in real-time

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The system implements a feedback mechanism where user corrections are processed to update the language model. The feedback loop allows the system to learn from user corrections by decreasing likelihood values of incorrect predictions and increasing likelihood values of correct predictions, thereby improving future prediction accuracy

Inventive Principle:

Principle #23Feedback

2Measurement precision

If the computing device requires additional actions to correct erroneous predictions, then the accuracy of text input improves, but the efficiency of gesture-based text input deteriorates

Engineering Contradiction:

Improveaccuracy of text inputVSAvoidefficiency of gesture-based text input

Core Design Contradiction:

Measurement precisionVSProductivity

Solution Approach 1:

The system provides self-service by automatically learning from user corrections and improving its predictions over time. The language model autonomously adjusts likelihood values based on user feedback, reducing the need for manual corrections and improving input efficiency

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The system performs preliminary learning by continuously updating the language model with user feedback in the background. This preliminary action ensures that the model is already improved by the time the user needs to input text, reducing the need for corrective actions during actual typing

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentEP3245580B1Unlearning techniques for adaptive language models in text entry

Publication Date: 2019.01.02 GOOGLE LLC

Figure 1
Figure 2
Figure 3

AI summary

In some examples, a method includes outputting a graphical keyboard (120) for display and responsive to receiving an indication of a first input (124), determining a new character string that is not included in a language model. The method may include adding the new character string to the language model and associating a likelihood value with the new character string. The method may include, responsive to receiving an indication of a second input, predicting the new character string, and responsive to receiving an indication of a third input that rejects the new character string, decreasing the likelihood value associated with the new character string.