Text Rewriting Model Using Knowledge Distillation for Client Devices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing text style transfer methods, particularly large language models (LLMs), face deficiencies in transferring writing styles into more vivid or expressive forms, leading to inconsistencies in semantic meaning and potential confusion, especially when attempting to enhance communication in interactive domains like social media, document editing, and videoconferencing.
Innovation Solution
A computer-implemented system and method that utilizes a few-shot trained LLM to generate a large rewriting corpus, which is then distilled into a smaller model for client devices, allowing for personalized text rewriting with options for different styles, tones, and emojification, enabling vivid textual and graphical information generation based on user input and context.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a large language model is used for text style transfer, then the text can be rewritten with various styles, but the model size is large and cannot be implemented directly on client devices
Solution Approach 1:
The patent extracts the essential rewriting capabilities from a large language model and transfers them to a smaller model through knowledge distillation. The smaller model is trained on data generated by the larger model, capturing the style transfer abilities while reducing the model size to fit on client devices.
Solution Approach 2:
The patent creates a copy of the rewriting knowledge from the large model to a small model. The large model generates training data that encapsulates its rewriting capabilities, which are then used to train a smaller model that replicates these capabilities in a compact form.
2Speed
If a smaller model is used for client device implementation, then the model can run locally with low latency, but the text rewriting quality may be reduced
Solution Approach 1:
The patent performs preliminary action by having the large language model generate extensive training data before the smaller model is deployed. This pre-generated data contains high-quality rewriting examples that teach the smaller model effective rewriting techniques, enabling it to achieve good quality results despite its smaller size.
Solution Approach 2:
The patent changes the parameters of the training process by using synthetic data generated by the large model with controlled style attributes. This allows the smaller model to learn specific rewriting patterns and parameters that optimize both quality and efficiency for client-side operation.
3Adaptability or versatility
If general text rewriting is used, then the model can handle various styles, but the semantic meaning may become inconsistent with the source text
Solution Approach 1:
The patent implements feedback mechanisms where the large language model evaluates and refines the rewriting data before it is used for training the smaller model. This feedback loop ensures that the generated examples maintain semantic consistency while achieving the desired style transfer, and the smaller model learns these validated patterns.
Data Source
AI summary
The technology provides rewriting suggestions for communication styles in different pre-defined styles. Rewriting may convert text in different ways, including visually enhancing the message. A method provides input to a trained large language model, the input including curated examples associated with one or more writing style choices. The set of curated examples has a first size. The method also includes generating, using the model, a rewriting corpus according to one or more writing style choices. The rewriting corpus has a size two or more orders of magnitude larger than a size of curated examples. The writing style choices include at least one of tone, conversion, application context, or conversation type. A text rewriting model is trained using at least a subset of the rewriting corpus. The model is configured to generate vivid textual information in response to user input in an interactive domain, according to specific writing style choices.