Memory-enhanced deep personalized input method system based on end-side large language model

Abstract

The present application relates to a memory-enhanced deep personalized input method system based on end-side large language model, and belongs to the technical field of artificial intelligence. The present application solves the problems of the prior art, such as the loose coupling of the model and the input core process, the shallow personalization mechanism, the delay and privacy risk caused by cloud deployment, and the insufficient efficiency of the end-side operation that cannot meet the real-time input requirements. The system includes a front-end real-time interaction module, a stylized large model post-training module, a hierarchical memory mechanism module, and an end-side reasoning optimization module. The present application can realize low-delay real-time text generation under the condition of highly limited mobile terminal resources, and at the same time, through continuous modeling of user input behavior, it can complete deep personalization adaptation and ensure the end-side retention and privacy security of user input data throughout the process.

Description

Technical Field

[0001] This invention relates to a memory-enhanced deep personalized input method system based on a large edge language model, belonging to the field of artificial intelligence technology. Background Technology

[0002] Mobile input methods are the most basic and frequently used text input interface in mobile human-computer interaction. Traditional Chinese input methods have gradually evolved from N-gram statistical models to neural language models, but their core capabilities have always been limited to the conversion and mapping from Pinyin to Chinese characters. They cannot deeply model the user's input intent, nor can they capture the user's long-term personalized writing habits and expression characteristics. As large language models have shown disruptive potential in the fields of text generation and writing assistance, mainstream input method manufacturers have begun to explore a hybrid architecture that combines traditional candidate word generation with intelligent writing driven by large models. Some solutions have also tried to introduce generative functions such as preset personas and basic text rewriting. However, most of these solutions are incremental optimizations to traditional input methods and do not fundamentally reconstruct the core workflow of the input method. The generative large model exists only as an additional function in the cloud, rather than a core component of the core process of the input method. Therefore, it has many inherent defects.

[0003] Currently, the closest implementations to this invention are mainly cloud-first or hybrid deployment-based large-model enhanced Chinese input methods launched by various manufacturers, including SwiftKey based on GPT-4 Turbo, Baidu Input Method based on the Wenxin large model, Sogou Input Method and WeChat Input Method based on the Hunyuan large model, Xunfei Input Method based on the Xinghuo large model, and Doubao Input Method based on the Doubao large model. These existing solutions generally suffer from three core limitations. First, the generative large model is loosely coupled with the traditional key-to-candidate input process. The model is only invoked in scenarios such as long text rewriting and full-sentence suggestions, and cannot perform fine-grained modeling for high-frequency input scenarios such as short sentence completion and interactive editing, thus limiting the application of large-model generation capabilities in core input method scenarios. Second, the personalization mechanism is superficial. The personalization capabilities of existing solutions are mostly limited to short context windows or static character presets, failing to accumulate users' long-term input knowledge and behavioral characteristics, and unable to complete adaptive iteration based on continuous user input feedback, making it difficult to achieve true deep personalized adaptation. Thirdly, there are the experience and privacy defects brought about by cloud reliance. Cloud-first inference architectures introduce unpredictable network latency, and user input data at the key level needs to be uploaded to the cloud for processing, posing serious risks of privacy leaks and data exposure. In high-frequency input scenarios, neither real-time response nor user data security can be guaranteed. Even with a hybrid deployment solution, advanced generation functions still rely on cloud authentication, and performance degrades significantly offline, making it impossible to achieve stable full-featured on-device operation. In addition, existing technologies generally lack on-device large model optimization solutions tailored for mobile input method scenarios. When general large models run on resource-constrained mobile terminals, they suffer from high inference latency, large memory consumption, and poor heat management, failing to meet the millisecond-level response requirements of input method scenarios and severely impacting the user's input experience. Summary of the Invention

[0004] This invention addresses the numerous shortcomings of existing large-model-enhanced input methods, such as loose coupling between the model and the core input process, shallow personalization mechanisms, latency and privacy risks caused by cloud deployment, and insufficient on-device operating efficiency to meet real-time input requirements. Therefore, it proposes a memory-enhanced deep personalized input method system based on a large on-device language model.

[0005] The technical solution adopted by the present invention to solve the above problems is as follows: The present invention includes a front-end real-time interaction module, a stylized large model post-training module, a hierarchical memory mechanism module, and an edge-side inference optimization module; The front-end real-time interaction module integrates the completion content generated by the large model into the candidate word-based input workflow. By displaying ghost text at the cursor position and alternative generated results in the secondary panel, it achieves a deep integration of the large model generation capability with the core process of the input method. The stylized large model post-training module transforms the general lightweight large model into a dedicated backbone network for input methods with human-person perception capabilities through a closed-loop pipeline of corpus collection, synthesis expansion, filtering and iterative optimization. The hierarchical memory mechanism module adopts a three-level decoupled architecture consisting of an L1 high-speed cache layer, an L2 plaintext fact layer, and an L3 parameter weight layer. It separates the front-end completion generation from the back-end memory management and controls the boundaries of memory retrieval, extraction, and rejection through a memory trigger based on the GRPO algorithm. The edge-side inference optimization module achieves low-latency inference of large models on resource-constrained mobile terminals through prefix tree-based KV cache reuse, thread affinity scheduling, and memory injection mechanisms.

[0006] Furthermore, the stylized large model post-training module achieves model transformation in the following way: Collect publicly available dialogue data from the internet and expand the data by synthesizing dialogues using large-scale models; A large model-assisted filtering and style classification pipeline is used to remove low-quality samples and samples that do not meet style requirements. The filtered corpus was divided into a training set and a test set. A structured training context was constructed by splicing personalized system prompts with dialogue history. The model is fine-tuned based on the structured training context, and the style fidelity and sentence fluency of the generated content are evaluated. The corpus construction process was iteratively optimized based on the evaluation feedback.

[0007] Furthermore, the three-level decoupling architecture of the hierarchical memory mechanism module includes: The L1 cache layer is used to store KV cache blocks corresponding to specific styles and single memories, and injects the pre-computed KV state into the front-end generation path; The L2 plaintext fact layer is used to append structured memory records in plaintext form and maintain a lightweight HNSW vector index to achieve fast semantic retrieval of memory content. The L3 parameter weight layer is used to accumulate user interaction trajectories and model decision logs, and automatically generate a dataset that meets the requirements for subsequent edge-side incremental fine-tuning.

[0008] Furthermore, the memory trigger based on the GRPO algorithm controls the memory function by pre-defining direct completion tasks, memory retrieval tasks, memory extraction tasks, and invalid information rejection tasks. The GRPO algorithm is used to sharpen the boundary between retrieval and rejection. The reward function of the GRPO algorithm comprehensively considers format compliance, task execution correctness and delay constraints.

[0009] Furthermore, the edge-side inference optimization module manages the KV cache as a compressed prefix tree structure, enabling structure sharing between overlapping prefixes generated by user input and editing. Automatically locate the longest common prefix, continue decoding from the matching node, and transform the interactive editing process into an incremental tree traversal process to achieve KV cache reuse.

[0010] Furthermore, in the edge-side inference optimization module, for the big.LITTLE architecture of mobile devices, computationally intensive attention calculations and GEMM kernels are scheduled to high-performance cores for execution, achieving thread affinity scheduling.

[0011] Furthermore, the edge-side inference optimization module injects pre-computed memory KV fragments into the active decoding path through a lightweight KV injection mechanism with the help of position offset mapping. By combining phase shift correction and selective tail recomputation techniques to preserve the causal attention mechanism, cache context splicing is achieved and recomputation overhead is minimized, thus realizing memory injection.

[0012] Furthermore, it also includes a cross-application context synchronization mechanism based on the model context protocol, which is implemented in the following ways: Under the constraints of Android system sandbox isolation, lightweight cross-process context sharing is achieved; Perform graceful degradation when the protocol becomes unavailable to ensure the normal operation of basic input functions.

[0013] Furthermore, the front-end real-time interaction module integrates large model generation capabilities with the core input method process in the following ways: Set up a ghost text area immediately adjacent to the cursor position to display the best short sentence candidate content in real time; Set up a two-layer candidate interface in the secondary panel to display the alternative generated results.

[0014] Furthermore, the memory-enhanced, deeply personalized input method system achieves end-to-end retention of user input data on the device side without any cloud dependency. This device side deployment includes: The end-to-end inference operation of the large language model, the end-to-end operation of the hierarchical memory mechanism, and the end-to-end incremental fine-tuning of user-personalized features.

[0015] The beneficial effects of this invention are: 1. This invention proposes a three-level hierarchical memory architecture and a matching GRPO enhanced memory triggering mechanism. This architecture, through the decoupled design of the L1 high-speed cache layer, the L2 plaintext fact layer, and the L3 parameter weight layer, takes into account both the low latency requirements of front-end generation and the long-term user behavior modeling requirements of the back-end. At the same time, the memory trigger optimized based on the GRPO algorithm can accurately control the boundaries of memory retrieval, extraction, and rejection. Under the premise of minimizing computational overhead, it can realize the continuous capture and efficient utilization of user input information, supporting the continuous self-evolution capability of the input method on the device side.

[0016] 2. This invention establishes a customized end-to-end optimization scheme for mobile input method scenarios. This scheme achieves low-latency inference of large models on resource-constrained mobile terminals by using core technologies such as KV cache reuse based on prefix trees, thread affinity scheduling, memory injection mechanism, and position-independent caching strategy. It amortizes the first candidate generation time in the interaction process to near zero, while strictly controlling memory usage to ensure stable and smooth operation of the system on various mobile terminals.

[0017] 3. This invention establishes a cross-application context synchronization mechanism based on the model context protocol. This mechanism can achieve lightweight cross-process context sharing under the constraints of Android system sandbox isolation, and has a complete graceful degradation capability, which can still ensure the normal operation of basic input functions when the protocol is unavailable.

[0018] 4. End-to-end operation, privacy and security: This invention has no cloud dependency throughout the process, and user input data is only stored on the terminal, thus avoiding privacy leaks and data upload risks from the source.

[0019] 5. Low latency and high efficiency inference on the device side: Through optimizations such as prefix tree KV multiplexing, thread scheduling, and memory injection, this invention achieves millisecond-level response on mobile terminals, meeting real-time input requirements. Attached Figure Description

[0020] Figure 1 This is a framework diagram of a memory-enhanced deep personalized input method system based on a large on-device language model. Figure 2 A flowchart outlining the workflow of each module in a memory-enhanced, deeply personalized input method system; Figure 3 This is a flowchart of the training module after stylizing a large model. Detailed Implementation

[0021] like Figure 1 As shown, the memory-enhanced deep personalized input method system based on a large edge language model described in this embodiment includes: a front-end real-time interaction module, a stylized large model post-training module, a hierarchical memory mechanism module, and an edge-side inference optimization module.

[0022] The HUOZIIME system proposed in this invention is a fully device-side large language model-driven input method system designed specifically for resource-constrained mobile environments. Its overall architecture consists of four core parts: a front-end real-time interactive UI module, a stylized large model post-training module, a hierarchical memory mechanism module, and a device-side inference efficiency optimization module. It also features two core operational processes: an online interactive loop and asynchronous memory update, forming a complete closed loop for input generation and personalized iteration. The specific process is as follows: Figure 2 As shown.

[0023] The front-end real-time interactive UI design of this invention seamlessly integrates the completion content generated by the large model into the user's familiar candidate word-based workflow. It adopts a design scheme of ghost text adjacent to the cursor position and a two-layer candidate interface. The ghost text area displays the best short sentence candidate content in real time, while the secondary panel displays the alternative generation results. This achieves a deep integration of large model generation capabilities and traditional input processes, avoiding the forced modification of user input behavior.

[0024] The stylized large-scale model post-training module of this invention addresses the core issues of neutral and unpersonalized content generation by general large-scale models. It performs full-process stylized post-training for three predefined persona styles. Specifically, it first collects a large-scale public internet dialogue corpus and expands it by synthesizing dialogues using a high-quality large-scale model, enriching the diversity of scene coverage. Then, a large-scale model-assisted filtering and style classification pipeline is used to eliminate low-quality samples that do not meet style requirements, ensuring the style purity of the training corpus. The filtered corpus is then divided into training and testing sets. A structured training context is constructed by concatenating personalized system prompts with dialogue history, enabling the model to explicitly complete the generation task based on a specified persona. After fine-tuning the model on this stylized corpus, a comprehensive evaluation of the style fidelity and fluency of the generated content is conducted. Based on the evaluation feedback, the corpus construction process is iteratively optimized, forming a complete closed-loop pipeline that continuously strengthens the model's persona expression ability and the quality of generated content. The overall process is as follows: Figure 3 As shown.

[0025] Inspired by the hierarchical architecture of computer memory, the layered memory mechanism module adopts a three-level decoupled memory architecture (L1 / L2 / L3) to separate front-end completion generation from back-end memory management, enabling hierarchical management and efficient retrieval of user information at different levels. The L1 high-speed cache layer stores key-value (KV) cache blocks corresponding to specific styles and individual memories, allowing pre-calculated KV states to be directly injected into the front-end generation path, ensuring fast front-end response times. The L2 plaintext fact layer serves as an auditable source of user factual information, appending structured memory records in plaintext while maintaining a lightweight HNSW vector index for rapid semantic retrieval of memory content. The L3 parameter weight layer accumulates user interaction trajectories and model decision logs, automatically generating compliant datasets for subsequent incremental fine-tuning on the client side, achieving deep parameterization of user features. Since performing full memory retrieval in every prediction step incurs significant unnecessary computational overhead and severely impacts the real-time performance of input, this invention introduces a memory trigger based on the GRPO algorithm to selectively activate the memory retrieval function at appropriate times. This trigger predefines four core tasks: direct completion task supporting foreground input, memory retrieval task triggering query when factual information is required, memory retrieval task converting interaction trajectories into structured records and storing them in memory, and invalid information rejection task rejecting noise or sensitive segments. After stylization and training, the model's boundary between retrieval and rejection remains unclear. Therefore, this invention uses the GRPO algorithm to sharpen this boundary. Its reward function comprehensively considers three dimensions: format compliance, task execution correctness, and latency constraints, ensuring accurate triggering of the memory function while avoiding additional computational overhead that could affect the real-time response speed of the input.

[0026] The edge-side inference efficiency optimization module manages the KV cache as a compressed prefix tree structure, enabling structure sharing between overlapping prefixes generated by user input and editing. Instead of performing full pre-filling from the beginning each time, it automatically locates the longest common prefix and continues decoding from the matching node, transforming the interactive editing process into an incremental tree traversal process, significantly reducing redundant computation overhead. For the mainstream big.LITTLE architecture on mobile devices, this invention further applies a thread affinity scheduling strategy, scheduling computationally intensive attention calculations and GEMM kernels to high-performance cores, effectively reducing long-tail latency in inference. For memory-enhanced generation scenarios, this invention uses a lightweight KV injection mechanism, leveraging position offset mapping to inject pre-computed memory KV fragments into active decoding paths. Combined with phase shift correction and selective tail recomputation techniques, it fully preserves the causal attention mechanism, achieving flexible cache context splicing while minimizing recomputation overhead. This significantly reduces the cost of injecting memory content while ensuring smooth and coherent generated content.

[0027] While there are alternative solutions that can achieve similar objectives, these solutions have significant shortcomings in terms of overall effectiveness and design, failing to achieve the comprehensive technical effect of this invention. For example, in the personalized generation stage, a static character preset scheme based on prompt word engineering can be used instead of the stylized post-training scheme of this invention. This eliminates the need for model fine-tuning, guiding the model to generate content in the corresponding style simply by inputting prompt words. However, this scheme results in low style fidelity, failing to achieve fine-grained personalized expression. Furthermore, each generation requires a large number of prompt words, increasing context length and computational overhead, and failing to meet the real-time requirements of high-frequency input scenarios. In the memory mechanism stage, a single vector database retrieval scheme can be used instead of the three-level hierarchical memory architecture of this invention, storing all user historical information in a vector database and performing a full retrieval each time. However, this scheme cannot distinguish between high-frequency and low-frequency information retrieval efficiency, resulting in high retrieval latency. It also lacks the ability to parameterize user long-term behavior, failing to achieve continuous self-evolution. Frequent full retrievals lead to a large amount of redundant computation, severely impacting edge operating efficiency and device battery life.

[0028] In the edge-side inference optimization stage, a general edge-side inference framework can be used to replace the customized end-to-end optimization scheme of this invention, directly using an open-source framework to deploy the model on the edge. However, the general framework is not customized for scenarios with high-frequency input and frequent editing in input methods, and cannot achieve KV cache sharing and incremental decoding of the prefix tree structure. In interactive input scenarios, it suffers from high latency and large memory consumption, failing to meet the user experience requirements of millisecond-level response. In the quality control stage of generated content, a cloud-based large model can be used as a judge to evaluate and filter the candidate content generated on the edge. However, this approach requires uploading user input data to the cloud, violating the core design principle of privacy protection in this invention. At the same time, cloud calls will bring unpredictable network latency, failing to guarantee real-time performance in high-frequency input scenarios. Therefore, although there are multiple alternative technical paths, the HUOZIIME system proposed in this invention, which features full edge-side integration, deep fusion, hierarchical memory, and end-to-end optimization, has comprehensive advantages that existing alternatives cannot match in achieving the comprehensive goals of deep personalization, low-latency real-time response, and end-to-end privacy protection.

[0029] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent substitutions, and improvements made to the above embodiments without departing from the scope of the present invention, based on the technical essence of the present invention and within the spirit and principles of the present invention, shall still fall within the protection scope of the present invention.

Claims

1. A memory-enhanced deep personalized input method system based on end-side large language model, characterized in that, include: The module includes a front-end real-time interaction module, a stylized large model post-training module, a hierarchical memory mechanism module, and an edge-side inference optimization module. The front-end real-time interaction module integrates the completion content generated by the large model into the candidate word-based input workflow. By displaying ghost text at the cursor position and alternative generation results in the secondary panel, it achieves a deep integration of the large model generation capability with the core process of the input method. The stylized large model post-training module transforms the general lightweight large model into a dedicated backbone network for input methods with human-person perception capabilities through a closed-loop pipeline of corpus collection, synthesis expansion, filtering and iterative optimization. The hierarchical memory mechanism module adopts a three-level decoupled architecture consisting of an L1 high-speed cache layer, an L2 plaintext fact layer, and an L3 parameter weight layer. It separates the front-end completion generation from the back-end memory management and controls the boundaries of memory retrieval, extraction, and rejection through a memory trigger based on the GRPO algorithm. The edge-side inference optimization module achieves low-latency inference of large models on resource-constrained mobile terminals through KV cache reuse based on prefix trees, thread affinity scheduling, and memory injection mechanisms.

2. The memory-enhanced deep personalized input method system based on a large edge language model according to claim 1, wherein the stylized large model post-training module achieves model conversion in the following manner: Collect publicly available dialogue data from the internet and expand the data by synthesizing dialogues using large-scale models; A large model-assisted filtering and style classification pipeline is used to remove low-quality samples and samples that do not meet style requirements. The filtered corpus was divided into a training set and a test set. A structured training context was constructed by splicing personalized system prompts with dialogue history. The model is fine-tuned based on the structured training context, and the style fidelity and sentence fluency of the generated content are evaluated. Based on the evaluation feedback, the corpus construction process is iteratively optimized to form a closed-loop pipeline.

3. The memory-enhanced deep personalized input method system based on a large edge-side language model according to claim 1, wherein the three-level decoupling architecture of the hierarchical memory mechanism module includes: The L1 cache layer is used to store KV cache blocks corresponding to specific styles and single memories, and injects the pre-computed KV state into the front-end generation path; The L2 plaintext fact layer is used to append structured memory records in plaintext form and maintain a lightweight HNSW vector index to achieve fast semantic retrieval of memory content. The L3 parameter weight layer is used to accumulate user interaction trajectories and model decision logs, and automatically generate a dataset that meets the requirements for subsequent edge-side incremental fine-tuning.

4. The memory-enhanced deep personalized input method system based on a large language model on the device side according to claim 1, wherein the memory trigger based on the GRPO algorithm controls the memory function by pre-defining direct completion task, memory retrieval task, memory extraction task and invalid information rejection task; The GRPO algorithm is used to sharpen the boundary between retrieval and rejection. The reward function of the GRPO algorithm comprehensively considers format compliance, task execution correctness and delay constraints.

5. The memory-enhanced deep personalized input method system based on a large language model on the client side according to claim 1, wherein the client-side inference optimization module manages the KV cache as a compressed prefix tree structure, thereby achieving structure sharing between the overlapping prefixes generated by user input and editing; Automatically locate the longest common prefix, continue decoding from the matching node, and transform the interactive editing process into an incremental tree traversal process to achieve KV cache reuse.

6. In the memory-enhanced deep personalized input method system based on the terminal-side large language model as described in claim 1, in the terminal-side inference optimization module, for the big.LITTLE architecture of the mobile terminal, the computationally intensive attention calculation and GEMM kernel are scheduled to be executed by the high-performance core to achieve thread affinity scheduling.

7. The memory-enhanced deep personalized input method system based on a large language model on the edge as described in claim 1, wherein the edge inference optimization module injects pre-computed memory KV fragments into the active decoding path through a lightweight KV injection mechanism and by means of position offset mapping; By combining phase shift correction and selective tail recomputation techniques to preserve the causal attention mechanism, cache context splicing is achieved and recomputation overhead is minimized, thus realizing memory injection.

8. The memory-enhanced deep personalized input method system based on a large edge-side language model according to claim 1 further includes a cross-application context synchronization mechanism based on a model context protocol, wherein the cross-application context synchronization mechanism is implemented in the following ways: Under the constraints of Android system sandbox isolation, lightweight cross-process context sharing is achieved; Perform graceful degradation when the protocol becomes unavailable to ensure the normal operation of basic input functions.

9. The memory-enhanced deep personalized input method system based on a large language model on the client side according to claim 1, wherein the front-end real-time interaction module integrates the large model generation capability with the core input method process in the following way: Set up a ghost text area immediately adjacent to the cursor position to display the best short sentence candidate content in real time; Set up a two-layer candidate interface in the secondary panel to display the alternative generated results.

10. The memory-enhanced deep personalized input method system based on a large language model on the client side according to claim 1, wherein the memory-enhanced deep personalized input method system achieves end-to-end client-side retention of user input data through client-side deployment without any cloud dependency, wherein the client-side deployment includes: The end-to-end inference operation of the large language model, the end-to-end operation of the hierarchical memory mechanism, and the end-to-end incremental fine-tuning of user-personalized features.

Images