An Associative Input Method Based on a Large Model
By constructing a large language model based on Transformer, directly using original Mongolian characters as the basis, the problems of information loss and shallow semantic understanding in Mongolian input technology are solved, achieving high-fidelity word-level association and fast interaction, which is suitable for Mongolian text input and human-computer dialogue.
Patent Information
- Application Number
- CN202611553493.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-10-09
- Publication Date
- 2026-11-10
AI Technical Summary
Existing Mongolian input technology relies on Latin transliteration, which leads to information loss and shallow semantic understanding. It cannot achieve end-to-end context awareness and word-level association, and confuses the boundaries between input completion and association functions.
We construct a large language model based on the Transformer architecture, directly using original Mongolian Unicode characters as basic units to train an autoregressive model, realize multi-round interactive word association, capture long-distance dependencies through a self-attention mechanism, and introduce key-value caching to optimize inference speed.
It achieves high-fidelity Mongolian semantic understanding, supports multi-turn chain association, improves the coherence and naturalness of input content, is suitable for Mongolian text input, writing assistance and human-computer dialogue, and maintains fast response on resource-constrained devices.
Smart Images

Figure CN123067732A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of natural language processing and intelligent human-computer interaction, specifically to a large-model-based associative input method. This method utilizes deep learning technology to model the semantic structure and contextual dependencies of Mongolian text, achieving intelligent associative prediction at the complete word level. It is applicable to scenarios such as Mongolian text input, writing assistance, and human-computer dialogue. Background Technology
[0002] With the rapid development of deep learning and Large Language Model (LLM) technologies, intelligent input methods have gradually evolved from traditional methods based on lexicon matching or n-gram statistical models to generative interactive systems centered on context awareness and semantic understanding. In high-resource languages such as Chinese and English, large language models based on the Transformer architecture have been widely applied to tasks such as word-level association, sentence completion, and even paragraph generation, significantly improving user input efficiency and naturalness of expression. The core advantage of this type of system lies in its ability to capture long-distance dependencies through autoregressive modeling and intelligently predict the next most likely complete word in the subsequent context after the user inputs one or more complete words, thereby achieving true "semantic association" rather than simple character or syllable completion.
[0003] However, in the fields of low-resource minority languages such as Mongolian, the development of intelligent input technology is still severely lagging behind. Currently, mainstream Mongolian input methods mostly employ two technical paths: one is a prefix matching mechanism based on a static dictionary, which only supports spell completion for currently incomplete words; the other is the introduction of simplified recurrent neural networks (RNNs) or shallow Transformer models, but the training corpus is limited in scale, and often Mongolian is first transcribed into Latin letters, then modeled on the Latin sequence, and finally restored to traditional Mongolian characters through a mapping table. This indirect processing method of transcription-modeling-restoration has multiple drawbacks: First, the Latin transcription process inevitably loses Mongolian's unique orthographic information (such as vowel harmony, consonant assimilation, and intra-word ligature rules); second, the transcription mapping is not a one-to-one correspondence, easily leading to ambiguity or incorrect restoration; more importantly, the model learns the statistical patterns of Latin symbols rather than the linguistic structure of Mongolian itself, making it difficult to accurately reflect real pragmatic habits.
[0004] Furthermore, existing Mongolian input systems generally blur the functional boundaries between "input completion" and "contextual association." The former addresses incomplete words (e.g., when a user inputs "ᠪᠢᠳ," it completes to "ᠪᠢᠳᠡ"), which is a character-level task. The latter, however, should predict semantically related subsequent words such as "ᠬᠡᠦᠬᠡᠨ" (girl) and "ᠦᠵᠡᠮᠵᠢ" (scenery) after the user completes a full word (e.g., "ᠰᠠᠶᠢᠬᠠᠨ"), which is a word-level semantic reasoning task. Currently, there is no publicly available technical solution that can construct an end-to-end large language model using raw Mongolian Unicode characters as direct input and output units, achieving multi-turn, autoregressive, and context-aware complete word association.
[0005] Therefore, there is an urgent need for an associative input method driven by a large language model that does not rely on Latin transliteration, does not involve character encoding conversion, and is entirely based on training on the original Mongolian corpus, in order to break through the bottlenecks of existing technologies in semantic depth, language fidelity and interactive intelligence, and provide core technical support for the informatization of Mongolian. Summary of the Invention
[0006] To overcome the shortcomings of existing Mongolian input technologies, such as shallow semantic understanding, reliance on Latin transliteration, and obfuscation completion and association functions, this invention proposes a large-model-based associative input method. This method uses original Mongolian Unicode characters as the basic processing unit to construct an end-to-end autoregressive language model. After a user inputs a complete Mongolian word, it dynamically predicts the next most likely complete word to appear in the context and supports multi-round interactive chain-like association.
[0007] The specific technical solution provided by this invention is as follows:
[0008] An associative input method based on a large model includes the following steps:
[0009] Step 1: Construction of Mongolian Large Language Model: Collect a large-scale Mongolian text corpus, clean and segment it, and train an autoregressive Mongolian large language model based on the Transformer architecture using the original Mongolian characters as the basic units.
[0010] Step 2: Receive the current Mongolian word input by the user, wherein the Mongolian word is composed of standard Mongolian Unicode characters;
[0011] Step 3: Input the Mongolian words as tokens into the large language model to obtain the probability distribution of the next complete Mongolian word predicted by the model based on the current context.
[0012] Step 4: Select high-probability words from the probability distribution to form a candidate word list. The candidate word list contains only grammatically valid and semantically coherent complete Mongolian words.
[0013] Step 5: Show the user the list of candidate related words and receive the user's selection of one of the candidate words;
[0014] Step 6: Append the user-selected word to the end of the input sequence and repeat steps 3 to 5 to perform multiple rounds of autoregressive word association until the model outputs a termination signal or the user actively ends the process.
[0015] Furthermore, the specific construction process of the Mongolian language model is as follows:
[0016] Step 1.1: Obtain the original Mongolian text corpus. All texts are represented using traditional Mongolian characters encoded in Unicode standard. Then, perform corpus cleaning and standardization.
[0017] Step 1.2: The Mongolian text is segmented using an unsupervised word segmentation strategy based on sub-word units. Specifically, the word segmenter is trained directly on the original Mongolian character sequence without introducing any intermediate representation.
[0018] Step 1.3: Construct an autoregressive Mongolian large language model using the standard Transformer decoder architecture. This large language model consists of... It consists of stacked Transformer layers, and training uses the maximum likelihood estimation criterion.
[0019] Furthermore, the Transformer layer includes a multi-head self-attention mechanism and a position feedforward neural network, supplemented by residual connections and layer normalization;
[0020] Given an input sequence of Mongolian words, the large language model first maps it into a dense vector through an embedding layer, and then introduces positional encoding to obtain the final input.
[0021] Subsequently, each Transformer layer computes the context-aware representation in turn. The multi-head self-attention mechanism allows large language models to pay attention to different parts of the context in parallel in different subspaces.
[0022] The large language model training adopts the maximum likelihood estimation criterion, and learns the natural language co-occurrence rules and semantic dependencies among Mongolian words by minimizing the negative log-likelihood loss function;
[0023] The predicted probability is obtained by linear projection and Softmax normalization of the last hidden state of the Transformer decoder.
[0024] Furthermore, the Mongolian word input by the user is a grammatically complete and semantically independent word unit.
[0025] Furthermore, the candidate associative word list is obtained through a restricted sampling strategy, and then the sampling results are post-filtered. This post-filtering includes removing punctuation marks, function words, sub-word fragments, and outliers that do not conform to the basic word class distribution of Mongolian, ultimately forming a list of... A list of candidate related words composed of complete Mongolian words.
[0026] Furthermore, it includes a built-in intelligent termination mechanism: during each prediction, a special termination term is calculated simultaneously. <eos>The system outputs a probability. When this probability exceeds a preset threshold, the system determines that the current semantic unit is complete, automatically stops the association, and prompts the user to end the input.
[0027] Furthermore, the multi-round autoregressive word association is specifically as follows: each round predicts only one subsequent complete Mongolian word, which is added to the context sequence after user confirmation, serving as a new prefix for the next round of prediction; in the... During round prediction, before system cache The key-value matrix calculated for each word in each Transformer layer only needs to calculate its corresponding query matrix and perform attention calculation with the key-value matrix in the cache when processing new input words, without having to repeatedly process the historical sequence.
[0028] The beneficial effects of this invention are as follows:
[0029] 1. High language fidelity and avoidance of information loss: This invention directly uses original Mongolian Unicode characters as the basic processing unit, without the need for Latin transliteration or character set mapping. This effectively avoids the loss of Mongolian-specific orthographic information (such as vowel harmony, consonant assimilation, and intra-word ligature rules) during transliteration, ensuring that the model learns the linguistic structure of Mongolian itself rather than the statistical patterns of Latin symbols, significantly improving the accuracy and fidelity of language processing.
[0030] 2. Deep Semantic Understanding, Achieving Word-Level Association: Unlike traditional input methods that only provide character-level spell completion, this invention achieves complete word-level semantic association based on a large language model. The model can capture long-distance dependencies and predict the most likely subsequent complete word in the context after the user inputs a complete word, achieving true semantic association and significantly improving the coherence and naturalness of the input content.
[0031] 3. Strong interactive intelligence and support for chain-like expansion: This invention supports multi-round interactive chain-like association, where the prediction result of each round can serve as context for the next round. This autoregressive interaction method can assist users in constructing entire sentences or even paragraphs, and is particularly suitable for scenarios such as Mongolian text input, writing assistance, and human-computer dialogue.
[0032] 4. High inference efficiency and fast response speed: In the multi-round association process, this invention introduces key-value caching (KV Cache) technology to cache the key and value matrix of the historical sequence, avoiding repeated calculation of the historical context during each prediction. This reduces the time complexity of each round of inference from linear to constant level, significantly improving the response speed and ensuring smooth interaction even on resource-constrained devices.
[0033] 5. Strong generalization ability and adaptability to open domain scenarios: The model adopts an unsupervised word segmentation strategy based on sub-word units (such as BPE or SentencePiece), which can control the vocabulary size and handle out-of-vocabulary (OOV) words through sub-word combination mechanism. This enables the model to still have good generalization ability and prediction accuracy when facing new words, proper nouns or low-frequency words. Attached Figure Description
[0034] Figure 1 This is the overall flowchart of the method.
[0035] Figure 2 A flowchart for model building and training.
[0036] Figure 3 This is a flowchart of the online reasoning and association interaction process.
[0037] Figure 4 The diagram illustrates the working principle of key-value caching, where (a) represents the traditional method and (b) represents the KV Cache optimization of this invention. Detailed Implementation
[0038] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.
[0039] An associative input method based on a large model, such as Figure 1 As shown, it includes the following steps:
[0040] The first stage is the construction and training of the large language model.
[0041] Step 1: Construction of the Mongolian Large Language Model. A large-scale Mongolian text corpus was collected, cleaned, and segmented. Using original Mongolian characters as basic units, an autoregressive Mongolian large language model based on the Transformer architecture was trained. The vocabulary of the constructed Mongolian large language model is directly built from Mongolian words and sub-words, without the need for intermediate transcription steps. For example... Figure 2 As shown, in a specific example, there are the following sub-steps.
[0042] Step 1.1: Obtain a large-scale corpus of raw Mongolian texts covering news, literature, government affairs, education, and other fields. All texts are represented using traditional Mongolian characters encoded in the Unicode standard (U+1800–U+18AF), including basic Mongolian letters, long vowel symbols, separators, and legal connected forms. After cleaning, HTML tags, special symbols, duplicate paragraphs, and low-quality segments are removed from the corpus. Standardization processing is also performed, such as unifying full-width / half-width punctuation and correcting illegal character combinations, to ensure the standardization and consistency of the corpus.
[0043] Step 1.2: Mongolian text is segmented using an unsupervised word segmentation strategy based on subword units. Specifically, the segmenter is trained directly on the original Mongolian character sequence using SentencePiece or Byte Pair Encoding (BPE) algorithms, without introducing any intermediate representations (such as Latin transliteration). This process automatically learns lexical boundaries and common affix structures by iteratively merging high-frequency adjacent character pairs, ultimately generating a vocabulary that balances the decomposability of high-frequency complete words and low-frequency words. This design effectively controls the vocabulary size, avoiding the sparsity problem caused by vocabulary explosion in traditional word-level modeling, and also handles out-of-vocabulary (OOV) words through subword combination mechanisms, significantly improving the model's generalization ability.
[0044] Step 1.3: Subsequently, an autoregressive Mongolian large language model is constructed using the standard Transformer decoder architecture. This model is... It consists of stacked Transformer layers, each containing a multi-head self-attention mechanism and a position-wise feed-forward network, supplemented by residual connections and layer normalization. Given an input word sequence... Each of them As a Mongolian word element, This is a vocabulary set containing all possible Mongolian lexical units. These Mongolian lexical units can be complete words or sub-words, and the model first maps them into dense vectors through an embedding layer.
[0045]
[0046] in For learnable word embedding matrix, For vocabulary size, To hide the dimension, positional encoding is introduced to preserve sequence order information. Typically, sine / cosine functions or learnable position embeddings are used. This represents the length of the input sequence, i.e., the number of tokens. The final input is represented as follows.
[0047]
[0048] in, This represents the initial input representation of the Transformer model. Subsequently, each Transformer layer computes its context-aware representation in turn:
[0049]
[0050] in, Indicates the first The contextualized representation output by the Transformer layer. Indicates the first Transformer layer functions of the layer, This represents the total number of layers in the Transformer model. Multi-head self-attention allows the model to focus on different parts of the context in parallel across different subspaces; its calculation formula is:
[0051]
[0052]
[0053]
[0054] Here These are query, key, and value matrices, respectively. Indicates the number of attention heads. Scaling factor The linear projection matrix after multi-head attention output. They represent the first The query, key, and value weight matrix corresponding to each head. This represents the output of the multi-head attention mechanism. This indicates a splicing operation. Indicates the first The output of each attention head This represents the Scaled Dot-ProductAttention function. The normalized exponential function is used to transform attention scores into a probability distribution. This mechanism allows large language models to capture dependencies between any two lexical units, regardless of their distance. Training of large language models employs the maximum likelihood estimation criterion, which minimizes the negative log-likelihood loss function. :
[0055]
[0056] in Indicates the first The entire sequence of context words preceding each word, For all trainable parameters of the model, To train the sequence length, the large language model learns the natural language co-occurrence patterns and semantic dependencies among Mongolian words using this loss function. Specifically, the first... Predicted probability of step Hidden state through the last layer of the Transformer decoder The result obtained after linear projection and Softmax normalization is:
[0057]
[0058] in To output the projection matrix, For bias terms, as a word element Embedded vector, For traversing any lexical index in the vocabulary The embedding vector (usually shares weights with the input embedding to reduce the number of parameters). Indicates a word list The summation of all lexical units in the model is used for probability normalization. Through this training objective, the model gradually learns the deep linguistic rules of Mongolian, such as syntactic structure, semantic collocation, and topic coherence, laying the foundation for subsequent associative reasoning.
[0059] Next is the interactive associative reasoning process. After the model training is complete, it enters the online reasoning stage. The core of this invention lies in achieving multi-turn, context-aware, word-level intelligent association, and its workflow strictly follows the user interaction rhythm, such as... Figure 3 As shown, the specific steps are as follows.
[0060] Step 2: Receive the current Mongolian word input by the user, perform word encoding and context construction. The word is composed of standard Mongolian Unicode characters.
[0061] Specifically, the system receives the current Mongolian words input by the user. It is emphasized here that... It must be a grammatically complete and semantically independent word unit, such as the noun "ᠰᠠᠶᠢᠬᠠᠨ" (beautiful), the verb "ᠪᠠᠷᠢᠬᠤ" (to catch), or the pronoun "ᠲᠡᠷᠡ" (he / she), rather than partial syllables (such as "ᠰᠠᠶ") or incomplete stems (such as "ᠪᠠᠷᠢ"). This design fundamentally distinguishes this invention from traditional "input completion" technology—the latter aims to complete the spelling of the current word, while this invention predicts the next logically occurring complete word in the context after the user completes a complete word. The initial context sequence is constructed as follows: And it is fed as input into the pre-trained large language model.
[0062] Step 3: Input the Mongolian words as tokens into the large language model to obtain the probability distribution of the next complete Mongolian word predicted by the model based on the current context.
[0063] The model performs one forward propagation, computing the results at a given time. Given the condition, the conditional probability distribution of all possible next words:
[0064]
[0065] in, For the decoder at time step The context of hidden state, For model parameters, and This is for outputting projection parameters.
[0066] Step 4: Select several high-probability words from the probability distribution to form a candidate word list. The candidate word list only contains grammatically valid and semantically coherent complete Mongolian words, excluding stems, affixes, incomplete forms, or intermediate representations requiring further morphological changes.
[0067] To generate a high-quality and diverse candidate list, this embodiment employs a restricted sampling strategy. This avoids the monotonicity caused by directly using greedy search (taking only the highest probability item) and also prevents unrestricted sampling from introducing unreasonable words, thus achieving a balance between semantic rationality and lexical diversity. Specific optional strategies include:
[0068] Top-K sampling: Selecting the highest probability The candidate set consists of 1 word elements. After renormalizing the probability, sampling is performed.
[0069] Nucleus Sampling: Dynamically Determining the Minimal Set , making the cumulative probability (generally Then sample from that set.
[0070] Beam Search: Maintenance The optimal partial sequences are expanded, and those with the highest overall scores are retained. This is suitable for scenarios where high continuity is required.
[0071] Regardless of the strategy employed, the system will perform post-filtering on the sampling results: removing punctuation marks, function words (such as particles and conjunctions), word fragments (that do not constitute complete semantic units), and outliers that do not conform to the basic word class distribution of Mongolian, ultimately forming a result consisting of... A list of candidate related words composed of complete Mongolian words. .
[0072] Step 5: Show the user the list of candidate suggested words. Receive user input for one of the candidate words The choice.
[0073] Step 6: Append the user-selected word to the end of the input sequence and repeat steps 3 to 5 to perform multiple rounds of autoregressive word association until the model outputs a termination signal or the user actively ends the process.
[0074] The system will immediately receive Append to the current context sequence and update to And immediately trigger the next round of predictions, in which This indicates a splicing operation. This process constitutes an autoregressive chain-like reasoning loop, which can be performed continuously for multiple rounds to gradually construct a semantically coherent and structurally sound word sequence such as "ᠪᠢ → ᠰᠠᠶᠢᠬᠠᠨ → ᠠᠮᠢᠳᠤᠷᠠᠬᠤ→ ᠪᠣᠯᠤᠨᠠ" (I → will → live well).
[0075] To prevent infinite generation, this invention incorporates a built-in intelligent termination mechanism. During each prediction, the model simultaneously calculates specific termination terms. <eos>(End-of-Sequence) Output Probability When the probability exceeds a preset threshold (For example When the system determines that the current semantic unit is complete, it automatically stops the association and prompts the user to end input. Alternatively, the user can also actively terminate the interaction at any time through explicit actions (such as pressing Enter or selecting the end option). This dual-safety mechanism ensures that the association process is both intelligent and controllable.
[0076] This invention supports multi-round interactive association: each round predicts only one subsequent complete Mongolian word, which is then added to the context sequence after user confirmation, serving as a new prefix for the next round's prediction, thus achieving chain-like semantic expansion. Furthermore, as the context sequence grows progressively during multi-round association, calculating attention from scratch each time would lead to severe redundancy and latency, resulting in a cumulative complexity of [missing information]. (Squared order). Therefore, this invention introduces key-value caching (KV Cache) technology. In the... During round prediction, before system cache The key-value matrix calculated for each lexical unit in each Transformer layer: Where l represents the level index, Indicates key-value caching. Indicates that the l-th layer is in the first place. During round prediction, the system caches the first... The key matrix calculated for each lexical unit in each Transformer layer. Indicates that the l-th layer is in the first place. During round prediction, the system caches the first... The key matrix is calculated for each lexical unit in each Transformer layer.
[0077] The cached data structure is maintained independently layer by layer from the first layer to the last layer (the Lth layer) of the Transformer. The historical key-value calculation results of each layer are persistently stored and will not be overwritten in subsequent inference rounds.
[0078] Compared to traditional cacheless inference methods, such as Figure 4 As shown in (a), each round of prediction requires an analysis of data from the first round. The word element to the current word The entire sequence of each word is recalculated using the complete KV matrix. The cumulative computational complexity increases quadratically with the sequence length.
[0079] In this KV Cache optimization scheme, such as Figure 4 As shown in (b), in the first round, only the key-value matrix corresponding to the first word is calculated and cached; from the second round onwards, each round of prediction directly reads all the cached previous key-value results, and only needs to calculate the current new input word. The corresponding key-value matrix, and only its corresponding query matrix needs to be calculated. and in the preceding cache , Attention calculations are performed without repeatedly processing the historical sequence. This mechanism reduces the cumulative time complexity of each round of inference from quadratic to quadratic levels compared to traditional methods. Reduced to linear level This significantly improves response speed and ensures smooth interaction even on resource-constrained devices.
[0080] It should be noted that this method operates entirely at the language model inference level and is only used to generate semantically coherent subsequent complete Mongolian words. Both the input and output are standard Mongolian Unicode character sequences, and it does not involve character encoding conversion, Latin transliteration, glyph rendering, or input method front-end control.
[0081] It is understood that the present invention has been described through some embodiments, and those skilled in the art will recognize that various changes or equivalent substitutions can be made to these features and embodiments without departing from the spirit and scope of the invention. Furthermore, under the teachings of the present invention, these features and embodiments can be modified to adapt to specific situations and materials without departing from the spirit and scope of the invention. Therefore, the present invention is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the protection scope of the present invention.< / eos> < / eos>
Claims
1. An associative input method based on a large model, characterized in that, Includes the following steps: Step 1: Construction of Mongolian Language Model: Collect Mongolian text corpus, clean and segment it, and train an autoregressive Mongolian language model based on the Transformer architecture using the original Mongolian characters as the basic units. Step 2: Receive the current Mongolian word input by the user, wherein the Mongolian word is composed of standard Mongolian Unicode characters; Step 3: Input the Mongolian words as word tokens into the large language model to obtain the probability distribution of the next complete Mongolian word predicted by the model based on the current context; Step 4: Select high-probability words from the probability distribution to form a candidate word list. The candidate word list contains only grammatically valid and semantically coherent complete Mongolian words. Step 5: Show the user the list of candidate related words and receive the user's selection of one of the candidate words; Step 6: Append the user-selected word to the end of the input sequence and repeat steps 3 to 5 to perform multiple rounds of autoregressive word association until the model outputs a termination signal or the user actively ends the process.
2. The associative input method based on a large model according to claim 1, characterized in that, The specific construction process of the Mongolian language model is as follows: Step 1.1: Obtain the original Mongolian text corpus. All texts are represented using traditional Mongolian characters encoded in Unicode standard. Then, perform corpus cleaning and standardization. Step 1.2: The Mongolian text is segmented using an unsupervised word segmentation strategy based on sub-word units. Specifically, the word segmenter is trained directly on the original Mongolian character sequence without introducing any intermediate representation. Step 1.3: Construct an autoregressive Mongolian large language model using the standard Transformer decoder architecture. This large language model consists of... It consists of stacked Transformer layers, and training uses the maximum likelihood estimation criterion.
3. The associative input method based on a large model according to claim 2, characterized in that, The Transformer layer includes a multi-head self-attention mechanism and a position feedforward neural network, supplemented by residual connections and layer normalization; Given an input sequence of Mongolian words, the large language model first maps it into a dense vector through an embedding layer, and then introduces positional encoding to obtain the final input. Subsequently, each Transformer layer computes the context-aware representation in turn. The multi-head self-attention mechanism allows large language models to pay attention to different parts of the context in parallel in different subspaces. The large language model training adopts the maximum likelihood estimation criterion, and learns the natural language co-occurrence rules and semantic dependencies among Mongolian words by minimizing the negative log-likelihood loss function; The predicted probability is obtained by linear projection and Softmax normalization of the last hidden state of the Transformer decoder.
4. The associative input method based on a large model according to claim 3, characterized in that, The Mongolian word entered by the user is a grammatically complete and semantically independent word unit.
5. The associative input method based on a large model according to claim 4, characterized in that, The candidate word list is obtained through a restricted sampling strategy, and then the sampling results are post-filtered. This post-filtering includes removing punctuation marks, function words, sub-word fragments, and outliers that do not conform to the basic word class distribution of Mongolian. The final result is a list of... A list of candidate related words composed of complete Mongolian words.
6. The associative input method based on a large model according to claim 5, characterized in that, It also includes a built-in intelligent termination mechanism: during each prediction, a special termination term is calculated simultaneously. <eos> The system outputs a probability. When this probability exceeds a preset threshold, the system determines that the current semantic unit is complete, automatically stops the association, and prompts the user to end the input.< / eos> 7. The associative input method based on a large model according to claim 6, characterized in that, The multi-round autoregressive word association is described in detail below: In each round, only one subsequent complete Mongolian word is predicted. After user confirmation, it is added to the context sequence as a new prefix for the next round of prediction; in the... During round prediction, before system cache The key-value matrix calculated for each lexical unit in each Transformer layer only needs to calculate its corresponding query matrix and perform attention calculation with the key-value matrix in the cache when processing new input lexical units, without having to repeatedly process the historical sequence.
8. The associative input method based on a large model according to claim 7, characterized in that, The restricted sampling strategies include Top-K sampling, kernel sampling, or beam search.
