A retrieval enhancement generation method based on traditional mongolian roots

Abstract

The application discloses a retrieval enhancement generation method based on traditional Mongolian roots, and belongs to the technical field of natural language processing. The method first realizes the unification and normalization of text representation; secondly, the normalized text is subjected to root-morpheme disassembly analysis, and through root-oriented semantic enhancement training, combined with the root-level sub-word coding strategy of OOV words, the completeness and accuracy of semantic representation are significantly improved; finally, the semantic knowledge obtained through retrieval is deeply fused with the internal semantics of the generative language model, and through the generation control and consistency constraint mechanism at the root level, the accuracy and reliability of the model output are ensured. The application effectively solves the semantic matching problem caused by the traditional Mongolian word diversity, significantly improves its performance in information retrieval and generative tasks, and can be widely applied to multiple technical scenes such as question and answer systems, machine translation, knowledge base construction and text generation.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, and in particular to a retrieval enhancement generation (RAG) method based on traditional Mongolian word roots. Background Technology

[0002] Traditional Mongolian script, also known as Uyghur-style Mongolian script, is a script used to write the Mongolian language. It is written vertically from top to bottom and left to right. Traditional Mongolian script is an important script for the Mongolian people and one of the major minority scripts used in my country. It is a typical agglutinative language, with words typically composed of roots and multiple affixes. Affixes include word-forming affixes, inflectional affixes, and suffixes, and a single root can connect to dozens of different affixes. Due to the high agglutinative nature of its word formation, a single word can exhibit dozens or even hundreds of inflections, greatly increasing the complexity of natural language processing for Mongolian.

[0003] The complex structure and writing system of Mongolian further complicate its digitization. Traditional Mongolian is written vertically with connected strokes, and the morphology of letters varies significantly depending on their position within a word (beginning, middle, or end). Furthermore, the existence of free character variants (FVS) means that the same character may correspond to multiple encoding forms. Linguists estimate that the number of lexical combinations in Mongolian is enormous, exceeding tens of millions. Due to the complexity of its language structure and the scarcity of annotated corpora, traditional Mongolian has long been considered one of the most difficult scripts to automate.

[0004] In the field of natural language processing, traditional Mongolian has long suffered from poor performance in tasks such as information retrieval, syntactic analysis, machine translation, question answering, and text summarization. One key issue is that different word forms of the same root word vary greatly in text, making it difficult for word-based retrieval methods to effectively match cognate word forms, thus affecting the overall recall and accuracy of the model. Especially with the rapid development of large-scale language models, Retrieval Augmentation (RAG) technology has become a mainstream solution to improve the knowledge and accuracy of models. However, traditional RAG models are mainly designed for languages ​​with stable writing structures, such as Chinese and English, and lack adaptability for highly cohesive languages ​​like traditional Mongolian.

[0005] Existing research on Mongolian information retrieval mainly focuses on word segmentation, root word identification, and stemming. For example, methods such as "Automatic Segmentation System for Mongolian Roots, Stems, and Endings" and "Slavic Mongolian Word Segmentation System Based on Dictionary and Rules" primarily rely on dictionaries and rules to achieve coarse word segmentation. In contrast, methods like "Mongolian Word Segmentation Based on Statistical Language Models" and "Mongolian Word Segmentation System Based on Dictionary, Rules, and Statistics" combine language models for statistical word segmentation. These methods have improved the accuracy of traditional Mongolian word form analysis to some extent, but they still suffer from the following common problems.

[0006] (1) The word form changes are highly complex, making it difficult to extract features. Because the word form changes in traditional Mongolian are extremely rich, it is difficult for rule-based or statistical feature-based methods to cover all language phenomena. As a result, semantic matching has long relied on limited rules or manual features, which cannot adapt to unknown word forms or complex combinations.

[0007] (2) High labor costs and lack of system scalability. Traditional methods generally require a large amount of manual annotation of word roots, affixes, parts of speech, and maintenance of dictionaries and rule bases. Given the scarcity of traditional Mongolian corpora, the high cost severely restricts the system's iterative capabilities.

[0008] (3) The problems of out-of-vocabulary (OOV) words and sparse word forms are serious. Due to the limited training corpus and the huge variations in word forms, a large number of words in traditional Mongolian texts are out-of-vocabulary (OOV) words. Schemes based on statistical models are prone to problems of insufficient training and weak generalization ability.

[0009] (4) Existing retrieval methods cannot handle the diversity of traditional Mongolian word forms. Traditional retrieval methods based on word literal matching or general embedding have difficulty identifying the semantic consistency of cognate word forms, resulting in the inability to recall relevant documents and seriously affecting the performance of retrieval-based question answering, summarizing and knowledge enhancement tasks.

[0010] (5) Existing RAG technology is not adapted to the characteristics of traditional Mongolian language. Mainstream RAG methods rely on accurate retrieval to improve the knowledge content of generative language models. However, in traditional Mongolian language scenarios, due to inconsistent encoding, inconsistent word forms, and lack of root-level retrieval capabilities, the retrieval results are poor, which affects the accuracy, controllability, and stability of generative language models.

[0011] Therefore, in order to improve the processing capabilities of traditional Mongolian in information retrieval, knowledge question answering and generative tasks, a new method is needed that can effectively extract word roots and normalize word forms based on the characteristics of traditional Mongolian word form structure, and support high-quality semantic retrieval enhancement generation. Summary of the Invention

[0012] This invention addresses the aforementioned problems by proposing a retrieval enhancement generation method based on traditional Mongolian word roots. It unifies word form representation from the perspective of word roots, improves retrieval recall and generation accuracy, thereby solving a long-standing difficulty in traditional Mongolian natural language processing.

[0013] To achieve the above objectives, this invention proposes a retrieval enhancement generation method based on traditional Mongolian word roots, comprising the following steps:

[0014] S1. Standardization of Traditional Mongolian Text: The input traditional Mongolian text is processed sequentially with Unicode normalization, free glyph selector standardization, glyph normalization, continuous writing structure parsing, and special symbol cleaning to obtain standardized traditional Mongolian text.

[0015] S2. Traditional Mongolian word root extraction: The standardized traditional Mongolian text is analyzed by word root-affix splitting using a bidirectional long short-term memory network-conditional random field (BiLSTM-CRF) model to extract stable word root sequences.

[0016] S3. Root-enhanced vector representation generation: Root vectors and affix vectors are constructed based on the root sequence, and enhanced vector representations are obtained through root-guided semantic enhancement training and root-level sub-word encoding of out-of-vocabulary words.

[0017] S4. Root-based retrieval enhancement generation mechanism: Using the enhanced vector representation as the retrieval key, a root semantic neighborhood-oriented knowledge retrieval mechanism is constructed, and the retrieval results are integrated with the generative language model. At the same time, through root-level generation control and consistency constraints, semantically consistent generated text is output.

[0018] Furthermore, the standardization process for the traditional Mongolian text is as follows:

[0019] Unicode standardization: unifying character combination methods;

[0020] Standardization of free glyph selectors: Construct a standard variant mapping table to uniformly map non-standard free glyph selector combinations to standard-defined glyph variants;

[0021] Glyph normalization: Mapping the position-dependent glyphs of the first, middle, and last glyphs to a unified basic letter code;

[0022] Connected writing structure analysis: Deconstructing the connected writing structure, breaking down the connected characters into standard character sequences;

[0023] Special symbol cleaning: Removes illegal control characters, scans optical characters to identify incorrect symbols, non-Mongolian encoded characters mixed in, and duplicate or semantically meaningless variant marks.

[0024] Furthermore, the extraction of traditional Mongolian word roots is as follows:

[0025] S201. Deconstruct and analyze the root-affix structure of traditional Mongolian words, and construct a traditional Mongolian affix library that includes case affixes, plural markers, derived suffixes, and functional suffixes;

[0026] S202. A bidirectional long short-term memory network (BiLSTM) expression coding model is used to perform bidirectional context encoding on the character-level sequence to obtain the character-level bidirectional hidden state; the character-level sequence is a traditional Mongolian character-level input sequence composed of normalized Mongolian individual characters in the order of writing;

[0027] S203. The bidirectional hidden state is input into the Conditional Random Field (CRF) sequence labeling boundary prediction model to perform global optimal sequence labeling, identify the start and end positions of word roots and affix boundaries, and output the set word root-affix boundary labeling sequence.

[0028] Furthermore, the enhanced vector representation is obtained through the following process:

[0029] S301. The letter-level Transformer encoding model is used to learn the input character vectors, and the letter-level Transformer encoding model is jointly trained through masked language modeling, autoregressive prediction and semantic contrastive learning to construct word root vectors.

[0030] S302. Construct independent embedding tables for affixes of different grammatical categories and dynamically fuse them with root vectors to obtain a fused representation of root vectors and affix vectors;

[0031] S303. Comparative learning is performed using words derived from the same root as positive samples and semantically irrelevant words as negative samples to optimize the vector space structure and obtain an enhanced root vector representation.

[0032] S304. Perform root-based sub-word encoding on out-of-vocabulary words: First, perform root-affix splitting and parsing on out-of-vocabulary words, then perform sub-word encoding on the remaining segments, and dynamically adjust the semantic contribution weight of sub-words.

[0033] Furthermore, S4 is specifically as follows:

[0034] S401. Merge the root vector, affix vector and query context encoding into a query vector;

[0035] S402. Filter candidate documents in the knowledge base based on the consistency or similarity of the root words between the query vector and the document to obtain a set of highly relevant texts;

[0036] S403. Dynamically inject highly relevant text into the hidden state of the generative language model through an attention mechanism, and calculate the probability distribution based on the updated hidden state of the generative language model.

[0037] S404. Add root word consistency constraints to the generation loss to ensure that the output text and the query root words remain semantically coherent.

[0038] Compared with the prior art, the present invention has the following technical advantages:

[0039] 1. Significantly Improves Retrieval Performance of Traditional Mongolian: As an agglutinative language, traditional Mongolian can be followed by numerous derived affixes, morphological affixes, and functional suffixes, resulting in dozens or even hundreds of different morphological variations for the same semantic unit. Traditional word-based retrieval methods cannot achieve stable cross-morphological matching. This invention unifies the originally scattered morphological forms into a standard root representation through Unicode normalization, FVS standardization, and unified glyph mapping. Subsequently, through a root-enhanced vector representation method, different derived morphological forms of the same root are aggregated in the vector space, significantly improving semantic consistency. This invention can significantly improve accuracy in cross-morphological retrieval scenarios in traditional Mongolian, effectively solving the problems of low recall and inaccurate matching in traditional methods, and providing quantifiable performance gains for traditional Mongolian information retrieval tasks.

[0040] 2. Enhancing the Reliability and Consistency of Generative Language Models: Traditional applications of traditional Mongolian based generative language models are often limited by scarce corpora and the complexity of language structure, leading to factual biases and semantic inconsistencies within the generated content. This invention constructs a root-based RAG structure, using high-quality traditional Mongolian knowledge fragments as contextual input to the generative language model. This enables the model to reason based on reliable knowledge sources when generating answers, summaries, translations, etc. Because the retrieval part uses a root-enhancement strategy, it can recall real knowledge semantically relevant to the user's input with a higher probability, thus significantly improving the accuracy, coherence, and consistency of the generated content.

[0041] 3. Fully adaptable to the linguistic characteristics of traditional Mongolian script, compatible with various writing and encoding variations: The vertical layout, continuous writing, position-dependent glyphs, and free glyph selector variations of traditional Mongolian script make its processing significantly more difficult than horizontal scripts. The standardization module proposed in this invention encompasses steps such as Unicode normalization, FVS mapping, first-middle-last glyph normalization, and continuous writing structure parsing, which can solve the text inconsistency problem caused by the lack of uniform encoding in traditional Mongolian script. The root word extraction module can stably extract root words from traditional Mongolian text. Furthermore, the processing flow of this invention relies entirely on the vertical writing logic design of traditional Mongolian script, enabling unified processing of various traditional Mongolian fonts, encodings, and variations, laying an accurate data foundation for subsequent retrieval and generation.

[0042] 4. Highly scalable and applicable to all traditional Mongolian natural language processing tasks: Because this invention establishes a unified representation system starting from the root structure of the language, its vector representation, retrieval structure, and generation enhancement modules are all universal and can be extended to various traditional Mongolian natural language processing (NLP) application scenarios such as question answering systems, semantic retrieval, document summarization, machine translation, knowledge base construction, dialogue systems, and text analysis. By uniformly using root representations in the knowledge base construction, retrieval enhancement, and generation enhancement stages, this invention forms a reusable, transferable, and generalizable traditional Mongolian processing framework, providing a solid primitive technology foundation for the future construction of a large-scale traditional Mongolian model ecosystem, a traditional Mongolian knowledge question answering platform, and an intelligent search system. Attached Figure Description

[0043] Figure 1 This is a flowchart of the method provided by the present invention. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0045] The application principle of the present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0046] A retrieval enhancement generation method based on traditional Mongolian word roots, such as Figure 1 As shown, it includes the following steps:

[0047] S1. Standardization of Traditional Mongolian Text: The input traditional Mongolian text is processed sequentially with Unicode normalization, FVS (Free Character Selector) standardization, character normalization, contiguous structure parsing, and special symbol cleaning to obtain standardized traditional Mongolian text.

[0048] This step is fundamental to the entire retrieval enhancement generation method. Its core objective is to address the inconsistencies in traditional Mongolian text caused by its agglutinative nature, unique writing style, and encoding variations, ultimately achieving a stable and consistent text representation. This normalization module encompasses five closely related sub-steps, fully adapting to the linguistic characteristics of traditional Mongolian and laying an accurate data foundation for subsequent root word extraction and retrieval.

[0049] S101. Perform Unicode normalization to unify character combination methods;

[0050] Traditional Mongolian script may generate various Unicode encoding forms during input, encoding, and storage, such as encoding splitting caused by different systems using Unicode composite and decomposed characters, and differences in font rendering. This step performs Unicode normalization on the original text, unifies the character combination method, eliminates the phenomenon of the same character having different codes due to encoding differences, and ensures that the same semantic unit has a consistent basic encoding representation at the character level.

[0051] S102. Construct a standard variant mapping table to standardize FVS (Free Character Selector);

[0052] Traditional Mongolian script uses FVS (Form Variant) in Unicode to decode different variants of the same letter. However, the use of FVS in actual text lacks a unified standard. Different fonts, input methods, and operating systems have different encoding habits for FVS, and variants of the same character are often arbitrarily inserted or omitted, leading to a chaotic encoding structure. This step solves the problem of encoding confusion for homographs by constructing a standard variant mapping table, uniformly mapping non-standard FVS combinations to standard-defined glyph variants, and removing redundant or incorrectly used FVS, thereby improving the consistency of word form recognition.

[0053] S103, Glyph Normalization: Mapping the position-dependent glyphs of the first, middle, and last glyphs to a unified basic letter encoding;

[0054] Traditional Mongolian characters exhibit three distinct glyphs—initial, middle, and final—depending on their position within a word, resulting in significant visual differences. This directly hinders text-level search and language model learning. This step addresses this by mapping all position-dependent glyphs to a unified basic alphabetic encoding using glyph mapping rules, eliminating character differences caused by writing position. As shown in Table 1, this Mongolian character... (No specific meaning) Different positions have different character shapes. This normalization can significantly improve the stability of word root recognition and vector modeling.

[0055] Table 1 Mongolian characters Font display in different positions

[0056] beginning of word In the words suffix

[0057] S104. Parse the continuous writing structure and restore the standard character sequence;

[0058] Traditional Mongolian script uses a connected stroke structure, with certain connected strokes forming special connected characters, such as the Mongolian character meaning "second". The ligature results in a visually significantly different form compared to the character-by-character representation: This step deconstructs the connected writing structure, breaking down the connected characters into standard character sequences to restore the true semantic structure. This enables the subsequent root extraction and morphological analysis processes to accurately locate root and affix boundaries.

[0059] S105, Cleaning of special symbols;

[0060] Traditional Mongolian text originates from various platforms, such as optical character recognition (OCR), web page capture, and input method autocorrect, and may contain redundant characters, invisible control characters, custom font symbols, and other noise. This step uses a self-built noise symbol filter table to remove the following abnormal symbols: illegal control characters (such as hidden characters like 0x200C, 0x200D, etc.), OCR error symbols, non-Mongolian encoded mixed characters, and duplicate or semantically meaningless variant markers, ultimately resulting in a clean and standardized Mongolian character sequence.

[0061] S2. Traditional Mongolian word root extraction: The standardized traditional Mongolian text is analyzed by word root-affix splitting using a bidirectional long short-term memory network-conditional random field (BiLSTM-CRF) model to extract stable word root sequences;

[0062] After completing the standardization process of the traditional Mongolian text in S1, the text has been unified into a standardized character sequence without redundant variants. In order to further address the problem of word form diversity caused by the agglutinative structure of traditional Mongolian, this step extracts word roots from the standardized text, separating the word roots from the complex word forms and establishing a unified semantic representation.

[0063] Traditional Mongolian words are generally composed of a root word plus multiple derived affixes, inflectional affixes, and functional affixes. A single root word can derive dozens or even hundreds of word forms. If direct matching is performed based on the literal meaning of the word, semantically consistent words will be scattered in the statistical space, affecting retrieval results and the performance of generative language models. Therefore, this invention uses the root word as the center for a unified expression, merging all word forms into a unique and stable semantic representation level, providing high-quality input for semantic retrieval and knowledge augmentation generation.

[0064] This step includes an explanation of the root and affix structure, as well as an automatic segmentation process based on the BiLSTM-CRF model.

[0065] S201. Deconstruct and analyze the root-affix structure of traditional Mongolian words, and construct a traditional Mongolian affix library that includes case affixes, plural markers, derived suffixes, and functional suffixes;

[0066] Before performing model-based automatic word segmentation, it is necessary to clarify the structural characteristics of traditional Mongolian word forms and the targets for root extraction from a linguistic perspective. Traditional Mongolian is a typical agglutinative language, and its word form construction follows the following principles:

[0067]

[0068] In this context, "Word" represents the traditional Mongolian script, and "Root" represents the word's root. Indicates derived affixes, morphological affixes, or grammatical affixes, where n represents the number of affixes in the word and i is the affix index.

[0069] Affixes are typically linked hierarchically according to functional categories, following the order in which outer grammatical affixes wrap inner derived structures. The affix library constructed in this invention includes:

[0070] Case affixes: including genitive, dative, instrumental, abductor, etc., such as (of) etc.;

[0071] Plural notation: such as wait;

[0072] Derivative suffixes: used to derive new words from a root word—potentially changing the part of speech (e.g., noun → verb / adjective) or introducing new meanings. For example... wait;

[0073] Functional / grammatical suffixes for verbs / adjectives / nouns: such as wait;

[0074] It should be noted that the above Mongolian affixes are suffixes that modify the part of speech and have no actual meaning.

[0075] By employing a sequential stripping strategy, registered affixes are identified and deleted level by level from bottom to top, ultimately retaining the core word roots. For example:

[0076] (Skilled) → (Yes, root word) + (civilized, derived affix) + (Exercise, derived affixes) + (Skilled, derived affix)

[0077] Therefore, the goal of word root extraction is to identify word root boundaries and separate the semantic core from the outer grammatical structure:

[0078]

[0079] in, This represents word root extraction, an operation that extracts the core word root from the complete Mongolian word form, and outputs the unique core word root corresponding to that word form. This represents the probability of a word root appearing given a complete word form. This theoretical framework lays the foundation for subsequent boundary recognition and sequence labeling based on neural networks.

[0080] S202. The BiLSTM representation coding model is used to perform bidirectional context encoding on character-level sequences.

[0081] To accurately identify root and affix boundaries, character-level sequence context modeling is required. This invention employs a BiLSTM model to capture semantic dependencies bidirectionally. Let the normalized character sequence be:

[0082]

[0083] in, The input sequence is a normalized traditional Mongolian character-level sequence, serving as the input data for the BiLSTM model. It consists of individual Mongolian characters, processed by S1 text normalization, arranged in the order of their writing, with a length of T. For the T-th input character;

[0084] BiLSTM generates hidden vectors:

[0085]

[0086] in, The vector representation of the t-th input character. and These are forward semantic encoding and backward semantic encoding, respectively. For the forward long short-term memory network function, the normalized character-level input sequence X is processed from left to right / from front to back according to the traditional Mongolian writing order, and the semantic dependencies of the preceding text are encoded character by character; This is the inverse long short-term memory network function; the final expression is:

[0087]

[0088] in Let be the hidden state representation of the t-th input character, used to provide the semantic context required for root word determination. This model is able to understand the semantic coherence between affixes and roots, providing a semantic table for boundary prediction.

[0089] S203. Use the CRF model to perform globally optimal sequence labeling and output the set root and affix label sequences.

[0090] After obtaining the hidden state representation using the BiLSTM model, this invention uses a CRF model for globally optimal sequence prediction, identifying the start and end positions of word roots and affix boundaries. The CRF model decodes the output label sequence, avoiding illegally labeled sequences, thereby improving recognition accuracy.

[0091] The labeling system is shown in Table 2:

[0092] Table 2. CRF model label description.

[0093] BR Root begin position IR The middle position of the word root (root inside) BA Affix begin position IA Affix inside

[0094] The overall input to the BiLSTM-CRF model is a standardized traditional Mongolian text character sequence processed by S1 and its corresponding character features. The core input to the CRF model is the character-level bidirectional hidden state representation generated by BiLSTM encoding. and The model output is a root-affix boundary label sequence: BR, IR, ..., BA, IA, ..., BA, IA, ..., ...

[0095] The CRF sequence path scoring function is:

[0096]

[0097] in, Score the path. To output the label sequence, The transfer score between labels represents the score from the label. Go to Fixed scoring; Represents character-level input sequences The t-th input character Labeled The launch score, where T is the character-level input sequence. The length of the label path; the final output is the optimal label path. :

[0098]

[0099] in, This represents selecting Y that maximizes the path score.

[0100] CRF ensures the output of valid labeled sequences through transition constraints (e.g., avoiding IR from appearing at the beginning of the sequence and BR from appearing in the middle of the sequence), thereby significantly improving the accuracy of word root extraction. After the above steps, the system generates a structured representation for each word form:

[0101] [Original word form] → [Standardized word form] → [Root word] + [Affix sequence] + [Part of speech information], where the arrows indicate transformations.

[0102] S3. Root-enhanced vector representation generation: Root vectors and affix vectors are constructed based on the root sequence, and enhanced vector representations are obtained through root-guided semantic enhancement training and OOV (out-of-vocabulary) root-level sub-word encoding.

[0103] After completing the traditional Mongolian root extraction in S2, the text has been uniformly mapped from complex and varied word forms to a stable root structure representation. To aggregate different derived word forms of the same root in the semantic space and improve the semantic expressive power of retrieval and generative language models, this step constructs a multi-layered enhanced vector representation for the rooted text, including root vector construction, affix vector construction, semantic enhancement training, and a root-based OOV subword encoding mechanism. By modeling the semantic center of the roots and integrating the grammatical and semantic features of affixes, this invention achieves consistency and generalization ability in traditional Mongolian semantic representation, providing high-quality embedded representations for subsequent retrieval enhancement generation.

[0104] S301. A letter-level Transformer encoding model is used, and word root vectors are constructed through joint training of masked language modeling (MLM), autoregressive prediction, and semantic contrastive learning.

[0105] This step uses word roots as the core semantic encoding unit of the model, constructing highly expressive word root vectors through self-supervised learning. To fully capture the semantic dependencies and structural patterns in traditional Mongolian alphabet sequences, an alphabet-level Transformer encoding model is used for vector learning. Model training employs parallel optimization using three types of supervision signals: masked language modeling (MLM), autoregressive word root prediction task, and semantic contrastive learning, to obtain robust semantic center representations of word roots.

[0106] In the MLM task, several characters in a normalized traditional Mongolian character-level sequence X are randomly masked, and the masked characters are predicted based on the context. The training objective is:

[0107]

[0108] in The loss function for modeling masked languages, used to optimize the model's ability to predict masked characters, is one of the core training objectives for constructing word root vectors. Let be the vector representation of the t-th input character, that is, the character vector corresponding to the masked position. For the set of mask locations, This represents the entire sequence after removing the t-th input character. The model predicts the probability of characters at mask positions. By predicting missing characters, the model improves its semantic inference ability. The purpose of the mask is to learn letter-level / word-root-level contextual representations and improve the semantic sensitivity of the embedding.

[0109] To learn the word formation rules within word roots in traditional Mongolian, this invention designs an autoregressive task based on context prediction. By predicting the components of the next word root, the model is guided to learn word root combination patterns. Its objective function is:

[0110]

[0111] in The autoregressive word root prediction loss function is one of the core training objectives for constructing word root vectors. It is used to optimize the model's ability to learn the word formation rules within traditional Mongolian word roots. The first word root segment sequence item, For the first The term preceding the term; the purpose of this formula is to learn the combination patterns within word roots.

[0112] In addition, to ensure the aggregation of word vectors derived from the same root, semantic contrast loss is introduced. This minimizes the distance between cognate word vectors and maximizes the distance between disconate word vectors.

[0113]

[0114] in A positive sample pair consists of the i-th sample and the j-th sample. For the negative sample set, For negative samples, The similarity function measures the semantic closeness between vectors; temperature parameter To contrast with the scaling control factor in the loss function, this formula aims to bring word vectors with similar roots closer together and those with different roots further apart, creating a root-level cluster structure in the vector space. The ultimate training objective is multi-task joint optimization.

[0115]

[0116] in For word root loss function, , , These are hyperparameters, tuned using a validation set. This mechanism allows the model to naturally form a semantic clustering structure centered on word roots, achieving a unified expression at the word root level.

[0117] S302, Affix Vector Construction: Construct independent embedding tables for affixes of different grammatical categories and dynamically merge them with root vectors;

[0118] Traditional Mongolian suffixes not only have derivational word-forming functions but also carry rich grammatical attributes, such as case, tense, aspect, mood, and person. Therefore, modeling only the root words cannot fully express the grammatical ideas. This step constructs independent embedding tables for different categories of suffixes, achieving differentiated modeling through grammatical category division. At the same time, it interacts and merges with the root word vectors to maintain semantic unity while expressing structural changes.

[0119] Affix vectors and root vectors are jointly represented using a trainable fusion structure. The core idea is to control the grammatical contribution ratio of affixes through weights, enabling the model to dynamically adjust the influence of affixes based on context. Fusion Representation Defined as:

[0120]

[0121] in The word root vectors obtained through joint training with S301 are generated by encoding traditional Mongolian word root sequences using letter-level Transformer through MLM, autoregressive prediction, and semantic contrastive learning. Affix vectors are constructed independently for affixes of different grammatical categories, and are obtained by classifying and training them according to categories such as case affixes, plural markers, derived suffixes, and function / grammatical suffixes; These are learnable coefficients used to control the grammatical weight contribution of different affixes in different contexts.

[0122] This structure achieves a balance between semantic sharing and grammatical expression, enabling traditional Mongolian derived word forms to share semantic centers while retaining semantic differences, thus achieving refined task expression capabilities.

[0123] S303. Semantic enhancement training based on word roots: Comparative learning is performed using words derived from the same word root as positive samples and semantically irrelevant words as negative samples to optimize the vector space structure;

[0124] To further improve the consistency and discriminative nature of vector representations, this step employs a semantic enhancement training strategy centered on word roots. Words derived from the same root form positive sample pairs, while semantically unrelated words form negative sample pairs. Through contrastive learning, the model achieves supervised structural convergence in the vector space. The contrastive loss is responsible for bringing semantically similar vectors closer together and distancing semantically unrelated vectors apart, resulting in a highly structured representation of the word vector space.

[0125] Contrast learning loss function Defined as:

[0126]

[0127] in The fusion representation of the main sample represents the semantic vector of the current sample; This represents the fusion of positive samples. This represents the fusion of negative samples. To ensure the interpretability and stability of the training, this invention further introduces affix classification and root boundary prediction auxiliary tasks to form a joint learning framework:

[0128]

[0129] in, For the affix-root joint loss function, and It is an adjustable hyperparameter. The affix classification loss function serves as an auxiliary training loss to optimize the model's ability to identify and distinguish features of affixes of different functional categories, such as case affixes, plural markers, and derived suffixes. This strategy enables the model to simultaneously optimize root aggregation, affix differentiation, and semantic comparison objectives during training, thereby improving the structural consistency and cross-task transferability of the traditional Mongolian representation system.

[0130] S304. Perform root-based sub-word encoding on OOV words: First, perform root-affix splitting and parsing on OOV words, then perform BPE (byte-pair encoding) sub-word encoding on the remaining segments, and dynamically adjust the semantic contribution weight of sub-words.

[0131] Traditional Mongolian corpora are limited in size and have extremely high OOV (Out-of-Vocabulary) frequencies, especially academic vocabulary, loanwords, and derived neologisms, which have complex structures and pose a performance bottleneck to the model. To address this, this step employs a root-based sub-word encoding mechanism, decomposing OOV words into root and affix or BPE sub-word structures, and maintaining vector continuity through semantic fusion.

[0132] OOV word meaning Defined as:

[0133]

[0134] in, For sub-word vectors, A dynamic adjustment factor to control the contribution of sub-words to the overall semantics.

[0135] For new OOV words, a BiLSTM model is first used to encode the character / syllable features of the OOV word, and a CRF model learns traditional Mongolian morphological rules to jointly predict the boundary positions of the root and affixes. If there are still unregistered segments after root-affix splitting and parsing, BPE sub-word splitting is performed on the segment to generate a sub-word set. And through dynamic adjustment factors Controlling sub-word vectors The semantic contribution weight.

[0136] This mechanism enables the construction and expansion of new words within a root-centered semantic framework, effectively enhancing the knowledge coverage, adaptability, and scalability of traditional Mongolian language models.

[0137] S4. Root-based retrieval enhancement generation mechanism: Using the enhanced vector representation as the retrieval key, a root semantic neighborhood-oriented knowledge retrieval mechanism is constructed, and the retrieval results are deeply integrated with the generative language model. At the same time, through root-level generation control and consistency constraints, accurate and semantically consistent generated text is output.

[0138] After generating S3 root-enhanced vector representations, traditional Mongolian texts already possess a structured, aggregateable, and interpretable unified semantic representation. This step constructs a retrieval enhancement generation mechanism based on the root vector space, deeply integrating knowledge retrieval with a generative language model. This enables the generative language model to not only possess linguistic expression capabilities but also to provide high-precision answers based on external knowledge bases and semantic neighborhoods. Through root-based retrieval vector construction, knowledge retrieval, semantic enhancement fusion, and root context-based generation control, this step achieves significant performance improvements for traditional Mongolian knowledge question answering, text completion, summarization, and domain-specific generation tasks.

[0139] S401. Construction of semantic retrieval vector based on word roots: Integrating word root vectors, affix vectors and query context encoding into a retrieval vector;

[0140] Traditional Mongolian is characterized by rich word inflections and complex suffix connections. If surface word forms are directly used to construct retrieval vectors, semantically consistent content will be severely fragmented, leading to a decline in retrieval recall. Therefore, this step uses root vectors as the primary key of the retrieval space and integrates affixes and context vectors to construct a semantic retrieval representation:

[0141]

[0142] in, For query vector, To supplement the context vector, These are learnable weights. When the query q is not just a single word, but a phrase / sentence, roots and affixes alone are insufficient to express the complete semantics; therefore, a supplementary vector needs to be constructed from the context. :

[0143]

[0144] Among them, Encoder is a character-level encoding network adapted to traditional Mongolian script. This refers to non-root components in the query, context letter sequences, or the character sequence of the entire sentence. Semantic similarity is measured using the cosine distance in the vector space.

[0145]

[0146] in, It represents the Euclidean magnitude (norm 2) of a vector. This is a document vector. In this way, derivative words with the same root are automatically clustered in a neighborhood space, and semantically consistent content can achieve high recall, thus significantly improving retrieval accuracy and semantic relevance.

[0147] S402. Knowledge retrieval mechanism based on semantic neighborhood of word roots: filter candidate documents in the knowledge base based on word root consistency or similarity to obtain a set of highly relevant texts;

[0148] After obtaining the retrieval vector, the system retrieves the set of most relevant candidate information from the knowledge base based on semantic distance:

[0149]

[0150] in, To initially retrieve the candidate set, This is a set of document vectors, with Top-K representing the top K best candidate results. To avoid semantic drift and retrieval errors, this step incorporates a root word consistency check mechanism, filtering out pseudo-related information by determining whether candidate content shares the same root word or has a similar root word sequence.

[0151]

[0152] in, For filtering operations, As candidate documents, Represents the set of word roots for candidate documents. The set of word roots for the query; The logical symbol represents OR; This is the filtering threshold. This mechanism ensures semantic aggregation in retrieval, effectively supporting high-precision retrieval scenarios such as professional terminology, domain knowledge, and ancient texts.

[0153] S403. Semantic Fusion of Retrieval Results and Generative Language Model: Dynamically inject the semantic representation of retrieval into the latent state of the generative language model through an attention mechanism;

[0154] The search results are not directly used as model output, but rather injected as semantic enhancement input into a generative language model adapted to the characteristics of traditional Mongolian. This step employs an attention fusion mechanism to dynamically weight the hidden state within the generative language model with the retrieval semantic representation:

[0155]

[0156] in Let H be the updated hidden state matrix of the generative language model after fusing retrieval semantics. H is the original internal hidden state matrix obtained by encoding the generated text sequence during the generation process, containing contextual semantic information prior to the current generation step. The semantic fusion strength coefficient. This represents the attention mechanism. This is used for the initial retrieval of the candidate set. The final generated probability distribution is defined as follows:

[0157]

[0158] in, The conditional probability for generating the t-th Mongolian character unit; For the Mongolian script units to be generated, The generated sequence; For normalization function, It is a learnable linear transformation matrix.

[0159] By integrating retrieved information, the model can significantly improve factual accuracy, professionalism, and semantic consistency during the generation process.

[0160] S404. Root-based generation control and consistency constraints: Root consistency constraints are added to the generation loss to ensure that the output text and the query roots remain semantically coherent.

[0161] To maintain root consistency and semantic coherence in the generated content, this step introduces a root-based generation control mechanism by incorporating root consistency constraints. To prioritize the generation of derivative structures with the same root word:

[0162]

[0163] The ultimate training objective is:

[0164]

[0165] in The standard generated loss for the model, This is a hyperparameter used to adjust the strength of the root consistency constraint, in order to achieve a balance between the degree of freedom of generation and the consistency constraint.

[0166] This mechanism ensures that the generated content is consistent in semantics, structure, and style, and is particularly suitable for the continuation of traditional Mongolian texts, terminology explanation, and the generation of professional knowledge.

[0167] It is understood that the present invention has been described through some embodiments, and those skilled in the art will recognize that various changes or equivalent substitutions can be made to these features and embodiments without departing from the spirit and scope of the invention. Furthermore, under the teachings of the present invention, these features and embodiments can be modified to adapt to specific situations and materials without departing from the spirit and scope of the invention. Therefore, the present invention is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the protection scope of the present invention.

Claims

1. A retrieval enhancement generation method based on traditional Mongolian word roots, characterized in that, Includes the following steps: S1. Standardization of Traditional Mongolian Text: The input traditional Mongolian text is processed sequentially with Unicode normalization, free glyph selector standardization, glyph normalization, continuous writing structure parsing, and special symbol cleaning to obtain standardized traditional Mongolian text. S2. Traditional Mongolian word root extraction: The standardized traditional Mongolian text is analyzed by word root-affix splitting using a bidirectional long short-term memory network-conditional random field (BiLSTM-CRF) model to extract stable word root sequences. S3. Root-enhanced vector representation generation: Root vectors and affix vectors are constructed based on the root sequence, and enhanced vector representations are obtained through root-guided semantic enhancement training and root-level sub-word encoding of out-of-vocabulary words; S4. Root-based retrieval enhancement generation mechanism: Using the enhanced vector representation as the retrieval key, a root semantic neighborhood-oriented knowledge retrieval mechanism is constructed, and the retrieval results are integrated with the generative language model. At the same time, through root-level generation control and consistency constraints, semantically consistent generated text is output.

2. The retrieval enhancement generation method based on traditional Mongolian word roots according to claim 1, characterized in that, The specific steps for standardizing traditional Mongolian texts are as follows: Unicode standardization: unifying character combination methods; Standardization of free glyph selectors: Construct a standard variant mapping table to uniformly map non-standard free glyph selector combinations to standard-defined glyph variants; Glyph normalization: Mapping the position-dependent glyphs of the first, middle, and last glyphs to a unified basic letter code; Connected writing structure analysis: Deconstructing the connected writing structure, breaking down the connected characters into standard character sequences; Special symbol cleaning: Removes illegal control characters, scans optical characters to identify incorrect symbols, non-Mongolian encoded characters mixed in, and duplicate or semantically meaningless variant marks.

3. The retrieval enhancement generation method based on traditional Mongolian word roots according to claim 2, characterized in that, The specific details of traditional Mongolian word root extraction are as follows: S201. Deconstruct and analyze the root-affix structure of traditional Mongolian words, and construct a traditional Mongolian affix library that includes case affixes, plural markers, derived suffixes, and functional suffixes; S202. A bidirectional long short-term memory network (BiLSTM) expression coding model is used to perform bidirectional context encoding on the character-level sequence to obtain the character-level bidirectional hidden state; the character-level sequence is a traditional Mongolian character-level input sequence composed of normalized Mongolian individual characters in the order of writing; S203. The bidirectional hidden state is input into the Conditional Random Field (CRF) sequence labeling boundary prediction model to perform global optimal sequence labeling, identify the start and end positions of word roots and affix boundaries, and output the set word root-affix boundary labeling sequence.

4. The retrieval enhancement generation method based on traditional Mongolian word roots according to claim 3, characterized in that, The enhanced vector representation is obtained through the following process: S301. The letter-level Transformer coding model is used to learn the input character vectors, and the letter-level Transformer coding model is jointly trained by masked language modeling, autoregressive prediction and semantic contrastive learning to construct word root vectors. S302. Construct independent embedding tables for affixes of different grammatical categories and dynamically fuse them with root vectors to obtain a fused representation of root vectors and affix vectors; S303. Comparative learning is performed using words derived from the same root as positive samples and semantically irrelevant words as negative samples to optimize the vector space structure and obtain an enhanced root vector representation. S304. Perform root-based sub-word encoding on out-of-vocabulary words: First, perform root-affix splitting and parsing on out-of-vocabulary words, then perform sub-word encoding on the remaining segments, and dynamically adjust the semantic contribution weight of sub-words.

5. The retrieval enhancement generation method based on traditional Mongolian word roots according to claim 4, characterized in that, S4 is specifically as follows: S401. Merge the root vector, affix vector and query context encoding into a query vector; S402. Filter candidate documents in the knowledge base based on the consistency or similarity of the root words between the query vector and the document to obtain a set of highly relevant texts; S403. Dynamically inject highly relevant text into the hidden state of the generative language model through an attention mechanism, and calculate the probability distribution based on the updated hidden state of the generative language model. S404. Add root word consistency constraints to the generation loss to ensure that the output text and the query root words remain semantically coherent.

6. The retrieval enhancement generation method based on traditional Mongolian word roots according to claim 5, characterized in that, The labeling system in the root-affix boundary annotation sequence includes the root start position BR, the root middle position IR, the affix start position BA, and the affix middle position IA.

7. The retrieval enhancement generation method based on traditional Mongolian word roots according to claim 6, characterized in that, The CRF sequence path score function for the root-affix boundary annotation sequence is: in, Score the path. The output is a sequence of labels, and T is a character-level input sequence. Length, The transfer score between labels represents the score from the label. Go to Fixed scoring; Represents character-level input sequences The t-th input character Labeled The launch score is calculated; the optimal tag path is then output. : in, This represents taking Y that maximizes the path score; after the above steps, the system generates a structured representation for each word form: [root word] + [affix sequence] + [part of speech information].

8. The retrieval enhancement generation method based on traditional Mongolian word roots according to claim 7, characterized in that, The formula for fusing the query vectors is as follows: in, For query vector, To supplement the context vector, For learnable weights, Affix vectors are constructed independently for affixes of different grammatical categories. The root vectors and context supplement vectors are obtained by training with S301. Encode using the following method: Among them, Encoder is a character-level encoding network adapted to traditional Mongolian script. It refers to non-root components, contextual letter sequences, or character sequences of the entire sentence in the query.

9. The retrieval enhancement generation method based on traditional Mongolian word roots according to claim 8, characterized in that, The codes for the out-of-vocabulary words are as follows: in, This indicates an unregistered word. For sub-word vectors, A dynamic adjustment factor to control the contribution of sub-words to the overall semantics; New out-of-vocabulary (OV) words are first encoded using a BiLSTM representation-encoding model to represent their character / syllable features. A CRF model then learns traditional Mongolian morphological rules and jointly predicts the boundary positions of roots and affixes. If OV segments still exist after root-affix splitting and parsing, BPE sub-word splitting is performed on those segments to generate a sub-word set. And through dynamic adjustment factors Controlling sub-word vectors The semantic contribution weight.

Images