Large language model training method and device for low-resource language, equipment and medium

Abstract

The invention provides a low-resource language-oriented large language model training method and device, equipment and a medium, and relates to the technical field of artificial intelligence. The method comprises the following steps: acquiring low-resource language data and preprocessing the low-resource language data to obtain low-resource language unsupervised training data; generating a low-resource language vocabulary and performing vocabulary expansion on the existing model; performing unsupervised pre-training on the model after word list expansion according to the low-resource language unsupervised training data in combination with a first low-rank adaptive module; performing instruction fine tuning on the pre-trained model according to the low-resource language instruction fine tuning data set in combination with a second low-rank adaptive module; and respectively inputting a high-resource language supervised corpus into the model subjected to instruction fine tuning and the existing mature model to respectively obtain negative example data and positive example data so as to generate a reinforcement learning low-resource language data set based on model feedback, and carrying out reinforcement learning training on the model subjected to instruction fine tuning in combination with a third low-rank adaptive module.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and more specifically, to a large language model training method, device, equipment and medium for low-resource languages. Background Art

[0002] In recent years, with the rapid development of artificial intelligence (AI) technology, natural language processing (NLP) based on deep learning has been widely used in many application scenarios. As one of the core technologies in the current NLP field, the Large Language Model (LLM) has demonstrated its powerful ability to process natural language in a variety of tasks, such as machine translation, text generation, information retrieval, question-answering systems, and sentiment analysis. Through training on a large-scale corpus, the Large Language Model can understand and generate complex patterns of natural language and has achieved remarkable results in many practical applications.

[0003] In the process of realizing the concepts disclosed herein, the inventors discovered that there are at least the following defects in the related art: the training of these large language models is highly dependent on abundant high-resource language data (such as English, Chinese, French, etc.), while for low-resource languages ​​(such as minority languages ​​or endangered languages ​​in certain regions), the practical application of the models is greatly limited due to the lack of sufficient training data. Summary of the invention

[0004] In view of the above problems, the present disclosure provides a large language model training method, device, equipment and medium for low-resource languages, so as to solve the technical problem that the existing large language model has insufficient support for low-resource languages.

[0005] One aspect of the present disclosure provides a large language model training method for low-resource languages, including: obtaining low-resource language data, and preprocessing the low-resource language data to obtain unsupervised training data for the low-resource language; generating a low-resource language vocabulary based on the low-resource language unsupervised training data, and using the low-resource language vocabulary to expand the vocabulary of an existing first large language model, so that the first large language model after the vocabulary expansion can support low-resource languages; based on the low-resource language unsupervised training data, unsupervised pre-training is performed on the first large language model after the vocabulary expansion in combination with a first low-rank adaptive module, so that the pre-trained first large language model can adapt to the language pattern and grammatical characteristics of the low-resource language; obtaining a supervised corpus of a high-resource language, and generating a supervised corpus of a low-resource language based on the supervised corpus of the high-resource language ; Perform instruction fine-tuning conversion processing on the supervised corpus of the low-resource language to generate a low-resource language instruction fine-tuning dataset; Based on the low-resource language instruction fine-tuning dataset, combine the second low-rank adaptive module to perform instruction fine-tuning on the pre-trained first large language model, so that the first large language model after instruction fine-tuning can execute different task instructions in a variety of low-resource language task environments; Input the supervised corpus of the high-resource language into the first large language model after instruction fine-tuning and the existing second large language model respectively, and obtain negative example data and positive example data respectively, and generate a reinforcement learning low-resource language dataset based on model feedback according to the negative example data and the positive example data; Based on the reinforcement learning low-resource language dataset, combine the third low-rank adaptive module to perform reinforcement learning training on the first large language model after instruction fine-tuning to obtain the target large language model.

[0006] According to an embodiment of the present disclosure, preprocessing low-resource language data to obtain low-resource language unsupervised training data includes: performing noise removal processing on the low-resource language data; performing format standardization processing on the low-resource language data after the noise removal processing; performing multi-language data filtering processing on the low-resource language data after the format standardization processing; performing privacy information removal processing on the low-resource language data after the multi-language data filtering processing; and generating low-resource language unsupervised training data based on the low-resource language data after the privacy information removal processing.

[0007] According to an embodiment of the present disclosure, using a low-resource language vocabulary to expand the vocabulary of the existing first largest language model includes: merging the low-resource language vocabulary and the vocabulary of the existing first largest language model to obtain a merged vocabulary; based on the merged vocabulary, expanding the head layer of the existing first largest language model to adapt it to the expanded vocabulary to achieve support for low-resource languages, wherein the head layer includes an embedding layer and an output layer.

[0008] According to an embodiment of the present disclosure, unsupervised pre-training of the first largest language model after vocabulary expansion is performed in combination with the first low-rank adaptive module, including: adding a first side branch to the first largest language model after vocabulary expansion, wherein the first side branch is configured to simplify parameter adjustment of the model by introducing a first low-rank matrix; inputting low-resource language unsupervised training data into the first largest language model after vocabulary expansion for unsupervised pre-training, wherein, during the pre-training process, the model initialization parameters are fixed and unchanged, and only the parameters of the first side branch are trained.

[0009] According to an embodiment of the present disclosure, obtaining a supervised corpus in a high-resource language and generating a supervised corpus in a low-resource language based on the supervised corpus in the high-resource language includes: obtaining a supervised corpus in a high-resource language and translating the supervised corpus in the high-resource language into a low-resource language using machine translation technology; mining and annotating pseudo-annotated data of a low-resource language from the supervised corpus in the high-resource language using a parallel sentence mining method and a label transfer method based on self-supervised learning; performing confidence scoring on the pseudo-annotated data using a corpus quality assessment algorithm; and obtaining a supervised corpus in a low-resource language by screening and removing pseudo-annotated data that is below a confidence score threshold.

[0010] According to an embodiment of the present disclosure, fine-tuning the instructions of the pre-trained first large language model based on a low-resource language instruction fine-tuning dataset in combination with a second low-rank adaptive module includes: adding a second side branch to the pre-trained first large language model, wherein the second side branch is configured to simplify the parameter adjustment of the model by introducing a second low-rank matrix; inputting the low-resource language instruction fine-tuning dataset into the pre-trained first large language model for instruction fine-tuning, wherein, during the instruction fine-tuning process, the fixed model initialization parameters remain unchanged, and only the parameters of the second side branch are trained.

[0011] According to an embodiment of the present disclosure, reinforcement learning training is performed on the first large language model after fine-tuning the instructions according to the reinforcement learning low-resource language dataset to obtain a target large language model, including: according to the reinforcement learning low-resource language dataset, reinforcement learning training is performed on the first large language model after fine-tuning the instructions using a direct preference optimization algorithm combined with a third low-rank adaptive module to obtain a target large language model.

[0012] Another aspect of the present disclosure provides a large language model training device for low-resource languages, including: a first acquisition module, used to acquire low-resource language data, and pre-process the low-resource language data to obtain unsupervised training data for low-resource languages; a vocabulary expansion module, used to generate a low-resource language vocabulary based on the low-resource language unsupervised training data, and use the low-resource language vocabulary to expand the vocabulary of an existing first large language model, so that the first large language model after the vocabulary expansion can support low-resource languages; an unsupervised pre-training module, used to perform unsupervised pre-training on the first large language model after the vocabulary expansion based on the low-resource language unsupervised training data in combination with a first low-rank adaptive module, so that the pre-trained first large language model can adapt to the language pattern and grammatical characteristics of low-resource languages; a second acquisition module, used to acquire high-resource language supervised corpus, and generate low-resource language supervised corpus based on the high-resource language supervised corpus a conversion module for performing instruction fine-tuning conversion processing on the supervised corpus of the low-resource language to generate a low-resource language instruction fine-tuning dataset; an instruction fine-tuning module for performing instruction fine-tuning on the pre-trained first large language model based on the low-resource language instruction fine-tuning dataset in combination with the second low-rank adaptive module, so that the first large language model after instruction fine-tuning can execute different task instructions in a variety of low-resource language task environments; a model feedback data generation module for inputting the supervised corpus of the high-resource language into the first large language model after instruction fine-tuning and the existing second large language model respectively, to obtain negative example data and positive example data respectively, and to generate a reinforcement learning low-resource language dataset based on model feedback according to the negative example data and the positive example data; a reinforcement learning module for performing reinforcement learning training on the first large language model after instruction fine-tuning based on the reinforcement learning low-resource language dataset in combination with the third low-rank adaptive module to obtain a target large language model.

[0013] Another aspect of the present disclosure provides an electronic device, including: one or more processors; a memory for storing one or more programs; wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the above method.

[0014] Another aspect of the present disclosure provides a computer-readable storage medium storing computer-executable instructions, which are used to implement the above method when executed.

[0015] Another aspect of the present disclosure provides a computer program product, which includes computer executable instructions, and the instructions are used to implement the above method when executed.

[0016] Compared with the prior art, the large language model training method, device, equipment and medium for low-resource languages ​​provided by the present disclosure have at least the following beneficial effects:

[0017] (1) The large language model training method, device, equipment and medium for low-resource languages ​​provided in the present disclosure, by constructing a multi-task learning framework suitable for low-resource languages, and combining low-rank adaptation (LoRA) technology, continues to pre-train and fine-tune the existing large language model, thereby achieving the goal of improving the processing capabilities of low-resource languages ​​under limited hardware resources and data conditions. This method effectively utilizes the data and model experience of high-resource languages, and enhances the performance of the model on low-resource languages ​​through cross-language transfer learning strategies, especially in language understanding and text generation tasks. Significant improvements have been achieved.

[0018] (2) The large language model training method, device, equipment and medium for low-resource languages ​​provided by the present disclosure expand the training data set of low-resource languages ​​by integrating low-resource data from multiple sources, solve the problem of data scarcity, and further optimize the performance of the model in a low-resource language environment for the modeling technology of low-resource languages, so that the model can more flexibly adapt to different language characteristics and structures, and enhance the operability and accuracy in practical applications. Through this method, it is possible to achieve efficient training of large language models in low-resource language environments, and promote the popularization and application of natural language processing technology in more languages, which has important practical significance and broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The above and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0020] Figure 1 A flowchart of a large language model training method for low-resource languages ​​according to an embodiment of the present disclosure is schematically shown;

[0021] Figure 2 A schematic diagram showing the construction principle of a word segmenter in a large language model training method for low-resource languages ​​according to an embodiment of the present disclosure is shown;

[0022] Figure 3 A schematic diagram of a principle diagram of expanding the vocabulary of a first large language model in a large language model training method for low-resource languages ​​according to an embodiment of the present disclosure is shown;

[0023] Figure 4 Schematically shows a schematic diagram of unsupervised pre-training of causal language modeling of the first large language model after vocabulary expansion according to an embodiment of the present disclosure;

[0024] Figure 5 A schematic diagram of fine-tuning instructions on a pre-trained first language model according to an embodiment of the present disclosure is shown;

[0025] Figure 6 A structural block diagram of a large language model training device for low-resource languages ​​according to an embodiment of the present disclosure is schematically shown;

[0026] Figure 7 The structural block diagram of an electronic device suitable for implementing a large language model training method for low-resource languages ​​according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0027] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0028] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0029] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0030] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0031] In the embodiments of the present disclosure, the collection, updating, analysis, processing, use, transmission, provision, disclosure, storage, etc. of the data involved (for example, including but not limited to user personal information) are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. In particular, necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information, network security, and national security.

[0032] In recent years, with the rapid development of artificial intelligence (AI) technology, natural language processing (NLP) based on deep learning has been widely used in many application scenarios. As one of the core technologies in the current NLP field, the Large Language Model (LLM) has demonstrated its powerful ability to process natural language in a variety of tasks, such as machine translation, text generation, information retrieval, question-answering systems, and sentiment analysis. Through training on a large-scale corpus, the Large Language Model can understand and generate complex patterns of natural language and has achieved remarkable results in many practical applications.

[0033] In the process of realizing the concepts disclosed herein, the inventors discovered that there are at least the following defects in the related art: the training of these large language models is highly dependent on abundant high-resource language data (such as English, Chinese, French, etc.), while for low-resource languages ​​(such as minority languages ​​or endangered languages ​​in certain regions), the practical application of the models is greatly limited due to the lack of sufficient training data.

[0034] Low-Resource Languages ​​generally refer to languages ​​with relatively few speakers and scarce digital resources, such as minority languages ​​or endangered languages ​​in certain regions. Due to the scarcity of data resources for these languages, the lack of large-scale parallel corpora and pre-annotated data, existing large language models perform poorly on low-resource languages. At the same time, low-resource languages ​​often have unique language structures and complex grammatical rules, making these languages ​​particularly difficult to perform in natural language processing tasks. In addition, some low-resource languages ​​lack standardized writing systems and annotation specifications, which increases the difficulty of data processing.

[0035] For the research on natural language processing of low-resource languages, existing methods mainly focus on technical means such as multilingual modeling, cross-language transfer, and data enhancement. For example, many researchers try to improve the language processing ability of low-resource languages ​​by sharing parameters or weights between models of low-resource languages ​​and high-resource languages, and taking advantage of the rich corpus and model training experience of high-resource languages. Although this method can alleviate the problem of insufficient data in low-resource languages ​​to a certain extent, due to the large differences between languages, the model still faces the problems of insufficient semantic understanding and low generation accuracy when facing low-resource languages. In addition, cross-language transfer learning usually transfers between high-resource languages ​​and low-resource languages, but for low-resource languages ​​with large differences in grammatical structures, this method has limited effect. In order to meet these challenges, some researchers have proposed a large language model training method based on pre-training and instruction fine-tuning strategies. The pre-trained model captures the common features of multiple languages ​​by unsupervised training on a large amount of multilingual corpus, and then fine-tunes on specific low-resource language tasks so that the model can adapt to specific language features. However, such methods still face several key problems: First, it is difficult to obtain corpora for low-resource languages. The proportion of low-resource languages ​​in existing multilingual parallel corpora is small, which cannot meet the training needs of large language models. Second, even if low-resource language data can be obtained, these data are usually of uneven quality, contain a lot of noise and incomplete information, and are difficult to use directly for high-quality model training. In addition, how to effectively expand the vocabulary of low-resource languages ​​in the model and ensure that the expanded model can support the processing of low-resource languages ​​without affecting the performance of the original high-resource languages ​​is also a major technical challenge.

[0036] In order to solve the above defects, some new technical directions have gradually emerged in recent years, such as Low-Rank Adaptation (LoRA) technology. It is an innovative method for lightweight fine-tuning of large models. It can adjust the parameters of large language models without increasing too much computational overhead, so that they can adapt to new languages ​​or tasks, especially when resources are limited. At the same time, through self-supervised learning (Self-Supervised Learning) and multi-task learning (Multi-Task Learning) technology, the model's understanding ability and generation quality of low-resource languages ​​can be effectively improved. The introduction of these technologies provides new ideas and directions for building large language models for low-resource languages. However, despite the progress made in these methods, the training of large language models for low-resource languages ​​still faces many challenges. For example, how to systematically collect and process unsupervised and supervised data of low-resource languages, how to efficiently clean and normalize data, how to reasonably expand and adapt the vocabulary of low-resource languages ​​in the model, and how to design effective training algorithms to improve the performance of models on low-resource languages ​​are all urgent problems to be solved in the current field of natural language processing. Therefore, there is an urgent need for a large language model training method for low-resource languages. By combining a variety of innovative technologies, we can effectively utilize existing low-resource language data resources, and through reasonable model architecture design and optimization strategies, we can improve the model's understanding and generation capabilities for low-resource languages, providing strong support for low-resource language processing applications around the world.

[0037] Based on this, the embodiment of the present disclosure provides a large language model training method for low-resource languages ​​to solve the technical problem that the existing large language model does not support low-resource languages. The method includes:

[0038] Obtain low-resource language data, and preprocess the low-resource language data to obtain unsupervised training data for the low-resource language;

[0039] Generate a low-resource language vocabulary according to the low-resource language unsupervised training data, and use the low-resource language vocabulary to expand the vocabulary of the existing first language model, so that the first language model after the vocabulary expansion can support the low-resource language;

[0040] According to the unsupervised training data of the low-resource language, unsupervised pre-training is performed on the first language model after the vocabulary expansion in combination with the first low-rank adaptive module, so that the pre-trained first language model can adapt to the language pattern and grammatical characteristics of the low-resource language;

[0041] Obtain supervised corpora in high-resource languages, and generate supervised corpora in low-resource languages ​​based on the supervised corpora in high-resource languages;

[0042] Perform instruction fine-tuning conversion processing on supervised corpus of low-resource languages ​​to generate a low-resource language instruction fine-tuning dataset;

[0043] According to the low-resource language instruction fine-tuning dataset, the pre-trained first language model is fine-tuned in combination with the second low-rank adaptive module, so that the first language model after instruction fine-tuning can execute different task instructions in a variety of low-resource language task environments;

[0044] The supervised corpus of high-resource language is input into the first language model after instruction fine-tuning and the existing second language model respectively, and negative data and positive data are obtained respectively. Based on the negative data and positive data, a reinforcement learning low-resource language dataset based on model feedback is generated;

[0045] According to the reinforcement learning low-resource language dataset, the first large language model after instruction fine-tuning is trained by reinforcement learning in combination with the third low-rank adaptive module to obtain the target large language model.

[0046] The large language model training method based on low-resource language provided in the embodiment of the present disclosure, by constructing a multi-task learning framework suitable for low-resource language, and combining the Low-Rank Adaptation (LoRA) technology, continues to pre-train and fine-tune the instructions of the existing large language model, and achieves the goal of improving the processing ability of low-resource language under limited hardware resources and data conditions. This method effectively utilizes the data and model experience of high-resource language, and enhances the performance of the model on low-resource language through cross-language transfer learning strategy, especially in language understanding and text generation tasks. Significant improvement has been achieved.

[0047] In order to make the objectives, technical solutions and advantages of the present disclosure more clearly understood, the present disclosure is further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings.

[0048] Figure 1 The flowchart of the large language model training method for low-resource languages ​​according to an embodiment of the present disclosure is schematically shown.

[0049] like Figure 1 As shown, the large language model training method for low-resource languages ​​in this embodiment may include operations S1 to S8, for example.

[0050] In operation S1, low-resource language data is acquired and preprocessed to obtain low-resource language unsupervised training data.

[0051] In this embodiment, we first use the existing multilingual translation platform and parallel corpus to obtain the control corpus of low-resource languages, and organize the low-resource language data from various public sources. On this basis, we convert the public low-resource language documents that do not have ready-made digital versions into usable text resources to obtain low-resource language data resources.

[0052] Then, the acquired low-resource language data is preprocessed to obtain low-resource language unsupervised training data. Specifically:

[0053] According to an embodiment of the present disclosure, preprocessing low-resource language data to obtain low-resource language unsupervised training data may include, for example:

[0054] Noise removal for low-resource language data;

[0055] Standardize the format of low-resource language data after noise removal;

[0056] Perform multilingual data filtering on low-resource language data after format standardization;

[0057] Remove privacy information from low-resource language data after multilingual data filtering;

[0058] Generate low-resource language unsupervised training data based on the low-resource language data after privacy information removal.

[0059] In this embodiment, the acquired low-resource language data is preprocessed, mainly using a heuristic data cleaning method, and a set of detailed rules are formulated for multilingual scenarios, mainly including noise removal, format standardization (such as unified encoding format, punctuation specifications), multilingual data filtering (to avoid mixing of erroneous languages) and privacy information removal.

[0060] Among them, data cleaning may include, for example, language filtering, statistical feature filtering, keyword filtering, etc., to retain high-quality text as much as possible.

[0061] For example, privacy information removal can use a rule-based algorithm to detect personal information such as name, address, email address and phone number and delete or replace them.

[0062] In addition, the preprocessing process can also include advertising filtering and data deduplication. If the data contains a large amount of advertising information, it will seriously affect the generation effect of the model. Therefore, the advertising content in the corpus can be filtered by keyword filtering. Repeated redundant data will reduce the diversity of the language model and cause the training process to be unstable, affecting the performance of the model. Therefore, the minimum hash algorithm can be used to complete the corpus deduplication in a shorter time.

[0063] Through these preprocessing steps, high-quality, privacy-free low-resource language data is retained to build a dataset for unsupervised model training. Since the data sources of low-resource languages ​​usually contain inconsistent, messy or incomplete content, preprocessing can significantly improve the purity of the data and the training effect.

[0064] In operation S2, a low-resource language vocabulary is generated according to the low-resource language unsupervised training data, and the low-resource language vocabulary is used to expand the vocabulary of the existing first language model, so that the first language model after the vocabulary expansion can support the low-resource language.

[0065] In this embodiment, a low-resource language vocabulary is generated based on the preprocessed corpus.

[0066] A vocabulary is a list of all the words in a particular language and is often used to map words in a text to numerical representations (such as word embeddings) that a model can understand.

[0067] In another embodiment, based on the vocabulary, a word segmenter can also be generated according to the preprocessed corpus.

[0068] The tokenizer is an important component in natural language processing. It is mainly used to break down text into smaller units (called "words" or "tokens"), which can be words, characters, or subwords, depending on the word segmentation strategy. In this embodiment, the construction principle of the tokenizer is shown in Figure 2 .

[0069] Figure 2 The schematic diagram shows a construction principle diagram of a word segmenter in a large language model training method for low-resource languages ​​according to an embodiment of the present disclosure.

[0070] like Figure 2 As shown, unsupervised data for low-resource languages ​​is processed into a word segmenter via a byte pair encoding algorithm.

[0071] Byte-Pair Encoding (BPE) is an algorithm for breaking words into subword units, which is particularly suitable for processing low-resource languages ​​and reducing vocabulary sparsity. The tokenizer includes a lemma method for low-resource languages, which helps the model effectively break down text into subwords or lemmas when processing such languages.

[0072] After generating the low-resource language vocabulary, the low-resource language vocabulary is used to expand the vocabulary of the existing first language model, so that the first language model after the vocabulary expansion can support the low-resource language. Specifically:

[0073] According to an embodiment of the present disclosure, using a low-resource language vocabulary to expand the vocabulary of the existing first language model may include, for example:

[0074] Merging the low-resource language vocabulary with the vocabulary of the existing largest language model to obtain a merged vocabulary;

[0075] According to the merged vocabulary, the head layer of the existing first language model is expanded to adapt it to the expanded vocabulary and support low-resource languages, wherein the head layer includes an embedding layer and an output layer.

[0076] In this embodiment, after generating the low-resource language vocabulary, the principle of using the low-resource language vocabulary to expand the vocabulary of the existing first language model is shown in Figure 3 .

[0077] Figure 3 The schematic diagram shows a principle diagram of expanding the vocabulary of the first large language model in the large language model training method for low-resource languages ​​according to an embodiment of the present disclosure.

[0078] like Figure 3 As shown, the vocabulary expansion is achieved by merging the vocabulary of the low-resource language with the existing vocabulary, and expanding the head layer (especially the embedding layer and the output layer) of the first language model according to the size of the merged vocabulary, ensuring that the model can handle the expanded vocabulary content, thereby achieving support for low-resource languages.

[0079] In operation S3, unsupervised pre-training is performed on the first language model after the vocabulary expansion according to the unsupervised training data of the low-resource language in combination with the first low-rank adaptation (LoRA) module, so that the pre-trained first language model can adapt to the language pattern and grammatical characteristics of the low-resource language.

[0080] In this embodiment, based on the design of the LoRA module and the unsupervised continued pre-training strategy adopted, the first language model can quickly adapt to the special patterns and grammatical characteristics of low-resource languages ​​while retaining the original multi-language capabilities. Specifically:

[0081] According to an embodiment of the present disclosure, unsupervised pre-training of the first language model after vocabulary expansion in combination with the first low-rank adaptive module may include, for example:

[0082] Adding a first side branch to the first language model after the vocabulary expansion, wherein the first side branch is configured to simplify parameter adjustment of the model by introducing a first low-rank matrix;

[0083] The low-resource language unsupervised training data is input into the first language model after the vocabulary is expanded for unsupervised pre-training. During the pre-training process, the model initialization parameters are fixed and unchanged, and only the parameters of the first side branch are trained.

[0084] In this embodiment, LoRA simplifies the parameter adjustment of the model by introducing a low-rank matrix, reduces the training cost and computational burden, and thus can achieve significant model performance improvement even with low data volume.

[0085] The principle of LoRA is relatively simple. It adds a bypass next to the first language model and performs dimensionality reduction and dimensionality increase operations. Assuming that a large language model needs to be fine-tuned in downstream tasks, the pre-trained model parameters need to be updated. The formula is as follows:

[0086] W o +ΔW

[0087] Among them, W o represents the parameters initialized for training the first language model, and ΔW represents the parameters that need to be updated.

[0088] If full parameter fine-tuning is used, the number of training parameters is W o For example, the number of parameters of GPT-3 is as high as 175B. The cost of fine-tuning all parameters of a large language model is very high, while LORA only needs to fine-tune ΔW. Specifically, assuming that the pre-trained weight parameter matrix is , the parameter update method of LoRA is expressed as follows:

[0089]

[0090] Among them, the rank ,During the training process of LoRA, W o Fixed, only the training matrix is ​​needed and Parameters.

[0091] In the forward process W o and ∆W will multiply the same input , and finally add them together, the formula is as follows:

[0092]

[0093] Assume that the loss function of the model is , then the parameters in the training process and The gradient calculation of is as follows:

[0094]

[0095] Therefore, the LoRA technique reduces the number of parameters during fine-tuning through low-rank factorization without sacrificing the performance of the model. The specific formula for unsupervised pre-training is as follows:

[0096]

[0097] Among them, represents the model parameters, represents the currently predicted token, represents the conditional context.

[0098] Figure 4 Schematically shows the schematic diagram of unsupervised pre-training for causal language modeling of the first large language model after vocabulary expansion according to an embodiment of the present disclosure.

[0099] As Figure 4 shown, it shows the unsupervised pre-training process of causal language modeling of the first large language model after vocabulary expansion. The model gradually predicts the next token of the input through time steps and gradually generates a complete sentence. Each time step in the figure shows the process of the model predicting the current token given the previous context. Specifically:

[0100] Time step 1:

[0101] The input token is "is".

[0102] Based on the expanded vocabulary, the model predicts this token. At this time, the model has read and processed "is", but the next step still needs to predict the next word.

[0103] The initial part of the predicted sentence is "This is...".

[0104] Time step 2:

[0105] The input token is "a".

[0106] The model then reads and processes the next token "a" and continues to search for a matching word in the expanded vocabulary. At this time, the model can already speculate based on the previous word "is" and has successfully read "a".

[0107] Part of the information of the sentence at this time is "This is a...".

[0108] Time step 3:

[0109] The input token is "one".

[0110] The model processes the next token "one" and continues to infer based on the context. At this time, the model has successfully predicted a more complete part of the sentence.

[0111] Part of the information of the sentence is "This is a...".

[0112] Time step 4:

[0113] The input token is "table".

[0114] The model predicts "table" at this step, which is the last token of the sentence. At this time, the model infers through the previous context and combines the extended vocabulary to draw a reasonable conclusion.

[0115] The completed sentence is "This is a table".

[0116] In operation S4, obtain the supervised corpus of the high-resource language and generate the supervised corpus of the low-resource language according to the supervised corpus of the high-resource language.

[0117] According to the embodiments of the present disclosure, obtaining the supervised corpus of the high-resource language and generating the supervised corpus of the low-resource language according to the supervised corpus of the high-resource language may include, for example:

[0118] Obtain the supervised corpus of the high-resource language and translate the supervised corpus of the high-resource language into the low-resource language by using machine translation technology;

[0119] Use the parallel sentence mining method and label transfer method based on self-supervised learning to mine and label the pseudo-labeled data of the low-resource language from the supervised corpus of the high-resource language;

[0120] Use the corpus quality evaluation algorithm to perform confidence scoring on the pseudo-labeled data;

[0121] Obtain the supervised corpus of the low-resource language by filtering out the pseudo-labeled data below the confidence scoring threshold.

[0122] In this embodiment, collect the supervised texts of large-scale high natural language processing tasks and translate the supervised texts into the low-resource language by using machine translation technology.

[0123] In order to generate high-quality supervised corpus of the low-resource language, the supervised task corpus (such as question answering, text classification, named entity recognition, etc.) in the high-resource language (such as English, Chinese) can be translated into the target low-resource language by using machine translation technology. In this way, rich low-resource language task data can be quickly obtained in the case of scarce original corpus.

[0124] Based on the parallel sentence mining method and label transfer method of self-supervised learning, the pseudo-annotated data of low-resource languages ​​are automatically mined and annotated from high-resource language corpora through knowledge transfer from high-resource languages ​​to low-resource languages. These pseudo-annotated corpora are used for knowledge transfer through model self-supervised learning to ensure that the knowledge mapping of low-resource languages ​​is strengthened. The pseudo-annotated data is screened through confidence scoring to remove unreliable generated corpora and obtain supervised corpora of low-resource languages, ensuring the quality of the supervised corpora of low-resource languages ​​used for training.

[0125] In operation S5, instruction fine-tuning conversion processing is performed on the supervised corpus in the low-resource language to generate a low-resource language instruction fine-tuning dataset.

[0126] In this example, all supervised corpora of low-resource languages ​​are converted to instructions, and different tasks (such as classification, generation, question answering, etc.) are unified into a consistent input format to build a low-resource language instruction fine-tuning dataset. The purpose of this step is to create a unified and structured instruction dataset to ensure that the model can handle multiple tasks and execute instructions correctly.

[0127] In operation S6, the pre-trained first large language model is fine-tuned according to the low-resource language instruction fine-tuning dataset in combination with the second low-rank adaptive module, so that the first large language model after instruction fine-tuning can execute different task instructions in a variety of low-resource language task environments.

[0128] In this embodiment, according to the low-resource language instruction fine-tuning dataset, and in combination with the new LoRA module (i.e., the second LoRA module), the pre-trained first large language model is fine-tuned. The fine-tuning process enables the large language model to execute different task instructions in a variety of low-resource language task environments. Specifically:

[0129] According to an embodiment of the present disclosure, fine-tuning the instruction of the pre-trained first large language model using the second low-rank adaptive module according to the low-resource language instruction fine-tuning dataset may include, for example:

[0130] Adding a second side branch to the pre-trained first language model, wherein the second side branch is configured to simplify parameter adjustment of the model by introducing a second low-rank matrix;

[0131] The low-resource language instruction fine-tuning dataset is input into the pre-trained first language model for instruction fine-tuning. During the instruction fine-tuning process, the model initialization parameters are fixed and only the parameters of the second branch are trained. The process is similar to the unsupervised pre-training of the first language model after the vocabulary expansion using the first low-rank adaptive module. The principle is shown in Figure 5 .

[0132] Figure 5The schematic diagram shows a principle diagram of fine-tuning instructions on a pre-trained first large language model according to an embodiment of the present disclosure.

[0133] like Figure 5 As shown in the figure, by introducing the new LoRA module, the computing resources required for the model during fine-tuning are further reduced, ensuring that the large language model has higher efficiency and accuracy when executing multi-task instructions of low-resource languages. The instruction fine-tuning formula is as follows:

[0134]

[0135] in, represents the model parameters, Represents a set of instruction fine-tuning input-output pairs.

[0136] In this embodiment, the instruction fine-tuning method includes the following features: during the fine-tuning process, task instructions at different levels of the model are gradually optimized by means of hierarchical fine-tuning to ensure the robustness of the large language model in processing different task instructions; during the fine-tuning process, a multi-task learning framework is combined to ensure that the large language model can simultaneously process multiple low-resource language tasks, including question-answering, text generation, sentiment analysis and other task instructions; a low-rank adaptive module is used to perform lightweight parameter updates to reduce the computational overhead and storage requirements of the model, especially to perform parameter adaptive fine-tuning on specific language subsets of the model in a multilingual environment; during the fine-tuning process, a strategy combining supervised learning and self-supervised learning is used to strengthen the model's understanding and execution of low-resource language instructions, thereby improving the model's instruction execution capabilities and response accuracy in different task scenarios.

[0137] In operation S7, the high-resource language supervised corpus is input into the first language model after instruction fine-tuning and the existing second language model respectively to obtain negative example data and positive example data, and a reinforcement learning low-resource language dataset based on model feedback is generated according to the negative example data and the positive example data.

[0138] In this embodiment, the input part of the supervised corpus in the high-resource language is used to input the first large language model (low-resource language large language model) and the existing mature second large language model (high-resource language large language model) after instruction fine-tuning, respectively, to generate negative data and positive data, and translate the positive and negative data into the low-resource language through translation technology, so as to construct a low-resource language data set for reinforcement learning based on model feedback. In this step, the supervised data in the high-resource language is used to construct a reinforcement learning data set through the mechanism of generating positive and negative data.

[0139] This method generates negative and positive data by comparing the output of the existing mature model with the output of the fine-tuned low-resource model. The output of the large language model for the high-resource language is used as the positive data, and the error output of the large language model for the low-resource language is used as the negative data. After the positive and negative data are generated, they are translated into low-resource languages ​​through machine translation technology so that these data can be used in the reinforcement learning process of low-resource languages. The generated positive and negative data are further used to construct a reinforcement learning low-resource language dataset based on model feedback. This dataset is used to guide the large language model to optimize its task performance on low-resource languages ​​through reinforcement learning, especially the accuracy of instruction execution and generation tasks. In the process of generating negative data, an adversarial learning method is used to annotate and optimize the model's error output by designing a generative adversarial network (GAN) to enhance the model's ability to distinguish and correct error data in low-resource languages.

[0140] In operation S8, reinforcement learning training is performed on the first large language model after instruction fine-tuning according to the reinforcement learning low-resource language dataset in combination with the third low-rank adaptive module to obtain a target large language model.

[0141] According to an embodiment of the present disclosure, based on the reinforcement learning low-resource language dataset, the first large language model after instruction fine-tuning is subjected to reinforcement learning training in combination with the third low-rank adaptive module, and the target large language model is obtained, for example, may include:

[0142] According to the reinforcement learning low-resource language dataset, the direct preference optimization algorithm combined with the third low-rank adaptive module is used to perform reinforcement learning training on the first large language model after instruction fine-tuning to obtain the target large language model.

[0143] In this embodiment, the Direct Preference Optimization (DPO) algorithm is used to further optimize the first language model after instruction fine-tuning through reinforcement learning training. By reinforcing the positive and negative feedback of the model, it is ensured that the model can perform better on low-resource language tasks and improve its adaptability to user preferences. The optimization objectives are expressed as follows:

[0144]

[0145] Among them, the reward function for:

[0146]

[0147] in, represents the parameter distribution of the model, represents the distribution of the current model, which is initialized to the supervised fine-tuned model; represents the model distribution under ideal conditions; Indicates the source language that needs to be translated; Indicates the selected answer, that is, the output that the user or model considers to be better; Represents the rejected answer, that is, the output that was not selected; represents the logistic function, which in this case may be the sigmoid function; Represents the payoff or preference scores associated with the chosen answers and the rejected answers.

[0148] Among them, about the above formula :

[0149] It describes how to optimize the model distribution by directly using preference scores or rewards , to compare the selected answers and the rejected answers, so as to adjust the model to generate outputs that are more in line with user preferences, making it gradually approach the ideal model state .

[0150] In this embodiment, the principle of the third low-rank adaptive module is the same as the principle of the first and second low-rank adaptive modules, that is, the parameter adjustment of the model is simplified by adding a third side branch to the first large language model after fine-tuning the instructions, so the specific process is not repeated here.

[0151] In this embodiment, the direct preference optimization (DPO) algorithm training process includes the following features: during the training process, the DPO algorithm learns the user or model's preference for different output results based on the generated positive and negative feedback data, and directly optimizes the model's preference response to low-resource language tasks; the DPO algorithm gradually enhances the model's response ability to low-resource languages ​​in different tasks by utilizing the model's preference feedback mechanism, especially the preference processing for complex instruction tasks, to ensure that the model can output the optimal response result preferentially; during the DPO algorithm optimization process, the policy gradient algorithm of reinforcement learning is combined, and the model is iteratively trained to gradually improve its understanding and generation capabilities of low-resource languages ​​in the process of continuous feedback learning; through the optimization of the DPO algorithm, the instruction execution accuracy and task processing efficiency of the large language model in low-resource language scenarios are further improved, especially in cross-language task conversion, to ensure that the model has high robustness and accuracy in semantic conversion between different languages.

[0152] The large language model training method for low-resource languages ​​provided in the embodiments of the present disclosure expands the training data set of low-resource languages ​​by integrating low-resource data from multiple sources, solves the problem of data scarcity, and further optimizes the performance of the model in a low-resource language environment for the modeling technology of low-resource languages, so that the model can adapt to different language characteristics and structures more flexibly, and enhances operability and accuracy in practical applications. Through this method, it is possible to achieve efficient training of large language models in low-resource language environments, and promote the popularization and application of natural language processing technology in more languages, which has important practical significance and broad application prospects.

[0153] Figure 6 The structural block diagram of a large language model training device for low-resource languages ​​according to an embodiment of the present disclosure is schematically shown.

[0154] like Figure 6 As shown, the large language model training device 600 for low-resource languages ​​of the embodiment of the present disclosure includes: a first acquisition module 610, a vocabulary expansion module 620, an unsupervised pre-training module 630, a second acquisition module 640, a conversion module 650, an instruction fine-tuning module 660, a model feedback data generation module 670, and a reinforcement learning module 680.

[0155] The first acquisition module 610 is used to acquire low-resource language data and preprocess the low-resource language data to obtain low-resource language unsupervised training data.

[0156] The vocabulary expansion module 620 is used to generate a low-resource language vocabulary based on the low-resource language unsupervised training data, and use the low-resource language vocabulary to expand the vocabulary of the existing first language model so that the first language model after the vocabulary expansion can support the low-resource language.

[0157] The unsupervised pre-training module 630 is used to perform unsupervised pre-training on the first language model after the vocabulary expansion based on the unsupervised training data of the low-resource language in combination with the first low-rank adaptive module, so that the pre-trained first language model can adapt to the language pattern and grammatical characteristics of the low-resource language.

[0158] The second acquisition module 640 is used to acquire high-resource language supervised corpus and generate low-resource language supervised corpus according to the high-resource language supervised corpus.

[0159] The conversion module 650 is used to perform instruction fine-tuning conversion processing on the low-resource language supervised corpus to generate a low-resource language instruction fine-tuning dataset.

[0160] The instruction fine-tuning module 660 is used to fine-tune the instructions of the pre-trained first large language model according to the low-resource language instruction fine-tuning dataset in combination with the second low-rank adaptive module, so that the first large language model after instruction fine-tuning can execute different task instructions in a variety of low-resource language task environments.

[0161] The model feedback data generation module 670 is used to input the high-resource language supervised corpus into the first language model after instruction fine-tuning and the existing second language model, respectively, to obtain negative example data and positive example data, and generate a reinforcement learning low-resource language data set based on model feedback according to the negative example data and the positive example data.

[0162] The reinforcement learning module 680 is used to perform reinforcement learning training on the first large language model after instruction fine-tuning according to the reinforcement learning low-resource language dataset in combination with the third low-rank adaptive module to obtain a target large language model.

[0163] According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits, or at least part of the functions of any one of them can be implemented in one module. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be split into multiple modules for implementation. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be at least partially implemented as hardware circuits, such as field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), systems on chips, systems on substrates, systems on packages, application specific integrated circuits (ASICs), or can be implemented by hardware or firmware in any other reasonable way of integrating or packaging circuits, or can be implemented in any one of the three implementation methods of software, hardware, and firmware, or in any appropriate combination of any of them. Alternatively, according to the embodiments of the present invention, one or more of the modules, submodules, units, and subunits can be at least partially implemented as computer program modules, and when the computer program modules are run, the corresponding functions can be performed.

[0164] For example, any of the first acquisition module 610, the vocabulary expansion module 620, the unsupervised pre-training module 630, the second acquisition module 640, the conversion module 650, the instruction fine-tuning module 660, the model feedback data generation module 670, and the reinforcement learning module 680 can be combined in one module / unit / sub-unit for implementation, or any of the modules / units / sub-units can be split into multiple modules / units / sub-units. Alternatively, at least part of the functions of one or more of these modules / units / sub-units can be combined with at least part of the functions of other modules / units / sub-units and implemented in one module / unit / sub-unit. According to an embodiment of the present disclosure, at least one of the first acquisition module 610, the vocabulary expansion module 620, the unsupervised pre-training module 630, the second acquisition module 640, the conversion module 650, the instruction fine-tuning module 660, the model feedback data generation module 670, and the reinforcement learning module 680 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or by any one of the three implementation methods of software, hardware and firmware or by an appropriate combination of any of them. Alternatively, at least one of the first acquisition module 610, the vocabulary expansion module 620, the unsupervised pre-training module 630, the second acquisition module 640, the conversion module 650, the instruction fine-tuning module 660, the model feedback data generation module 670, and the reinforcement learning module 680 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be performed.

[0165] It should be noted that the part of the large language model training device based on low-resource language in the embodiments of the present disclosure corresponds to the part of the large language model training method based on low-resource language in the embodiments of the present disclosure. The description of the part of the large language model training device based on low-resource language specifically refers to the part of the large language model training method based on low-resource language, which will not be repeated here.

[0166] Figure 7 The structural block diagram of an electronic device suitable for implementing a large language model training method for low-resource languages ​​according to an embodiment of the present disclosure is schematically shown. Figure 7 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0167] like Figure 7As shown, the electronic device 700 according to an embodiment of the present disclosure includes a processor 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage part 708 to a random access memory (RAM) 703. The processor 701 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The processor 701 may also include an onboard memory for caching purposes. The processor 701 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0168] In RAM 703, various programs and data required for the operation of electronic device 700 are stored. Processor 701, ROM 702 and RAM 703 are connected to each other via bus 704. Processor 701 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 702 and / or RAM 703. It should be noted that the program can also be stored in one or more memories other than ROM 702 and RAM 703. Processor 701 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in the one or more memories.

[0169] According to an embodiment of the present disclosure, the electronic device 700 may further include an input / output (I / O) interface 705, which is also connected to the bus 704. The electronic device 700 may further include one or more of the following components connected to the input / output (I / O) interface 705: an input portion 706 including a keyboard, a mouse, etc.; an output portion 707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 708 including a hard disk, etc.; and a communication portion 709 including a network interface card such as a LAN card, a modem, etc. The communication portion 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the input / output (I / O) interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 710 as needed, so that a computer program read therefrom is installed into the storage portion 708 as needed.

[0170] According to an embodiment of the present disclosure, the method flow according to an embodiment of the present disclosure can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 709, and / or installed from the removable medium 711. When the computer program is executed by the processor 701, the above-mentioned functions defined in the system of the embodiment of the present disclosure are executed. According to an embodiment of the present disclosure, the system, equipment, device, module, unit, etc. described above can be implemented by a computer program module.

[0171] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0172] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium. For example, it may include, but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, apparatus, or device.

[0173] For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include the ROM 702 and / or the RAM 703 described above and / or one or more memories other than the ROM 702 and the RAM 703 .

[0174] The embodiments of the present disclosure also include a computer program product, which includes a computer program, and the computer program contains program code for executing the method provided by the embodiments of the present disclosure. When the computer program product runs on an electronic device, the program code is used to enable the electronic device to implement the method provided by the embodiments of the present disclosure.

[0175] When the computer program is executed by the processor 701, the above functions defined in the system / device of the embodiment of the present disclosure are executed. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0176] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 709, and / or installed from the removable medium 711. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0177] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0178] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram may represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions. It can be understood by those skilled in the art that the features recorded in the various embodiments of the present disclosure can be combined and / or combined in a variety of ways, even if such a combination or combination is not explicitly recorded in the present disclosure. In particular, without departing from the spirit and teaching of the present disclosure, the features described in the various embodiments of the present disclosure may be combined and / or combined in a variety of ways. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0179] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A large language model training method for low-resource languages, characterized in that: The method comprises: Acquire low-resource language data, and preprocess the low-resource language data to obtain low-resource language unsupervised training data; Generate a low-resource language vocabulary according to the low-resource language unsupervised training data, and use the low-resource language vocabulary to expand the vocabulary of the existing first language model, so that the first language model after the vocabulary expansion can support the low-resource language; According to the low-resource language unsupervised training data, in combination with the first low-rank adaptive module, unsupervised pre-training is performed on the first language model after the vocabulary expansion, so that the pre-trained first language model can adapt to the language pattern and grammatical characteristics of the low-resource language; Acquire a supervised corpus in a high-resource language, and generate a supervised corpus in a low-resource language based on the supervised corpus in the high-resource language; Performing instruction fine-tuning conversion processing on the supervised corpus of the low-resource language to generate a low-resource language instruction fine-tuning dataset; According to the low-resource language instruction fine-tuning dataset, the pre-trained first large language model is fine-tuned in combination with the second low-rank adaptive module, so that the first large language model after instruction fine-tuning can execute different task instructions in a variety of low-resource language task environments; Inputting the high-resource language supervised corpus into the first language model after instruction fine-tuning and the existing second language model respectively, obtaining negative example data and positive example data respectively, and generating a reinforcement learning low-resource language dataset based on model feedback according to the negative example data and the positive example data; According to the reinforcement learning low-resource language dataset, reinforcement learning training is performed on the first large language model after instruction fine-tuning in combination with the third low-rank adaptive module to obtain a target large language model.

2. The method according to claim 1, characterized in that The preprocessing of the low-resource language data to obtain low-resource language unsupervised training data includes: performing noise removal processing on the low-resource language data; Standardize the format of low-resource language data after noise removal; Perform multilingual data filtering on low-resource language data after format standardization; Remove privacy information from low-resource language data after multilingual data filtering; Generate low-resource language unsupervised training data based on the low-resource language data after privacy information removal.

3. The method according to claim 2, characterized in that The using the low-resource language vocabulary to expand the vocabulary of the existing first language model includes: Merging the low-resource language vocabulary with the vocabulary of the existing first largest language model to obtain a merged vocabulary; According to the merged vocabulary, the head layer of the existing first language model is expanded to adapt it to the expanded vocabulary and support low-resource languages, wherein the head layer includes an embedding layer and an output layer.

4. The method according to claim 1, characterized in that: The performing unsupervised pre-training on the first language model after the vocabulary expansion in combination with the first low-rank adaptive module according to the low-resource language unsupervised training data comprises: Adding a first side branch to the first language model after the vocabulary expansion, wherein the first side branch is configured to simplify parameter adjustment of the model by introducing a first low-rank matrix; The low-resource language unsupervised training data is input into the first language model after the vocabulary is expanded for unsupervised pre-training, wherein during the pre-training process, the model initialization parameters are fixed and remain unchanged, and only the parameters of the first side branch are trained.

5. The method according to claim 1, characterized in that The obtaining of high-resource language supervised corpus and generating low-resource language supervised corpus according to the high-resource language supervised corpus includes: Obtaining supervised corpus in a high-resource language, and translating the supervised corpus in the high-resource language into a low-resource language using machine translation technology; Using a parallel sentence mining method and a label transfer method based on self-supervised learning, mining and labeling pseudo-labeled data of low-resource languages ​​from the supervised corpus of the high-resource languages; Using a corpus quality assessment algorithm to perform confidence scoring on the pseudo-annotated data; By filtering and removing pseudo-annotated data below the confidence score threshold, supervised corpus for low-resource languages ​​is obtained.

6. The method according to claim 1, characterized in that Fine-tuning the low-resource language instruction data set and fine-tuning the instruction of the pre-trained first large language model in combination with the second low-rank adaptive module includes: Adding a second side branch to the pre-trained first large language model, wherein the second side branch is configured to simplify parameter adjustment of the model by introducing a second low-rank matrix; The low-resource language instruction fine-tuning dataset is input into the pre-trained first large language model for instruction fine-tuning, wherein during the instruction fine-tuning process, the model initialization parameters are fixed and remain unchanged, and only the parameters of the second side branch are trained.

7. The method according to claim 1, characterized in that The reinforcement learning low-resource language dataset is used to perform reinforcement learning training on the first large language model after fine-tuning the instructions in combination with the third low-rank adaptive module to obtain the target large language model, including: According to the reinforcement learning low-resource language dataset, the first large language model after instruction fine-tuning is subjected to reinforcement learning training using a direct preference optimization algorithm combined with a third low-rank adaptive module to obtain a target large language model.

8. A large language model training device for low-resource languages, characterized in that: The device comprises: A first acquisition module is used to acquire low-resource language data and pre-process the low-resource language data to obtain low-resource language unsupervised training data; A vocabulary expansion module, configured to generate a low-resource language vocabulary according to the low-resource language unsupervised training data, and use the low-resource language vocabulary to expand the vocabulary of the existing first language model, so that the first language model after the vocabulary expansion can support the low-resource language; An unsupervised pre-training module, used for performing unsupervised pre-training on the first language model after the vocabulary expansion according to the unsupervised training data of the low-resource language in combination with the first low-rank adaptive module, so that the pre-trained first language model can adapt to the language pattern and grammatical characteristics of the low-resource language; A second acquisition module is used to acquire a high-resource language supervised corpus and generate a low-resource language supervised corpus based on the high-resource language supervised corpus; A conversion module, used to perform instruction fine-tuning conversion processing on the supervised corpus in the low-resource language to generate a low-resource language instruction fine-tuning dataset; An instruction fine-tuning module is used to perform instruction fine-tuning on the pre-trained first large language model according to the low-resource language instruction fine-tuning dataset in combination with the second low-rank adaptive module, so that the first large language model after instruction fine-tuning can execute different task instructions in a variety of low-resource language task environments; A model feedback data generation module is used to input the high-resource language supervised corpus into the first language model after instruction fine-tuning and the existing second language model, respectively, to obtain negative example data and positive example data, and generate a reinforcement learning low-resource language data set based on model feedback according to the negative example data and the positive example data; The reinforcement learning module is used to perform reinforcement learning training on the first large language model after instruction fine-tuning according to the reinforcement learning low-resource language dataset in combination with the third low-rank adaptive module to obtain a target large language model.

9. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having executable instructions stored thereon, characterized in that: When the instruction is executed by a processor, the processor executes the method according to any one of claims 1 to 7.

Images