Low-resource language-oriented large language model training method and device, equipment and medium

Abstract

The present disclosure provides a large language model training method and device for low-resource languages, equipment and medium, relating to the technical field of artificial intelligence. The method comprises: obtaining low-resource language data and preprocessing to obtain low-resource language unsupervised training data; generating a low-resource language vocabulary and expanding the vocabulary of an existing model; unsupervised pre-training the model after vocabulary expansion according to the low-resource language unsupervised training data combined with a first low-rank adaptive module; instruction fine-tuning of the pre-trained model according to the low-resource language instruction fine-tuning data set combined with a second low-rank adaptive module; inputting high-resource language supervised corpus into the instruction fine-tuned model and the existing mature model respectively to obtain negative and positive example data respectively, thereby generating a reinforcement learning low-resource language data set based on model feedback, and combining a third low-rank adaptive module to perform reinforcement learning training on the instruction fine-tuned model.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of artificial intelligence, and more particularly, to a large language model training method and device for low-resource languages, equipment and medium. BACKGROUND

[0002] In recent years, with the rapid development of artificial intelligence (AI) technology, natural language processing (NLP) based on deep learning has been widely applied in many application scenarios. As one of the core technologies in the current NLP field, large language models (LLM) have shown strong capabilities in processing natural language in various tasks, such as machine translation, text generation, information retrieval, question-answering systems, and sentiment analysis, etc. Through the training of large-scale corpus, large language models can understand and generate complex patterns of natural language, and have achieved remarkable results in many practical applications.

[0003] In the process of implementing the present disclosure, the inventors have found that at least the following defects exist in the related art: The training of these large language models is highly dependent on rich high-resource language data (such as English, Chinese, French, etc.), and for low-resource languages (such as some minority languages or endangered languages in certain regions), due to the lack of sufficient training data, the practical application of the model is greatly limited. SUMMARY

[0004] In view of the above problems, the present disclosure provides a large language model training method and device for low-resource languages, equipment and medium, to solve the technical problem of insufficient support for low-resource languages by existing large language models.

[0005] One aspect of the present disclosure provides a large language model training method for low-resource languages, including: obtaining low-resource language data, and preprocessing the low-resource language data to obtain low-resource language unsupervised training data; generating a low-resource language vocabulary table according to the low-resource language unsupervised training data, and expanding the vocabulary table of an existing first large language model using the low-resource language vocabulary table, so that the first large language model after vocabulary table expansion can support the low-resource language; unsupervised pre-training the first large language model after vocabulary table expansion according to the low-resource language unsupervised training data, in combination with a first low-rank adaptive module, so that the first large language model after pre-training can adapt to the language mode and grammar characteristics of the low-resource language; obtaining high-resource language supervised corpus, and generating low-resource language supervised corpus according to the high-resource language supervised corpus; performing instruction fine-tuning conversion processing on the low-resource language supervised corpus to generate a low-resource language instruction fine-tuning data set; performing instruction fine-tuning on the first large language model after pre-training according to the low-resource language instruction fine-tuning data set, in combination with a second low-rank adaptive module, so that the first large language model after instruction fine-tuning can execute different task instructions in a variety of low-resource language task environments; inputting the high-resource language supervised corpus into the first large language model after instruction fine-tuning and an existing second large language model respectively to obtain negative example data and positive example data respectively, and generating a reinforcement learning low-resource language data set based on model feedback according to the negative example data and the positive example data; performing reinforcement learning training on the first large language model after instruction fine-tuning according to the reinforcement learning low-resource language data set, in combination with a third low-rank adaptive module, to obtain a target large language model.

[0006] According to the embodiments of the present disclosure, the preprocessing of the low-resource language data to obtain the low-resource language unsupervised training data includes: noise removal processing of the low-resource language data; format standardization processing of the low-resource language data after noise removal processing; multilingual data filtering processing of the low-resource language data after format standardization processing; privacy information removal processing of the low-resource language data after multilingual data filtering processing; and generating the low-resource language unsupervised training data according to the low-resource language data after privacy information removal processing.

[0007] According to the embodiments of the present disclosure, the vocabulary table expansion of the existing first large language model using the low-resource language vocabulary table includes: merging the low-resource language vocabulary table and the vocabulary table of the existing first large language model to obtain a merged vocabulary table; expanding the head layer of the existing first large language model according to the merged vocabulary table to adapt to the expanded vocabulary table, so as to support the low-resource language, wherein the head layer includes an embedding layer and an output layer.

[0008] According to an embodiment of the present disclosure, the unsupervised pre-training of the first large language model after the vocabulary expansion in combination with the first low-rank adaptive module comprises: adding a first side branch to the first large language model after the vocabulary expansion, wherein the first side branch is configured to be able to simplify the parameter adjustment of the model by introducing a first low-rank matrix; inputting the low-resource language unsupervised training data into the first large language model after the vocabulary expansion for unsupervised pre-training, wherein in the pre-training process, the model initialization parameters are fixed and only the parameters of the first side branch are trained.

[0009] According to an embodiment of the present disclosure, the obtaining of the high-resource language supervised corpus and the generation of the low-resource language supervised corpus from the high-resource language supervised corpus comprise: obtaining the high-resource language supervised corpus, translating the high-resource language supervised corpus into a low-resource language by using a machine translation technology; mining and labeling pseudo-labeled data of the low-resource language from the high-resource language supervised corpus by using a parallel sentence mining method based on self-supervised learning and a label migration method; performing confidence scoring on the pseudo-labeled data by using a corpus quality evaluation algorithm; and obtaining the low-resource language supervised corpus by removing pseudo-labeled data below a confidence score threshold through screening.

[0010] According to an embodiment of the present disclosure, the instruction fine-tuning of the pre-trained first large language model in combination with the second low-rank adaptive module according to the low-resource language instruction fine-tuning dataset comprises: adding a second side branch to the pre-trained first large language model, wherein the second side branch is configured to be able to simplify the parameter adjustment of the model by introducing a second low-rank matrix; inputting the low-resource language instruction fine-tuning dataset into the pre-trained first large language model for instruction fine-tuning, wherein in the instruction fine-tuning process, the model initialization parameters are fixed and only the parameters of the second side branch are trained.

[0011] According to an embodiment of the present disclosure, the reinforcement learning training of the instruction fine-tuned first large language model according to the reinforcement learning low-resource language dataset to obtain a target large language model comprises: performing reinforcement learning training of the instruction fine-tuned first large language model according to the reinforcement learning low-resource language dataset by using a direct preference optimization algorithm in combination with a third low-rank adaptive module to obtain the target large language model.

[0012] Another aspect of the present disclosure provides a large language model training device for low-resource languages, comprising: a first acquisition module configured to acquire low-resource language data, and preprocess the low-resource language data to obtain low-resource language unsupervised training data; a vocabulary expansion module configured to generate a low-resource language vocabulary based on the low-resource language unsupervised training data, and expand the vocabulary of an existing first large language model using the low-resource language vocabulary, so that the first large language model after vocabulary expansion can support low-resource languages; an unsupervised pre-training module configured to pre-train the first large language model after vocabulary expansion based on the low-resource language unsupervised training data, in combination with a first low-rank adaptive module, so that the first large language model after pre-training can adapt to the language patterns and grammatical characteristics of low-resource languages; a second acquisition module configured to acquire high-resource language supervised corpus, and generate low-resource language supervised corpus based on the high-resource language supervised corpus; a conversion module configured to perform instruction fine-tuning conversion processing on the low-resource language supervised corpus to generate a low-resource language instruction fine-tuning dataset; an instruction fine-tuning module configured to fine-tune the first large language model after pre-training based on the low-resource language instruction fine-tuning dataset, in combination with a second low-rank adaptive module, so that the first large language model after instruction fine-tuning can execute different task instructions in a variety of low-resource language task environments; a model feedback data generation module configured to input the high-resource language supervised corpus into the first large language model after instruction fine-tuning and an existing second large language model respectively, to obtain negative example data and positive example data respectively, and generate a reinforcement learning low-resource language dataset based on model feedback based on the negative example data and the positive example data; and a reinforcement learning module configured to train the first large language model after instruction fine-tuning based on the reinforcement learning low-resource language dataset, in combination with a third low-rank adaptive module, to obtain a target large language model.

[0013] Another aspect of the present disclosure provides an electronic device, comprising: one or more processors; a memory for storing one or more programs; wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method as above.

[0014] Another aspect of the present disclosure provides a computer-readable storage medium storing computer-executable instructions that, when executed, implement the method as above.

[0015] Another aspect of the present disclosure provides a computer program product comprising computer-executable instructions that, when executed, implement the method as above.

[0016] Compared with the prior art, the large language model training method, device, equipment and medium for low-resource languages provided by the present disclosure have at least the following beneficial effects:

[0017] (1) The low-resource language-oriented large language model training method, device, equipment and medium provided by the present disclosure, by constructing a multi-task learning framework suitable for low-resource languages, and combining the Low-Rank Adaptation (LoRA) technology, the existing large language model is further pre-trained and fine-tuned, which realizes the goal of improving the processing capability of low-resource languages under the condition of limited hardware resources and data. This method effectively utilizes the data and model experience of high-resource languages, and enhances the performance of the model on low-resource languages through cross-language transfer learning strategy, especially in language understanding and text generation tasks, which has achieved significant improvement.

[0018] (2) The low-resource language-oriented large language model training method, device, equipment and medium provided by the present disclosure, by fusing low-resource data from multiple sources, the training data set of low-resource languages is expanded, solving the problem of data scarcity, and further optimizing the performance of the model in the low-resource language environment according to the modeling technology of low-resource languages, so that the model can more flexibly adapt to different language characteristics and structures, and enhance the operability and accuracy in practical applications. Through this method, efficient training of large language models in low-resource language environments can be realized, and the popularization and application of natural language processing technology in more languages can be promoted, which has important practical significance and wide application prospect. BRIEF DESCRIPTION OF DRAWINGS

[0019] The above and other objects, features and advantages of the present disclosure will become more apparent from the following description of the embodiments of the present disclosure, taken in conjunction with the accompanying drawings, in which:

[0020] Figure 1 The flowchart of the low-resource language-oriented large language model training method according to the embodiment of the present disclosure is schematically shown;

[0021] Figure 2 The construction principle diagram of the word segmenter in the low-resource language-oriented large language model training method according to the embodiment of the present disclosure is schematically shown;

[0022] Figure 3 The principle diagram of the word table expansion of the first large language model in the low-resource language-oriented large language model training method according to the embodiment of the present disclosure is schematically shown;

[0023] Figure 4 The principle diagram of the first large language model after word table expansion causal language modeling unsupervised pre-training according to the embodiment of the present disclosure is schematically shown;

[0024] Figure 5 The principle diagram of the instruction fine-tuning of the pre-trained first large language model according to the embodiment of the present disclosure is schematically shown;

[0025] Figure 6 A structural block diagram of a large language model training apparatus for a low-resource language according to an embodiment of the disclosure is schematically shown;

[0026] Figure 7 A structural block diagram of an electronic device adapted to implement a large language model training method for a low-resource language according to an embodiment of the disclosure is schematically shown. DETAILED DESCRIPTION

[0027] Hereinafter, embodiments of the disclosure will be described with reference to the accompanying drawings. It should be understood, however, that the description which follows is merely illustrative and is not intended to limit the scope of the disclosure. In the following detailed description of embodiments of the disclosure, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the disclosure. However, it will be apparent to one skilled in the art that the embodiments of the disclosure can be practiced without these specific details. In other instances, well-known structures and functions have not been described in detail in order to avoid obscuring the concepts of the disclosure.

[0028] The terms used herein are merely used to describe specific embodiments, and are not intended to limit the disclosure. The terms "include" and "have" and the like used herein indicate the presence of the described features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0029] All terms used herein, including technical and scientific terms, have the same meanings as those generally understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having meanings consistent with the context of the specification, and should not be interpreted in an idealized or overly formal manner.

[0030] In the case of using expressions similar to "at least one of A, B, and C, etc.", it should be generally interpreted as including at least one of the items enumerated, but not limited to the items enumerated (e.g., "a system having at least one of A, B, and C" should include a system having A alone, a system having B alone, a system having C alone, a system having A and B together, a system having A and C together, a system having B and C together, and / or a system having A, B, and C together, etc.).

[0031] In the embodiments of the disclosure, the collection, updating, analysis, processing, use, transmission, provision, disclosure, storage, etc. of the data involved (for example, including but not limited to user personal information) comply with the relevant legal regulations, are used for legal purposes, and do not violate public order and good customs. In particular, necessary measures are taken for user personal information to prevent illegal access to user personal information data, and to maintain user personal information security, network security, and national security.

[0032] In recent years, with the rapid development of Artificial Intelligence (AI) technology, Natural Language Processing (NLP) based on deep learning has been widely applied in many application scenarios. Large Language Model (LLM) as one of the core technologies in the current NLP field, shows strong ability in processing natural language in various tasks, such as machine translation, text generation, information retrieval, question answering system and sentiment analysis, etc. Through the training of large-scale corpus, large language model can understand and generate complex patterns of natural language, and has achieved remarkable results in many practical applications.

[0033] In the process of implementing the present disclosure, the inventors found that at least the following defects exist in the related art: the training of these large language models is highly dependent on rich high-resource language data (such as English, Chinese, French, etc.), and for low-resource languages (such as some minority languages or endangered languages in certain regions), due to the lack of sufficient training data, the practical application of the model is greatly limited.

[0034] Low-Resource Languages generally refer to languages with relatively small number of users and lack of digital resources, such as some minority languages or endangered languages in certain regions. Due to the scarcity of data resources for these languages, there is a lack of large-scale parallel corpus and pre-labeled data, and the existing large language models perform poorly on low-resource languages. At the same time, low-resource languages often have unique language structures and complex grammatical rules, making it particularly difficult for these languages to perform well in natural language processing tasks. In addition, some low-resource languages lack standardized writing systems and annotation specifications, increasing the difficulty of data processing.

[0035] For natural language processing research on low-resource languages, existing methods mainly focus on multilingual modeling, cross-language transfer, and data enhancement techniques. For example, many researchers attempt to share parameters or weights between low-resource languages and high-resource languages, leveraging the rich corpus and model training experience of high-resource languages to improve the language processing capabilities of low-resource languages. While this approach can alleviate the problem of data scarcity in low-resource languages to some extent, the model still faces problems of insufficient semantic understanding and low generation accuracy when dealing with low-resource languages due to the large differences between languages. In addition, cross-language transfer learning usually transfers between high-resource languages and low-resource languages, but for low-resource languages with large differences in grammatical structure, the effectiveness of this approach is limited. To address these challenges, some researchers have proposed large language model training methods based on pre-training and instruction fine-tuning strategies. Pre-trained models are trained unsupervised on a large amount of multilingual corpus to capture general features of multiple languages, and then fine-tuned on specific low-resource language tasks to adapt to specific language features. However, this approach still faces several key problems: First, it is difficult to obtain low-resource language data, and the proportion of low-resource languages in existing multilingual parallel corpus is small, which cannot meet the training needs of large language models. Second, even if low-resource language data can be obtained, the quality of these data is uneven, with a lot of noise and incomplete information, making it difficult to be directly used for high-quality model training. In addition, how to effectively expand the vocabulary of low-resource languages in the model and ensure that the expanded model can support the processing of low-resource languages without affecting the performance of high-resource languages is also a major technical challenge.

[0036] To address the above shortcomings, in recent years, some new technical directions have begun to emerge, such as the Low-Rank Adaptation (LoRA) technology. It is an innovative method for fine-tuning large models, which can adjust the parameters of large language models without increasing excessive computational overhead, enabling them to adapt to new languages or tasks, especially in resource-limited situations. At the same time, through Self-Supervised Learning and Multi-Task Learning techniques, the understanding ability and generation quality of the model for low-resource languages can be effectively improved. The introduction of these technologies provides new ideas and directions for building large language models for low-resource languages. However, despite the progress made by these methods, training large language models for low-resource languages still faces many challenges. For example, how to systematically collect and process unsupervised and supervised data for low-resource languages, how to efficiently clean and standardize data, how to reasonably expand and adapt the vocabulary of low-resource languages in the model, and how to design effective training algorithms to improve the performance of the model on low-resource languages, are all problems that need to be solved in the field of natural language processing. Therefore, there is an urgent need for a large language model training method for low-resource languages, which effectively utilizes existing low-resource language data resources by combining various innovative technologies, and improves the understanding and generation capabilities of the model on low-resource languages through reasonable model architecture design and optimization strategies, providing strong support for low-resource language processing applications worldwide.

[0037] Based on this, the embodiments of the present disclosure provide a large language model training method for low-resource languages to solve the technical problem of insufficient support for low-resource languages by existing large language models. The method comprises:

[0038] Obtaining low-resource language data and preprocessing the low-resource language data to obtain low-resource language unsupervised training data;

[0039] Generating a low-resource language vocabulary based on the low-resource language unsupervised training data, and using the low-resource language vocabulary to expand the vocabulary of an existing first large language model, so that the first large language model after vocabulary expansion can support low-resource languages;

[0040] According to the low-resource language unsupervised training data, combining the first low-rank adaptive module to unsupervisedly pre-train the first large language model after vocabulary expansion, so that the pre-trained first large language model can adapt to the language patterns and grammatical characteristics of low-resource languages;

[0041] Obtaining high-resource language supervised corpus and generating low-resource language supervised corpus based on the high-resource language supervised corpus;

[0042] Supervised corpora of low-resource languages ​​are processed for instruction fine-tuning transformation to generate a low-resource language instruction fine-tuning dataset.

[0043] Based on the low-resource language instruction fine-tuning dataset, the first pre-trained language model is fine-tuned by combining the second low-rank adaptive module, so that the first language model after instruction fine-tuning can execute different task instructions in a variety of low-resource language task environments.

[0044] The high-resource language supervised corpus is input into the first large language model after fine-tuning and the existing second large language model respectively to obtain negative and positive examples. Based on the negative and positive examples, a reinforcement learning low-resource language dataset based on model feedback is generated.

[0045] Based on the low-resource language dataset for reinforcement learning, the first large language model after instruction fine-tuning is trained by reinforcement learning in conjunction with the third low-rank adaptive module to obtain the target large language model.

[0046] The large language model training method based on low-resource languages ​​provided in this disclosure constructs a multi-task learning framework suitable for low-resource languages ​​and combines it with low-rank adaptation (LoRA) technology to further pre-train and fine-tune existing large language models, achieving the goal of improving the processing capabilities of low-resource languages ​​under limited hardware resources and data conditions. This method effectively utilizes data and model experience from high-resource languages ​​and enhances the model's performance on low-resource languages ​​through cross-language transfer learning strategies, achieving significant improvements, particularly in language understanding and text generation tasks.

[0047] To make the objectives, technical solutions, and advantages of this disclosure clearer, the following detailed description is provided in conjunction with specific embodiments and the accompanying drawings.

[0048] Figure 1 A flowchart illustrating a method for training large language models for low-resource languages ​​according to an embodiment of the present disclosure is shown.

[0049] like Figure 1 As shown, the large language model training method for low-resource languages ​​in this embodiment may include, for example, operations S1 to S8.

[0050] In operation S1, low-resource language data is acquired and preprocessed to obtain low-resource language unsupervised training data.

[0051] In the embodiment, first, the existing multilingual translation platform and parallel corpus are used to obtain the low-resource language corpus, and the low-resource language data of various sources is sorted out, and on this basis, the low-resource language data resource is obtained by converting the public low-resource language literature without a ready-made digital version into a usable text resource.

[0052] Then, the obtained low-resource language data is preprocessed to obtain low-resource language unsupervised training data, and specifically:

[0053] According to the embodiment of the disclosure, the preprocessing of the low-resource language data to obtain the low-resource language unsupervised training data may include, for example:

[0054] The low-resource language data is subjected to noise removal processing;

[0055] The low-resource language data subjected to the noise removal processing is subjected to format standardization processing;

[0056] The low-resource language data subjected to the format standardization processing is subjected to multilingual data filtering processing;

[0057] The low-resource language data subjected to the multilingual data filtering processing is subjected to privacy information removal processing;

[0058] The low-resource language data subjected to the privacy information removal processing is used to generate the low-resource language unsupervised training data.

[0059] In the embodiment, the preprocessing of the obtained low-resource language data mainly adopts a heuristic data cleaning method, and a set of detailed rules are formulated for the multilingual scene, mainly including noise removal, format standardization (such as unified coding format, punctuation symbol specification), multilingual data filtering (avoiding the mixing of incorrect languages), and privacy information removal.

[0060] The data cleaning may include, for example, language filtering, statistical feature filtering, keyword filtering, and the like, and high-quality text is retained as much as possible.

[0061] The privacy information removal may use, for example, a rule-based algorithm to detect and delete or replace personal information content such as names, addresses, email addresses, and telephone numbers.

[0062] In addition, the preprocessing process may also include advertisement filtering and data deduplication. If the data contains a large amount of advertising information, it will seriously affect the generation effect of the model, and therefore, the advertisement content in the corpus can be filtered by keyword filtering. The repeated redundant data will reduce the diversity of the language model and cause the training process to be unstable, affecting the performance of the model, and therefore, the minimum hash algorithm can be used to complete the corpus deduplication in a short time.

[0063] Through these preprocessing steps, high-quality, privacy-free low-resource language data is retained, and a dataset for model unsupervised training is constructed. Since the data sources of low-resource languages often contain inconsistent, chaotic or incomplete content, preprocessing can significantly improve the purity and training effect of the data.

[0064] In operation S2, a low-resource language vocabulary is generated according to the low-resource language unsupervised training data, and the existing first large language model is expanded with the low-resource language vocabulary, so that the first large language model after vocabulary expansion can support the low-resource language.

[0065] In this embodiment, a low-resource language vocabulary is generated according to the preprocessed corpus.

[0066] The vocabulary is a list containing all the words in a specific language, and is usually used to map the words in the text to a numerical representation (such as word embedding) that the model can understand.

[0067] In another embodiment, based on the vocabulary, a tokenizer can also be generated according to the preprocessed corpus.

[0068] The tokenizer is an important component in natural language processing, mainly used to divide text into smaller units (called "word pieces" or "tokens"), which can be words, characters or subwords, depending on the tokenization strategy. In this embodiment, the construction principle of the tokenizer is shown in Figure 2 .

[0069] Figure 2 The construction principle diagram of the tokenizer in the large language model training method for low-resource languages according to the embodiments of the present disclosure is schematically shown.

[0070] As shown in Figure 2 , the unsupervised data of the low-resource language is processed into the tokenizer by the byte pair encoding algorithm.

[0071] The byte pair encoding algorithm (BPE) is an algorithm for decomposing words into subword units, which is particularly suitable for processing low-resource languages and reducing vocabulary sparsity. This tokenizer contains a word piece method for low-resource languages, which helps the model effectively decompose text into subwords or word pieces when processing the language.

[0072] After generating the low-resource language vocabulary, the existing first large language model is expanded with the low-resource language vocabulary, so that the first large language model after vocabulary expansion can support the low-resource language, specifically:

[0073] According to an embodiment of the present disclosure, the table expansion of the existing first large language model by using the low-resource language table may, for example, include:

[0074] The low-resource language table and the table of the existing first large language model are merged to obtain a merged table;

[0075] According to the merged table, the head layer of the existing first large language model is expanded to adapt to the expanded table, and support for the low-resource language is realized, wherein the head layer includes an embedding layer and an output layer.

[0076] In the present embodiment, after generating the low-resource language table, the principle of table expansion of the existing first large language model by using the low-resource language table is shown in Figure 3 .

[0077] Figure 3 The principle diagram of the table expansion of the first large language model in the large language model training method for low-resource languages according to an embodiment of the present disclosure is schematically shown.

[0078] As shown in Figure 3 , the table expansion is performed by merging the table of the low-resource language with the existing table, and expanding the head layer (especially the embedding layer and the output layer) of the first large language model according to the size of the merged table, to ensure that the model can process the expanded table content, thereby realizing support for the low-resource language.

[0079] In operation S3, the first large language model after table expansion is unsupervised pre-trained according to the low-resource language unsupervised training data combined with the first low-rank adaptive (LoRA) module, so that the pre-trained first large language model can adapt to the language mode and grammar characteristics of the low-resource language.

[0080] In the present embodiment, based on the design of the LoRA module and the adopted unsupervised continued pre-training strategy, the first large language model can quickly adapt to the special mode and grammar characteristics of the low-resource language while retaining the original multilingual capabilities, specifically:

[0081] According to an embodiment of the present disclosure, the unsupervised pre-training of the first large language model after table expansion combined with the first low-rank adaptive module may, for example, include:

[0082] A first side branch is added to the first large language model after table expansion, wherein the first side branch is configured to be able to simplify the parameter adjustment of the model by introducing a first low-rank matrix;

[0083] The low-resource language unsupervised training data is input into the first large language model after the vocabulary expansion, and in the pre-training process, the model initialization parameters are fixed and only the parameters of the first bypass are trained.

[0084] In this embodiment, LoRA simplifies the parameter adjustment of the model by introducing a low-rank matrix, reduces the training cost and computational burden, and thus can still achieve significant improvement of model performance under low data volume.

[0085] The principle of LoRA is relatively simple, that is, a bypass is added to the first large language model, and dimension reduction and dimension increase operations are performed. Assuming that a large language model is fine-tuned in a downstream task, the pre-training model parameters need to be updated, which is expressed by the following formula:

[0086] W o +ΔW

[0087] wherein W o represents the initialized parameters of the first large language model, and ΔW represents the parameters that need to be updated.

[0088] If full-parameter fine-tuning is performed, the training parameter amount is W o , for example, the parameter amount of GPT-3 is as high as 175B, and the cost of full-parameter fine-tuning of the large language model is very high, while LORA only needs to fine-tune ΔW. Specifically, assuming that the pre-training weight parameter matrix is , the parameter updating mode of LoRA is expressed as follows:

[0089]

[0090] wherein the rank , in the training process of LoRA, W o is fixed and unchanged, and only the parameters of the matrix and need to be trained.

[0091] In the forward process, W o and ΔW are multiplied by the same input , and finally added, which is expressed by the following formula:

[0092]

[0093] Assuming that the loss function of the model is , then the gradient calculation of the parameters and in the training process is as follows:

[0094]

[0095] Therefore, the LoRA technique reduces the parameter amount during fine-tuning by low-rank decomposition without sacrificing the performance of the model. The specific formula of unsupervised pre-training is as follows:

[0096]

[0097] wherein, denotes the model parameters, denotes the current predicted token, denotes the conditional context.

[0098] Figure 4 The principle diagram of the unsupervised pre-training of the first large language model after the expansion of the vocabulary is schematically shown according to the embodiment of the present disclosure.

[0099] As Figure 4 shown, the unsupervised pre-training process of the first large language model after the expansion of the vocabulary is shown, and the model gradually predicts the next word element of the input through the time step, and gradually generates a complete sentence. Each time step in the figure shows the process of the model predicting the current word element given the previous context, in particular:

[0100] Time step 1:

[0101] The input word element is "is".

[0102] The model predicts the word element based on the expanded vocabulary. At this time, the model has read and processed "is", but the next step still needs to predict the next word.

[0103] The initial part of the predicted sentence is "this is …".

[0104] Time step 2:

[0105] The input word element is "one".

[0106] The model then reads and processes the next word element "one" and continues to search for a matching word in the expanded vocabulary. At this time, the model can already make an inference based on the previous word "is" and successfully read "one".

[0107] The partial information of the sentence at this time is "this is a …".

[0108] Time step 3:

[0109] The input word element is "a".

[0110] The model processes the next word element "a" and continues to infer based on the context. At this time, the model has successfully predicted a more complete part of the sentence.

[0111] The partial information of the sentence is "this is a …".

[0112] Time step 4:

[0113] The input token is "table".

[0114] The model predicts "table" at this step, which is the last token of the sentence. At this time, the model makes a reasonable conclusion by inferring from the previous context and combining the expanded vocabulary.

[0115] The sentence is completed as "This is a table".

[0116] In operation S4, a high-resource language supervised corpus is obtained, and a low-resource language supervised corpus is generated according to the high-resource language supervised corpus.

[0117] According to an embodiment of the present disclosure, obtaining a high-resource language supervised corpus and generating a low-resource language supervised corpus according to the high-resource language supervised corpus can include, for example:

[0118] The high-resource language supervised corpus is obtained, and the high-resource language supervised corpus is translated into a low-resource language by using a machine translation technology;

[0119] Parallel sentence mining methods and label transfer methods based on self-supervised learning are used to mine and label pseudo-labeled data of the low-resource language from the high-resource language supervised corpus;

[0120] The corpus quality evaluation algorithm is used to score the confidence of the pseudo-labeled data;

[0121] By screening, pseudo-labeled data below the confidence score threshold is removed, and a low-resource language supervised corpus is obtained.

[0122] In this embodiment, a large amount of supervised text of high natural language processing tasks is collected, and the supervised text is translated into a low-resource language by using a machine translation technology.

[0123] In order to generate high-quality low-resource language supervised corpus, supervised task corpus (such as question answering, text classification, named entity recognition, etc.) in high-resource language (such as English, Chinese) can be translated into target low-resource language by using machine translation technology. In this way, rich low-resource language task data can be quickly obtained in the case of scarce original corpus.

[0124] Further based on the parallel sentence mining method and label transfer method of self-supervised learning, through knowledge transfer from high-resource language to low-resource language, pseudo-labeled data of low-resource language is automatically mined and labeled from high-resource language corpus. These pseudo-labeled corpora are used for knowledge transfer through model self-supervised learning, ensuring that the knowledge mapping of low-resource language is strengthened. The pseudo-labeled data is filtered by confidence score to remove unreliable generated corpora, obtaining low-resource language supervised corpus, ensuring the quality of low-resource language supervised corpus for final training.

[0125] In operation S5, the low-resource language supervised corpus is subjected to instruction fine-tuning conversion processing to generate a low-resource language instruction fine-tuning dataset.

[0126] In this embodiment, all low-resource language supervised corpora are subjected to instruction conversion, unifying different tasks (such as classification, generation, question and answer, etc.) into consistent input formats to construct a low-resource language instruction fine-tuning dataset. The purpose of this step is to create a unified and structured instruction dataset to ensure that the model can handle multiple tasks and execute instructions correctly.

[0127] In operation S6, according to the low-resource language instruction fine-tuning dataset, the pre-trained first large language model is subjected to instruction fine-tuning in combination with the second low-rank adaptive module, so that the instruction fine-tuned first large language model can execute different task instructions in multiple low-resource language task environments.

[0128] In this embodiment, according to the low-resource language instruction fine-tuning dataset, the pre-trained first large language model is subjected to instruction fine-tuning in combination with the new LoRA module (i.e., the second LoRA module). This fine-tuning process enables the large language model to execute different task instructions in multiple low-resource language task environments. Specifically:

[0129] According to an embodiment of the present disclosure, according to the low-resource language instruction fine-tuning dataset, the pre-trained first large language model is subjected to instruction fine-tuning using the second low-rank adaptive module, which may, for example, include:

[0130] A second side branch is added to the pre-trained first large language model, wherein the second side branch is configured to simplify the parameter adjustment of the model by introducing a second low-rank matrix;

[0131] The low-resource language instruction fine-tuning dataset is input into the pre-trained first large language model for instruction fine-tuning. During the instruction fine-tuning process, the model initialization parameters remain unchanged, and only the parameters of the second side branch are trained. The process is similar to the unsupervised pre-training of the first large language model with the first low-rank adaptive module, and the principle is described in Figure 5 .

[0132] Figure 5A schematic diagram illustrating the principle of instruction fine-tuning of the pre-trained first large language model according to an embodiment of the present disclosure is shown.

[0133] As shown in Figure 5 , by introducing a new LoRA module, the computing resources required by the model during fine-tuning are further reduced, ensuring that the large language model has higher efficiency and accuracy when executing multi-task instructions in low-resource languages, where the instruction fine-tuning formula is as follows:

[0134]

[0135] wherein, represents the model parameters, represents the instruction fine-tuning input-output pair set.

[0136] In this embodiment, the instruction fine-tuning method includes the following features: during the fine-tuning process, the different layers of the model are gradually optimized for task instructions in a hierarchical fine-tuning manner to ensure the robustness of the large language model when processing different task instructions; during the fine-tuning process, a multi-task learning framework is combined to ensure that the large language model can handle multiple low-resource language tasks simultaneously, including question answering, text generation, sentiment analysis, and other task instructions; a low-rank adaptive module is used for lightweight parameter updating to reduce the computational overhead and storage requirements of the model, especially for parameter adaptive fine-tuning of specific language subsets of the model in a multi-language environment; during the fine-tuning process, a combination of supervised learning and self-supervised learning strategies is used to strengthen the model's understanding and execution of low-resource language instructions, improving the model's instruction execution capability and response accuracy in different task scenarios.

[0137] In operation S7, the high-resource language supervised corpus is input into the instruction fine-tuned first large language model and the existing second large language model, respectively, to obtain negative example data and positive example data, and based on the negative example data and the positive example data, a reinforcement learning low-resource language dataset based on model feedback is generated.

[0138] In this embodiment, the input part of the high-resource language supervised corpus is input into the instruction fine-tuned first large language model (low-resource language large language model) and the existing mature second large language model (high-resource language large language model), respectively, to generate negative example data and positive example data, and the positive and negative example data is translated into low-resource languages through translation technology, thereby constructing a reinforcement learning low-resource language dataset based on model feedback. In this step, the supervised data in the high-resource language is used to construct a reinforcement learning dataset through the mechanism of positive and negative example data generation.

[0139] The method generates negative and positive example data by comparing the outputs of the existing mature model and the fine-tuned low-resource model, wherein the output of the high-resource language large language model is taken as the positive example data, and the error output of the low-resource language large language model is taken as the negative example data. After the generation of the positive and negative example data, the machine translation technology is used to translate them into the low-resource language, so that these data can be used in the reinforcement learning process of the low-resource language; further, the generated positive and negative example data are used to construct a reinforcement learning low-resource language data set based on model feedback, which is used to guide the large language model to optimize its task performance in the low-resource language through reinforcement learning, especially the accuracy of instruction execution and generation tasks; wherein, in the negative example data generation process, an adversarial learning method is used, and a generative adversarial network (GAN) is designed to label and optimize the error output of the model, so as to enhance the model's ability to distinguish and correct the error data of the low-resource language.

[0140] In operation S8, the first large language model fine-tuned for instructions is trained through reinforcement learning according to the reinforcement learning low-resource language data set in combination with the third low-rank adaptive module, and a target large language model is obtained.

[0141] According to an embodiment of the present disclosure, the reinforcement learning training of the first large language model fine-tuned for instructions according to the reinforcement learning low-resource language data set in combination with the third low-rank adaptive module to obtain a target large language model may, for example, include:

[0142] According to an embodiment of the present disclosure, the reinforcement learning training of the first large language model fine-tuned for instructions according to the reinforcement learning low-resource language data set in combination with the third low-rank adaptive module to obtain a target large language model may, for example, include:

[0143] In this embodiment, the Direct Preference Optimization (DPO) algorithm is used to further optimize the first large language model fine-tuned for instructions through reinforcement learning training. Through reinforcement training of positive and negative feedback of the model, it is ensured that the model can have better performance on the low-resource language task and improve its adaptability to user preferences. The optimization target is represented as follows:

[0144]

[0145] wherein the reward function is:

[0146]

[0147] wherein represents the parameter distribution of the model, represents the distribution of the current model, which is initialized as the model after supervised fine-tuning; denotes the model distribution in the ideal state; denotes the source language that needs to be translated; denotes the selected answer, i.e., the output that the user or the model considers more optimal; denotes the rejected answer, i.e., the output that is not selected; denotes the logistic function, which in this case can be a sigmoid function; denotes the reward or preference score associated with the selected answer and the rejected answer.

[0148] where, with respect to the above formula :

[0149] which describes how to optimize the model distribution by directly using the preference score or reward to compare the selected answer and the rejected answer, so as to adjust the model to generate output that better meets the user's preferences, gradually approaching the ideal model state .

[0150] In this embodiment, the principle of the third low-rank adaptive module is the same as that of the first and second low-rank adaptive modules described above, i.e., a third branch is added to the first large language model after fine-tuning of the instructions to simplify the parameter adjustment of the model, so the specific process is not described again here.

[0151] In this embodiment, the Direct Preference Optimization (DPO) algorithm training process includes the following features: during the training process, the DPO algorithm learns the user's or the model's preferences for different output results based on the generated positive and negative feedback data, and directly optimizes the model's preference response to low-resource language tasks; the DPO algorithm gradually enhances the model's response ability to low-resource languages in different tasks by using the model's preference feedback mechanism, especially for complex instruction tasks, ensuring that the model can output the optimal response result first; during the DPO algorithm optimization process, combined with the policy gradient algorithm of reinforcement learning, the model is iteratively trained to gradually improve its understanding and generation capabilities for low-resource languages in the continuous feedback learning process; through the optimization of the DPO algorithm, the instruction execution accuracy and task processing efficiency of the large language model in the low-resource language scenario are further improved, especially when switching between cross-language tasks, ensuring that the model has high robustness and accuracy in semantic conversion between different languages.

[0152] The method for training a large language model for a low-resource language provided by the embodiments of the present disclosure expands the training data set of the low-resource language by fusing low-resource data from multiple sources, solves the problem of data scarcity, and further optimizes the performance of the model in a low-resource language environment by optimizing the modeling technology for the low-resource language, so that the model can more flexibly adapt to different language characteristics and structures, and the operability and accuracy in practical applications are enhanced. Through this method, efficient training of a large language model in a low-resource language environment can be achieved, and the popularization and application of natural language processing technology in more languages are promoted, which has important practical significance and wide application prospects.

[0153] Figure 6 The structure block diagram of the device for training a large language model for a low-resource language according to the embodiments of the present disclosure is schematically shown.

[0154] As shown in Figure 6 The device for training a large language model for a low-resource language 600 according to the embodiments of the present disclosure includes a first obtaining module 610, a vocabulary expansion module 620, an unsupervised pre-training module 630, a second obtaining module 640, a conversion module 650, an instruction fine-tuning module 660, a model feedback data generation module 670, and a reinforcement learning module 680.

[0155] The first obtaining module 610 is configured to obtain low-resource language data and pre-process the low-resource language data to obtain low-resource language unsupervised training data.

[0156] The vocabulary expansion module 620 is configured to generate a low-resource language vocabulary according to the low-resource language unsupervised training data, and expand the vocabulary of an existing first large language model using the low-resource language vocabulary, so that the first large language model after vocabulary expansion can support the low-resource language.

[0157] The unsupervised pre-training module 630 is configured to pre-train the first large language model after vocabulary expansion according to the low-resource language unsupervised training data in combination with a first low-rank adaptive module, so that the pre-trained first large language model can adapt to the language mode and grammar characteristics of the low-resource language.

[0158] The second obtaining module 640 is configured to obtain high-resource language supervised corpus and generate low-resource language supervised corpus according to the high-resource language supervised corpus.

[0159] The conversion module 650 is configured to perform instruction fine-tuning conversion processing on the low-resource language supervised corpus to generate a low-resource language instruction fine-tuning data set.

[0160] The instruction fine-tuning module 660 is configured to fine-tune the first large language model according to the low-resource language instruction fine-tuning dataset, so that the fine-tuned first large language model can execute different task instructions in a plurality of low-resource language task environments.

[0161] The model feedback data generation module 670 is configured to input the high-resource language supervised corpus into the fine-tuned first large language model and the existing second large language model respectively, to obtain negative example data and positive example data respectively, and to generate a reinforcement learning low-resource language dataset based on model feedback according to the negative example data and the positive example data.

[0162] The reinforcement learning module 680 is configured to perform reinforcement learning training on the fine-tuned first large language model according to the reinforcement learning low-resource language dataset, to obtain a target large language model.

[0163] Any one or more of the modules, sub-modules, units, sub-units according to the embodiments of the present disclosure, or at least part of the functions of any one or more of the modules, sub-modules, units, sub-units, can be implemented in one module. Any one or more of the modules, sub-modules, units, sub-units according to the embodiments of the present disclosure can be split into a plurality of modules for implementation. Any one or more of the modules, sub-modules, units, sub-units according to the embodiments of the present disclosure can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or any other reasonable way of hardware or firmware through integration or packaging of circuits, or in any one of software, hardware and firmware or in an appropriate combination of any of them. Alternatively, one or more of the modules, sub-modules, units, sub-units according to the embodiments of the present disclosure can be at least partially implemented as computer program modules that can perform corresponding functions when executed.

[0164] For example, any multiple of the first obtaining module 610, the vocabulary expansion module 620, the unsupervised pre-training module 630, the second obtaining module 640, the conversion module 650, the instruction fine-tuning module 660, the model feedback data generation module 670, and the reinforcement learning module 680 can be combined in one module / unit / sub-unit for implementation, or any one of the modules / units / sub-units can be split into multiple modules / units / sub-units. Alternatively, at least part of the functions of one or more of the modules / units / sub-units can be combined with at least part of the functions of other modules / units / sub-units, and implemented in one module / unit / sub-unit. According to embodiments of the present disclosure, at least one of the first obtaining module 610, the vocabulary expansion module 620, the unsupervised pre-training module 630, the second obtaining module 640, the conversion module 650, the instruction fine-tuning module 660, the model feedback data generation module 670, and the reinforcement learning module 680 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on board, a system on package, an application specific integrated circuit (ASIC), or any other reasonable manner that can integrate or package a circuit, etc. hardware or firmware, or in any one of software, hardware, and firmware implementation or in a proper combination of any of them. Alternatively, at least one of the first obtaining module 610, the vocabulary expansion module 620, the unsupervised pre-training module 630, the second obtaining module 640, the conversion module 650, the instruction fine-tuning module 660, the model feedback data generation module 670, and the reinforcement learning module 680 can be at least partially implemented as a computer program module that can perform corresponding functions when the computer program module is run.

[0165] It should be noted that the part of the device for training a large language model based on a low-resource language in the embodiments of the present disclosure corresponds to the part of the method for training a large language model based on a low-resource language in the embodiments of the present disclosure, and the description of the part of the device for training a large language model based on a low-resource language is specifically referred to the part of the method for training a large language model based on a low-resource language, which will not be repeated here.

[0166] Figure 7 An illustrative structural block diagram of an electronic device suitable for implementing the method for training a large language model for a low-resource language according to an embodiment of the present disclosure is shown. Figure 7 The electronic device shown is merely an example and should not impose any limitation on the functions and use range of the embodiments of the present disclosure.

[0167] As Figure 7As shown, the electronic device 700 according to an embodiment of the present disclosure includes a processor 701 which can perform various appropriate actions and processes according to a program stored in a read only memory (ROM) 702 or a program loaded into a random access memory (RAM) 703 from a storage section 708. The processor 701 can include, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor, and / or a related chipset, and / or a special purpose microprocessor (e.g., an application specific integrated circuit (ASIC)), and so on. The processor 701 can also include an on-board memory for cache use. The processor 701 can include a single processing unit or multiple processing units to perform the different actions of the method processes according to embodiments of the present disclosure.

[0168] In the RAM 703, various programs and data required for the operation of the electronic device 700 are stored. The processor 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. The processor 701 performs various operations of the method processes according to embodiments of the present disclosure by executing the programs in the ROM 702 and / or the RAM 703. Note that the programs can also be stored in one or more memories other than the ROM 702 and the RAM 703. The processor 701 can also perform various operations of the method processes according to embodiments of the present disclosure by executing the programs stored in the one or more memories.

[0169] According to an embodiment of the present disclosure, the electronic device 700 can further include an input / output (I / O) interface 705 which is also connected to the bus 704. The electronic device 700 can further include one or more of the following components connected to the input / output (I / O) interface 705: an input section 706 including a keyboard, a mouse, etc.; an output section 707 including a display such as a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN card, a modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the input / output (I / O) interface 705 as necessary. A removable medium 711 such as a magnetic disk, a magneto-optical disk, a semiconductor memory, etc. is attached to the drive 710 as necessary, so that a computer program read out therefrom is installed into the storage section 708 as necessary.

[0170] According to an embodiment of the present disclosure, the method flow according to the embodiments of the present disclosure can be implemented as a computer software program. For example, the embodiments of the present disclosure include a computer program product comprising a computer program carrying out the program codes for executing the method shown in the flow chart. In such an embodiment, the computer program can be downloaded and installed from a network through the communication part 709, and / or installed from the detachable medium 711. When the computer program is executed by the processor 701, the above-mentioned functions defined in the system / apparatus according to the embodiments of the present disclosure are executed. According to an embodiment of the present disclosure, the system, device, apparatus, module, unit, etc. described above can be implemented by computer program modules.

[0171] The present disclosure also provides a computer readable storage medium, which can be included in the device / apparatus / system described in the above embodiments, or exist separately without being assembled into the device / apparatus / system. The above computer readable storage medium carries one or more programs, which when executed, implement the method according to the embodiments of the present disclosure.

[0172] According to an embodiment of the present disclosure, the computer readable storage medium can be a non-volatile computer readable storage medium. For example, it can include but not limited to portable computer diskette, hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any appropriate combination thereof. In the present disclosure, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus or device.

[0173] For example, according to an embodiment of the present disclosure, the computer readable storage medium can include the ROM 702 and / or the RAM 703 described above and / or one or more memories other than the ROM 702 and the RAM 703.

[0174] The embodiments of the present disclosure also include a computer program product comprising a computer program containing program codes for executing the method provided by the embodiments of the present disclosure, which program codes are used to make the electronic device implement the method provided by the embodiments of the present disclosure when the computer program product is running on the electronic device.

[0175] When the computer program is executed by the processor 701, the above-mentioned functions defined in the system / apparatus according to the embodiments of the present disclosure are executed. According to an embodiment of the present disclosure, the system, device, apparatus, module, unit, etc. described above can be implemented by computer program modules.

[0176] In one embodiment, the computer program can be tangibly embodied in a non-transitory computer readable medium, such as the optical storage device, the magnetic storage device, etc. In another embodiment, the computer program can be tangibly embodied in a signal, such as a download singal, and be distributed over the networks, such as the Internet, and be downloaded into the computer system 700 via the communication interface 709, and / or the removable storage device 711. The computer program embodied in the computer program can be transmitted in any suitable form including, but not limited to, radio frequency, electrical, magnetic, optical, electromagnetic, infrared or other suitable transmission mediums. The computer program embodied in the computer program may, for example, be transmitted over communication network 713, including the Internet, using transmission mediums and transmission techniques.

[0177] According to an embodiment of the present disclosure, program code of a computer program for performing the methods provided by the embodiments of the present disclosure can be written in any combination of one or more programming languages, including a high-level procedural or object-oriented programming language, and / or an assembly or machine language. Programming languages include, but are not limited to, Java, C++, python, "C" language, or the like. Program code can execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on a remote computing device, or entirely on the remote computing device or server. In the latter scenario, the remote computing device can be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computing device, such as through the Internet using an Internet Service Provider. The embodiments of the present disclosure are not limited to any particular programming language or programming technique.

[0178] The flow diagrams and the block diagrams in the drawings are illustrations of architectures, functional processes, and operations that can be implemented in computer programs, systems, and methods in accordance with various embodiments of the present disclosure. In this regard, each block in the flow diagrams or block diagrams can represent a module, a segment, or a portion of code that comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks can sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flow diagrams, and combinations thereof, can be implemented by special purpose hardware-based systems that perform the specified functions or operations, or combinations of special purpose hardware and computer instructions. The embodiments of the present disclosure can be implemented by a variety of means, including, but not limited to, software, firmware, hardware, or a combination thereof. The disclosure is not limited to any particular combination of means. The various embodiments of the present disclosure can be implemented in software and / or firmware, for example, stored in one or more memory devices and executed on or by one or more computer processors that preferably carry out the operations of the various embodiments of the present disclosure. The software and / or firmware can be initially stored on a removable storage medium, which can be a floppy diskette, a CD-ROM, a DVD, a Blu-ray Disc, a memory stick, a memory card, a USB drive, an SD card, a flash drive, or a solid state drive, among others. The software and / or firmware can also be stored at a remote location from the computer processor, such as a file server on a local area network or a wide area network, cloud storage, or a server computer. The software and / or firmware can be loaded into the computer processor from the removable storage medium or the remote location, or both. The software and / or firmware can comprise computer program code that, when executed, carries out the methods provided by the various embodiments of the present disclosure.

[0179] The above described embodiments of the present disclosure. However, these embodiments are merely for illustrative purposes, and are not intended to limit the scope of the present disclosure. Although each embodiment is described above separately, this does not mean that the measures in each embodiment cannot be used advantageously in combination. Without departing from the scope of the present disclosure, those skilled in the art can make various substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A method for training a large language model for low-resource languages, the method comprising: The method comprises: acquiring low-resource language data, and preprocessing the low-resource language data to obtain low-resource language unsupervised training data; generating a low-resource language vocabulary according to the low-resource language unsupervised training data, and using the low-resource language vocabulary to expand the vocabulary of an existing first large language model, so that the first large language model after vocabulary expansion can support the low-resource language; unsupervised pre-training the first large language model after vocabulary expansion according to the low-resource language unsupervised training data and in combination with a first low-rank adaptive module, so that the pre-trained first large language model can adapt to the language mode and grammar characteristics of the low-resource language; acquiring high-resource language supervised corpus, and generating low-resource language supervised corpus according to the high-resource language supervised corpus; performing instruction fine-tuning conversion processing on the low-resource language supervised corpus to generate a low-resource language instruction fine-tuning data set; performing instruction fine-tuning on the pre-trained first large language model according to the low-resource language instruction fine-tuning data set and in combination with a second low-rank adaptive module, so that the first large language model after instruction fine-tuning can execute different task instructions in a variety of low-resource language task environments; inputting the high-resource language supervised corpus into the first large language model after instruction fine-tuning and an existing second large language model respectively to obtain negative example data and positive example data respectively, and generating a reinforcement learning low-resource language data set based on model feedback according to the negative example data and the positive example data; performing reinforcement learning training on the first large language model after instruction fine-tuning according to the reinforcement learning low-resource language data set and in combination with a third low-rank adaptive module to obtain a target large language model.

2. The method of claim 1, wherein, The preprocessing of the low-resource language data to obtain low-resource language unsupervised training data comprises: noise removal processing of the low-resource language data; format standardization processing of the low-resource language data after noise removal processing; multilingual data filtering processing of the low-resource language data after format standardization processing; privacy information removal processing of the low-resource language data after multilingual data filtering processing; generating low-resource language unsupervised training data according to the low-resource language data after privacy information removal processing.

3. The method of claim 2, wherein, The vocabulary expansion of the existing first large language model using the low-resource language vocabulary comprises: merging the low-resource language vocabulary and the vocabulary of the existing first large language model to obtain a merged vocabulary; expanding the head layer of the existing first large language model according to the merged vocabulary to adapt to the expanded vocabulary, thereby realizing support for the low-resource language, wherein the head layer comprises an embedding layer and an output layer.

4. The method of claim 1, wherein, The unsupervised pre-training of the first large language model after vocabulary expansion according to the low-resource language unsupervised training data and in combination with the first low-rank adaptive module comprises: adding a first side branch to the first large language model after vocabulary expansion, wherein the first side branch is configured to simplify the parameter adjustment of the model by introducing a first low-rank matrix; The low-resource language unsupervised training data is input into the first large language model with an expanded vocabulary for unsupervised pre-training, wherein, during the pre-training process, the model initialization parameters remain unchanged, and only the parameters of the first branch are trained.

5. The method of claim 1, wherein, The high-resource language supervised corpus is obtained, and the low-resource language supervised corpus is generated according to the high-resource language supervised corpus, which includes: A high-resource language supervised corpus is obtained, and the high-resource language supervised corpus is translated into a low-resource language by using a machine translation technology. Pseudo-labeled data of a low-resource language is mined and labeled from the high-resource language supervised corpus by using a parallel sentence mining method based on self-supervised learning and a label migration method. The pseudo-labeled data is scored for confidence by using a corpus quality evaluation algorithm. The low-resource language supervised corpus is obtained by screening and removing pseudo-labeled data below a confidence score threshold.

6. The method of claim 1, wherein, The first large language model after pre-training is fine-tuned according to the low-resource language instruction fine-tuning data set in combination with a second low-rank adaptive module, which includes: A second branch is added to the first large language model after pre-training, wherein the second branch is configured to simplify the parameter adjustment of the model by introducing a second low-rank matrix. The low-resource language instruction fine-tuning data set is input into the first large language model after pre-training for instruction fine-tuning, wherein, during the instruction fine-tuning process, the model initialization parameters remain unchanged, and only the parameters of the second branch are trained.

7. The method of claim 1, wherein, The first large language model after instruction fine-tuning is trained according to the reinforcement learning low-resource language data set in combination with a third low-rank adaptive module to obtain a target large language model, which includes: The first large language model after instruction fine-tuning is trained according to the reinforcement learning low-resource language data set by using a direct preference optimization algorithm in combination with a third low-rank adaptive module to obtain a target large language model.

8. A large language model training device for a low-resource language, comprising: The device includes: A first acquisition module is configured to acquire low-resource language data and pre-process the low-resource language data to obtain low-resource language unsupervised training data. A vocabulary expansion module is configured to generate a low-resource language vocabulary based on the low-resource language unsupervised training data, and expand the vocabulary of an existing first large language model by using the low-resource language vocabulary, so that the first large language model after vocabulary expansion can support the low-resource language. An unsupervised pre-training module is configured to pre-train the first large language model after vocabulary expansion based on the low-resource language unsupervised training data in combination with a first low-rank adaptive module, so that the first large language model after pre-training can adapt to the language patterns and grammatical characteristics of the low-resource language. A second acquisition module is configured to acquire a high-resource language supervised corpus and generate a low-resource language supervised corpus based on the high-resource language supervised corpus. A conversion module is configured to perform instruction fine-tuning conversion processing on the low-resource language supervised corpus to generate a low-resource language instruction fine-tuning data set. The instruction fine-tuning module is configured to fine-tune the low-resource language instruction fine-tuning dataset, and fine-tune the pre-trained first large language model in combination with the second low-rank adaptive module, so that the instruction fine-tuned first large language model can execute different task instructions in a variety of low-resource language task environments. The model feedback data generation module is configured to input the high-resource language supervised corpus into the instruction fine-tuned first large language model and an existing second large language model respectively, to obtain negative example data and positive example data respectively, and to generate a reinforcement learning low-resource language dataset based on model feedback according to the negative example data and the positive example data. The reinforcement learning module is configured to perform reinforcement learning training on the instruction fine-tuned first large language model in combination with a third low-rank adaptive module according to the reinforcement learning low-resource language dataset, to obtain a target large language model.

9. An electronic device, comprising: comprise: one or more processors; a memory device for storing one or more programs; wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the method according to any one of claims 1-7.

10. A computer-readable storage medium having stored thereon executable instructions that cause a processor-based system to perform steps comprising: The instruction is executed by the processor to make the processor perform the method according to any one of claims 1-7.

Images