Model knowledge migration method and device based on translation thinking chain

Abstract

A model knowledge migration method based on a translation thinking chain comprises the steps that a first data set is obtained, the first data set comprises translation thinking chain data, question translation pairs of high and low resource languages and answer translation pairs, and the translation thinking chain data is used for describing the process of converting low resource language questions into low resource language answers in a cross-language mode; on the basis of the first data set, a first model is finely adjusted to obtain a second model, the first model has the translation ability of high and low resource languages, and the second model is used for processing low resource language tasks. Therefore, the model is finely adjusted by translating the thinking chain data, so that the finely-adjusted model can simulate the process of processing the low-resource language by human beings, the model can process the low-resource language by simulating a human brain to process information, and the capability of processing the low-resource language by the model is improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence (AI) technology, and in particular to a model knowledge transfer method, device and computing device cluster based on translation thinking chain. Background Technology

[0002] Among the world's many languages, both high-resource languages ​​(such as English and Chinese) and low-resource languages ​​(such as Italian and Thai) have a large number of users. However, high-resource languages, due to their relatively broad user base, have become the primary focus for training large-scale language models, resulting in particularly outstanding model performance on these languages. Although low-resource languages ​​have relatively fewer users, they also face needs related to information retrieval and natural language processing. Therefore, developing high-quality large-scale models for these languages ​​will help meet user needs and promote linguistic diversity. Thus, how to construct a large-scale model that performs well on low-resource languages ​​is a pressing technical problem that needs to be solved. Summary of the Invention

[0003] This application provides a model knowledge transfer method, device, computing device cluster, computer storage medium, and computer product based on translation thinking chain, which can construct a neural network model that performs well on low-resource languages.

[0004] In a first aspect, this application provides a model knowledge transfer method based on translation thought chain, comprising: obtaining a first dataset, the first dataset including: translation thought chain data, question translation pairs and answer translation pairs in high-resource and low-resource languages, wherein the translation thought chain data is used to describe the process of converting low-resource language questions into low-resource language answers across languages; and fine-tuning a first model based on the first dataset to obtain a second model, wherein the first model has the ability to translate between high-resource and low-resource languages, and the second model is used to handle low-resource language tasks.

[0005] Therefore, by fine-tuning the model through translation of thought chain data, the fine-tuned model can simulate the human process of processing low-resource language, thereby enabling the model to process low-resource language in a way that mimics the human brain's information processing method, thus improving the model's ability to process low-resource language.

[0006] In one possible implementation, a first model is fine-tuned based on a first dataset to obtain a second model. This includes: processing question-to-answer translation pairs according to the translation thought chain data to construct chained question-and-answer translation data; and fine-tuning the first model based on the question translation data to obtain the second model. In this way, the model fine-tuning is guided by the translation thought chain, enabling the model to learn the knowledge transfer process based on translation thinking.

[0007] In one possible implementation, before fine-tuning the first model based on the first dataset, the process includes: acquiring a second dataset, which comprises: high-resource language data, low-resource language data, and parallel pairs of high- and low-resource translations; and training a third model based on the second dataset to obtain the first model. In this way, the trained first model possesses the translation capabilities for both high- and low-resource languages.

[0008] In one possible implementation, after obtaining the second model, the method further includes: utilizing the second model to process low-resource language tasks.

[0009] Secondly, this application provides a model knowledge transfer device based on a translation thought chain, comprising: an acquisition module and a processing module. The acquisition module is used to acquire a first dataset, which includes: translation thought chain data, question-to-answer pairs in high-resource and low-resource languages, and the translation thought chain data describes the process of converting low-resource language questions into low-resource language answers across languages. The processing module is used to fine-tune a first model based on the first dataset to obtain a second model, wherein the first model possesses translation capabilities in both high- and low-resource languages, and the second model is used to handle low-resource language tasks.

[0010] In one possible implementation, when the processing module fine-tunes the first model based on the first dataset to obtain the second model, it specifically performs the following: processes question translation pairs and answer translation pairs according to the translation thought chain data to construct chained question-answer translation data; and fine-tunes the first model based on the question translation data to obtain the second model.

[0011] In one possible implementation, before the processing module fine-tunes the first model based on the first dataset, the acquisition module is also used to acquire a second dataset, which includes: high-resource language data, low-resource language data, and parallel pairs of high- and low-resource translations. The processing module is also used to train a third model based on the second dataset to obtain the first model.

[0012] In one possible implementation, after obtaining the second model, the processing module is also used to: process low-resource language tasks using the second model.

[0013] Thirdly, this application provides a computing device cluster, including at least one computing device, each computing device including a processor and a memory; the processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster performs the method described in the first aspect or any possible implementation of the first aspect.

[0014] Fourthly, this application provides a computer-readable storage medium including computer program instructions, which, when executed by a computing device, perform the method described in the first aspect or any possible implementation thereof; or, when executed by a cluster of computing devices, the cluster of computing devices performs the method described in the first aspect or any possible implementation thereof. Exemplarily, the cluster of computing devices may include one or more computing devices.

[0015] Fifthly, this application provides a computer program product containing instructions that, when executed by a computing device, cause the computing device to perform the method described in the first aspect or any possible implementation thereof; or, when executed by a cluster of computing devices, cause the cluster of computing devices to perform the method described in the first aspect or any possible implementation thereof. Exemplarily, a cluster of computing devices may include one or more computing devices.

[0016] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0017] Figure 1 This is a schematic diagram illustrating the technical concept of knowledge transfer in a high-low resource model provided in an embodiment of this application;

[0018] Figure 2 This is a schematic diagram illustrating the construction process of translation thought chain data provided in an embodiment of this application;

[0019] Figure 3 This is a flowchart illustrating a model knowledge transfer method based on translation thinking chain provided in an embodiment of this application;

[0020] Figure 4 This is a schematic diagram illustrating a comparison of model capabilities provided in an embodiment of this application;

[0021] Figure 5 This is a schematic diagram of the structure of a model knowledge transfer device based on translation thinking chain provided in an embodiment of this application;

[0022] Figure 6 This is a schematic diagram of the structure of a computing device provided in an embodiment of this application;

[0023] Figure 7 This is a schematic diagram of the structure of a computing device cluster provided in an embodiment of this application;

[0024] Figure 8 This is a schematic diagram of another computing device cluster structure provided in an embodiment of this application. Detailed Implementation

[0025] In this article, the term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The symbol " / " in this article indicates that the related objects are in an "or" relationship; for example, A / B means A or B.

[0026] The terms "first" and "second," etc., used in the specification and claims herein are used to distinguish different objects, not to describe a specific order of objects. For example, "first response message" and "second response message," etc., are used to distinguish different response messages, not to describe a specific order of response messages.

[0027] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0028] In the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more, for example, multiple processing units means two or more processing units, multiple elements means two or more elements, etc.

[0029] First, the relevant technical terms involved in the technical solution provided in this application will be introduced.

[0030] (1) Chain of thought (COT)

[0031] Mind chain is a method for designing cue words, in which the cue words contain not only the input and output of the task, but also intermediate steps of reasoning (intermediate thinking).

[0032] (2) Translation Chain of Thought (TCOT)

[0033] Translation mind chain is a method for designing language translation cue words. In addition to source and target language translation pairs, the cue words also include intermediate steps of translation reasoning.

[0034] (3) Knowledge transfer

[0035] Knowledge transfer refers to the process of successfully applying knowledge, skills, or experience gained in one domain or task to another domain or task. In this application, it specifically refers to translation knowledge transfer of language models, which means the process of transferring translation knowledge, skills, and abilities learned by a large-scale trained language model on one language pair to another language pair.

[0036] (4) High-resource languages ​​and low-resource languages

[0037] High-resource languages ​​are those with abundant written texts, translation resources, labeled data, and research literature. These languages ​​are often spoken by a large number of people or are widely used globally. Examples include English and Chinese.

[0038] Low-resource languages ​​are those that lack a large amount of written text, labeled data, and research resources. These languages ​​may be minority or regional languages ​​with a relatively small number of speakers, or limited digital resources. Examples include certain minority or tribal languages.

[0039] (5) Parallel pair data for high- and low-resource translation

[0040] In the field of machine translation, parallel pairs refer to a set of texts that correspond between two or more languages; that is, they are mutually translated versions. High-low resource translation parallel pairs refer to a set of texts that are mutually translated between high- and low-resource languages. For example, when the high-resource language is English and the low-resource language is Thai, a high-low resource translation parallel pair might be (hello, ).

[0041] The technical solution provided in this application will be described below.

[0042] Generally, when building models for low-resource languages, a model architecture of "low-resource language to high-resource language translator + high-resource language-based model + high-resource language to low-resource language translator" can be adopted. Under this architecture, when processing low-resource language issues, the low-resource language question is first translated using the low-resource language to high-resource language translator to obtain the corresponding high-resource language question. Then, the high-resource language-based model answers the high-resource language question, obtaining the high-resource language answer. Finally, the high-resource language-to-low-resource language translator translates the high-resource language answer to obtain the corresponding low-resource language answer. This approach indirectly transfers knowledge from the high-resource language by translating from low-resource language to high-resource language, outputting the result through the high-resource language model, and then translating it back to low-resource language. However, this method cannot utilize the native knowledge of the low-resource language, and due to limitations in translation capabilities, translation errors accumulate gradually during the two rounds of translation, thus affecting the knowledge transfer effect.

[0043] In view of this, this application provides a general knowledge transfer method for high- and low-resource models. In the model pre-training, high-resource language data and limited low-resource language data are introduced to enable the model to learn certain language knowledge. Then, translation thought chain data is constructed to fine-tune the model, so that the model learns to think about how to utilize the knowledge of high-resource and low-resource languages, which greatly reduces the demand for low-resource language data and makes full use of the rich knowledge of high-resource languages.

[0044] For example, Figure 1 This illustration shows a schematic diagram of a technical concept for knowledge transfer in a high-low resource model according to an embodiment of this application. Figure 1 As shown, the overall model architecture used for knowledge transfer between high and low resource models includes: a pre-training data construction module 110, a pre-training module 120, a translation thought chain construction module 130, a fine-tuning data construction module 140, and a fine-tuning module 150.

[0045] The pre-training data construction module 110 is mainly used to introduce massive amounts of high-resource language data and limited low-resource language data for preprocessing in order to construct the training data (i.e., pre-training data) required for pre-training. The pre-training data may include: high-resource language data, low-resource language data, and parallel pairs of high- and low-resource translations.

[0046] The pre-training module 120 is primarily used to pre-train a neural network model with high-resource language processing capabilities using the masked language model (MLM) method and the pre-training data constructed by the pre-training data construction module 110, thereby obtaining a pre-trained neural network model. For example, this neural network model can be, but is not limited to, built based on a transformer architecture. The ultimate goal of the pre-training module 120 is to obtain a neural network model capable of encoding the input corpus (e.g., high-resource language data or low-resource language data). As one possible implementation, the pre-training module 120 can first pre-train the neural network model with high-resource language processing capabilities based on low-resource language data, thereby modeling the low-resource language. Then, the pre-training module 120 can further perform translation (hybrid) pre-training on the neural network model based on high-resource language data and parallel translation data of both high and low resources, thereby enabling the neural network model to acquire translation capabilities for both high and low resource languages. Because low-resource language data has limited model knowledge, it requires a large amount of low-resource monolingual data to add new knowledge. Therefore, it's necessary to pre-train a neural network model with high-resource language processing capabilities on low-resource language data. The high-resource language processing model itself is already trained; after learning knowledge related to low-resource languages, it needs to be retrained using a small amount of high-resource language data to recover its original capabilities. Furthermore, training the model on parallel data can also establish knowledge alignment between high- and low-resource languages.

[0047] The Translation Mind Chain Construction Module 130 is primarily used to construct bilingual knowledge distillation question-and-answer data from raw language data, i.e., to construct translation mind chain data. This data describes the reasoning process from low-resource language questions to low-resource language responses (i.e., the process of converting low-resource language questions into low-resource language answers across languages). This reasoning process involves: translating the low-resource language question into a high-resource language question; responding to the high-resource language question to obtain a high-resource language response (i.e., a high-resource language answer); and translating the high-resource language response back into a low-resource language response. In other words, this reasoning process mainly involves mutual translation between high- and low-resource languages, as well as responses to high-resource language inputs. For example, taking the knowledge transfer model of English (high-resource) - Thai (low-resource) as an example... Figure 2The entire process of constructing translation thought chain data is demonstrated: First, the neural network model, pre-trained by module 120, translates Thai input into English; then, the neural network model responds to the translated English input based on English knowledge, obtaining an English response; next, the neural network model translates the English response back into Thai; finally, the Thai input and the subsequent thought chain (English input, English response, Thai response) are combined into a training data pair (i.e., translation thought chain data).<x=query_ti,y=("[En]-> The query_en->[response]->answer_en->[Ti]->answer_ti"> prepares for subsequent model fine-tuning, where [En], [Ti], and [response] are the language prompt and response prompt, respectively.

[0048] The fine-tuning data construction module 140 is mainly used to construct fine-tuning data. Fine-tuning data includes question-to-answer pairs and answer-to-answer pairs in high-resource and low-resource languages. Specifically, a translation model can be used to translate questions and answers in the high-resource language separately to obtain corresponding questions and answers in the low-resource language, thus constructing the fine-tuning data. For example, taking the English (high-resource) - Thai (low-resource) model knowledge transfer as an example, the form of the question-to-answer pairs can be, but is not limited to, as follows:<query_en,query_ti> The correct translation of the answer can be in the following form:<answer_en,answer_ti> .

[0049] The fine-tuning module 150 primarily uses the translation thought chain data constructed by the translation thought chain construction module 130 and the fine-tuning data constructed by the fine-tuning data construction module 140 to fine-tune the neural network model pre-trained by the pre-training module 120, thereby constructing a neural network model that performs well on low-resource languages. Furthermore, the fine-tuned neural network model can be applied to various low-resource language tasks (i.e., the fine-tuned model can be used to process low-resource language tasks), such as intelligent customer service question answering. For example, when fine-tuning the neural network model pre-trained by the pre-training module 120, low-rank adaptation (LoRA) techniques can be used, but are not limited to, to fine-tune the weights of the network layers in the neural network model, so that the fine-tuned model can be directly used for various downstream low-resource language scenario tasks.

[0050] As described above, the knowledge transfer task between high-resource and low-resource language models can be abstracted into two steps: pre-training and fine-tuning. The pre-training process primarily involves training a neural network model capable of understanding both high-resource and low-resource language knowledge using massive amounts of high-resource data and limited low-resource data, laying the foundation for the next step. Fine-tuning involves constructing question-translation pairs, answer-translation pairs, and thought chain data to fine-tune the neural network model obtained in the previous step, enabling the model to learn the knowledge transfer process based on translation thinking.

[0051] The following describes the specific implementation process of the above technical concept.

[0052] For example, Figure 3 This illustration shows a flowchart of a model knowledge transfer method based on a translation-based thought chain, as provided in an embodiment of this application. It is understood that this method can be executed by any device, equipment, platform, or cluster of devices with computing and processing capabilities. For example, this method can be executed by a model knowledge transfer device based on a translation-based thought chain, wherein the device can be implemented by software and / or hardware, and can be configured in, but is not limited to, electronic devices or servers; typically, it can be configured on a cloud computing platform. Figure 3 As shown, this model-based knowledge transfer method based on translation thinking chains may include the following steps:

[0053] S301. Obtain the first dataset, which includes: translation mind chain data, question translation pairs and answer translation pairs in high- and low-resource languages. The translation mind chain data is used to describe the process of converting low-resource language questions into low-resource language answers across languages.

[0054] In this embodiment, translation thinking data can be constructed during knowledge transfer. This translation thinking chain data describes the process of converting a low-resource language problem into a low-resource language answer across languages. This process may include: first translating the low-resource language problem into a high-resource language problem, then reasoning about the high-resource language problem to obtain a high-resource language answer, and finally translating the high-resource language answer back into a low-resource language answer. For example, the process of constructing the translation thinking chain data can be found in the aforementioned description of the translation thinking chain construction module 130, and will not be repeated here. Furthermore, a translation model can be used to translate the high-resource language problem and answer separately to obtain the corresponding low-resource language problem and answer, thus constructing high-low resource language problem-to-translation pairs and answer-to-translation pairs. Of course, a translation model can also be used to translate the low-resource language problem and answer separately to obtain the corresponding high-resource language problem and answer; the specific method depends on the actual situation and is not limited here.

[0055] S302. Based on the first dataset, fine-tune the first model to obtain the second model, wherein the first model has the ability to translate high- and low-resource languages.

[0056] In this embodiment, after obtaining the first dataset, the first model with high- and low-resource language translation capabilities can be fine-tuned using the first dataset to obtain the second model. This second model is primarily used to handle various downstream low-resource language scenario tasks. During the fine-tuning process, the first model can process question-and-answer translation pairs according to the process described in the translation thinking data, enabling the first model to simulate the human process of processing low-resource languages. This allows the fine-tuned second model to learn the knowledge transfer process based on translation thinking. In some embodiments, question-and-answer translation pairs can be processed first according to the translation thinking chain data to construct chained question-and-answer translation data. For example, taking the knowledge transfer from the English (high-resource) to Thai (low-resource) model as an example, the constructed question translation data can be... Introduce yourself->I am a LLM-> After constructing the chained question translation data, the first model can be fine-tuned using this data, for example, by sampling LoRA techniques, to obtain the second model. In this way, by constructing chained question-and-answer translation data pairs through the translation thought chain for model fine-tuning, the model can implicitly perform language translation and knowledge transfer. Furthermore, the superior question translation data includes: high-resource language source sentences, low-resource language translated sentences, low-resource language response sentences, and high-resource language response translated sentences. Therefore, during fine-tuning, the model can learn information from each sequence (sentence), thereby achieving cross-sequence sequence learning. This allows the fine-tuned model to not only learn sentences and semantic features in a single text sequence but also model cross-sequence dependencies. In this embodiment, the first model needs to have high- and low-resource language translation capabilities so that during fine-tuning, the first model can translate low-resource language questions into high-resource language questions and high-resource language answers into low-resource language answers. The third model can be trained using high-resource language data, low-resource language data, and parallel pairs of high- and low-resource translation data to obtain the first model. For example, the data used to train the third model can also be referred to as the "second dataset". Furthermore, the third model can be a neural network model with high-resource language processing capabilities. For the process of obtaining the first model, please refer to the aforementioned description of the pre-training module 120, which will not be repeated here.

[0057] Therefore, by fine-tuning the model through translation of thought chain data, the fine-tuned model can simulate the human process of processing low-resource language, thereby enabling the model to process low-resource language in a way that mimics the human brain's information processing method, thus improving the model's ability to process low-resource language.

[0058] The effects of the solutions provided in the embodiments of this application will be described below.

[0059] like Figure 4 As shown, Figure 4 The paper presents a performance comparison between the neural network models obtained using the methods provided in this application and open-source and closed-source models (i.e., models that are not open-source). ChatGPT and GPT4 are closed-source models, while LlaMA2-Chat(EN), PolyLM, X-LlaMA, Typhoon, PLUG, and NLLB-bridge are open-source models. Figure 4 The comparison between the neural network model (Ours) obtained using the method provided in this application and models obtained using other methods in various aspects of single-turn and multi-turn dialogue tasks shows that the neural network model obtained using the method provided in this application outperforms the open-source model and approaches the performance of the closed-source model in all aspects of single-turn and multi-turn dialogue tasks. This demonstrates that the neural network model obtained using the method provided in this application significantly improves the ability to handle low-resource languages.

[0060] It is understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. In addition, the various embodiments or technical features involved in the embodiments described above can be combined according to the actual situation, and the combined solution is still within the protection scope of this application.

[0061] Based on the methods in the above embodiments, this application also provides a model knowledge transfer device based on translation thinking chain.

[0062] For example, Figure 5 This illustration shows a schematic diagram of a model knowledge transfer device based on a translation thought chain, provided in an embodiment of this application. Figure 5As shown, the model knowledge transfer device 500 based on translation thought chain includes: an acquisition module 501 and a processing module 502. The acquisition module 501 is used to acquire a first dataset, which includes: translation thought chain data, question-to-answer pairs in high-resource and low-resource languages, and translation thought chain data describing the process of converting low-resource language questions into low-resource language answers across languages. The processing module 502 is used to fine-tune a first model based on the first dataset to obtain a second model, wherein the first model has the ability to translate between high-resource and low-resource languages, and the second model is used to handle low-resource language tasks.

[0063] In some embodiments, when the processing module 502 fine-tunes the first model based on the first dataset to obtain the second model, it is specifically used to: process the question translation pairs and answer translation pairs according to the translation thought chain data to construct chained question-answer translation data; and fine-tune the first model based on the question translation data to obtain the second model.

[0064] In some embodiments, before the processing module 502 fine-tunes the first model based on the first dataset, the acquisition module 501 is further configured to acquire a second dataset, which includes: high-resource language data, low-resource language data, and parallel pairs of high- and low-resource translations. The processing module 502 is further configured to train a third model based on the second dataset to obtain the first model.

[0065] In some embodiments, after obtaining the second model, the processing module 502 is further configured to: process low-resource language tasks using the second model.

[0066] In some embodiments, Figure 5 Both the acquisition module 501 and the processing module 502 shown can be implemented in software or in hardware. For example, the implementation of the acquisition module 501 will be described below. Similarly, the implementation of the processing module 502 can refer to the implementation of the acquisition module 501.

[0067] As an example of a software functional unit, module 501 may include code running on a computing instance. The computing instance may include at least one of a physical host (computing device), a virtual machine, or a container. Further, the aforementioned computing instance may be one or more. For example, module 501 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed within the same region or in different regions. Further, the multiple hosts / virtual machines / containers used to run the code may be distributed within the same availability zone (AZ) or in different AZs, each AZ including one or more geographically proximate data centers. Typically, a region may include multiple AZs.

[0068] Similarly, multiple hosts / virtual machines / containers used to run this code can be distributed within the same Virtual Private Cloud (VPC) or across multiple VPCs. Typically, a VPC is set up within a region. Communication between two VPCs within the same region, as well as between VPCs in different regions, requires a communication gateway to be set up within each VPC to enable interconnection between VPCs.

[0069] As an example of a hardware functional unit, the acquisition module 501 may include at least one computing device, such as a server. Alternatively, the acquisition module 501 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be implemented using a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), generic array logic (GAL), or any combination thereof.

[0070] The multiple computing devices included in the acquisition module 501 can be distributed in the same region or in different regions. Similarly, the multiple computing devices included in the acquisition module 501 can be distributed in the same Availability Zone (AZ) or in different AZs. Likewise, the multiple computing devices included in the acquisition module 501 can be distributed in the same Virtual Private Cloud (VPC) or in multiple VPCs. These multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0071] It should be noted that, in other embodiments, the acquisition module 501 can be used to execute any step in the model knowledge transfer method based on translation thinking chain described in the above embodiments, and the processing module 502 can be used to execute any step in the model knowledge transfer method based on translation thinking chain described in the above embodiments. Furthermore, the steps implemented by the acquisition module 501 and the processing module 502 can be specified as needed, and different steps in the model knowledge transfer method based on translation thinking chain described in the above embodiments can be implemented by the acquisition module 501 and the processing module 502 respectively. Figure 5 The diagram shows all the functions of the model knowledge transfer device 900 based on translation thinking chain.

[0072] This application also provides a computing device 600. For example... Figure 6 As shown, the computing device 600 includes a bus 602, a processor 604, a memory 606, and a communication interface 608. The processor 604, the memory 606, and the communication interface 608 communicate with each other via the bus 602. The computing device 600 can be a server or an electronic device. It should be understood that this application does not limit the number of processors and memories in the computing device 600.

[0073] Bus 602 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be divided into address buses, data buses, control buses, etc. For ease of representation, Figure 6 The bus 604 is represented by a single line, but this does not mean that there is only one bus or one type of bus. The bus 604 may include a path for transmitting information between various components of the computing device 600 (e.g., memory 606, processor 604, communication interface 608).

[0074] Processor 604 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0075] Memory 606 may include volatile memory, such as random access memory (RAM). Processor 604 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0076] The memory 606 stores executable program code, and the processor 604 executes the executable program code to implement the aforementioned functions. Figure 5 The functions of the acquisition module 501 and processing module 502 shown are configured to implement the model knowledge transfer method based on translation thought chain described in the above embodiments. That is, the memory 606 stores instructions for executing the model knowledge transfer method based on translation thought chain described in the above embodiments.

[0077] Alternatively, the memory 606 stores executable code, and the processor 604 executes the executable code to implement the aforementioned functions respectively. Figure 5 The model knowledge transfer device 900 based on translation thinking chain shown in the figure functions to realize the model knowledge transfer method based on translation thinking chain described in the above embodiments. That is, the memory 606 stores instructions for executing the model knowledge transfer method based on translation thinking chain described in the above embodiments.

[0078] The communication interface 603 uses transceiver modules, such as, but not limited to, network interface cards and transceivers, to enable communication between the computing device 600 and other devices or communication networks.

[0079] This application also provides a computing device cluster. The computing device cluster includes at least one computing device. This computing device can be a server, such as a central server, an edge server, or a local server in a local data center.

[0080] like Figure 7As shown, the computing device cluster includes at least one computing device 600. The memory 606 of one or more computing devices 600 in the computing device cluster may store the same instructions for executing the translation-based thought chain-based model knowledge transfer method described in the above embodiments.

[0081] In some possible implementations, the memory 606 of one or more computing devices 600 in the computing device cluster may also store partial instructions for executing the translation-based thought chain-based model knowledge transfer method described in the above embodiments. In other words, a combination of one or more computing devices 600 can jointly execute instructions for executing the translation-based thought chain-based model knowledge transfer method described in the above embodiments.

[0082] It should be noted that the memory 606 in different computing devices 600 within the computing device cluster can store different instructions, each used to execute the aforementioned instructions. Figure 5 The diagram illustrates some functions of the translation-based thought chain-based model knowledge transfer device 900. Specifically, the instructions stored in the memory 606 of different computing devices 600 can implement the functions of one or more modules in the acquisition module 501 and processing module 502.

[0083] In some possible implementations, one or more computing devices in a computing device cluster can be connected via a network. This network can be a wide area network (WAN) or a local area network (LAN), etc. Figure 8 One possible implementation is shown. For example... Figure 8 As shown, two computing devices 600A and 600B are connected via a network. Specifically, they are connected to the network through communication interfaces in each computing device. In this possible implementation, the memory 606 in computing device 600A stores instructions for executing the functions of the acquisition module 501. Simultaneously, the memory 606 in computing device 600B stores instructions for executing the functions of the processing module 502.

[0084] It should be understood that Figure 8 The functions of computing device 600A shown can also be performed by multiple computing devices 600. Similarly, the functions of computing device 600B can also be performed by multiple computing devices 600.

[0085] This application also provides another computing device cluster. The connection relationships between the computing devices in this computing device cluster can be similarly referred to... Figure 7 and Figure 8 The connection method of the computing device cluster is different in that the memory 606 of one or more computing devices 600 in the computing device cluster can store the same instructions for executing the methods in the above embodiments.

[0086] In some possible implementations, the memory 606 of one or more computing devices 600 in the computing device cluster may also store partial instructions for executing the aforementioned methods. In other words, a combination of one or more computing devices 600 can jointly execute the instructions for executing the aforementioned methods.

[0087] Based on the methods in the above embodiments, this application provides a computer-readable storage medium including computer program instructions. When the computer program instructions are executed by a computing device, the computing device performs the methods in the above embodiments; or, when the computer program instructions are executed by a cluster of computing devices, the cluster of computing devices performs the methods in the above embodiments. Exemplarily, the computer-readable storage medium can be any available medium that the computing device can store, or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive), etc.

[0088] Based on the methods in the above embodiments, this application provides a computer program product containing instructions that, when executed by a computing device, cause the computing device to perform the methods in the above embodiments, or, when executed by a cluster of computing devices, cause the cluster of computing devices to perform the methods in the above embodiments.

[0089] It is understood that the processor in the embodiments of this application can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A general-purpose processor can be a microprocessor or any conventional processor.

[0090] The method steps in the embodiments of this application can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can reside in an ASIC.

[0091] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, cloud computing platform, or data center to another website, computer, cloud computing platform, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a cloud computing platform or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0092] It is understood that the various numerical designations used in the embodiments of this application are merely for descriptive convenience and are not intended to limit the scope of the embodiments of this application.

[0093] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of this application.

Claims

1. A model-based knowledge transfer method based on translation thinking chains, characterized in that, The method includes: Obtain a first dataset, which includes: translation mind chain data, question translation pairs and answer translation pairs in high- and low-resource languages, wherein the translation mind chain data is used to describe the process of converting low-resource language questions into low-resource language answers across languages; Based on the first dataset, the first model is fine-tuned to obtain the second model, wherein the first model has the ability to translate both high- and low-resource languages, and the second model is used to handle low-resource language tasks.

2. The method according to claim 1, characterized in that, The step of fine-tuning the first model based on the first dataset to obtain the second model includes: Based on the translation thought chain data, the question translation pairs and the answer translation pairs are processed to construct chain-like question-and-answer translation data; Based on the translation data of the problem, the first model is fine-tuned to obtain the second model.

3. The method according to claim 1 or 2, characterized in that, Before fine-tuning the first model based on the first dataset, the method further includes: Obtain a second dataset, which includes: high-resource language data, low-resource language data, and parallel pairs of high- and low-resource translations; The third model is trained based on the second dataset to obtain the first model.

4. The method according to any one of claims 1-3, characterized in that, After obtaining the second model, the following is also included: The second model is used to process low-resource language tasks.

5. A model knowledge transfer device based on translation thinking chain, characterized in that, include: The acquisition module is used to acquire a first dataset, which includes: translation mind chain data, question translation pairs and answer translation pairs in high- and low-resource languages, wherein the translation mind chain data is used to describe the process of converting low-resource language questions into low-resource language answers across languages; The processing module is used to fine-tune the first model based on the first dataset to obtain a second model, wherein the first model has the ability to translate high- and low-resource languages, and the second model is used to process low-resource language tasks.

6. The apparatus according to claim 5, characterized in that, When the processing module fine-tunes the first model based on the first dataset to obtain the second model, it is specifically used for: Based on the translation thought chain data, the question translation pairs and the answer translation pairs are processed to construct chain-like question-and-answer translation data; Based on the translation data of the problem, the first model is fine-tuned to obtain the second model.

7. The apparatus according to claim 5 or 6, characterized in that, Before the processing module fine-tunes the first model based on the first dataset, the acquisition module is further configured to acquire a second dataset, which includes: high-resource language data, low-resource language data, and parallel pair data of high- and low-resource translations; The processing module is further configured to train the third model based on the second dataset to obtain the first model.

8. A computing device cluster, characterized in that, It includes at least one computing device, each computing device including a processor and memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device to cause the cluster of computing devices to perform the method as described in any one of claims 1-4.

9. A computer-readable storage medium, characterized in that, The method includes computer program instructions that, when executed by a cluster of computing devices, cause the cluster of computing devices to perform the method as described in any one of claims 1-4, wherein the cluster of computing devices includes at least one computing device.

10. A computer program product containing instructions, characterized in that, When the instruction is executed by the computing device cluster, the computing device cluster causes the computing device cluster to perform the method as described in any one of claims 1-4, wherein the computing device cluster includes at least one computing device.

Images