Information processing methods, apparatus, electronic devices, and storage media

Abstract

To provide an information processing method, an apparatus, an electronic device, and a storage medium for improving translation quality for a first language sentence, and obtaining a second bilingual sentence pair that is more accurate in translation.SOLUTION: An information processing method using machine learning includes: obtaining a first bilingual sentence pair including a source language sentence and a target language sentence; and obtaining a distilled second bilingual sentence pair by distilling a first language sentence in the first bilingual sentence pair based on a large language model. The first language sentence is the source language sentence or the target language sentence.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, specifically to the fields of machine translation, deep learning, and large language models, and particularly to an information processing method, apparatus, electronic device, and storage medium.

Background Art

[0002] Machine translation is a discipline that uses computers to perform human language translation and is a core technology for breaking down language barriers. Currently, neural network machine translation is the mainstream technology, and neural network machine translation has significantly improved in the quality of translation compared to conventional machine translation methods.

[0003] Currently, large language models can strongly express the abilities of understanding, generation, memory, and reasoning and also achieve excellent effects in cross-language task machine translation. However, large language models also face the challenge of having a large number of parameters and high computing power requirements, and for machine translation technology, the cost of directly using large language models is high.

Summary of the Invention

Problems to be Solved by the Invention

[0004] The present disclosure provides an information processing method, apparatus, electronic device, and storage medium.

Means for Solving the Problems

[0005] According to one aspect of the present disclosure, an information processing method is provided, an acquisition step of acquiring a first bilingual sentence pair including a source language sentence and a target language sentence, a step of distilling a first language sentence in the first bilingual sentence pair based on a large language model to obtain a second bilingual sentence pair after distillation, wherein the first language sentence is the source language sentence or the target language sentence.

[0006] According to another aspect of this disclosure, an information processing device is provided, A retrieval module for obtaining a first bilingual sentence pair containing a source language sentence and a target language sentence, A distillation module for distilling a first language sentence in a first bilingual sentence pair based on a large-scale language model and obtaining a second bilingual sentence pair after distillation, wherein the first language sentence is the source language sentence or the target language sentence of the distillation module.

[0007] According to a third aspect of the present disclosure, an electronic device is provided, comprising at least one processor and a memory communicably connected to the at least one processor, the memory storing instructions executable by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor performs the method according to the first aspect.

[0008] A fourth aspect of the present disclosure further provides a non-temporary computer-readable storage medium in which computer instructions are stored, the computer instructions being used to cause the computer to perform the method described in the first aspect.

[0009] A fifth aspect of this disclosure provides a computer program which, when executed by a processor, implements the steps of the method described in the first aspect. [Effects of the Invention]

[0010] In the embodiments of this disclosure, a large-scale language model is used to distill the first language sentence in the first bilingual sentence pair, improving the translation effect on the first language sentence, thereby obtaining a second bilingual sentence pair with a more accurate translation, resulting in higher translation quality compared to conventional machine translation.

[0011] The contents described in this section are not intended to identify any essential or important features of the embodiments of this disclosure, nor to limit the scope of this disclosure. Other features of this disclosure will be readily apparent through the following description. [Brief explanation of the drawing]

[0012] The drawings are provided to better understand the solution and do not limit this disclosure. [Figure 1] This is a schematic diagram of the information processing method provided in the embodiments of this disclosure. [Figure 2] This is a schematic diagram of another information processing method provided in the embodiments of this disclosure. [Figure 3] This is a schematic diagram of another information processing method provided in the embodiments of this disclosure. [Figure 4] This is a schematic diagram of another information processing method provided in the embodiments of this disclosure. [Figure 5] This is a schematic diagram of another information processing method provided in the embodiments of this disclosure. [Figure 6] This is a schematic diagram of another information processing device provided in the embodiments of this disclosure. [Figure 7] This is a block diagram of an electronic device for realizing the information processing method of the embodiments of this disclosure. [Modes for carrying out the invention]

[0013] Illustrative embodiments of the present disclosure are described below with reference to the drawings, and for the sake of ease of understanding, these include various details of the embodiments of the present disclosure, which should be considered merely illustrative. Accordingly, as will be aware to those skilled in the art, various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for clarity and conciseness, well-known descriptions of functions and structures are omitted below.

[0014] Artificial Intelligence, abbreviated as AI in English, is a new technical science that researches and develops theories, methods, technologies, and application systems for simulating, exploring, and expanding human intelligence. It can simulate the information processes of human consciousness and thinking. The main purpose of artificial intelligence research is for machines to undertake complex tasks that can only be completed when they have the intelligence of ordinary humans.

[0015] Machine Translation, also known as automatic translation, is a process of using a computer to convert one natural language (source language) into another natural language (target language). The development of machine translation technology has always been closely related to the development of disciplines such as computer technology, information theory, and linguistics.

[0016] Deep Learning is a research direction of information in the field of machine learning. It learns the inherent rules and representation hierarchies of sample data. The information obtained in the learning process is of great help for the interpretation of data such as text, images, and voices. The ultimate goal is for machines to have the ability of analytical learning like humans and be able to recognize data such as text, images, and voices.

[0017] Large Language Model (LLM) is an artificial intelligence model aimed at understanding and generating language. It can execute a wide range of tasks, including text summarization, translation, and sentiment analysis. Its characteristic is that it has an extremely large scale, including tens of billions to trillions of parameters, and learns complex patterns in language.

[0018] Figure 1 is a schematic diagram of the information processing method provided by an embodiment of the present disclosure. As shown in Figure 1, the method includes the following.

[0019] In S101, obtain a first bilingual sentence pair.

[0020] The first bilingual sentence pair includes a source language sentence and a target language sentence.

[0021] The source language text is the text to be translated, and the target language text is the text after translation. For example, when the translation requirement is to translate a Chinese text into an English text, the source Chinese text to be translated is the source language text, and the translated English text is the target language text.

[0022] In some implementations, the first bilingual sentence pair can be read from a database or collected from a network. As can be understood, the source language may be an existing language such as Chinese, English, Korean, and Japanese, and accordingly, the target language may also be an existing language such as Chinese, English, Korean, and Japanese.

[0023] In some implementations, the source language text and the target language text in the first bilingual sentence are sentences that appear in pairs. That is, the source language text and the target language text are sentences with the same meaning expressed in different languages. For example, the source language text is A expressed in Chinese, and the target language text is A expressed in English.

[0024] In S102, distill the first language text in the first bilingual sentence pair based on a large language model, and obtain a second bilingual sentence pair after distillation.

[0025] The first language text is the source language text or the target language text.

[0026] The purpose of distillation is to extract the features and capabilities in the model and compress them into a form that can be processed with less effort. Optionally, the distillation in this embodiment may perform translation distillation or optimization distillation on the sentence, and obtain a second bilingual sentence pair with better translation quality through the two types of distillation methods.

[0027] In some implementations, translation distillation is a translation capability that translates sentences and extracts a large-scale language model. As understood, after translating the first language sentence in a first bilingual sentence pair based on the large-scale language model, the translated sentence can be obtained, and the translated sentence and the sentence to be input into the large-scale language model can be constructed into a second bilingual sentence pair.

[0028] To illustrate with an example, if the first bilingual sentence pair contains source language sentence A and target language sentence B, then source language sentence A in the first bilingual sentence pair is input into a large-scale language model and distilled to obtain the translated sentence C. At this point, the second bilingual sentence pair may contain translated sentence C and source language sentence A.

[0029] In another implementation, the distillation in this embodiment can be further optimized or embellished for the sentence; that is, optimized distillation optimizes or embellishes the first language sentence in the first bilingual sentence pair to obtain a sentence that is more accurate and smoother in expression.

[0030] To illustrate with an example, suppose the first bilingual sentence pair contains source language sentence A and target language sentence B. The target language sentence B in the first bilingual sentence is input into a large-scale language model for distillation, and an optimized and refined sentence D is obtained. At this point, the second bilingual sentence pair can contain the optimized sentence D and target language sentence B.

[0031] In this embodiment, a first bilingual sentence pair containing a source language sentence and a target language sentence is obtained, the first bilingual sentence pair is distilled by a large-scale language model, a second bilingual sentence pair is obtained after distillation by the large-scale language model, and the second bilingual sentence pair, after distillation by the large-scale language model, has better translation effect and expression, thus guaranteeing translation quality.

[0032] Figure 2 is a schematic diagram of another information processing method provided in an embodiment of the present disclosure. As shown in Figure 2, the method includes the following:

[0033] In S201, obtain the first bilingual sentence pair.

[0034] In the embodiments of this disclosure, the method for realizing step S201 can be implemented using any one of the methods in each embodiment of this disclosure, and this is not limited to this, nor is a detailed explanation provided here.

[0035] In S202, the distillation target for the first bilingual sentence pair is determined.

[0036] The distillation target is translational distillation or emulsification distillation.

[0037] In some implementations, when the distillation target is translational distillation, the main task of the large-scale language model is to accurately translate the sentence. When the distillation target is embellishment distillation, the main task of the large-scale language model is to embellish and optimize the sentence to improve its expressive effect.

[0038] In S203, if the distillation target is translational distillation, the first presented word of the large-scale language model is generated based on the distillation target and the second language sentence in the first bilingual sentence pair.

[0039] Selectively, the second language sentence may be either a source language sentence or a target language sentence. If the first language sentence is a source language sentence, the second language sentence is a target language sentence, and conversely, if the first language sentence is a target language sentence, the second language sentence is a source language sentence.

[0040] In some implementations, if the second language sentence is the source language sentence, translation distillation is determined to be target language distillation, the second language sentence becomes the target language sentence, and a first suggestion is generated that translates the target language sentence into the target language. In other words, if the second language sentence is the source language sentence, translation distillation is target language distillation, the source language sentence becomes the target language sentence, and a first suggestion is generated that translates the said source language sentence into the source language, for example, the first suggestion is "translate the source language sentence into the target language."

[0041] Selectively, if the second language sentence is the target language sentence, translation distillation is determined to be source language distillation, the second language sentence is designated as the target language sentence, and a first suggestion is generated that translates the target language sentence into the source language. In other words, if the second language sentence is the target language sentence, translation distillation is source language distillation, the target language sentence is designated as the target language sentence, and a first suggestion is generated that translates the target language sentence into the source language. For example, the first suggestion is "translate the target language sentence into the source language," and the suggestion guides the large-scale language model to output a third language sentence with better translation efficiency.

[0042] In S204, at least one language sentence is determined from the first bilingual sentence pair to be input into the large-scale language model. The first presented word and at least one language sentence from the first bilingual sentence pair are input into the large-scale language model for distillation to obtain a third language sentence corresponding to the first language sentence.

[0043] Selectively, a second language sentence can be determined from the first bilingual sentence pair to be input into a large-scale language model.

[0044] To understand this, if the first language sentence is the source language sentence, then the second language sentence is the target language sentence, translation distillation is source language distillation, inputting the target language sentence (language sentence to be translated) and the first presented word into a large-scale language model to obtain a third language sentence corresponding to the first language sentence can be understood as reconstructing the first language sentence (source language sentence), that is, retranslating the second language sentence (target language sentence) by the large-scale language model to obtain a source language sentence corresponding to the first language sentence (original source language sentence). Accordingly, if the first language sentence is the target language sentence, then the second language sentence is the source language sentence, translation distillation is target language distillation, inputting the source language sentence (language sentence to be translated) and the first presented word into a large-scale language model to obtain a third language sentence corresponding to the first language sentence, that is, retranslating the second language sentence (source language sentence) by the large-scale language model to obtain a target language sentence corresponding to the first language sentence (original target language sentence).

[0045] In S205, a second bilingual sentence pair is generated based on the second and third language sentences in the first bilingual sentence pair.

[0046] If the first language sentence is the source language sentence, the second language sentence is the target language sentence; and if the first language sentence is the target language sentence, the second language sentence is the source language sentence.

[0047] To make it clear, after inputting the first presented word and the second language sentence into a large-scale language model, a third language sentence corresponding to the first language sentence is obtained. At this point, the first and third language sentences are different representations of the same type of language. For example, the first language sentence X and the second language sentence Y are a pair of Chinese-English translations. The large-scale language model obtains the third language sentence X1 in Chinese by retranslating the second language sentence Y in English. At this point, the first language sentence X in Chinese and the third language sentence X1 in Chinese are translations of the second language sentence Y in English, and the third language sentence X1 and the second language sentence Y are constructed as a second bilingual sentence pair.

[0048] In this embodiment, when the distillation target is translation distillation, the second language sentence determines whether the translation distillation is target language distillation or source language distillation, multi-language distillation is realized, the corresponding first presented word is determined, the second language sentence and the first presented word are input into a large-scale language model, a third language sentence corresponding to the first language sentence is obtained, that is, the third language sentence after the first language sentence has been reconstructed, the third language sentence and the original second language sentence are constructed into a new second bilingual sentence pair, the quality of the translation during translation is ensured by retranslation by the large-scale language model, and the effect of poor translation quality is avoided.

[0049] Figure 3 is a schematic diagram of another information processing method provided in an embodiment of the present disclosure. As shown in Figure 3, the method includes the following:

[0050] In S301, the first bilingual sentence pair is obtained.

[0051] In the embodiments of this disclosure, the method for realizing step S301 can be implemented using any one of the methods in each embodiment of this disclosure, and this is not limited to this, nor is a detailed explanation provided here.

[0052] In S302, the distillation target for the first bilingual sentence pair is determined.

[0053] In the embodiments of this disclosure, the method for realizing step S302 can be implemented using any one of the methods in each embodiment of this disclosure, and this is not limited to this, nor is a detailed explanation provided here.

[0054] In S303, if the distillation target is embellishment distillation, a second presented word of the large-scale language model is generated based on the distillation target and the first language sentence.

[0055] Selectively, the first language sentence can be designated as the language sentence to be modified, and a second suggestion word can be generated to modify the language sentence to be modified. In other words, when the distillation target is modification distillation, the sentence needs to be modified, the first language sentence in the first bilingual sentence pair is designated as the language sentence to be modified, and a second suggestion word can be generated to modify that language sentence to be modified.

[0056] In some implementations, the first language sentence may be a source language sentence or a target language sentence. If the first language sentence is a source language sentence, the second suggestion may be "enhance the source language sentence according to the expression conventions of the source language to make it smoother." If the first language sentence is a target language sentence, the second suggestion may be "enhance the target language sentence according to the expression conventions of the target language to make it smoother." Based on the second suggestion, a large-scale language model is guided to perform the enhancement process on the first language sentence.

[0057] In S304, at least one language sentence is determined from the first bilingual sentence pair to be input into the large-scale language model, and the second presented word and at least one language sentence from the first bilingual sentence pair are input into the large-scale language model for distillation to obtain a third language sentence corresponding to the first language sentence.

[0058] By selectively inputting the first language sentence and its second presented word into a large-scale language model, an embellishment distillation can be performed to obtain the embellished sentence, which is a third language sentence corresponding to the first language sentence.

[0059] Selectively, both language sentences and their corresponding second presentation words in the first bilingual sentence pair are input into a large-scale language model. The large-scale language model then performs embellishment distillation on the first and second language sentences, respectively, to obtain the embellished sentence. In other words, both the first and second language sentences in the first bilingual sentence pair are input into the large-scale language model. The second language sentence, the first language sentence, and their corresponding second presentation words are also input into the large-scale language model. The model outputs a third language sentence corresponding to the first language sentence and a second language sentence, achieving the goal of embellishing the sentence and making it more clear and smooth.

[0060] In S305, a second bilingual sentence pair is generated based on the second and third language sentences in the first bilingual sentence pair.

[0061] As can be understood, in the first bilingual sentence pair, the first language sentence is the sentence that undergoes distillation, and the third language sentence is the sentence obtained after embellishment distillation by a large-scale language model. The expression of the third language sentence is clearer and smoother, and therefore the third language sentence and the second language sentence can be constructed into a second bilingual sentence pair. This second bilingual sentence pair is the sentence after embellishment, and its expression is clearer and smoother.

[0062] In the embodiments of this disclosure, the method for realizing step S305 can be implemented using any one of the methods in each embodiment of this disclosure, and this is not limited to this, nor is a detailed explanation provided here.

[0063] In this embodiment, when the distillation target is embellishment distillation, a third language sentence with smoother and clearer expression is obtained by performing embellishment distillation on the first language sentence. The first language sentence may be either the source language sentence or the target language sentence. Distillation embellishment of a large-scale language model improves the expression and translation quality of the sentence, thereby generating a high-quality second bilingual sentence pair based on the third and second language sentences and guaranteeing translation quality.

[0064] Figure 4 is a schematic diagram of another information processing method provided in an embodiment of the present disclosure. As shown in Figure 4, the method includes the following:

[0065] In S401, the first bilingual sentence pair is obtained.

[0066] In the embodiments of this disclosure, the method for realizing step S401 can be implemented using any one of the methods in each embodiment of this disclosure, and this is not limited to this, nor is a detailed explanation provided here.

[0067] In S402, the first language sentence in the first bilingual sentence pair is distilled based on a large-scale language model, and the second bilingual sentence pair after distillation is obtained.

[0068] In the embodiments of this disclosure, the method for realizing step S402 can be implemented using any one of the methods in each embodiment of this disclosure, and this is not limited to this, nor is a detailed explanation provided here.

[0069] In S403, the second bilingual sentence pair and the first bilingual sentence pair are integrated to generate a reinforcement corpus library, and the student model is trained based on the reinforcement corpus library.

[0070] To understand this, the first bilingual sentence pair is the original sentence pair obtained by the database, and the second bilingual sentence pair is the sentence pair with better translation quality after being distilled by a large-scale language model. The first and second bilingual sentence pairs are combined to obtain an augmentation corpus library, which contains multiple corpora, each containing source language sentences and target language sentences, i.e., multiple bilingual sentence pairs. The student model is trained using the augmentation corpus library, and the student model learns the translation capabilities of the large-scale language model, ensuring the student model's translation effectiveness for sentences.

[0071] Selectively, the student model may be a language model with a small number of parameters (compared to a large language model) or a conventional neural network model, such as a Long Short-Term Memory (LSTM) network or a Transformer. As can be understood, the teacher model in this embodiment is a large language model with a large number of parameters, and by distillation, the translation capabilities of the large language model are migrated to the student model, giving the student model the translation capabilities of the large language model while retaining the characteristic that the student model is highly efficient and easy to deploy.

[0072] In some implementations, the quality of corpora in the augmentation corpus library can be further evaluated. Based on the corpus quality evaluation information, a selection operation is performed on the corpora in the augmentation corpus library, a target corpus library is obtained, and the student model is trained based on the target augmentation corpus library. In other words, the quality of corpora in the augmentation corpus library is evaluated, corpora with better quality are selected to obtain the target corpus library, the student model is trained using the target corpus library, and the student model is obtained that retains the translation capabilities of the large-scale language model.

[0073] Selectively, the purpose of evaluating the quality of a corpus in a reinforcement corpus library is to evaluate the quality of translations within that corpus. In other words, evaluating the quality of a set of corpora characterizes the correspondence between the source language and the target language within that corpus, and the better the translation between the source language and the target language, the better the quality of the corpus.

[0074] In some implementations, the corpus quality evaluation information may be a value obtained for the quality of the corpus, with a higher quality value indicating better translation quality for that corpus. The selection process for corpora in an augmented corpus library based on quality evaluation information may involve determining corpus pairs corresponding to the same source language sentences and selecting at least one target corpus from the corpus pairs corresponding to the same source language sentences based on the quality evaluation information of each corpus in the corpus pair.

[0075] To illustrate with an example, for a source language sentence A, corpora a and corpora b corresponding to that source language sentence A are determined as the first bilingual sentence pair and the second bilingual sentence pair, respectively. Quality evaluation information is obtained for corpora a and corpora b, and the target corpus is determined based on the quality evaluation information.

[0076] By selectively comparing the quality evaluation information of each corpus in a corpus set, the corpus with the highest quality can be determined as the target corpus. Alternatively, by sorting the corpora in a corpus set based on the quality evaluation information of each corpus in the corpus set, the corpus sorted to the front can be selected as the target corpus. Another method is to compare the quality evaluation information of each corpus in a corpus set with a pre-set quality evaluation threshold, and select a corpus that is equal to or above the quality evaluation threshold as the target corpus.

[0077] To illustrate with an example, for a corpus set corresponding to source language sentence A (including corpus a and corpus b), the corpus with the highest quality among corpus a and corpus b can be selected as the target corpus. Alternatively, if the corpus set contains many corpora, the quality evaluation information of all corpora can be sorted in descending order, and the corpus sorted to the front can be selected as the target corpus. Furthermore, the quality evaluation information of each corpus in the corpus set can be compared with a pre-set quality evaluation threshold, and corpora that are above the quality evaluation threshold can be selected as the target corpus. The specific method of selecting the target corpus is not limited and should be understood as such. The purpose of selecting the target corpus is to select a corpus with better translation quality. A target corpus library constructed based on the target corpus is used as a training sample, that is, a corpus with better translation quality is used as a training sample, ensuring the training effect of the student model and allowing the student model to learn the translation capabilities of a large-scale language model.

[0078] In this embodiment, after obtaining a second pair of bilingual sentences, a target corpus with better translation quality is determined based on a comparison between the second pair and the first pair of bilingual sentences. A target corpus library is then constructed based on the target corpus to train the student model. This allows the student model to learn the translation capabilities of a large-scale language model, ensures the quality of the student model's translations by utilizing a target corpus with better translation quality, improves resource utilization, and reduces the cost of directly deploying a large-scale model.

[0079] Figure 5 is a schematic diagram of another information processing method provided in an embodiment of the present disclosure. As shown in Figure 5, the method includes the following:

[0080] In S501, the first bilingual sentence pair is obtained.

[0081] In the embodiments of this disclosure, the method for realizing step S501 can be implemented using any one of the methods in each embodiment of this disclosure, and this is not limited to this, nor is a detailed explanation provided here.

[0082] In S502, the distillation target for the first bilingual sentence pair is determined.

[0083] In the embodiments of this disclosure, the method for realizing step S502 can be implemented using any one of the methods in each embodiment of this disclosure, and this is not limited to this, nor is a detailed explanation provided here.

[0084] In S503, if the distillation target is translational distillation, the first presented word of the large-scale language model is generated based on the distillation target and the second language sentence in the first bilingual sentence pair.

[0085] In the embodiments of this disclosure, the method for realizing step S503 can be implemented using any one of the methods in each embodiment of this disclosure, and this is not limited to this, nor is a detailed explanation provided here.

[0086] In S504, at least one language sentence is determined from the first bilingual sentence pair to be input into the large-scale language model. The first presented word and at least one language sentence from the first bilingual sentence pair are input into the large-scale language model for distillation to obtain a third language sentence corresponding to the first language sentence.

[0087] In the embodiments of this disclosure, the method for realizing step S504 can be implemented using any one of the methods in each embodiment of this disclosure, and this is not limited to this, nor is a detailed explanation provided here.

[0088] In S505, if the distillation target is embellishment distillation, a second presented word of the large-scale language model is generated based on the distillation target and the first language sentence.

[0089] In the embodiments of this disclosure, the method for realizing step S505 can be implemented using any one of the methods in each embodiment of this disclosure, and this is not limited to this, nor is a detailed explanation provided here.

[0090] In S506, at least one language sentence is determined from the first bilingual sentence pair to be input into the large-scale language model, and the second presented word and at least one language sentence from the first bilingual sentence pair are input into the large-scale language model for distillation to obtain a third language sentence corresponding to the first language sentence.

[0091] In the embodiments of this disclosure, the method for realizing step S506 can be implemented using any one of the methods in each embodiment of this disclosure, and this is not limited to this, nor is a detailed explanation provided here.

[0092] In S507, a second bilingual sentence pair is generated based on the second and third language sentences in the first bilingual sentence pair.

[0093] In the embodiments of this disclosure, the method for realizing step S507 can be implemented using any one of the methods in each embodiment of this disclosure, and this is not limited to this, and the explanation is omitted here.

[0094] In S508, the second bilingual sentence pair and the first bilingual sentence pair are merged to generate a reinforcement corpus library, and the student model is trained based on the reinforcement corpus library.

[0095] In the embodiments of this disclosure, the method for realizing step S508 can be implemented using any one of the methods in each embodiment of this disclosure, and this is not limited to this, and the explanation is omitted here.

[0096] In this embodiment, a first bilingual sentence pair containing a source language sentence and a target language sentence is obtained, the first bilingual sentence pair is distilled by a large-scale language model, a second bilingual sentence pair is obtained after the distillation process of the large-scale language model, and the second bilingual sentence pair, through distillation by the large-scale language model, has better translation effect and expression, guarantees translation quality, trains a student model based on a corpus library with better translation quality, ensures the student model learns the translation capabilities of the large-scale language model, and provides better translation effect.

[0097] Figure 6 is a schematic diagram of an information processing device provided in an embodiment of the present disclosure. As shown in Figure 6, the information processing device 600 is A retrieval module 601 for obtaining a first bilingual sentence pair containing a source language sentence and a target language sentence, A distillation module 602 for distilling a first language sentence in a first bilingual sentence pair based on a large-scale language model and obtaining a second bilingual sentence pair after distillation, wherein the first language sentence includes the distillation module 602 which is either a source language sentence or a target language sentence.

[0098] In some implementations, the distillation module 602 is used. The distillation target for the first bilingual sentence pair is determined, and the distillation target is either translation distillation or embellishment distillation. A large-scale language model is used to distill a first language sentence according to the distillation target, and to obtain a second bilingual sentence pair.

[0099] In some implementations, the distillation module 602 is used. Based on the distillation target and the first bilingual sentence pair, the presented words of the large-scale language model are generated. The presented word and at least one language sentence in the first bilingual sentence pair are input into a large-scale language model for distillation, and a third language sentence corresponding to the first language sentence is obtained. Used to generate a second bilingual sentence pair based on the second and third language sentences in the first bilingual sentence pair, If the first language sentence is the source language sentence, the second language sentence is the target language sentence; and if the first language sentence is the target language sentence, the second language sentence is the source language sentence.

[0100] In some implementations, the distillation module 602 is used. Based on the distillation target, at least one language sentence is determined from the first bilingual sentence pair to be input into the large-scale language model. The presented words and at least one linguistic sentence are input into a large-scale linguistic model for distillation.

[0101] In some implementations, the distillation module 602 is used. When the distillation target is translational distillation, it is used to generate the first presented word of the large-scale language model based on the distillation target and the second language sentence in the first bilingual sentence pair.

[0102] In some implementations, the distillation module 602 is used. If the second language sentence is the source language sentence, then translation distillation is determined to be source language distillation. The second language sentence is used as the target language sentence for translation, and the first suggested words are used to generate the first suggested words that translate the target sentence into the source language.

[0103] In some implementations, the distillation module 602 is used. If the second language sentence is the target language sentence, then translation distillation is determined to be target language distillation. The second language sentence is used as the target language sentence, and the first suggested words are used to generate the first suggested words that translate the target language sentence into the target language.

[0104] In some implementations, the distillation module 602 is used. This is used to determine the second language sentence from the first bilingual sentence pair as the language sentence to be input into a large-scale language model.

[0105] In some implementations, the distillation module 602 is used. When the distillation target is embellishment distillation, it is used to generate a second presented word in a large-scale language model based on the distillation target and the first language sentence.

[0106] In some implementations, the distillation module 602 is used. It is used to generate a second set of suggested words that modify the first set of language sentences, which are designated as the target language sentences for modification.

[0107] In some implementations, the distillation module 602 is used. The first and second language sentences in the first bilingual sentence pair are used to become the language sentences input into a large-scale language model.

[0108] In some implementations, the distillation module 602 further... The second pair of bilingual sentences and the first pair of bilingual sentences are merged to generate a reinforcement corpus library, which is then used to train a student model, with each corpus containing both source language and target language sentences.

[0109] In some implementations, the distillation module 602 is used. We perform quality evaluations on corpora in augmented corpus libraries. Based on the corpus quality evaluation information, a selection operation is performed on the corpora in the augmentation corpus library, and the target corpus library is obtained. It is used to train student models based on a target reinforcement corpus library.

[0110] In some implementations, the distillation module 602 is used. Determine the corpus pairs corresponding to the same source language sentences, This is used to select at least one target corpus from the corpus set that corresponds to the same source language sentence, based on the quality evaluation information of each corpus in the corpus set.

[0111] In some implementations, the distillation module 602 is used. By comparing the quality evaluation information of each corpus in the corpus set, the corpus with the highest quality is designated as the target corpus, or Based on the quality evaluation information of each corpus in the corpus set, the corpora in the corpus set are sorted, and the corpus sorted to the front is selected as the target corpus, or This method is used to compare the quality evaluation information of each corpus in a corpus set with a pre-set quality evaluation threshold, and to select corpora that meet or exceed the quality evaluation threshold as the target corpus.

[0112] In this embodiment, a first bilingual sentence pair containing a source language sentence and a target language sentence is obtained, the first bilingual sentence pair is distilled by a large-scale language model, a second bilingual sentence pair is obtained after the distillation process of the large-scale language model, and the second bilingual sentence pair, through distillation by the large-scale language model, has better translation effect and expression, guarantees translation quality, trains a student model based on a corpus library with better translation quality, ensures the student model learns the translation capabilities of the large-scale language model, and provides better translation effect.

[0113] The acquisition, storage, and application of such user personal information in the technical solutions disclosed herein all comply with the provisions of relevant laws and regulations and do not violate public order and morals.

[0114] According to embodiments of the present disclosure, the present disclosure further provides electronic devices, readable storage media, and computer programs.

[0115] Figure 7 shows an exemplary block diagram of an exemplary electronic device 700 that can implement embodiments of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, mobile phones, smartphones, wearable devices, and other similar computing devices. The components, their connections and relationships, and their functions shown herein are merely examples and are not intended to limit the description herein and / or the implementation of the present disclosure as required.

[0116] As shown in Figure 7, the device 700 includes a computing unit 701 capable of performing various appropriate operations and processes according to a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. The RAM 703 may store various programs and data necessary for the operation of the device 700. The computing unit 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is further connected to the bus 704.

[0117] The I / O interface 705 is connected to multiple components of the device 700, including input units 706 such as a keyboard and mouse, output units 707 such as various types of displays and speakers, storage units 708 such as magnetic disks and optical disks, and communication units 709 such as a network card, modem, and wireless communication transceiver. The communication unit 709 enables the device 700 to exchange information / data with other devices via computer networks such as the Internet and / or various telecommunication networks.

[0118] The computing unit 701 may be a variety of general-purpose and / or dedicated processing components having processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, a computing unit that executes various machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs each of the methods and processes described above, for example, an information processing method. For example, in some embodiments, the information processing method may be implemented as a computer software program tangibly embedded in a mechanically readable medium such as a storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed into the device 700 via ROM 702 and / or communication unit 709. Once the computer program is loaded into RAM 703 and executed by the computing unit 701, one or more steps of the information processing method described above may be performed. Optionally, in other embodiments, the computing unit 701 may be configured to perform the information processing method in any other suitable manner (for example, by firmware).

[0119] Various embodiments of the systems and technologies described herein can be implemented as digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-chip (SOCs), load-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may be implemented within one or more computer programs that run and / or are interpreted and acquired on a programmable system including at least one programmable processor, which is a dedicated or general-purpose programmable processor, and may include receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, at least one input device, and at least one output device.

[0120] Program code for carrying out the methods of this disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a dedicated computer, or other programmable data processing device, and when the program code is executed by the processor or controller, the functions / operations specified in the flowchart and / or block diagrams will be performed. The program code may run entirely on a machine, partially on a machine, partially on a machine and partially on a remote machine as a standalone software package, or entirely on a remote machine or server.

[0121] In the context of this disclosure, a mechanically readable medium may be a tangible medium that contains or can store a program for use by, or in combination with, an instruction execution system, device, or apparatus. A mechanically readable medium may be a mechanically readable signal medium or a mechanically readable storage medium. A mechanically readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or apparatus, or any suitable combination thereof. More specific examples of mechanically readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0122] To provide user interaction, the systems and technologies described herein can be implemented on a computer having a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) and a keyboard and pointing device (e.g., a mouse or trackball) on which the user can provide input to the computer. Other types of devices can also be used to provide user interaction; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback), and input from the user may be received in any form (including voice input, speech input, or tactile input).

[0123] The systems and technologies described herein can be implemented in computing systems including backend components (e.g., as data servers), computing systems including middleware components (e.g., application servers), computing systems including frontend components (e.g., user computers having a graphical user interface or web browser that allows users to interact with embodiments of the systems and technologies described herein), or computing systems including any combination of such backend components, middleware components, or frontend components. The components of the system can be connected to one another via any form or medium of digital data communication (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0124] A computer system can include clients and servers. Clients and servers are generally geographically separated and typically interact via a communication network. The client-server relationship is generated by computer programs running on each computer that have a client-server relationship with each other. A server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0125] It should be understood that the steps can be rearranged, added, or deleted using the various forms of flows shown above. For example, each step described herein may be performed in parallel, sequentially, or in a different order, as long as the technical solution disclosed herein can achieve the desired result.

[0126] The specific embodiments described above do not limit the scope of protection of this disclosure. Those skilled in the art can make various modifications, combinations, subcombinations, and substitutions based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. An information processing method performed by an information processing device, The steps include obtaining a first bilingual sentence pair containing a source language sentence and a target language sentence, A step of distilling the first language sentence in the first bilingual sentence pair based on a large-scale language model to obtain a second bilingual sentence pair after distillation, wherein the first language sentence is the source language sentence or the target language sentence, A step comprising: integrating the second bilingual sentence pair and the first bilingual sentence pair to generate an augmentation corpus library; and training a student model based on the augmentation corpus library, wherein each corpus includes the source language sentences and the target language sentences; Information processing methods.

2. The step of distilling the first language sentence in the first bilingual sentence pair based on the large-scale language model and obtaining the second bilingual sentence pair after distillation is: A step of determining the distillation target of the first bilingual sentence pair, wherein the distillation target is translation distillation or embellishment distillation, The process includes the step of distilling the first language sentence according to the distillation target using the large-scale language model to obtain the second bilingual sentence pair, The information processing method according to claim 1.

3. The step of distilling the first language sentence according to the distillation target using the large-scale language model to obtain the second bilingual sentence pair is: The steps include generating the presented words of the large-scale language model based on the distillation target and the first bilingual sentence pair, The steps include inputting the presented word and at least one language sentence from the first bilingual sentence pair into the large-scale language model for distillation to obtain a third language sentence corresponding to the first language sentence, The process includes the step of generating a second bilingual sentence pair based on the second language sentence and the third language sentence in the first bilingual sentence pair, If the first language sentence is the source language sentence, the second language sentence is the target language sentence, and if the first language sentence is the target language sentence, the second language sentence is the source language sentence. The information processing method according to claim 2.

4. The step of inputting the aforementioned presented word and at least one linguistic sentence in the first bilingual sentence pair into the large-scale linguistic model and distilling it is: The steps include determining at least one language sentence to be input to the large-scale language model from the first bilingual sentence pair based on the distillation target, The process includes the step of inputting the aforementioned presented word and the at least one language sentence into the large-scale language model and distilling it, The information processing method according to claim 3.

5. The step of generating the presented words of the large-scale language model based on the distillation target and the first bilingual sentence pair is: If the distillation target is translational distillation, the step includes generating a first presentation word of the large-scale language model based on the distillation target and the second language sentence in the first bilingual sentence pair. The information processing method according to claim 3.

6. The step of generating the first presented word of the large-scale language model based on the distillation target and the second language sentence in the first bilingual sentence pair is: If the second language sentence is the source language sentence, the step of determining that the translation distillation is source language distillation, The process includes the step of using the second language sentence as the target language sentence for translation and generating a first suggestion word that translates the target language sentence into the source language, The information processing method according to claim 5.

7. The step of generating the first presented word of the large-scale language model based on the distillation target and the second language sentence in the first bilingual sentence pair is: If the second language sentence is the target language sentence, the step of determining that the translation distillation is target language distillation, The process includes the step of using the second language sentence as the target language sentence and generating a first suggestion word that translates the target language sentence into the target language, The information processing method according to claim 5.

8. The step of determining at least one language sentence to be input to the large-scale language model from the first bilingual sentence pair based on the distillation target is: The process includes the step of determining the second language sentence from the first bilingual sentence pair as the language sentence to be input to the large-scale language model, The information processing method according to claim 5.

9. The step of generating the presented words of the large-scale language model based on the distillation target and the first language sentence is: If the distillation target is embellishment distillation, the step includes generating a second presentation word of the large-scale language model based on the distillation target and the first language sentence. The information processing method according to claim 3.

10. The step of generating a second presented word of the large-scale language model based on the distillation target and the first language sentence is: The process includes the step of using the first language sentence as the language sentence to be modified, and generating a second suggestion word that modifies the language sentence to be modified. The information processing method according to claim 9.

11. The step of determining at least one language sentence to be input to the large-scale language model from the first bilingual sentence pair based on the distillation target is: The process includes the step of making the first language sentence and the second language sentence in the first bilingual sentence pair into language sentences to be input into the large-scale language model. The information processing method according to claim 9.

12. The step of training the student model based on the augmentation corpus library is: The steps include: performing a quality evaluation on the corpus in the aforementioned augmentation corpus library; Based on the quality evaluation information of the aforementioned corpus, a selection operation is performed on the corpus in the augmentation corpus library to obtain the target corpus library. The steps include training a student model based on the aforementioned target corpus library, The information processing method according to claim 1.

13. The step of performing a selection operation on the corpora in the augmentation corpus library based on the quality evaluation information of the aforementioned corpora and obtaining the target corpus library is: The steps include determining the corpus pairs corresponding to the same source language sentences, The process includes the step of selecting at least one target corpus corresponding to the same source language sentence from the corpus set based on quality evaluation information for each corpus in the corpus set, The information processing method according to claim 12.

14. The step of selecting at least one target corpus corresponding to the same source language sentence from the corpus set based on the quality evaluation information of each corpus in the corpus set is: A step of determining the corpus with the highest quality as the target corpus by comparing the quality evaluation information of each corpus in the aforementioned corpus set, or A step of selecting the target corpus by sorting the corpora in the corpus set based on the quality evaluation information of each corpus in the corpus set, or, The process includes comparing the quality evaluation information of each corpus in the aforementioned corpus set with a pre-set quality evaluation threshold, and selecting a corpus that is equal to or better than the quality evaluation threshold as the target corpus. The information processing method according to claim 13.

15. A retrieval module for obtaining a first bilingual sentence pair containing a source language sentence and a target language sentence, A distillation module for distilling a first language sentence in a first bilingual sentence pair based on a large-scale language model and obtaining a second bilingual sentence pair after distillation, comprising: a distillation module in which the first language sentence is the source language sentence or the target language sentence; The aforementioned distillation module further, The second pair of bilingual sentences and the first pair of bilingual sentences are combined to generate an augmentation corpus library, which is used to train a student model, and each corpus includes a step that includes the source language sentences and the target language sentences, Information processing device.

16. The distillation module is The distillation target of the first bilingual sentence pair is determined, and the distillation target is translation distillation or embellishment distillation. The large-scale language model is used to distill the first language sentence according to the distillation target and to obtain the second bilingual sentence pair. The information processing apparatus according to claim 15.

17. The distillation module is Based on the distillation target and the first bilingual sentence pair, the present words of the large-scale language model are generated. The presented word and at least one language sentence in the first bilingual sentence pair are input into the large-scale language model and distilled to obtain a third language sentence corresponding to the first language sentence. Used to generate the second bilingual sentence pair based on the second language sentence and the third language sentence in the first bilingual sentence pair, If the first language sentence is the source language sentence, the second language sentence is the target language sentence, and if the first language sentence is the target language sentence, the second language sentence is the source language sentence. The information processing apparatus according to claim 16.

18. The distillation module is Based on the distillation target, at least one language sentence to be input to the large-scale language model is determined from the first bilingual sentence pair. The aforementioned presented word and the at least one language sentence are input into the large-scale language model and used for distillation. The information processing apparatus according to claim 17.

19. The distillation module is If the distillation target is translational distillation, the distillation target and the second language sentence in the first bilingual sentence pair are used to generate the first presented word of the large-scale language model. The information processing apparatus according to claim 17 or 18.

20. The distillation module is If the second language sentence is the source language sentence, then the translation distillation is determined to be source language distillation. The second language sentence is used as the target language sentence for translation, and a first suggestion word is used to generate a first suggestion word that translates the target language sentence into the source language. The information processing apparatus according to claim 19.

21. The distillation module is If the second language sentence is the target language sentence, then the translation distillation is determined to be target language distillation. The second language sentence is used as the target language sentence for translation, and a first presentation word is used to generate a first presentation word that translates the target language sentence into the target language. The information processing apparatus according to claim 19.

22. The distillation module is Used to determine the second language sentence from the first bilingual sentence pair as the language sentence to be input into the large-scale language model, The information processing apparatus according to claim 19.

23. The distillation module is If the distillation target is embellishment distillation, the distillation target and the first language sentence are used to generate a second presentation word of the large-scale language model. The information processing apparatus according to claim 18.

24. The distillation module is The first language sentence is used as the language sentence to be modified, and a second presentation word is used to modify the language sentence to be modified. The information processing apparatus according to claim 23.

25. The distillation module is The first language sentence and the second language sentence in the first bilingual sentence pair are used to be language sentences input into the large-scale language model. The information processing apparatus according to claim 23 or 24.

26. The distillation module is The quality of the corpora in the aforementioned augmentation corpus library is evaluated. Based on the quality evaluation information of the aforementioned corpus, a selection operation is performed on the corpus in the augmentation corpus library, and the target corpus library is obtained. Used to train a student model based on the aforementioned target corpus library, The information processing apparatus according to claim 15.

27. The distillation module is Determine the corpus pairs corresponding to the same source language sentences, Based on the quality evaluation information of each corpus in the aforementioned corpus set, a method is used to select at least one target corpus from the corpus set that corresponds to the same source language sentence. The information processing apparatus according to claim 26.

28. The distillation module is By comparing the quality evaluation information of each corpus in the aforementioned corpus set, the corpus with the highest quality is determined as the target corpus, or Based on the quality evaluation information of each corpus in the aforementioned corpus set, the corpora in the aforementioned corpus set are sorted, and the corpus sorted to the front is selected as the target corpus, or This method is used to compare the quality evaluation information of each corpus in the aforementioned corpus set with a pre-set quality evaluation threshold, and to select a corpus that is equal to or better than the quality evaluation threshold as the target corpus. The information processing apparatus according to claim 27.

29. It is an electronic device, At least one processor, Includes a memory that is communicably connected to at least one of the processors, The memory stores instructions that can be executed by the at least one processor, and when an instruction is executed by the at least one processor, the at least one processor executes the information processing method described in any one of claims 1 to 14. electronic equipment.

30. A non-temporary, computer-readable storage medium in which computer instructions are stored, The computer instruction causes the computer to execute the information processing method described in any one of claims 1 to 14. A non-temporary, computer-readable storage medium.

31. It is a computer program, When the computer program is executed by a processor, the steps of the information processing method described in any one of claims 1 to 14 are realized. Computer program.

Images