Cross-language task processing method, electronic equipment and storage medium

Abstract

The invention provides a cross-language task processing method, electronic equipment and a storage medium, and the method comprises the steps: obtaining task data inputted by a user, the task data comprising a task instruction and task content; according to the task instruction and the task content, a priori knowledge token corresponding to the task data is generated, and the priori knowledge token comprises vocabularies different from the source language in language; according to the priori knowledge token and the task data, generating a priori text; and inputting the prior text into a base language model for iterative decoding processing, and outputting a target execution result indicated by the task instruction. Therefore, the base language model can output the accurate target execution result based on the input prior text, supervised training and additional training or fine tuning of the model are not needed, and the method can be suitable for processing of different cross-language tasks.

Description

Technical Field

[0001] The present application relates to the technical field of natural language processing. Specifically, it relates to a cross-language task processing method, an electronic device, and a storage medium. Background Art

[0002] The base language model is the foundation and core for building various large language model applications. Although the base language model has pre-trained and modeled a large amount of knowledge, it does not have the ability to understand and execute instructions related to downstream tasks. Facing complex instructions, it still exhibits the behavior of text continuation rather than executing specific tasks. That is to say, the function of the base language model is to complete the input text.

[0003] In the prior art, in order to align the specific pre-trained knowledge in the base model with the requirements described in the natural language instructions given by users in actual scenarios, a common approach is to perform supervised training on the base model using interactive dialogue data to obtain an instruction-following model.

[0004] However, the existing supervised training methods are difficult to cover cross-language scenario tasks in all languages. Moreover, the base language model is built with English corpus as the center and cannot work properly in cross-language tasks. Instead, it performs the behavior of continuing to write with an English text as the prefix. Summary of the Invention

[0005] An object of the present application is to provide a cross-language task processing method, an electronic device, and a storage medium, which can accurately understand and execute cross-language tasks, aiming at the deficiencies in the above-mentioned prior art.

[0006] To achieve the above object, the technical solutions adopted in the embodiments of the present application are as follows:

[0007] In a first aspect, an embodiment of the present application provides a cross-language task processing method, the method includes:

[0008] Obtain task data input by a user, where the task data includes a task instruction and task content, the task instruction is used to indicate the execution of a cross-language task, and the task content includes at least one vocabulary in a source language;

[0009] Generate a prior knowledge token corresponding to the task data according to the task instruction and the task content, where the prior knowledge token includes a vocabulary different from the source language;

[0010] Generate prior text according to the prior knowledge token and the task data;

[0011] Input the prior text into a base language model for iterative decoding processing, and output a target execution result indicated by the task instruction.

[0012] Optionally, generating the prior knowledge tokens corresponding to the task data according to the task instruction and the task content includes:

[0013] Determine the task type, the source language and the target language according to the task instruction, where the task type includes a translation task type or a summarization task type, and the target language is different from the source language;

[0014] Determine the set of source language vocabulary to be mapped according to the task type and the task content, where the set of source language vocabulary to be mapped contains at least one source language vocabulary to be mapped;

[0015] Generate the prior knowledge tokens according to the target language, the source language, the set of source language vocabulary to be mapped, the preset number of prior knowledge tokens, and the pre-established bilingual dictionary.

[0016] Optionally, determining the set of source language vocabulary to be mapped according to the task type and the task content includes:

[0017] If the task type is a translation task type, then use the set of each source language vocabulary in the task content as the set of source language vocabulary to be mapped;

[0018] If the task type is a summarization task type, then use a pre-trained keyword extraction model to extract at least one keyword from the task content, and use the set of each keyword as the set of source language vocabulary to be mapped.

[0019] Optionally, generating the prior knowledge tokens according to the target language, the source language, the set of source language vocabulary to be mapped, the preset number of prior knowledge tokens, and the pre-established bilingual dictionary includes:

[0020] Determine the target bilingual sub-dictionary from the pre-established bilingual dictionary according to the target language and the source language, where the target bilingual sub-dictionary includes at least one vocabulary in the target language, at least one vocabulary in the source language, and the mapping relationship between each vocabulary in the target language and each vocabulary in the source language;

[0021] Generate the prior knowledge tokens according to the target bilingual sub-dictionary, the set of source language vocabulary to be mapped, and the preset number of prior knowledge tokens.

[0022] Optionally, generating the prior knowledge tokens according to the target bilingual sub-dictionary, the set of source language vocabulary to be mapped, and the preset number of prior knowledge tokens includes:

[0023] Determine the target language vocabulary corresponding to each source language vocabulary to be mapped in the source language vocabulary set to be mapped according to the target bilingual sub-dictionary;

[0024] Use each target language vocabulary less than or equal to the preset number of prior knowledge tokens as the prior knowledge tokens.

[0025] Optionally, the determining the target language vocabulary corresponding to each source language vocabulary to be mapped in the source language vocabulary set to be mapped according to the target bilingual sub-dictionary includes:

[0026] Traverse each source language vocabulary to be mapped in the source language vocabulary set to be mapped in sequence. For the currently traversed source language vocabulary to be mapped, search for the currently traversed source language vocabulary in the target bilingual sub-dictionary, and use the language vocabulary having a mapping relationship with the currently traversed source language vocabulary as the target language vocabulary corresponding to the currently traversed source language vocabulary. When the number of target language vocabularies traversed reaches the preset number of prior knowledge tokens, the traversal ends.

[0027] Optionally, the generating the prior text according to the prior knowledge tokens and the task data includes:

[0028] Concatenate the prior knowledge tokens and the task data to generate the prior text.

[0029] Optionally, the inputting the prior text into the base language model for iterative decoding processing and outputting the target execution result indicated by the task instruction includes:

[0030] The base language model generates a text vector corresponding to the prior text according to the currently input prior text, and outputs a result word according to the text vector and by using the highest probability strategy, and updates the result word to the prior text to obtain the updated prior text, and inputs the updated prior text back into the base language model to generate a new result word until the iterative cutoff condition is satisfied, and uses the text composed of the prior knowledge tokens and each result word when the iterative cutoff condition is satisfied as the target execution result.

[0031] In a second aspect, an embodiment of the present application further provides a cross-language task processing device, and the device includes:

[0032] An acquisition module, configured to acquire task data input by a user, where the task data includes a task instruction and task content, the task instruction is used to indicate the execution of a cross-language task, and the task content includes at least one vocabulary in a source language;

[0033] A generation module, configured to generate prior knowledge tokens corresponding to the task data according to the task instruction and the task content, where the prior knowledge tokens include words different from the source language.

[0034] A generation module, configured to generate prior text according to the prior knowledge tokens and the task data.

[0035] A decoding module, configured to input the prior text into a base language model for iterative decoding processing, and output a target execution result indicated by the task instruction.

[0036] Optionally, the generation module is specifically configured to:

[0037] Determine a task type, a source language, and a target language according to the task instruction, where the task type includes a translation task type or a summarization task type, and the target language is different from the source language;

[0038] Determine a set of source language words to be mapped according to the task type and the task content, where the set of source language words to be mapped contains at least one source language word to be mapped;

[0039] Generate the prior knowledge tokens according to the target language, the source language, the set of source language words to be mapped, a preset number of prior knowledge tokens, and a pre-established bilingual dictionary.

[0040] Optionally, the generation module is specifically configured to:

[0041] If the task type is a translation task type, use the set of source language words in the task content as the set of source language words to be mapped;

[0042] If the task type is a summarization task type, extract at least one keyword from the task content by using a pre-trained keyword extraction model, and use the set of keywords as the set of source language words to be mapped.

[0043] Optionally, the generation module is specifically configured to:

[0044] Determine a target bilingual sub-dictionary from the pre-established bilingual dictionary according to the target language and the source language, where the target bilingual sub-dictionary includes at least one word in the target language, at least one word in the source language, and a mapping relationship between each word in the target language and each word in the source language;

[0045] Generate the prior knowledge tokens according to the target bilingual sub-dictionary, the set of source language words to be mapped, and the preset number of prior knowledge tokens.

[0046] Optionally, the generating module is specifically configured to:

[0047] Determine the target language vocabulary corresponding to each to-be-mapped source language vocabulary in the to-be-mapped source language vocabulary set according to the target bilingual sub-dictionary;

[0048] Use each target language vocabulary that is less than or equal to the preset number of prior knowledge tokens as the prior knowledge tokens.

[0049] Optionally, the generating module is specifically configured to:

[0050] Traverse each to-be-mapped source language vocabulary in the to-be-mapped source language vocabulary set in sequence. For the currently traversed to-be-mapped source language vocabulary, look up the currently traversed to-be-mapped source language vocabulary in the target bilingual sub-dictionary, and use the language vocabulary that has a mapping relationship with the currently traversed to-be-mapped source language vocabulary as the target language vocabulary corresponding to the currently traversed to-be-mapped source language vocabulary. When the number of target language vocabularies traversed reaches the preset number of prior knowledge tokens, end the traversal.

[0051] Optionally, the generating module is specifically configured to:

[0052] Concatenate the prior knowledge tokens with the task data to generate the prior text.

[0053] Optionally, the decoding module is specifically configured to:

[0054] The base language model generates a text vector corresponding to the prior text according to the currently input prior text, outputs a result word according to the text vector and using the highest probability strategy, updates the result word to the prior text to obtain an updated prior text, and re-inputs the updated prior text into the base language model to generate a new result word until the iteration termination condition is met, and uses the text composed of the prior knowledge tokens and each result word when the iteration termination condition is met as the target execution result.

[0055] In a third aspect, an embodiment of the present application further provides an electronic device, including: a processor, a storage medium, and a bus. The storage medium stores program instructions executable by the processor. When the application program runs, the processor communicates with the storage medium through the bus. The processor executes the program instructions to perform the steps of the cross-language task processing method described in the first aspect above.

[0056] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium. A computer program is stored on the computer-readable storage medium, and the computer program is read and executed to perform the steps of the cross-language task processing method described in the first aspect above.

[0057] The beneficial effects of the present application are as follows:

[0058] A cross - language task processing method, an electronic device, and a storage medium provided by the present application, after obtaining a task instruction and task content input by a user, first generate prior knowledge tokens corresponding to task data according to the task instruction and the task content. Since the generated prior knowledge tokens are words in a language different from the source language in the task content, therefore, generating a prior text according to the prior knowledge tokens and the task data has prior constraint conditions. Guided by the prior constraint conditions, the decoding process of the base language model is guided to a path more in line with the task requirements. This enables the base language model to output an accurate target execution result based on the input prior text, without supervised training and without additional training or fine - tuning of the model, and can be suitable for different cross - language task processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] To more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings required for use in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, without creative efforts, other related drawings can also be obtained based on these drawings.

[0060] Figure 1 It is a schematic flowchart of a cross - language task processing method provided by an embodiment of the present application;

[0061] Figure 2 It is a schematic flowchart of another cross - language task processing method provided by an embodiment of the present application;

[0062] Figure 3 It is a schematic flowchart of yet another cross - language task processing method provided by an embodiment of the present application;

[0063] Figure 4 It is a schematic flowchart of still another cross - language task processing method provided by an embodiment of the present application;

[0064] Figure 5 It is a complete schematic diagram of cross - language task processing provided by an embodiment of the present application;

[0065] Figure 6 It is a schematic diagram of a device for a cross - language task processing method provided by an embodiment of the present application;

[0066] Figure 7 It is a structural block diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0067] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of this application. It should be understood that the accompanying drawings in this application are only for the purpose of illustration and description, and are not used to limit the protection scope of this application. Additionally, it should be understood that the schematic drawings are not drawn to the scale of the actual objects. The flowcharts used in this application illustrate the operations implemented according to some embodiments of this application. It should be understood that the operations in the flowchart may not be implemented in sequence, and steps without a logical context relationship may be reversed in order or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart or remove one or more operations from the flowchart under the guidance of the content of this application.

[0068] In addition, the described embodiments are only a part of the embodiments of this application, rather than all of the embodiments. The components of the embodiments of this application usually described and illustrated in the accompanying drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of this application that is claimed, but merely represents the selected embodiments of this application. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative efforts fall within the scope of protection of this application.

[0069] It should be noted that the term "including" will be used in the embodiments of this application to indicate the presence of the features stated hereinafter, but does not exclude the addition of other features.

[0070] The base language model is the foundation and core for building various large language model applications, but it does not have the ability to understand and execute instructions related to downstream tasks. The function of the base model is to complete the input text (i.e., text continuation).

[0071] In the prior art, supervised fine-tuning is the main method for adjusting the output of the base language model to adapt to downstream tasks. Nowadays, the main construction method of mainstream intelligent dialogue assistants is to perform supervised training on the base language model with a large amount of specific task data ("instruction input - task output" data pairs).

[0072] However, the existing supervised training methods are difficult to cover all actual scenario tasks, especially cross-language tasks, and there are mainly three challenges:

[0073] One is the problem of data scarcity. Labeled dialogue instruction data is scarce and mainly focuses on English single-language tasks, making it difficult to cover all cross-language tasks and languages. And manually constructing the instruction data for relevant cross-language tasks requires hiring bilingual and multilingual individuals, which will consume a huge amount of time and labor costs.

[0074] Second, there is a risk of catastrophic forgetting. Supervised training on the base model will cause the model to forget the knowledge learned during the language modeling stage and instead focus on the knowledge in the instruction fine-tuning task.

[0075] Third, it is difficult to balance multilingual resources. Supervised training requires careful balancing of multilingual data in the data distribution. Otherwise, it will cause the short-board effect, resulting in poor performance of the finally trained model in resource-scarce languages and limiting the universality of the multilingual model.

[0076] Therefore, based on the technical problems existing in the prior art, the present application proposes a cross-language task processing method that does not require supervised training and can complete cross-language tasks.

[0077] Optionally, the cross-language task processing method provided in the embodiments of the present application is applied to an electronic device, which can be, for example, a terminal device such as a mobile phone, a tablet computer, a laptop computer, a handheld computer, a desktop computer, etc. that has computing and processing capabilities and a display function, or it can also be a server. Specifically, it can be applied to an application program in the terminal device, such as: an application (APP) of a mobile phone, an application system on a computer, etc.

[0078] Next, the specific implementation process of cross-language task processing provided in the embodiments of the present application will be specifically explained.

[0079] Figure 1 FIG. is a schematic flowchart of a cross-language task processing method provided in an embodiment of the present application. The execution subject of this method is the aforementioned electronic device. As Figure 1 shown, the method includes:

[0080] S101. Obtain task data input by the user.

[0081] Among them, the task data input by the user may include a task instruction and task content. The task instruction is used to indicate the execution of a cross-language task, and the task content includes a vocabulary indicating a source language. The cross-language task can be a language task regarding two different language languages, such as a Chinese-English language translation task, a Chinese-Korean language translation task, an English-Korean language translation task, etc. The task content is the specific content for executing the task instruction. The task data can be represented by, for example, the sequence [· ins .

[0082] For example, if the task data input by the user is "Translate from English to Chinese: We now have 4-month-old mice.", the task instruction is "Translate from English to Chinese", the task content is "We now have 4-month-old mice.", "We", "now", "have", etc. in the task content are each a word, and the source language is English.

[0083] S102: Generate a priori knowledge token corresponding to the task data according to the task instruction and task content.

[0084] The prior knowledge token may include vocabulary different from the source language, and the prior knowledge token also includes at least one vocabulary. The prior knowledge token can add prior constraints to the task data. For example, when the source language is English, the language of the prior knowledge token can be vocabulary in languages other than English, such as Chinese, Korean, etc. Specifically, the language of the prior knowledge token can be determined using a preset method based on the task instructions and task content. The generated prior knowledge token can be generated using, for example, [X pri ] to indicate.

[0085] For example, taking the aforementioned task content "We now have 4-month-old mice." as an example, the source language in the task content is English. Then, a prior knowledge token can be generated according to the task instructions and task content using a preset method. For example, if the generated prior knowledge token is "we", then "we" is a Chinese word different from the source language.

[0086] S103: Generate prior text according to the prior knowledge token and task data.

[0087] Specifically, the priori text may be generated using a preset method according to the priori knowledge token generated in S102 and the task data input by the user in S101, and the generated priori text includes at least one vocabulary and includes vocabulary in two languages.

[0088] For example, taking the aforementioned task data “Translate from English to Chinese: We now have 4-month-old mice.” and the prior knowledge token “we” as an example, the generated prior text may be, for example, “Translate from English to Chinese: We now have 4-month-old mice. We”.

[0089] S104: Input the priori text into the base language model for iterative decoding processing, and output the target execution result indicated by the task instruction.

[0090] Among them, the base language model can be a forward propagation model. Specifically, the prior text guided by the prior knowledge token is input into the base language model. After the base language model receives the prior text, since the prior text contains the prior knowledge token, the base language model can use a small amount of prior knowledge tokens as decoding guidance information, and can iteratively decode the received prior text and output the target execution result indicated by the task instruction. Specifically, the prior knowledge token adjusts the probability distribution of the subsequent text generated by the base language model, so that the target execution result generated by the base language model is closer to the distribution of the supervised training model, thereby guiding the base language model to generate the target execution result.

[0091] For example, continuing with the prior text “Translate from English to Chinese: We now have 4-month-old mice.”, the base language model can perform iterative decoding processing on “Translate from English to Chinese: We now have 4-month-old mice.” Since the task instruction is “Translate from English to Chinese,” the target execution result output by the base language model is “We now have a 4-month-old mouse.”

[0092] In this embodiment, after obtaining the task instruction and task content input by the user, first generate prior knowledge tokens corresponding to the task data according to the task instruction and task content. Since the generated prior knowledge tokens are words in a language different from the source language in the task content, therefore, generating prior text according to the prior knowledge tokens and task data has prior constraint conditions. Under the guidance of the prior constraint conditions, the decoding process of the base language model is guided to a path more in line with the task requirements. This enables the base language model to output accurate target execution results based on the input prior text, without supervised training and without additional training or fine-tuning of the model, and can be suitable for different cross-language task processing.

[0093] Figure 2 It is a schematic flowchart of another cross-language task processing method provided by an embodiment of the present application. As Figure 2 shown, in S102 above, generating prior knowledge tokens corresponding to the task data according to the task instruction and task content may include:

[0094] S201. Determine the task type, source language, and target language according to the task instruction.

[0095] Among them, the task type may include a translation task type or a summarization task type. The translation task of the translation task type is also a language translation for two different languages, and the summarization task of the summarization task type is also a summarization task for two different languages.

[0096] Among them, the target language is different from the source language. For example, the target language is Chinese and the source language is English; it can also be that the source language is English and the target language is Korean; it can also be that the source language is Chinese and the target language is English, etc.

[0097] Exemplarily, continuing with the task instruction "Translate from English to Chinese:" in the foregoing as an example, through the analysis of the task instruction, it can be determined that the source language is "English", that is, English, and the target language is "Chinese", that is, Chinese. The task type is a translation task type. Specifically, it is a translation task of translating English into Chinese.

[0098] S202. Determine the set of source language vocabulary to be mapped according to the task type and task content.

[0099] Among them, the set of source language vocabulary to be mapped contains at least one source language vocabulary to be mapped. The set of source language vocabulary to be mapped can be represented by S for example, then S = [s1, s2,..., s i , s i is the i-th source language vocabulary to be mapped.

[0100] S203. Generate prior knowledge tokens according to the target language, the source language, the set of source language vocabulary to be mapped, the preset number of prior knowledge tokens, and the pre-established bilingual dictionary.

[0101] Among them, the number of prior knowledge tokens can be determined in advance. The number of prior knowledge tokens refers to the number of words that make up the prior knowledge tokens. For example, the number of prior knowledge tokens is represented by k, and k can be a number less than or equal to 3, that is, at most 3 prior knowledge tokens can be constructed.

[0102] Among them, the bilingual dictionary can be a language dictionary of any two languages. The bilingual dictionary can include at least one bilingual sub-dictionary, such as Chinese-English language sub-dictionary, Korean-Chinese language sub-dictionary, English-Korean language sub-dictionary, English-Japanese language sub-dictionary, Chinese-Japanese language sub-dictionary, etc. Each bilingual sub-dictionary contains at least one vocabulary in the first language, at least one vocabulary in the second language, and the mapping relationship between the vocabulary in each first language and the vocabulary in each second language. Each vocabulary in the first language corresponds to a vocabulary in the second language one by one.

[0103] Optionally, in the above S202 step, determining the set of source language vocabulary to be mapped according to the task type and the task content may include:

[0104] Specifically, if the task type is a translation task type, then the set of each source language vocabulary in the task content is used as the set of source language vocabulary to be mapped.

[0105] Exemplarily, taking "Translate from English to Chinese: We now have 4-month-old mice." mentioned above as an example, since the task type is a translation task type, the set of all vocabulary in the task content "We now have 4-month-old mice" is used as the set of source language vocabulary to be mapped, so S = [We, now, have, month, old, mice].

[0106] Optionally, if the task type is a summarization task type, use a pre-trained keyword extraction model to extract at least one keyword from the task content, and use the set of each keyword as the set of source language vocabulary to be mapped. Among them, this summarization task is, for example, a cross-language abstract generation task, then a keyword extraction model can be used to extract at least one keyword from the task content, and the set of the extracted keywords is used as the set of source language vocabulary to be mapped.

[0107] In this embodiment, for different task types, different strategies are used to determine the set of source language vocabulary to be mapped, so as to generate prior knowledge tokens based on the determined set of source language vocabulary to be mapped, making the generated prior knowledge tokens more in line with the requirements of each task type.

[0108] Figure 3 It is a schematic flowchart of another cross - language task processing method provided by an embodiment of the present application. As Figure 3 shown, the above - mentioned S203. Generate prior knowledge tokens according to the target language variety, source language variety, set of source language vocabulary to be mapped, preset number of prior knowledge tokens, and pre - established bilingual dictionary, which may include:

[0109] S301. Determine the target bilingual sub - dictionary from the pre - established bilingual dictionary according to the target language variety and source language variety.

[0110] Among them, the target bilingual sub - dictionary includes at least one vocabulary of the target language variety, at least one vocabulary of the source language variety, and the mapping relationship between the vocabulary of each target language variety and the vocabulary of each source language variety.

[0111] Exemplarily, in the foregoing "Translate from English to Chinese", the source language variety is English and the target language variety is Chinese, then the determined target bilingual sub - dictionary is the Chinese - English language sub - dictionary.

[0112] S302. Generate prior knowledge tokens according to the target bilingual sub - dictionary, the set of source language vocabulary to be mapped, and the preset number of prior knowledge tokens.

[0113] Exemplarily, according to the Chinese - English language sub - dictionary determined in step S301, the set of source language vocabulary to be mapped S = [We, now, have, month, old, mice], and the preset number of prior knowledge tokens k, a preset method can be used to generate prior knowledge tokens.

[0114] Figure 4 It is a schematic flowchart of yet another cross - language task processing method provided by an embodiment of the present application. As Figure 4 shown, the above - mentioned S302. Generate prior knowledge tokens according to the target bilingual sub - dictionary, the set of source language vocabulary to be mapped, and the preset number of prior knowledge tokens, which may include:

[0115] S401. Determine the target language vocabulary corresponding to each source language vocabulary to be mapped in the set of source language vocabulary to be mapped according to the target bilingual sub - dictionary.

[0116] Specifically, the target bilingual sub-dictionary contains at least one vocabulary in the source language and the corresponding target language vocabulary for each vocabulary in the source language. Therefore, the target language vocabulary corresponding to each source language vocabulary to be mapped can be determined from the target bilingual sub-dictionary. In the target bilingual sub-dictionary, the target language vocabulary corresponding to each source language vocabulary to be mapped may or may not exist.

[0117] S402. Use each target language vocabulary that is less than or equal to the preset number of prior knowledge tokens as prior knowledge tokens.

[0118] Specifically, the number of determined target language vocabularies can be less than or equal to the preset number of prior knowledge tokens, that is, each target language vocabulary with a quantity less than or equal to k can be used as a prior knowledge token. If there are k or more target language vocabularies corresponding to the source language vocabularies to be mapped in the target bilingual sub-dictionary, then each target language vocabulary with a quantity of k can be used as a prior knowledge token; if there are less than k and more than 0 target language vocabularies corresponding to the source language vocabularies to be mapped in the target bilingual sub-dictionary, then the existing target language vocabularies corresponding to the source language vocabularies to be mapped are used as prior knowledge tokens; if there are no target language vocabularies corresponding to the source language vocabularies to be mapped in the target bilingual sub-dictionary, then the preset target language vocabulary is used as a prior knowledge token. Among them, the preset target language vocabulary can be, for example, <unk>。

[0119] Specifically, it can be represented by the formula to express.

[0120] Among them, s i is each source language word to be mapped, D(s i ) is the target language word corresponding to each source language word to be mapped in the target bilingual sub-dictionary, t i is the target language word, and n is the total number of source language words to be mapped in the set of source language words to be mapped, <unk>is a preset prior knowledge token. i where i≤k.

[0121] Optionally, in the above step S401, determining the target language vocabulary corresponding to each source language vocabulary to be mapped in the source language vocabulary set to be mapped according to the target bilingual sub-dictionary may include:

[0122] Specifically, each source language word to be mapped in the set of source language words to be mapped is traversed in turn, and for the current source language word to be mapped that is traversed, the current source language word to be mapped is searched from the target bilingual sub-dictionary, and the language word that has a mapping relationship with the current source language word to be mapped is used as the target language word corresponding to the current source language word to be mapped. When the number of traversed target language words reaches the preset number of prior knowledge tokens, the traversal is terminated.

[0123] Exemplarily, if the current source language vocabulary to be mapped is "We", then "We" is searched in the target bilingual sub-dictionary. If there is a target language vocabulary such as "we" that has a mapping relationship with "We", then "we" is used as the target language vocabulary of "We". If the preset number of prior knowledge tokens is 1, the traversal is terminated after the target language vocabulary "we" is determined; if there is no target language vocabulary that has a mapping relationship with "We", then the traversal continues to the next source language vocabulary to be mapped of "We" in the source language vocabulary set to be mapped, such as "now", until the number of traversed target language vocabulary reaches the preset number of prior knowledge tokens. If all the source language vocabulary to be mapped cannot be mapped, that is, the target language vocabulary of each source language vocabulary to be mapped does not exist in the target bilingual sub-dictionary, then the preset target language vocabulary is used as the prior knowledge token corresponding to the task data. For example, the aforementioned " <unk>” as the prior knowledge token corresponding to the task data.

[0124] Optionally, the above S103, generating a priori text according to the priori knowledge token and the task data, may include:

[0125] Specifically, we can concatenate the prior knowledge tokens with the task data to generate the prior text. The prior text obtained can be [X ins , X pri ], where [X ins ] is the task data, [X pri ] is a prior knowledge token.

[0126] For example, taking the aforementioned task data "Translate from English to Chinese: We now have 4-month-old mice." and the prior knowledge token "we" as an example, by concatenating "we" after "Translatefrom English to Chinese: We now have 4-month-old mice.", we can get the prior text "Translate from English to Chinese: We now have4-month-old mice.we".

[0127] Optionally, the above S104, inputting the priori text into the base language model for iterative decoding processing, and outputting the target execution result indicated by the task instruction, may include:

[0128] Optionally, the base language model can generate a text vector corresponding to the prior text based on the input current prior text, and output the result word based on the text vector and the highest probability strategy, and update the result word to the prior text to obtain an updated prior text. Specifically, the result word can be concatenated to the current prior text to obtain an updated prior text. The updated prior text is re-input into the base language model to generate a new result word until the iteration cutoff condition is met, and the text composed of the prior knowledge token and each result word when the iteration cutoff condition is met is used as the target execution result.

[0129] Specifically, the prior text [X ins ,X pri ] is input into the base language model, and the base language model is forward propagated to initialize the hidden state As the decoding starting point, d model is the size of the model’s hidden state dimension. When decoding for the first time, the hidden state h is calculated by stacking the model layers. t =Transformax(X ins , X pri ), project h t onto the vocabulary space V (|V| = N is the size of the predefined vocabulary), generating a logits vector l t ∈ R N , and apply temperature scaling and the softmax function to obtain P(y t+1 ) = softmax(l′ t / τ). Here, τ > 0 is the temperature hyperparameter used to control the sharpness of the distribution. Then, the resulting word output using the highest probability strategy is y t+1 = argmax1 v∈V P(v|X ins , X pri ), concatenate the resulting word to the prior text of the current input to obtain the updated prior text, and the updated prior text is [X ins , X pri , y 1:t , where, [y 1:t is the resulting word obtained from the t-th decoding. Then, take the updated prior text [X ins , X pri , y 1:t and re-enter it into the base language model for decoding until the iteration cutoff condition is met and then end. Among them, the iteration cutoff condition can refer to the resulting word obtained from the last decoding being a terminator (such as <eos>) Or the length of the obtained updated prior text reaches a preset threshold L max . Then, the text composed of the prior knowledge tokens and each result word when the iterative cutoff condition is met is used as the target execution result. For example, if the obtained result words are "There is a 4-month-old mouse.", and the prior knowledge token is "We", then the obtained result words are concatenated after the prior knowledge token in sequence to obtain the target execution result. For example, the obtained target execution result is "We there is a 4-month-old mouse.".

[0130] Figure 5 This is a complete schematic diagram of cross-language task processing provided by an embodiment of the present application. As Figure 5 shown, after the base language model receives the task data input by the user and pauses decoding before entering the decoding stage, it first determines the task type according to the task instruction and generates prior knowledge tokens, concatenates the prior knowledge tokens after the task data to obtain the prior text, and inputs the prior text into the base language model. The base language model can perform decoding based on the knowledge of the prior knowledge tokens to obtain the target execution result.

[0131] Through the method in the above specific embodiments of the present application, the prior art cannot solve the task scenarios outside the training data distribution and has the risk of catastrophic forgetting, while the present application has good generalization and scalability and can flexibly use different language auxiliary resources to guide the base language model to decode. The present application does not require training and can enable the base model to decode complex instructions without additional data. The prior art needs to optimize all or part of the model parameters and requires a large amount of computing resources to complete the tuning process. In the present application, only the input needs to be modified, and the time cost is much less than the tuning.

[0132] Figure 6 This is a schematic diagram of an apparatus for a cross-language task processing method provided by an embodiment of the present application. As Figure 6 shown, the apparatus includes:

[0133] An acquisition module 501, configured to acquire task data input by a user, where the task data includes a task instruction and task content, the task instruction is used to indicate the execution of a cross-language task, and the task content includes at least one vocabulary in a source language;

[0134] A generation module 502, configured to generate prior knowledge tokens corresponding to the task data according to the task instruction and the task content, where the prior knowledge tokens include vocabulary different from the source language;

[0135] A generation module 502, configured to generate a prior text according to the prior knowledge tokens and the task data;

[0136] The decoding module 503 is configured to input the prior text into the base language model for iterative decoding processing, and output the target execution result indicated by the task instruction.

[0137] Optionally, the generating module 502 is specifically configured to:

[0138] Determine the task type, the source language, and the target language according to the task instruction, where the task type includes a translation task type or a summarization task type, and the target language is different from the source language;

[0139] Determine the set of source language vocabulary to be mapped according to the task type and the task content, where the set of source language vocabulary to be mapped contains at least one source language vocabulary to be mapped;

[0140] Generate the prior knowledge tokens according to the target language, the source language, the set of source language vocabulary to be mapped, the preset number of prior knowledge tokens, and the pre-established bilingual dictionary.

[0141] Optionally, the generating module 502 is specifically configured to:

[0142] If the task type is a translation task type, use the set of each source language vocabulary in the task content as the set of source language vocabulary to be mapped;

[0143] If the task type is a summarization task type, extract at least one keyword from the task content using a pre-trained keyword extraction model, and use the set of each keyword as the set of source language vocabulary to be mapped.

[0144] Optionally, the generating module 502 is specifically configured to:

[0145] Determine the target bilingual sub-dictionary from the pre-established bilingual dictionary according to the target language and the source language, where the target bilingual sub-dictionary includes at least one vocabulary in the target language, at least one vocabulary in the source language, and the mapping relationship between each vocabulary in the target language and each vocabulary in the source language;

[0146] Generate the prior knowledge tokens according to the target bilingual sub-dictionary, the set of source language vocabulary to be mapped, and the preset number of prior knowledge tokens.

[0147] Optionally, the generating module 502 is specifically configured to:

[0148] Determine the target language vocabulary corresponding to each source language vocabulary to be mapped in the set of source language vocabulary to be mapped according to the target bilingual sub-dictionary;

[0149] Take each target language vocabulary less than or equal to the preset number of prior knowledge tokens as the prior knowledge tokens.

[0150] Optionally, the generation module 502 is specifically configured to:

[0151] Traverse each to-be-mapped source language vocabulary in the to-be-mapped source language vocabulary set in sequence. For the currently traversed to-be-mapped source language vocabulary, look up the currently traversed to-be-mapped source language vocabulary in the target bilingual sub-dictionary, and take the language vocabulary having a mapping relationship with the currently traversed to-be-mapped source language vocabulary as the target language vocabulary corresponding to the currently traversed to-be-mapped source language vocabulary. When the number of target language vocabularies traversed reaches the preset number of prior knowledge tokens, end the traversal.

[0152] Optionally, the generation module 502 is specifically configured to:

[0153] Concatenate the prior knowledge tokens with the task data to generate the prior text.

[0154] Optionally, the decoding module 503 is specifically configured to:

[0155] The base language model generates a text vector corresponding to the prior text according to the currently input prior text, and outputs a result word according to the text vector and by adopting a highest probability strategy, and updates the result word to the prior text to obtain an updated prior text, and re-inputs the updated prior text into the base language model to generate a new result word until an iteration termination condition is met, and takes the text composed of the prior knowledge tokens and each result word when the iteration termination condition is met as the target execution result.

[0156] Figure 7 The structural block diagram of an electronic device 600 provided by an embodiment of the present application Figure 7 . As Figure 7 shown, the electronic device may include: a processor 601 and a memory 602.

[0157] Optionally, a bus 603 may further be included, where the memory 602 is used to store machine-readable instructions executable by the processor 601. When the electronic device 600 runs, the processor 601 communicates with the memory 602 through the bus 603, and when the machine-readable instructions are executed by the processor 601, the method steps in the above method embodiments are executed.

[0158] An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is run by a processor, the method steps in the above cross-language task processing method embodiments are executed.

[0159] Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the systems and devices described above can refer to the corresponding processes in the method embodiments, and will not be elaborated herein. In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation. For another example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection to each other can be through some communication interfaces. The indirect coupling or communication connection of the devices or modules can be in electrical, mechanical or other forms.

[0160] In addition, in each embodiment of the present application, each functional unit can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art or a part of this technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks or optical disks and other various media that can store program codes.

[0161] The above is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present application, and all should be covered by the protection scope of the present application.< / eos> < / unk> < / unk> < / unk>

Claims

1. A cross - language task processing method, characterized in that The method includes: Obtain task data input by a user, where the task data includes a task instruction and task content, the task instruction is used to indicate the execution of a cross - language task, and the task content includes at least one vocabulary in the source language; Generate a prior knowledge token corresponding to the task data according to the task instruction and the task content, where the prior knowledge token includes a vocabulary different from the source language; Generate a prior text according to the prior knowledge token and the task data; Input the prior text into a base language model for iterative decoding processing, and output the target execution result indicated by the task instruction.

2. The cross-language task processing method according to claim 1, wherein The generating the prior knowledge token corresponding to the task data according to the task instruction and the task content includes: Determine the task type, the source language, and the target language according to the task instruction, where the task type includes a translation task type or a summarization task type, and the target language is different from the source language; Determine a set of source language vocabulary to be mapped according to the task type and the task content, where the set of source language vocabulary to be mapped contains at least one source language vocabulary to be mapped; Generate the prior knowledge token according to the target language, the source language, the set of source language vocabulary to be mapped, a preset number of prior knowledge tokens, and a pre - established bilingual dictionary.

3. The cross-language task processing method according to claim 2, wherein The determining the set of source language vocabulary to be mapped according to the task type and the task content includes: If the task type is a translation task type, use the set of each source language vocabulary in the task content as the set of source language vocabulary to be mapped; If the task type is a summarization task type, use a pre - trained keyword extraction model to extract at least one keyword from the task content, and use the set of each keyword as the set of source language vocabulary to be mapped.

4. The cross-language task processing method according to claim 2, wherein, The generating the prior knowledge token according to the target language, the source language, the set of source language vocabulary to be mapped, a preset number of prior knowledge tokens, and a pre - established bilingual dictionary includes: Determine a target bilingual sub - dictionary from the pre - established bilingual dictionary according to the target language and the source language, where the target bilingual sub - dictionary includes at least one vocabulary in the target language, at least one vocabulary in the source language, and the mapping relationship between each vocabulary in the target language and each vocabulary in the source language; Generate the prior knowledge token according to the target bilingual sub - dictionary, the set of source language vocabulary to be mapped, and the preset number of prior knowledge tokens.

5. The cross-language task processing method according to claim 4, wherein The generating the prior knowledge token according to the target bilingual sub - dictionary, the set of source language vocabulary to be mapped, and the preset number of prior knowledge tokens includes: Determine the target language vocabulary corresponding to each source language vocabulary to be mapped in the set of source language vocabulary to be mapped according to the target bilingual sub - dictionary; Use each target language vocabulary less than or equal to the preset number of prior knowledge tokens as the prior knowledge token.

6. The cross-language task processing method according to claim 5, wherein, Determining the target language vocabulary corresponding to each to-be-mapped source language vocabulary in the to-be-mapped source language vocabulary set according to the target bilingual sub-dictionary includes: Traversing each to-be-mapped source language vocabulary in the to-be-mapped source language vocabulary set in sequence. For the currently traversed to-be-mapped source language vocabulary, searching for the currently traversed to-be-mapped source language vocabulary in the target bilingual sub-dictionary, and taking the language vocabulary having a mapping relationship with the currently traversed to-be-mapped source language vocabulary as the target language vocabulary corresponding to the currently traversed to-be-mapped source language vocabulary. When the number of target language vocabularies traversed reaches the preset number of prior knowledge tokens, the traversal ends.

7. The cross-language task processing method according to claim 1, wherein Generating prior text according to the prior knowledge tokens and the task data includes: Concatenating the prior knowledge tokens and the task data to generate the prior text.

8. The cross-language task processing method according to claim 1, characterized in that Inputting the prior text into the base language model for iterative decoding processing and outputting the target execution result indicated by the task instruction includes: The base language model generates a text vector corresponding to the prior text according to the currently input prior text, and outputs a result word according to the text vector and by adopting the highest probability strategy, and updates the result word to the prior text to obtain an updated prior text. The updated prior text is re-input into the base language model to generate a new result word until the iterative termination condition is satisfied, and the text composed of the prior knowledge tokens and each result word when the iterative termination condition is satisfied is used as the target execution result.

9. An electronic device, characterized in that, It includes a memory and a processor. The memory stores a computer program executable by the processor. When the processor executes the computer program, the steps of the cross-language task processing method according to any one of claims 1-8 above are implemented.

10. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium. When the computer program is run by the processor, the steps of the cross-language task processing method according to any one of claims 1-8 are executed.

Images