A vocabulary initialization method and device for multi-language translation based on graph network

Abstract

The present invention discloses a vocabulary initialization method and device for multilingual translation based on a graph network, comprising: constructing a vocabulary mapping heterogeneous graph between each language and a core language based on the decomposition and composition relationship of bilingual dictionaries, words, and word units corresponding to each language; learning the word unit mapping relationship between the word units of different language vocabularies and the core language word unit on the vocabulary mapping heterogeneous graph based on a heterogeneous graph neural network and a contrastive learning algorithm; initializing the word units of the core language word unit in a multilingual machine translation task using the word unit of the core language BERT model, and mapping other languages ​​to the word unit space of the core language BERT model based on the word unit mapping relationship, thereby completing the initialization of all word units in the multilingual machine translation task. The present invention uses the method of constructing a word unit heterogeneous graph to complete the mapping of word units in different languages ​​to initialize word units, thereby improving the translation accuracy of the multilingual machine translation model and also accelerating the model convergence speed.

Description

Technical Field

[0001] The present invention relates to the technical field of machine translation, and in particular to a method and device for initializing a vocabulary for multilingual translation based on a graph network. Background Art

[0002] Neural machine translation uses neural networks to build models to implement end-to-end machine translation. Currently, the best performing neural network architecture is the Transformer architecture.

[0003] Multilingual machine translation (MT) uses a single machine translation model to translate between multiple languages. Compared to training a separate translation model for each language pair, MT significantly reduces the number of model parameters, effectively reducing deployment space requirements and increasing translation efficiency.

[0004] However, existing multilingual machine translation models require the construction and training of large vocabulary lists, which not only takes a lot of training time, but also, due to the lack of bilingual corpus for some languages, the translation accuracy output by the trained multilingual machine translation models is low.

[0005] Therefore, the existing technology needs to be improved. Summary of the Invention

[0006] The technical problem to be solved by the present invention is that, in response to the defects of the existing technology, the present invention provides a vocabulary initialization method and device for multilingual translation based on graph network to solve the technical problem of low translation accuracy of the existing multilingual machine translation model.

[0007] The technical solutions adopted by the present invention to solve the technical problems are as follows:

[0008] In a first aspect, the present invention provides a method for initializing a vocabulary for multilingual translation based on a graph network, comprising:

[0009] Based on the decomposition and composition relationship of bilingual dictionaries, words and lemmas corresponding to each language, a heterogeneous graph of vocabulary mapping between each language and the core language is constructed;

[0010] Learning word-element mapping relationships between different language vocabularies and a core language vocabulary on the vocabulary mapping heterogeneous graph based on a heterogeneous graph neural network and a contrastive learning algorithm;

[0011] The word-grams of the core language vocabulary in the multilingual machine translation task are initialized using the vocabulary of the core language BERT model, and other languages ​​are mapped to the vocabulary space of the core language BERT model according to the word-gram mapping relationship to complete the initialization of all word-grams in the multilingual machine translation task.

[0012] In one implementation, constructing a vocabulary mapping heterogeneous graph between each language and the core language based on the decomposition and composition relationship of the bilingual dictionary, words, and word units corresponding to each language includes:

[0013] Train the corresponding vocabulary for each language;

[0014] According to the vocabulary and bilingual dictionary corresponding to each language, a vocabulary mapping heterogeneous graph between each language and the core language is constructed.

[0015] In one implementation, the training of a corresponding vocabulary for each language includes:

[0016] Split all words in each language into tokens according to the BPE algorithm;

[0017] Count the probabilities of all word units appearing adjacently, and concatenate the word units with the highest probability to generate new subwords.

[0018] Based on all the generated new subwords, the vocabulary corresponding to each language is obtained.

[0019] In one implementation, constructing a vocabulary mapping heterogeneous graph between each language and the core language based on the vocabulary and bilingual dictionary corresponding to each language includes:

[0020] Setting all word-grams in the vocabulary and the corresponding words in the bilingual dictionary as nodes of the vocabulary mapping heterogeneous graph;

[0021] Adding inclusion relationships between word units and words of the same language into the vocabulary mapping heterogeneous graph;

[0022] Adding bilingual correspondences with the same referential meaning between different languages ​​into the vocabulary mapping heterogeneous graph;

[0023] Adding substring relationships between word units of the same language to the word table mapping heterogeneous graph;

[0024] The vocabulary mapping heterogeneous graph is obtained according to the added mapping relationship.

[0025] In one implementation, the vocabulary mapping heterogeneous graph is a directed heterogeneous graph: ;

[0026] in, Represents a node in the map, Represents an edge in the mapping graph.

[0027] In one implementation, learning the word-element mapping relationship between different language vocabularies and the core language vocabulary on the vocabulary mapping heterogeneous graph based on the heterogeneous graph neural network and the contrastive learning algorithm includes:

[0028] According to the heterogeneous graph neural network and the contrastive learning algorithm, the representation of each word element node is learned on the vocabulary mapping heterogeneous graph from different angles to obtain the embedding representation and vector corresponding to each word element node.

[0029] In one implementation, learning the representation of each word-meta node on the vocabulary mapping heterogeneous graph from different angles includes:

[0030] Learn the connection relationship between each word node and surrounding nodes based on the network model perspective.

[0031] In one implementation, the learning of representations of each word-meta node on the vocabulary mapping heterogeneous graph from different angles further includes:

[0032] Based on the meta-path perspective, the path connection relationship between each word meta-node and surrounding nodes is learned.

[0033] In one implementation, initializing the word-grams of the core language vocabulary in the multilingual machine translation task using the vocabulary of the core language BERT model, and mapping other languages ​​to the vocabulary space of the core language BERT model according to the word-gram mapping relationship includes:

[0034] Using the representation of the graph space to learn the correspondence between the core language and other languages;

[0035] Mapping the representation of the core language and the representation of other languages ​​into the embedding layer space of the BERT model according to the corresponding relationship;

[0036] The learned word-gram representations of each language are combined to initialize the vocabulary of the multilingual machine translation task.

[0037] In one implementation, learning the correspondence between the core language and other languages ​​using a representation in a graph space includes:

[0038] Calculate the similarity between each word in the core language and all words in other languages ​​in the graph space to obtain a similarity matrix;

[0039] A set of similar word-grams between the core language and other languages ​​is calculated according to the similarity matrix.

[0040] In a second aspect, the present invention provides a vocabulary initialization device for multilingual translation based on a graph network, comprising:

[0041] The vocabulary mapping heterogeneous graph module is used to construct a vocabulary mapping heterogeneous graph between each language and the core language based on the decomposition and composition relationship of the bilingual dictionary, words and word units corresponding to each language;

[0042] a word-meta mapping relationship module, configured to learn word-meta mapping relationships between different language vocabularies and a core language vocabularies on the vocabulary mapping heterogeneous graph based on a heterogeneous graph neural network and a contrastive learning algorithm;

[0043] The word-gram initialization module is used to initialize the word-grams of the core language vocabulary in the multilingual machine translation task using the vocabulary of the core language BERT model, and map other languages ​​to the vocabulary space of the core language BERT model according to the word-gram mapping relationship, thereby completing the initialization of all word-grams in the multilingual machine translation task.

[0044] In a third aspect, the present invention provides a terminal comprising: a processor and a memory, wherein the memory stores a vocabulary initialization program for multilingual translation based on a graph network, and when the vocabulary initialization program for multilingual translation based on a graph network is executed by the processor, it is used to implement the operation of the vocabulary initialization method for multilingual translation based on a graph network as described in the first aspect.

[0045] In a fourth aspect, the present invention also provides a medium, which is a computer-readable storage medium, and which stores a vocabulary initialization program for multilingual translation based on a graph network. When the vocabulary initialization program for multilingual translation based on a graph network is executed by a processor, it is used to implement the operation of the vocabulary initialization method for multilingual translation based on a graph network as described in the first aspect.

[0046] The present invention adopts the above technical solution to achieve the following effects:

[0047] The present invention can construct a vocabulary mapping heterogeneous graph between each language and the core language based on the decomposition and composition relationship of bilingual dictionaries, words, and word units corresponding to each language; and, based on the heterogeneous graph neural network and contrastive learning algorithm, can learn the word unit mapping relationship between different language word units and the core language word unit on the vocabulary mapping heterogeneous graph; and initialize the word units of the core language word unit in the multilingual machine translation task using the word unit of the core language BERT model, and map other languages ​​to the word unit space of the core language BERT model according to the word unit mapping relationship, thereby completing the initialization of all word units in the multilingual machine translation task. The present invention uses the method of constructing a word unit heterogeneous graph to complete the mapping of word units in different languages ​​to initialize word units, thereby improving the translation accuracy of the multilingual machine translation model and also accelerating the model convergence speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying any creative work.

[0049] Figure 1 This is a flowchart of a method for initializing a vocabulary for multilingual translation based on a graph network in one implementation of the present invention.

[0050] Figure 2 This is a schematic diagram of vocabulary initialization based on heterogeneous graph neural networks for multilingual machine translation in one implementation of the present invention.

[0051] Figure 3 It is a functional principle diagram of a terminal in one implementation of the present invention.

[0052] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solutions and advantages of the present invention more clear and distinct, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0054] Exemplary Methods

[0055] Existing multilingual machine translation models require the construction and training of large vocabulary lists, which not only takes a lot of training time, but also, due to the lack of bilingual corpus for some languages, the translation accuracy output by the trained multilingual machine translation models is low.

[0056] In response to the above technical problems, this embodiment provides a vocabulary initialization method for multilingual translation based on a graph network. In this embodiment, a word-unit heterogeneous graph is constructed to complete the mapping of word-units in different languages ​​to initialize word-units, thereby improving the translation accuracy of the multilingual machine translation model and accelerating the model convergence speed.

[0057] like Figure 1 As shown, an embodiment of the present invention provides a vocabulary initialization method for multilingual translation based on a graph network, comprising the following steps:

[0058] Step S100 : constructing a vocabulary mapping heterogeneous graph between each language and the core language based on the decomposition and composition relationship of the bilingual dictionary, words, and word-grams corresponding to each language.

[0059] In this embodiment, the graph network-based multi-language translation vocabulary initialization method is applied to a terminal, which includes but is not limited to: a computer and other devices.

[0060] In neural network tasks, a suitable data initialization method will generally effectively reduce the time it takes to learn model parameters and improve the model's performance to a certain extent. In the context of multilingual machine translation, an excellent initialization method needs to shorten the distance between words with the same meaning in different language vocabularies. The vocabulary of current machine translation models is generally based on subword granularity, and direct alignment of subwords across languages ​​is very difficult. Therefore, this embodiment proposes using a cross-language bilingual dictionary to construct a word-unit heterogeneous graph for subword-level alignment; a heterogeneous graph refers to a graph in which the sum of the number of node types and the number of edge types in the graph is greater than two. A heterogeneous graph neural network is a graph neural network constructed for heterogeneous graphs.

[0061] In this embodiment, the purpose is to provide an excellent vocabulary initialization for multilingual machine translation tasks, and then proposes to use the method of constructing a word-unit heterogeneous graph to complete the mapping of word units in different languages ​​to initialize word units, which can improve the model effect and accelerate the model convergence speed to a certain extent.

[0062] In this embodiment, a heterogeneous graph of vocabulary mapping is constructed between each multilingual machine translation and English through the decomposition and construction relationship of bilingual dictionaries, words and word units. Then, the word unit mapping relationship between the vocabularies of different languages ​​and the English vocabulary is learned on the heterogeneous graph using a heterogeneous graph neural network and contrastive learning. Finally, the word units of the English vocabulary in the multilingual machine translation task are initialized through the vocabulary embedding layer representation of the language model BERT. The mapping relationships learned in these graph spaces are used to map other languages ​​to the BERT vocabulary space, completing the initialization of all word units in the multilingual machine translation vocabulary.

[0063] Specifically, in one implementation of this embodiment, step S100 includes the following steps:

[0064] Step S101: training the corresponding vocabulary for each language.

[0065] like Figure 2As shown in the figure, in this embodiment, a vocabulary needs to be trained separately for each language; in existing natural language processing tasks, a vocabulary is generally constructed before training, and the basic constituent unit of the vocabulary is a token. During model training, the input natural language is tokenized and then mapped into an embedding representation through the vocabulary for training, and each token corresponds to an embedding representation (i.e., an embedding, a vector). The vocabulary is generally at the word level, that is, a word is a token, such as "apple", but this word-level representation may encounter the situation of words not seen during training (i.e., out-of-vocabulary words) during testing. To reduce this situation, a subword-level vocabulary can be used, that is, a subword is represented as an embedding representation. For example, "apple" can be represented as "app" and "le", and when encountering an unseen word, a new word can be formed by generating subwords and splicing them together.

[0066] Specifically, in one implementation manner of this embodiment, step S101 includes the following steps:

[0067] Step S101a, split all words in each language into tokens according to the BPE algorithm;

[0068] Step S101b, count the probability of adjacent occurrences of all tokens, and splice the tokens with the highest probability to generate new subwords;

[0069] Step S101c, obtain the vocabulary corresponding to each language according to all the generated new subwords.

[0070] In this embodiment, the subword training BPE algorithm is used. The main training method of the BPE algorithm is to first split all words into characters, then count the probability of adjacent occurrences of all characters, splice the ones with the highest probability to generate new subwords, and iterate this process until the number of iterations meets the requirements or the size of the vocabulary meets the requirements.

[0071] Specifically, in one implementation manner of this embodiment, step S100 includes the following steps:

[0072] Step S102, construct a vocabulary mapping heterogeneous graph between each language and the core language according to the vocabulary corresponding to each language and the bilingual dictionary.

[0073] In this embodiment, according to these vocabularies and the bilingual dictionary (i.e., the mapping relationship dictionary between cross-language words, such as "apple" and "苹果"), a vocabulary mapping heterogeneous graph is constructed for each language and the core language; among them, the core language can be languages such as English and Portuguese. In this embodiment, English is used as an example for illustration.

[0074] In order to solve the problem of unregistered words, the current vocabulary is generally at the subword granularity, which makes it difficult to build semantic relationships between subwords in different languages. Therefore, this embodiment introduces a bilingual dictionary to complete the mapping between subwords. In this embodiment, the cross-language vocabulary mapping graph is defined as a directed heterogeneous graph , where represents a node in the map, Represents an edge in the mapping graph.

[0075] Specifically, in one implementation of this embodiment, step S102 includes the following steps:

[0076] Step S102a, setting all word-grams in the vocabulary and the corresponding words in the bilingual dictionary as nodes of the vocabulary mapping heterogeneous graph;

[0077] Step S102b, adding the inclusion relationship between word-grams and words of the same language to the vocabulary mapping heterogeneous graph;

[0078] Step S102c, adding bilingual correspondences with the same referential meaning between different languages ​​into the vocabulary mapping heterogeneous graph;

[0079] Step S102d, adding substring relationships between word-grams of the same language to the vocabulary mapping heterogeneous graph;

[0080] Step S102e: obtaining the vocabulary mapping heterogeneous graph according to the added mapping relationship.

[0081] like Figure 2 As shown, assuming that the languages ​​involved in multilingual machine translation are Chinese, English and Japanese, a vocabulary mapping heterogeneous graph is constructed based on this background. In this embodiment, vocabulary mapping heterogeneous graphs need to be constructed for English and Chinese, and English and Japanese respectively.

[0082] In this embodiment, the vocabulary mapping graph (a directed heterogeneous graph) is specifically constructed as follows:

[0083] First, all subwords in the vocabulary and words in the bilingual dictionary are used as nodes of the vocabulary mapping graph;

[0084] Secondly, add the inclusion relationship between the same-language word units and words in the vocabulary mapping graph. By using the tokenizer of each language when constructing the vocabulary (i.e., the word unit tokenizer, which is the product during vocabulary construction, one for each language, and can divide a word or sentence of that language into a set of tokens according to the vocabulary, such as dividing "apple" into "##app" and "le"), tokenize all the words in the bilingual dictionary. For example, when tokenizing word A to obtain the sub-word set A_set, there is an inclusion relationship between word A and all the tokens in A_set, that is, the inclusion relationship between the same-language word units and words.

[0085] Thirdly, add the bilingual corresponding relationship with the same referential meaning between words of different languages in the vocabulary mapping graph. This relationship is added through the bilingual dictionary, and this relationship is naturally added between the corresponding words in the bilingual dictionary, such as "apple" and "苹果".

[0086] Finally, add the substring relationship between word units of the same language in the vocabulary mapping graph. Add it to the sub-vocabulary of each language. Traverse all the tokens in the vocabulary and judge the inclusion relationship between word units through dynamic programming. If one word unit completely contains another word unit (such as "application" contains "appli"), then there is a substring relationship between them.

[0087] After the above four steps, the vocabulary mapping graph (i.e., the vocabulary mapping graph between English and other languages) is constructed according to the vocabularies of different languages and the bilingual dictionary (i.e., the bilingual dictionary between English and other languages).

[0088] In this embodiment, there are two types of nodes: words of different languages in the bilingual dictionary, such as "apple", "苹果", etc. in the figure; word units in the vocabulary of each language, that is, sub-words, such as "#app", "#苹果", etc. in the figure.

[0089] The relationships in the vocabulary mapping heterogeneous graph include three types: the inclusion relationship between the same-language word units and words, as Figure 2 shown, the English word "apple" contains the English word units "#app" and "le"; the bilingual corresponding relationship with the same referential meaning between words of different languages, such as the relationship between the English word "apple" and the Chinese word "苹果"; the substring relationship between word units of the same language, such as there is a substring relationship between the English word units "#app" and "#ap".

[0090] As Figure 1 shown, in one implementation manner of the embodiment of the present invention, the vocabulary initialization method for multi-language translation based on the graph network further includes the following steps:

[0091] Step S200 , learning word-element mapping relationships between different language vocabularies and a core language vocabulary on the vocabulary mapping heterogeneous graph based on a heterogeneous graph neural network and a contrastive learning algorithm.

[0092] In this embodiment, it is necessary to learn representations for all nodes in the constructed heterogeneous graph (i.e., each node receives an embedded representation, a vector). This is primarily to learn representations for word-unit nodes in the graph, which are then used for learning relationships. The graph representation learning method used in this embodiment is a heterogeneous graph representation method based on contrastive learning.

[0093] Specifically, in one implementation of this embodiment, step S200 includes the following steps:

[0094] Step S201 , based on the heterogeneous graph neural network and the contrastive learning algorithm, learn the representation of each word-unit node on the vocabulary mapping heterogeneous graph from different angles to obtain the embedding representation and vector corresponding to each word-unit node.

[0095] In this embodiment, contrastive learning is used, a learning method that aims to keep the representations of similar nodes as close as possible and the representations of dissimilar nodes as far apart as possible. This contrastive learning method primarily constructs a model from the following two perspectives: calculating the contrastive learning cost function (based on positive and negative examples), and then training using gradient descent. Parameters include node representations and some fully connected layer parameters.

[0096] Specifically, in one implementation of this embodiment, step S201 includes the following steps:

[0097] Step S201a, learning the connection relationship between each word node and surrounding nodes based on the network model perspective;

[0098] Step S201b: learning the path connection relationship between each word-meta node and surrounding nodes based on the meta-path perspective.

[0099] In this embodiment, after constructing heterogeneous graphs for each language and English, it is necessary to learn the representation of nodes on each graph, that is, to learn the representation of word units by using a heterogeneous graph self-supervised representation learning method based on contrastive learning.

[0100] This method learns the representation of core nodes in the graph from two perspectives (the core nodes of the heterogeneous graph in this embodiment are all word nodes ), which are the network schema view and meta-path view respectively.

[0101] From the perspective of network model, the focus is on the connection relationship between the core node and the peripheral nodes. In this embodiment, all word nodes in the bilingual dictionary are involved (in this embodiment, the English word nodes are marked as , marking words in other languages ​​as )

[0102] From the perspective of meta-path, we focus on the path connection relationship between core nodes. In this embodiment, the meta-path is defined as , that is, English words English words Other language words For the positive and negative examples involved in contrastive learning in the model, in this embodiment, core nodes with more than two meta-paths between them are considered mutually positive examples, and the rest that do not meet this condition are considered mutually negative examples.

[0103] like Figure 1 As shown, in one implementation of the embodiment of the present invention, the method for initializing a vocabulary for multilingual translation based on a graph network further includes the following steps:

[0104] In step S300, the word-grams of the core language vocabulary in the multilingual machine translation task are initialized using the vocabulary of the core language BERT model, and other languages ​​are mapped to the vocabulary space of the core language BERT model according to the word-gram mapping relationship, thereby completing the initialization of all word-grams in the multilingual machine translation task.

[0105] In this embodiment, the neural network parameters need to be initialized before training, and then updated using methods such as gradient descent. Typical neural networks use random initialization; the initialization in this embodiment involves the BERT model, a pre-trained language model that also involves vocabulary issues. In this embodiment, the trained English BERT vocabulary representation is used to initialize the English vocabulary for multilingual machine translation.

[0106] Specifically, in one implementation of this embodiment, step S300 includes the following steps:

[0107] Step S301, using the representation of the graph space to learn the correspondence between the core language and other languages;

[0108] Step S302: Mapping the representation of the core language and the representation of other languages ​​into the embedding layer space of the BERT model according to the corresponding relationship;

[0109] Step S303 : merging the learned word-gram representations of each language to initialize the vocabulary of the multilingual machine translation task.

[0110] In this embodiment, the graph network-based method obtains the representation of English words in the graph space. and representations of words in other languages Next, we need to use the representation of the graph space to learn the correspondence between English and other languages, so that the representation of English and other languages ​​are in the same space, that is, the embedding layer space of the language model BERT.

[0111] Specifically, in one implementation of this embodiment, step S301 includes the following steps:

[0112] Step S301a, calculating the similarity between each word-gram of the core language and all word-grams of another language in the graph space to obtain a similarity matrix;

[0113] Step S301b: Calculate a set of similar word-grams between the core language and other languages ​​according to the similarity matrix.

[0114] In this embodiment, the correspondence between English and other languages ​​is learned using a similarity matrix;

[0115] like Figure 2 As shown, first calculate the similarity between each English word and all words in another language in the graph space to obtain a similarity matrix :

[0116] ;

[0117] in, Represents English A statement, Represents other languages Representation. The word unit of the English vocabulary initialized by BERT is represented as , then for each word in other languages , can be obtained through the similarity matrix Calculation yields:

[0118] ;

[0119] in, Represents the similarity matrix obtained in the graph space with the other language word The closest A collection of English words.

[0120] Perform the above operation for each language in the multilingual machine translation (except the core language English) to obtain the representation of the word-grams in the vocabulary involved in each language. At this point, you can use the learned word-gram representations of each language to merge and initialize the vocabulary of the multilingual machine translation.

[0121] When designing a heterogeneous graph, you can also build a single heterogeneous graph for all languages. However, if the original model has a large number of languages, the resulting mapping heterogeneous graph will be very large, and the training cost will continue to increase. To avoid this, in this embodiment, the mapping heterogeneous graph can be simplified by setting a core language, English, and mapping other languages ​​to the core language separately.

[0122] This embodiment achieves the following technical effects through the above technical solution:

[0123] This embodiment can construct a vocabulary mapping heterogeneous graph between each language and the core language based on the decomposition and composition relationship of the bilingual dictionary, words, and word-grams corresponding to each language; and, based on the heterogeneous graph neural network and contrastive learning algorithm, the word-gram mapping relationship between the word-grams of different languages ​​and the core language word-gram can be learned on the vocabulary mapping heterogeneous graph; and by initializing the word-grams of the core language word-gram in the multilingual machine translation task using the word-gram of the core language BERT model, and mapping other languages ​​to the word-gram space of the core language BERT model based on the word-gram mapping relationship, the initialization of all word-grams in the multilingual machine translation task is completed. This embodiment uses the method of constructing a word-gram heterogeneous graph to complete the mapping of word-grams of different languages ​​to initialize word-grams, thereby improving the translation accuracy of the multilingual machine translation model and also accelerating the model convergence speed.

[0124] Exemplary devices

[0125] Based on the above embodiments, the present invention further provides a vocabulary initialization device for multilingual translation based on a graph network, comprising:

[0126] The vocabulary mapping heterogeneous graph module is used to construct a vocabulary mapping heterogeneous graph between each language and the core language based on the decomposition and composition relationship of the bilingual dictionary, words and word units corresponding to each language;

[0127] a word-meta mapping relationship module, configured to learn word-meta mapping relationships between different language vocabularies and a core language vocabularies on the vocabulary mapping heterogeneous graph based on a heterogeneous graph neural network and a contrastive learning algorithm;

[0128] The word-gram initialization module is used to initialize the word-grams of the core language vocabulary in the multilingual machine translation task using the vocabulary of the core language BERT model, and map other languages ​​to the vocabulary space of the core language BERT model according to the word-gram mapping relationship, thereby completing the initialization of all word-grams in the multilingual machine translation task.

[0129] Based on the above embodiment, the present invention further provides a terminal, whose principle block diagram can be shown as follows: Figure 3 shown.

[0130] The terminal includes: a processor, memory, interface, display screen and communication module connected via a system bus; wherein the processor of the terminal is used to provide computing and control capabilities; the memory of the terminal includes a storage medium and an internal memory; the storage medium stores an operating system and computer programs; the internal memory provides an environment for the operation of the operating system and computer programs in the storage medium; the interface is used to connect to external devices, such as mobile terminals and computers; the display screen is used to display corresponding information; and the communication module is used to communicate with a cloud server or a mobile terminal.

[0131] When the computer program is executed by a processor, it is used to implement the operation of a vocabulary initialization method for multi-language translation based on a graph network.

[0132] It will be understood by those skilled in the art that Figure 3 The principle block diagram shown in the figure is only a block diagram of a partial structure related to the solution of the present invention, and does not constitute a limitation on the terminal to which the solution of the present invention is applied. The specific terminal may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0133] In one embodiment, a terminal is provided, which includes: a processor and a memory, wherein the memory stores a vocabulary initialization program for multi-language translation based on a graph network, and when the vocabulary initialization program for multi-language translation based on a graph network is executed by the processor, it is used to implement the operation of the vocabulary initialization method for multi-language translation based on a graph network as described above.

[0134] In one embodiment, a storage medium is provided, wherein the storage medium stores a vocabulary initialization program for multilingual translation based on a graph network, and when the vocabulary initialization program for multilingual translation based on a graph network is executed by a processor, it is used to implement the operation of the vocabulary initialization method for multilingual translation based on a graph network as described above.

[0135] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware through a computer program. The computer program can be stored in a non-volatile storage medium. When executed, the computer program can include the processes in the above-described method embodiments. Any reference to memory, database, or other media used in the various embodiments provided herein may include non-volatile and / or volatile memory.

[0136] In summary, the present invention provides a method and device for initializing a vocabulary for multilingual translation based on a graph network. The method includes: constructing a vocabulary mapping heterogeneous graph between each language and a core language based on the decomposition and composition relationship of bilingual dictionaries, words, and word units corresponding to each language; learning the word unit mapping relationship between different language vocabularies and the core language vocabulary on the vocabulary mapping heterogeneous graph based on a heterogeneous graph neural network and a contrastive learning algorithm; initializing the word units of the core language vocabulary in the multilingual machine translation task using the vocabulary of the core language BERT model, and mapping other languages ​​to the word unit space of the core language BERT model based on the word unit mapping relationship, thereby completing the initialization of all word units in the multilingual machine translation task. The present invention uses the method of constructing a word unit heterogeneous graph to complete the mapping of word units in different languages ​​to initialize word units, thereby improving the translation accuracy of the multilingual machine translation model and also accelerating the model convergence speed.

[0137] It should be understood that the application of the present invention is not limited to the above examples. For those skilled in the art, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A vocabulary initialization method for multilingual translation based on graph network, characterized in that: include: Based on the decomposition and composition relationship of bilingual dictionaries, words and lemmas corresponding to each language, a heterogeneous graph of vocabulary mapping between each language and the core language is constructed; Learning word-element mapping relationships between different language vocabularies and a core language vocabulary on the vocabulary mapping heterogeneous graph based on a heterogeneous graph neural network and a contrastive learning algorithm; The word-grams of the core language vocabulary in the multilingual machine translation task are initialized using the vocabulary of the core language BERT model, and other languages ​​are mapped to the vocabulary space of the core language BERT model according to the word-gram mapping relationship to complete the initialization of all word-grams in the multilingual machine translation task.

2. The method for initializing a vocabulary for multilingual translation based on a graph network according to claim 1, characterized in that: The method of constructing a vocabulary mapping heterogeneous graph between each language and the core language based on the decomposition and composition relationship of the bilingual dictionary, words, and word units corresponding to each language includes: Train the corresponding vocabulary for each language; According to the vocabulary and bilingual dictionary corresponding to each language, a vocabulary mapping heterogeneous graph between each language and the core language is constructed.

3. The method for initializing a vocabulary for multilingual translation based on a graph network according to claim 2, characterized in that: The training of the corresponding vocabulary for each language includes: Split all words in each language into tokens according to the BPE algorithm; Count the probabilities of all word units appearing adjacently, and concatenate the word units with the highest probability to generate new subwords. Based on all the generated new subwords, the vocabulary corresponding to each language is obtained.

4. The method for initializing a vocabulary for multilingual translation based on a graph network according to claim 2, characterized in that: The step of constructing a vocabulary mapping heterogeneous graph between each language and the core language based on the vocabulary and bilingual dictionary corresponding to each language includes: Setting all word-grams in the vocabulary and the corresponding words in the bilingual dictionary as nodes of the vocabulary mapping heterogeneous graph; Adding inclusion relationships between word units and words of the same language into the vocabulary mapping heterogeneous graph; Adding bilingual correspondences with the same referential meaning between different languages ​​into the vocabulary mapping heterogeneous graph; Adding substring relationships between word units of the same language to the word table mapping heterogeneous graph; The vocabulary mapping heterogeneous graph is obtained according to the added mapping relationship.

5. The method for initializing a vocabulary for multilingual translation based on a graph network according to claim 1, characterized in that: The vocabulary mapping heterogeneous graph is a directed heterogeneous graph: ; in, represents a node in the vocabulary mapping heterogeneous graph, Represents the edges in the vocabulary mapping heterogeneous graph.

6. The method for initializing a vocabulary for multilingual translation based on a graph network according to claim 1, characterized in that: The method of learning the word-element mapping relationship between different language vocabularies and the core language vocabularies on the vocabularies mapping heterogeneous graph based on the heterogeneous graph neural network and the contrastive learning algorithm includes: According to the heterogeneous graph neural network and the contrastive learning algorithm, the representation of each word element node is learned on the vocabulary mapping heterogeneous graph from different angles to obtain the embedding representation and vector corresponding to each word element node.

7. The method for initializing a vocabulary for multilingual translation based on a graph network according to claim 6, characterized in that: The learning of representations of each word element node on the vocabulary mapping heterogeneous graph from different angles includes: Learn the connection relationship between each word node and surrounding nodes based on the network model perspective.

8. The method for initializing a vocabulary for multilingual translation based on a graph network according to claim 6, characterized in that: The learning of representations of each word element node on the vocabulary mapping heterogeneous graph from different angles further includes: Based on the meta-path perspective, the path connection relationship between each word meta-node and surrounding nodes is learned.

9. The method for initializing a vocabulary for multilingual translation based on a graph network according to claim 1, characterized in that: Initializing the word grammar of the core language vocabulary in the multilingual machine translation task using the vocabulary of the core language BERT model, and mapping other languages ​​to the vocabulary space of the core language BERT model according to the word gram mapping relationship, includes: Using the representation of the graph space to learn the correspondence between the core language and other languages; Mapping the representation of the core language and the representation of other languages ​​into the embedding layer space of the BERT model according to the corresponding relationship; The learned word-gram representations of each language are combined to initialize the vocabulary of the multilingual machine translation task.

10. The method for initializing a vocabulary for multilingual translation based on a graph network according to claim 9, characterized in that: The learning of the correspondence between the core language and other languages ​​using the representation of the graph space includes: Calculate the similarity between each word in the core language and all words in other languages ​​in the graph space to obtain a similarity matrix; A set of similar word units between the core language and other languages ​​is calculated according to the similarity matrix.

11. A vocabulary initialization device for multilingual translation based on graph network, characterized in that: include: The vocabulary mapping heterogeneous graph module is used to construct a vocabulary mapping heterogeneous graph between each language and the core language based on the decomposition and composition relationship of the bilingual dictionary, words and word units corresponding to each language; a word-meta mapping relationship module, configured to learn word-meta mapping relationships between different language vocabularies and a core language vocabularies on the vocabulary mapping heterogeneous graph based on a heterogeneous graph neural network and a contrastive learning algorithm; The word-gram initialization module is used to initialize the word-grams of the core language vocabulary in the multilingual machine translation task using the vocabulary of the core language BERT model, and map other languages ​​to the vocabulary space of the core language BERT model according to the word-gram mapping relationship, thereby completing the initialization of all word-grams in the multilingual machine translation task.

12. A terminal, characterized in that: include: A processor and a memory, wherein the memory stores a vocabulary initialization program for multilingual translation based on a graph network, and when the vocabulary initialization program for multilingual translation based on a graph network is executed by the processor, it is used to implement the operation of the vocabulary initialization method for multilingual translation based on a graph network as described in any one of claims 1-10.

13. A medium, characterized in that The medium is a computer-readable storage medium, which stores a vocabulary initialization program for multilingual translation based on a graph network. When the vocabulary initialization program for multilingual translation based on a graph network is executed by a processor, it is used to implement the operation of the vocabulary initialization method for multilingual translation based on a graph network as described in any one of claims 1-10.

Images