Big data based machine translation system and document-level vocabulary translation method

Abstract

The application provides a machine translation system based on big data and a document-level vocabulary translation method. Global attention of vocabulary in a target document in a translation process is extracted; semantic role analysis is performed on the vocabulary to obtain semantic role features of the vocabulary, a word chain of the vocabulary is constructed according to the semantic role features and semantic similarity between the vocabulary; semantic contribution degrees of each homograph in the word chain to a theme of the target document are determined, further attention interaction is performed on each homograph in the word chain to obtain semantic attention of each homograph in the word chain; context information between each homograph is interactively semantically embedded through the semantic attention of each homograph in the word chain and the word chain semantic relationship between each homograph to obtain semantic embedding amounts of each homograph, and then document-level vocabulary translation is performed based on all the semantic embedding amounts. By adopting the scheme, semantic drift generated by cross-sentence translation of vocabulary context in a document can be eliminated.

Description

Technical Field

[0001] The present application relates to the technical field of machine translation, and more specifically, to a big data-based machine translation system and a document-level vocabulary translation method. Background Art

[0002] Machine translation is a technology that uses computers to automatically convert one natural language into another. Its core includes rule-based translation, statistical machine translation, and neural machine translation. Rule-based translation relies on linguistic rules, statistical machine translation uses statistical models of large amounts of bilingual corpora, and neural machine translation uses deep learning and neural networks to improve the fluency and accuracy of translation. In recent years, neural machine translation has become mainstream and is widely used in cross-language communication, information retrieval, and intelligent assistants, helping to break down language barriers and promote global information sharing.

[0003] With the continuous advancement of natural language processing technology, especially in the field of machine translation, how to make full use of cross-sentence context information to improve translation quality has become a research hotspot. Traditional sentence-level translation models find it difficult to effectively capture cross-sentence semantic connections, resulting in an increasingly prominent problem of semantic drift of vocabulary in different sentences. Semantic drift refers to the change in meaning of the same vocabulary in different contexts. This phenomenon is particularly evident in cross-sentence translation, because the expansion of the context range makes word meaning dependence more complex and diverse. For example, a word may assume different grammatical roles or semantic functions in the previous and next sentences, which leads to deviations in the translation model's understanding of word meaning and affects the accuracy and consistency of the overall translation. Traditional methods often find it difficult to fully model this dynamically changing semantic information, resulting in the failure to effectively resolve word meaning ambiguity. Therefore, how to resolve the semantic drift caused by cross-sentence translation of lexical context in documents has become a difficult problem facing the industry. Summary of the Invention

[0004] The present application provides a big data-based machine translation system and a document-level vocabulary translation method, which can eliminate the semantic drift caused by cross-sentence translation of vocabulary context in documents.

[0005] In a first aspect, the present application provides a document-level vocabulary translation method for performing vocabulary translation in a big data-based machine translation system, the method comprising the following steps:

[0006] Obtain the target document and extract the global attention of the words in the target document during the translation process based on the global attention mechanism;

[0007] Performing semantic role analysis on the words in the target document to obtain semantic role features of the words, and then constructing word chains of the words based on the semantic role features and the semantic similarity between the words;

[0008] Determining the semantic contribution of each cognate word in the word chain to the target document topic, performing attention interaction on each cognate word in the word chain through all semantic contributions and the global attention of the vocabulary during the translation process, and obtaining the semantic attention of each cognate word in the word chain;

[0009] By interactively semantically embedding the contextual information between cognate words through the semantic attention of each cognate word in the word chain and the word chain semantic relationship between each cognate word, the semantic embedding amount of each cognate word is obtained, and then document-level vocabulary translation is performed based on all the semantic embedding amounts.

[0010] In some embodiments, extracting the global attention of the words in the target document during the translation process based on the global attention mechanism specifically includes:

[0011] Vectorize each word in the target document to obtain the word vector for each word;

[0012] Based on the global attention mechanism, the attention weight of each word to other words in the target document is determined by the word vector of each word;

[0013] The global attention of each word in the target document during translation is extracted from all attention weights.

[0014] In some embodiments, performing semantic role analysis on the words in the target document to obtain semantic role features of the words specifically includes:

[0015] Perform syntactic dependency analysis on the sentence containing the word in the target document to obtain the dependency relationship between the word and other words in the sentence;

[0016] Label the semantic roles of words through all dependency relationships;

[0017] The semantic role feature of the vocabulary is determined according to the semantic role label.

[0018] In some embodiments, constructing a word chain of words based on the semantic role features and the semantic similarity between words specifically includes:

[0019] Perform threshold screening on all words in the target document according to the semantic role features to obtain a candidate word set;

[0020] Screening the candidate word set for cognates based on semantic similarity between words to obtain a cognate word set;

[0021] Based on the index position of each cognate word in the target document, all cognate words in the cognate word set are linked in index order to obtain a word chain of the vocabulary.

[0022] In some embodiments, determining the semantic contribution of each cognate word in the word chain to the target document topic specifically includes:

[0023] Extract topic keywords from the target document based on the preset document topic model;

[0024] Determining the semantic association strength and co-occurrence frequency between each cognate word in the word chain and the subject keyword;

[0025] The semantic contribution of each cognate word in the word chain to the target document topic is determined according to the semantic association strength and co-occurrence frequency between each cognate word in the word chain and the topic keyword.

[0026] In some embodiments, the semantic attention of each cognate word in the word chain is obtained by performing attention interaction on each cognate word in the word chain through all semantic contributions and the global attention of the vocabulary during the translation process, specifically including:

[0027] Determining the local attention of each cognate word in the word chain according to the semantic contribution of each cognate word in the word chain to the target document topic and the global attention of the vocabulary during the translation process;

[0028] Perform attention propagation on the local attention of each cognate word in the word chain to obtain the attention interaction coefficient of each cognate word in the word chain;

[0029] The semantic weights of the local attention of each cognate word in the word chain are distributed through all the attention interaction coefficients to obtain the semantic attention of each cognate word in the word chain.

[0030] In some embodiments, interactive semantic embedding of contextual information between cognate words is performed through semantic attention of each cognate word in a word chain and the word chain semantic relationship between each cognate word, and the semantic embedding amount of each cognate word is obtained specifically including:

[0031] Determine the semantic relationship between each cognate word in the word chain;

[0032] The context information between the cognate words is interactively transferred through the semantic relationship between the cognate words in the word chain to obtain the context interaction representation vector of each cognate word;

[0033] According to the semantic attention of each cognate word in the word chain and the context interaction representation vector of each cognate word, the semantic embedding of each cognate word is obtained.

[0034] In some embodiments, performing document-level vocabulary translation based on all semantic embeddings specifically includes:

[0035] Initialize the pre-trained large language model;

[0036] Each word in the target document is translated consistently based on the large language model and all semantic embeddings to obtain a translation result for each word.

[0037] In some embodiments, the target document is an English technical document.

[0038] In a second aspect, the present application provides a machine translation system based on big data, wherein the machine translation system includes a document-level vocabulary translation unit, wherein the document-level vocabulary translation unit includes:

[0039] The acquisition module is used to obtain the target document and extract the global attention of the words in the target document during the translation process based on the global attention mechanism;

[0040] A processing module, configured to perform semantic role analysis on the words in the target document to obtain semantic role features of the words, and then construct a word chain of the words based on the semantic role features and the semantic similarity between the words;

[0041] The processing module is further configured to determine the semantic contribution of each cognate word in the word chain to the target document topic, and to obtain the semantic attention of each cognate word in the word chain by performing attention interaction on each cognate word in the word chain through all semantic contributions and the global attention of the vocabulary during the translation process;

[0042] The execution module is used to interactively semantically embed the contextual information between cognate words through the semantic attention of each cognate word in the word chain and the word chain semantic relationship between each cognate word, obtain the semantic embedding amount of each cognate word, and then perform document-level vocabulary translation based on all the semantic embedding amounts.

[0043] The technical solutions provided by the embodiments disclosed in this application have the following beneficial effects:

[0044] In the big data-based machine translation system and document-level vocabulary translation method provided by the present application, by acquiring a target document, the global attention of the vocabulary in the target document during the translation process is extracted based on the global attention mechanism; semantic role analysis is performed on the vocabulary in the target document to obtain the semantic role characteristics of the vocabulary, and then a word chain of the vocabulary is constructed according to the semantic role characteristics and the semantic similarity between the vocabulary; the semantic contribution of each cognate word in the word chain to the target document topic is determined, and attention interaction is performed on each cognate word in the word chain through all semantic contributions and the global attention of the vocabulary during the translation process to obtain the semantic attention of each cognate word in the word chain; interactive semantic embedding is performed on the context information between each cognate word through the semantic attention of each cognate word in the word chain and the word chain semantic relationship between each cognate word to obtain the semantic embedding amount of each cognate word, and then document-level vocabulary translation is performed based on all semantic embedding amounts.

[0045] It can be seen that in this application, the contextual information between each cognate word can be interactively semantically embedded through the semantic attention of each cognate word in the word chain and the word chain semantic relationship between each cognate word to obtain the semantic embedding amount of each cognate word; among them, firstly, the global attention of the vocabulary in the target document during the translation process is extracted, which can break the limitation of the traditional translation model that only focuses on the information within the sentence, and can capture the semantic association of vocabulary in different contexts in a larger range, and enhance the perception of cross-sentence semantic connections; secondly, the semantic role analysis of the vocabulary in the target document can identify the semantic function and grammatical role of vocabulary in different sentences, especially in cross-sentence translation, the same vocabulary may appear in multiple sentences. By extracting the semantic role features of words, we can identify the functional differences of words in different sentences, thus avoiding confusing words with different meanings but the same superficial form. Furthermore, by combining the semantic role features with the semantic similarity between words to construct word chains, we can identify semantically related and functionally similar words at the document level and link them in a chain structure. This kind of word chain can cross sentence boundaries and aggregate words that express similar concepts or meanings in the context, thus making up for the deficiency of traditional sentence-level translation models in identifying cross-sentence word meaning connections, enhancing the ability to grasp the dynamic semantic changes of words, and avoiding the deviation in word meaning understanding caused by isolating single words. Then, the semantic contribution of each cognate word in the word chain to the target document theme is determined, which can quantify the importance and influence of different words in the overall semantic structure; furthermore, attention interaction is performed on each cognate word in the word chain, which can dynamically adjust and strengthen the semantic association between words. The attention interaction comprehensively considers the importance of the word in the document theme and its attention in translation, so as to more accurately capture the semantic changes of words in different contexts. The semantic attention of each cognate word in the word chain can identify and adjust the semantic drift caused by context differences, ensuring that cognate words maintain semantic consistency and coherence when translating across sentences; then, the semantic attention of each cognate word in the word chain and the interaction between cognate words are used to identify and adjust the semantic drift caused by context differences, ensuring that cognate words maintain semantic consistency and coherence when translating across sentences; then, the semantic attention of each cognate word in the word chain and the interaction between cognate words are used to identify and adjust the semantic drift caused by context differences, ensuring that cognate words maintain semantic consistency and coherence when translating across sentences The interactive semantic embedding not only strengthens the semantic connection between cognates, but also fully considers the mutual influence of cognates in different contexts, making the semantic representation of vocabulary richer and more accurate. The generated semantic embeddings can effectively alleviate the semantic drift caused by cross-sentence context changes, improve the accuracy and consistency of word meaning discrimination, and thus ensure the stable transmission of word meaning during the translation process. Finally, document-level vocabulary translation is performed based on all semantic embeddings. In summary, the solution of the present application can achieve the elimination of semantic drift caused by cross-sentence translation of vocabulary context in documents. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 is an exemplary flow chart of a document-level vocabulary translation method according to some embodiments of the present application;

[0047] Figure 2 is a schematic diagram of a process for determining semantic contribution according to some embodiments of the present application;

[0048] Figure 3 is a schematic diagram of a process for determining semantic embedding amount according to some embodiments of the present application;

[0049] Figure 4 is a schematic diagram of the structure of a document-level vocabulary translation unit according to some embodiments of the present application;

[0050] Figure 5 1 is a schematic diagram of the structure of a computer device for implementing a document-level vocabulary translation method according to some embodiments of the present application. DETAILED DESCRIPTION

[0051] In order to better understand the technical solution of the present application, the technical solution of the present application will be described in detail below with reference to the accompanying drawings and specific implementation methods.

[0052] refer to Figure 1 , which is an exemplary flow chart of a document-level vocabulary translation method according to some embodiments of the present application. The document-level vocabulary translation method mainly includes the following steps:

[0053] In step 101, a target document is obtained, and the global attention of the words in the target document during the translation process is extracted based on the global attention mechanism.

[0054] It should be noted that the target document described in this application is an English technical document. English technical documents focus on accurately conveying professional and technical information. They usually contain professional content in the fields of computer science, engineering technology, biomedicine, etc., and have the characteristics of rigorous structure, standardized terminology, and clear logic.

[0055] In some embodiments, extracting the global attention of each word in the target document during the translation process based on the global attention mechanism can be achieved by the following steps:

[0056] Vectorize each word in the target document to obtain the word vector for each word;

[0057] Based on the global attention mechanism, the attention weight of each word to other words in the target document is determined by the word vector of each word;

[0058] The global attention of each word in the target document during translation is extracted from all attention weights.

[0059] It should be noted that the attention weight described in this application represents a parameter that represents the degree of contextual contribution of a word to other words in the target document; the global attention represents the distribution vector of the word's attention to the overall context of the target document.

[0060] In the specific implementation, first, each word in the target document is vectorized to obtain the word vector of each word. This can be achieved in the following way: using the Word2Vec model to convert each word in the target document into a vector, and using the obtained vectors as the word vectors of the corresponding words; secondly, based on the global attention mechanism, the attention weight of each word to other words in the target document is determined by the word vector of each word. This can be achieved in the following way: inputting the word vector of each word in the target document into the Transformer model, extracting the attention distribution value of each word to other words in the target document through the self-attention mechanism of the Transformer model, and using the obtained attention distribution value as the attention weight of the corresponding word to other words in the target document; then, extracting the global attention of each word in the target document during the translation process from all the attention weights can be achieved in the following way: for each word, multiplying the attention weight of the word to other words in the target document by the value vector of other words in the target document, then summing all the multiplied values, and using the summed vector as the global attention of the word during the translation process, thereby obtaining the global attention of each word in the target document during the translation process.

[0061] It should be noted that the Word2Vec model described in this application is a word embedding method based on a shallow neural network. It maps each word into a continuous low-dimensional vector by utilizing the co-occurrence information of vocabulary in a large-scale corpus, so that words with similar semantics are closer in the vector space, thereby capturing the semantic relationship between words; the Transformer model is a deep learning architecture based on the self-attention mechanism, which can process sequence data in parallel. By calculating the mutual attention weights between elements in the sequence, it effectively models long-distance dependencies. It is widely used in natural language processing tasks such as machine translation and text generation, which greatly improves the model's ability to understand contextual information.

[0062] In step 102, semantic role analysis is performed on the words in the target document to obtain semantic role features of the words, and then word chains of the words are constructed based on the semantic role features and the semantic similarity between the words.

[0063] In some embodiments, performing semantic role analysis on words in a target document to obtain semantic role features of the words can be achieved by using the following steps:

[0064] Perform syntactic dependency analysis on the sentence containing the word in the target document to obtain the dependency relationship between the word and other words in the sentence;

[0065] Label the semantic roles of words through all dependency relationships;

[0066] The semantic role feature of the vocabulary is determined according to the semantic role label.

[0067] It should be noted that the dependency relationship described in this application represents the structural subordinate relationship between words; the semantic role label represents the formal identification of the semantic function of the word in the sentence in which it is located; and the semantic role feature represents the feature that reflects the semantic role of the word in the sentence in which it is located.

[0068] In the specific implementation, first, perform syntactic dependency analysis on the target document to obtain the dependency relationship between the vocabulary and other vocabulary in the sentence where it is located. This can be achieved in the following way: use the SpaCy syntactic analysis tool to perform syntactic dependency analysis on the vocabulary in the sentence where it is located in the target document, so as to obtain the dependency relationship between the vocabulary and other vocabulary in the sentence where it is located; secondly, the semantic role labels of the vocabulary can be annotated through all the dependency relationships. This can be achieved in the following way: input all the dependency relationships and the vocabulary vectors of the vocabulary into the semantic role labeling model. After model inference, the semantic role labeling model identifies the predicate in the sentence where the vocabulary is located. Word-argument structure, thereby outputting the role label of the vocabulary, and using the obtained role label as the semantic role label of the vocabulary; then, determining the semantic role feature of the vocabulary based on the semantic role label can be implemented in the following way, namely: converting the semantic role label into a vector through the Word2Vec model, and using the obtained vector as the role label vector, and then obtaining the vocabulary vector of the vocabulary, further, performing weighted averaging on the vocabulary vector and the role label vector of the vocabulary, and using the vector obtained by weighted averaging as the semantic role feature of the vocabulary, wherein, when performing weighted averaging, the following formula is adopted in this application, namely: semantic role feature ,in is a weight hyperparameter that can be set empirically. is the vocabulary vector of the vocabulary, is the role label vector.

[0069] It should be noted that the semantic role labeling model described in this application is a type of natural language processing model used to automatically identify predicates and their related arguments (such as semantic roles such as agent and patient) in sentences. The semantic role labeling model is usually based on deep learning technology, such as recurrent neural networks and long short-term memory networks. By learning the semantic structures in a large amount of annotated corpus, it can accurately determine the semantic role distribution of words in a sentence.

[0070] In some embodiments, constructing a word chain of words based on the semantic role features and the semantic similarity between words can be achieved by using the following steps:

[0071] Perform threshold screening on all words in the target document according to the semantic role features to obtain a candidate word set;

[0072] Screening the candidate word set for cognates based on semantic similarity between words to obtain a cognate word set;

[0073] Based on the index position of each cognate word in the target document, all cognate words in the cognate word set are linked in index order to obtain a word chain of the vocabulary.

[0074] It should be noted that the cognate word set described in this application represents a set of words that captures the dynamic evolution of the semantics of the vocabulary in the target document; wherein, the cognate words represent words with a common semantic source in the target document; the word chain represents a chain structure that captures the semantic extension of the vocabulary in different paragraphs in the target document.

[0075] In the specific implementation, first, all the words in the target document are threshold-screened according to the semantic role features, and the candidate word set can be obtained in the following way, namely: all the words in the target document are converted into vectors using the Word2Vec model, and the obtained vectors are used as word vectors, the cosine similarity between the semantic role features of the vocabulary and the vocabulary vectors of other vocabulary in the sentence where the vocabulary is located is calculated, and then the mean of all cosine similarities is used as the screening threshold, and further, the cosine similarity between the vocabulary vector and the word vector of each word in the target document is calculated, all cosine similarities are compared with the screening threshold, and the words corresponding to the cosine similarities greater than or equal to the screening threshold are used as candidate words, so that the set composed of all candidate words is used as the candidate word set; Secondly, the candidate word set is screened for cognates by the semantic similarity between words, and the cognate word set can be obtained in the following manner, namely: calculating the cosine similarity between all words in the target document, and taking all the obtained cosine similarities as the semantic similarity between words, and taking the average of the semantic similarities between all words as the cognate word threshold; further, calculating the cosine similarity between all candidate words in the candidate word set, taking the candidate words corresponding to the cosine similarity greater than or equal to the cognate word threshold as cognate words, and then taking the set of all cognate words as the cognate word set; then, all the cognate words in the cognate word set are linked in index order according to the index position of each cognate word in the target document, and taking the index link obtained after linking as the word chain of the vocabulary.

[0076] In step 103, the semantic contribution of each cognate word in the word chain to the target document topic is determined, and attention interaction is performed on each cognate word in the word chain through all semantic contributions and the global attention of the vocabulary in the translation process to obtain the semantic attention of each cognate word in the word chain.

[0077] In some embodiments, reference Figure 2 As shown in FIG, this figure is a flow chart of determining semantic contribution in some embodiments of the present application. In this embodiment, determining the semantic contribution of each cognate word in the word chain to the target document topic can be achieved by using the following steps:

[0078] Extract topic keywords from the target document based on the preset document topic model;

[0079] Determining the semantic association strength and co-occurrence frequency between each cognate word in the word chain and the subject keyword;

[0080] The semantic contribution of each cognate word in the word chain to the target document topic is determined according to the semantic association strength and co-occurrence frequency between each cognate word in the word chain and the topic keyword.

[0081] It should be noted that the document topic model described in this application adopts the latent Dirichlet allocation model, which is a commonly used document topic modeling algorithm. It is based on the probability generation model and assumes that each document consists of several topics, and each topic is represented by a set of words according to the probability distribution. The latent Dirichlet allocation model automatically mines the potential topic structure in the document by modeling the document collection, and can generate a topic distribution and a keyword set for each topic for each document.

[0082] In addition, it should be noted that the subject keywords described in this application represent the keywords of the core content of the target document theme; the semantic association strength represents the semantic proximity of cognate words to the subject keywords in the semantic space; the co-occurrence frequency represents the number of times cognate words and subject keywords appear together in the target document; the semantic contribution represents the importance of cognate words in expressing the semantic information of the target document theme.

[0083] In the specific implementation, first, the topic keywords are extracted from the target document based on the preset document topic model, which can be implemented in the following way, namely: the preset document topic model is used to perform topic modeling on the target document, thereby generating multiple topics, and then the keywords with the highest probability in each topic are selected as the topic keywords; secondly, the semantic association strength and co-occurrence frequency between each cognate word and the topic keyword in the word chain are determined, which can be implemented in the following way, namely: the topic keyword is converted into a vector through the Word2Vec model and the obtained vector is used as the keyword vector, the Euclidean distance between the word vector of each cognate word in the word chain and the keyword vector of the topic keyword is calculated, and the obtained Euclidean distance is used as the semantic association strength between the corresponding cognate word and the topic keyword, based on the sliding window technology, a sliding window (for example, 5 words) is set, and the sliding window is moved from front to back in the entire target document. Each time the window slides, it is detected whether there is a cognate word in the sliding window. When the cognate words in the word chain and any subject keyword are included, if they exist, it is regarded as a co-occurrence number, and the co-occurrence number of each cognate word in the word chain and any subject keyword in all windows is counted, and the sum of all co-occurrence numbers is used as the co-occurrence frequency between the corresponding cognate words and the subject keyword, thereby obtaining the sum co-occurrence frequency between each cognate word in the word chain and the subject keyword; then, according to the semantic association strength and co-occurrence frequency between each cognate word in the word chain and the subject keyword, the semantic contribution of each cognate word in the word chain to the target document theme is determined, which can be achieved in the following way, namely: for each cognate word in the word chain, a negative exponential function with the natural logarithm e as the base is calculated for the co-occurrence frequency between the cognate word and the subject keyword, and then the value obtained after calculating the negative exponential function with the natural logarithm e as the base is multiplied by the semantic association strength between the cognate word and the subject keyword, and the multiplied value is used as the semantic contribution of the cognate word to the target document theme.

[0084] In some embodiments, the semantic attention of each cognate word in the word chain is obtained by performing attention interaction on each cognate word in the word chain through all semantic contributions and the global attention of the vocabulary in the translation process. The following steps can be used to achieve the semantic attention of each cognate word in the word chain:

[0085] Determining the local attention of each cognate word in the word chain according to the semantic contribution of each cognate word in the word chain to the target document topic and the global attention of the vocabulary during the translation process;

[0086] Perform attention propagation on the local attention of each cognate word in the word chain to obtain the attention interaction coefficient of each cognate word in the word chain;

[0087] The semantic weights of the local attention of each cognate word in the word chain are distributed through all the attention interaction coefficients to obtain the semantic attention of each cognate word in the word chain.

[0088] It should be noted that the local attention described in this application represents the distribution vector of the attention intensity of cognate words on the target document topic in the current semantic context.

[0089] In specific implementation, the local attention of each cognate word in the word chain is determined according to the semantic contribution of each cognate word in the word chain to the target document topic and the global attention of the vocabulary in the translation process. This can be achieved in the following way, namely: for each cognate word in the word chain, the semantic contribution of the cognate word to the target document topic is multiplied by the global attention of the vocabulary in the translation process, and the vector obtained by the multiplication is used as the local attention of the cognate word, thereby obtaining the local attention of each cognate word in the word chain.

[0090] It should be noted that the attention interaction coefficient described in this application represents a weight parameter of the degree of mutual influence between cognate words in the process of semantic transmission.

[0091] In specific implementation, attention propagation is performed on the local attention of each cognate word in the word chain, and the attention interaction coefficient of each cognate word in the word chain can be obtained in the following way, namely: a graph structure is constructed based on a graph neural network, and each cognate word in the word chain is used as a node in the graph. The edges between the nodes are established according to the similarity between the word vectors corresponding to the cognate words, and the edge weight can be measured by the cosine similarity between the word vectors. Then, the initial feature of each node is set as the local attention of the node corresponding to the cognate word, and then the graph neural network is used to perform several rounds of message passing and feature aggregation operations on the graph structure, so that each node can fuse the local attention information from the neighboring nodes; finally, the final output feature of each node in the graph neural network is used as the semantic attention of the node corresponding to the cognate word, so as to obtain the attention interaction coefficient of each cognate word in the word chain.

[0092] It should be noted that graph neural networks are a type of deep learning model that specializes in processing graph-structured data. The core idea is to realize the propagation of information and aggregation of features in the graph through the edge connection relationship between nodes. Each node not only retains its own feature information, but also exchanges and fuses features with neighboring nodes through multiple rounds of "message passing" mechanism, thereby achieving semantic complementarity and dynamic enhancement in the context.

[0093] In addition, it should be noted that the semantic attention described in this application represents the vector of cognate words that recognize context changes in cross-sentence contexts.

[0094] In specific implementation, semantic weights are distributed to the local attention of each cognate word in the word chain through all attention interaction coefficients. The semantic attention of each cognate word in the word chain can be obtained in the following way, namely: for each cognate word in the word chain, the attention interaction coefficient of the cognate word is multiplied by the local attention of the cognate word, and then the multiplied vector is Softmax normalized, and the normalized vector is used as the semantic attention of the cognate word, thereby obtaining the semantic attention of each cognate word in the word chain.

[0095] In step 104, interactive semantic embedding is performed on the contextual information between the cognate words through the semantic attention of each cognate word in the word chain and the word chain semantic relationship between each cognate word to obtain the semantic embedding amount of each cognate word, and then document-level vocabulary translation is performed based on all the semantic embedding amounts.

[0096] In some embodiments, reference Figure 3 As shown in FIG, this figure is a flow chart of determining the semantic embedding amount in some embodiments of the present application. In this embodiment, the context information between each cognate word is interactively semantically embedded by the semantic attention of each cognate word in the word chain and the word chain semantic relationship between each cognate word. The semantic embedding amount of each cognate word can be obtained by the following steps:

[0097] First, in step 1041, the word chain semantic relationship between each cognate word in the word chain is determined;

[0098] Next, in step 1042, context information between the cognate words is interactively transferred through the word chain semantic relationship between the cognate words in the word chain to obtain a context interaction representation vector for each cognate word;

[0099] Then, in step 1043, semantic embedding is performed on each cognate word according to the semantic attention of each cognate word in the word chain and the context interaction representation vector of each cognate word to obtain the semantic embedding amount of each cognate word.

[0100] It should be noted that the word chain semantic relationship described in this application represents a parameter of the degree of semantic coherence between different cognate words in the same word chain at the semantic level.

[0101] In specific implementation, the word chain semantic relationship between each cognate word in the word chain can be determined in the following way: first, the Euclidean distance is calculated for the word vectors of any two cognate words in the word chain, and the obtained Euclidean distance is substituted into the Gaussian kernel function, and the result obtained by the Gaussian kernel function operation is used as the word chain semantic relationship between the two cognate words, that is, the word chain semantic relationship ,in, and Represent the word vectors of two different cognate words, Represents the bandwidth parameter, which is used to control the decay rate of the semantic relationship of the word chain. The bandwidth parameter can be set to the median of the Euclidean distance between the word vectors of two cognate words, that is: ,in, and are the components of the word vectors of two different cognate words, and then the word chain semantic relationship between each cognate word in the word chain is obtained.

[0102] It should be noted that the context information described in this application represents a collection of semantic and contextual information shared between cognate words in a word chain, and the context information specifically includes the vocabulary and sentences before and after the cognate words.

[0103] In addition, it should be noted that the context interaction representation vector described in this application refers to a vector that enhances the semantic expression of context information of cognate words in the case of semantic drift.

[0104] In specific implementation, the context information between each cognate word is interactively transmitted through the word chain semantic relationship between each cognate word in the word chain, and the context interaction representation vector of each cognate word is obtained. This can be achieved in the following way, namely: first, the context information between each cognate word is converted into a vector, and then all the vectors are arranged according to the arrangement order of each cognate word in the word chain, and then all the vectors are input into the recurrent neural network in sequence. Then, the word chain semantic relationship between each cognate word is used as the weight for adjusting the information transmission intensity between the hidden states of each cognate word in the recurrent neural network. During the recurrent neural network encoding process, the information flow is dynamically controlled through the gating mechanism, and the context information between each cognate word is focused on retaining and transmitting. After the encoding is completed, the hidden state vector of each cognate word is output through the recurrent neural network, and the hidden state vector of each cognate word is used as the context interaction representation vector, thereby obtaining the context interaction representation vector of each cognate word.

[0105] It should be noted that the recurrent neural network in this application is a type of neural network model specifically used to process sequence data. The recurrent neural network introduces a loop structure into the network so that the output at the current moment depends not only on the current input, but also on the hidden state at the previous moment, thereby being able to remember and utilize previous context information.

[0106] It should also be noted that the semantic embedding described in this application represents a vector that captures the semantic changes of cognate words due to semantic drift in the context of the target document.

[0107] In specific implementation, semantic embedding is performed on each cognate word according to the semantic attention of each cognate word in the word chain and the context interaction representation vector of each cognate word. The semantic embedding amount of each cognate word can be obtained by the following method, namely: first, the context interaction representation vector of each cognate word is spliced ​​with its corresponding semantic attention, and the spliced ​​vector is used as the enhanced vector of the corresponding cognate word. Then, the enhanced vector of the cognate word is input into the multi-head self-attention model, and then the output result of the multi-head self-attention model is residually connected with the enhanced vector, and layer normalization is performed. Then, the layer-normalized vector is input into the feedforward network, and the output result of the feedforward network is used as the semantic embedding amount of the cognate word, thereby obtaining the semantic embedding amount of each cognate word.

[0108] It should be noted that the multi-head self-attention model in this application is a mechanism that can capture the correlation between different positions in a sequence from multiple perspectives at the same time. The multi-head self-attention model maps the input vector to multiple different subspaces (called "heads") and calculates the attention weight of each head separately, thereby paying attention to different semantic features and contextual information in the sequence in parallel. The output results of each head are then spliced ​​and linearly transformed to obtain a representation that integrates multiple semantic perspectives. Multi-head self-attention not only enhances the model's ability to capture long-distance dependencies and complex semantic relationships, but also improves the richness and flexibility of information expression.

[0109] In some embodiments, document-level vocabulary translation based on all semantic embeddings can be achieved using the following steps:

[0110] Initialize the pre-trained large language model;

[0111] Each word in the target document is translated consistently based on the large language model and all semantic embeddings to obtain a translation result for each word.

[0112] It should be noted that the pre-trained large language model in this application refers to a neural network model trained through self-supervised tasks on a large-scale text corpus. Its core capability lies in understanding and modeling contextual relationships in language, and is particularly suitable for dealing with text-level semantic consistency and lexical ambiguity. The large language model used in this application is mBART (i.e. Multilingual BART). mBART can capture contextual information across sentences or even entire documents, thereby effectively identifying changes in the meaning of words in different contexts, and improving the translation accuracy and consistency of the same word in different contexts. By initializing these models and introducing semantic embeddings, more coherent and context-sensitive word translation can be achieved, which is the core foundation for the current implementation of document-level machine translation.

[0113] In specific implementation, each word in the target document is consistently translated based on the large language model and all semantic embeddings, and the translation result of each word can be obtained in the following manner, namely: first, the semantic embeddings of each cognate word in the word chain of each word are obtained, and the vocabulary vector of each word is input into the encoder of the pre-trained large language model. At the same time, the semantic embeddings of each cognate word in the word chain of each word are input into the encoder of the pre-trained large language model as additional context features. The decoder of the large language model generates a translation result for the corresponding word based on the context information output by the encoder, thereby obtaining the translation result of each word.

[0114] In addition, in another aspect of the present application, in some embodiments, the present application provides a machine translation system based on big data, the machine translation system includes a document-level vocabulary translation unit, reference Figure 4 , which is a schematic diagram of the structure of a document-level vocabulary translation unit according to some embodiments of the present application. The document-level vocabulary translation unit 400 includes: an acquisition module 401, a processing module 402, and an execution module 403, which are described as follows:

[0115] Acquisition module 401, in this application, is mainly used to acquire a target document and extract the global attention of the vocabulary in the target document during the translation process based on the global attention mechanism;

[0116] Processing module 402, in this application, is used to perform semantic role analysis on the words in the target document to obtain semantic role features of the words, and then construct word chains of the words based on the semantic role features and the semantic similarity between the words;

[0117] It should be noted that the processing module 402 in the present application is also used to determine the semantic contribution of each cognate word in the word chain to the target document topic, and to obtain the semantic attention of each cognate word in the word chain by performing attention interaction on each cognate word in the word chain through all semantic contributions and the global attention of the vocabulary during the translation process;

[0118] Execution module 403. In this application, execution module 403 is mainly used to interactively semantically embed the contextual information between each cognate word through the semantic attention of each cognate word in the word chain and the word chain semantic relationship between each cognate word, obtain the semantic embedding amount of each cognate word, and then perform document-level vocabulary translation based on all the semantic embedding amounts.

[0119] In addition, the present application also provides a computer device, which includes a memory and a processor, wherein the memory stores code, and the processor is configured to obtain the code and execute the above-mentioned document-level vocabulary translation method.

[0120] In some embodiments, reference Figure 5 , which is a schematic diagram of the structure of a computer device for implementing a document-level vocabulary translation method according to some embodiments of the present application. The document-level vocabulary translation method in the above embodiment can be Figure 5 The computer device 500 shown in FIG. 5 is implemented as shown in FIG. 5 . The computer device 500 includes at least one processor 501 , a communication bus 502 , a memory 503 , and at least one communication interface 504 .

[0121] The processor 501 may be a general-purpose central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more processors for controlling the execution of the document-level vocabulary translation method of the present application.

[0122] The communication bus 502 may be used to transmit information between the aforementioned components.

[0123] The memory 503 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, a random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, an optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), a magnetic disk or other magnetic storage device, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto. The memory 503 may be independent and connected to the processor 501 via the communication bus 502. The memory 503 may also be integrated with the processor 501.

[0124] The memory 503 is used to store program code for executing the solution of the present application, and is controlled by the processor 501. The processor 501 is used to execute the program code stored in the memory 503. The program code may include one or more software modules. The method described in the above method embodiment can be implemented by the processor 501 and one or more software modules in the program code in the memory 503.

[0125] The communication interface 504 uses any device such as a transceiver to communicate with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area network (WLAN), etc.

[0126] In a specific implementation, as an example, a computer device may include multiple processors, each of which may be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. A processor herein may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0127] The aforementioned computer device can be a general-purpose computer device or a dedicated computer device. In a specific implementation, the computer device can be a desktop computer, a portable computer, a network server, a personal digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device, or an embedded device. The embodiments of this application do not limit the type of computer device.

[0128] In addition, the present application also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned document-level vocabulary translation method is implemented.

[0129] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0130] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A document-level vocabulary translation method for performing vocabulary translation in a big data-based machine translation system, characterized in that: The method comprises the following steps: Obtain the target document and extract the global attention of the words in the target document during the translation process based on the global attention mechanism; Performing semantic role analysis on the words in the target document to obtain semantic role features of the words, and then constructing word chains of the words based on the semantic role features and the semantic similarity between the words; Determining the semantic contribution of each cognate word in the word chain to the target document topic, performing attention interaction on each cognate word in the word chain through all semantic contributions and the global attention of the vocabulary during the translation process, and obtaining the semantic attention of each cognate word in the word chain; By interactively embedding the contextual information between cognate words through the semantic attention of each cognate word in the word chain and the word chain semantic relationship between each cognate word, the semantic embedding of each cognate word is obtained, and then the document-level vocabulary translation is performed based on all the semantic embeddings; The semantic role analysis of the words in the target document is performed to obtain the semantic role features of the words, including: Perform syntactic dependency analysis on the sentence containing the word in the target document to obtain the dependency relationship between the word and other words in the sentence; Label the semantic roles of words through all dependency relationships; Determining the semantic role characteristics of the vocabulary according to the semantic role label; The step of constructing a word chain based on the semantic role features and the semantic similarity between words specifically includes: Perform threshold screening on all words in the target document according to the semantic role features to obtain a candidate word set; Screening the candidate word set for cognates based on semantic similarity between words to obtain a cognate word set; Based on the index position of each cognate word in the target document, all cognate words in the cognate word set are linked in index order to obtain a word chain of the vocabulary; Among them, the context information between each cognate word is interactively semantically embedded through the semantic attention of each cognate word in the word chain and the word chain semantic relationship between each cognate word, and the semantic embedding amount of each cognate word is obtained specifically including: Determine the semantic relationship between each cognate word in the word chain; The context information between the cognate words is interactively transferred through the semantic relationship between the cognate words in the word chain to obtain the context interaction representation vector of each cognate word; According to the semantic attention of each cognate word in the word chain and the context interaction representation vector of each cognate word, the semantic embedding of each cognate word is obtained.

2. The method according to claim 1, wherein The global attention mechanism is used to extract the global attention of the words in the target document during the translation process. Specifically, it includes: Vectorize each word in the target document to obtain the word vector for each word; Based on the global attention mechanism, the attention weight of each word to other words in the target document is determined by the word vector of each word; The global attention of each word in the target document during translation is extracted from all attention weights.

3. The method according to claim 1, wherein Determining the semantic contribution of each cognate word in the word chain to the target document topic specifically includes: Extract topic keywords from the target document based on the preset document topic model; Determining the semantic association strength and co-occurrence frequency between each cognate word in the word chain and the subject keyword; The semantic contribution of each cognate word in the word chain to the target document topic is determined according to the semantic association strength and co-occurrence frequency between each cognate word in the word chain and the topic keyword.

4. The method according to claim 1, wherein By using all semantic contributions and the global attention of the vocabulary during the translation process to interact with each cognate word in the word chain, the semantic attention of each cognate word in the word chain is obtained, which specifically includes: Determining the local attention of each cognate word in the word chain according to the semantic contribution of each cognate word in the word chain to the target document topic and the global attention of the vocabulary during the translation process; Perform attention propagation on the local attention of each cognate word in the word chain to obtain the attention interaction coefficient of each cognate word in the word chain; The semantic weights of the local attention of each cognate word in the word chain are distributed through all the attention interaction coefficients to obtain the semantic attention of each cognate word in the word chain.

5. The method according to claim 1, wherein Document-level vocabulary translation based on all semantic embeddings specifically includes: Initialize the pre-trained large language model; Each word in the target document is translated consistently based on the large language model and all semantic embeddings to obtain a translation result for each word.

6. The method according to claim 1, wherein The target document is a technical document in English.

7. A machine translation system based on big data, which uses the method according to any one of claims 1 to 6 to perform document-level vocabulary translation, the machine translation system including a document-level vocabulary translation unit, characterized in that: The document-level vocabulary translation unit includes: The acquisition module is used to obtain the target document and extract the global attention of the words in the target document during the translation process based on the global attention mechanism; A processing module, configured to perform semantic role analysis on the words in the target document to obtain semantic role features of the words, and then construct a word chain of the words based on the semantic role features and the semantic similarity between the words; The processing module is further configured to determine the semantic contribution of each cognate word in the word chain to the target document topic, and to obtain the semantic attention of each cognate word in the word chain by performing attention interaction on each cognate word in the word chain through all semantic contributions and the global attention of the vocabulary during the translation process; The execution module is used to interactively semantically embed the contextual information between cognate words through the semantic attention of each cognate word in the word chain and the word chain semantic relationship between each cognate word, obtain the semantic embedding amount of each cognate word, and then perform document-level vocabulary translation based on all the semantic embedding amounts.

Images