Machine translation system and method based on artificial intelligence

Abstract

The invention provides a machine translation system and method based on artificial intelligence, and the method comprises the steps: carrying out the word segmentation of each sentence in a source language text through the semantic complexity of the sentences, and obtaining a word set of each sentence; determining the semantic contribution degree of each sentence on the sentence level in the machine translation process according to the co-occurrence distance and the syntactic dependency relationship among the words in each sentence; further determining the semantic association degree between each sentence and all other sentences on a chapter structure, and determining the chapter-level translation attention of each sentence in the machine translation process through all the semantic association degrees and the semantic complexity of each sentence; and performing text-level machine translation on the source language text based on the semantic contribution degree of each sentence on the sentence level and the translation attention of each sentence on the text level to obtain a target language text. By means of the scheme, semantic association translation between chapter-level sentences can be achieved, and therefore the semantic accuracy of machine translation is improved.

Description

Technical Field

[0001] The present application relates to the technical field of machine translation, and more specifically, to an artificial intelligence-based machine translation system and method. Background Art

[0002] As an important branch of natural language processing, machine translation (MT) has made remarkable progress in recent years. The development of deep learning technology has made neural machine translation (NMT) the mainstream, especially the Transformer model based on recurrent neural networks and self-attention mechanism, which has greatly improved the fluency and semantic accuracy of translation. Neural machine translation can learn translation patterns at the word, phrase and sentence levels from data through end-to-end learning, significantly reducing the reliance on artificial rules, and can better capture contextual information and long-distance dependencies, thereby generating more natural and fluent translations.

[0003] In the prior art, machine translation mainly relies on neural network-based translation models, which are trained through a large number of parallel corpora to achieve translation of source language texts. However, these translation models usually have problems with insufficient understanding of semantic associations and contexts between sentences. Especially when dealing with long sentences, complex sentence structures or polysemous words, the translation results are prone to semantic deviations or unnatural situations. At the same time, existing translation models often ignore the association between sentences at the paragraph level and fail to fully consider the impact of contextual information on translation. In contrast, improving the semantic coherence between sentences through a paragraph-level translation attention mechanism can reduce the impact of semantic inaccuracy and insufficient context understanding on the translation of source language texts in the prior art. Therefore, how to achieve semantic association translation between sentences at the paragraph level, thereby improving the semantic accuracy of machine translation, has become a difficult problem faced by the industry. Summary of the invention

[0004] The present application provides an artificial intelligence-based machine translation system and method, which can realize the semantic association translation between paragraph-level sentences, thereby improving the semantic accuracy of machine translation.

[0005] In a first aspect, the present application provides a machine translation method based on artificial intelligence, comprising the following steps:

[0006] Obtain source language text for machine translation;

[0007] Extracting the semantic complexity of each sentence in the source language text based on an artificial intelligence semantic model, and performing adaptive word segmentation processing on each sentence in the source language text according to the semantic complexity of each sentence to obtain a word set of each sentence;

[0008] For each sentence, determine the co-occurrence distance between each word in the word set, and then determine the semantic contribution of the sentence at the sentence level in the machine translation process based on all the co-occurrence distances and the syntactic dependency between each word, and then obtain the semantic contribution of each sentence at the sentence level in the machine translation process;

[0009] Based on the context information of the source language text, a semantic association analysis of the text structure is performed on each sentence to obtain the semantic association between each sentence and all other sentences in the text structure, and the translation attention of each sentence at the text level during the machine translation process is determined by all the semantic associations and the semantic complexity of each sentence;

[0010] Based on the semantic contribution of each sentence at the sentence level and the translation attention of each sentence at the paragraph level, the source language text is machine translated at the paragraph level to obtain the target language text.

[0011] Preferably, extracting the semantic complexity of each sentence in the source language text based on the artificial intelligence semantic model specifically includes:

[0012] Build artificial intelligence-based semantic models;

[0013] Using the source language text as an initialization parameter of the semantic model;

[0014] The semantic complexity of each sentence in the source language text is determined by the semantic model.

[0015] Preferably, performing adaptive word segmentation processing on each sentence in the source language text according to the semantic complexity of each sentence to obtain a word set for each sentence specifically includes:

[0016] For each sentence, the word segmentation granularity of the word segmentation strategy in the preset word segmentation model is determined according to the semantic complexity of the sentence;

[0017] According to the word segmentation granularity, the sentences are adaptively segmented by a preset word segmentation model to obtain the word set of the sentence, and then the word set of each sentence is obtained.

[0018] Preferably, determining the co-occurrence distance between each word in the word set specifically includes:

[0019] Select a word from the word set as a selected word;

[0020] Determine the co-occurrence frequency between the selected word and every other word;

[0021] Determine the inter-word distance between the selected word and every other word;

[0022] The co-occurrence distance between the selected word and all other words is determined by all co-occurrence frequencies and all inter-word distances, and the co-occurrence distance between the remaining words and all other words is continued to be determined.

[0023] Preferably, determining the semantic contribution of a sentence at the sentence level during machine translation based on all co-occurrence distances and the syntactic dependencies between each word specifically includes:

[0024] Select a word from the word set as a selected word;

[0025] Get the co-occurrence distance between the selected word and all other words;

[0026] Determine the syntactic dependency between the selected word and all other words based on the source language knowledge graph;

[0027] Determine the attention coefficient of the selected word in the machine translation process by the co-occurrence distance and syntactic dependency between the selected word and all other words, and continue to determine the attention coefficients of the remaining words in the machine translation process;

[0028] The semantic contribution of a sentence at the sentence level during machine translation is determined based on the attention coefficients of all words.

[0029] Preferably, determining the translation attention of each sentence at the paragraph level during the machine translation process by all semantic relevances and the semantic complexity of each sentence specifically includes:

[0030] Select a sentence as the selected sentence;

[0031] Determine the priority of the selected sentence in the machine translation process according to the semantic relevance between the selected sentence and all other sentences in the text structure;

[0032] By selecting the semantic complexity of the sentence, the priority is constrained in importance, so as to obtain the translation attention of the selected sentence at the paragraph level during the machine translation process;

[0033] Continue to determine the translation attention of the remaining sentences at the paragraph level during the machine translation process.

[0034] Preferably, performing a paragraph-level machine translation on the source language text based on the semantic contribution of each sentence at the sentence level and the translation attention of each sentence at the paragraph level to obtain the target language text specifically includes:

[0035] Build a machine translation model for paragraph-level translation;

[0036] The semantic contribution of each sentence at the sentence level is set as the sentence-level translation weight in the machine translation model;

[0037] Setting the translation attention of each sentence at the paragraph level as the translation weight at the paragraph level in the machine translation model;

[0038] The source language text is translated into a target language text by using the machine translation model.

[0039] In a second aspect, the present application provides a machine translation system based on artificial intelligence, comprising:

[0040] An acquisition module, used to acquire source language text in machine translation;

[0041] A processing module, configured to extract the semantic complexity of each sentence in the source language text based on an artificial intelligence semantic model, and perform adaptive word segmentation processing on each sentence in the source language text according to the semantic complexity of each sentence to obtain a word set of each sentence;

[0042] The processing module is further used to determine, for each sentence, the co-occurrence distance between each word in the word set, and then determine the semantic contribution of the sentence at the sentence level in the machine translation process based on all the co-occurrence distances and the syntactic dependency between each word, and then obtain the semantic contribution of each sentence at the sentence level in the machine translation process;

[0043] The processing module is further used to perform a semantic association analysis of the text structure on each sentence based on the context information of the source language text, obtain the semantic association between each sentence and all other sentences in the text structure, and determine the translation attention of each sentence at the text level during the machine translation process through all the semantic associations and the semantic complexity of each sentence;

[0044] The execution module is used to perform a machine translation of the source language text at the paragraph level based on the semantic contribution of each sentence at the sentence level and the translation attention of each sentence at the paragraph level to obtain a target language text.

[0045] In a third aspect, the present application provides a computer device, comprising a memory and a processor, wherein the memory stores codes, and the processor is configured to obtain the codes and execute the above-mentioned artificial intelligence-based machine translation method.

[0046] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned artificial intelligence-based machine translation method is implemented.

[0047] The technical solution provided by the embodiments disclosed in this application has the following beneficial effects:

[0048] In an embodiment of the present application, a source language text in machine translation is obtained; the semantic complexity of each sentence in the source language text is extracted based on a semantic model of artificial intelligence, and adaptive word segmentation processing is performed on each sentence in the source language text according to the semantic complexity of each sentence to obtain a word set of each sentence; for each sentence, the co-occurrence distance between each word in the word set is determined, and then the semantic contribution of the sentence at the sentence level in the machine translation process is determined according to all the co-occurrence distances and the syntactic dependency relationship between each word, and then the semantic contribution of each sentence at the sentence level in the machine translation process is obtained; based on the context information of the source language text, a semantic association analysis of the chapter structure is performed on each sentence to obtain the semantic association between each sentence and all other sentences in the chapter structure, and the translation attention of each sentence at the chapter level in the machine translation process is determined according to all the semantic associations and the semantic complexity of each sentence; the source language text is subjected to chapter-level machine translation based on the semantic contribution of each sentence at the sentence level and the translation attention of each sentence at the chapter level to obtain a target language text.

[0049] It can be seen that the present application performs chapter-level machine translation on the source language text based on the semantic contribution of each sentence at the sentence level and the translation attention of each sentence at the chapter level to obtain the target language text; first, at the sentence level, the semantic contribution of each sentence at the sentence level in the machine translation process is determined according to all co-occurrence distances and the syntactic dependencies between each word, and the semantic contribution of each sentence in the entire translation process is determined by analyzing the co-occurrence distances and syntactic dependencies between words, which can realize dynamic adjustment of the importance and translation priority of different sentences, and help to improve the translation quality of long or complex sentences, thereby reducing sentence semantic deviations; then, at the chapter level, the translation attention of each sentence at the chapter level in the machine translation process is determined by all semantic associations and the semantic complexity of each sentence, and the contextual information in the source language text is analyzed to determine the translation attention of each sentence in the chapter. The semantic relevance of each sentence is evaluated, so that different translation attention is allocated to each sentence, which can ensure the coherence and contextual consistency between sentences, so that the translation result is not only more accurate at the single sentence level, but also maintains the overall semantic consistency and fluency at the chapter level, which can reduce the translation deviation caused by semantic breaks and context incoherence commonly seen in the prior art; finally, the scheme performs semantic association translation on the source language text through the semantic contribution and translation attention at the sentence level and the chapter level, and by fully considering the semantic relationship between sentences and the chapter as a whole, it can significantly improve the performance of the machine translation system in processing complex texts, enhance the naturalness and semantic accuracy of the translation, thereby improving the translation quality in practical applications and meeting diverse and high-quality translation needs; in summary, the embodiments of the present application can realize semantic association translation between sentences at the chapter level, thereby improving the semantic accuracy of machine translation. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 is an exemplary flow chart of an artificial intelligence-based machine translation method according to some embodiments of the present application;

[0051] Figure 2 is a schematic diagram of the structure of a codec according to some embodiments of the present application;

[0052] Figure 3 is a schematic diagram of a process for determining a co-occurrence distance according to some embodiments of the present application;

[0053] Figure 4 is a schematic diagram of the structure of an artificial intelligence-based machine translation system according to some embodiments of the present application;

[0054] Figure 5 It is a structural diagram of a computer device for implementing an artificial intelligence-based machine translation method according to some embodiments of the present application. DETAILED DESCRIPTION

[0055] In order to better understand the technical solution of the present application, the technical solution of the present application will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0056] refer to Figure 1 , which is an exemplary flow chart of an artificial intelligence-based machine translation method according to some embodiments of the present application. The artificial intelligence-based machine translation method 100 mainly includes the following steps:

[0057] In step 101, a source language text in machine translation is obtained.

[0058] In specific implementation, the source language text refers to the source language text data to be translated into the target language. The source language text data can be, for example, text data containing the source language translated by a machine translation device, wherein the source language can be Chinese, English, German and Korean, and the target language can also be Chinese, English, German and Korean, which are not specifically limited here.

[0059] It should be noted that the machine translation method in this application can translate between two or more languages, which will not be elaborated here.

[0060] In step 102, the semantic complexity of each sentence in the source language text is extracted based on an artificial intelligence-based semantic model, and adaptive word segmentation processing is performed on each sentence in the source language text according to the semantic complexity of each sentence to obtain a word set for each sentence.

[0061] It should be noted that the semantic model in this application can be obtained by training based on the bidirectional Transformer architecture. The structural diagram of the encoder-decoder in the Transformer architecture can be referenced. Figure 2 As shown, this figure is a schematic diagram of the structure of the encoder-decoder in some embodiments of the present application. The input first passes through the encoder, and the encoder compresses the input data into an abstract representation. Then, the decoder converts this abstract representation back to the original data space to generate an output. It should be noted that the encoder-decoder in the embodiments of the present application adopts a double-layer attention mechanism, which is introduced in the subsequent embodiments and will not be repeated here. In addition, the semantic model based on artificial intelligence in the present application aims to capture and represent the semantic information in the text through deep learning and language understanding technology. The semantic model can process complex structures in the language, help machines understand, reason, generate and translate text, and is widely used in machine translation, sentiment analysis, speech recognition and other fields.

[0062] In some embodiments, extracting the semantic complexity of each sentence in the source language text based on an artificial intelligence semantic model may be implemented by the following steps:

[0063] Build artificial intelligence-based semantic models;

[0064] Using the source language text as an initialization parameter of the semantic model;

[0065] The semantic complexity of each sentence in the source language text is determined by the semantic model.

[0066] In specific implementation, first, a semantic model can be designed based on the bidirectional Transformer architecture, and the semantic model can analyze the words, syntactic structure and contextual relationships of each sentence in the source language text, and then quantify the complexity of each sentence at the semantic level; then, the content of the source language text is used as the initial condition for the input of the semantic model, and by passing the sentence structure, vocabulary and contextual information in the source language text to the pre-trained semantic model, the semantic model can perform further semantic understanding and processing based on this information; finally, each sentence is subjected to in-depth semantic analysis through the semantic model to evaluate its complexity at the semantic level. This process usually involves comprehensive consideration of the vocabulary, grammatical structure, dependency relationship and contextual information in the sentence to quantify the difficulty of the sentence in semantic expression, and the semantic complexity of each sentence in the source language text is output through the semantic model.

[0067] It should be noted that the semantic complexity in this application is an indicator to measure the complexity of a sentence at the semantic level. A sentence structure with higher semantic complexity usually has more semantic levels and dependencies, while a sentence structure with lower semantic complexity is more straightforward and simple in structure.

[0068] In some embodiments, adaptive word segmentation is performed on each sentence in the source language text according to the semantic complexity of each sentence to obtain a word set for each sentence, which can be achieved by the following steps:

[0069] For each sentence, the word segmentation granularity of the word segmentation strategy in the preset word segmentation model is determined according to the semantic complexity of the sentence;

[0070] According to the word segmentation granularity, the sentences are adaptively segmented by a preset word segmentation model to obtain the word set of the sentence, and then the word set of each sentence is obtained.

[0071] It should be noted that in this application, by introducing the self-attention mechanism of word segmentation granularity, more detailed word segmentation processing can be performed on syntactically complex sentences, and more lightweight word segmentation processing can be performed on syntactically simple sentences, thereby improving the word segmentation accuracy of the word segmentation model for sentences.

[0072] In specific implementation, for each sentence, first, the natural exponential function value of the inverse of the semantic complexity of the sentence can be used as the fine-grained adjustment coefficient in the word segmentation strategy, and the product of the basic fine-grainedness preset in the word segmentation strategy and the adjustment coefficient can be used as the word segmentation granularity of the word segmentation strategy in the preset word segmentation model. It should be noted that the basic fine-grainedness preset in the word segmentation strategy can be determined based on the historical word segmentation data. In the implementation of this application, the average value of the fine-grainedness of all historical word segmentations can be used as the basic fine-grainedness. In other embodiments, other methods can also be used to preset the basic fine-grainedness, which is not limited here; then, the obtained word segmentation granularity is used as a conditional parameter of the preset word segmentation model. Through this conditional parameter, the word segmentation model can be guided to determine the degree of refinement of the word segmentation processing of the sentence, and then the sentence is segmented through the preset word segmentation model, and the set of all words obtained by the segmentation is used as the word set of the sentence. Through the above steps, the word set of each sentence can be obtained.

[0073] It should be noted that the adaptive word segmentation processing in the present application dynamically adjusts the word segmentation granularity according to the semantic complexity of the sentence, so that the machine translation system can process different types of sentences more accurately. For sentences with high semantic complexity, fine-grained word segmentation can retain more lexical details and syntactic structures, thereby improving semantic understanding ability and reducing information loss. For sentences with low semantic complexity, coarse-grained word segmentation is used to improve processing efficiency. Through the adaptive word segmentation method, it can ensure that the machine translation system neither ignores details nor avoids excessive cutting when dealing with various sentences, thereby improving the accuracy, fluency and naturalness of the translation.

[0074] In step 103, for each sentence, the co-occurrence distance between each word in the word set is determined, and then the semantic contribution of the sentence at the sentence level in the machine translation process is determined based on all the co-occurrence distances and the syntactic dependencies between each word, and then the semantic contribution of each sentence at the sentence level in the machine translation process is obtained.

[0075] In some embodiments, reference Figure 3 As shown, this figure is a schematic diagram of the process of determining the co-occurrence distance in some embodiments of the present application. In this embodiment, determining the co-occurrence distance between each word in the word set can be achieved by using the following steps:

[0076] In step 1031, a word is selected from the word set as a selected word;

[0077] In step 1032, the co-occurrence frequency between the selected word and each of the other words is determined;

[0078] In step 1033, determine the inter-word distance between the selected word and each of the other words;

[0079] In step 1034, the co-occurrence distances between the selected word and all other words are determined by using all co-occurrence frequencies and all inter-word distances, and the co-occurrence distances between the remaining words and all other words are continued to be determined.

[0080] In the specific implementation, first, a word is selected from the word set and recorded as the selected word; secondly, by traversing all other words related to the selected word in the sentence, the co-occurrence frequency of each pair of words is counted. The co-occurrence frequency can be calculated by a sliding window method within a fixed window size, and the words in the window are considered to be co-occurring. For example, a window of size 3 is used, and the selected word and the words in the window appear together and are considered to be co-occurring; then, based on the word sequence number in the sentence, the relative position difference between the selected word and other words in the sentence can be counted, and the relative position difference can be used as the inter-word distance between the selected word and other words. For example, assuming that the selected word is in the third position and the other word is in the fifth position, the inter-word distance is 2, and the inter-word distance between the selected word and each other word can be counted; finally, the product of the co-occurrence frequency between the selected word and each other word and the natural exponential function value of the opposite number of the inter-word distance can be used as the dependency distance between the selected word and each other word, and then the average value of all dependency distances is used as the co-occurrence distance between the selected word and all other words. Repeat the above steps to calculate the co-occurrence distance between the remaining words and all other words.

[0081] It should be noted that the co-occurrence distance in this application is a metric to measure the strength of the co-occurrence relationship between words. A smaller co-occurrence distance usually indicates that the words have a stronger semantic association, and vice versa, it indicates that the association between them is weaker.

[0082] In some embodiments, determining the semantic contribution of a sentence at the sentence level during machine translation based on all co-occurrence distances and syntactic dependencies between words can be implemented by the following steps:

[0083] Select a word from the word set as a selected word;

[0084] Get the co-occurrence distance between the selected word and all other words;

[0085] Determine the syntactic dependency between the selected word and all other words based on the source language knowledge graph;

[0086] Determine the attention coefficient of the selected word in the machine translation process by the co-occurrence distance and syntactic dependency between the selected word and all other words, and continue to determine the attention coefficients of the remaining words in the machine translation process;

[0087] The semantic contribution of a sentence at the sentence level during machine translation is determined based on the attention coefficients of all words.

[0088] It should be noted that the source language knowledge graph in this application refers to the language knowledge graph of the source language. The language knowledge graph may be FrameNet, and in other embodiments it may also be other knowledge graphs, which is not limited here.

[0089] In the specific implementation, first, a word is selected from the word set as the selected word, and the co-occurrence distance between the selected word and all other words is obtained from the above embodiment, which will not be repeated here; secondly, a semantic analysis model based on the source language knowledge graph is preset, and the semantic analysis model is a machine model trained based on a large amount of source language text data, and then the sentence is used as the input of the semantic analysis model, and the selected words in the sentence (the expression form of the sentence is a word set) are marked, and the semantic analysis model can be used to learn and analyze the dependency of the marked word and each other word in the syntactic structure, and all the dependencies are arranged in a symmetrical manner. Matrix, and then use the arranged symmetric matrix as the dependency matrix of the selected word, so that the syntactic dependency relationship between the selected word and all other words can be described by the dependency matrix; then, the average value of all dependencies in the dependency matrix can be used as the relationship weight of the syntactic dependency relationship, and then the product of the relationship weight and the co-occurrence distance between other words can be used as the attention coefficient of the selected word in the machine translation process, and the words in the word set can be repeatedly selected to determine the attention coefficients of the remaining words in the machine translation process; finally, the average value of the attention coefficients of all words can be used as the semantic contribution of the sentence at the sentence level in the machine translation process.

[0090] It should be noted that the syntactic dependency in this application is an indicator that quantifies the grammatical structure and semantic relationship between each word in a sentence. It reveals the grammatical function and hierarchical structure of each word component in a sentence by expressing the dependency relationship between words, thereby helping to understand the interaction between words in a sentence; in addition, the semantic contribution in this application is an indicator that measures the importance of word components in a sentence to the overall semantic expression of the sentence. By determining the semantic contribution of a sentence at the sentence level during machine translation, it helps to identify and emphasize key information in a sentence, ensuring that the translation model can give priority to the most semantically important parts when processing text, thereby improving the quality of translation, generation, and information extraction.

[0091] In step 104, a semantic association analysis of the text structure is performed on each sentence based on the context information of the source language text to obtain the semantic association between each sentence and all other sentences in the text structure, and the translation attention of each sentence at the text level during the machine translation process is determined through all the semantic associations and the semantic complexity of each sentence.

[0092] In some embodiments, the semantic association analysis of the text structure is performed on each sentence based on the context information of the source language text to obtain the semantic association between each sentence and all other sentences in the text structure. The following steps can be used to achieve this:

[0093] Selecting a sentence from the source language text as a selected sentence;

[0094] Determining a textual relationship between the selected sentence and each of the other sentences based on contextual information of the source language text;

[0095] Determine the semantic similarity between the selected sentence and every other sentence;

[0096] The semantic relevance between the selected sentence and all other sentences in the text structure is determined by the semantic similarity and text relationship between the selected sentence and each other sentence, and the semantic relevance between the remaining sentences and all other sentences in the text structure is further determined.

[0097] In the specific implementation, first, a sentence is selected from the source language text as the selected sentence; secondly, an implicit chapter relationship recognition model is initialized. The implicit chapter relationship recognition model can be trained based on a deep learning network. During the training process, the implicit chapter relationship recognition model is trained by annotated data (including sentence pairs with clear chapter relationships) to learn the implicit relationships between different sentences (such as causality and progression). Further, the implicit chapter relationship recognition model will encode each pair of sentences and predict the chapter relationship between them through their context information. The context information of the source language text can be used as the input parameter of the implicit chapter relationship recognition model, and then the implicit chapter relationship recognition model is used to learn the chapter relationship between the selected sentence and each other sentence. The chapter relationship specifically includes causality and progression, which needs to be explained. In the embodiment of the present application, the chapter relationship is numerically mapped to obtain the mapping value of the chapter relationship, and the mapping value can reflect the strength of the chapter relationship between sentences. It should also be noted that in other embodiments, the chapter relationship also includes other types of relationships, which are not specifically limited here; then, the semantic similarity between the selected sentence and each other sentence can be quantified by the cosine similarity in the existing semantic matching model; finally, the semantic similarity between the selected sentence and each other sentence and the mapping value of the chapter relationship can be weighted averaged, and the value obtained by the weighted average is used as the semantic association between the selected sentence and all other sentences in the chapter structure, wherein the semantic similarity is used as the weighting coefficient of the mapping value of the chapter relationship. Repeat the above steps to continue to determine the semantic association between the remaining sentences and all other sentences in the chapter structure.

[0098] It should be noted that the semantic relevance in this application is an indicator to measure the degree of semantic relevance between sentences in the text structure; the text relationship refers to the logical and structural connection between sentences in the text.

[0099] In some embodiments, determining the translation attention of each sentence at the paragraph level during the machine translation process by all semantic relevances and the semantic complexity of each sentence can be implemented by the following steps:

[0100] Select a sentence as the selected sentence;

[0101] Determine the priority of the selected sentence in the machine translation process according to the semantic relevance between the selected sentence and all other sentences in the text structure;

[0102] By selecting the semantic complexity of the sentence, the priority is constrained in importance, so as to obtain the translation attention of the selected sentence at the paragraph level during the machine translation process;

[0103] Continue to determine the translation attention of the remaining sentences at the paragraph level during the machine translation process.

[0104] In the specific implementation, first, a sentence is selected from the source language text as the selected sentence; then, the semantic correlation between each sentence and all other sentences in the text structure can be obtained, and the sum of all semantic correlations is taken as the sum of correlations, and then the ratio between the semantic correlation between the selected sentence and all other sentences in the text structure and the sum of the correlations is taken as the priority of the selected sentence in the machine translation process; finally, the semantic complexity of the selected sentence is taken as the constraint weight of the importance constraint within the sentence. The greater the semantic complexity, the greater the constraint weight of the importance constraint within the sentence, indicating that the attention to the words in the sentence is greater. Then the product of the constraint weight and the priority is taken as the translation attention of the selected sentence at the text level in the machine translation process. Repeating the above steps can continue to determine the translation attention of the remaining sentences at the text level in the machine translation process.

[0105] It should be noted that the translation attention in this application is an indicator to measure the degree of attention required for a sentence in the process of machine translation. Translation attention can adjust the degree of attention of the machine translation model to different parts according to the characteristics of the source language, thereby improving the translation adaptability in the process of machine translation.

[0106] In step 105, the source language text is machine translated at the paragraph level based on the semantic contribution of each sentence at the sentence level and the translation attention of each sentence at the paragraph level to obtain the target language text.

[0107] In some embodiments, based on the semantic contribution of each sentence at the sentence level and the translation attention of each sentence at the paragraph level, the source language text is subjected to paragraph-level machine translation to obtain the target language text, which can be achieved by the following steps:

[0108] Build a machine translation model for paragraph-level translation;

[0109] The semantic contribution of each sentence at the sentence level is set as the sentence-level translation weight in the machine translation model;

[0110] Setting the translation attention of each sentence at the paragraph level as the translation weight at the paragraph level in the machine translation model;

[0111] The source language text is translated into a target language text by using the machine translation model.

[0112] In the specific implementation, first, a machine translation model based on deep learning is constructed. Common architectures include Transformer. The machine translation model can handle long-distance dependencies and contextual information and is suitable for paragraph-level translation. Then, a dual attention network is designed in the machine translation model. The dual attention network specifically includes a sentence-level attention network and a paragraph-level attention network. The semantic contribution of each sentence at the sentence level is set as the translation weight of the sentence-level corresponding attention network in the machine translation model, and the translation attention of each sentence at the paragraph level is set as the translation weight of the paragraph-level corresponding attention network in the machine translation model. Finally, the source language text is input into the machine translation model, and the target language text is obtained by translation through the machine translation model.

[0113] It should be noted that the present application scheme provides multi-dimensional optimization for the machine translation process by introducing semantic complexity, syntactic dependency, co-occurrence distance and semantic association analysis in the text structure; firstly, by accurately extracting the semantic complexity of each sentence and combining it with adaptive word segmentation processing, it can better adapt to the diversity of the source language text and improve the translation quality at the sentence level. The analysis of semantic complexity enables the translation model to understand the inherent difficulty of each sentence and adjust the word segmentation strategy according to its complexity, thereby avoiding translation errors when processing complex or polysemous sentences; secondly, by conducting in-depth analysis of the co-occurrence distance and syntactic dependency between words, the translation model can identify the semantic structure within the sentence and accurately calculate the semantic contribution of each sentence. This mechanism ensures that at the sentence level, the translation model can better capture the key components and semantic focus, thereby optimizing the translation results at the sentence level and reducing language errors. Then, in terms of paragraph-level optimization, the semantic relevance of each sentence is evaluated through in-depth mining of contextual information, and the translation attention of the sentence in the entire paragraph is determined based on these relevances. The paragraph-level translation attention mechanism helps the translation model to maintain semantic consistency and coherence when processing long texts, avoids the problem of context disconnection in the translation process of a single sentence, and makes the translation result more natural and fluent; finally, the present application scheme improves the performance of the machine translation system in processing complex texts through multi-level semantic analysis and context association mechanisms, significantly enhances the accuracy and fluency of translation, especially in the translation of multiple sentences and long texts, can ensure the semantic consistency, contextual coherence and grammatical correctness of the translation results, provides a more intelligent and flexible translation solution, and solves the problems of semantic inaccuracy and insufficient context understanding in traditional translation models.

[0114] On the other hand, in some embodiments, the present application provides a machine translation system based on artificial intelligence, referring to Figure 4, which is a schematic diagram of the structure of an artificial intelligence-based machine translation system according to some embodiments of the present application. The artificial intelligence-based machine translation system 400 includes: an acquisition module 401, a processing module 402 and an execution module 403, which are described as follows:

[0115] Acquisition module 401, in this application, acquisition module 401 is mainly used to acquire source language text in machine translation;

[0116] Processing module 402, in the present application, is used to extract the semantic complexity of each sentence in the source language text based on an artificial intelligence semantic model, and perform adaptive word segmentation processing on each sentence in the source language text according to the semantic complexity of each sentence to obtain a word set of each sentence;

[0117] The processing module 402 in the present application is also used to determine the co-occurrence distance between each word in the word set for each sentence, and then determine the semantic contribution of the sentence at the sentence level in the machine translation process according to all the co-occurrence distances and the syntactic dependency relationship between each word, and then obtain the semantic contribution of each sentence at the sentence level in the machine translation process;

[0118] The processing module 402 in the present application is also used to perform a semantic association analysis of the text structure on each sentence based on the context information of the source language text, obtain the semantic association between each sentence and all other sentences in the text structure, and determine the translation attention of each sentence at the text level during the machine translation process through all the semantic associations and the semantic complexity of each sentence;

[0119] Execution module 403, in the present application, execution module 403 is mainly used to perform machine translation of the source language text at the chapter level based on the semantic contribution of each sentence at the sentence level and the translation attention of each sentence at the chapter level to obtain the target language text.

[0120] In addition, the present application also provides a computer device, which includes a memory and a processor, the memory stores code, and the processor is configured to obtain the code and execute the above-mentioned artificial intelligence-based machine translation method.

[0121] In some embodiments, reference Figure 5 , which is a schematic diagram of the structure of a computer device for implementing an artificial intelligence-based machine translation method according to some embodiments of the present application. The artificial intelligence-based machine translation method in the above embodiment can be Figure 5 The computer device 500 shown in the figure is implemented, and the computer device 500 includes at least one processor 501, a communication bus 502, a memory 503 and at least one communication interface 504.

[0122] The processor 501 may be a general-purpose central processing unit (CPU) or an application-specific integrated circuit (ASIC).

[0123] The communication bus 502 may be used to transmit information between the above-mentioned components.

[0124] The memory 503 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a disk or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 503 may exist independently and be connected to the processor 501 via the communication bus 502. The memory 503 may also be integrated with the processor 501.

[0125] The memory 503 is used to store the program code for executing the solution of the present application, and the execution is controlled by the processor 501. The processor 501 is used to execute the program code stored in the memory 503. The program code may include one or more software modules. The machine translation method based on artificial intelligence in the above embodiment can be implemented by the processor 501 and one or more software modules in the program code in the memory 503.

[0126] The communication interface 504 uses any transceiver or other device for communicating with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area networks (WLAN), etc.

[0127] In a specific implementation, as an embodiment, a computer device may include multiple processors, each of which may be a single-CPU processor or a multi-CPU processor. The processor here may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0128] The above-mentioned computer device can be a general-purpose computer device or a special-purpose computer device. In a specific implementation, the computer device can be a desktop computer, a portable computer, a network server, a personal digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device or an embedded device. The embodiment of the present application does not limit the type of computer device.

[0129] In addition, the present application also provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the above-mentioned artificial intelligence-based machine translation method is implemented.

[0130] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0131] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A machine translation method based on artificial intelligence, characterized in that: The steps include: Obtain source language text for machine translation; Extracting the semantic complexity of each sentence in the source language text based on an artificial intelligence semantic model, and performing adaptive word segmentation processing on each sentence in the source language text according to the semantic complexity of each sentence to obtain a word set of each sentence; For each sentence, determine the co-occurrence distance between each word in the word set, and then determine the semantic contribution of the sentence at the sentence level in the machine translation process based on all the co-occurrence distances and the syntactic dependency between each word, and then obtain the semantic contribution of each sentence at the sentence level in the machine translation process; Based on the context information of the source language text, a semantic association analysis of the text structure is performed on each sentence to obtain the semantic association between each sentence and all other sentences in the text structure, and the translation attention of each sentence at the text level during the machine translation process is determined by all the semantic associations and the semantic complexity of each sentence; Based on the semantic contribution of each sentence at the sentence level and the translation attention of each sentence at the paragraph level, the source language text is machine translated at the paragraph level to obtain the target language text.

2. The method according to claim 1, characterized in that Extracting the semantic complexity of each sentence in the source language text based on the artificial intelligence semantic model specifically includes: Build artificial intelligence-based semantic models; Using the source language text as an initialization parameter of the semantic model; The semantic complexity of each sentence in the source language text is determined by the semantic model.

3. The method according to claim 1, characterized in that Adaptive word segmentation is performed on each sentence in the source language text according to the semantic complexity of each sentence, and the word set of each sentence obtained specifically includes: For each sentence, the word segmentation granularity of the word segmentation strategy in the preset word segmentation model is determined according to the semantic complexity of the sentence; According to the word segmentation granularity, the sentences are adaptively segmented by a preset word segmentation model to obtain the word set of the sentence, and then the word set of each sentence is obtained.

4. The method according to claim 1, characterized in that Determining the co-occurrence distance between words in a word set specifically includes: Select a word from the word set as a selected word; Determine the co-occurrence frequency between the selected word and every other word; Determine the inter-word distance between the selected word and every other word; The co-occurrence distance between the selected word and all other words is determined by all co-occurrence frequencies and all inter-word distances, and the co-occurrence distance between the remaining words and all other words is continued to be determined.

5. The method according to claim 1, characterized in that The semantic contribution of a sentence at the sentence level in the machine translation process is determined based on all co-occurrence distances and the syntactic dependencies between each word. Specifically, it includes: Select a word from the word set as a selected word; Get the co-occurrence distance between the selected word and all other words; Determine the syntactic dependency between the selected word and all other words based on the source language knowledge graph; Determine the attention coefficient of the selected word in the machine translation process by the co-occurrence distance and syntactic dependency between the selected word and all other words, and continue to determine the attention coefficients of the remaining words in the machine translation process; The semantic contribution of a sentence at the sentence level during machine translation is determined based on the attention coefficients of all words.

6. The method according to claim 1, characterized in that The translation attention of each sentence at the paragraph level in the machine translation process is determined by all semantic relevance and semantic complexity of each sentence, specifically including: Select a sentence as the selected sentence; Determine the priority of the selected sentence in the machine translation process according to the semantic relevance between the selected sentence and all other sentences in the text structure; By selecting the semantic complexity of the sentence, the priority is constrained in importance, so as to obtain the translation attention of the selected sentence at the paragraph level during the machine translation process; Continue to determine the translation attention of the remaining sentences at the paragraph level during the machine translation process.

7. The method according to claim 1, characterized in that Based on the semantic contribution of each sentence at the sentence level and the translation attention of each sentence at the paragraph level, the source language text is machine translated at the paragraph level to obtain the target language text, which specifically includes: Build a machine translation model for paragraph-level translation; The semantic contribution of each sentence at the sentence level is set as the sentence-level translation weight in the machine translation model; Setting the translation attention of each sentence at the paragraph level as the translation weight at the paragraph level in the machine translation model; The source language text is translated into a target language text by using the machine translation model.

8. A machine translation system based on artificial intelligence, characterized in that: include: An acquisition module, used to acquire source language text in machine translation; A processing module, configured to extract the semantic complexity of each sentence in the source language text based on an artificial intelligence semantic model, and perform adaptive word segmentation processing on each sentence in the source language text according to the semantic complexity of each sentence to obtain a word set of each sentence; The processing module is further used to determine, for each sentence, the co-occurrence distance between each word in the word set, and then determine the semantic contribution of the sentence at the sentence level in the machine translation process based on all the co-occurrence distances and the syntactic dependency between each word, and then obtain the semantic contribution of each sentence at the sentence level in the machine translation process; The processing module is further used to perform a semantic association analysis of the text structure on each sentence based on the context information of the source language text, obtain the semantic association between each sentence and all other sentences in the text structure, and determine the translation attention of each sentence at the text level during the machine translation process through all the semantic associations and the semantic complexity of each sentence; The execution module is used to perform a machine translation of the source language text at the paragraph level based on the semantic contribution of each sentence at the sentence level and the translation attention of each sentence at the paragraph level to obtain a target language text.

9. A computer device, comprising a memory and a processor, wherein the memory stores a code, characterized in that: The processor is configured to obtain the code and execute the artificial intelligence-based machine translation method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the artificial intelligence-based machine translation method according to any one of claims 1 to 7 is implemented.

Images