Artificial intelligence-based translation method and apparatus

Abstract

The application provides an artificial intelligence-based translation method and device; the method comprises the following steps: encoding an i-th element in a sentence to be translated into an i-th vector, and matching the i-th vector with a translation vocabulary to obtain a plurality of intermediate vectors; performing fusion processing on the plurality of intermediate vectors and candidate translation vectors of the first i-1 elements of the sentence to be translated respectively, and performing probability prediction processing on the plurality of fused candidate translation vectors to obtain a conditional probability of each candidate translation vector; storing the candidate translation vector corresponding to the conditional probability not less than the lower bound probability into a data structure, wherein the data structure stores K candidate translation vectors corresponding to the conditional probability greater than the lower bound probability in the first i translation processes; and decoding the candidate translation vectors in the data structure to obtain a translation result of the first i elements of the sentence to be translated. Through the application, an accurate translation result can be accurately searched.

Description

Technical Field

[0001] This application relates to artificial intelligence technology, and more particularly to an artificial intelligence-based translation method, apparatus, electronic device, and computer-readable storage medium. Background Technology

[0002] With the research and advancement of artificial intelligence (AI) technology, AI is being studied and applied in various fields, such as smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, autonomous driving, drones, robots, smart healthcare, smart customer service, connected vehicles, intelligent transportation, and machine translation. It is believed that with the development of technology, AI will be applied in more fields and play an increasingly important role.

[0003] Neural machine translation (NMT) is gradually becoming the mainstream of new translation models. However, existing NMT models often suffer from significant search errors during the testing phase, and the magnitude of these errors cannot be guaranteed. Currently, there is no effective solution for how to accurately search for and obtain precise translation results. Summary of the Invention

[0004] This application provides an artificial intelligence-based translation method, apparatus, electronic device, and computer-readable storage medium that can retrieve accurate translation results.

[0005] The technical solution of this application embodiment is implemented as follows:

[0006] This application provides an artificial intelligence-based translation method, including:

[0007] The i-th element in the sentence to be translated is encoded into the i-th vector, and the i-th vector is matched with the translation vocabulary to obtain multiple intermediate vectors, where i is an integer variable greater than 1;

[0008] The multiple intermediate vectors are fused with the candidate translation vectors of the first i-1 elements of the sentence to be translated, and the resulting multiple fused candidate translation vectors are subjected to probability prediction to obtain the conditional probability of each candidate translation vector.

[0009] The candidate translation vectors whose corresponding conditional probability is not less than the lower bound probability are stored in a data structure, wherein the data structure stores K candidate translation vectors whose corresponding conditional probability is greater than the lower bound probability in the previous i translation processes, and K is an integer constant greater than 1.

[0010] The candidate translation vectors in the data structure are decoded to obtain the translation results of the first i elements of the sentence to be translated.

[0011] This application provides an artificial intelligence-based translation device, including:

[0012] The matching module is used to encode the i-th element in the sentence to be translated into the i-th vector, and match the i-th vector with the translation vocabulary to obtain multiple intermediate vectors, where i is an integer variable greater than 1;

[0013] The fusion module is used to fuse the multiple intermediate vectors with the candidate translation vectors of the first i-1 elements of the sentence to be translated.

[0014] The prediction module is used to perform probability prediction processing on the multiple fused candidate translation vectors obtained, and to obtain the conditional probability of each candidate translation vector.

[0015] The storage module is used to store candidate translation vectors whose corresponding conditional probabilities are not less than the lower bound probability into a data structure. The data structure stores K candidate translation vectors whose corresponding conditional probabilities are greater than the lower bound probability in the previous i translation processes, where K is an integer constant greater than 1.

[0016] The decoding module is used to decode the candidate translation vectors in the data structure to obtain the translation results of the first i elements of the sentence to be translated.

[0017] In the above scheme, the storage module is further configured to compare the conditional probability of each candidate translation vector with the lower bound probability to obtain at least one candidate translation vector with a probability not less than the lower bound probability; and store the at least one candidate translation vector with a probability not less than the lower bound probability in the data structure.

[0018] In the above scheme, the AI-based translation device further includes: a deletion module, used to delete the candidate translation vector with the minimum conditional probability in the data structure when the number of candidate translation vectors in the data structure exceeds K, until the number of candidate translation vectors in the data structure is K; and an update module, used to update the lower bound probability to the minimum conditional probability of the K candidate translation vectors in the data structure.

[0019] In the above scheme, the decoding module is further configured to sort and filter the K candidate translation vectors in the data structure to obtain the candidate translation vector with the highest conditional probability among the K candidate translation vectors; and call the decoder based on the candidate translation vector with the highest conditional probability among the K candidate translation vectors to obtain the translation result of the first i elements of the sentence to be translated.

[0020] In the above scheme, the fusion module is further used to fuse the plurality of intermediate vectors with the candidate translation vectors of the first i-1 elements of the sentence to be translated, respectively, to obtain the plurality of fused candidate translation vectors corresponding to the plurality of intermediate vectors.

[0021] In the above scheme, the prediction module is further used to call the translation model based on the multiple fused candidate translation vectors to obtain the conditional probability of each candidate translation vector.

[0022] In the above scheme, the fusion module is further used to fuse the intermediate vector with the candidate translation vectors of the first i-1 elements of the sentence to be translated, so as to obtain the plurality of fused candidate translation vectors corresponding to any one of the plurality of intermediate vectors.

[0023] In the above scheme, the fusion module is further configured to perform parallel fusion processing on at least two intermediate vectors in the batch with the candidate translation vectors of the first i-1 elements of the sentence to be translated, so as to obtain the plurality of fused candidate translation vectors corresponding to at least two intermediate vectors in the batch.

[0024] In the above scheme, the AI-based translation device further includes a sorting module, which sorts the plurality of intermediate vectors according to the probability of each intermediate vector being selected, to obtain the sorted plurality of intermediate vectors.

[0025] In the above scheme, the fusion module is further used to fuse the sorted intermediate vectors with the candidate translation vectors of the first i-1 elements of the sentence to be translated.

[0026] In the above scheme, the fusion module is further configured to fuse the intermediate vector with the candidate translation vectors of the first i-1 elements of the sentence to be translated in sorting order, so as to obtain the plurality of fused candidate translation vectors corresponding to any one of the plurality of intermediate vectors.

[0027] In the above scheme, the fusion module is further configured to take at least two adjacent intermediate vectors among the sorted intermediate vectors as a batch, and different batches include different intermediate vectors, the order between the different batches satisfies the sorting order, and perform the following processing for each batch: fuse at least two intermediate vectors in the batch with the candidate translation vectors of the first i-1 elements of the sentence to be translated in parallel to obtain the multiple fused candidate translation vectors corresponding to at least two intermediate vectors in the batch.

[0028] This application provides an artificial intelligence-based translation device, including:

[0029] Memory, used to store executable instructions;

[0030] A processor, when executing executable instructions stored in the memory, implements the method provided in the embodiments of this application.

[0031] This application provides a computer-readable storage medium storing executable instructions for inducing a processor to execute and implement the artificial intelligence-based translation method provided in this application.

[0032] This application provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the artificial intelligence-based translation method described above in this application.

[0033] The embodiments of this application have the following beneficial effects:

[0034] Since the data structure stores K candidate translation vectors whose conditional probabilities are greater than the lower bound probability in the previous i translation processes, it ensures that the data structure stores the K candidate translation vectors that best match the sentence to be translated during the search process. Decoding the K candidate translation vectors that best match the sentence to be translated during the search process yields the translation results of the first i elements of the sentence to be translated, thereby improving the accuracy of the translation results. Attached Figure Description

[0035] Figure 1 This is a schematic diagram of the architecture of an artificial intelligence-based translation system provided in an embodiment of this application;

[0036] Figure 2 This is an optional structural schematic diagram of the artificial intelligence-based translation device provided in the embodiments of this application;

[0037] Figure 3 This is an optional flowchart illustrating an artificial intelligence-based translation method provided in an embodiment of this application.

[0038] Figure 4 This is an optional flowchart illustrating an artificial intelligence-based translation method provided in an embodiment of this application.

[0039] Figure 5 This is an optional flowchart illustrating an artificial intelligence-based translation method provided in an embodiment of this application;

[0040] Figure 6This is an optional flowchart illustrating an artificial intelligence-based translation method provided in an embodiment of this application;

[0041] Figure 7 This is an optional flowchart illustrating an artificial intelligence-based translation method provided in an embodiment of this application;

[0042] Figure 8 This is an optional flowchart illustrating an artificial intelligence-based translation method provided in an embodiment of this application;

[0043] Figure 9a This is a schematic diagram comparing the AI-based translation method and beam search provided in the embodiments of this application;

[0044] Figure 9b This is a schematic diagram comparing the AI-based translation method and beam search provided in the embodiments of this application;

[0045] Figure 9c This is a schematic diagram comparing the AI-based translation method and beam search provided in the embodiments of this application;

[0046] Figure 10 This is a schematic diagram illustrating the principle of the translation model provided in the embodiments of this application. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0048] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0049] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0050] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0051] 1) Natural Language Processing (NLP): This is an important area within computer science and artificial intelligence. It studies the theories and methods for enabling effective communication between humans and computers using natural language. NLP is a science that integrates linguistics, computer science, and mathematics. Therefore, research in this field involves natural language, that is, the language people use in daily life, and thus it has a close relationship with linguistic research. NLP techniques typically include text processing, semantic understanding, machine translation, question answering, and knowledge graphs.

[0052] 2) Machine Learning (ML): This is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, and inductive learning.

[0053] 3) Convolutional Neural Networks (CNNs): A class of feedforward neural networks (FNNs) that incorporate convolutional computations and have a deep structure, CNNs are one of the representative algorithms of deep learning. CNNs possess representation learning capabilities, enabling them to perform shift-invariant classification of input images according to their hierarchical structure.

[0054] 4) Machine Translation: Machine translation is characterized by five main features: automation, mechanical nature, sentence-based translation, second-order imitation, and limited contextual constraints. Furthermore, the relationship between machine translation and human translation is not contradictory or zero-sum, but rather complementary and mutually reinforcing. When language changes, new words, or loanwords appear, Google Translate can upgrade and improve its corpus, adding corresponding texts of new words and loanwords to adapt the translation to language development, which is very convenient.

[0055] 5) Neural machine translation (abbreviated as NMT): It uses a neural network model to learn the statistical model of machine translation. Its main advantage is that a system can be directly trained through source text and target text, without requiring a dedicated statistical machine learning system. Unlike the traditional phrase-based translation system which consists of many small independently adjusted sub-components, neural machine translation attempts to construct and train a single large neural network that can read a sentence and output a correct translation.

[0056] Machine translation refers to the process of using a computer to convert one natural language into another natural language: for example, translating a Chinese sentence into an English sentence. Typically, the translation model adopts an encoder-decoder model structure. In the process of translating a to-be-translated sentence, the translation model takes the to-be-translated sentence as the input of the encoder, encodes the to-be-translated sentence and outputs an intermediate representation: the intermediate representation is taken as the input of the decoder, and the intermediate representation is decoded into a translation result.

[0057] 6) To-be-translated sentence / translation result: In the process of natural language translation, a sentence of a first language category is translated into a sentence of a second language category, the sentence of the first language category is the to-be-translated sentence, and the sentence of the second language category is the translation result. Illustratively, when a Chinese sentence is translated into an English sentence, the Chinese sentence is the to-be-translated sentence, and the English sentence is the translation result: for example, the translation model translates the sentence "I must solve this problem." into the sentence "I must solve the problem.", the sentence "I must solve this problem." is the to-be-translated sentence, and the sentence "I must solve the problem." is the translation result.

[0058] In existing neural network machine translation systems, Beam Search is often used in the test phase to decode the translation result of the to-be-translated text. However, as a heuristic algorithm, beam search itself inherently has large search errors, and the magnitude of the errors cannot be guaranteed.

[0059] In beam search, a larger beam size leads to fewer search errors, because these candidates have higher ranking positions. As usual, the applicant calculates the BLEU scores of these candidates. The results show that higher-ranked candidates exhibit poorer BLEU scores (see Figure 9b ), which indicates that the NMT system has a defect in ranking, that is, it tends to place poorer candidates in relatively higher positions. This phenomenon is the beam search curse — the translation quality decreases as the beam size increases.

[0060] Based on the above analysis, the applicant found that beam search curse is closely related to the ranking ability of NMT models. Therefore, existing technologies cannot accurately search for translation results, and the accuracy of the translation results obtained by existing technologies is not high. Poor translation results alone cannot fully reflect the errors of the model itself, limiting the development of translation model evaluation methods. To address the above problems, this application provides an AI-based translation method that can reflect the probability distribution of the model at the translation result level.

[0061] The AI-based translation method, apparatus, electronic device, and computer-readable storage medium provided in this application can accurately search for the most matching translation result, significantly improving the accuracy of the translation result. The following describes exemplary applications of the AI-based translation electronic device provided in this application. The electronic device provided in this application can be implemented as various types of user terminals such as laptops, tablets, desktop computers, set-top boxes, and mobile devices (e.g., mobile phones, portable music players, personal digital assistants, dedicated messaging devices, portable gaming devices), or it can be implemented as a server.

[0062] See Figure 1 , Figure 1 This is a schematic diagram of the architecture of an AI-based translation system 100 provided in this application embodiment. To realize AI-based translation application scenarios, the terminal (exemplarily shown as terminal 400) connects to the server 200 through a network 300. The network 300 can be a wide area network or a local area network, or a combination of both.

[0063] Terminal 400 is used by users to access client 410, and is displayed on graphical interface 410-1 (graphical interface 410-1 is shown as an example). Terminal 400 and server 200 are interconnected via wired or wireless network.

[0064] In some embodiments, the terminal 400 includes a microphone: the terminal 400 acquires speech content through the microphone and sends the acquired speech content to the server 200. The server 200 includes a speech translation module: the speech translation module includes a translation model.

[0065] For example, server 200 receives voice content sent by terminal 400, recognizes the voice content to obtain the text to be translated, translates the text using a translation model, obtains the translation result, and sends the translation result to terminal 400 for display. Alternatively, terminal 400 may include a voice translation module: This voice translation module includes a translation model. After acquiring voice content, terminal 400 recognizes the voice content to obtain the text to be translated, translates the voice content using a translation model, obtains the translation result, and displays the translation result.

[0066] In some embodiments, the terminal 400 includes a text input control: the terminal 400 obtains text content through the text input control and sends the obtained text content to the server 200.

[0067] For example, server 200 includes a text translation module containing a trained translation model. Server 200 receives text content from terminal 400, translates the text content using the translation model, obtains a translation result, and sends the translation result to terminal 400 for display. Alternatively, terminal 400 may also include a text translation module containing a trained translation model. After obtaining the text content, terminal 400 translates the text content using the translation model, obtains a translation result, and displays the translation result.

[0068] In some embodiments, the terminal 400 may display information in either voice or text format.

[0069] As an illustration, the AI-based translation method provided in this application can be applied to various scenarios such as document translation, article reading, foreign language website translation, foreign language learning queries, oral conversation assistance, and overseas travel services. Examples are provided below:

[0070] Translation models are applied in text translation software. They can take documents such as contracts, files, materials, papers, and emails as input data, perform word embedding on the input data, and finally obtain the translation result.

[0071] Translation models are applied to various websites to achieve translation assistance. For example, when reading an article on a website, triggering the website's text translation function retrieves the text content from the article, uses this text as input data, performs word embedding processing on the input data, and finally obtains the translation result. Similarly, when browsing a foreign language website, triggering the website's translation function retrieves the text content from the webpage, uses this text as input data, performs word embedding processing on the input data, and finally obtains the translation result. This achieves website translation.

[0072] The translation model is applied to an electronic dictionary. When a sentence is input into the electronic dictionary, the dictionary first performs word embedding processing on the sentence, and finally obtains the translation result. This enables the translation of sentences input into the electronic dictionary.

[0073] The translation model is applied to real-time translation tools, such as machine translation systems. These systems acquire audio via a microphone, process the audio to obtain the sentence to be translated, perform word embedding on the sentence, and finally obtain the translation result. The translation result is then displayed, or converted into speech and played through a speaker. This translation tool can provide rapid responses in multinational conferences and while traveling abroad.

[0074] The translation model is applied to instant messaging applications. When a user sends a voice / video message through the instant messaging application, the terminal acquires the audio, processes it to obtain the sentence to be translated, performs word embedding on the sentence, and finally obtains the translation result. The translation result is then displayed, or converted into speech and played through a speaker.

[0075] The translation model is applied to video playback programs. When a video is played on the terminal, the terminal collects audio, processes the audio to obtain the sentence to be translated, and performs word embedding processing on the sentence to be translated. Finally, the translation result is obtained. The translation result is then displayed, or the translation result is converted into speech and played through a speaker.

[0076] Alternatively, the terminal can extract the subtitles, use the text content of the subtitles as input data, and finally obtain the translation result.

[0077] In some embodiments, server 200 may be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. Terminal 400 may be a smartphone, tablet computer, laptop computer, desktop computer, smart speaker, smartwatch, etc., but is not limited thereto. Terminals and servers can be directly or indirectly connected via wired or wireless communication, which is not limited in this embodiment.

[0078] The AI-based translation method, apparatus, electronic device, and computer-readable storage medium provided in this application can accurately search for translation results. The following describes an exemplary application of the AI ​​translation electronic device provided in this application. The electronic device provided in this application can be implemented as various types of user terminals such as laptops, tablets, desktop computers, set-top boxes, and mobile devices (e.g., mobile phones, portable music players, personal digital assistants, dedicated messaging devices, portable gaming devices), or as a server.

[0079] See Figure 2 , Figure 2 This is a schematic diagram of the structure of terminal 400 of the artificial intelligence-based translation method provided in this application embodiment. Figure 2 The terminal 400 shown includes at least one processor 410, a memory 450, at least one network interface 420, and a user interface 430. The various components in the terminal 400 are coupled together via a bus system 440. It is understood that the bus system 440 is used to implement communication between these components. In addition to a data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in… Figure 2 The general labeled all buses as Bus System 440.

[0080] The processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0081] User interface 430 includes one or more output devices 431 that enable the presentation of media content, including one or more speakers and / or one or more visual displays. User interface 430 also includes one or more input devices 432, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0082] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices physically located away from the processor 410.

[0083] The memory 450 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 450 described in this application embodiment is intended to include any suitable type of memory.

[0084] In some embodiments, memory 450 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0085] Operating system 451 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, and driver layer, for implementing various basic business functions and handling hardware-based tasks.

[0086] The network communication module 452 is used to reach other computing devices via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 including Bluetooth, WiFi, and Universal Serial Bus (USB).

[0087] Presentation module 453 is configured to enable the presentation of information (e.g., a user interface for operating peripheral devices and displaying content and information) via one or more output devices 431 (e.g., a display screen, a speaker, etc.) associated with user interface 430;

[0088] The input processing module 454 is used to detect and translate one or more user inputs or interactions from one or more input devices 432.

[0089] In some embodiments, the AI-based translation device provided in this application can be implemented in software. Figure 2 A device 455 stored in memory 450 is shown, which can be software in the form of programs and plug-ins, including the following software modules: matching module 4551, fusion module 4552, prediction module 4553, storage module 4554, and decoding module 4555. These modules are logically connected and can therefore be arbitrarily combined or further split according to the functions they implement. The functions of each module will be described below.

[0090] In other embodiments, the translation apparatus provided in this application can be implemented in hardware. As an example, the apparatus provided in this application can be a processor in the form of a hardware decoding processor, which is programmed to execute the artificial intelligence-based translation method provided in this application. For example, the processor in the form of a hardware decoding processor can adopt one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0091] The following will describe the AI-based translation method provided in this application embodiment, using exemplary applications and implementations of the terminal provided in the embodiments of this application.

[0092] The translation model provided in this application uses a sequence-to-sequence modeling approach to model translation and decoding. See also... Figure 10 , Figure 10 This is an optional flowchart of the translation model provided in this application embodiment. The input sequence to be translated (e.g., a Chinese sequence) is encoded by an encoder, and then decoded by a decoder to obtain the translation result (e.g., the Chinese translation of the sentence to be translated). Both the encoder and decoder can be Long Short-Term Memory (LSTM) models (sometimes GRU models). The context vector of the final unit of the encoder is input to the first unit of the decoder network. Using these initial states, the decoder begins to generate output sequences (e.g., English sequences), and future outputs will also take these outputs into account.

[0093] See Figure 3 , Figure 3 This is an optional flowchart illustrating an artificial intelligence-based translation method provided in an embodiment of this application, which will be combined with... Figure 3 The steps shown are explained.

[0094] In step 101, the i-th element in the sentence to be translated is encoded as the i-th vector, and the i-th vector is matched with the translation vocabulary to obtain multiple intermediate vectors, where i is an integer variable greater than 1.

[0095] In some embodiments, before step 101, the first element in the sentence to be translated is encoded as a first vector, and the first vector is matched with the translation vocabulary to obtain multiple intermediate vectors.

[0096] In some embodiments, step 101 can be implemented as follows: call the encoder based on the i-th element in the sentence to be translated to obtain the encoded i-th vector, pass the i-th vector to the decoder, call the decoder based on the i-th vector, match the i-th vector with the translation vocabulary, and obtain multiple intermediate vectors.

[0097] For example, when i=2, the second element in the sentence to be translated can be encoded as a second vector, and the second vector can be matched with the translation vocabulary to obtain multiple intermediate vectors.

[0098] For example, when i=3, the third element in the sentence to be translated can be encoded as a third vector, and the third vector can be matched with the translation vocabulary to obtain multiple intermediate vectors.

[0099] In some embodiments, the element can be a source word, punctuation mark, special symbol, etc., in the sentence to be translated, and the specific type of the element does not constitute a limitation on this application. Special symbols can be added after each sentence to be translated; the special symbols added after each sentence can be "". <eos>"end of sequence" indicates the end of a sentence to be translated.

[0100] For example, the translation vocabulary can be {X, Y, T, F, H, S}. By matching the i-th vector with each vector in the translation vocabulary, multiple intermediate vectors that have been successfully matched can be obtained. For example, the multiple intermediate vectors can be {X, F, S}.

[0101] See Figure 4 , Figure 4 This is an optional flowchart of an artificial intelligence-based translation method provided in this application embodiment. In step 101, the i-th vector is matched with the translation vocabulary to obtain multiple intermediate vectors, and then step 105 is executed.

[0102] In step 105, the multiple intermediate vectors are sorted according to the probability of each intermediate vector being selected, resulting in sorted intermediate vectors.

[0103] By sorting the intermediate vectors in descending order, the intermediate vectors with higher probability of being selected can be prioritized for fusion processing, effectively saving running time and improving translation efficiency.

[0104] In step 102, multiple intermediate vectors are fused with the candidate translation vectors of the first i-1 elements of the sentence to be translated, and the resulting multiple fused candidate translation vectors are subjected to probability prediction to obtain the conditional probability of each candidate translation vector.

[0105] For example, when i=2, the second element of the sentence to be translated can be encoded into a second vector, and this second vector can be matched with the translation vocabulary to obtain multiple intermediate vectors. These multiple intermediate vectors are then fused with the candidate translation vector of the first element of the sentence to be translated, and probability prediction is performed on the resulting fused candidate translation vectors to obtain the conditional probability of each candidate translation vector.

[0106] For example, when i=3, the third element in the sentence to be translated can be encoded as a third vector, and this third vector can be matched with the translation vocabulary to obtain multiple intermediate vectors. These intermediate vectors are then fused with the candidate translation vectors of the first two elements of the sentence to be translated, and probability prediction is performed on the resulting fused candidate translation vectors to obtain the conditional probability of each candidate translation vector.

[0107] In some embodiments, the fusion process may be splicing, merging, etc., and the specific type of fusion process does not constitute a limitation on this application.

[0108] For example, multiple intermediate vectors can be {X, F, S}, and the candidate translation vectors for the first i-1 elements of the sentence to be translated can be {XY, XYTF, TFHS, YXFTHS}. By fusing multiple intermediate vectors with the candidate translation vectors for the first i-1 elements of the sentence to be translated, multiple fused candidate translation vectors can be {XYX, XYF, XYTFS, TFHSX}.

[0109] In some embodiments, probabilistic prediction processing of the obtained multiple fused candidate translation vectors can be achieved through the following scheme: a translation model is invoked based on the multiple fused candidate translation vectors to perform probabilistic prediction processing, wherein the expression of the translation model can be:

[0110] log p(Y t |X;Y 1:t-1 )=M(X,Y 1:t-1 (1)

[0111] Among them, Y 1：t-1 This represents the candidate translation vector of the first i-1 elements of the sentence to be translated, where X represents the middle element, and logp(Y) = ... t |X;Y 1：t-1 ) represents the logarithmic form of conditional probability.

[0112] In some embodiments, the translation can be based on multiple fused candidate translation vectors (X, Y). 1：t-1 Calling the translation model M, we obtain the logarithmic form of the conditional probability of each candidate translation vector: log p(Y). t |X;Y 1：t-1 ).

[0113] See Figure 5 , Figure 5 This is an optional flowchart illustrating an artificial intelligence-based translation method provided in an embodiment of the present invention. Figure 5 Step 102 shown can be implemented through steps 1021 to 1022.

[0114] In step 1021, the multiple intermediate vectors are fused with the candidate translation vectors of the first i-1 elements of the sentence to be translated to obtain multiple fused candidate translation vectors corresponding to the multiple intermediate vectors.

[0115] In some embodiments, step 1021 can be implemented by the following technical solution: merging multiple intermediate vectors with the candidate translation vectors of the first i-1 elements of the sentence to be translated, where each intermediate vector corresponds to multiple merged candidate translation vectors. It is understood that if the number of intermediate vectors is N, and the number of candidate translation vectors for the first i-1 elements of the sentence to be translated is M, then the number of merged candidate translation vectors is M*N.

[0116] In some embodiments, step 1021 can be implemented by the following technical solution: performing the following processing on each of the multiple intermediate vectors: fusing the intermediate vector with the candidate translation vectors of the first i-1 elements of the sentence to be translated to obtain multiple fused candidate translation vectors corresponding to any one of the multiple intermediate vectors.

[0117] For example, by fusing an intermediate vector with the candidate translation vectors of the first i-1 elements of N sentences to be translated, we can obtain N fused candidate translation vectors corresponding to the intermediate vector.

[0118] In some embodiments, step 1021 can be implemented by the following technical solution: taking any two intermediate vectors from a plurality of intermediate vectors as a batch, and different batches include different intermediate vectors, and performing the following processing for each batch: fusing the at least two intermediate vectors in the batch with the candidate translation vectors of the first i-1 elements of the sentence to be translated in parallel to obtain a plurality of fused candidate translation vectors corresponding to the at least two intermediate vectors in the batch.

[0119] For example, any two intermediate vectors from multiple intermediate vectors are taken as a batch, and the at least two intermediate vectors in a batch are fused in parallel with the N candidate translation vectors of the first i-1 elements of the sentence to be translated, to obtain at least 2N fused candidate translation vectors corresponding to the at least two intermediate vectors in the batch.

[0120] By treating any two of the multiple intermediate vectors as a batch and fusing them in parallel with the candidate translation vectors of the first i-1 elements of the sentence to be translated, the fusion efficiency can be significantly improved and the fusion time can be effectively shortened.

[0121] In step 1022, the translation model is invoked based on multiple fused candidate translation vectors to obtain the conditional probability of each candidate translation vector.

[0122] In some embodiments, step 1022 can be implemented by inputting multiple fused candidate translation vectors into a translation model to obtain the conditional probability of each candidate translation vector.

[0123] In some embodiments, the sorting of multiple intermediate vectors in step 105 above according to the probability of each intermediate vector being selected, to obtain multiple sorted intermediate vectors, can be achieved by: fusing the multiple sorted intermediate vectors with the candidate translation vectors of the first i-1 elements of the sentence to be translated.

[0124] Since multiple candidate translation vectors are sorted, the candidate translation vector with a higher probability of being selected can be preferentially fused with the candidate translation vectors of the first i-1 elements of the sentence to be translated.

[0125] In some embodiments, step 102 can be implemented by the following technical solution: the sorted intermediate vectors are fused with the candidate translation vectors of the first i-1 elements of the sentence to be translated according to the sorting order, so as to obtain a plurality of fused candidate translation vectors corresponding to the sorted intermediate vectors.

[0126] In some embodiments, the above-mentioned fusion processing of the sorted intermediate vectors with the candidate translation vectors of the first i-1 elements of the sentence to be translated in the sorted order to obtain multiple fused candidate translation vectors corresponding to the sorted intermediate vectors can also be achieved by the following technical solution: the following processing can be performed on each of the sorted intermediate vectors: the intermediate vectors are fused with the candidate translation vectors of the first i-1 elements of the sentence to be translated in the sorted order to obtain multiple fused candidate translation vectors corresponding to any one of the intermediate vectors.

[0127] For example, the sorted intermediate vector is {4, 3, 2, 1}, and the unsorted intermediate vector is {2, 3, 4, 1}. The intermediate vector {4} can be fused with the candidate translation vectors of the first i-1 elements of the sentence to be translated. Then, the intermediate vector {3} can be fused with the candidate translation vectors of the first i-1 elements of the sentence to be translated. Next, the intermediate vector {2} can be fused with the candidate translation vectors of the first i-1 elements of the sentence to be translated. Finally, the intermediate vector {1} can be fused with the candidate translation vectors of the first i-1 elements of the sentence to be translated.

[0128] By fusing the intermediate vector with the candidate translation vectors of the first i-1 elements of the sentence to be translated in sorted order, the multiple fused candidate translation vectors can also follow the sorted order. Therefore, when performing probability prediction on multiple fused candidate translation vectors, the probability prediction can also be performed in sorted order, which further improves the efficiency of the algorithm and effectively shortens the algorithm's runtime.

[0129] In some embodiments, the above-mentioned fusion processing of the sorted intermediate vectors with the candidate translation vectors of the first i-1 elements of the sentence to be translated according to the sorting order to obtain multiple fused candidate translation vectors corresponding to the sorted intermediate vectors can also be achieved by the following technical solution: at least two adjacent intermediate vectors in the sorted intermediate vectors can be regarded as a batch, and different batches include different intermediate vectors, and the order between different batches satisfies the sorting order. For each batch, the following processing is performed: at least two intermediate vectors in the batch are fused in parallel with the candidate translation vectors of the first i-1 elements of the sentence to be translated to obtain multiple fused candidate translation vectors corresponding to at least two intermediate vectors in the batch.

[0130] For example, at least two adjacent intermediate vectors from a sorted set of intermediate vectors can be grouped into a batch. The batches are then fused sequentially with the candidate translation vectors of the first i-1 elements of the sentence to be translated, in the sorted order. At least two intermediate vectors within a batch are then fused in parallel with the candidate translation vectors of the first i-1 elements of the sentence to be translated.

[0131] First, sort the multiple intermediate vectors, and then batch process at least two adjacent intermediate vectors among the sorted intermediate vectors, which greatly reduces the time consumption of the algorithm and effectively improves the running speed of the algorithm.

[0132] In step 103, the candidate translation vectors with conditional probabilities not less than the lower bound probability are stored in a data structure. The data structure stores K candidate translation vectors with conditional probabilities greater than the lower bound probability in the previous i translation processes, where K is an integer constant greater than 1.

[0133] In some embodiments, the lower bound probability can be the conditional probability of the candidate translation vector with the minimum conditional probability in the data structure.

[0134] In some embodiments, the data structure can be a min-heap. A heap is a type of tree structure; it is a complete binary tree. A min-heap or max-heap always satisfies the following conditions: the value of any node in the heap is never greater than or never less than the value of its parent node, and a heap is a complete binary tree. In a min-heap data structure, the minimum value is at the root node, and the value of any node is always less than the minimum value of its subtree.

[0135] For example, the fused candidate translation vectors are {XYX, XYF, XYTF\S, TFH\SX}, with conditional probabilities of 0.2 for XYX, 0.5 for XYF, 0.1 for XYTF\S, and 0.7 for TFH\SX, and a lower bound probability of 0.4. Therefore, the candidate translation vectors {XYF, TFH\SX} with conditional probabilities not less than the lower bound probability can be stored in the data structure.

[0136] For example, during the (i-2)th translation, candidate translation vectors with conditional probabilities greater than the lower bound, during the (i-1)th translation, and during the ith translation, can all be stored in a data structure. The data structure can store K candidate translation vectors.

[0137] See Figure 6 , Figure 6 This is an optional flowchart of the translation method based on artificial intelligence provided in the embodiments of this application. After step 103, steps 106 to 107 can be executed.

[0138] In step 106, when there are more than K candidate translation vectors in the data structure, the candidate translation vector with the minimum conditional probability is deleted from the data structure until the number of candidate translation vectors in the data structure is K.

[0139] For example, when there are K+2 candidate translation vectors in the data structure, the candidate translation vector with the minimum conditional probability is deleted, that is, two candidate translation vectors are deleted from the data structure, until the number of candidate translation vectors in the data structure is K.

[0140] When the number of candidate translation vectors in the data structure exceeds K, the candidate translation vector with the lowest conditional probability is deleted from the data structure until the number of candidate translation vectors in the data structure is K. This ensures that the data structure always stores K candidate translation vectors with the highest conditional probabilities.

[0141] In step 107, the lower bound probability is updated to the minimum conditional probability of the K candidate translation vectors in the data structure.

[0142] For example, when there are more than K candidate translation vectors in the data structure, after deleting the candidate translation vector with the minimum conditional probability from the data structure, the candidate translation vector with the minimum probability in the deleted data structure will be updated. Therefore, the lower bound probability needs to be updated to the minimum conditional probability of the K candidate translation vectors in the data structure.

[0143] In the first i translation iterations, the lower bound probability is updated to the minimum conditional probability of the K candidate translation vectors in the data structure after each translation iteration, whenever the candidate translation vector with the minimum conditional probability is deleted from the data structure. Understandably, the value of the lower bound probability increases gradually after each update. Specifically, the lower bound probability after the i-th update will be higher than the lower bound probability after the (i-2)-th update. Because the lower bound probability gradually increases, the difficulty for candidate translation vectors to enter the data structure gradually increases, thus further ensuring that the data structure stores the K optimal candidate translation vectors from the first i translation results.

[0144] See Figure 7 , Figure 7 This is an optional flowchart illustrating an artificial intelligence-based translation method provided in an embodiment of the present invention. Figure 7 Show Figure 3 Step 103 can be achieved through Figure 7 Steps 1031 to 1032 shown are implemented.

[0145] In step 1031, the conditional probability of each candidate translation vector is compared with the lower bound probability to obtain at least one candidate translation vector with a probability not less than the lower bound probability.

[0146] In some embodiments, step 1031 above can be implemented by the following technical solution: comparing one of the multiple fused candidate translation vectors with the lower bound probability; if the conditional probability of the candidate translation vector is greater than the lower bound probability, then storing the candidate translation vector in a data structure.

[0147] For example, the conditional probability of the candidate translation vector for the first i-1 elements of the sentence to be translated is smaller than the conditional probability of the fused candidate translation vector. Specifically, the conditional probability of the candidate translation vector for the first i-1 elements of the sentence to be translated and the conditional probability of the fused candidate translation vector can be related as follows:

[0148] P1=P+log p(v|x,y) (2)

[0149] Where P1 represents the conditional probability of the fused candidate translation vector, P represents the conditional probability of the candidate translation vector of the first i-1 elements of the sentence to be translated, and log p(v|x,y) represents the candidate translation vector of the i-th element in the sentence to be translated.

[0150] In step 1032, at least one candidate translation vector with a probability not less than the lower bound is stored in the data structure.

[0151] For example, a candidate translation vector with a probability not less than the lower bound is a relatively better candidate translation vector among all candidate translation vectors. Storing the relatively better candidate translation vector in the data structure can ensure that the data structure always stores the best candidate translation vector in the previous i translation processes.

[0152] In step 104, the candidate translation vectors in the data structure are decoded to obtain the translation results of the first i elements of the sentence to be translated.

[0153] In some embodiments, decoding the candidate translation vectors in the data structure to obtain the translation results of the first i elements of the statement to be translated can be achieved through the following technical solution: calling the decoder based on the candidate translation vectors in the data structure to obtain the translation results of the first i elements of the statement to be translated.

[0154] For example, the candidate translation vectors in the data structure are all candidate translation vectors whose conditional probability is greater than the lower bound probability during the first i translation processes. This allows the data structure to store the optimal candidate translation vectors during the first i translation processes. Based on the optimal candidate translation vectors during the first i translation processes, the decoder is called to decode, thereby obtaining the optimal translation result of the first i elements of the sentence to be translated.

[0155] See Figure 8 , Figure 8 This is an optional flowchart illustrating an artificial intelligence-based translation method provided in an embodiment of the present invention. Figure 8 Show Figure 3 Step 104 in the process can be achieved through Figure 8 Steps 1041 to 1042 shown are implemented.

[0156] In step 1041, the K candidate translation vectors in the data structure are sorted and filtered to obtain the candidate translation vector with the highest conditional probability among the K candidate translation vectors.

[0157] In some embodiments, step 1041 can be implemented by the following technical solution: the K candidate translation vectors in the data structure can be sorted in descending order of conditional probability value, and the candidate translation vector with the largest conditional probability value can be selected after sorting to obtain the candidate translation vector with the largest conditional probability among the K candidate translation vectors.

[0158] In step 1042, the decoder is invoked based on the candidate translation vector with the highest conditional probability among the K candidate translation vectors to obtain the translation results of the first i elements of the sentence to be translated.

[0159] In some embodiments, step 1042 can be implemented by the following technical solution: calling the decoder to decode based on the candidate translation vector with the highest conditional probability among K candidate translation vectors, and obtaining the translation result of the first i elements of the sentence to be translated.

[0160] The following will describe an exemplary application of the embodiments of this application in a real-world application scenario.

[0161] During the implementation of the embodiments of this application, the applicant discovered the following problems with the related technology:

[0162] In existing neural network machine translation systems, beam search is often used to decode the translation results of the text to be translated during the testing phase. However, beam search itself is a heuristic algorithm, which inherently has a large search error and cannot guarantee the magnitude of the error.

[0163] In existing neural network machine translation systems, a common decoding method is to find the candidate code with the highest conditional probability, often referred to as the maximum-a-posterical (MAP) decoding algorithm. Among all MAP decoding methods, beam search is the most widely used decoding method in neural network machine translation systems. Beam search typically uses a fixed beam size for each decoding step. Naive beam search with logarithmic probability has some drawbacks, such as favoring short translations and monotonic constraints. Therefore, to improve the performance of beam search, many regularization methods or beam search variants have been proposed.

[0164] In existing neural network machine translation systems, error evaluation focuses on the gap between the machine translation result and the human-translated reference. Typically, statistical matching metrics (such as BLE U, METEOR, etc.) dominate error evaluation. However, these statistical matching metrics only evaluate a single best candidate (i.e., sentence or document). They do not evaluate the system-level ranking of the generated candidates. In contrast, the embodiments of this application consider system-level ranking.

[0165] Beam search is a commonly used decoding method in machine translation. The specific algorithm of beam search starts with an empty sentence and uses a beam of width b. Each time, an additional word is added to the translation result, and the model's result is decoded from left to right. During the decoding process within the b beams, each beam provides |V| candidate words based on a translation vocabulary. The algorithm selects the b largest words from the b*|V| candidate words in the b beams. Once a terminator is encountered, the current beam ends; otherwise, after updating the log probability score, the selected words are added to the decoded sentence until all beams have ended.

[0166] See Figure 9a , Figure 9b , Figure 9c , Figure 9a This is a schematic diagram comparing the AI-based translation method and beam search provided in the embodiments of this application; Figure 9b This is a schematic diagram comparing the AI-based translation method and beam search provided in the embodiments of this application; Figure 9c This is a schematic diagram comparing the AI-based translation method and beam search provided in the embodiments of this application.

[0167] First, the AI-based translation method and beam search algorithm from this application were run on English and German test sets, respectively. The differences in the results of these two methods can help understand the degree of their divergence. Then, plotting... Figure 9a The ranking position of the beam search results is precisely output. It is clearly shown that larger beam sizes result in fewer search errors, as these candidates rank higher. As usual, applicants calculate the BLEU scores for these candidates. The results show that higher-ranked candidates exhibit worse BLEU scores (see...). Figure 9b This indicates a flaw in the NMT system's ranking mechanism, namely, its tendency to place weaker candidates at relatively high positions. This phenomenon is known as the beam search curse—translation quality deteriorates as beam size increases.

[0168] In addition, the applicant generated translations for each source sentence with different beam sizes. The applicant then calculated and compared the sentence-level BLEU scores of these translations. Figure 9a The results shown suggest that applicants can infer that the changes in BLEU at the sentence level are consistent with the increments in beam size. In contrast, these changes are random (see...). Figure 9c This means that as the beam size increases, the source statement may be randomly converted into a better or worse target statement. This indicates that NMT models also suffer from a large number of ranking errors, meaning that two very close candidates may be incorrectly ranked by the model.

[0169] Based on the above analysis, the applicant found that beam search curse is closely related to the ranking ability of NMT models. Therefore, existing technologies cannot accurately search for translation results, and the accuracy of the translation results obtained by existing technologies is not high. Poor translation results alone cannot fully reflect the errors of the model itself, which limits the development of translation model evaluation methods. Therefore, this application provides an AI-based translation method that can reflect the probability distribution of the model at the translation result level.

[0170] This application's embodiments approximate the probability distribution of the model at the level of the complete translation result by focusing on the translation results with K maximum conditional probabilities, providing a necessary foundation for designing more robust evaluation metrics.

[0171] The i-th element in the sentence to be translated is encoded into the i-th vector, and the i-th vector is matched with the translation vocabulary to obtain multiple intermediate vectors, where i is an integer variable greater than 1.

[0172] Each intermediate vector is compared with the candidate translation vector of the first i-1 elements of the sentence to be translated.

[0173] The translation result Y of the translated sentence t The following modeling is performed:

[0174] log p(Y t |X;Y 1:t-1 )=M(X,y 1:t-1 (3)

[0175] Among them, Y 1：t-1 This represents the candidate translation vector of the first i-1 elements of the sentence to be translated, where X represents the middle element, and logp(Y) = ... t |X;Y 1：t-1 ) represents the logarithmic form of conditional probability.

[0176] In some embodiments, translation can be based on multiple fused candidate translation vectors (X, Y). 1：t-1 Calling the translation model M, we obtain the logarithmic form of the conditional probability log_p(Y) for each candidate translation vector. t |X;Y 1：t-1 ).

[0177] During the translation process, the translation model M uses the t-th vector of the t-th element in the sentence to be translated and the translated result Y. 1：t-1 As input, predict the translation result Y t Where t represents each translation time step, Y t Represents the candidate word predicted at time t, log p(Y) t The ) represents the logarithmic probability that the candidate word is selected.

[0178] A min-heap is a data structure, and the lower bound probability can be simply referred to as the lower bound. The lower bound probability can be the probability value of the candidate translation vector with the minimum probability in the min-heap data structure. In the embodiments of this application, the lower bound probability and the lower bound are both the probability values ​​of the candidate translation vector with the minimum probability in the current min-heap data structure.

[0179] An additional min-heap is used to help find the k translations that the model considers optimal. First, the algorithm takes as input the source sentence X, the currently translated sentence Y (translation prefix), the log probability output by the model, and a hyperparameter k to control which of the k largest translations to be returned. Throughout the algorithm's operation, a global min-heap and a lower bound γ are used.

[0180] Check if the currently translated part has ended (the most recently translated word is 0). If it has ended, add the current translation candidate (the sentence that has been translated) to the min-heap.

[0181] If the data structure contains more than k candidate translation vectors, then the candidate translation vectors in the data structure need to be deleted. This is done by deleting the candidate translation vector with the lowest conditional probability in the data structure until the number of candidate translation vectors in the data structure is maintained at k, so as to ensure that the data structure maintains the k candidate translation vectors with the highest conditional probability in the first i search processes.

[0182] Next, the lower bound value will be updated to the score of the translation candidate with the smallest score in the data structure.

[0183] By calling the translation model M, the score of the translation model M for all candidate vectors in the translation vocabulary for the i-th vector and the first i-1 elements of the sentence to be translated is calculated.

[0184] By iteratively traversing the entire translation vocabulary, the fused candidate translation vectors are obtained, and the algorithm is then called again.

[0185] Next, the pseudocode of the AI-based translation method provided in this application embodiment is shown below:

[0186] Algorithm 1: Top-k Exact Search Process Based on Depth-First Search

[0187] Global min-heap

[0188] lower bound γ

[0189] Function DFS_TOP-k(x,y,p)

[0190] if the currently translated portion has been completed then

[0191] Add the translated candidate translation vectors to the min-heap.

[0192] If there are more than K candidate translation vectors that have been translated in the min-heap, then...

[0193] Pop the candidate translation vector with the minimum conditional probability from the min-heap.

[0194] If the number of candidate translation vectors that have been translated in the min-heap equals K then

[0195] The lower bound γ is updated to the score of the candidate translation vector with the smallest score in the min-heap.

[0196] The for loop iterates through the entire translation vocabulary.

[0197] Assign the conditional probabilities of the first i-1 candidate translation vectors to the first i candidate translation vectors.

[0198] If the conditional probability of the first i candidate translation vectors is greater than the lower bound γ, then

[0199] Continue executing the (i+1)th time

[0200] Save the new global min-heap

[0201] Save the candidate translation vectors after translation is complete.

[0202] Since the algorithm needs to traverse a very large possible search space, the algorithm speed may be slow. Therefore, this application also proposes some solutions to accelerate the algorithm itself, including using lower bound pruning of the search space, batch processing of the search, and the iteration order of the greedy search, etc.

[0203] The search process employs batch processing: During the recursive use of the algorithm, the most time-consuming part is the forward computation of the model. Therefore, this embodiment uses a batch computation mode. Instead of iterating through the vocabulary word by word, it iterates every b candidate words, simultaneously feeding b concatenated translation histories into the next algorithm call. Then, in the next call, a single forward model operation is used to simultaneously calculate the probabilities of b*|V| candidate words. This reduces the number of times the model probabilities are calculated using the GPU.

[0204] Greedy search iteration order: During the traversal of the entire candidate word list, the probabilities of all candidate words are first sorted. Then, all candidate words are greedily iterated in descending order. This allows for faster updates to the lower bound value and reduces the amount of computation in the experiments.

[0205] The evaluation results of the TOP-K-based translation algorithm proposed in this application, the algorithm proposed by Stahlberg and byrne et al., and the beam search algorithm are compared. The evaluation results are shown in Table 1:

[0206] Table 1 Evaluation Results

[0207]

[0208] It can be seen that beam search is 10 to 20 times faster than the exact search algorithm. However, compared with the exact top 10w / Minheap BS implementation, the exact top 10w / Minheap BS implementation proposed in this application is only twice as slow. This is a reasonable additional overhead for the newly proposed translation method in this application, and this reasonable additional overhead can significantly improve translation accuracy. As shown in Table 1 above, using the exact top 10w / Minheap BS proposed in this application can accelerate the decoding algorithm from 19312.8 seconds to 18908.4 seconds.

[0209] The evaluation results of the TOP-K translation algorithm proposed in this application under different K values ​​are compared with those of the algorithm proposed by Stahlberg and Byrne et al., and the evaluation results are shown in Table 2:

[0210] Table 2 Evaluation results for different K values

[0211]

[0212] The evaluation results in Tables 1 and 2 above are based on a random sample of sentences to be translated, translated using the methods described above, and the corresponding runtime and number of expansion operations are statistically analyzed. In the exact search algorithm, the expansion operation, i.e., the forward pass of the model, is the most time-consuming operation. The applicant reported the computational costs of four different algorithms, including beam search, exact Top-1, exact Top-10, and exact Top-20 min-heap lower bound. The figures reported for each algorithm are the average of four runs using different samples as input.

[0213] The following will continue to combine Figure 1 The implementation of the AI-based translation device 455 provided in this application is an exemplary structure of a software module. In some embodiments, such as... Figure 1 As shown, the software modules stored in the AI-based translation device 455 in the memory 450 may include: a matching module 4551, a fusion module 4552, a prediction module 4553, a storage module 4554, and a decoding module 4555.

[0214] The matching module 4551 is used to encode the i-th element in the sentence to be translated into the i-th vector, and match the i-th vector with the translation vocabulary to obtain multiple intermediate vectors, where i is an integer variable greater than 1.

[0215] The fusion module 4552 is used to fuse multiple intermediate vectors with the candidate translation vectors of the first i-1 elements of the sentence to be translated.

[0216] The prediction module 4553 is used to perform probability prediction processing on the multiple fused candidate translation vectors to obtain the conditional probability of each candidate translation vector.

[0217] The storage module 4554 is used to store the candidate translation vectors whose corresponding conditional probability is not less than the lower bound probability into the data structure. The data structure stores K candidate translation vectors whose corresponding conditional probability is greater than the lower bound probability in the first i translation processes, where K is an integer constant greater than 1.

[0218] The decoding module 4555 is used to decode the candidate translation vectors in the data structure to obtain the translation results of the first i elements of the sentence to be translated.

[0219] In some embodiments, the storage module 4554 is further configured to compare the conditional probability of each candidate translation vector with the lower bound probability to obtain at least one candidate translation vector with a probability not less than the lower bound probability; and store at least one candidate translation vector with a probability not less than the lower bound probability in a data structure.

[0220] In some embodiments, the AI-based translation device further includes: a deletion module, configured to delete the candidate translation vector with the minimum conditional probability in the data structure when the number of candidate translation vectors in the data structure exceeds K, until the number of candidate translation vectors in the data structure is K; and an update module, configured to update the lower bound probability to the minimum conditional probability of the K candidate translation vectors in the data structure.

[0221] In some embodiments, the decoding module 4555 is further configured to sort and filter the K candidate translation vectors in the data structure to obtain the candidate translation vector with the highest conditional probability among the K candidate translation vectors; and call the decoder based on the candidate translation vector with the highest conditional probability among the K candidate translation vectors to obtain the translation result of the first i elements of the sentence to be translated.

[0222] In some embodiments, the fusion module 4552 is further configured to fuse the multiple intermediate vectors with the candidate translation vectors of the first i-1 elements of the sentence to be translated, respectively, to obtain multiple fused candidate translation vectors corresponding to the multiple intermediate vectors.

[0223] In some embodiments, the prediction module 4553 is further configured to invoke the translation model based on multiple fused candidate translation vectors to obtain the conditional probability of each candidate translation vector.

[0224] In some embodiments, the fusion module 4552 is further configured to fuse the intermediate vector with the candidate translation vectors of the first i-1 elements of the sentence to be translated, so as to obtain a plurality of fused candidate translation vectors corresponding to any one of the plurality of intermediate vectors.

[0225] In some embodiments, the fusion module 4552 is further configured to perform parallel fusion processing on at least two intermediate vectors in the batch with the candidate translation vectors of the first i-1 elements of the sentence to be translated, to obtain a plurality of fused candidate translation vectors corresponding to at least two intermediate vectors in the batch.

[0226] In some embodiments, the AI-based translation device further includes a sorting module, configured to sort multiple intermediate vectors according to the probability of each intermediate vector being selected, to obtain sorted intermediate vectors.

[0227] In some embodiments, the fusion module 4552 is further configured to fuse the sorted intermediate vectors with the candidate translation vectors of the first i-1 elements of the sentence to be translated.

[0228] In some embodiments, the fusion module 4552 is further configured to fuse the intermediate vectors with the candidate translation vectors of the first i-1 elements of the sentence to be translated in sorted order, so as to obtain multiple fused candidate translation vectors corresponding to any one of the multiple intermediate vectors.

[0229] In some embodiments, the fusion module 4552 is further configured to take at least two adjacent intermediate vectors among the sorted intermediate vectors as a batch, and different batches include different intermediate vectors, and the order between different batches satisfies the sorting order, and perform the following processing for each batch: fuse at least two intermediate vectors in the batch with the candidate translation vectors of the first i-1 elements of the sentence to be translated in parallel to obtain multiple fused candidate translation vectors corresponding to at least two intermediate vectors in the batch.

[0230] This application provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the artificial intelligence-based translation method described above in this application.

[0231] This application provides a computer-readable storage medium storing executable instructions. When these executable instructions are executed by a processor, they cause the processor to execute the AI-based translation method provided in this application. For example, ... Figure 3 The translation method based on artificial intelligence is shown.

[0232] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EP ROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0233] In some embodiments, executable instructions may take the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0234] As an example, executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple collaborative files (e.g., files that store one or more modules, subroutines, or code sections).

[0235] As an example, executable instructions can be deployed to execute on a single computing device, or on multiple computing devices located in one location, or on multiple computing devices distributed across multiple locations and interconnected via a communication network.

[0236] In summary, the embodiments of this application have the following beneficial effects:

[0237] (1) Store the candidate translation vectors whose corresponding conditional probability is not less than the lower bound probability in the data structure. Since the data structure stores K candidate translation vectors whose corresponding conditional probability is greater than the lower bound probability in the first i translation processes, it ensures that the data structure stores the K candidate translation vectors that best match the sentence to be translated during the search process. Furthermore, decode the K candidate translation vectors that best match the sentence to be translated during the search process to obtain the translation results of the first i elements of the sentence to be translated, so as to accurately search for the best matching translation result and significantly improve the accuracy of the translation result.

[0238] (2) Take at least two adjacent intermediate vectors from the sorted intermediate vectors as a batch, and different batches include different intermediate vectors. The order of different batches satisfies the sorting order. For each batch, perform the following processing: fuse at least two intermediate vectors in the batch with the candidate translation vectors of the first i-1 elements of the sentence to be translated in parallel to obtain multiple fused candidate translation vectors corresponding to at least two intermediate vectors in the batch. By using the batch parallel fusion processing method, the number of times conditional probabilities need to be calculated through the probability prediction model can be effectively reduced.

[0239] (3) Based on the probability of each intermediate vector being selected, the multiple intermediate vectors are sorted to obtain sorted intermediate vectors. Furthermore, when the number of candidate translation vectors in the data structure exceeds K, the candidate translation vector with the minimum conditional probability is deleted until the number of candidate translation vectors in the data structure is K. The lower bound probability is then updated to the minimum conditional probability of the K candidate translation vectors in the data structure. Since the lower bound probability value is constantly updated, by sorting the multiple intermediate vectors, the fused candidate translation vectors also have the same sorting order. Therefore, during the process of comparing the lower bound probability sequentially according to the sorting order, if the conditional probability of any candidate translation vector is smaller than the lower bound probability, the conditional probability of the candidate translation vectors following that candidate translation vector will also be smaller than the lower bound probability. Thus, the candidate translation vectors following that candidate translation vector do not need to continue comparing their probability values ​​with the lower bound probability, effectively reducing the number of probability value comparisons, significantly improving running efficiency, and effectively shortening the running time.

[0240] The above are merely embodiments of this application and are not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.< / eos>

Claims

1. An artificial intelligence-based translation method, characterized in that, The method includes: The i-th element in the sentence to be translated is encoded into the i-th vector, and the i-th vector is matched with the translation vocabulary to obtain multiple intermediate vectors, where i is an integer variable greater than 1; The multiple intermediate vectors are fused with the candidate translation vectors of the first i-1 elements of the sentence to be translated, and the resulting multiple fused candidate translation vectors are subjected to probability prediction to obtain the conditional probability of each candidate translation vector. The candidate translation vectors whose corresponding conditional probability is not less than the lower bound probability are stored in a data structure, wherein the data structure stores K candidate translation vectors whose corresponding conditional probability is greater than the lower bound probability in the previous i translation processes, and K is an integer constant greater than 1. When the number of candidate translation vectors in the data structure exceeds K, the candidate translation vector with the minimum conditional probability in the data structure is deleted until the number of candidate translation vectors in the data structure is K; the lower bound probability is updated to the minimum conditional probability of the K candidate translation vectors in the data structure. The candidate translation vectors in the data structure are decoded to obtain the translation results of the first i elements of the sentence to be translated.

2. The method according to claim 1, characterized in that, Candidate translation vectors with conditional probabilities not less than the lower bound are stored in a data structure, including: The conditional probability of each candidate translation vector is compared with the lower bound probability to obtain at least one candidate translation vector whose probability is not less than the lower bound probability. The candidate translation vector with a probability not less than the lower bound is stored in the data structure.

3. The method according to claim 1, characterized in that, The step of decoding the candidate translation vectors in the data structure to obtain the translation results of the first i elements of the sentence to be translated includes: The K candidate translation vectors in the data structure are sorted and filtered to obtain the candidate translation vector with the highest conditional probability among the K candidate translation vectors; Based on the candidate translation vector with the highest conditional probability among the K candidate translation vectors, the decoder is invoked to perform decoding processing to obtain the translation results of the first i elements of the sentence to be translated.

4. The method according to claim 1, characterized in that, The process of fusing the plurality of intermediate vectors with the candidate translation vectors of the first i-1 elements of the sentence to be translated includes: The plurality of intermediate vectors are fused with the candidate translation vectors of the first i-1 elements of the sentence to be translated to obtain the plurality of fused candidate translation vectors corresponding to the plurality of intermediate vectors; The step of performing probability prediction processing on the multiple fused candidate translation vectors to obtain the conditional probability of each candidate translation vector includes: Based on the multiple fused candidate translation vectors, the translation model is invoked to perform probabilistic prediction processing to obtain the conditional probability of each candidate translation vector.

5. The method according to claim 4, characterized in that, The step of fusing the plurality of intermediate vectors with the candidate translation vectors of the first i-1 elements of the sentence to be translated, respectively, to obtain the plurality of fused candidate translation vectors corresponding to the plurality of intermediate vectors, includes: For each of the plurality of intermediate vectors, perform the following processing: The intermediate vector is fused with the candidate translation vectors of the first i-1 elements of the sentence to be translated to obtain the multiple fused candidate translation vectors corresponding to any one of the multiple intermediate vectors.

6. The method according to claim 4, characterized in that, The step of fusing the plurality of intermediate vectors with the candidate translation vectors of the first i-1 elements of the sentence to be translated, respectively, to obtain the plurality of fused candidate translation vectors corresponding to the plurality of intermediate vectors, includes: Take any two intermediate vectors from the plurality of intermediate vectors as a batch, and different batches include different intermediate vectors. Perform the following processing on each batch: At least two intermediate vectors from the batch are fused in parallel with the candidate translation vectors of the first i-1 elements of the sentence to be translated to obtain the plurality of fused candidate translation vectors corresponding to at least two intermediate vectors from the batch.

7. The method according to claim 1, characterized in that, After matching the i-th vector with the translation vocabulary to obtain multiple intermediate vectors, the method further includes: Based on the probability of each of the intermediate vectors being selected, the plurality of intermediate vectors are sorted to obtain the sorted plurality of intermediate vectors; The process of fusing the plurality of intermediate vectors with the candidate translation vectors of the first i-1 elements of the sentence to be translated includes: The sorted intermediate vectors are then fused with the candidate translation vectors of the first i-1 elements of the sentence to be translated.

8. The method according to claim 7, characterized in that, The step of fusing the sorted intermediate vectors with the candidate translation vectors of the first i-1 elements of the sentence to be translated includes: For each of the sorted intermediate vectors, perform the following processing: Following the sorting order, the intermediate vectors are fused with the candidate translation vectors of the first i-1 elements of the sentence to be translated, respectively, to obtain the plurality of fused candidate translation vectors corresponding to any one of the plurality of intermediate vectors; or, Take at least two adjacent intermediate vectors from the sorted intermediate vectors as a batch, and different batches include different intermediate vectors. The order between the different batches satisfies the sorting order. For each batch, perform the following processing: merge at least two intermediate vectors in the batch with the candidate translation vectors of the first i-1 elements of the sentence to be translated to obtain the multiple fused candidate translation vectors corresponding to at least two intermediate vectors in the batch.

9. A translation device based on artificial intelligence, characterized in that, include: The matching module is used to encode the i-th element in the sentence to be translated into the i-th vector, and match the i-th vector with the translation vocabulary to obtain multiple intermediate vectors, where i is an integer variable greater than 1; The fusion module is used to fuse the multiple intermediate vectors with the candidate translation vectors of the first i-1 elements of the sentence to be translated. The prediction module is used to perform probability prediction processing on the multiple fused candidate translation vectors obtained to obtain the conditional probability of each candidate translation vector; The storage module is used to store candidate translation vectors whose corresponding conditional probabilities are not less than the lower bound probability into a data structure. The data structure stores K candidate translation vectors whose corresponding conditional probabilities are greater than the lower bound probability in the previous i translation processes, where K is an integer constant greater than 1. The deletion module is used to delete the candidate translation vector with the minimum conditional probability in the data structure when the number of candidate translation vectors in the data structure exceeds K, until the number of candidate translation vectors in the data structure is K; The update module is used to update the lower bound probability to the minimum conditional probability of the K candidate translation vectors in the data structure; The decoding module is used to decode the candidate translation vectors in the data structure to obtain the translation results of the first i elements of the sentence to be translated.

10. The artificial intelligence-based translation device according to claim 9, characterized in that, The storage module is also used for: The conditional probability of each candidate translation vector is compared with the lower bound probability to obtain at least one candidate translation vector whose probability is not less than the lower bound probability. The candidate translation vector with a probability not less than the lower bound is stored in the data structure.

11. The artificial intelligence-based translation device according to claim 9, characterized in that, The decoding module is also used for: The K candidate translation vectors in the data structure are sorted and filtered to obtain the candidate translation vector with the highest conditional probability among the K candidate translation vectors; Based on the candidate translation vector with the highest conditional probability among the K candidate translation vectors, the decoder is invoked to perform decoding processing to obtain the translation results of the first i elements of the sentence to be translated.

12. The artificial intelligence-based translation device according to claim 9, characterized in that, The fusion module is also used for: The plurality of intermediate vectors are fused with the candidate translation vectors of the first i-1 elements of the sentence to be translated to obtain the plurality of fused candidate translation vectors corresponding to the plurality of intermediate vectors; The prediction module is also used for: Based on the multiple fused candidate translation vectors, the translation model is invoked to perform probabilistic prediction processing to obtain the conditional probability of each candidate translation vector.

13. The artificial intelligence-based translation device according to claim 12, characterized in that, The fusion module is also used for: For each of the plurality of intermediate vectors, perform the following processing: The intermediate vector is fused with the candidate translation vectors of the first i-1 elements of the sentence to be translated to obtain the multiple fused candidate translation vectors corresponding to any one of the multiple intermediate vectors.

14. The artificial intelligence-based translation device according to claim 12, characterized in that, The fusion module is also used for: Take any two intermediate vectors from the plurality of intermediate vectors as a batch, and different batches include different intermediate vectors. Perform the following processing on each batch: At least two intermediate vectors from the batch are fused in parallel with the candidate translation vectors of the first i-1 elements of the sentence to be translated to obtain the plurality of fused candidate translation vectors corresponding to at least two intermediate vectors from the batch.

15. The artificial intelligence-based translation device according to claim 9, characterized in that, The AI-based translation device further includes: The sorting module is used to sort the plurality of intermediate vectors according to the probability of each of the intermediate vectors being selected, so as to obtain the sorted plurality of intermediate vectors. The fusion module is also used for: The sorted intermediate vectors are then fused with the candidate translation vectors of the first i-1 elements of the sentence to be translated.

16. The artificial intelligence-based translation device according to claim 15, characterized in that, The fusion module is also used for: For each of the sorted intermediate vectors, perform the following processing: Following the sorting order, the intermediate vectors are fused with the candidate translation vectors of the first i-1 elements of the sentence to be translated, respectively, to obtain the plurality of fused candidate translation vectors corresponding to any one of the plurality of intermediate vectors; or, Take at least two adjacent intermediate vectors from the sorted intermediate vectors as a batch, and different batches include different intermediate vectors. The order between the different batches satisfies the sorting order. For each batch, perform the following processing: merge at least two intermediate vectors in the batch with the candidate translation vectors of the first i-1 elements of the sentence to be translated to obtain the multiple fused candidate translation vectors corresponding to at least two intermediate vectors in the batch.

17. A computer-readable storage medium storing executable instructions, characterized in that, When the executable instructions are executed by the processor, the processor causes the processor to execute the artificial intelligence-based translation method according to any one of claims 1 to 8.

18. A computer program product, the computer program product comprising computer instructions, characterized in that, The computer instructions are stored in a computer-readable storage medium, and the processor of the computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the artificial intelligence-based translation method according to any one of claims 1 to 8.

Images