Text character-based machine translation method, device and equipment and storage medium
Patent Information
- Application Number
- CN202611175448.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-04
- Publication Date
- 2026-10-27
Smart Images
Figure CN122987973A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of machine translation technology, specifically to a machine translation method, apparatus, device, and storage medium based on text glyphs. Background Technology
[0002] Significant progress has been made in machine translation technology based on deep neural networks, achieving excellent translation results in machine translation tasks between high-resource languages. Deep neural network machine translation models are trained using a deep neural network architecture.
[0003] Because deep neural network machine translation models have a large number of parameters, they require large-scale bilingual parallel sentences for training. This condition is relatively easy to meet for high-resource languages, but for low-resource languages, the construction of bilingual parallel sentence datasets is difficult and costly. As a result, neural machine translation models cannot achieve ideal translation results in low-resource language translation tasks due to insufficient training data.
[0004] In related technologies, the character vectors used in machine translation models are obtained by training based on the contextual information of characters in the text. However, when learning the character vector representations of the source and target languages, machine translation methods based on character vectors usually establish a mapping relationship between the character tokens in the text and their corresponding word embedding vectors in the form of ID numbers.
[0005] In this approach, the representational power of characters is limited, and for words that do not appear in the training data, it is impossible to obtain the corresponding character vectors, which leads to a decrease in the translation quality of the model. For characters with spelling errors, it is difficult to automatically denoise and correct them into vector representations that conform to the contextual semantics, resulting in poor robustness of the model to input noise. Summary of the Invention
[0006] This invention provides a machine translation method, apparatus, device, and storage medium based on text glyphs to solve the problem of poor machine translation results caused by bottlenecks such as vocabulary limitations, out-of-vocabulary words, and spelling errors in related machine translation methods.
[0007] In a first aspect, the present invention provides a machine translation method for text glyphs, the machine translation method comprising: The source language text to be translated is segmented to obtain the word sequence of the source language text; The word sequence is converted into a binary image sequence, and the binary image sequence is input into the residual network of a pre-trained machine translation model to extract the glyph vector sequence of the source language text. Generate a character vector sequence of the source language text based on the lexical sequence; Based on the glyph vector sequence and character vector sequence, an input feature vector sequence is generated; The input feature vector sequence is input into the end-to-end network of the machine translation model for machine translation to obtain the target language text.
[0008] In some embodiments, converting the word sequence into a binary image sequence includes: For each word in the word sequence, each character in the word is converted into a fixed-size binary bitmap image to obtain the image sequence corresponding to the word. The binary image sequence includes the image sequence of each word.
[0009] In some embodiments, the step of inputting the binary image sequence into the residual network of a pre-trained machine translation model to extract the glyph vector sequence of the source language text includes: The binary image sequence is input into the residual network, and the glyph features of each character are extracted through a deep convolutional network composed of stacked residual units; After processing the character line features of each character through global average pooling and a fully connected layer, the glyph vector of each character is obtained. The glyph vector sequence of the source language text includes the glyph vector of each character in the source language text.
[0010] In some embodiments, generating a character vector sequence of the source language text based on the lexical sequence includes: For each lexical in the lexical sequence, the character vector corresponding to the lexical is obtained according to the index of the lexical in the preset vocabulary. The character vector sequence includes the character vectors corresponding to each lexical.
[0011] In some embodiments, generating an input feature vector sequence based on the glyph vector sequence and the character vector sequence includes: Use the sequence of character shape vectors as the input feature vector sequence; or... The glyph vector sequence and the character vector sequence are fused to obtain the input feature vector sequence.
[0012] In some embodiments, the step of fusing the glyph vector sequence and the character vector sequence to obtain the input feature vector sequence includes: The character vector sequence and the glyph vector sequence are concatenated along the channel dimension to obtain a concatenated vector sequence; The concatenated vector sequence is input into a fully connected network, and the concatenated features are fused through the linear transformation and nonlinear activation function of the fully connected network to obtain the input feature vector sequence.
[0013] In some embodiments, the machine translation model is trained using the following methods: Obtain the training dataset, each training dataset including source language text samples and target language text samples; The source language text sample and the target language text are respectively segmented to obtain the source language word sequence and the target language word sequence; The source language word sequence and the target language word sequence are respectively converted into corresponding binary image sequences; The binary image sequences of the source language and the target language are respectively input into the residual network of the machine translation model to be trained for glyph feature extraction, resulting in glyph vector sequences of the source language and the target language. Generate character vector sequences for the source language and the target language respectively based on the source language lexical sequence and the target language lexical sequence; Based on the glyph vector sequence and character vector sequence of the source language, generate the input feature vector sequence of the source language; and based on the glyph vector sequence and character vector sequence of the target language, generate the input feature vector sequence of the target language. The input feature vector sequence of the source language is input into the encoder of the end-to-end network of the machine translation model, and the output of the encoder and the input feature vector sequence of the target language are input into the decoder of the end-to-end network for end-to-end machine translation training. With minimizing the translation loss function as the optimization objective, the model parameters of the machine translation model are updated simultaneously through the backpropagation algorithm until the model converges and a trained machine translation model is obtained.
[0014] In a second aspect, the present invention provides a machine translation apparatus, the machine translation apparatus comprising: The word segmentation unit is used to segment the source language text to be translated to obtain the word sequence of the source language text. The glyph extraction unit is used to convert the word sequence into a binary image sequence and input the binary image sequence into the residual network of a pre-trained machine translation model to extract the glyph vector sequence of the source language text. A character processing unit is configured to generate a character vector sequence of the source language text based on the lexical sequence; The fusion processing unit is used to generate an input feature vector sequence based on the glyph vector sequence and the character vector sequence; The translation unit is used to input the input feature vector sequence into the end-to-end network of the machine translation model for machine translation to obtain the target language text.
[0015] Thirdly, the present invention provides an electronic device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the machine translation method of the first aspect or any corresponding embodiment described above.
[0016] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to perform the machine translation method of the first aspect or any corresponding embodiment thereof.
[0017] The machine translation method based on text glyphs provided by this invention employs a residual network to extract deep glyph features of each source character from the binary image sequence of each word in the source language text. By obtaining the input feature vector sequence of the model based on the glyph vectors and character vectors, the model can obtain a richer input representation than the character vectors, making the model's understanding of the input text more accurate and improving the accuracy and robustness of machine translation. Since the generation of glyph vectors depends only on the image information of the characters and not on their frequency of occurrence in the training data, effective vector representations can be generated based on the glyphs of words not appearing in the training data, allowing the model to process and translate them correctly. This overcomes the limitation of vocabulary size and effectively solves the bottleneck problem of out-of-vocabulary words. Even when characters have spelling errors, their glyphs usually still maintain the same appearance as the correct characters. Maintaining high similarity, the model can extract representations close to actual words from noisy input through glyph vectors, achieving automatic error correction and denoising. This enhances the ability to correct misspelled characters, enabling the model to maintain good translation quality even when faced with input text containing spelling errors. Since glyph feature extraction is based on binary images of words, and characters from all languages can be converted into binary images of a unified format, the glyph feature extraction process is not limited by language type and can be applied to characters of various languages. It supports machine translation model training between any language pairs, possessing good versatility and not limited to specific languages. It can utilize the deep character features and glyph features of text to fully extract multi-dimensional feature information from limited data for translation model training, thereby achieving better translation results under data-constrained conditions, which is beneficial for training translation models for low-resource languages. Attached Figure Description
[0018] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0019] Figure 1 This is a schematic diagram illustrating an application scenario of an embodiment of the present invention; Figure 2 A flowchart illustrating a machine translation method based on text glyphs provided in an embodiment of the present invention; Figure 3 A graphical representation of multi-character lexical units and single-character lexical units; Figure 4 This is a schematic diagram of the model architecture of the machine translation model in an embodiment of the present invention; Figure 5 This is a schematic diagram of the network structure of a residual network in an embodiment of the present invention; Figure 6 This is a structural block diagram of a machine translation device provided in an embodiment of the present invention; Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation
[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0021] Figure 1 This is a schematic diagram of one application scenario of the present invention. As an optional application scenario of the present invention, such as... Figure 1 As shown, a machine translation system may include at least one terminal device and at least one server. Figure 1 The system is illustrated in the example, which includes a computer 101, a mobile terminal 102, and a server 103, and the terminal devices such as the computer 101 and the mobile terminal 102 are connected to the server 103 through a network 110.
[0022] Specifically, the terminal device can be a smartphone, tablet, laptop, PDA, desktop computer, game console, smart TV, smart wearable device, in-vehicle terminal, VR (Virtual Reality) device, AR (Augmented Reality) device, etc. Server 103 can be a standalone physical server, a server cluster, a distributed system, or a cloud server providing cloud services. Network 110 can be a wired or wireless network, examples of which include, but are not limited to, the Internet, corporate intranet, local area network, wide area network, mobile communication network, and combinations thereof.
[0023] According to an embodiment of the present invention, a machine translation method based on text glyphs is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.
[0024] This invention provides a machine translation method based on text glyphs, which can be used in the aforementioned terminal devices or servers. Figure 2 A flowchart illustrating a text-based glyph-based machine translation method provided in this embodiment of the invention is shown below. Figure 2 As shown, the machine translation method includes the following steps: steps S201 to S205.
[0025] Step S201: Perform word segmentation on the source language text to be translated to obtain the word sequence of the source language text.
[0026] It receives source language text to be translated from user input. The source language text is text described in the source language (e.g., Chinese, English).
[0027] In machine translation tasks, the input source language text first needs to be segmented into word sequences, which are collections of words in the source language text. Because different languages use different character sets, the number of characters contained in each word sequence after segmentation varies. For example, Chinese can be segmented directly according to Chinese characters, with each word containing only one Chinese character; while in English, each word may contain one or more alphabetic characters.
[0028] Step S202: Convert the word sequence into a binary image sequence, and input the binary image sequence into the residual network of the pre-trained machine translation model to extract the glyph vector sequence of the source language text.
[0029] In this embodiment of the invention, the binary image sequence is a collection of image sequences of each word in the word sequence.
[0030] In some embodiments, in step S202, converting the word sequence into a binary image sequence includes: for each word in the word sequence, converting each character in the word into a fixed-size binary bitmap image to obtain the image sequence corresponding to the word, wherein the binary image sequence includes the image sequence of each word.
[0031] In some application scenarios, for multi-character languages that use letters, such as English, each word in the word sequence is first decomposed into a letter sequence. Then, each letter is converted into a binary bitmap image of the corresponding font, thus obtaining an image sequence composed of binary bitmap images of multiple letters. The binary bitmap image of each letter adopts a uniform size (e.g., 28×28 pixels) and is represented in the form of a fixed-width binary bitmap.
[0032] In some application scenarios, for single-character languages such as Chinese, each word in the word sequence is directly converted into a binary bitmap image of the corresponding font. Since each word is a character, the length of the image sequence corresponding to each word is 1. The image also adopts the same size and format as multi-character languages that use letters.
[0033] In the process of image representation, the differences in linguistic features between single-character and multi-character lexical characters are fully considered, enabling the converted images to more accurately reflect the glyph characteristics of different character types. To achieve a unified representation framework, single-character characters such as Chinese are represented as image sequences of length 1, while multi-character characters such as English are represented as image sequences equal to the number of letters. Finally, a source language text of length N (with N lexical units) is converted into a tensor of N image sequences, each image sequence corresponding to one lexical unit. This scheme has good versatility and scalability, and can uniformly support characters from different languages.
[0034] Figure 3 This is a graphical illustration of multi-character and single-character lexical units, exemplified as follows: Figure 3 As shown, in the English scenario, taking the multi-character word "cat" as an example, the word "cat" is converted into an image sequence composed of binary bitmap images of the character "c", the character "a", and the character "t". In the Chinese scenario, taking the single-character word "cat" as an example, the word "cat" is converted into an image sequence composed of binary bitmap images of the character "cat".
[0035] Figure 4 This is a schematic diagram of the model architecture of the machine translation model in an embodiment of the present invention. In an embodiment of the present invention, as shown... Figure 4 As shown, the machine translation model includes a glyph input layer and a model layer. The glyph input layer includes a residual network for extracting glyph features. The glyph input layer uses the residual network to convert the input binary image sequence into a glyph vector sequence. The model layer includes an end-to-end network, which is used to train and infer the translation of the input feature vector sequence.
[0036] In some embodiments, in step S202, the binary image sequence is input into the residual network of a pre-trained machine translation model to extract the glyph vector sequence of the source language text, including: The binary image sequence is input into the residual network, and the glyph features of each character are extracted through a deep convolutional network composed of stacked residual units; After processing the glyph features of each character through global average pooling and fully connected layers, the glyph vector of each character is obtained. The glyph vector sequence of the source language text includes the glyph vector of each character in the source language text.
[0037] Figure 5 This is a schematic diagram of the network structure of a residual network in an embodiment of the present invention, as shown below. Figure 5 As shown, the main body of this residual network consists of multiple stacked residual units. Each residual unit comprises an identity mapping layer and a residual function. The identity mapping layer is implemented through a single convolutional layer with a kernel size of 1×1, ensuring the integrity of information during inter-layer transmission. The residual function consists of several convolutional layers (with kernel size 3×3) and corresponding activation functions (such as ReLU), used to learn the residual between the input and output.
[0038] The calculation process of the residual element can be expressed as follows: x l+1 =f(h(x l )+F(x l W l )); In the formula, x l+1 For the output of the residual unit, x l h(x) is the input to the residual unit. l F(x) is the identity mapping function (implemented by 1×1 convolution). l W l The residual function is composed of a 3×3 convolutional layer and its activation function. The introduction of residual units changes the optimization objective of the network from directly learning the original mapping function to learning the residual function, effectively reducing the training and convergence difficulty of deep convolutional neural networks.
[0039] like Figure 5As shown, the residual network consists of eight layers, from input to output: a 2D convolutional layer (Conv2d), an activation layer (ReLU), a max pooling layer (MaxPool), residual units (ResBlock), residual units (ResBlock), residual units (ResBlock), average pooling (AvgPool), and a fully connected and activated layer (FC&Tanh). As the network depth increases, the size of the intermediate feature maps gradually decreases, while the number of channels gradually increases. Specifically, the shallow layers maintain a large spatial resolution to capture the detailed stroke features of characters, while the higher layers gradually reduce the spatial resolution and increase the number of channels through strided convolutions or pooling operations to extract higher-level abstract glyph features.
[0040] At the network output, global average pooling is used to aggregate features from all spatial locations to obtain the feature mean for each channel. Subsequently, a fully connected layer is used to map the aggregated multi-channel features to a fixed-dimensional vector space, and a non-linear transformation is performed using the Tanh activation function to finally obtain the glyph vectors of the words.
[0041] In some embodiments, in step S202, after obtaining the above-mentioned binary image sequence, the following steps are taken: Figure 5 The residual network shown obtains the vector representation of the glyph.
[0042] Step S203: Generate a character vector sequence of the source language text based on the word sequence.
[0043] In some embodiments, generating a character vector sequence of source language text based on a lexical sequence includes: for each lexical in the lexical sequence, obtaining the character vector corresponding to the lexical according to the index of the lexical in a preset vocabulary, wherein the character vector sequence includes the character vectors corresponding to each lexical.
[0044] Specifically, the preset vocabulary is a pre-built mapping table between lexical units and lexical unit indices. By inputting a lexical unit into the preset vocabulary, the corresponding index is obtained. The embedding vector corresponding to the index is obtained by looking up the table through the embedding matrix, which is the character vector corresponding to the lexical unit.
[0045] In some embodiments, the method for converting a word sequence into a character vector may also include: TF-IDF, bag-of-words model, Word2Vec, GloVe, FastText, BERT, using custom word vectors, etc., and the embodiments of the present invention do not impose any special limitations on this.
[0046] Step S204: Generate the input feature vector sequence based on the glyph vector sequence and the character vector sequence.
[0047] In this embodiment of the invention, the machine translation model supports a pure character shape input mode and a character shape vector fusion input mode. When the pure character shape input mode is used, an input feature vector sequence is generated based on the character shape vector sequence and the character vector sequence, including: using the character shape vector sequence as the input feature vector sequence.
[0048] When using the character glyph vector fusion input mode, an input feature vector sequence is generated based on the glyph vector sequence and the character vector sequence, including: fusing the glyph vector sequence and the character vector sequence to obtain the input feature vector sequence.
[0049] In some embodiments, the glyph input layer further includes a fully connected network, which uses the fully connected network to concatenate and fuse the glyph vector sequence with the character vector sequence, enabling the model to support pure glyph input mode and character glyph vector fusion input mode.
[0050] In some embodiments, feature fusion of glyph vector sequences and character vector sequences to obtain an input feature vector sequence includes: concatenating the character vector sequence and glyph vector sequence along the channel dimension to obtain a concatenated vector sequence; inputting the concatenated vector sequence into a fully connected network, and fusing the concatenated features through linear transformation and nonlinear activation functions of the fully connected network to obtain the input feature vector sequence.
[0051] Traditional character vectors and the glyph vectors extracted in this embodiment of the invention represent two different modalities of the same text. If the two are simply added element by element, the information is only superimposed in a single dimension, making it difficult to reflect the complementary relationship between the two modalities. The resulting merged vector often fails to fully leverage the advantages of each modality.
[0052] To overcome the above problems, some embodiments employ a fusion strategy based on fully connected networks to fuse character vector sequences (of dimension d). id ) and glyph vector sequence (dimension d) glyph By concatenating along the channel dimension, we obtain a dimension d. id + d glyphThe concatenated vector sequence is then input into one or more fully connected networks. The concatenated features are deeply fused through linear transformations and nonlinear activation functions (such as ReLU or Tanh) of the fully connected network, ensuring the output dimension of the fully connected network matches the expected input dimension of the model layer. After the nonlinear transformation of the fully connected network, the information from both modalities is effectively integrated in the hidden space, resulting in a fused vector sequence. This fused vector sequence serves as the input feature vector sequence for the model layer, enabling the model to simultaneously utilize both the semantic distribution and glyphic structure information of characters. This results in a richer representational capability than using either vector alone. This fusion method effectively integrates the advantages of both modalities, significantly enhancing the model's representational ability and robustness.
[0053] Step S205: Input the input feature vector sequence into the end-to-end network of the machine translation model for machine translation to obtain the target language text.
[0054] An end-to-end network includes an encoder and a decoder, and can employ neural network structures including but not limited to Transformer encoder-decoder or Transformer pure decoder. The encoder encodes the input feature vector sequence, and the decoder decodes the output of the encoder to obtain the target language text.
[0055] In some embodiments, the machine translation model is trained using the following methods: Step 1: Obtain the training dataset. Each training dataset includes source language text samples and target language text samples.
[0056] Step 2: Perform word segmentation on the source language text sample and the target language text respectively to obtain the source language word sequence and the target language word sequence; Step 3: Convert the source language word sequence and the target language word sequence into their corresponding binary image sequences; Step 4: Input the binary image sequences of the source language and the target language into the residual network of the machine translation model to be trained to extract glyph features, and obtain the glyph vector sequence of the source language and the glyph vector sequence of the target language. Step 5: Generate character vector sequences for the source language and the target language respectively based on the source language word sequence and the target language word sequence; Step 6: Generate the input feature vector sequence of the source language based on the glyph vector sequence and character vector sequence of the source language, and generate the input feature vector sequence of the target language based on the glyph vector sequence and character vector sequence of the target language. Step 7: Input the input feature vector sequence of the source language into the encoder of the end-to-end network of the machine translation model, and input the output of the encoder and the input feature vector sequence of the target language into the decoder of the end-to-end network for end-to-end machine translation training. Step 8: With minimizing the translation loss function as the optimization objective, the model parameters of the machine translation model are updated simultaneously through the backpropagation algorithm until the model converges and a trained machine translation model is obtained.
[0057] Translation loss functions can include cross-entropy, and backpropagation algorithms can include gradient descent.
[0058] According to the text glyph-based machine translation method of this invention, a residual network is used to extract deep glyph features of each source character from the binary image sequence of each word in the source language text. The input feature vector sequence of the model is obtained by combining the glyph vectors and character vectors, enabling the model to obtain a richer input representation than the character vectors. This makes the model's understanding of the input text more accurate, improving the accuracy and robustness of machine translation. Since the generation of glyph vectors depends only on the image information of the characters and not on their frequency of occurrence in the training data, effective vector representations can be generated based on the glyphs of words not appearing in the training data, allowing the model to process and translate them correctly. This overcomes the limitation of vocabulary size and effectively solves the bottleneck problem of out-of-vocabulary words. Even when characters have spelling errors, their glyphs usually remain highly similar to the correct characters. The model can extract representations close to actual words from noisy inputs using glyph vectors, enabling automatic error correction and denoising. This enhances the ability to correct misspelled characters, allowing the model to maintain good translation quality even when faced with input text containing spelling errors. Since glyph feature extraction is based on binary images of words, and characters from all languages can be converted into binary images of a unified format, the glyph feature extraction process is not limited by language type and can be applied to characters from various languages. It supports machine translation model training between any language pairs, possessing good versatility and not limited to specific languages. The machine translation method of this invention can utilize the deep character features and glyph features of text to fully extract multi-dimensional feature information from limited data for translation model training, thereby achieving better translation results under data-constrained conditions and benefiting the training of translation models for low-resource languages.
[0059] This invention also provides a machine translation apparatus for implementing the above embodiments and preferred embodiments, which will not be repeated hereafter. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.
[0060] Figure 6 This is a structural block diagram of a machine translation device provided in an embodiment of the present invention. The embodiment of the present invention provides a machine translation device, such as... Figure 6 As shown, the device includes: Word segmentation unit 601 is used to segment the source language text to be translated to obtain the word sequence of the source language text. The glyph extraction unit 602 is used to convert the word sequence into a binary image sequence and input the binary image sequence into the residual network of the pre-trained machine translation model to extract the glyph vector sequence of the source language text. The character processing unit 603 is used to generate a character vector sequence of the source language text based on the word sequence; The fusion processing unit 604 is used to generate an input feature vector sequence based on the glyph vector sequence and the character vector sequence; The translation unit 605 is used to input the input feature vector sequence into the end-to-end network of the machine translation model for machine translation to obtain the target language text.
[0061] The machine translation apparatus provided in this embodiment of the invention can execute the text-based glyph machine translation method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects for executing the method. Further functional descriptions of the various modules and units described above are the same as in the corresponding embodiments described above, and will not be repeated here.
[0062] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention.
[0063] The following is a detailed reference. Figure 7 This diagram illustrates a suitable structural schematic for implementing an electronic device according to embodiments of the present invention. The electronic device may include a processor (e.g., a central processing unit, graphics processor, etc.) 701, which can perform various appropriate actions and processes based on a program stored in read-only memory (ROM) 702 or a program loaded from memory 708 into random access memory (RAM) 703. The RAM 703 also stores various programs and data required for the operation of the electronic device. The processor 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.
[0064] Typically, the following devices can be connected to I / O interface 705: input devices 706 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 707 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; memory devices 708 including, for example, magnetic tapes, hard disks, etc.; and communication devices 709. Communication device 709 allows electronic devices to exchange data via wireless or wired communication with other devices. Although Figure 7 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.
[0065] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 709, or installed from a memory 708, or installed from a ROM 702. When the computer program is executed by the processor 701, it performs the functions defined in the text glyph-based machine translation method of the embodiments of the present invention.
[0066] Figure 7 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.
[0067] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the text-based glyph-based machine translation method shown in the above embodiments is implemented.
[0068] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.
[0069] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and all such modifications and variations fall within the scope defined by the appended claims.
Claims
1. A machine translation method based on text glyphs, characterized in that, The method includes: The source language text to be translated is segmented to obtain the word sequence of the source language text; The word sequence is converted into a binary image sequence, and the binary image sequence is input into the residual network of a pre-trained machine translation model to extract the glyph vector sequence of the source language text. Generate a character vector sequence of the source language text based on the lexical sequence; Based on the glyph vector sequence and character vector sequence, an input feature vector sequence is generated; The input feature vector sequence is input into the end-to-end network of the machine translation model for machine translation to obtain the target language text.
2. The method according to claim 1, characterized in that, Converting the word sequence into a binary image sequence includes: For each word in the word sequence, each character in the word is converted into a fixed-size binary bitmap image to obtain the image sequence corresponding to the word. The binary image sequence includes the image sequence of each word.
3. The method according to claim 1, characterized in that, The step of inputting the binary image sequence into the residual network of a pre-trained machine translation model to extract the glyph vector sequence of the source language text includes: The binary image sequence is input into the residual network, and the glyph features of each character are extracted through a deep convolutional network composed of stacked residual units; After processing the character line features of each character through global average pooling and a fully connected layer, the glyph vector of each character is obtained. The glyph vector sequence of the source language text includes the glyph vector of each character in the source language text.
4. The method according to claim 1, characterized in that, The step of generating a character vector sequence of the source language text based on the lexical sequence includes: For each lexical in the lexical sequence, the character vector corresponding to the lexical is obtained according to the index of the lexical in the preset vocabulary. The character vector sequence includes the character vectors corresponding to each lexical.
5. The method according to claim 1, characterized in that, The step of generating an input feature vector sequence based on the glyph vector sequence and the character vector sequence includes: Use the sequence of character shape vectors as the input feature vector sequence; or... The glyph vector sequence and the character vector sequence are fused to obtain the input feature vector sequence.
6. The method according to claim 5, characterized in that, The step of fusing the glyph vector sequence and the character vector sequence to obtain the input feature vector sequence includes: The character vector sequence and the glyph vector sequence are concatenated along the channel dimension to obtain a concatenated vector sequence; The concatenated vector sequence is input into a fully connected network, and the concatenated features are fused through the linear transformation and nonlinear activation function of the fully connected network to obtain the input feature vector sequence.
7. The method according to claim 1, characterized in that, The machine translation model was trained using the following method: Obtain the training dataset, each training dataset including source language text samples and target language text samples; The source language text sample and the target language text are respectively segmented to obtain the source language word sequence and the target language word sequence; The source language word sequence and the target language word sequence are respectively converted into corresponding binary image sequences; The binary image sequences of the source language and the target language are respectively input into the residual network of the machine translation model to be trained for glyph feature extraction, resulting in glyph vector sequences of the source language and the target language. Generate character vector sequences for the source language and the target language respectively based on the source language lexical sequence and the target language lexical sequence; Based on the glyph vector sequence and character vector sequence of the source language, generate the input feature vector sequence of the source language; and based on the glyph vector sequence and character vector sequence of the target language, generate the input feature vector sequence of the target language. The input feature vector sequence of the source language is input into the encoder of the end-to-end network of the machine translation model, and the output of the encoder and the input feature vector sequence of the target language are input into the decoder of the end-to-end network for end-to-end machine translation training. With minimizing the translation loss function as the optimization objective, the model parameters of the machine translation model are updated simultaneously through the backpropagation algorithm until the model converges and a trained machine translation model is obtained.
8. A machine translation device, characterized in that, The device includes: The word segmentation unit is used to segment the source language text to be translated to obtain the word sequence of the source language text. The glyph extraction unit is used to convert the word sequence into a binary image sequence and input the binary image sequence into the residual network of a pre-trained machine translation model to extract the glyph vector sequence of the source language text. A character processing unit is configured to generate a character vector sequence of the source language text based on the lexical sequence; The fusion processing unit is used to generate an input feature vector sequence based on the glyph vector sequence and the character vector sequence; The translation unit is used to input the input feature vector sequence into the end-to-end network of the machine translation model for machine translation to obtain the target language text.
9. An electronic device, characterized in that, include: A memory and a processor are communicatively connected, the memory storing computer instructions, and the processor executing the computer instructions to perform the text glyph-based machine translation method according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the text glyph-based machine translation method according to any one of claims 1 to 7.