A machine translation post-processing method for HTML tag translation

Abstract

The present invention discloses a machine translation post-processing method for HTML tag translation, comprising a labeled source language sentence, an unlabeled source language sentence, a machine translation model, an HTML post-processing model, and a labeled translation. The present invention has the beneficial effects of: the method is divided into a machine translation stage and a tag translation stage, achieving decoupling between text content translation and HTML tag translation, thereby hardly affecting the quality of machine translation. Because the machine translation model and the post-processing model are unrelated, the post-processing model can be used in conjunction with any other commercial machine translation engine, thereby improving flexibility. In addition, the post-processing model can also mark the content of the original text and the target text, and can thus also be used for other purposes such as term replacement.

Description

Technical Field

[0001] The present invention relates to a method for processing machine-translated text, in particular to a method for post-processing machine translation for HTML tag translation, and belongs to the technical field of neural machine translation. Background Art

[0002] Neural machine translation has become the mainstream translation method due to its excellent translation quality. Specifically, its translation method accepts a source language sentence as input and then outputs a target language sentence in an end-to-end manner.

[0003] However, there are some problems when machine translation engines handle the translation of web page content: since web page content contains a large number of HTML tag sentences, the HTML tags in these sentences play a role in formatting the web page text. If the machine translation engine ignores these HTML tags when translating the text, the translation result will lose the formatting information of the web page, and users may see text content without any typeset. The loss of important HTML tags may even cause the web page to fail to load, resulting in the user being unable to see any information. Summary of the Invention

[0004] The purpose of the present invention is to provide a machine translation post-processing method for HTML tag translation in order to solve at least one of the above technical problems. When translating web page text, this machine translation post-processing method not only correctly translates the text content in the web page, but also correctly migrates the HTML tags to the target sentence.

[0005] The present invention achieves the above-mentioned purpose through the following technical solutions: A machine translation post-processing method for HTML tag translation comprises the following steps:

[0006] Step 1: There are labeled source language sentences. Extract the text content in the labeled source language sentences and obtain the labeled source language sentences S of the input text content. tag ;

[0007] Step 2: Unlabeled source language sentences, use regular expressions to identify S tag All the tags in , and delete all the tags to get the unlabeled source language sentence S no_tag ;

[0008] Step 3: Using a trained machine translation model to translate the source language sentence into a machine translation.

[0009] Step 4: Train the HTML post-processing model by synthesizing HTML data to obtain the post-processing model PE;

[0010] Step 5: Labeled translation, label the source language sentence Stag and the untagged translation T no_tag Splicing is used as input, and the trained post-processing model PE is used to obtain the labeled translation T tag .

[0011] As a further solution of the present invention: the step three specifically includes:

[0012] a. Prepare parallel corpus data (S, T);

[0013] b. Use parallel corpus to train the machine translation model MT;

[0014] c. Steps a and b can be replaced by using other existing commercial machine translation models;

[0015] d. Use the machine translation model MT to translate the unlabeled source language sentence S no_tag Translate into untagged translation T no_tag .

[0016] As a further solution of the present invention: the step 4 specifically includes:

[0017] a. Prepare parallel corpus data (S, T) and phrase translation table data (S phrase , T phrase );

[0018] b. For each sentence pair in the parallel corpus, determine whether there is a phrase pair that only appears once. If so, add a pair of random tags before and after the phrase to obtain (S, T, S tag , T tag )Such quadruple data;

[0019] c. Filter the quadruple data according to the node correspondence: If the phrase S in the source language S phrase contains a subphrase, and the target sentence corresponds to T phrase If there is no corresponding sub-phrase, the data is deleted;

[0020] d. Replace S in the quadruple data tag and T are concatenated as the input of the HTML post-processing model, T tag The post-processing model PE is trained as a training target of the post-processing model.

[0021] The beneficial effects of the present invention are:

[0022] 1) This method divides the HTML page into a machine translation stage and an HTML tag translation stage: the machine translation stage is only responsible for translating the text content in the HTML page into the target language, while the HTML tag translation stage is responsible for migrating the tags in the HTML text into the target sentence;

[0023] 2) The HTML tag translation stage does not have any other requirements for the machine translation stage, thus achieving decoupling between text content translation and HTML tag translation. Therefore, the machine translation stage in this method can use any other commercial machine translation engine, which improves flexibility;

[0024] 3) Since the tags in HTML text are invisible during the machine translation stage, this method blocks the impact of HTML tags on the quality of machine translation. This allows for both high-quality machine translation results and the preservation of HTML tags in the translation results, thereby enabling typesetting and modification of the translation results.

[0025] 4) Because the post-processing model in the HTML tag translation stage can also play the role of marking the content of the original text and the target text, this method can even be used for other purposes such as term replacement. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0027] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0028] Example 1

[0029] like Figure 1 As shown, a machine translation post-processing method for HTML tag translation includes the following steps

[0030] First: For labeled source language sentences, extract the text content in the labeled source language sentences and obtain the labeled source language sentences S of the input text content. tag , and use regular expressions to identify S tag All the tags in , and delete all the tags to get the unlabeled source language sentence S no_tag ;

[0031] Second: a machine translation model, using a trained machine translation model to translate the text content of the source language sentence to obtain a machine translation.

[0032] Specifically including: a. Prepare parallel corpus data (S, T); b. Use parallel corpus to train machine translation model MT; c. Steps a and b can be replaced by using other existing commercial machine translation models (the machine translation model consists of two parts: model structure and model parameters. The defined model structure will not change, while the model parameters are learned from the data through training. Without training, the model parameters are random and meaningless. At this time, the machine translation model cannot be used normally. Therefore, a machine translation model that can work is trained using training data, i.e. parallel corpus. All commercial machine translation engines are trained. Therefore, using a public commercial machine translation model can replace the two steps of preparing data and training the translation model yourself); d. Use the machine translation model MT to convert the unlabeled source language sentence S no_tag Translate into untagged translation T no_tag .

[0033] Third: HTML post-processing model, which takes the labeled source language sentences and machine translation text as input to the HTML post-processing model to obtain the labeled machine translation text.

[0034] Specifically include: a) training HTML post-processing model; b) converting the labeled source language sentence S tag and the untagged translation T no_tag splicing as input; c) using the trained post-processing model PE to obtain the labeled translation T tag .

[0035] The HTML post-processing model processing process includes: 1) preparing parallel corpus data (S, T) and phrase translation table data (S phrase , T phrase ); 2) For each sentence pair in the parallel corpus, determine whether there is a phrase pair that only appears once. If so, add a pair of random tags before and after the phrase to obtain (S, T, S tag , T tag ) such quad-tuple data; 3) filtering the quad-tuple data according to the node correspondence: if the phrase S in the source language S phrase contains a subphrase, and the target sentence corresponds to T phrase If there is no corresponding sub-phrase, delete the data; 4) S in the quadruple data tag and T are concatenated as the input of the HTML post-processing model, T tag The post-processing model PE is trained as a training target of the post-processing model.

[0036] Further explanation is needed: a sentence pair refers to a source language sentence and its corresponding target language sentence (the translation sentence T of the source language sentence S); a phrase refers to a sentence fragment (also known as a shorter sentence); a phrase pair is similar to a sentence pair, which refers to two shorter sentence fragments (a source language sentence fragment, a target language sentence fragment).

[0037] S is the training data source language sentence; T is the training data target language sentence; S tag T is the generated source language sentence with labels or the labeled sentence in the HTML page; tag The generated labeled target language sentences or the labeled translations output by the post-processing model; S no_tag The original HTML sentence with tags is the sentence after the tags are deleted. Unlike the above S, S is the training data sentence and has no tags.

[0038] A pair of random labels is , so that a random HTML start tag and the corresponding HTML end tag are added before and after the phrase respectively to wrap the phrase.

[0039] For step 3), let’s take an example to illustrate:

[0040] Original: For <hb> many rich families< / hb> ,the other part of the solution isbodyguards– <hr> lots ofthem.

[0041] Translation: Yes <hb> <hr> Many wealthy families< / hb> Another part of the response is bodyguards—lots of them.

[0042] The original phrase <hb> many rich families< / hb> No subtags, but corresponding phrases in the translation <hb> <hr> Many wealthy families< / hb> Also contains sub-tags <hr> Many, therefore the data is wrong and needs to be deleted;

[0043] Example 2

[0044] A machine translation post-processing method for HTML tag translation includes the following steps:

[0045] 1. Enter the original text with tags:

[0046] And the light breeze <hj>moves <hp>me <hk> to caress< / hk> < / hp> < / hj> herlong ear

[0047] In this example, <hj>< / hj> 、 <hp>< / hp> 、 <hk>< / hk>Use regular expressions or third-party HTML parsing tools such as Beautiful Soup to identify these tags and delete them to get the original text without tags.

[0048] 2. Clear the tags to get the untagged original text:

[0049] And the light breeze moves me to caress her long ear

[0050] 3. Use the machine translation model to translate the text:

[0051] The breeze pushes me to caress its long ears

[0052] 4. Take the labeled original text and the machine translation as input, and use the post-processing model to obtain the labeled translation:

[0053] New input:

[0054] And the light breeze <hj>moves <hp>me <hk> to caress< / hk> < / hp> < / hj> herlong ear

[0055] The breeze pushes me to caress its long ears

[0056] Output of the post-processing model:

[0057] breeze <hj>Pushing <hp>I <hk> to caress< / hk> < / hp> < / hj> Its long ears

[0058] The labels in the output are the results automatically output by the post-processing model.

[0059] Working principle: Extract the text content from the labeled source language sentence; use the trained machine translation model to translate the text content of the source language sentence to obtain the machine translation version; use the labeled source language sentence and the machine translation version as input to the HTML post-processing model to obtain the labeled machine translation version.

[0060] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the invention can be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims, not the foregoing description, and all variations within the meaning and range of equivalents of the claims are intended to be included therein. Any reference sign in a claim should not be construed as limiting the claim to which it relates.

[0061] In addition, it should be understood that although this specification is described in terms of implementation methods, not every implementation method contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

Claims

1. A machine translation post-processing method for HTML tag translation, characterized in that: The machine translation post-processing method comprises the following steps: 1) Extract the text content from the labeled source language sentence. For the input labeled source language sentence, record it as S tag ; and use regular expressions to identify S tag All the tags in , and delete all the tags to get the unlabeled source language sentence S no_tag ; 2) using a machine translation model to translate the text content of the source language sentence to obtain a machine translation; 3) using the labeled source language sentence and the machine translation text as input to an HTML post-processing model to obtain a labeled machine translation text; The step 3) includes: a) Train HTML post-processing model; b) The labeled source language sentence S tag and the untagged translation T no_tag splicing as input; c) Use the trained post-processing model PE to obtain the labeled translation T tag .

2. The machine translation post-processing method according to claim 1, characterized in that: The step 2) specifically includes: a) Prepare parallel corpus data (S, T); b) Using parallel corpora to train machine translation models; c) Use the machine translation model MT to translate the unlabeled source language sentence S no_tag Translate into untagged translation T no_tag .

3. The machine translation post-processing method according to claim 1, wherein: The HTML post-processing model includes: 1) Prepare parallel corpus data (S, T) and phrase translation table data (S phrase , T phrase ); 2) For each sentence pair in the parallel corpus data, determine whether there is a phrase pair in each sentence pair, and the phrase pair only appears once. If so, add a pair of random tags before and after the phrase to obtain (S, T, S tag , T tag ) such quadruple data; 3) Filter the quadruple data according to the node correspondence: If the phrase S in the source language S phrase contains a sub-phrase, and the target sentence corresponds to T phrase If there is no corresponding sub-phrase, the data is deleted; 4) S in the quadruple data tag and T are concatenated as the input of the HTML post-processing model, T tag The post-processing model PE is trained as the training target of the html post-processing model.

Images