Bilingual alignment method, device, electronic device and readable storage medium

Abstract

The present disclosure provides a method for bilingual alignment, including: obtaining an original Chinese text and an original English text, the original Chinese text and the original English text being texts with corresponding meanings; dividing the original Chinese text into sentences to obtain a sentence set of the original Chinese text; dividing the original English text into sentences to obtain a sentence set of the original English text; translating the sentence set of the original Chinese text to obtain an English translation sentence set of the sentence set of the original Chinese text; based on the English translation sentence set, the sentence set of the original Chinese text, and the sentence set of the original English text, dividing the sentence set of the original Chinese text and the sentence set of the original English text into a number of corresponding segments respectively; and aligning the corresponding segments. The present disclosure also provides a bilingual alignment device, an electronic device, and a readable storage medium.

Description

Technical Field

[0001] The present disclosure relates to the technical field of natural language processing and translation, and in particular to a method, device, electronic device and readable storage medium for bilingual alignment. Background Art

[0002] The great value of bilingual corpus resources for natural language processing research has been increasingly recognized by all sectors of society. In particular, sentence-level aligned bilingual parallel corpora have become an indispensable resource for machine translation, translation knowledge acquisition, and bilingual dictionary compilation research. This technology is mainly aimed at Chinese and English. When bilingual alignment is performed based on Chinese and English, especially when processing the alignment of tens of thousands of bilingual sentences, the existing bilingual alignment methods have the problems of low accuracy and slow speed, and cannot meet the needs of fast-response bilingual alignment scenarios. Summary of the invention

[0003] In order to solve at least one of the above technical problems, a bilingual alignment method and a readable storage medium are provided.

[0004] According to one aspect of the present disclosure, a method for bilingual alignment is provided, comprising:

[0005] Acquire an original Chinese text and an original English text, wherein the original Chinese text and the original English text are texts with corresponding meanings;

[0006] Segment the original Chinese text to obtain a sentence set of the original Chinese text;

[0007] Segment the original English text to obtain a sentence set of the original English text;

[0008] Translate the sentence set of the original Chinese text to obtain an English translation sentence set of the sentence set of the original Chinese text;

[0009] Based on the English translation sentence set, the original Chinese text sentence set and the original English text sentence set, the original Chinese text sentence set and the original English text sentence set are divided into a number of corresponding segments respectively; and,

[0010] The corresponding fragments are aligned.

[0011] According to at least one embodiment of the present disclosure, a bilingual alignment method is provided, in which an original Chinese text and an original English text are obtained, wherein the original Chinese text and the original English text are texts with corresponding meanings, including:

[0012] By extracting original Chinese text and original English text from the document.

[0013] According to at least one embodiment of the present disclosure, the bilingual alignment method is provided, the original Chinese text is divided into sentences, and a sentence set of the original Chinese text is obtained, including:

[0014] The original Chinese text is divided into sentences according to punctuation marks in the original Chinese text to obtain a sentence set of the original Chinese text.

[0015] According to at least one embodiment of the present disclosure, the bilingual alignment method is provided, wherein the original English text is divided into sentences to obtain a sentence set of the original English text, including:

[0016] The original English text is divided into sentences according to punctuation marks in the original English text to obtain a sentence set of the original English text.

[0017] According to at least one embodiment of the present disclosure, the bilingual alignment method provides a method of translating the sentence set of the original Chinese text to obtain an English translation sentence set of the sentence set of the original Chinese text, including:

[0018] The sentence set of the original Chinese text is translated by a translation engine to obtain an English translation sentence set of the sentence set of the original Chinese text.

[0019] According to at least one embodiment of the present disclosure, a bilingual alignment method is provided, based on the English translation sentence set, the original Chinese text sentence set and the original English text sentence set, dividing the original Chinese text sentence set and the original English text sentence set into a plurality of corresponding segments, including:

[0020] Find the first anchor point: calculate the similarity of the i-th sentence en[i] in the English translation sentence set and the sentences {eno[j]~eno[j+M]} in the range of j to j+M in the sentence set of the original English text one by one. If the maximum similarity is greater than the similarity threshold m, assign the subscript value of the corresponding sentence in {eno[j]~eno[j+M]} to offset according to the maximum similarity, and record the anchor point position as i, j=j+offset, and retain the anchor point position (i, j);

[0021] Find the next anchor point: calculate the similarity between the i+Lth sentence en[i+L] in the English translation sentence set and the sentences eno[j]~eno[j+M] in the range from j to j+M in the sentence set of the original English text one by one. If the maximum similarity is not greater than the similarity threshold m, it is not found, then calculate the similarity between the i+L+1th sentence en[i+L+1] in the English translation sentence set and the sentences {eno[j]~eno[j+M]} in the range from j to j+M in the sentence set of the original English text one by one. If the maximum similarity is greater than the similarity threshold m, then take the subscript value of the corresponding sentence in {eno[j]~eno[j+M]} according to the maximum similarity and assign it to offset, and record the anchor point positions as i=i+L+1, j=j+offset, and retain the second anchor point position (i, j);

[0022] According to the above process, anchor points are searched one by one until the English translation sentence set en is traversed; and,

[0023] Based on the anchor points, segmenting the sentence set zh of the original Chinese text and the sentence set eno of the original English text;

[0024] The meanings of the various parameters are as follows:

[0025] i: a natural number, initialized to 1, indicating the first sentence;

[0026] j: a natural number, initialized to 1, indicating the first sentence;

[0027] L: Anchor point step, that is, the interval step of selecting sentences, the initial value is 100, which can be adjusted;

[0028] M: The step size when selecting the sentence range during similarity calculation. The initial value is 500 and can be adjusted in the range of 1 to the number of sentences in the sentence set.

[0029] offset: the position of the sentence in the sentence set, initialized to 0;

[0030] en: English translation sentence set;

[0031] eno: a collection of sentences of the original English text;

[0032] zh: a set of sentences of original Chinese text;

[0033] m: similarity threshold.

[0034] According to the bilingual alignment method provided by at least one embodiment of the present disclosure, aligning the corresponding segments includes:

[0035] The corresponding segments are aligned using an alignment algorithm based on the translation engine.

[0036] According to at least one embodiment of the present disclosure, a bilingual alignment method is provided, which uses an alignment algorithm based on a translation engine to align corresponding segments, including:

[0037] The Bleu similarity calculation method is used to calculate the similarity between the English translation of the corresponding segment and the two sentences of the original English, where the two sentences are taken from the corresponding segments respectively, and the calculation formula is as follows:

[0038]

[0039]

[0040] The BP calculation method is as follows: matching is performed based on an N-gram matching rule, wherein the N-gram matching rule refers to matching phrases formed by N adjacent words of the two sentences, and calculating the ratio of the number of matched phrases to the total number of phrases in the translation divided by N groups of words;

[0041] The meanings of the parameters are as follows:

[0042] Bleu, represents Bleu similarity;

[0043] BP, indicates the matching degree;

[0044] Pn represents the N-gram matching accuracy.

[0045] According to another aspect of the present disclosure, a bilingual alignment device is provided, comprising:

[0046] An acquisition module, used to acquire an original Chinese text and an original English text, wherein the original Chinese text and the original English text are texts with corresponding meanings;

[0047] A Chinese sentence processing module, used for sentence segmentation of the original Chinese text to obtain a sentence set of the original Chinese text;

[0048] An English sentence processing module, used for sentence segmentation of the original English text to obtain a sentence set of the original English text;

[0049] A translation module, used for translating the sentence set of the original Chinese text to obtain an English translation sentence set of the sentence set of the original Chinese text;

[0050] A segmentation module, for dividing the sentence set of the original Chinese text and the sentence set of the original English text into a plurality of corresponding segments based on the sentence set of the English translation, the sentence set of the original Chinese text and the sentence set of the original English text; and

[0051] An alignment module is used to align the corresponding fragments.

[0052] According to a bilingual alignment device provided by at least one embodiment of the present disclosure, an original Chinese text and an original English text are obtained, wherein the original Chinese text and the original English text are texts with corresponding meanings, including:

[0053] By extracting original Chinese text and original English text from the document.

[0054] According to at least one embodiment of the present disclosure, the bilingual alignment device is provided, the original Chinese text is divided into sentences, and a sentence set of the original Chinese text is obtained, including:

[0055] The original Chinese text is divided into sentences according to punctuation marks in the original Chinese text to obtain a sentence set of the original Chinese text.

[0056] According to at least one embodiment of the present disclosure, the bilingual alignment device is provided, the original English text is divided into sentences, and a sentence set of the original English text is obtained, including:

[0057] The original English text is divided into sentences according to punctuation marks in the original English text to obtain a sentence set of the original English text.

[0058] According to the bilingual alignment device provided by at least one embodiment of the present disclosure, the sentence set of the original Chinese text is translated to obtain an English translation sentence set of the sentence set of the original Chinese text, including:

[0059] The sentence set of the original Chinese text is translated by a translation engine to obtain an English translation sentence set of the sentence set of the original Chinese text.

[0060] According to at least one embodiment of the present disclosure, a bilingual alignment device is provided, based on the English translation sentence set, the original Chinese text sentence set and the original English text sentence set, the sentence set of the original Chinese text and the sentence set of the original English text are divided into a plurality of corresponding segments, including:

[0061] Find the first anchor point: calculate the similarity of the i-th sentence en[i] in the English translation sentence set and the sentences {eno[j]~eno[j+M]} in the range of j to j+M in the sentence set of the original English text one by one. If the maximum similarity is greater than the similarity threshold m, assign the subscript value of the corresponding sentence in {eno[j]~eno[j+M]} to offset according to the maximum similarity, and record the anchor point position as i, j=j+offset, and retain the anchor point position (i, j);

[0062] Find the next anchor point: calculate the similarity between the i+Lth sentence en[i+L] in the English translation sentence set and the sentences eno[j]~eno[j+M] in the range from j to j+M in the sentence set of the original English text one by one. If the maximum similarity is not greater than the similarity threshold m, it is not found, then calculate the similarity between the i+L+1th sentence en[i+L+1] in the English translation sentence set and the sentences {eno[j]~eno[j+M]} in the range from j to j+M in the sentence set of the original English text one by one. If the maximum similarity is greater than the similarity threshold m, then take the subscript value of the corresponding sentence in {eno[j]~eno[j+M]} according to the maximum similarity and assign it to offset, and record the anchor point positions as i=i+L+1, j=j+offset, and retain the second anchor point position (i, j);

[0063] According to the above process, anchor points are searched one by one until the English translation sentence set en is traversed; and,

[0064] Based on the anchor points, segmenting the sentence set zh of the original Chinese text and the sentence set eno of the original English text;

[0065] The meanings of the various parameters are as follows:

[0066] i: a natural number, initialized to 1, indicating the first sentence;

[0067] j: a natural number, initialized to 1, indicating the first sentence;

[0068] L: Anchor point step, that is, the interval step of selecting sentences, the initial value is 100, which can be adjusted;

[0069] M: The step size when selecting the sentence range during similarity calculation. The initial value is 500 and can be adjusted in the range of 1 to the number of sentences in the sentence set.

[0070] offset: the position of the sentence in the sentence set, initialized to 0;

[0071] en: English translation sentence set;

[0072] eno: a collection of sentences of the original English text;

[0073] zh: a set of sentences of original Chinese text;

[0074] m: similarity threshold.

[0075] According to the bilingual alignment device provided by at least one embodiment of the present disclosure, aligning the corresponding segments includes:

[0076] The corresponding segments are aligned using an alignment algorithm based on the translation engine.

[0077] According to at least one embodiment of the present disclosure, a bilingual alignment device is provided, which uses an alignment algorithm based on a translation engine to align corresponding segments, including:

[0078] The Bleu similarity calculation method is used to calculate the similarity between the English translation of the corresponding segment and the two sentences of the original English, where the two sentences are taken from the corresponding segments respectively, and the calculation formula is as follows:

[0079]

[0080]

[0081] The BP calculation method is as follows: matching is performed based on an N-gram matching rule, wherein the N-gram matching rule refers to matching phrases formed by N adjacent words of the two sentences, and calculating the ratio of the number of matched phrases to the total number of phrases in the translation divided by N groups of words;

[0082] The meanings of the parameters are as follows:

[0083] Bleu, represents Bleu similarity;

[0084] BP, indicates the matching degree;

[0085] Pn represents the N-gram matching accuracy.

[0086] According to another aspect of the present disclosure, there is provided an electronic device, comprising:

[0087] a memory storing execution instructions; and,

[0088] A processor is configured to execute the execution instruction stored in the memory, so that the processor executes any one of the above-mentioned bilingual alignment methods.

[0089] According to another aspect of the present disclosure, a readable storage medium is provided, wherein the readable storage medium stores execution instructions, and when the execution instructions are executed by a processor, the execution instructions are used to implement any of the above-mentioned bilingual alignment methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0090] The accompanying drawings illustrate exemplary embodiments of the present disclosure and together with the description serve to explain the principles of the present disclosure. These drawings are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this specification.

[0091] Figure 1 The figure is a flowchart of a method for bilingual alignment according to one embodiment of the present disclosure.

[0092] Figure 2 It is a flowchart of a method for finding an anchor point according to an embodiment of the present disclosure.

[0093] Figure 3 It is a schematic diagram of N-gram matching of bleu similarity calculation according to an embodiment of the present disclosure.

[0094] Figure 4 It is a schematic diagram of the structure of a bilingual alignment device according to an embodiment of the present disclosure.

[0095] Description of Reference Numerals

[0096] 1000 Bilingual Alignment Device

[0097] 1002 Get module

[0098] 1004 Chinese sentence processing module

[0099] 1006 English sentence processing module

[0100] 1008 Translation Module

[0101] 1010 Segment Module

[0102] 1012 Alignment Module

[0103] 1100 Bus

[0104] 1200 processor

[0105] 1300 Memory

[0106] 1400 Other circuits. DETAILED DESCRIPTION

[0107] The present disclosure is further described in detail below in conjunction with the accompanying drawings and implementations. It is understood that the specific implementations described herein are only used to explain the relevant content, rather than to limit the present disclosure. It should also be noted that, for ease of description, only the parts related to the present disclosure are shown in the accompanying drawings.

[0108] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in the present disclosure can be combined with each other. The technical solution of the present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0109] Unless otherwise specified, the exemplary embodiments / embodiments shown will be understood as providing exemplary features of various details of some ways in which the technical concept of the present disclosure can be implemented in practice. Therefore, unless otherwise specified, the features of the various embodiments / embodiments can be combined, separated, interchanged and / or rearranged without departing from the technical concept of the present disclosure.

[0110] The use of cross-hatching and / or shading in the accompanying drawings is generally used to make the boundaries between adjacent components clear. As such, unless otherwise specified, the presence or absence of cross-hatching or shading does not convey or indicate any preference or requirement for the specific materials, material properties, dimensions, proportions, commonalities between the components shown, and / or any other characteristics, attributes, properties, etc. of the components. In addition, in the accompanying drawings, the sizes and relative sizes of the components may be exaggerated for clarity and / or descriptive purposes. When the exemplary embodiments can be implemented differently, the specific process sequence can be performed in a different order than described. For example, two successively described processes can be performed substantially simultaneously or in an order opposite to the described order. In addition, the same figure numbers represent the same components.

[0111] When a component is referred to as being "on," "over," "connected to," or "coupled to" another component, the component may be directly on, directly connected to, or directly coupled to the other component, or intervening components may be present. However, when a component is referred to as being "directly on," "directly connected to," or "directly coupled to" another component, there are no intervening components. For this purpose, the term "connected" may refer to a physical connection, an electrical connection, etc., with or without intervening components.

[0112] The terms used herein are for the purpose of describing specific embodiments, and are not intended to be restrictive. As used herein, unless the context clearly indicates otherwise, the singular forms "one (kind, person)" and "said (the)" are also intended to include plural forms. In addition, when the terms "comprise" and / or "include" and their variations are used in this specification, it is explained that there are stated features, integral bodies, steps, operations, parts, assemblies and / or their groups, but it is not excluded that there are or add one or more other features, integral bodies, steps, operations, parts, assemblies and / or their groups. It should also be noted that, as used herein, the terms "substantially", "approximately" and other similar terms are used as approximate terms and not as degree terms, so that they are used to explain the inherent deviations of the measured values, calculated values ​​and / or the values ​​provided that will be recognized by those of ordinary skill in the art.

[0113] Figure 1 The figure is a flowchart of a method for bilingual alignment according to one embodiment of the present disclosure.

[0114] like Figure 1 As shown, a bilingual alignment method S100 includes:

[0115] S102: Acquire an original Chinese text and an original English text, where the original Chinese text and the original English text are texts with corresponding meanings;

[0116] S104: Segment the original Chinese text to obtain a sentence set of the original Chinese text;

[0117] S106: Segment the original English text to obtain a sentence set of the original English text;

[0118] S108: translating the sentence set of the original Chinese text to obtain an English translation sentence set of the sentence set of the original Chinese text;

[0119] S110: based on the English translation sentence set, the original Chinese text sentence set and the original English text sentence set, the original Chinese text sentence set and the original English text sentence set are divided into a number of corresponding segments respectively; and,

[0120] S112: Align corresponding fragments.

[0121] The original Chinese text and the original English text are obtained, and the original Chinese text and the original English text are texts with corresponding meanings, including:

[0122] By extracting original Chinese text and original English text from the document.

[0123] The original Chinese text is divided into sentences to obtain a sentence set of the original Chinese text, including:

[0124] The original Chinese text is divided into sentences according to punctuation marks in the original Chinese text to obtain a sentence set of the original Chinese text.

[0125] The punctuation marks in the original Chinese text include: period, exclamation mark and question mark, namely: “.!?”.

[0126] The original English text is divided into sentences to obtain a sentence set of the original English text, including:

[0127] The original English text is divided into sentences according to punctuation marks in the original English text to obtain a sentence set of the original English text.

[0128] The punctuation marks in the original English text include: period, exclamation mark and question mark, namely: “.!?”.

[0129] The sentence set of the original Chinese text is translated to obtain the English translation sentence set of the sentence set of the original Chinese text, including:

[0130] The sentence set of the original Chinese text is translated by a translation engine to obtain an English translation sentence set of the sentence set of the original Chinese text.

[0131] Wherein, based on the English translation sentence set, the original Chinese text sentence set and the original English text sentence set, the original Chinese text sentence set and the original English text sentence set are divided into a number of corresponding segments, including:

[0132] Find the first anchor point: calculate the similarity between the i-th sentence en[i] in the English translation sentence set and the sentences {eno[j]~eno[j+M]} in the range j to j+M in the original English text sentence set one by one. If the maximum similarity is greater than the similarity threshold m, assign the subscript value of the corresponding sentence in {eno[j]~eno[j+M]} to offset according to the maximum similarity, and record the anchor point position as i, j=j+offset, and retain the anchor point position (i,j);

[0133] Find the next anchor point: calculate the similarity between the i+Lth sentence en[i+L] in the English translation sentence set and the sentences eno[j]~eno[j+M] in the range from j to j+M in the sentence set of the original English text one by one. If the maximum similarity is not greater than the similarity threshold m, it is not found. Then calculate the similarity between the i+L+1th sentence en[i+L+1] in the English translation sentence set and the sentences {eno[j]~eno[j+M]} in the range from j to j+M in the sentence set of the original English text one by one. If the maximum similarity is greater than the similarity threshold m, take the subscript value of the corresponding sentence in {eno[j]~eno[j+M]} according to the maximum similarity and assign it to offset, and record the anchor point positions as i=i+L+1, j=j+offset, and retain the second anchor point position (i, j);

[0134] According to the above process, anchor points are searched one by one until the English translation sentence set en is traversed; and,

[0135] Based on the anchor points, the sentence set zh of the original Chinese text and the sentence set eno of the original English text are segmented;

[0136] The meanings of the various parameters are as follows:

[0137] i: a natural number, initialized to 1, indicating the first sentence;

[0138] j: a natural number, initialized to 1, indicating the first sentence;

[0139] L: Anchor point step, that is, the interval step of selecting sentences, the initial value is 100, which can be adjusted;

[0140] M: The step size when selecting the sentence range during similarity calculation. The initial value is 500 and can be adjusted in the range of 1 to the number of sentences in the sentence set.

[0141] offset: position offset, that is, the position of the sentence in the sentence set, initialized to 0;

[0142] en: English translation sentence set;

[0143] eno: a collection of sentences of the original English text;

[0144] zh: a set of sentences of original Chinese text;

[0145] m: similarity threshold.

[0146] Among them, aligning the corresponding fragments includes:

[0147] The corresponding segments are aligned using an alignment algorithm based on the translation engine.

[0148] According to at least one embodiment of the present disclosure, a bilingual alignment method is provided, which uses an alignment algorithm based on a translation engine to align corresponding segments, including:

[0149] The Bleu similarity calculation method is used to calculate the similarity between the English translation of the corresponding segment and the two sentences of the original English. The two sentences are taken from the corresponding segments respectively. The calculation formula is as follows:

[0150]

[0151]

[0152] The calculation method of BP is as follows: matching is performed based on the N-gram matching rule, which refers to matching phrases formed by N adjacent words in two sentences, and calculating the ratio of the number of matched phrases to the total number of phrases in the translation divided by N groups of words;

[0153] The meanings of the parameters are as follows:

[0154] Bleu, represents Bleu similarity;

[0155] BP, indicates the matching degree;

[0156] Pn represents the N-gram matching accuracy.

[0157] When aligning sentences, the Chinese and English texts are divided into several small Chinese-English segments through anchor point division. Each segment contains less than 500 Chinese-English sentence pairs. When there are tens of thousands of sentences, this method improves the translation speed by dividing them into small segments. When aligning each small segment, the English is first translated through the existing translation engine to obtain a Chinese translation with good translation quality. The alignment accuracy is improved through the existing translation engine, and the translated text is aligned with the original translation and mapped to the original text to obtain the Chinese-English sentence pair result.

[0158] When aligning large documents, the content in the large document is first parsed and the text is extracted. The extracted Chinese and English texts are divided into sentences, and the text after sentence division is anchored. When dividing the anchor points, the Chinese and English texts are aligned in sequence to calculate the similarity. Here, the Chinese-English comparison range is limited to 500 sentences, and a threshold is set. When the similarity is greater than the threshold and greater than the threshold within the comparison range, the Chinese-English sentence pair is used as an anchor point to record its position. The Chinese and English texts are divided into several small segments according to the divided anchor points, and the Chinese and English segments are aligned separately.

[0159] This method uses the existing translation engine and anchor point division to perform alignment, solving the problems of slow alignment speed and low accuracy.

[0160] Figure 2 It is a flowchart of a method for finding an anchor point according to an embodiment of the present disclosure.

[0161] like Figure 2 As shown, the method for finding the anchor point includes:

[0162] a. Parameter definition and initialization, English translation sentence set en, original Chinese text sentence set zh and original English text sentence set eno, similarity threshold m, definition of anchor point step size L, maximum step size M, position offset offset, i and j;

[0163] b. Calculate the similarity between en[i] and each sentence in {eno[j]~eno[j+M]}, and take the position of the sentence in {eno[j]~eno[j+M]} corresponding to the maximum similarity sim as the position offset offset;

[0164] c. Determine whether the maximum similarity sim is greater than the similarity threshold m for comparison. If so, go to step d. Otherwise, reset i=i+1 and go to step e.

[0165] d. Confirm the anchor point position (i, j+offset), reset i=i+L, reset j=j+offset+1, go to step e; and,

[0166] e. Determine whether en[i] is the last sentence of en, that is, determine whether i is greater than length.en, where length.en represents the total number of sentences in en. If so, end; otherwise, go to step b.

[0167] Figure 3 It is a schematic diagram of N-gram matching based on Bleu similarity calculation according to an embodiment of the present disclosure.

[0168] like Figure 3 As shown in the figure, N-gram matching based on Bleu's similarity calculation. Take 3-gram with N value of 3 as an example. The idea of ​​the Bleu algorithm is to align the machine translation results with the original English. The higher the similarity value, the greater the matching possibility. Bleu uses an N-gram matching rule to compare the similarity ratio of n groups of words between the English translation and the original English.

[0169] Original Chinese text: Today is a nice day

[0170] Original English text: Today is a nice day

[0171] English translation: It is a nice day today

[0172] The machine translation can be divided into four 3-gram phrases, two of which can hit the original translation, and the 3-gram matching degree is 2 / 4.

[0173] The accuracy of each order of N-gram can be calculated according to the following formula:

[0174]

[0175] The accuracy of N-gram statistics decreases exponentially with the increase of the order, so in order to balance the effects of statistics of each order, the geometric mean is used to calculate the average and then weighted:

[0176]

[0177] Bleu, represents Bleu similarity;

[0178] BP, indicates the matching degree;

[0179] Pn represents the N-gram matching accuracy.

[0180] Figure 4 It is a schematic diagram of the structure of a bilingual alignment device according to an embodiment of the present disclosure.

[0181] like Figure 4 As shown, a bilingual alignment device 1000 includes:

[0182] An acquisition module 1002 is used to acquire an original Chinese text and an original English text, where the original Chinese text and the original English text are texts with corresponding meanings;

[0183] The Chinese sentence processing module 1004 is used to sentence the original Chinese text and obtain a sentence set of the original Chinese text;

[0184] English sentence processing module 1006, used to sentence the original English text and obtain a sentence set of the original English text;

[0185] A translation module 1008 is used to translate the sentence set of the original Chinese text to obtain an English translation sentence set of the sentence set of the original Chinese text;

[0186] A segmentation module 1010 is used to divide the sentence set of the original Chinese text and the sentence set of the original English text into a plurality of corresponding segments based on the sentence set of the English translation, the sentence set of the original Chinese text and the sentence set of the original English text; and

[0187] The alignment module 1012 is used to align corresponding segments.

[0188] The original Chinese text and the original English text are obtained, and the original Chinese text and the original English text are texts with corresponding meanings, including:

[0189] By extracting original Chinese text and original English text from the document.

[0190] The original Chinese text is divided into sentences to obtain a sentence set of the original Chinese text, including:

[0191] The original Chinese text is divided into sentences according to punctuation marks in the original Chinese text to obtain a sentence set of the original Chinese text.

[0192] The original English text is divided into sentences to obtain a sentence set of the original English text, including:

[0193] The original English text is divided into sentences according to punctuation marks in the original English text to obtain a sentence set of the original English text.

[0194] The sentence set of the original Chinese text is translated to obtain the English translation sentence set of the sentence set of the original Chinese text, including:

[0195] The sentence set of the original Chinese text is translated by a translation engine to obtain an English translation sentence set of the sentence set of the original Chinese text.

[0196] Wherein, based on the English translation sentence set, the original Chinese text sentence set and the original English text sentence set, the original Chinese text sentence set and the original English text sentence set are divided into a number of corresponding segments, including:

[0197] Find the first anchor point: calculate the similarity between the i-th sentence en[i] in the English translation sentence set and the sentences {eno[j]~eno[j+M]} in the range j to j+M in the original English text sentence set one by one. If the maximum similarity is greater than the similarity threshold m, assign the subscript value of the corresponding sentence in {eno[j]~eno[j+M]} to offset according to the maximum similarity, and record the anchor point position as i, j=j+offset, and retain the anchor point position (i,j);

[0198] Find the next anchor point: calculate the similarity between the i+Lth sentence en[i+L] in the English translation sentence set and the sentences eno[j]~eno[j+M] in the range from j to j+M in the sentence set of the original English text one by one. If the maximum similarity is not greater than the similarity threshold m, it is not found. Then calculate the similarity between the i+L+1th sentence en[i+L+1] in the English translation sentence set and the sentences {eno[j]~eno[j+M]} in the range from j to j+M in the sentence set of the original English text one by one. If the maximum similarity is greater than the similarity threshold m, take the subscript value of the corresponding sentence in {eno[j]~eno[j+M]} according to the maximum similarity and assign it to offset, and record the anchor point positions as i=i+L+1, j=j+offset, and retain the second anchor point position (i, j);

[0199] According to the above process, anchor points are searched one by one until the English translation sentence set en is traversed; and,

[0200] Based on the anchor points, the sentence set zh of the original Chinese text and the sentence set eno of the original English text are segmented;

[0201] The meanings of the various parameters are as follows:

[0202] i: a natural number, initialized to 1, indicating the first sentence;

[0203] j: a natural number, initialized to 1, indicating the first sentence;

[0204] L: Anchor point step, that is, the interval step of selecting sentences, the initial value is 100, which can be adjusted;

[0205] M: The step size when selecting the sentence range during similarity calculation. The initial value is 500 and can be adjusted in the range of 1 to the number of sentences in the sentence set.

[0206] offset: the position of the sentence in the sentence set, initialized to 0;

[0207] en: English translation sentence set;

[0208] eno: a collection of sentences of the original English text;

[0209] zh: a set of sentences of original Chinese text;

[0210] m: similarity threshold.

[0211] Among them, aligning the corresponding fragments includes:

[0212] The corresponding segments are aligned using an alignment algorithm based on the translation engine.

[0213] Among them, the corresponding segments are aligned using an alignment algorithm based on the translation engine, including:

[0214] The Bleu similarity calculation method is used to calculate the similarity between the English translation of the corresponding segment and the two sentences of the original English. The two sentences are taken from the corresponding segments respectively. The calculation formula is as follows:

[0215]

[0216]

[0217] The calculation method of BP is as follows: matching is performed based on the N-gram matching rule, which refers to matching phrases formed by N adjacent words in two sentences, and calculating the ratio of the number of matched phrases to the total number of phrases in the translation divided by N groups of words;

[0218] The meanings of the parameters are as follows:

[0219] Bleu, represents Bleu similarity;

[0220] BP, indicates the matching degree;

[0221] Pn represents the N-gram matching accuracy.

[0222] According to another aspect of the present disclosure, there is provided an electronic device, comprising:

[0223] a memory storing execution instructions; and,

[0224] A processor executes the execution instructions stored in the memory, so that the processor performs any of the above-mentioned bilingual alignment methods.

[0225] According to another aspect of the present disclosure, a readable storage medium is provided, in which execution instructions are stored. When the execution instructions are executed by a processor, they are used to implement any of the above-mentioned bilingual alignment methods.

[0226] The device for bilingual alignment may include corresponding modules for executing each or several steps in the above flowchart. Therefore, each step or several steps in the above flowchart may be executed by the corresponding module, and the device may include one or more modules in these modules. The module may be one or more hardware modules specially configured to execute the corresponding steps, or implemented by a processor configured to execute the corresponding steps, or stored in a computer-readable medium for implementation by a processor, or implemented by some combination.

[0227] The hardware structure can be implemented using a bus architecture. The bus architecture can include any number of interconnecting buses and bridges, depending on the specific application and overall design constraints of the hardware. The bus 1100 connects various circuits including one or more processors 1200, memory 1300, and / or hardware modules together. The bus 1100 can also connect various other circuits 1400 such as peripherals, voltage regulators, power management circuits, external antennas, etc.

[0228] The bus 1100 may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Component (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the figure only uses one connecting line, but does not mean that there is only one bus or one type of bus.

[0229] Any process or method description in the flowchart or otherwise described herein can be understood as a module, fragment or portion of a code representing one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present disclosure includes other implementations, in which the functions may not be performed in the order shown or discussed, including performing the functions in a substantially simultaneous manner or in a reverse order according to the functions involved, which should be understood by a person skilled in the art of the present disclosure. The processor performs the various methods and processes described above. For example, the method implementation in the present disclosure can be implemented as a software program, which is tangibly contained in a machine-readable medium, such as a memory. In some embodiments, part or all of the software program can be loaded and / or installed via a memory and / or a communication interface. When the software program is loaded into the memory and executed by the processor, one or more steps in the method described above can be performed. Alternatively, in other embodiments, the processor can be configured to perform one of the above methods in any other appropriate manner (e.g., by means of firmware).

[0230] The logic and / or steps represented in the flowchart or otherwise described herein may be embodied in any readable storage medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other system that can fetch instructions from and execute instructions on an instruction execution system, apparatus, or device), or in combination with such instruction execution systems, apparatuses, or devices.

[0231] For the purposes of this specification, a "readable storage medium" may be any device that can contain, store, communicate, propagate or transmit a program for use with or in conjunction with an instruction execution system, device or apparatus. More specific examples of readable storage media (a non-exhaustive list) include the following: an electrical connection with one or more wirings (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and editable read-only memory (EPROM or flash memory), a fiber optic device, and a portable read-only memory (CDROM). In addition, the readable storage medium may even be paper or other suitable medium on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a memory.

[0232] It should be understood that the various parts of the present disclosure can be implemented in hardware, software, or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented in software stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0233] A person skilled in the art may understand that all or part of the steps of implementing the above-mentioned implementation method may be completed by instructing related hardware through a program, and the program may be stored in a readable storage medium, which, when executed, includes one or a combination of the steps of the implementation method.

[0234] In addition, each functional unit in each embodiment of the present disclosure may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a readable storage medium. The storage medium may be a read-only memory, a disk or an optical disk, etc.

[0235] In the description of this specification, the description with reference to the terms "one embodiment / method", "some embodiments / methods", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment / method or example are included in at least one embodiment / method or example of the present application. In this specification, the schematic representations of the above terms are not necessarily the same embodiment / method or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments / methods or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments / methods or examples described in this specification and the features of the different embodiments / methods or examples, unless they are contradictory.

[0236] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0237] Those skilled in the art should understand that the above embodiments are only for the purpose of clearly illustrating the present disclosure, and are not intended to limit the scope of the present disclosure. For those skilled in the art, other changes or modifications may be made based on the above disclosure, and these changes or modifications are still within the scope of the present disclosure.

Claims

1. A bilingual alignment method, characterized in that: include: Acquire an original Chinese text and an original English text, wherein the original Chinese text and the original English text are texts with corresponding meanings; Segment the original Chinese text to obtain a sentence set of the original Chinese text; Segment the original English text to obtain a sentence set of the original English text; Translate the sentence set of the original Chinese text to obtain an English translation sentence set of the sentence set of the original Chinese text; Based on the English translation sentence set, the original Chinese text sentence set and the original English text sentence set, the sentence set of the original Chinese text and the sentence set of the original English text are divided into a plurality of corresponding segments respectively; as well as Aligning the corresponding fragments; Wherein, based on the English translation sentence set, the original Chinese text sentence set and the original English text sentence set, the original Chinese text sentence set and the original English text sentence set are respectively divided into a number of corresponding segments, including: Find the first anchor point: calculate the similarity of the i-th sentence en[i] in the English translation sentence set and the sentences {eno[j]~eno[j+M]} in the range of j to j+M in the sentence set of the original English text one by one. If the maximum similarity is greater than the similarity threshold m, assign the subscript value of the corresponding sentence in {eno[j]~eno[j+M]} to offset according to the maximum similarity, and record the anchor point position as i, j=j+offset, and retain the anchor point position (i, j); Find the next anchor point: calculate the similarity between the i+Lth sentence en[i+L] in the English translation sentence set and the sentences eno[j]~eno[j+M] in the range from j to j+M in the sentence set of the original English text one by one. If the maximum similarity is not greater than the similarity threshold m, it is not found, then calculate the similarity between the i+L+1th sentence en[i+L+1] in the English translation sentence set and the sentences {eno[j]~eno[j+M]} in the range from j to j+M in the sentence set of the original English text one by one. If the maximum similarity is greater than the similarity threshold m, then take the subscript value of the corresponding sentence in {eno[j]~eno[j+M]} according to the maximum similarity and assign it to offset, and record the anchor point positions as i=i+L+1, j=j+offset, and retain the second anchor point position (i, j); According to the above process, anchor points are searched one by one until the English translation sentence set en is traversed; and Based on the anchor points, segmenting the sentence set zh of the original Chinese text and the sentence set eno of the original English text; The meanings of each parameter are as follows: i: a natural number, initialized to 1, indicating the first sentence; j: a natural number, initialized to 1, indicating the first sentence; L: Anchor point step, that is, the interval step of selecting sentences, the initial value is 100, which can be adjusted; M: The step size when selecting the sentence range during similarity calculation. The initial value is 500 and can be adjusted in the range of 1 to the number of sentences in the sentence set. offset: the position of the sentence in the sentence set, initialized to 0; en: English translation sentence set; eno: a collection of sentences of the original English text; zh: a set of sentences of original Chinese text; m: similarity threshold.

2. The method for bilingual alignment according to claim 1, characterized in that: Obtaining an original Chinese text and an original English text, wherein the original Chinese text and the original English text are texts with corresponding meanings, including: By extracting original Chinese text and original English text from the document.

3. The method for bilingual alignment according to claim 1 or 2, characterized in that: Segmenting the original Chinese text to obtain a sentence set of the original Chinese text includes: The original Chinese text is divided into sentences according to punctuation marks in the original Chinese text to obtain a sentence set of the original Chinese text.

4. The method for bilingual alignment according to claim 1 or 2, characterized in that: The original English text is divided into sentences to obtain a sentence set of the original English text, including: The original English text is divided into sentences according to punctuation marks in the original English text to obtain a sentence set of the original English text.

5. The method for bilingual alignment according to claim 1, characterized in that: The sentence set of the original Chinese text is translated to obtain an English translation sentence set of the sentence set of the original Chinese text, including: The sentence set of the original Chinese text is translated by a translation engine to obtain an English translation sentence set of the sentence set of the original Chinese text.

6. The method for bilingual alignment according to claim 1, characterized in that: The aligning the corresponding fragments comprises: The corresponding segments are aligned using an alignment algorithm based on the translation engine.

7. A bilingual alignment device, characterized in that: include: An acquisition module, used to acquire an original Chinese text and an original English text, wherein the original Chinese text and the original English text are texts with corresponding meanings; A Chinese sentence processing module, used for sentence segmentation of the original Chinese text to obtain a sentence set of the original Chinese text; An English sentence processing module, used for sentence segmentation of the original English text to obtain a sentence set of the original English text; A translation module, used for translating the sentence set of the original Chinese text to obtain an English translation sentence set of the sentence set of the original Chinese text; A segmentation module, for dividing the sentence set of the original Chinese text and the sentence set of the original English text into a plurality of corresponding segments based on the sentence set of the English translation, the sentence set of the original Chinese text and the sentence set of the original English text; as well as An alignment module, used for aligning the corresponding fragments; Wherein, based on the English translation sentence set, the original Chinese text sentence set and the original English text sentence set, the original Chinese text sentence set and the original English text sentence set are respectively divided into a number of corresponding segments, including: Find the first anchor point: calculate the similarity of the i-th sentence en[i] in the English translation sentence set and the sentences {eno[j]~eno[j+M]} in the range of j to j+M in the sentence set of the original English text one by one. If the maximum similarity is greater than the similarity threshold m, assign the subscript value of the corresponding sentence in {eno[j]~eno[j+M]} to offset according to the maximum similarity, and record the anchor point position as i, j=j+offset, and retain the anchor point position (i, j); Find the next anchor point: calculate the similarity between the i+Lth sentence en[i+L] in the English translation sentence set and the sentences eno[j]~eno[j+M] in the range from j to j+M in the sentence set of the original English text one by one. If the maximum similarity is not greater than the similarity threshold m, it is not found, then calculate the similarity between the i+L+1th sentence en[i+L+1] in the English translation sentence set and the sentences {eno[j]~eno[j+M]} in the range from j to j+M in the sentence set of the original English text one by one. If the maximum similarity is greater than the similarity threshold m, then take the subscript value of the corresponding sentence in {eno[j]~eno[j+M]} according to the maximum similarity and assign it to offset, and record the anchor point positions as i=i+L+1, j=j+offset, and retain the second anchor point position (i, j); According to the above process, anchor points are searched one by one until the English translation sentence set en is traversed; and Based on the anchor points, segmenting the sentence set zh of the original Chinese text and the sentence set eno of the original English text; The meanings of each parameter are as follows: i: a natural number, initialized to 1, indicating the first sentence; j: a natural number, initialized to 1, indicating the first sentence; L: Anchor point step, that is, the interval step of selecting sentences, the initial value is 100, which can be adjusted; M: The step size when selecting the sentence range during similarity calculation. The initial value is 500 and can be adjusted in the range of 1 to the number of sentences in the sentence set. offset: the position of the sentence in the sentence set, initialized to 0; en: English translation sentence set; eno: a collection of sentences of the original English text; zh: a set of sentences of original Chinese text; m: similarity threshold.

8. An electronic device, characterized in that: include: A memory storing execution instructions; as well as A processor, wherein the processor executes the execution instruction stored in the memory, so that the processor executes the method according to any one of claims 1 to 6.

9. A readable storage medium, characterized in that: The readable storage medium stores execution instructions, which are used to implement the method according to any one of claims 1 to 6 when executed by a processor.

Images