Method for marking and translating repeated sentences

Abstract

The invention discloses a method for marking and translating repeated sentences. The method comprises the following steps: acquiring an original text, normalizing spaces and non-translated contents of the original text, and acquiring feature information of the original text; based on the feature information of the original text, screening out sentences with consistent feature information as duplicated sentences, and constructing a duplicated sentence judgment standard; based on the duplicate sentence judgment standard, judging to obtain a text mode of the duplicate sentences; analyzing the repetition rate of the duplicated sentences, and setting duplicated sentence marks based on the text mode of the duplicated sentences; and translating the duplicated sentences subjected to marking processing to obtain a target duplicated sentence translation. According to the method, the text mode is judged and correspondingly processed by constructing the duplicate sentence judgment standard, and meanwhile, the processing of the original text style information and the regular processing of the sentence space and the non-translated content are combined, so that the consistency and accuracy of the duplicate sentences in the marking and translating stages are ensured, and the translating time and the translating cost are greatly reduced.

Description

Technical Field

[0001] The present invention relates to the field of repeated sentence processing, and in particular to a repeated sentence marking and translation method. Background Art

[0002] With the advancement of computer-assisted translation technology, the content processed in the system is also increasing. In the documents that need to be processed, repeated sentences often appear, especially in descriptive documents. At present, in the process of processing repeated sentences, different text modes are not processed separately, and the processing of original style information and the regular processing of sentence spaces and non-translation content are not combined. The consistency and accuracy of repeated sentences in the marking and translation stages cannot be guaranteed, which greatly increases the translation time and translation cost. Summary of the invention

[0003] In order to solve the above-mentioned technical problems, the present invention provides a method for marking and translating repeated sentences, which determines the text pattern by constructing a repeated sentence determination standard and processes it accordingly. At the same time, it combines the processing of original text style information with the regular processing of sentence spaces and non-translated content to ensure the consistency and accuracy of repeated sentences in the marking and translation stages, thereby greatly reducing translation time and translation costs.

[0004] The technical solution adopted by the present invention is:

[0005] A method for marking and translating repeated sentences comprises the following specific steps:

[0006] S1. Obtain the original text, regularize the spaces and non-translated content of the original text, and obtain the characteristic information of the original text;

[0007] S2. Based on the feature information of the original text, select sentences with consistent feature information as repeated sentences, and construct a criterion for determining repeated sentences;

[0008] S3, determining the text pattern of the repeated sentence based on the repeated sentence determination standard;

[0009] S4, analyzing the repetition rate of repeated sentences, and setting repeated sentence marks based on the text pattern of the repeated sentences;

[0010] S5. Translate the marked repeated sentences to obtain target repeated sentence translations.

[0011] Furthermore, the characteristic information of the original text includes: source Sim Hash, source Urt Hash, source Repeat Hash and source Placeholder Urt Hash;

[0012] The characteristic information is defined, and the specific process is as follows:

[0013] 1) Calculate the MD5 value of the sentence text as feature information, and regard sentences with the same MD5 value, i.e., the same feature information, as repeated sentences;

[0014] 2) Determine the text type and perform corresponding processing:

[0015] ① If it is plain text, only the information after the regular spaces will be processed, as follows:

[0016] a. No regularization: no processing;

[0017] b. Regular spaces: source Sim Hash;

[0018] c. Regularization of non-translation content: no processing;

[0019] d. Regularize spaces and non-translation content: source Urt Hash;

[0020] ② If it is rich text, only the content with irregular spaces will be processed, as follows:

[0021] a. No regularization: source Repeat Hash;

[0022] b. Regular spaces: no processing;

[0023] c. Regularize non-translated content: source Placeholder Urt Hash;

[0024] d. Regularize spaces and regularize non-translation content: no processing.

[0025] Furthermore, the criteria for constructing repeated sentences include:

[0026] 1) The source Sim Hash is equal or the source Urt Hash is equal;

[0027] 2) The source Repeat Hash is equal or the source Placeholder Urt Hash is equal.

[0028] Furthermore, the step S3 specifically includes:

[0029] When the source Sim Hash or source Urt Hash of the original text are equal, the repeated sentence is determined to be a text pattern;

[0030] When the source Repeat Hash or source Placeholder Urt Hash of the original text are equal, the repeated sentence is determined to be a text and style mode.

[0031] Furthermore, the step S5 specifically includes:

[0032] 1) Before machine translation, all repeated sentences without translations are obtained based on the repeated sentence judgment criteria and form a repeated sentence set, recorded as [a1, a2…][b1, b2…][c1, c2][d1, d2…]…;

[0033] Extract the feature information of the set of repeated sentences, recorded as [a, b, c…];

[0034] The feature information of the extracted repeated sentence set is used as the query condition to query the repeated sentences with translations, which are recorded as [a0, c0…];

[0035] Use the translation of a0 to fill the set [a1, a2…] in the repeated sentence set, and use the translation of c0 to fill the set [c1, c2…] in the repeated sentence set… until all repeated sentence translations are filled;

[0036] 2) For sets [b1, b2…] and sets [d1, d2…]… for which no existing translations are found, the first record in these sets, i.e. [b1, d1…], is machine translated;

[0037] 3) After machine translation, use the translation of b1 to update the translation of the set [b1, b2…], and use the translation of d1 to update the translation of the set [d1, d2…]… until the update is completed;

[0038] 4) Send the completed and machine-translated repeated sentences to human translators;

[0039] 5) Manually group and check the repeated sentence translations to ensure that the repeated sentence translations are consistent and obtain the target repeated sentence translations.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] 1) Based on the judgment criteria of repeated sentences, the text mode is selected to minimize the translation workload and reduce the typesetting workload; among them, the plain text mode can match more repeated sentences; the text and style mode fills in the style information together when filling repeated sentences, which improves the efficiency of repeated sentence processing;

[0042] 2) The present invention can not only process the original text style information, but also process the sentence spaces and non-translated content in a regular manner, thereby ensuring the consistency and accuracy of repeated sentences in the marking and translation stages. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments:

[0044] Figure 1 It is a flow chart of a method for marking and translating repeated sentences of the present invention. DETAILED DESCRIPTION

[0045] Example

[0046] like Figure 1 As shown, a method for marking and translating repeated sentences includes the following specific steps:

[0047] S1. Obtain the original text, regularize the spaces and non-translated content of the original text, and obtain the characteristic information of the original text;

[0048] The characteristic information of the original text includes: source Sim Hash, source Urt Hash, sourceRepeat Hash and source Placeholder Urt Hash;

[0049] The characteristic information is defined, and the specific process is as follows:

[0050] 1) Calculate the MD5 value of the sentence text as feature information, and regard sentences with the same MD5 value, i.e., the same feature information, as repeated sentences;

[0051] 2) Determine the text type and perform corresponding processing:

[0052] ① If it is plain text, only the information after the regular spaces will be processed, as follows:

[0053] a. No regularization: no processing;

[0054] b. Regular spaces: source Sim Hash;

[0055] c. Regularization of non-translation content: no processing;

[0056] d. Regularize spaces and non-translation content: source Urt Hash;

[0057] ② If it is rich text, only the content with irregular spaces will be processed, as follows:

[0058] a. No regularization: source Repeat Hash;

[0059] b. Regular spaces: no processing;

[0060] c. Regularize non-translated content: source Placeholder Urt Hash;

[0061] d. Regularize spaces and non-translation content: no processing;

[0062] S2. Based on the feature information of the original text, select sentences with consistent feature information as repeated sentences, and construct a criterion for determining repeated sentences;

[0063] Specifically, the criteria for constructing repeated sentences include:

[0064] 1) The source Sim Hash is equal or the source Urt Hash is equal;

[0065] 2) The source Repeat Hash is equal or the source Placeholder Urt Hash is equal;

[0066] S3, determining the text pattern of the repeated sentence based on the repeated sentence determination standard;

[0067] Specifically, when the source Sim Hash of the original feature information is equal or the source Urt Hash is equal, the repeated sentence is determined to be a text pattern;

[0068] When the source Repeat Hash or source Placeholder Urt Hash of the original text are equal, the repeated sentence is determined to be a text and style mode;

[0069] Among them, in the plain text mode, only text information is consistently regarded as repeated sentences, and when filling in the translation of repeated sentences, only the text is filled in, and the precise style information of the repeated sentences is not processed;

[0070] The text and style mode means that the text information and style information must be completely consistent to be considered as a repeated sentence. The repeated sentence fills the text while retaining the precise style information of the repeated sentence.

[0071] S4, analyzing the repetition rate of repeated sentences, and setting repeated sentence marks based on the text pattern of the repeated sentences;

[0072] S5. Translate the marked repeated sentences to obtain target repeated sentence translations.

[0073] Specifically, the step S5 includes:

[0074] 1) Before machine translation, all repeated sentences without translations are obtained based on the repeated sentence judgment criteria and form a repeated sentence set, recorded as [a1, a2…][b1, b2…][c1, c2][d1, d2…]…;

[0075] Extract the feature information of the set of repeated sentences, recorded as [a, b, c…];

[0076] The feature information of the extracted repeated sentence set is used as the query condition to query the repeated sentences with translations, which are recorded as [a0, c0…];

[0077] Use the translation of a0 to fill the set [a1, a2…] in the repeated sentence set, and use the translation of c0 to fill the set [c1, c2…] in the repeated sentence set… until all repeated sentence translations are filled;

[0078] 2) For sets [b1, b2…] and sets [d1, d2…]… for which no existing translations are found, the first record in these sets, i.e. [b1, d1…], is machine translated;

[0079] 3) After machine translation, use the translation of b1 to update the translation of the set [b1, b2…], and use the translation of d1 to update the translation of the set [d1, d2…]… until the update is completed;

[0080] 4) Send the completed and machine-translated repeated sentences to human translators;

[0081] 5) Manually group and check the translations of repeated sentences to ensure that the translations of repeated sentences are consistent and obtain the target translations of repeated sentences;

[0082] In addition, when a translator saves a translation, if the sentence has repeated sentences, the system will simultaneously fill in the repeated sentences according to the repeated sentence judgment criteria to obtain the repeated sentence translation;

[0083] Among them, after processing a part of the content that has been machine translated or manually translated in step 1), when the newly uploaded file is processed for repeated sentences, the newly added repeated sentence translations can be filled with the existing translations to improve the translation efficiency;

[0084] Step 2) is to reduce the number of sentences sent to machine translation. A group of repeated sentences only needs one machine translation, and the remaining sentences are filled with the machine translated sentences.

[0085] Step 5) is to further shorten the translator's review and checking time and improve work efficiency.

[0086] The embodiments described above are merely descriptions of preferred implementation modes of the present invention and are not intended to limit the scope of the present invention. Without departing from the principles and essence of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the scope of protection determined by the claims of the present invention.

Claims

1. A method for marking and translating repeated sentences, characterized in that: The specific steps include: S1. Obtain the original text, regularize the spaces and non-translated content of the original text, and obtain the characteristic information of the original text; S2. Based on the feature information of the original text, select sentences with consistent feature information as repeated sentences, and construct a criterion for determining repeated sentences; S3, determining the text pattern of the repeated sentence based on the repeated sentence determination standard; S4, analyzing the repetition rate of repeated sentences, analyzing the repetition rate of repeated sentences, and setting repeated sentence marks based on the text mode of the repeated sentences; S5. Translate the marked repeated sentences to obtain target repeated sentence translations.

2. The method for marking and translating repeated sentences according to claim 1, characterized in that: The characteristic information of the original text includes: source Sim Hash, source Urt Hash, source Repeat Hash and sourcePlaceholder Urt Hash; The characteristic information is defined, and the specific process is as follows: 1) Calculate the MD5 value of the sentence text as feature information, and regard sentences with the same MD5 value, i.e., the same feature information, as repeated sentences; 2) Determine the text type and perform corresponding processing: ① If it is plain text, only the information after the regular spaces will be processed, as follows: a. No regularization: no processing; b. Regular spaces: source Sim Hash; c. Regularization of non-translation content: no processing; d. Regularize spaces and non-translation content: source Urt Hash; ② If it is rich text, only the content with irregular spaces will be processed, as follows: a. No regularization: source Repeat Hash; b. Regular spaces: no processing; c. Regularize non-translated content: source Placeholder Urt Hash; d. Regularize spaces and regularize non-translation content: no processing.

3. The method for marking and translating repeated sentences according to claim 1, characterized in that: The criteria for constructing repeated sentences include: 1) The source Sim Hash is equal or the source Urt Hash is equal; 2) The source Repeat Hash is equal or the source Placeholder Urt Hash is equal.

4. The method for marking and translating repeated sentences according to claim 1, characterized in that: The step S3 specifically includes: When the source Sim Hash or source Urt Hash of the original text are equal, the repeated sentence is determined to be a text pattern; When the source Repeat Hash or source Placeholder Urt Hash of the original text are equal, the repeated sentence is determined to be a text and style mode.

5. The method for marking and translating repeated sentences according to claim 1, characterized in that: The step S5 specifically includes: 1) Before machine translation, all repeated sentences without translations are obtained based on the repeated sentence judgment criteria and form a repeated sentence set, recorded as [a1, a2…][b1, b2…][c1, c2][d1, d2…]…; Extract the feature information of the set of repeated sentences, recorded as [a, b, c…]; The feature information of the extracted repeated sentence set is used as the query condition to query the repeated sentences with translations, which are recorded as [a0, c0…]; Use the translation of a0 to fill the set [a1, a2…] in the repeated sentence set, and use the translation of c0 to fill the set [c1, c2…] in the repeated sentence set… until all repeated sentence translations are filled; 2) For sets [b1, b2…] and sets [d1, d2…]… for which no existing translations are found, the first record in these sets, i.e. [b1, d1…], is machine translated; 3) After machine translation, use the translation of b1 to update the translation of the set [b1, b2…], and use the translation of d1 to update the translation of the set [d1, d2…]… until the update is completed; 4) Send the completed and machine-translated repeated sentences to human translators; 5) Manually group and check the repeated sentence translations to ensure that the repeated sentence translations are consistent and obtain the target repeated sentence translations.

Images