Translation detection method and device
通过获取和分析翻译文本的局部和全局信息,解决了现有技术中检测基于NMT模型翻译文本是否为机翻译文的准确率低的问题,实现了更高的检测准确率和更低的人工成本 Essence
Patent Information
- Application Number
- CN201910477659.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2019-06-03
- Publication Date
- 2025-01-28
- Estimated Expiration
- 2039-06-03
AI Technical Summary
The prior art is difficult to accurately detect whether the translated text translated by the NMT-based text translation model is a machine-translated text, resulting in a low detection accuracy.
By obtaining the first local information of the translation text and the first global information, including the semantic information and grammar information of the sub -text, and the semantic logic correlation between the translation text and the original text to determine whether the translation text is a machine translation.
It has improved whether the translation text obtained by NMT -based translation model translation is the accuracy of the detection accuracy of the machine translation, which reduces labor costs.
Smart Images

Figure CN112036193B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer technology, and in particular to a translation detection method and device. Background Art
[0002] At present, text translation plays an increasingly important role in all walks of life. Initially, text translation was based on manual translation, but the labor cost of manual translation is high. Therefore, in order to reduce the labor cost, translation based on text translation models is gradually used. The text translation model can be trained in advance so that users can translate text with the help of the text translation model. The quality of the text translation model used initially is limited, that is, the accuracy of the text translated using the text translation model is limited.
[0003] In order to improve the accuracy of translation, it is necessary to train a high-quality text translation model. When training a high-quality text translation model, bilingual corpus is often needed. The bilingual corpus includes the original text in one language and the translated text of the original text in another language. For example, the bilingual corpus includes English text and the translated text of the English text in Chinese. The English text includes "It's a nice day today", and the translated text in Chinese includes "The weather is very good today".
[0004] Among them, a large amount of bilingual corpora can be collected on the Internet. However, the translation texts of some bilingual corpora are manually translated, and the translation texts of some bilingual corpora are translated using existing low-quality text translation models. Since the accuracy of the translated texts translated using the existing low-quality text translation models is low, when the bilingual prediction training text translation model containing the translation texts with low accuracy is used, the quality of the trained text translation model will still be low.
[0005] Therefore, it is necessary to remove some bilingual corpora from the collected bilingual predictions. The translated texts included in these bilingual corpora are translated by the text translation model, and then use the remaining bilingual corpora to train the text translation model.
[0006] Among them, the low-quality text translation model mentioned above is usually a model based on SMT (Statistical Machine Translation). Therefore, the features of the translated text obtained by translating based on the SMT model can be used to detect whether the translated text is a machine-translated text. Since the quality of the translated text obtained by translating based on the SMT model is relatively low, when detecting whether the translated text is a machine-translated text, language experts usually manually select the features of the translated text. The features include the length of the characters included in the translated text, the number of functional words such as conjunctions and function words in the translated text, and the position of the functional words in the translated text. Then, whether the translated text is a machine-translated text is detected based on whether these features conform to normal grammatical habits.
[0007] However, with the rapid development of technology, text translation models based on NMT (Neural Machine Translation) have gradually come into people's view. The quality of text translation models based on NMT is often higher, that is, the accuracy of the translated texts translated by the text translation model based on NMT is higher, and the above-mentioned features of the translated texts translated by the text translation model based on NMT usually conform to normal grammatical habits. Therefore, the translated texts translated by the text translation model based on NMT are often determined as not being machine-translated texts in the above-mentioned manner. However, the translated texts translated by the text translation model based on NMT may also have problems such as wrong translations and missing translations.
[0008] Therefore, the detection accuracy of the existing method when detecting whether the translated text obtained by the NMT-based text translation model is a machine-translated text is low. Summary of the invention
[0009] In order to solve the above technical problems, the embodiments of the present application show a translation detection method and device.
[0010] In a first aspect, an embodiment of the present application shows a translation detection method, the method comprising:
[0011] Get the translated text;
[0012] Acquire first local information and first global information of the translation text;
[0013] It is determined whether the translated text is a machine translated text at least according to the first local information and the first global information.
[0014] In an optional implementation, the first local information includes first semantic information of each subtext in the translation text and first grammatical information of each subtext;
[0015] The first global information includes second semantic information of the translated text and first semantic logical correlations between the sub-texts.
[0016] In an optional implementation, determining whether the translated text is a machine translated text at least according to the first local information and the first global information includes:
[0017] Acquire a first semantic fluency of the translated text according to the first grammatical information and / or the first semantic logical relevance;
[0018] Acquire a first semantic similarity between the first semantic information and the second semantic information;
[0019] Determine whether the translated text is a machine translated text according to the first semantic fluency and the first semantic similarity.
[0020] In an optional implementation, the acquiring the first semantic fluency of the translated text according to the first grammatical information and / or the first semantic logic relevance includes:
[0021] The first grammatical information and / or the first semantic logic relevance is input into a semantic fluency acquisition model to obtain a first semantic fluency of the translated text output by the semantic fluency acquisition model.
[0022] In an optional implementation, the method further includes:
[0023] Acquire multiple sample information sets, each sample information set includes sample grammatical information of each sample sub-text in the sample text and / or sample semantic logical relevance between each sample sub-text in the sample text, and the sample semantic fluency of the sample text is annotated in the sample grammatical information or the sample semantic logical relevance;
[0024] The model is trained using sample grammatical information of each sample sub-text in the sample text in each sample information set and / or sample semantic logical relevance between each sample sub-text in the sample text until all weights in the model converge to obtain a semantic fluency acquisition model.
[0025] In an optional implementation, the obtaining the first semantic similarity between the first semantic information and the second semantic information includes:
[0026] A first semantic similarity between the first semantic information and the second semantic information is acquired according to a semantic similarity acquisition model.
[0027] In an optional implementation, the method further includes:
[0028] Acquire multiple sample information sets, each of which includes first sample semantic information of each sample subtext in the sample text and second sample semantic information of the sample text as a whole, and the first sample semantic information or the second sample semantic information is marked with a sample semantic similarity between the first sample semantic information and the second sample semantic information;
[0029] The model is trained using the first sample semantic information and the second sample semantic information in each sample information set until all weights in the model converge, thereby obtaining a semantic similarity acquisition model.
[0030] In an optional implementation, determining whether the translated text is a machine translated text according to the first semantic fluency and the first semantic similarity includes:
[0031] The first semantic fluency and the first semantic similarity are input into a classification model to obtain a result output by the classification model to indicate whether the translated text is a machine translated text.
[0032] In an optional implementation, the method further includes:
[0033] Acquire multiple sample information sets, each of which includes sample semantic fluency and sample semantic similarity of a sample text, wherein the sample semantic fluency is acquired based on sample grammatical information and / or sample semantic logical relevance of the sample text, and the sample semantic similarity is acquired based on first sample semantic information of each sample subtext in the sample text and second sample semantic information of the sample text as a whole, and the sample semantic fluency or the sample semantic similarity is annotated with a marking result indicating whether the sample text is a machine translated text;
[0034] The model is trained using the sample semantic fluency and sample semantic similarity in each sample information set until the weights in the model converge to obtain a classification model.
[0035] In an optional implementation, determining whether the translated text is a machine translated text at least according to the first local information and the first global information includes:
[0036] Obtaining the original text corresponding to the translated text;
[0037] Acquire the second local information and the second global information of the original text;
[0038] Determine whether the translation text is a machine translation text according to the first local information, the first global information, the second local information and the second global information.
[0039] In an optional implementation, the second local information includes third semantic information of each subtext in the original text and second grammatical information of each subtext in the original text;
[0040] The second global information includes fourth semantic information of the original text and second semantic logical correlations between subtexts in the original text.
[0041] In an optional implementation, determining whether the translated text is a machine translated text according to the first local information, the first global information, the second local information, and the second global information includes:
[0042] Acquire a first semantic fluency of the translated text according to the first grammatical information and / or the first semantic logical relevance;
[0043] Acquire a first semantic similarity between the first semantic information and the second semantic information;
[0044] Acquire a second semantic fluency of the original text according to the second grammatical information and / or the second semantic logical relevance;
[0045] Acquire a second semantic similarity between the third semantic information and the fourth semantic information;
[0046] Determine whether the translated text is a machine translated text according to the first semantic fluency, the first semantic similarity, the second semantic fluency, and the second semantic similarity.
[0047] In a second aspect, an embodiment of the present application shows a translation detection device, the device comprising:
[0048] A first acquisition module, used to acquire a translation text;
[0049] A second acquisition module, used to acquire first local information and first global information of the translation text;
[0050] A determination module is used to determine whether the translation text is a machine translation text based at least on the first local information and the first global information.
[0051] In an optional implementation, the first local information includes first semantic information of each subtext in the translation text and first grammatical information of each subtext;
[0052] The first global information includes second semantic information of the translated text and first semantic logical correlations between the sub-texts.
[0053] In an optional implementation, the determining module includes:
[0054] A first acquiring unit, configured to acquire a first semantic fluency of the translated text according to the first grammatical information and / or the first semantic logical relevance;
[0055] A second acquiring unit, configured to acquire a first semantic similarity between the first semantic information and the second semantic information;
[0056] The first determining unit is used to determine whether the translated text is a machine translated text according to the first semantic fluency and the first semantic similarity.
[0057] In an optional implementation, the first acquiring unit includes:
[0058] The first input subunit is used to input the first grammatical information and / or the first semantic logic relevance into a semantic fluency acquisition model to obtain a first semantic fluency of the translated text output by the semantic fluency acquisition model.
[0059] In an optional implementation, the first acquiring unit further includes:
[0060] A first acquisition subunit is used to acquire a plurality of sample information sets, each of which includes sample grammatical information of each sample subtext in the sample text and / or sample semantic logical relevance between each sample subtext in the sample text, and the sample semantic fluency of the sample text is annotated in the sample grammatical information or the sample semantic logical relevance;
[0061] The first training sub-unit is used to train the model using the sample grammatical information of each sample sub-text in the sample text in each sample information set and / or the sample semantic logical relevance between each sample sub-text in the sample text until the weights in the model converge to obtain a semantic fluency acquisition model.
[0062] In an optional implementation, the second acquiring unit includes:
[0063] The second acquisition subunit is used to acquire the first semantic similarity between the first semantic information and the second semantic information according to a semantic similarity acquisition model.
[0064] In an optional implementation, the second acquiring unit further includes:
[0065] A third acquisition subunit is used to acquire multiple sample information sets, each of which includes first sample semantic information of each sample subtext in the sample text and second sample semantic information of the sample text as a whole, and the first sample semantic information or the second sample semantic information is marked with a sample semantic similarity between the first sample semantic information and the second sample semantic information;
[0066] The second training subunit is used to train the model using the first sample semantic information and the second sample semantic information in each sample information set until the weights in the model converge to obtain a semantic similarity acquisition model.
[0067] In an optional implementation, the first determining unit includes:
[0068] The second input subunit is used to input the first semantic fluency and the first semantic similarity into a classification model to obtain a result output by the classification model to indicate whether the translated text is a machine translated text.
[0069] In an optional implementation, the first determining unit further includes:
[0070] A fourth acquisition subunit is used to acquire multiple sample information sets, each of which includes a sample semantic fluency and a sample semantic similarity of a sample text, wherein the sample semantic fluency is acquired based on the sample grammatical information and / or the sample semantic logical relevance of the sample text, and the sample semantic similarity is acquired based on the first sample semantic information of each sample subtext in the sample text and the second sample semantic information of the sample text as a whole, and the sample semantic fluency or the sample semantic similarity is annotated with a marking result indicating whether the sample text is a machine translated text;
[0071] The third training subunit is used to train the model using the sample semantic fluency and sample semantic similarity in each sample information set until all weights in the model converge to obtain a classification model.
[0072] In an optional implementation, the determining module includes:
[0073] A third acquisition unit, used to acquire an original text corresponding to the translated text;
[0074] A fourth acquisition unit, used for acquiring second local information and second global information of the original text;
[0075] The second determining unit is used to determine whether the translation text is a machine translation text according to the first local information, the first global information, the second local information and the second global information.
[0076] In an optional implementation, the second local information includes third semantic information of each subtext in the original text and second grammatical information of each subtext in the original text;
[0077] The second global information includes fourth semantic information of the original text and second semantic logical correlations between subtexts in the original text.
[0078] In an optional implementation, the second determining unit includes:
[0079] a fifth acquiring subunit, configured to acquire a first semantic fluency of the translated text according to the first grammatical information and / or the first semantic logical relevance;
[0080] a sixth acquisition subunit, configured to acquire a first semantic similarity between the first semantic information and the second semantic information;
[0081] a seventh acquisition subunit, configured to acquire a second semantic fluency of the original text according to the second grammatical information and / or the second semantic logical relevance;
[0082] an eighth acquiring subunit, configured to acquire a second semantic similarity between the third semantic information and the fourth semantic information;
[0083] A determination subunit is used to determine whether the translation text is a machine translation text according to the first semantic fluency, the first semantic similarity, the second semantic fluency, and the second semantic similarity.
[0084] In a third aspect, an embodiment of the present application shows an electronic device, the electronic device comprising:
[0085] Processor; and
[0086] A memory having executable codes stored thereon, which, when executed, causes the processor to execute the translation detection method as described in the first aspect.
[0087] In a fourth aspect, an embodiment of the present application illustrates one or more machine-readable media on which executable codes are stored. When the executable codes are executed, the processor executes the translation detection method as described in the first aspect.
[0088] Compared with the prior art, the embodiments of the present application have the following advantages:
[0089] In the present application, a translation text is obtained; first local information of the translation text is obtained and first global information of the translation text is obtained; and whether the translation text is a machine translation text is determined at least based on the first local information and the first global information. If the translation text is translated based on a text translation model based on NMT, the present application can accurately detect whether the translation text is a machine translation text through the first local information of the translation text and the first global information of the translation text. Therefore, compared with the prior art, the present application can improve the detection accuracy when detecting whether the translation text translated based on the text translation model of NMT is a machine translation text.
[0090] Secondly, in the prior art, the participation of language experts is required when selecting the features of the translated text, resulting in high labor costs. However, in the present application, the participation of language experts is not required when obtaining the first local information of the translated text and the first global information of the translated text, thereby saving labor costs. BRIEF DESCRIPTION OF THE DRAWINGS
[0091] Figure 1 The figure is a flowchart of a translation detection method according to an exemplary embodiment.
[0092] Figure 2 The figure is a flowchart of a translation detection method according to an exemplary embodiment.
[0093] Figure 3 The figure is a flowchart of a translation detection method according to an exemplary embodiment.
[0094] Figure 4 It is a block diagram of a translation detection device according to an exemplary embodiment.
[0095] Figure 5 It is a block diagram of a translation detection device according to an exemplary embodiment. DETAILED DESCRIPTION
[0096] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.
[0097] Figure 1 is a flowchart of a translation detection method according to an exemplary embodiment. Figure 1 As shown, the method is used in an electronic device, the electronic device includes a terminal or a server, etc., and the method includes the following steps.
[0098] In step S101, a translation text is obtained;
[0099] In this application, it is necessary to detect whether the translation text in each bilingual forecast collected on the Internet is a machine translation. If the translation text in some bilingual corpora is a machine translation, the bilingual forecast is discarded. In this way, the translation texts in the remaining bilingual forecasts contain as few machine translations as possible. Using the remaining bilingual corpora to train the translation model can improve the quality of the trained translation model.
[0100] Each bilingual corpus contains an original text and a translated text, and the method of this application can be used to detect whether the translated text in each bilingual corpus is a machine-translated text. This application only uses the detection of a translated text in a bilingual corpus as an example, but it does not limit the scope of protection of this application.
[0101] In step S102, first local information and first global information of the translation text are obtained;
[0102] In this application, a translation text may include multiple subtexts. For example, if the translation text is a paragraph, and the paragraph consists of multiple sentences, each sentence may be a subtext of the paragraph. Or, if the translation text is a sentence, and the sentence consists of multiple words, each word may be a subtext of the sentence.
[0103] The first local information includes first semantic information of each sub-text in the translation text and first grammatical information of each sub-text.
[0104] The first semantic information of each sub-text is used to describe the meaning or theme expressed by the sub-text, and the first grammatical information is used to describe the degree to which the sub-text conforms to normal grammatical habits.
[0105] The first global information includes second semantic information of the translated text and first semantic logical correlations between the sub-texts.
[0106] The second semantic information is used to describe the meaning or theme expressed by the translated text as a whole.
[0107] The first semantic logical relevance is used to describe the degree of correlation between the contents or semantics of each sub-text. For example, if one sub-text describes content about eating and one sub-text describes content about driving, the semantic logical relevance between the two sub-texts is low. If one sub-text describes content about eating and one sub-text describes content about cooking, the semantic logical relevance between the two sub-texts is high.
[0108] In one embodiment of the present application, a local information acquisition model can be trained in advance. The specific training method includes: obtaining a sample text set, the sample text set includes at least one sample text annotated with sample local information, and then using the sample text in the sample text set to train the model until the weights in the model converge to obtain a local information acquisition model.
[0109] Thus, in this step, the translation text may be input into the local information acquisition model to obtain the first local information of the translation text output by the local information acquisition model.
[0110] Among them, the model includes a model based on CNN (Convolutional Neural Networks).
[0111] In another embodiment of the present application, the global information acquisition model can be trained in advance. The specific training method includes: obtaining a sample text set, the sample text set includes at least one sample text annotated with sample global information, and then using the sample text in the sample text set to train the model until the weights in the model converge to obtain the global information acquisition model.
[0112] Thus, in this step, the translated text may be input into the global information acquisition model to obtain the first global information of the translated text output by the global information acquisition model.
[0113] The model includes RNN (Recurrent Neural Network)-based models, such as GRU (Gated Recurrent Unit), bidirectional Gated Recurrent Unit (bidirectional Gated Recurrent Unit), and LSTM (Long Short-Term Memory).
[0114] In this application, whenever a text needs to be applied to a model, the text can be segmented, and then the word vector of each segmented word can be obtained based on "word to vector". Then, each word vector is organized into a word vector sequence according to the position order of its corresponding segmented word in the text, and then the word vector sequence is input into the model.
[0115] For example, this step is taken as an example for illustration, and similar situations in subsequent embodiments can refer to the operation of this step, and no further detailed description is given.
[0116] In this step, the translation text can be segmented, and then the word vector of each segmented word can be obtained based on "word to vector". Then, each word vector is composed into a word vector sequence according to the position order of its corresponding segmented word in the translation text. Then, the word vector sequence is input into the local information acquisition model and the global information acquisition model respectively to obtain the first local information of the translation text output by the local information acquisition model and the first global information of the translation text output by the global information acquisition model.
[0117] In step S103, it is determined whether the translated text is a machine translated text at least according to the first local information and the first global information.
[0118] For details on this step, see Figure 2 or Figure 3 The illustrated embodiment will not be described in detail here.
[0119] In the present application, a translation text is obtained; first local information of the translation text is obtained and first global information of the translation text is obtained; and whether the translation text is a machine translation text is determined at least based on the first local information and the first global information. If the translation text is translated based on a text translation model based on NMT, the present application can accurately detect whether the translation text is a machine translation text through the first local information of the translation text and the first global information of the translation text. Therefore, compared with the prior art, the present application can improve the detection accuracy when detecting whether the translation text translated based on the text translation model of NMT is a machine translation text.
[0120] Secondly, in the prior art, the participation of language experts is required when selecting the features of the translated text, resulting in high labor costs. However, in the present application, the participation of language experts is not required when obtaining the first local information of the translated text and the first global information of the translated text, thereby saving labor costs.
[0121] In one embodiment of the present application, see Figure 2 , step S103 includes:
[0122] In step S201, a first semantic fluency of a translation text is obtained according to the first grammatical information and / or the first semantic logical relevance;
[0123] In one embodiment of the present application, a semantic fluency acquisition model can be trained in advance, and the specific training method includes: obtaining multiple sample information sets, each sample information set including sample grammatical information of each sample sub-text in the sample text and / or sample semantic logical relevance between each sample sub-text in the sample text, and the sample grammatical information or the sample semantic logical relevance is marked with the sample semantic fluency of the sample text, and then using the sample grammatical information of each sample sub-text in the sample text in each sample information set and / or the sample semantic logical relevance between each sample sub-text in the sample text to train the model until the weights in the model converge, thereby obtaining the semantic fluency acquisition model.
[0124] Thus, in this step, the first grammatical information and / or the first semantic logical relevance may be input into the semantic fluency acquisition model to obtain the first semantic fluency of the translated text output by the semantic fluency acquisition model.
[0125] Among them, when training the semantic fluency acquisition model, only the sample grammatical information of each sample sub-text in the sample text can be used, only the sample semantic logical correlation between each sample sub-text in the sample text can be used, or both the sample grammatical information of each sample sub-text in the sample text and the sample semantic logical correlation between each sample sub-text in the sample text can be used.
[0126] In this way, when obtaining the first semantic fluency of the translated text, only the first grammatical information can be used, only the first semantic logical relevance can be used, or both the first grammatical information and the first semantic logical relevance can be used. Which of the three methods is used depends on which method is used when training the semantic fluency acquisition model.
[0127] The accuracy of the first semantic fluency obtained by using both the first grammatical information and the first semantic logical relevance is greater than the accuracy of the first semantic fluency obtained by using only the first grammatical information or only the first semantic logical relevance.
[0128] Of course, the first semantic fluency of the translated text may also be acquired according to the first grammatical information and / or the first semantic logical relevance by other methods, which is not limited in the present application.
[0129] In step S202, a first semantic similarity between the first semantic information and the second semantic information is obtained;
[0130] In one embodiment of the present application, a semantic similarity acquisition model can be trained in advance, and the specific training method includes: obtaining multiple sample information sets, each sample information set includes first sample semantic information of each sample sub-text in the sample text and second sample semantic information of the sample text as a whole, and the first sample semantic information or the second sample semantic information is marked with the sample semantic similarity between the first sample semantic information and the second sample semantic information, and then the model is trained using the first sample semantic information and the second sample semantic information in each sample information set until the weights in the model converge, thereby obtaining a semantic similarity acquisition model.
[0131] Thus, in this step, the first semantic similarity between the first semantic information and the second semantic information can be obtained according to the semantic similarity acquisition model. For example, the first semantic information and the second semantic information are input into the semantic similarity acquisition model to obtain the first semantic similarity output by the semantic similarity acquisition model.
[0132] Of course, the first semantic similarity between the first semantic information and the second semantic information may also be acquired by other methods, which is not limited in the present application.
[0133] In step S203, it is determined whether the translated text is a machine translated text according to the first semantic fluency and the first semantic similarity.
[0134] In one embodiment of the present application, a classification model can be trained in advance, and the specific training method includes: obtaining multiple sample information sets, each sample information set including sample semantic fluency and sample semantic similarity of a sample text, the sample semantic fluency is obtained based on sample grammatical information and / or sample semantic logical relevance of the sample text, the sample semantic similarity is obtained based on first sample semantic information of each sample sub-text in the sample text and second sample semantic information of the sample text as a whole, and the sample semantic fluency or sample semantic similarity is marked with an annotation result of whether the sample text is a machine translated text, and then the model is trained using the sample semantic fluency and sample semantic similarity in each sample information set until the weights in the model converge to obtain a classification model.
[0135] Thus, in this step, a classification model can be used to determine whether the translated text is a machine translated text based on the first semantic fluency and the first semantic similarity. For example, the first semantic fluency and the first semantic similarity are input into the classification model to obtain a result output by the classification model to indicate whether the translated text is a machine translated text.
[0136] Of course, other methods may also be used to determine whether the translated text is a machine translated text based on the first semantic fluency and the first semantic similarity, and this application does not limit this. For example, the first semantic fluency and the first semantic similarity are weighted and summed to obtain a score. If the score is greater than or equal to a preset threshold, it is determined that the translated text is not a machine translated text. If the score is less than the preset threshold, it is determined that the translated text is a machine translated text.
[0137] In this application, generally, when a translated text is a manually translated text, the higher the degree to which the translated text conforms to normal grammatical habits and the higher the semantic logical correlation between the sub-texts in the translated text, the higher the semantic fluency of the translated text. When a translated text is a machine translated text, the lower the degree to which the translated text conforms to normal grammatical habits and / or the lower the semantic logical correlation between the sub-texts in the translated text, the lower the semantic fluency of the translated text.
[0138] That is, the higher the semantic fluency of the translated text, the lower the possibility that the translated text is a machine-translated text, and the lower the semantic fluency of the translated text, the higher the possibility that the translated text is a machine-translated text.
[0139] In the present application, for any original text in the collected bilingual corpus, the original text is used to describe a semantics, and each sub-text included in the original text is used to describe the semantics. If the translation text corresponding to the original text is obtained through manual translation, the translation text is also used to describe the semantics, and each sub-text included in the translation text is also used to describe the semantics.
[0140] However, if the translation text corresponding to the original text is obtained by translation through the translation model, the generalization degree and intelligence of the translation model are limited. Therefore, in the translation process, there is a situation where at least one sub-text included in the original text is translated incorrectly, so that the semantics of the incorrectly translated sub-text is different from the semantics of other sub-texts. The semantic information of each sub-text in the local information includes at least two different semantic information, while the overall semantic information of the translated text in the global information is still the original semantic information. Therefore, there is a difference between the two semantic information.
[0141] In this way, if the similarity between the first semantic information of each sub-text in the translation text and the second semantic information of the translation text is higher, the possibility that the translation text is a machine translation is lower; if the similarity between the first semantic information of each sub-text in the translation text and the second semantic information of the translation text is lower, the possibility that the translation text is a machine translation is higher.
[0142] In step S103 of the aforementioned embodiment, whether the translated text is a machine-translated text is detected only based on the translated text. However, sometimes the translated text obtained by machine translation may have a high first semantic fluency and a high first semantic similarity in individual cases, but compared with the original text corresponding to the translated text, the translated text has over-translations or omissions. In this case, the translated text is redundant or missing relative to the original text, which still affects the quality of the trained translation model.
[0143] Therefore, it is necessary to further improve the detection accuracy when detecting whether the translated text is a machine translated text, so as to improve the quality of the trained translation model. In another embodiment of the present application, the original text corresponding to the translated text can also be combined with the translated text to detect whether the translated text is a machine translated text. Figure 3 , step S103 includes:
[0144] In step S301, the original text corresponding to the translated text is obtained;
[0145] In the present application, the bilingual corpus in which the translation text is located can be determined among multiple bilingual corpora, and then the original text included in the determined bilingual corpus is obtained, that is, the original text corresponding to the translation text.
[0146] In step S302, second local information and second global information of the original text are obtained;
[0147] In the present application, the original text may include multiple subtexts. For example, if the original text is a paragraph, and the paragraph consists of multiple sentences, each sentence may be a subtext of the paragraph. Or, if the original text is a sentence, and the sentence consists of multiple words, each word may be a subtext of the sentence.
[0148] The second local information includes the third semantic information of each sub-text in the original text and the second grammatical information of each sub-text.
[0149] The third semantic information of each sub-text is used to describe the meaning or theme expressed by the sub-text, and the second grammatical information of each sub-text is used to describe the degree to which the sub-text conforms to normal grammatical habits.
[0150] The second global information includes fourth semantic information of the original text and second semantic logical correlations between the sub-texts.
[0151] The fourth semantic information is used to describe the meaning or theme expressed by the original text as a whole.
[0152] The second semantic logical relevance is used to describe the degree of correlation between the content or semantics of each sub-text. For example, if one sub-text describes content about eating and one sub-text describes content about driving, the semantic logical relevance between the two sub-texts is low. If one sub-text describes content about eating and one sub-text describes content about cooking, the semantic logical relevance between the two sub-texts is high.
[0153] In one embodiment of the present application, the original text may be input into a local information acquisition model to obtain the second local information of the original text output by the local information acquisition model.
[0154] In another embodiment of the present application, the original text may be input into the global information acquisition model to obtain the second global information of the original text output by the global information acquisition model.
[0155] In this application, whenever a text needs to be applied to a model, the text can be segmented, and then the word vector of each segmented word can be obtained based on "word to vector". Then, each word vector is organized into a word vector sequence according to the position order of its corresponding segmented word in the text, and then the word vector sequence is input into the model.
[0156] For example, this step is taken as an example for illustration, and similar situations in subsequent embodiments can refer to the operation of this step, and no further description is given. In this step, the original text can be segmented, and then the word vector of each segmented word is obtained based on "word to vector", and then each word vector is composed of a word vector sequence according to the position order of its corresponding segmented word in the original text, and then the word vector sequence is input into the local information acquisition model and the global information acquisition model respectively, to obtain the second local information of the original text output by the local information acquisition model and the second global information of the original text output by the global information acquisition model.
[0157] In step S303, it is determined whether the translated text is a machine translated text according to the first local information, the first global information, the second local information and the second global information.
[0158] This step can be achieved through the following process, including:
[0159] 3031. Obtain a first semantic fluency of the translated text according to the first grammatical information and / or the first semantic logical relevance;
[0160] For details of this step, please refer to step S201, which will not be described in detail here.
[0161] 3032. Obtain a first semantic similarity between the first semantic information and the second semantic information;
[0162] For details of this step, please refer to step S202, which will not be described in detail here.
[0163] 3033. Obtain a second semantic fluency of the original text according to the second grammatical information and / or the second semantic logical relevance;
[0164] In this step, the second grammatical information and / or the second semantic logical relevance may be input into the semantic fluency acquisition model to obtain the second semantic fluency of the original text output by the semantic fluency acquisition model.
[0165] For the specific training method of the semantic fluency acquisition model, please refer to the content in step S201.
[0166] In this way, when obtaining the second semantic fluency of the original text, only the second grammatical information can be used, only the second semantic logical relevance can be used, or both the second grammatical information and the second semantic logical relevance can be used. Which of the three methods is used depends on which method is used when training the semantic fluency acquisition model.
[0167] The accuracy of the second semantic fluency obtained by using both the second grammatical information and the second semantic logical relevance is greater than the accuracy of the second semantic fluency obtained by using only the second grammatical information or only the second semantic logical relevance.
[0168] Of course, the second semantic fluency of the original text may also be acquired according to the second grammatical information and / or the second semantic logical relevance by other methods, which is not limited in the present application.
[0169] 3034. Obtain a second semantic similarity between the third semantic information and the fourth semantic information;
[0170] In this step, the second semantic similarity between the third semantic information and the fourth semantic information can be obtained according to the semantic similarity acquisition model. For example, the third semantic information and the fourth semantic information are input into the semantic similarity acquisition model to obtain the second semantic similarity output by the semantic similarity acquisition model.
[0171] For the specific training method of the semantic similarity acquisition model, please refer to the content in step S202.
[0172] Of course, the second semantic similarity between the third semantic information and the fourth semantic information may also be acquired by other methods, which is not limited in the present application.
[0173] 3035. Determine whether the translated text is a machine translated text according to the first semantic fluency, the first semantic similarity, the second semantic fluency, and the second semantic similarity.
[0174] In one embodiment of the present application, the classification model can be trained in advance, and the specific training method includes: obtaining multiple sample information sets, each sample information set including a first sample semantic fluency and a first sample semantic similarity of a sample original text, and a second sample semantic fluency and a second sample semantic similarity of a sample translated text.
[0175] The first sample semantic fluency is obtained based on the first sample grammatical information and / or the first sample semantic logical relevance of the sample translation text, and the first sample semantic similarity is obtained based on the first sample semantic information of each sample sub-text in the sample translation text and the second sample semantic information of the sample translation text as a whole.
[0176] The second sample semantic fluency is obtained based on the second sample grammatical information and / or the second sample semantic logical relevance of the sample original text, and the second sample semantic similarity is obtained based on the third sample semantic information of each sample sub-text in the sample original text and the fourth sample semantic information of the sample original text as a whole.
[0177] The first sample semantic fluency, the first sample semantic similarity, the second sample semantic fluency or the second sample semantic similarity is marked with a marking result of whether the sample translation text is a machine translation text.
[0178] Then, the model is trained using the first sample semantic fluency and the first sample semantic similarity of the sample original text, and the second sample semantic fluency and the second sample semantic similarity of the sample translated text in each sample information set until the weights in the model converge to obtain a classification model.
[0179] Thus, in this step, a classification model can be used to determine whether the translated text is a machine translated text based on the first semantic fluency, the first semantic similarity, the second semantic fluency and the second semantic similarity. For example, the first semantic fluency, the first semantic similarity, the second semantic fluency and the second semantic similarity are input into the classification model to obtain a result output by the classification model to indicate whether the translated text is a machine translated text.
[0180] Of course, other methods may also be used to determine whether the translated text is a machine translated text based on the first semantic fluency and the first semantic similarity, and this application does not limit this.
[0181] For example, the first semantic fluency and the first semantic similarity are weighted and summed to obtain a first score, and then a first difference between the first semantic similarity and the second semantic similarity is determined; then a second difference between the second semantic fluency and the second semantic similarity is determined; the first difference and the first difference are weighted and summed, and the inverse of the value obtained by the weighted sum is calculated to obtain a second score, and then whether the translated text is a machine translated text is determined based on the first score and the second score. For example, the first score and the second score are weighted and summed to obtain a third score, and if the third score is greater than or equal to a preset threshold, it is determined that the translated text is not a machine translated text, and if the third score is less than the preset threshold, it is determined that the translated text is a machine translated text.
[0182] In this application, generally, when a translated text is a manually translated text, the higher the degree to which the translated text conforms to normal grammatical habits and the higher the semantic logical correlation between the sub-texts in the translated text, the higher the semantic fluency of the translated text. When a translated text is a machine translated text, the lower the degree to which the translated text conforms to normal grammatical habits and / or the lower the semantic logical correlation between the sub-texts in the translated text, the lower the semantic fluency of the translated text.
[0183] That is, the higher the semantic fluency of the translated text, the lower the possibility that the translated text is a machine-translated text, and the lower the semantic fluency of the translated text, the higher the possibility that the translated text is a machine-translated text.
[0184] In the present application, for any original text in the collected bilingual corpus, the original text is used to describe a semantics, and each sub-text included in the original text is used to describe the semantics. If the translation text corresponding to the original text is obtained through manual translation, the translation text is also used to describe the semantics, and each sub-text included in the translation text is also used to describe the semantics.
[0185] However, if the translation text corresponding to the original text is obtained by translation through the translation model, the generalization degree and intelligence of the translation model are limited. Therefore, in the translation process, there is a situation where at least one sub-text included in the original text is translated incorrectly, so that the semantics of the incorrectly translated sub-text is different from the semantics of other sub-texts. The semantic information of each sub-text in the local information includes at least two different semantic information, while the overall semantic information of the translated text in the global information is still the original semantic information. Therefore, there is a difference between the two semantic information.
[0186] In this way, if the similarity between the first semantic information of each sub-text in the translation text and the second semantic information of the translation text is higher, the possibility that the translation text is a machine translation is lower; if the similarity between the first semantic information of each sub-text in the translation text and the second semantic information of the translation text is lower, the possibility that the translation text is a machine translation is higher.
[0187] Moreover, when a translated text is a manually translated text, the meaning expressed by the original text and the translated text is the same, the difference between the semantic similarity of the original text and the semantic similarity of the translated text is small, and the difference between the semantic fluency of the original text and the semantic fluency of the translated text is also small.
[0188] If the difference between the semantic similarity of the original text and the semantic similarity of the translated text is large, or the difference between the semantic fluency of the original text and the semantic fluency of the translated text is large, the translated text may be a machine-translated text.
[0189] Therefore, when a translation text is a human-translated text, the meaning of the original text and the translation text is the same, the difference between the local information of the original text and the local information of the translation text is small, and the difference between the global information of the original text and the global information of the translation text is also small.
[0190] If the difference between the local information of the original text and the local information of the translated text is large, or the difference between the global information of the original text and the global information of the translated text is large, the translated text may be a machine-translated text.
[0191] It is determined whether the translated text is a machine translated text according to the first local information, the first global information, the second local information and the second global information.
[0192] Therefore, in the present application, the accuracy of the detection result can be improved when the second local information and the second global information of the original text and the first local information and the first global information of the translated text are combined to detect whether the translated text is a machine translated text.
[0193] Figure 4 is a block diagram of a translation detection device according to an exemplary embodiment. Figure 4 As shown, the device comprises:
[0194] A first acquisition module 11, used to acquire a translation text;
[0195] A second acquisition module 12, used to acquire first local information and first global information of the translation text;
[0196] The determination module 13 is used to determine whether the translation text is a machine translation text at least according to the first local information and the first global information.
[0197] In an optional implementation, the first local information includes first semantic information of each subtext in the translation text and first grammatical information of each subtext;
[0198] The first global information includes second semantic information of the translated text and first semantic logical correlations between the sub-texts.
[0199] In an optional implementation, the determining module 13 includes:
[0200] A first acquiring unit, configured to acquire a first semantic fluency of the translated text according to the first grammatical information and / or the first semantic logical relevance;
[0201] A second acquiring unit, configured to acquire a first semantic similarity between the first semantic information and the second semantic information;
[0202] The first determining unit is used to determine whether the translated text is a machine translated text according to the first semantic fluency and the first semantic similarity.
[0203] In an optional implementation, the first acquiring unit includes:
[0204] The first input subunit is used to input the first grammatical information and / or the first semantic logic relevance into a semantic fluency acquisition model to obtain a first semantic fluency of the translated text output by the semantic fluency acquisition model.
[0205] In an optional implementation, the first acquiring unit further includes:
[0206] A first acquisition subunit is used to acquire a plurality of sample information sets, each of which includes sample grammatical information of each sample subtext in the sample text and / or sample semantic logical relevance between each sample subtext in the sample text, and the sample semantic fluency of the sample text is annotated in the sample grammatical information or the sample semantic logical relevance;
[0207] The first training sub-unit is used to train the model using the sample grammatical information of each sample sub-text in the sample text in each sample information set and / or the sample semantic logical relevance between each sample sub-text in the sample text until the weights in the model converge to obtain a semantic fluency acquisition model.
[0208] In an optional implementation, the second acquiring unit includes:
[0209] The second acquisition subunit is used to acquire the first semantic similarity between the first semantic information and the second semantic information according to a semantic similarity acquisition model.
[0210] In an optional implementation, the second acquiring unit further includes:
[0211] A third acquisition subunit is used to acquire multiple sample information sets, each of which includes first sample semantic information of each sample subtext in the sample text and second sample semantic information of the sample text as a whole, and the first sample semantic information or the second sample semantic information is marked with a sample semantic similarity between the first sample semantic information and the second sample semantic information;
[0212] The second training subunit is used to train the model using the first sample semantic information and the second sample semantic information in each sample information set until the weights in the model converge to obtain a semantic similarity acquisition model.
[0213] In an optional implementation, the first determining unit includes:
[0214] The second input subunit is used to input the first semantic fluency and the first semantic similarity into a classification model to obtain a result output by the classification model to indicate whether the translated text is a machine translated text.
[0215] In an optional implementation, the first determining unit further includes:
[0216] A fourth acquisition subunit is used to acquire multiple sample information sets, each of which includes a sample semantic fluency and a sample semantic similarity of a sample text, wherein the sample semantic fluency is acquired based on the sample grammatical information and / or the sample semantic logical relevance of the sample text, and the sample semantic similarity is acquired based on the first sample semantic information of each sample subtext in the sample text and the second sample semantic information of the sample text as a whole, and the sample semantic fluency or the sample semantic similarity is annotated with a marking result indicating whether the sample text is a machine translated text;
[0217] The third training subunit is used to train the model using the sample semantic fluency and sample semantic similarity in each sample information set until all weights in the model converge to obtain a classification model.
[0218] In an optional implementation, the determining module 13 includes:
[0219] A third acquisition unit, used to acquire an original text corresponding to the translated text;
[0220] A fourth acquisition unit, used for acquiring second local information and second global information of the original text;
[0221] The second determining unit is used to determine whether the translation text is a machine translation text according to the first local information, the first global information, the second local information and the second global information.
[0222] In an optional implementation, the second local information includes third semantic information of each subtext in the original text and second grammatical information of each subtext in the original text;
[0223] The second global information includes fourth semantic information of the original text and second semantic logical correlations between subtexts in the original text.
[0224] In an optional implementation, the second determining unit includes:
[0225] a fifth acquiring subunit, configured to acquire a first semantic fluency of the translated text according to the first grammatical information and / or the first semantic logical relevance;
[0226] a sixth acquisition subunit, configured to acquire a first semantic similarity between the first semantic information and the second semantic information;
[0227] a seventh acquisition subunit, configured to acquire a second semantic fluency of the original text according to the second grammatical information and / or the second semantic logical relevance;
[0228] an eighth acquiring subunit, configured to acquire a second semantic similarity between the third semantic information and the fourth semantic information;
[0229] A determination subunit is used to determine whether the translation text is a machine translation text according to the first semantic fluency, the first semantic similarity, the second semantic fluency, and the second semantic similarity.
[0230] In the present application, a translation text is obtained; first local information of the translation text is obtained and first global information of the translation text is obtained; and whether the translation text is a machine translation text is determined at least based on the first local information and the first global information. If the translation text is translated based on a text translation model based on NMT, the present application can accurately detect whether the translation text is a machine translation text through the first local information of the translation text and the first global information of the translation text. Therefore, compared with the prior art, the present application can improve the detection accuracy when detecting whether the translation text translated based on the text translation model of NMT is a machine translation text.
[0231] Secondly, in the prior art, the participation of language experts is required when selecting the features of the translated text, resulting in high labor costs. However, in the present application, the participation of language experts is not required when obtaining the first local information of the translated text and the first global information of the translated text, thereby saving labor costs.
[0232] The embodiment of the present application also provides a non-volatile readable storage medium, which stores one or more modules (programs). When the one or more modules are applied to a device, the device can execute instructions (instructions) of each method step in the embodiment of the present application.
[0233] The embodiment of the present application provides one or more machine-readable media on which instructions are stored, and when executed by one or more processors, an electronic device executes the translation detection method described in one or more of the above embodiments. In the embodiment of the present application, the electronic device includes a server, a gateway, a sub-device, etc., and the sub-device is an Internet of Things device or other device.
[0234] The embodiments of the present disclosure may be implemented as an apparatus configured as desired using any appropriate hardware, firmware, software, or any combination thereof, and the apparatus may include electronic devices such as servers (clusters), terminal devices such as IoT devices, and the like.
[0235] Figure 5 An exemplary apparatus 1300 that can be used to implement various embodiments described in this application is schematically shown.
[0236] For one embodiment, Figure 5 An exemplary apparatus 1300 is shown having one or more processors 1302, a control module (chip set) 1304 coupled to at least one of the (one or more) processors 1302, a memory 1306 coupled to the control module 1304, a non-volatile memory (NVM) / storage device 1308 coupled to the control module 1304, one or more input / output devices 1310 coupled to the control module 1304, and a network interface 1312 coupled to the control module 1304.
[0237] The processor 1302 may include one or more single-core or multi-core processors, and the processor 1302 may include any combination of general-purpose processors or special-purpose processors (such as graphics processors, application processors, baseband processors, etc.). In some embodiments, the device 1300 can be used as a server device such as a gateway or controller described in the embodiments of the present application.
[0238] In some embodiments, the device 1300 may include one or more computer-readable media (e.g., memory 1306 or NVM / storage device 1308) having instructions 1314 and one or more processors 1302 configured to execute the instructions 1314 in combination with the one or more computer-readable media to implement a module to perform the actions described in the present disclosure.
[0239] For one embodiment, the control module 1304 may include any suitable interface controller to provide any suitable interface to at least one of the processor(s) 1302 and / or any suitable device or component in communication with the control module 1304 .
[0240] The control module 1304 may include a memory controller module to provide an interface to the memory 1306. The memory controller module may be a hardware module, a software module, and / or a firmware module.
[0241] The memory 1306 may be used, for example, to load and store data and / or instructions 1314 for the device 1300. For one embodiment, the memory 1306 may include any suitable volatile memory, such as a suitable DRAM. In some embodiments, the memory 1306 may include double data rate type four synchronous dynamic random access memory (DDR4 SDRAM).
[0242] For one embodiment, control module 1304 may include one or more input / output controllers to provide an interface to NVM / storage device 1308 and input / output device(s) 1310 .
[0243] For example, NVM / storage 1308 may be used to store data and / or instructions 1314. NVM / storage 1308 may include any suitable non-volatile memory (e.g., flash memory) and / or may include any suitable non-volatile storage device(s) (e.g., one or more hard disk drives (HDDs), one or more compact disk (CD) drives, and / or one or more digital versatile disk (DVD) drives).
[0244] NVM / storage device 1308 may include storage resources that are physically part of the device on which apparatus 1300 is installed, or it may be accessible to the device without being part of the device. For example, NVM / storage device 1308 may be accessed via input / output device(s) 1310 over a network.
[0245] (One or more) input / output devices 1310 may provide an interface for the apparatus 1300 to communicate with any other appropriate device, and the input / output device 1310 may include a communication component, an audio component, a sensor component, etc. The network interface 1312 may provide an interface for the apparatus 1300 to communicate through one or more networks, and the apparatus 1300 may wirelessly communicate with one or more components of a wireless network according to any of one or more wireless network standards and / or protocols, for example, accessing a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G, 5G, etc., or a combination thereof for wireless communication.
[0246] For one embodiment, at least one of the processor(s) 1302 may be packaged together with the logic of one or more controllers (e.g., a memory controller module) of the control module 1304. For one embodiment, at least one of the processor(s) 1302 may be packaged together with the logic of one or more controllers of the control module 1304 to form a system-in-package (SiP). For one embodiment, at least one of the processor(s) 1302 may be integrated on the same die with the logic of one or more controllers of the control module 1304. For one embodiment, at least one of the processor(s) 1302 may be integrated on the same die with the logic of one or more controllers of the control module 1304 to form a system-on-chip (SoC).
[0247] In various embodiments, the device 1300 may be, but is not limited to, a terminal device such as a server, a desktop computing device, or a mobile computing device (e.g., a laptop computing device, a handheld computing device, a tablet computer, a netbook, etc.). In various embodiments, the device 1300 may have more or fewer components and / or a different architecture. For example, in some embodiments, the device 1300 includes one or more cameras, a keyboard, a liquid crystal display (LCD) screen (including a touch screen display), a non-volatile memory port, multiple antennas, a graphics chip, an application-specific integrated circuit (ASIC), and a speaker.
[0248] An embodiment of the present application provides an electronic device, including: one or more processors; and one or more machine-readable media having instructions stored thereon, which, when executed by the one or more processors, enable the processor to execute the translation detection method as described in one or more of the embodiments of the present application.
[0249] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.
[0250] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.
[0251] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0252] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0253] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0254] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.
[0255] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.
[0256] The above is a detailed introduction to a translation detection method and device provided by the present application. Specific examples are used in this article to illustrate the principle and implementation method of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application. At the same time, for those skilled in the art, according to the idea of the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as limiting the present application.
Claims
1. A translation detection method, characterized in that: The method comprises: Get the translated text; Acquire first local information and first global information of the translation text; determining whether the translated text is a machine translated text at least according to the first local information and the first global information; The first global information includes the second semantic information of the translation text and the first semantic logical correlation between each subtext in the translation text; The second semantic information is used to describe the meaning or theme expressed by the translated text as a whole; The first local information includes first semantic information of each subtext in the translation text and first grammatical information of each subtext.
2. The method according to claim 1, characterized in that The determining whether the translated text is a machine translated text at least according to the first local information and the first global information includes: Acquire a first semantic fluency of the translated text according to the first grammatical information and / or the first semantic logical relevance; Acquire a first semantic similarity between the first semantic information and the second semantic information; Determine whether the translated text is a machine translated text according to the first semantic fluency and the first semantic similarity.
3. The method according to claim 2, characterized in that The obtaining the first semantic fluency of the translated text according to the first grammatical information and / or the first semantic logic relevance includes: The first grammatical information and / or the first semantic logic relevance is input into a semantic fluency acquisition model to obtain a first semantic fluency of the translated text output by the semantic fluency acquisition model.
4. The method according to claim 3, characterized in that: The method further comprises: Acquire multiple sample information sets, each sample information set includes sample grammatical information of each sample sub-text in the sample text and / or sample semantic logical relevance between each sample sub-text in the sample text, and the sample semantic fluency of the sample text is annotated in the sample grammatical information or the sample semantic logical relevance; The model is trained using sample grammatical information of each sample sub-text in the sample text in each sample information set and / or sample semantic logical relevance between each sample sub-text in the sample text until all weights in the model converge to obtain a semantic fluency acquisition model.
5. The method according to claim 2, characterized in that: The obtaining the first semantic similarity between the first semantic information and the second semantic information includes: A first semantic similarity between the first semantic information and the second semantic information is acquired according to a semantic similarity acquisition model.
6. The method according to claim 5, characterized in that The method further comprises: Acquire multiple sample information sets, each of which includes first sample semantic information of each sample subtext in the sample text and second sample semantic information of the sample text as a whole, and the first sample semantic information or the second sample semantic information is marked with a sample semantic similarity between the first sample semantic information and the second sample semantic information; The model is trained using the first sample semantic information and the second sample semantic information in each sample information set until all weights in the model converge, thereby obtaining a semantic similarity acquisition model.
7. The method according to claim 2, characterized in that The determining whether the translated text is a machine translated text according to the first semantic fluency and the first semantic similarity includes: The first semantic fluency and the first semantic similarity are input into a classification model to obtain a result output by the classification model to indicate whether the translated text is a machine translated text.
8. The method according to claim 7, characterized in that The method further comprises: Acquire multiple sample information sets, each of which includes sample semantic fluency and sample semantic similarity of a sample text, wherein the sample semantic fluency is acquired based on sample grammatical information and / or sample semantic logical relevance of the sample text, and the sample semantic similarity is acquired based on first sample semantic information of each sample subtext in the sample text and second sample semantic information of the sample text as a whole, and the sample semantic fluency or the sample semantic similarity is annotated with a marking result indicating whether the sample text is a machine translated text; The model is trained using the sample semantic fluency and sample semantic similarity in each sample information set until the weights in the model converge to obtain a classification model.
9. The method according to claim 1, characterized in that: The determining whether the translated text is a machine translated text at least according to the first local information and the first global information includes: Obtaining the original text corresponding to the translated text; Acquire the second local information and the second global information of the original text; Determine whether the translation text is a machine translation text according to the first local information, the first global information, the second local information and the second global information.
10. The method according to claim 9, characterized in that The second local information includes third semantic information of each subtext in the original text and second grammatical information of each subtext in the original text; The second global information includes fourth semantic information of the original text and second semantic logical correlations between subtexts in the original text.
11. The method according to claim 10, characterized in that The determining whether the translation text is a machine translation text according to the first local information, the first global information, the second local information and the second global information includes: Acquire a first semantic fluency of the translated text according to the first grammatical information and / or the first semantic logical relevance; Acquire a first semantic similarity between the first semantic information and the second semantic information; Acquire a second semantic fluency of the original text according to the second grammatical information and / or the second semantic logical relevance; Acquire a second semantic similarity between the third semantic information and the fourth semantic information; Determine whether the translated text is a machine translated text according to the first semantic fluency, the first semantic similarity, the second semantic fluency, and the second semantic similarity.
12. A translation detection device, characterized in that: The device comprises: A first acquisition module, used to acquire a translation text; A second acquisition module, used to acquire first local information and first global information of the translation text; a determination module, configured to determine whether the translation text is a machine translation text based at least on the first local information and the first global information; The first global information includes the second semantic information of the translation text and the first semantic logical correlation between each subtext in the translation text; The second semantic information is used to describe the meaning or theme expressed by the translated text as a whole; The first local information includes first semantic information of each subtext in the translation text and first grammatical information of each subtext.
13. The device according to claim 12, characterized in that The determination module comprises: A first acquiring unit, configured to acquire a first semantic fluency of the translated text according to the first grammatical information and / or the first semantic logical relevance; A second acquiring unit, configured to acquire a first semantic similarity between the first semantic information and the second semantic information; The first determining unit is used to determine whether the translated text is a machine translated text according to the first semantic fluency and the first semantic similarity.
14. The device according to claim 13, characterized in that The first acquiring unit includes: The first input subunit is used to input the first grammatical information and / or the first semantic logic relevance into a semantic fluency acquisition model to obtain a first semantic fluency of the translated text output by the semantic fluency acquisition model.
15. The device according to claim 14, characterized in that The first acquisition unit further includes: A first acquisition subunit is used to acquire a plurality of sample information sets, each of which includes sample grammatical information of each sample subtext in the sample text and / or sample semantic logical relevance between each sample subtext in the sample text, and the sample semantic fluency of the sample text is annotated in the sample grammatical information or the sample semantic logical relevance; The first training sub-unit is used to train the model using the sample grammatical information of each sample sub-text in the sample text in each sample information set and / or the sample semantic logical relevance between each sample sub-text in the sample text until the weights in the model converge to obtain a semantic fluency acquisition model.
16. The device according to claim 13, characterized in that The second acquiring unit includes: The second acquisition subunit is used to acquire the first semantic similarity between the first semantic information and the second semantic information according to a semantic similarity acquisition model.
17. The device according to claim 16, characterized in that The second acquiring unit further includes: A third acquisition subunit is used to acquire multiple sample information sets, each of which includes first sample semantic information of each sample subtext in the sample text and second sample semantic information of the sample text as a whole, and the first sample semantic information or the second sample semantic information is marked with a sample semantic similarity between the first sample semantic information and the second sample semantic information; The second training subunit is used to train the model using the first sample semantic information and the second sample semantic information in each sample information set until the weights in the model converge to obtain a semantic similarity acquisition model.
18. The device according to claim 13, characterized in that The first determining unit includes: The second input subunit is used to input the first semantic fluency and the first semantic similarity into a classification model to obtain a result output by the classification model to indicate whether the translated text is a machine translated text.
19. The device according to claim 18, characterized in that The first determining unit further includes: A fourth acquisition subunit is used to acquire multiple sample information sets, each of which includes a sample semantic fluency and a sample semantic similarity of a sample text, wherein the sample semantic fluency is acquired based on the sample grammatical information and / or the sample semantic logical relevance of the sample text, and the sample semantic similarity is acquired based on the first sample semantic information of each sample subtext in the sample text and the second sample semantic information of the sample text as a whole, and the sample semantic fluency or the sample semantic similarity is annotated with a marking result indicating whether the sample text is a machine translated text; The third training subunit is used to train the model using the sample semantic fluency and sample semantic similarity in each sample information set until all weights in the model converge to obtain a classification model.
20. The device according to claim 12, characterized in that The determination module comprises: A third acquisition unit, used to acquire an original text corresponding to the translated text; A fourth acquisition unit, used for acquiring second local information and second global information of the original text; The second determining unit is used to determine whether the translation text is a machine translation text according to the first local information, the first global information, the second local information and the second global information.
21. The device according to claim 20, characterized in that The second local information includes third semantic information of each subtext in the original text and second grammatical information of each subtext in the original text; The second global information includes fourth semantic information of the original text and second semantic logical correlations between subtexts in the original text.
22. The device according to claim 21, characterized in that The second determining unit includes: a fifth acquiring subunit, configured to acquire a first semantic fluency of the translated text according to the first grammatical information and / or the first semantic logical relevance; a sixth acquisition subunit, configured to acquire a first semantic similarity between the first semantic information and the second semantic information; a seventh acquisition subunit, configured to acquire a second semantic fluency of the original text according to the second grammatical information and / or the second semantic logical relevance; an eighth acquiring subunit, configured to acquire a second semantic similarity between the third semantic information and the fourth semantic information; A determination subunit is used to determine whether the translation text is a machine translation text according to the first semantic fluency, the first semantic similarity, the second semantic fluency, and the second semantic similarity.
Citation Information
Patent Citations
Translation detection method and device and electronic device
CN109344408A
Calibration optimization method and device for machine translation and electronic device
CN109670191A