Translation difficulty determination method and device based on multistage semantic features

Abstract

The invention provides a translation difficulty determination method and device based on multistage semantic features, and the method comprises the steps: carrying out the sentence segmentation of a target article, and obtaining a sentence segmentation result of the target article; based on the phrasing result, respectively determining word level complexity, sentence level complexity and semantic confusion of the target article; and based on the word level complexity, the sentence level complexity and the semantic confusion, determining a translation difficulty value of the target article. According to the translation difficulty determination method and device based on the multi-level semantic features, translation difficulty grades of the articles can be quickly and accurately given for different articles, and a reasonable basis is provided for subsequent translation task distribution.

Description

Technical Field

[0001] The present invention relates to the technical field of text translation, and in particular to a method and device for determining translation difficulty based on multi-level semantic features. Background Art

[0002] As a commercial activity, article translation requires speeding up the translation process and reducing costs while ensuring the quality of the translation. Therefore, rationally analyzing the difficulty of the article to be translated becomes a key step to improve translation efficiency.

[0003] In the purely manual translation process, different translation difficulty parts of the article are assigned to translation experts of different levels for translation review. The more difficult parts are translated and reviewed by senior experts, and the simpler parts are translated and reviewed by ordinary translators. This can greatly reduce the time and labor costs in the translation process and ensure that the overall translation quality of the article meets expectations. In the translation process combined with machine translation, articles of different translation difficulty can be handed over to different deep learning machine translation engines, which can reduce the overall translation time and cost, reduce the difficulty of manual review, and improve the speed and quality of delivery.

[0004] In short, reasonably judging the difficulty of article translation is a key link in reducing labor costs and improving translation quality in translation business. Therefore, a method for judging translation difficulty is urgently needed. Summary of the invention

[0005] The present invention provides a method and device for determining translation difficulty based on multi-level semantic features, which are used to solve the defects of the prior art that the difficulty of article translation cannot be reasonably judged and the translation efficiency is low, and to quickly and accurately give translation difficulty ratings for different articles, thereby providing a reasonable basis for subsequent translation task allocation.

[0006] The present invention provides a method for determining translation difficulty based on multi-level semantic features, comprising the following steps: Perform sentence segmentation processing on the target article to obtain a sentence segmentation result of the target article; Based on the sentence segmentation results, respectively determining the word-level complexity, sentence-level complexity, and semantic perplexity of the target article; A translation difficulty value of the target article is determined based on the word level complexity, the sentence level complexity and the semantic perplexity.

[0007] According to a method for determining translation difficulty based on multi-level semantic features provided by the present invention, based on the sentence segmentation results, the word-level complexity and sentence-level complexity of the target article are determined respectively, specifically including: Determining the language category of the target article; Based on the sentence segmentation results and the language category, the word-level complexity and sentence-level complexity of the target article are calculated.

[0008] According to a method for determining translation difficulty based on multi-level semantic features provided by the present invention, the word-level complexity of the target article is calculated in the following manner: Performing word segmentation processing on the sentence segmentation result to obtain a word segmentation result; Calculating the lexical complexity of each sentence in the sentence segmentation result based on the common vocabulary list corresponding to the language category and the word segmentation result; Calculate the word-level complexity of the target article based on the lexical complexity of each sentence; The common vocabulary list is obtained by sorting the vocabulary frequencies in multiple articles of the language category from high to low.

[0009] According to a method for determining translation difficulty based on multi-level semantic features provided by the present invention, the sentence-level complexity of the target article is calculated in the following manner: Based on the syntactic analyzer corresponding to the language category, parse and obtain the component syntactic tree of each sentence in the sentence segmentation result; Based on the depth features and width features of the component syntax tree, the sentence level complexity of the target article is calculated.

[0010] According to a method for determining translation difficulty based on multi-level semantic features provided by the present invention, based on the sentence segmentation result, the semantic perplexity of the target article is determined, specifically comprising: Based on a plurality of different neural network language models, determining the perplexity of each sentence in the sentence segmentation result, and obtaining an average perplexity of each sentence; Based on the average perplexity of each sentence, the semantic perplexity of the target article is determined.

[0011] According to a method for determining translation difficulty based on multi-level semantic features provided by the present invention, based on the word level complexity, the sentence level complexity and the semantic perplexity, determining the translation difficulty value of the target article specifically includes: Based on the average value and standard deviation of the lexical complexity of each sentence in the sentence segmentation result and the word level complexity, a word level complexity weight is obtained; Based on the average value and standard deviation of the depth feature and width feature of each component syntactic tree in the sentence segmentation result, and the sentence level complexity, a sentence level complexity weight is obtained; Obtaining a semantic complexity weight based on the average value and standard deviation of the average perplexity of each sentence in the sentence segmentation results and the semantic perplexity of the target article; A translation difficulty value of the target article is determined based on the word-level complexity weight, the sentence-level complexity weight, and the semantic complexity weight.

[0012] The present invention also provides a translation difficulty determination device based on multi-level semantic features, comprising the following modules: A processing module, used for performing sentence segmentation processing on the target article to obtain the sentence segmentation result of the target article; A calculation module, used to determine the word-level complexity, sentence-level complexity and semantic perplexity of the target article based on the sentence segmentation results; A determination module is used to determine a translation difficulty value of the target article based on the word level complexity, the sentence level complexity and the semantic perplexity.

[0013] According to a translation difficulty determination device based on multi-level semantic features provided by the present invention, based on the sentence segmentation result, the word level complexity and sentence level complexity of the target article are determined respectively, specifically including: Determining the language category of the target article; Based on the sentence segmentation results and the language category, the word-level complexity and sentence-level complexity of the target article are calculated.

[0014] According to a translation difficulty determination device based on multi-level semantic features provided by the present invention, the word level complexity of the target article is calculated in the following manner: Performing word segmentation processing on the sentence segmentation result to obtain a word segmentation result; Calculating the lexical complexity of each sentence in the sentence segmentation result based on the common vocabulary list corresponding to the language category and the word segmentation result; Calculate the word-level complexity of the target article based on the lexical complexity of each sentence; The common vocabulary list is obtained by sorting the vocabulary frequencies in multiple articles of the language category from high to low.

[0015] According to a translation difficulty determination device based on multi-level semantic features provided by the present invention, the sentence level complexity of the target article is calculated in the following manner: Based on the syntactic analyzer corresponding to the language category, parse and obtain the component syntactic tree of each sentence in the sentence segmentation result; Based on the depth features and width features of the component syntax tree, the sentence level complexity of the target article is calculated.

[0016] According to a translation difficulty determination device based on multi-level semantic features provided by the present invention, based on the sentence segmentation result, the semantic perplexity of the target article is determined, specifically comprising: Based on a plurality of different neural network language models, determining the perplexity of each sentence in the sentence segmentation result, and obtaining an average perplexity of each sentence; Based on the average perplexity of each sentence, the semantic perplexity of the target article is determined.

[0017] According to a translation difficulty determination device based on multi-level semantic features provided by the present invention, based on the word level complexity, the sentence level complexity and the semantic perplexity, the translation difficulty value of the target article is determined, specifically including: Based on the average value and standard deviation of the lexical complexity of each sentence in the sentence segmentation result and the word level complexity, a word level complexity weight is obtained; Based on the average value and standard deviation of the depth feature and width feature of each component syntactic tree in the sentence segmentation result, and the sentence level complexity, a sentence level complexity weight is obtained; Obtaining a semantic complexity weight based on the average value and standard deviation of the average perplexity of each sentence in the sentence segmentation results and the semantic perplexity of the target article; A translation difficulty value of the target article is determined based on the word-level complexity weight, the sentence-level complexity weight, and the semantic complexity weight.

[0018] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for determining translation difficulty based on multi-level semantic features as described above is implemented.

[0019] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method for determining translation difficulty based on multi-level semantic features as described above is implemented.

[0020] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned methods for determining translation difficulty based on multi-level semantic features.

[0021] The method and device for determining translation difficulty based on multi-level semantic features provided by the present invention obtain sentence segmentation results by performing sentence segmentation processing on the target article, and then determine the word level complexity, sentence level complexity and semantic perplexity of the target article according to the sentence segmentation results. Finally, the translation difficulty value of the target article is determined according to the word level complexity, sentence level complexity and semantic perplexity. The translation difficulty rating of the article can be given quickly and accurately for different articles, providing a reasonable basis for subsequent translation task allocation. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0023] Figure 1 It is a flowchart of the method for determining translation difficulty based on multi-level semantic features provided by the present invention.

[0024] Figure 2 It is a structural schematic diagram of a translation difficulty determination device based on multi-level semantic features provided by the present invention.

[0025] Figure 3 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] Figure 1 is a flow chart of the method for determining translation difficulty based on multi-level semantic features provided by the present invention, such as Figure 1 As shown, the method comprises the following steps: Step 100: Sentence segmentation is performed on the target article to obtain the sentence segmentation result of the target article.

[0028] Step 101: Based on the sentence segmentation results, determine the word-level complexity, sentence-level complexity, and semantic perplexity of the target article.

[0029] Step 102: Determine the translation difficulty value of the target article based on the word-level complexity, sentence-level complexity and semantic perplexity.

[0030] Specifically, the target article is an article for which the translation difficulty needs to be determined. In the embodiment of the present invention, the target article may first be segmented to obtain a sentence segmentation result.

[0031] It is understandable that for articles to be translated in different formats, they can be processed into a unified plain text format, retaining information such as formulas and tables in the article, using carriage returns to retain the paragraph structure of the article, and the article can be processed into sentences.

[0032] After obtaining the sentence segmentation results, the word-level complexity of the target article, the sentence-level complexity of the target article, and the semantic perplexity of the target article can be calculated according to the sentence segmentation results.

[0033] Among the methods for evaluating the difficulty of translating articles, word-level complexity, sentence-level complexity, and semantic perplexity measure the difficulty of translating articles at different levels. Among them, word-level complexity refers to the complexity of the vocabulary in the article; sentence-level complexity refers to the complexity of the syntactic structure of the sentences in the article; and semantic perplexity refers to the difficulty of semantic understanding of the sentences in the article.

[0034] The word-level complexity can be determined by the frequency of using uncommon words in the target article. The higher the value, the more difficult it is to translate the article at the lexical level. The sentence-level complexity can be determined by the structural complexity of the sentences in the article. The higher the value, the more difficult it is to translate the article at the syntactic level. The semantic perplexity can be determined by the difficulty of the article at the semantic understanding level, for example, it can be obtained through a neural network language model. The higher the value, the more difficult it is to translate the article at the semantic level.

[0035] Finally, the translation difficulty value of the target article can be determined based on the word-level complexity, sentence-level complexity and semantic perplexity of the target article. For example, the contribution of each part to the overall translation difficulty can be determined through the weighted coefficient, and finally the translation difficulty score of the target article can be obtained. This value can be used to scientifically evaluate the difficulty of translating the target article and provide a basis for the allocation of translation tasks.

[0036] The method for determining translation difficulty based on multi-level semantic features provided by the present invention obtains sentence segmentation results by performing sentence segmentation processing on a target article, and then determines the word level complexity and sentence level complexity of the target article according to the sentence segmentation results, and calculates the semantic perplexity of the target article. Finally, the translation difficulty value of the target article is determined according to the word level complexity, sentence level complexity and semantic perplexity of the target article. The method can quickly and accurately give translation difficulty ratings for different articles, and provide a reasonable basis for subsequent translation task allocation.

[0037] According to a method for determining translation difficulty based on multi-level semantic features provided by the present invention, based on the sentence segmentation results, the word-level complexity and sentence-level complexity of the target article are determined respectively, specifically including: Determine the language category of the target article; Based on the sentence segmentation results and language category, the word-level complexity and sentence-level complexity of the target article are calculated.

[0038] Specifically, in the process of determining the word-level complexity and sentence-level complexity of the target article according to the sentence segmentation results, the language category of the target article can be determined first, and then the word-level complexity and sentence-level complexity of the target article can be calculated according to the sentence segmentation results and the language category.

[0039] Different languages ​​have their own vocabulary systems and syntactic structures. Therefore, in order to accurately assess the complexity of vocabulary and sentence structure, it is necessary to evaluate different languages ​​based on the specific language characteristics of the language.

[0040] According to a method for determining translation difficulty based on multi-level semantic features provided by the present invention, the word-level complexity of the target article is calculated in the following manner: Perform word segmentation processing on the sentence segmentation result to obtain the word segmentation result; Based on the common vocabulary list and word segmentation results corresponding to the language category, calculate the lexical complexity of each sentence in the sentence segmentation results; Based on the lexical complexity of each sentence, the word-level complexity of the target article was calculated; Among them, the common vocabulary list is sorted from high to low according to the frequency of vocabulary in multiple articles of the language category.

[0041] Specifically, in the process of determining the word level complexity, the sentence segmentation result may be firstly processed by word segmentation, and each sentence in the sentence segmentation result may be split into multiple words.

[0042] Then, for each sentence, according to the common vocabulary list corresponding to the language category of the target article, it is determined whether multiple words in the sentence are common words, so as to calculate the lexical complexity of the sentence.

[0043] In some implementations, the translated articles of the language may be collected and sorted, and word segmentation and word form restoration may be performed, and the frequency of the words that have appeared may be counted and sorted from high to low to obtain a common vocabulary list of the language. For example, the top 5,000 words may be used as the common vocabulary list of the language.

[0044] In some implementations, the number of words in each sentence that do not appear in the common vocabulary can be counted and then divided by the total number of words in the sentence to obtain the word complexity value of the sentence. .

[0045] Then, the lexical complexity values ​​of all sentences in the target article can be summarized to calculate the word level complexity of the target article. In some implementations, the word level complexity of the target article can be calculated according to the following formula: :

[0046] in, is the total number of sentences in the target article; word-level complexity The higher the value, the more difficult the target article is at the lexical level.

[0047] According to a method for determining translation difficulty based on multi-level semantic features provided by the present invention, the sentence-level complexity of the target article is calculated in the following manner: Based on the syntactic analyzer corresponding to the language category, parse and obtain the component syntactic tree of each sentence in the sentence segmentation result; Based on the deep and wide features of the component syntactic tree, the sentence-level complexity of the target article is calculated.

[0048] Specifically, a syntactic parser is a tool for parsing the structure of natural language sentences. It can decompose sentences into smaller syntactic units (such as words, phrases, clauses, etc.) and construct a syntactic tree or dependency tree to describe the syntactic relationship between each word in the sentence.

[0049] In the process of determining the sentence-level complexity, we can first parse the component syntactic tree of each sentence in the sentence segmentation result according to the syntactic analyzer corresponding to the language category of the target article.

[0050] After obtaining the component syntactic tree of each sentence in the sentence segmentation result, the depth feature and width feature of the component syntactic tree can be obtained.

[0051] In some implementations, for each word in the component syntax tree, the following calculations may be performed: (1) The distance from each word to the root node of the syntax tree, and then calculate the average of all distances as the depth feature of the syntax tree .

[0052] (2) The number of child nodes connected to each intermediate node in the syntax tree, and then calculate the average number as the width feature of the syntax tree. .

[0053] Then, the depth and width features of all sentences in the target article can be summarized to calculate the sentence level complexity of the target article. In some implementations, the sentence level complexity of the target article can be calculated according to the following formula: :

[0054] in, is the total number of sentences in the target article; sentence-level complexity The higher the value, the more difficult the article is at the syntactic level.

[0055] According to a method for determining translation difficulty based on multi-level semantic features provided by the present invention, the semantic perplexity of a target article is determined based on sentence segmentation results, specifically comprising: Based on multiple different neural network language models, determine the perplexity of each sentence in the sentence segmentation results, and calculate the average perplexity of each sentence; Based on the average perplexity of each sentence, the semantic perplexity of the target article is determined.

[0056] Specifically, in the process of determining the semantic perplexity, multiple different neural network-based language models can be used. For example, three different neural network-based language models are used. For each sentence in the sentence segmentation result, the perplexity of each sentence under multiple different neural network language models is calculated respectively, and the average value is calculated as the average perplexity of the sentence. .

[0057] Then, the average perplexity of all sentences in the target article can be summarized, and the semantic perplexity of the target article can be determined based on the average perplexity of each sentence. In some implementations, the semantic perplexity of the target article can be calculated according to the following formula: :

[0058] in, is the total number of sentences in the target article; semantic perplexity The higher the value, the more difficult the target article is in terms of sentence comprehension.

[0059] According to a method for determining translation difficulty based on multi-level semantic features provided by the present invention, the translation difficulty value of a target article is determined based on word-level complexity, sentence-level complexity and semantic perplexity, specifically including: Based on the mean and standard deviation of the lexical complexity of each sentence in the sentence segmentation results, as well as the word-level complexity, the word-level complexity weight is obtained; Based on the average and standard deviation of the depth features and width features of each component syntactic tree in the sentence segmentation results, as well as the sentence level complexity, the sentence level complexity weight is obtained; Based on the average and standard deviation of the average perplexity of each sentence in the sentence segmentation results and the semantic perplexity of the target article, the semantic complexity weight is obtained; The translation difficulty value of the target article is determined based on the word-level complexity weight, sentence-level complexity weight and semantic complexity weight.

[0060] Specifically, in the process of calculating the lexical complexity of each sentence, the average lexical complexity of all sentences can also be calculated. With standard deviation , as the word-level complexity coefficient of the entire target article. Then the average value of the lexical complexity can be With standard deviation Combined with the word level complexity, the word level complexity weight is obtained.

[0061] In the process of parsing the component syntactic tree of each sentence, the average of the depth feature and width feature of each component syntactic tree can also be calculated. With standard deviation , as the sentence-level complexity coefficient of the entire target article. Then the average of the depth feature and width feature of each component syntactic tree can be With standard deviation Combined with the sentence level complexity, the sentence level complexity weight is obtained.

[0062] In the process of calculating the average perplexity of each sentence, the average perplexity of all sentences can also be averaged. and standard deviation , as the semantic perplexity feature coefficient of the entire target article. Then the average perplexity of each sentence can be With standard deviation Combined with the semantic perplexity, the semantic complexity weight is obtained.

[0063] Finally, the calculated word-level complexity weight, sentence-level complexity weight, and semantic complexity weight can be added together to jointly determine the translation difficulty value of the target article.

[0064] In some implementations, the translation difficulty value of the target article can be calculated according to the following formula: :

[0065] In the formula, is the word-level complexity of the target article; is the sentence-level complexity of the target article; is the semantic perplexity of the target article.

[0066] The method for determining translation difficulty based on multi-level semantic features provided by the present invention is further explained below through embodiments in specific application scenarios.

[0067] This embodiment includes the following steps: 1. Process the articles to be translated and identify the language

[0068] The articles to be translated in different formats are processed into a unified plain text format, retaining information such as formulas and tables in the articles, using carriage returns to retain the paragraph structure of the articles while dividing the articles into sentences. The language classification model is used to determine the language of the articles.

[0069] 2. Calculate the word-level complexity of the article

[0070] Build a common vocabulary list for multiple languages: For a language, collect and organize articles translated into that language, perform word segmentation and word form restoration, count the frequency of words that have appeared and sort them from high to low, and use the top 5,000 words as the common vocabulary list for that language.

[0071] Calculate vocabulary complexity: According to the language of the article, select the common vocabulary list of that language as a reference. Segment each sentence in the article and count the number of words in each sentence that do not appear in the common vocabulary list, and then divide it by the total number of words in the sentence to obtain the vocabulary complexity of the sentence. Summarize all the sentences in the article The value is used to calculate the vocabulary complexity feature of the article using the following formula, where is the total number of sentences in the article.

[0072]

[0073] The higher the value of this feature, the more difficult the article is at the lexical level. And calculate the average lexical complexity of all sentences in the article With standard deviation , as the word-level complexity coefficient of the entire article.

[0074] 3. Calculate the sentence-level complexity of the article

[0075] According to the language of the article, select the syntax analyzer of the corresponding language to parse the component syntax tree of each sentence. Then, for each word in the component syntax tree, calculate: (1) The distance from each word to the root node of the syntactic tree, and then calculate the average of all distances as the depth feature of the syntactic tree ; (2) The number of child nodes connected to each intermediate node in the syntax tree, and then calculate the average number as the syntax tree width feature .

[0076] Summarize the depth and width features of all sentences in the article and use the following formula to calculate the sentence-level complexity features of the article, where is the total number of sentences in the article.

[0077]

[0078] The higher the value of this feature, the more difficult the article is at the syntactic level.

[0079] Calculate the average of the depth and width features of the syntax tree With standard deviation , as the syntactic complexity coefficient of the entire article.

[0080] 4. Calculate semantic perplexity

[0081] Using three different neural network-based language models, the perplexity of each sentence is calculated under each language model, and the average is taken as the sentence perplexity of the sentence. .

[0082] Summarize the sentence perplexity of all sentences in the article and calculate the semantic perplexity feature of the article using the following formula, where is the total number of sentences in the article.

[0083]

[0084] The higher the value of this feature, the more difficult the article is in terms of sentence comprehension.

[0085] Average the sentence confusion of all sentences in the entire article and standard deviation , as the semantic perplexity feature coefficient of the entire article.

[0086] 5. Calculate the translation difficulty of the article

[0087] Based on the three complexity features and corresponding feature coefficients at the word level and sentence level obtained in steps 2 and 3, the translation difficulty score of the entire article is calculated using the following formula: .

[0088]

[0089] The higher the score, the more difficult it is to translate.

[0090] The following is a description of a device for determining translation difficulty provided by the present invention. The device for determining translation difficulty described below and the method for determining translation difficulty described above can be referred to each other in correspondence.

[0091] Figure 2 A schematic diagram of the structure of the translation difficulty determination device based on multi-level semantic features provided by the present invention, such as Figure 2 As shown, the device includes the following modules: The processing module 200 is used to perform sentence segmentation processing on the target article to obtain the sentence segmentation result of the target article; A calculation module 210 is used to determine the word level complexity, sentence level complexity and semantic perplexity of the target article based on the sentence segmentation results; The determination module 220 is used to determine the translation difficulty value of the target article based on the word level complexity, sentence level complexity and semantic perplexity.

[0092] According to a translation difficulty determination device based on multi-level semantic features provided by the present invention, based on the sentence segmentation results, the word-level complexity and sentence-level complexity of the target article are determined respectively, specifically including: Determine the language category of the target article; Based on the sentence segmentation results and language category, the word-level complexity and sentence-level complexity of the target article are calculated.

[0093] According to a translation difficulty determination device based on multi-level semantic features provided by the present invention, the word level complexity of the target article is calculated in the following manner: Perform word segmentation processing on the sentence segmentation result to obtain the word segmentation result; Based on the common vocabulary list and word segmentation results corresponding to the language category, calculate the lexical complexity of each sentence in the sentence segmentation results; Based on the lexical complexity of each sentence, the word-level complexity of the target article was calculated; Among them, the common vocabulary list is sorted from high to low according to the frequency of vocabulary in multiple articles of the language category.

[0094] According to a translation difficulty determination device based on multi-level semantic features provided by the present invention, the sentence level complexity of the target article is calculated in the following manner: Based on the syntactic analyzer corresponding to the language category, parse and obtain the component syntactic tree of each sentence in the sentence segmentation result; Based on the deep and wide features of the component syntactic tree, the sentence-level complexity of the target article is calculated.

[0095] According to the present invention, a translation difficulty determination device based on multi-level semantic features is provided, which determines the semantic perplexity of a target article based on sentence segmentation results, specifically comprising: Based on multiple different neural network language models, determine the perplexity of each sentence in the sentence segmentation results, and calculate the average perplexity of each sentence; Based on the average perplexity of each sentence, the semantic perplexity of the target article is determined.

[0096] According to a translation difficulty determination device based on multi-level semantic features provided by the present invention, based on word level complexity, sentence level complexity and semantic perplexity, the translation difficulty value of the target article is determined, specifically including: Based on the mean and standard deviation of the lexical complexity of each sentence in the sentence segmentation results, as well as the word-level complexity, the word-level complexity weight is obtained; Based on the average and standard deviation of the depth features and width features of each component syntactic tree in the sentence segmentation results, as well as the sentence level complexity, the sentence level complexity weight is obtained; Based on the average and standard deviation of the average perplexity of each sentence in the sentence segmentation results and the semantic perplexity of the target article, the semantic complexity weight is obtained; The translation difficulty value of the target article is determined based on the word-level complexity weight, sentence-level complexity weight and semantic complexity weight.

[0097] Figure 3 A schematic diagram of the structure of an electronic device provided by the present invention, such as Figure 3 As shown, the electronic device may include: a processor 310, a communications interface 320, a memory 330 and a communication bus 340, wherein the processor 310, the communications interface 320 and the memory 330 communicate with each other via the communication bus 340. The processor 310 may call the logic instructions in the memory 330 to execute the translation difficulty determination method based on multi-level semantic features provided by the above methods, and the method includes: Perform sentence segmentation processing on the target article to obtain the sentence segmentation result of the target article; Based on the sentence segmentation results, the word-level complexity, sentence-level complexity, and semantic perplexity of the target article are determined respectively; Determine the translation difficulty value of the target article based on word-level complexity, sentence-level complexity and semantic perplexity.

[0098] In addition, the logic instructions in the above-mentioned memory 330 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0099] On the other hand, the present invention further provides a computer program product, the computer program product comprising a computer program, the computer program can be stored in a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the method for determining translation difficulty based on multi-level semantic features provided by the above methods, the method comprising: Perform sentence segmentation processing on the target article to obtain the sentence segmentation result of the target article; Based on the sentence segmentation results, the word-level complexity, sentence-level complexity, and semantic perplexity of the target article are determined respectively; Determine the translation difficulty value of the target article based on word-level complexity, sentence-level complexity and semantic perplexity.

[0100] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform the method for determining translation difficulty based on multi-level semantic features provided by the above methods, the method comprising: Perform sentence segmentation processing on the target article to obtain the sentence segmentation result of the target article; Based on the sentence segmentation results, the word-level complexity, sentence-level complexity, and semantic perplexity of the target article are determined respectively; Determine the translation difficulty value of the target article based on word-level complexity, sentence-level complexity and semantic perplexity.

[0101] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0102] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0103] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for determining translation difficulty based on multi-level semantic features, characterized in that: include: Perform sentence segmentation processing on the target article to obtain a sentence segmentation result of the target article; Based on the sentence segmentation results, respectively determining the word-level complexity, sentence-level complexity, and semantic perplexity of the target article; A translation difficulty value of the target article is determined based on the word level complexity, the sentence level complexity and the semantic perplexity.

2. The method for determining translation difficulty based on multi-level semantic features according to claim 1, characterized in that: Based on the sentence segmentation results, the word-level complexity and sentence-level complexity of the target article are determined respectively, including: Determining the language category of the target article; Based on the sentence segmentation results and the language category, the word-level complexity and sentence-level complexity of the target article are calculated.

3. The method for determining translation difficulty based on multi-level semantic features according to claim 2, characterized in that: The word-level complexity of the target article is calculated as follows: Performing word segmentation processing on the sentence segmentation result to obtain a word segmentation result; Calculating the lexical complexity of each sentence in the sentence segmentation result based on the common vocabulary list corresponding to the language category and the word segmentation result; Calculate the word-level complexity of the target article based on the lexical complexity of each sentence; The common vocabulary list is obtained by sorting the vocabulary frequencies in multiple articles of the language category from high to low.

4. The method for determining translation difficulty based on multi-level semantic features according to claim 2, characterized in that: The sentence-level complexity of the target article is calculated as follows: Based on the syntactic analyzer corresponding to the language category, parse and obtain the component syntactic tree of each sentence in the sentence segmentation result; Based on the depth features and width features of the component syntax tree, the sentence level complexity of the target article is calculated.

5. The method for determining translation difficulty based on multi-level semantic features according to claim 1, characterized in that: Based on the sentence segmentation results, the semantic perplexity of the target article is determined, specifically including: Based on a plurality of different neural network language models, determining the perplexity of each sentence in the sentence segmentation result, and obtaining an average perplexity of each sentence; Based on the average perplexity of each sentence, the semantic perplexity of the target article is determined.

6. The method for determining translation difficulty based on multi-level semantic features according to any one of claims 3 to 5, characterized in that: Determining the translation difficulty value of the target article based on the word level complexity, the sentence level complexity and the semantic perplexity, specifically includes: Based on the average value and standard deviation of the lexical complexity of each sentence in the sentence segmentation result and the word level complexity, a word level complexity weight is obtained; Based on the average value and standard deviation of the depth feature and width feature of each component syntactic tree in the sentence segmentation result, and the sentence level complexity, a sentence level complexity weight is obtained; Obtaining a semantic complexity weight based on the average value and standard deviation of the average perplexity of each sentence in the sentence segmentation results and the semantic perplexity of the target article; The translation difficulty value of the target article is determined based on the word-level complexity weight, the sentence-level complexity weight, and the semantic complexity weight.

7. A translation difficulty determination device based on multi-level semantic features, characterized in that: include: A processing module, used for performing sentence segmentation processing on the target article to obtain the sentence segmentation result of the target article; A calculation module, used to determine the word-level complexity, sentence-level complexity and semantic perplexity of the target article based on the sentence segmentation results; A determination module is used to determine a translation difficulty value of the target article based on the word level complexity, the sentence level complexity and the semantic perplexity.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method for determining translation difficulty based on multi-level semantic features according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for determining translation difficulty based on multi-level semantic features as claimed in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for determining translation difficulty based on multi-level semantic features as claimed in any one of claims 1 to 6 is implemented.

Images