Cross-culture translation method and system based on context perception fusion

Abstract

The invention relates to the technical field of cross-culture translation, in particular to a cross-culture translation method and system based on context perception fusion, and the method comprises the steps: recognizing a text data set and an image data set, carrying out the paragraph division of the text data set, obtaining a text paragraph set, recognizing an image description text of image data, carrying out the word segmentation of the image description text, and obtaining a text paragraph set; calculating the matching similarity of the text paragraphs and the image description phrases, judging whether the matching similarity is greater than a standard matching similarity or not, if so, recording the matching similarity as a target matching similarity to obtain a target matching similarity set, and obtaining a plurality of associated text paragraph sets according to the target matching similarity set; and translating the associated text paragraph set to obtain an associated translated text, extracting an unassociated text paragraph set from the text paragraph set, and translating the unassociated text paragraphs to obtain an unassociated translated text. According to the invention, the accuracy and efficiency of cross-culture translation can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of cross-cultural translation, and in particular to a cross-cultural translation method and system based on context-aware fusion. Background Art

[0002] In a globalized working environment, cross-cultural translation can help companies and employees with different language backgrounds communicate effectively. At the same time, cross-cultural translation can convey the values, customs and beliefs of different cultures, and help enhance understanding and respect for other cultures. Therefore, in actual work applications, it is of great significance to achieve fast and accurate cross-cultural translation.

[0003] The current cross-cultural translation method is mainly achieved through simple machine translation. This method may have the problem of semantic analysis errors, which makes the translation result inconsistent with the meaning of the original article, reducing the accuracy of cross-cultural translation. At the same time, contextual errors require a lot of manual correction, which in turn reduces the efficiency of cross-cultural translation. Summary of the invention

[0004] The present invention provides a cross-cultural translation method and system based on context-aware fusion, the main purpose of which is to improve the accuracy and efficiency of cross-cultural translation.

[0005] To achieve the above object, the present invention provides a cross-cultural translation method based on context-aware fusion, comprising:

[0006] Receiving an article to be translated, and starting a data translation platform based on the article to be translated, wherein the article to be translated includes text data and image data, and the text data is separated by paragraphs, and the image data includes description text of related images, and the data translation platform includes intelligent software or a translation website;

[0007] Using the data translation platform, respectively identifying a text dataset and an image dataset in the article to be translated, dividing the text dataset into paragraphs, and obtaining a text paragraph set;

[0008] Extracting image data in the image data set in sequence, identifying image description text of the image data, and segmenting the image description text to obtain image description phrases;

[0009] Extracting text paragraphs from the text paragraph set in sequence, determining similarity between the text paragraphs and the image description phrases, and obtaining matching similarity;

[0010] Determining whether the matching similarity is greater than a preset standard matching similarity;

[0011] If the matching similarity is greater than the standard matching similarity, the matching similarity is recorded as the target matching similarity;

[0012] Summarizing the target matching similarities to obtain a target matching similarity set, identifying an associated similarity group corresponding to the image data in the target matching similarity set, and identifying a plurality of associated text paragraphs corresponding to the associated similarity group;

[0013] Summarizing the multiple associated text paragraphs to obtain an associated text paragraph set, wherein each associated text paragraph in the associated text paragraph set corresponds to one or more image data, and there is no duplicate text paragraph in the associated text paragraph set;

[0014] By using the image data set, the related text paragraph set is subjected to image context fusion translation to obtain related translated texts; based on the related text paragraph set, an unrelated text paragraph set is extracted from the text paragraph set; the unrelated text paragraph set is subjected to text translation to obtain unrelated translated texts; and based on the related translated texts and the unrelated translated texts, cross-cultural translation based on context-aware fusion is completed.

[0015] Optionally, segmenting the image description text to obtain image description phrases includes:

[0016] Constructing a word segmentation dataset based on the text paragraph set, wherein the word segmentation dataset includes all words in the text paragraph set and their corresponding word frequencies;

[0017] Segmenting the image description text to obtain original description phrases, wherein the original description phrases do not contain repeated original description words;

[0018] Extracting original description words from the original description word group in sequence, and counting the original description frequency of the original description words in the image description text and the comparative description frequency in the word segmentation data set respectively;

[0019] Identify the category to be translated of the article to be translated, obtain a text data set of the category to be translated, and count the overall description frequency of the original description words in the text data set;

[0020] Calculate the description key value based on the original description frequency, the comparative description frequency and the overall description frequency;

[0021] Determine whether the description key value is greater than a preset standard key value;

[0022] If the description key value is greater than the standard key value, the original description word is recorded as the target description word, and the target description words are summarized to obtain an image description phrase.

[0023] Optionally, calculating the description key value according to the original description frequency, the comparative description frequency and the overall description frequency includes:

[0024] Set the standard description frequency, and calculate the description key value using the following formula based on the standard description frequency, original description frequency, comparative description frequency, and overall description frequency:

[0025]

[0026] Among them, G represents the description key value, f y represents the original description frequency, e represents the natural constant, f z represents the overall description frequency, f d represents the comparison description frequency, and f0 represents the standard description frequency.

[0027] Optionally, constructing a word segmentation dataset based on the text paragraph set includes:

[0028] Performing word segmentation on each text paragraph in the text paragraph set to obtain a plurality of text paragraph word segmentation groups, wherein the number of the text paragraph word segmentation groups is equal to the number of text paragraphs in the text paragraph set;

[0029] Merge all the text paragraph segmentations in multiple text paragraph segmentation groups to obtain a text paragraph segmentation set, remove duplicate words from the text paragraph segmentation set, and obtain a single paragraph segmentation set;

[0030] Extracting single paragraph segmentation words in the single paragraph segmentation word set in turn, counting the frequency of occurrence of the single paragraph segmentation words in the single paragraph segmentation word set, performing key-value pairing of the paragraph occurrence frequency and the single paragraph segmentation word, and obtaining a single paragraph segmentation word group;

[0031] The single paragraph word segmentation groups are aggregated to obtain a word segmentation dataset.

[0032] Optionally, the similarity determination of the text paragraph and the image description phrase to obtain the matching similarity includes:

[0033] Recognize the period symbol in the text paragraph, divide the text paragraph into sentences based on the period symbol to obtain paragraph short sentence groups, and extract paragraph short sentences in the paragraph short sentence groups in sequence;

[0034] Segment the short sentences in the paragraph to obtain short sentence segmentation phrases, identify the overlapping words of the short sentence segmentation phrases and the image description phrases to obtain short sentence overlapping phrases, wherein there are no repeated words in the short sentence overlapping phrases, and the short sentence overlapping phrases may be an empty set;

[0035] In the short sentence word groups, a short sentence overlap frequency array of the short sentence overlapped word groups is counted, wherein the short sentence overlap times correspond to the short sentence overlapped words one by one;

[0036] According to the short sentence overlapping subarray and the image description phrase, constructing a phrase vector for the short sentence word segmentation group to obtain a short sentence word segmentation vector;

[0037] Summarize the short sentence word segmentation vectors to obtain a short sentence word segmentation vector set;

[0038] The image description phrase is vectorized to obtain the description phrase vector, where the description phrase vector is expressed as:

[0039]

[0040] in, represents the description word vector, K1 represents the description key value of the first image description word, K n represents the description key value of the nth image description word, where n represents the number of image description words in the image description phrase;

[0041] According to the description phrase vector and the short sentence word vector set, the matching similarity is calculated using the following formula:

[0042]

[0043] Among them, S p represents the matching similarity, l represents the number of short sentence segmentation vectors in the short sentence segmentation vector set, represents the a-th sentence segmentation vector, and || represents the norm symbol of the vector.

[0044] Optionally, constructing a phrase vector for the short sentence word group according to the short sentence overlapping subarray and the image description phrase to obtain the short sentence word segmentation vector includes:

[0045] Extracting image description words from the image description word group in sequence, and searching for the description overlap times corresponding to the image description words in the short sentence overlap times array, wherein the description overlap times may be 0;

[0046] Summarizing the description overlap times according to the arrangement order of the image description words in the image description phrase group to obtain a description overlap times array;

[0047] Based on the description overlap array, construct the short sentence segmentation vector, where the short sentence segmentation vector is expressed as:

[0048]

[0049] in, represents the short sentence segmentation vector, P1 represents the number of overlapping times of the first description, P n Indicates the number of times the nth description overlaps.

[0050] Optionally, identifying the associated similarity group corresponding to the image data in the target matching similarity set includes:

[0051] Identify a target matching paragraph set corresponding to the target matching similarity set;

[0052] Extracting a nearest matching paragraph group that is closest to the image data from the target matching paragraph set, wherein the nearest matching paragraph group contains one or two nearest matching paragraphs;

[0053] Sequentially extracting the nearest matching paragraphs from the nearest matching paragraph group, and identifying the number of associated lines between the nearest matching paragraphs and the image data;

[0054] Determine whether the number of associated rows is less than a preset maximum number of rows;

[0055] If the number of associated lines is not less than the maximum number of lines, the nearest similarity of the nearest matching paragraph is obtained, and the relevance of the nearest matching paragraph is evaluated according to the nearest similarity and the number of associated lines to obtain a relevance factor, wherein the relevance factor is expressed as:

[0056]

[0057] Among them, H represents the correlation factor, S z represents the nearest similarity, and L represents the number of associated rows;

[0058] Determining whether the correlation factor is greater than a preset standard correlation factor;

[0059] If the correlation factor is greater than the standard correlation factor, the nearest matching paragraph is recorded as a valid matching paragraph, and the valid matching paragraph is removed from the target matching paragraph set to obtain a correlated matching paragraph set;

[0060] Using the associated matching paragraph set and the valid matching paragraph to update the target matching paragraph set and the image data respectively, and returning to the step of extracting the closest matching paragraph group closest to the image data in the target matching paragraph set until there is no closest matching paragraph group in the target matching paragraph set;

[0061] If the number of associated lines is not less than the maximum number of lines, the nearest matching paragraph is recorded as a valid matching paragraph, and the valid matching paragraph is removed from the target matching paragraph set to obtain an associated matching paragraph set;

[0062] Using the associated matching paragraph set and the valid matching paragraph to update the target matching paragraph set and the image data respectively, and returning to the step of extracting the closest matching paragraph group closest to the image data in the target matching paragraph set until there is no closest matching paragraph group in the target matching paragraph set;

[0063] The valid matching paragraphs are summarized to obtain an associated similarity group.

[0064] Optionally, the using of the image data set to perform image context fusion translation on the associated text paragraph set to obtain associated translated text includes:

[0065] Extracting related text paragraphs in sequence from the related text paragraph set, identifying related image groups corresponding to the related text paragraphs in the image data set, wherein the related image group includes one or more related images;

[0066] Extracting related images from the related image group in sequence, and using a pre-built deep learning network to perform object recognition on the related images to obtain an image recognition vocabulary group, wherein the deep learning network includes: a convolutional neural network, and the language of the image recognition vocabulary group is the target translation language;

[0067] Obtain a similar recognition vocabulary group set of the image recognition vocabulary group, wherein the similar recognition vocabulary groups in the similar recognition vocabulary group set correspond one-to-one to the image recognition vocabulary in the image recognition vocabulary group;

[0068] Translating the associated text paragraphs to obtain an original translation text, wherein the language category of the original translation text is the target translation language;

[0069] Sequentially extracting similar recognition vocabulary groups from the similar recognition vocabulary group set, identifying a source recognition vocabulary and a plurality of branch recognition vocabulary of the similar recognition vocabulary group;

[0070] Based on the multiple branch recognition words, multiple branch related words are searched in the original translation text, and independent related words are determined from the multiple branch related words according to the preset related word frequency, wherein the frequency of the independent related words appearing in the original translation text is less than the related word frequency;

[0071] Calculating the semantic similarity between the independent associated words and the source identification words, and recording the independent associated words whose semantic similarity is greater than the preset standard semantic similarity as replaceable associated words;

[0072] The source recognition vocabulary is used to replace the replaceable associated vocabulary in the original translation text to obtain the associated translation text.

[0073] Optionally, the obtaining of a set of similar recognition vocabulary groups of the image recognition vocabulary group includes:

[0074] Acquire a target vocabulary library of the target translation language;

[0075] Extracting image recognition vocabulary in the image recognition vocabulary group in sequence, and identifying a plurality of similar recognition vocabulary of the image recognition vocabulary in the target vocabulary library;

[0076] Combining the image recognition vocabulary and a plurality of similar recognition vocabulary to obtain a similar recognition vocabulary group;

[0077] The same type of recognition vocabulary groups are summarized to obtain a same type of recognition vocabulary group set.

[0078] To achieve the above object, the present invention also provides a cross-cultural translation system based on context-aware fusion, comprising:

[0079] A text paragraph division module is used to receive an article to be translated, and start a data translation platform based on the article to be translated, wherein the article to be translated includes text data and image data, and the text data is divided into paragraphs, and the image data includes description text of related images, and the data translation platform includes intelligent software or a translation website. The data translation platform is used to identify a text data set and an image data set in the article to be translated, and the text data set is divided into paragraphs to obtain a text paragraph set;

[0080] A phrase similarity evaluation module is used to sequentially extract image data from the image data set, identify image description text of the image data, segment the image description text to obtain image description phrases, sequentially extract text paragraphs from the text paragraph set, perform similarity determination on the text paragraphs and the image description phrases, and obtain matching similarity;

[0081] An image-text matching module, used to determine whether the matching similarity is greater than a preset standard matching similarity, if the matching similarity is greater than the standard matching similarity, the matching similarity is recorded as a target matching similarity, the target matching similarities are summarized to obtain a target matching similarity set, an associated similarity group corresponding to the image data is identified in the target matching similarity set, and a plurality of associated text paragraphs corresponding to the associated similarity group are identified;

[0082] The associated text translation module is used to summarize the multiple associated text paragraphs to obtain an associated text paragraph set, wherein each associated text paragraph in the associated text paragraph set corresponds to one or more image data, and there is no repeated text paragraph in the associated text paragraph set, and the associated text paragraph set is subjected to image context fusion translation using the image data set to obtain associated translation text, and based on the associated text paragraph set, an unrelated text paragraph set is extracted from the text paragraph set, and text translation is performed on the unrelated text paragraph set to obtain an unrelated translation text.

[0083] In order to solve the above problem, the present invention further provides an electronic device, the electronic device comprising:

[0084] a memory storing at least one instruction; and

[0085] The processor executes the instructions stored in the memory to implement the above-mentioned cross-cultural translation method based on context-aware fusion.

[0086] In order to solve the above problem, the present invention also provides a computer-readable storage medium, in which at least one instruction is stored. The at least one instruction is executed by a processor in an electronic device to implement the above-mentioned cross-cultural translation method based on context-aware fusion.

[0087] The present invention solves the problem described in the background technology. First, a text data set and an image data set are respectively identified in the article to be translated. This process processes the text and image content in the article separately. Different processing strategies can be adopted for different types of data, thereby improving the accuracy of translation. Then, the image description text of the image data is identified to obtain the image description phrase. Extracting the image data and the description text is helpful to convert the image content into processable information. Meanwhile, the image description phrase can be associated with different paragraphs in the article, thereby improving the accuracy of text paragraph translation. Moreover, due to the intuitiveness of the image, the translation result is more in line with the context of the original article. By calculating the matching similarity of the text paragraph and the image description phrase, and judging whether the matching similarity is greater than the standard matching similarity, the correlation degree between the text paragraph and the image data is determined, so that images and texts with a high correlation degree can be integrated with each other, thereby improving the accuracy of the final translation result and also improving the efficiency of translation. The image data set is used to perform image context fusion translation on the associated text paragraph set to obtain the associated translation text. In this step, the image data and the text paragraph are contextually integrated, so that the translated associated translation text is more in line with the context, thereby improving the accuracy of cross-cultural translation. Meanwhile, the automated translation process also improves the efficiency of translation. Therefore, the present invention can improve the accuracy and efficiency of cross-cultural translation. BRIEF DESCRIPTION OF THE DRAWINGS

[0088] Figure 1 A schematic diagram of a process of a cross-cultural translation method based on context-aware fusion provided by an embodiment of the present invention;

[0089] Figure 2 A functional module diagram of a cross-cultural translation system based on context-aware fusion provided by an embodiment of the present invention;

[0090] Figure 3 A schematic diagram of the structure of an electronic device for implementing the cross-cultural translation method based on context-aware fusion provided by an embodiment of the present invention.

[0091] Description of reference numerals:

[0092] 1. Electronic device; 10. Processor; 11. Memory; 12. Bus.

[0093] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0094] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0095] The embodiment of the present application provides a cross-cultural translation method based on context-aware fusion. The execution subject of the cross-cultural translation method based on context-aware fusion includes but is not limited to at least one of the electronic devices such as a server and a terminal that can be configured to execute the method provided by the embodiment of the present application. In other words, the cross-cultural translation method based on context-aware fusion can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc.

[0096] Reference Figure 1 FIG. 1 is a flow chart of a cross-cultural translation method based on context-aware fusion provided by an embodiment of the present invention. In this embodiment, the cross-cultural translation method based on context-aware fusion includes:

[0097] S1. Receive an article to be translated from a target user, and start a data translation platform based on the article to be translated, wherein the article to be translated includes text data and image data, and the text data is separated by paragraphs, and the image data includes related image description text, and the data translation platform includes intelligent software or a translation website.

[0098] It is clear that the article to be translated refers to text data that needs to be translated across languages. The text data in the article to be translated is segmented according to paragraphs, and an image is inserted in a paragraph or between two paragraphs, and a descriptive text about the image is contained below the image.

[0099] For example, Xiao Zhang needs to translate an English article into Chinese. The English article contains text data and image data, so Xiao Zhang sends the English article to a translation website. The translation website completes the activation after receiving the English article.

[0100] S2. Using the data translation platform, respectively identifying a text dataset and an image dataset in the article to be translated, dividing the text dataset into paragraphs, and obtaining a text paragraph set.

[0101] It can be understood that the text dataset refers to the set of text data in the article to be translated, and the image dataset refers to the set of image data in the article to be translated. It should be noted that the descriptive text of the above-mentioned image data should be part of the image data, not the text dataset, and the text paragraph set refers to the text dataset after paragraph division.

[0102] For example, an English article contains 5 paragraphs, namely paragraph 1, paragraph 2, paragraph 3, paragraph 4 and paragraph 5, wherein each paragraph is separated by a line break symbol. The English article also includes 3 images, which are interspersed in these 5 paragraphs. Therefore, the text dataset of the English article consists of 5 paragraphs, and the image dataset consists of 3 images and their descriptive texts. Among them, the text paragraph set divided by the text dataset is: {paragraph 1, paragraph 2, paragraph 3, paragraph 4 and paragraph 5}.

[0103] S3. Extracting image data in the image data set in sequence, identifying image description text of the image data, and segmenting the image description text to obtain image description phrases.

[0104] It should be explained that the image description text refers to text information used to describe image data, and the image description phrase refers to the image description text after word segmentation, wherein the image description phrase includes all the words in the image description text.

[0105] Specifically, the image description text is segmented to obtain image description phrases, including:

[0106] Constructing a word segmentation dataset based on the text paragraph set, wherein the word segmentation dataset includes all words in the text paragraph set and their corresponding word frequencies;

[0107] Segmenting the image description text to obtain original description phrases, wherein the original description phrases do not contain repeated original description words;

[0108] Extracting original description words from the original description word group in sequence, and counting the original description frequency of the original description words in the image description text and the comparative description frequency in the word segmentation data set respectively;

[0109] Identify the category to be translated of the article to be translated, obtain a text data set of the category to be translated, and count the overall description frequency of the original description words in the text data set;

[0110] Calculate the description key value based on the original description frequency, the comparative description frequency and the overall description frequency;

[0111] Determine whether the description key value is greater than a preset standard key value;

[0112] If the description key value is greater than the standard key value, the original description word is recorded as the target description word, and the target description words are summarized to obtain an image description phrase.

[0113] It is understandable that the word segmentation of the image description text can be performed by different word segmentation tools. For different language types, the word segmentation tools are different. For example, in Python language, Jieba database can be used to segment Chinese sentences, and NLTK database can be used to segment English sentences. The word segmentation data set refers to the set of all words in the text paragraph set.

[0114] It should be explained that the original description phrase refers to a set of all words in the image description text, the original description frequency refers to the number of times the original description word appears in the image description text, the comparative description frequency refers to the frequency of the original description word appearing in the entire text paragraph set, and the comparative description frequency can be quickly queried in the word segmentation data set, the category to be translated refers to the language category of the article to be translated, such as: Chinese, English, etc., the text data set refers to a large text database corresponding to the category to be translated, for example: a word set composed of all words in multiple articles to be translated in the same language category, and the construction process of the text data set is the same as the construction process of the word segmentation data set below. The overall description frequency refers to the number of times the original description word appears in the text data set, and the description key value refers to the value used to indicate the importance of the original description word for the image data description.

[0115] For example, for an image about grape structure, its image description text is as follows: Figure 1 The image description phrases of the grape double cross V-shaped frame pattern are: {Figure, 1, grapes, double cross, V-shaped, frame, pattern, figure}, among which the words "grapes, double cross, V-shaped" are highly descriptive for the image description text, so their description keywords are relatively high.

[0116] Specifically, the calculation of the description key value according to the original description frequency, the comparative description frequency and the overall description frequency includes:

[0117] Set the standard description frequency, and calculate the description key value using the following formula based on the standard description frequency, original description frequency, comparative description frequency, and overall description frequency:

[0118]

[0119] Among them, G represents the description key value, f y represents the original description frequency, e represents the natural constant, f z represents the overall description frequency, f d represents the comparison description frequency, and f0 represents the standard description frequency.

[0120] It is clear that the standard description frequency refers to an artificially set constant.

[0121] In detail, constructing a word segmentation dataset based on the text paragraph set includes:

[0122] Performing word segmentation on each text paragraph in the text paragraph set to obtain a plurality of text paragraph word segmentation groups, wherein the number of the text paragraph word segmentation groups is equal to the number of text paragraphs in the text paragraph set;

[0123] Merge all the text paragraph segmentations in multiple text paragraph segmentation groups to obtain a text paragraph segmentation set, remove duplicate words from the text paragraph segmentation set, and obtain a single paragraph segmentation set;

[0124] Extracting single paragraph segmentation words in the single paragraph segmentation word set in turn, counting the frequency of occurrence of the single paragraph segmentation words in the single paragraph segmentation word set, performing key-value pairing of the paragraph occurrence frequency and the single paragraph segmentation word, and obtaining a single paragraph segmentation word group;

[0125] The single paragraph word segmentation groups are aggregated to obtain a word segmentation dataset.

[0126] It can be understood that the text paragraph word group refers to the word combination of the text paragraph, the text paragraph word set refers to the set of all text paragraph word segments, the single paragraph word set refers to the text paragraph word set after removing repeated words, the paragraph occurrence frequency refers to the number of times a single paragraph word appears in the single paragraph word set, and the single paragraph word group refers to the combination of the paragraph occurrence frequency and the single paragraph word.

[0127] S4, extracting text paragraphs in sequence from the text paragraph set, performing similarity determination on the text paragraphs and the image description phrases, and obtaining matching similarity.

[0128] It can be understood that the matching similarity refers to a numerical value used to represent the similarity between the image description phrase and the text paragraph.

[0129] Specifically, the similarity determination of the text paragraphs and the image description phrases to obtain the matching similarity includes:

[0130] Recognize the period symbol in the text paragraph, divide the text paragraph into sentences based on the period symbol to obtain paragraph short sentence groups, and extract paragraph short sentences in the paragraph short sentence groups in sequence;

[0131] Segment the short sentences in the paragraph to obtain short sentence segmentation phrases, identify the overlapping words of the short sentence segmentation phrases and the image description phrases to obtain short sentence overlapping phrases, wherein there are no repeated words in the short sentence overlapping phrases, and the short sentence overlapping phrases may be an empty set;

[0132] In the short sentence word groups, a short sentence overlap frequency array of the short sentence overlapped word groups is counted, wherein the short sentence overlap times correspond to the short sentence overlapped words one by one;

[0133] According to the short sentence overlapping subarray and the image description phrase, constructing a phrase vector for the short sentence word segmentation group to obtain a short sentence word segmentation vector;

[0134] Summarize the short sentence word segmentation vectors to obtain a short sentence word segmentation vector set;

[0135] The image description phrase is vectorized to obtain the description phrase vector, where the description phrase vector is expressed as:

[0136]

[0137] in, represents the description word vector, K1 represents the description key value of the first image description word, K n represents the description key value of the nth image description word, where n represents the number of image description words in the image description phrase;

[0138] According to the description phrase vector and the short sentence word vector set, the matching similarity is calculated using the following formula:

[0139]

[0140] Among them, S p represents the matching similarity, l represents the number of short sentence segmentation vectors in the short sentence segmentation vector set, represents the a-th sentence segmentation vector, and || represents the norm symbol of the vector.

[0141] It can be understood that the paragraph sentence group refers to the text paragraph after division, the sentence word group refers to the combination of all words in the paragraph sentence, wherein the step of word segmentation of the paragraph sentence is the same as the step of word segmentation of the image description text, the sentence overlapping word group refers to the overlapping part of the sentence word group and the image description word group, and the sentence overlapping times refers to the number of times the sentence overlapping words appear in the sentence word group.

[0142] For example, a short sentence word group is: {A, A, B, C, D, E}, and an image description word group is: {A, B, C, F, G}, then their short sentence overlapping word group is: {A, B, C}, and the number of times A, B, C appear in the short sentence word group is: 2, 1, 1 respectively, so the short sentence overlapping times group is: {2,1,1}.

[0143] It can be understood that the short sentence word segmentation vector points to a short sentence word segmentation group in the form of a quantity, and the description word group vector points to an image description word group in the form of a quantity.

[0144] In detail, constructing a phrase vector for the short sentence word group according to the short sentence overlapping subarray and the image description phrase to obtain the short sentence word segmentation vector includes:

[0145] Extracting image description words from the image description word group in sequence, and searching for the description overlap times corresponding to the image description words in the short sentence overlap times array, wherein the description overlap times may be 0;

[0146] Summarizing the description overlap times according to the arrangement order of the image description words in the image description phrase group to obtain a description overlap times array;

[0147] Based on the description overlap array, construct the short sentence segmentation vector, where the short sentence segmentation vector is expressed as:

[0148]

[0149] in, represents the short sentence segmentation vector, P1 represents the number of overlapping times of the first description, P n Indicates the number of times the nth description overlaps.

[0150] It can be understood that the description overlap times refers to the short sentence overlap times corresponding to the image description word in the short sentence overlap times array. When the image description word is not displayed in the short sentence overlap times, the description overlap times is 0.

[0151] For example, a short sentence overlap array is {2, 1, 1}, and its corresponding short sentence overlap phrase is: {A, B, C}, and a certain image description phrase is: {A, B, C, F, G}, and the short sentence overlap times corresponding to A, B, C, F, G in the short sentence overlap array are: 2, 1, 1, 0, 0, respectively, so the description overlap array is {2, 1, 1, 0, 0}, and then the short sentence word segmentation vector is expressed as:

[0152] S5. Determine whether the matching similarity is greater than a preset standard matching similarity.

[0153] It should be explained that the standard matching similarity refers to an artificially set similarity constant, which is used to determine the above matching similarity. When the matching similarity is greater than the standard matching similarity, it means that the image data has a greater correlation with this paragraph of text, and this paragraph of text can be translated based on the image data, thereby realizing cross-cultural translation based on context-aware fusion.

[0154] S6. If the matching similarity is greater than the standard matching similarity, the matching similarity is recorded as the target matching similarity.

[0155] It is understandable that when the matching similarity is greater than the standard matching similarity, in order to distinguish the matching similarity at this time from other matching similarities that are not greater than the standard matching similarity, the matching similarity at this time needs to be recorded as the target matching similarity.

[0156] S7. Summarize the target matching similarities to obtain a target matching similarity set, identify an associated similarity group corresponding to the image data in the target matching similarity set, and identify multiple associated text paragraphs corresponding to the associated similarity group.

[0157] It can be understood that the associated similarity group refers to the target matching similarity that has a strong correlation with the image data in the article position. Although the target matching similarity has a strong correlation with the image data in content, if the text paragraph corresponding to the target matching similarity is far away from the image data in the article, the relationship between the two is not strong. At this time, if the image data continues to be used as a translation reference for the text paragraph, it will cause a large content error. For example: the matching similarity between image A and text paragraph B is high, but text paragraph B is six paragraphs away from image A, and the distance between the two is too large. At this time, when translating text paragraph B according to image A, it is necessary to adjust the proportion of image data A referenced.

[0158] It can be understood that the associated text paragraph refers to the text paragraph corresponding to the associated similarity group.

[0159] In detail, the identifying the associated similarity group corresponding to the image data in the target matching similarity set includes:

[0160] Identify a target matching paragraph set corresponding to the target matching similarity set;

[0161] Extracting a nearest matching paragraph group that is closest to the image data from the target matching paragraph set, wherein the nearest matching paragraph group contains one or two nearest matching paragraphs;

[0162] Sequentially extracting the nearest matching paragraphs from the nearest matching paragraph group, and identifying the number of associated lines between the nearest matching paragraphs and the image data;

[0163] Determine whether the number of associated rows is less than a preset maximum number of rows;

[0164] If the number of associated lines is not less than the maximum number of lines, the nearest similarity of the nearest matching paragraph is obtained, and the relevance of the nearest matching paragraph is evaluated according to the nearest similarity and the number of associated lines to obtain a relevance factor, wherein the relevance factor is expressed as:

[0165]

[0166] Among them, H represents the correlation factor, S z represents the nearest similarity, and L represents the number of associated rows;

[0167] Determining whether the correlation factor is greater than a preset standard correlation factor;

[0168] If the correlation factor is greater than the standard correlation factor, the nearest matching paragraph is recorded as a valid matching paragraph, and the valid matching paragraph is removed from the target matching paragraph set to obtain a correlated matching paragraph set;

[0169] Using the associated matching paragraph set and the valid matching paragraph to update the target matching paragraph set and the image data respectively, and returning to the step of extracting the closest matching paragraph group closest to the image data in the target matching paragraph set until there is no closest matching paragraph group in the target matching paragraph set;

[0170] If the number of associated lines is not less than the maximum number of lines, the nearest matching paragraph is recorded as a valid matching paragraph, and the valid matching paragraph is removed from the target matching paragraph set to obtain an associated matching paragraph set;

[0171] Using the associated matching paragraph set and the valid matching paragraph to update the target matching paragraph set and the image data respectively, and returning to the step of extracting the closest matching paragraph group closest to the image data in the target matching paragraph set until there is no closest matching paragraph group in the target matching paragraph set;

[0172] The valid matching paragraphs are summarized to obtain an associated similarity group.

[0173] It can be understood that the target matching paragraph refers to the text paragraph corresponding to the target matching similarity, the nearest matching paragraph refers to the target matching paragraph closest to the image data in the target matching paragraph set, and the associated line number refers to the specific number of lines between the nearest matching paragraph and the image data.

[0174] For example, a target matching paragraph set is: {paragraph 1, paragraph 2, paragraph 3}, which are respectively 5 lines, 8 lines, and 3 lines away from a certain image data. Then, paragraph 3 corresponding to 3 lines is closest to the image data, so paragraph 3 is recorded as the closest matching paragraph, and 3 is the number of associated lines of the closest matching paragraph.

[0175] It is clear that the maximum number of lines refers to an artificially set number of lines, the nearest similarity refers to the matching similarity between the nearest matching paragraph and the image description phrase corresponding to the image data, and the correlation factor refers to a numerical value used to evaluate the overall correlation between the nearest matching paragraph and the image data. When the nearest matching paragraph is far away from the image data, the image data cannot usually be used for translation reference of the nearest matching paragraph, but the nearest matching paragraph still has the possibility of having a high content correlation with the image data. Therefore, it is necessary to conduct a comprehensive correlation evaluation of the nearest matching paragraph in combination with the position and content in this case.

[0176] S8. Summarize the multiple associated text paragraphs to obtain an associated text paragraph set, wherein each associated text paragraph in the associated text paragraph set corresponds to one or more image data, and there is no duplicate text paragraph in the associated text paragraph set.

[0177] Exemplarily, a set of associated text paragraphs is: {Paragraph 1, Paragraph 2, Paragraph 3, Paragraph 4}, wherein Paragraph 1 and Paragraph 2 are the associated text paragraphs corresponding to image data 1, Paragraph 1 and Paragraph 3 are the associated text paragraphs corresponding to image data 2, and Paragraph 3 and Paragraph 4 are the associated text paragraphs corresponding to image data 3.

[0178] S9. Using the image data set, perform image context fusion translation on the related text paragraph set to obtain related translated texts; based on the related text paragraph set, extract an unrelated text paragraph set from the text paragraph set; perform text translation on the unrelated text paragraph set to obtain unrelated translated texts; and complete cross-cultural translation based on context-aware fusion based on the related translated texts and the unrelated translated texts.

[0179] It can be understood that the associated translation text refers to the translation text obtained by translating the associated text paragraphs with reference to the image data, the unrelated translation text refers to the translation text obtained by translating the unrelated text paragraphs only through text information, and the text translation of the associated text paragraphs refers to translation through a translation model or algorithm, wherein the translation model or algorithm, for example: Seq2Seq model, Transformer model, etc.

[0180] In detail, the image dataset is used to perform image context fusion translation on the associated text paragraph set to obtain associated translated text, including:

[0181] Extracting related text paragraphs in sequence from the related text paragraph set, identifying related image groups corresponding to the related text paragraphs in the image data set, wherein the related image group includes one or more related images;

[0182] Extracting related images from the related image group in sequence, and using a pre-built deep learning network to perform object recognition on the related images to obtain an image recognition vocabulary group, wherein the deep learning network includes: a convolutional neural network, and the language of the image recognition vocabulary group is the target translation language;

[0183] Obtain a similar recognition vocabulary group set of the image recognition vocabulary group, wherein the similar recognition vocabulary groups in the similar recognition vocabulary group set correspond one-to-one to the image recognition vocabulary in the image recognition vocabulary group;

[0184] Translating the associated text paragraphs to obtain an original translation text, wherein the language category of the original translation text is the target translation language;

[0185] Sequentially extracting similar recognition vocabulary groups from the similar recognition vocabulary group set, identifying a source recognition vocabulary and a plurality of branch recognition vocabulary of the similar recognition vocabulary group;

[0186] Based on the multiple branch recognition words, multiple branch related words are searched in the original translation text, and independent related words are determined from the multiple branch related words according to the preset related word frequency, wherein the frequency of the independent related words appearing in the original translation text is less than the related word frequency;

[0187] Calculating the semantic similarity between the independent associated words and the source identification words, and recording the independent associated words whose semantic similarity is greater than the preset standard semantic similarity as replaceable associated words;

[0188] The source recognition vocabulary is used to replace the replaceable associated vocabulary in the original translation text to obtain the associated translation text.

[0189] It can be understood that the associated image group refers to the image data corresponding to the associated text paragraph, and the image recognition vocabulary group refers to the vocabulary group used to describe the image information in the associated image. The image recognition vocabulary group can be obtained through a deep learning network. For example, an image of grapes, the image is input into a convolutional neural network, and the vocabulary describing the image will be obtained: grapes, brackets, etc. The same type recognition vocabulary group refers to vocabulary with the same root as the image recognition vocabulary or the same etymology as the image recognition vocabulary. For example, grapes are an image recognition vocabulary, and its same type recognition vocabulary group is: raisins, wine, grape juice, etc.

[0190] It can be understood that the step of translating the associated text paragraphs is the same as the step of translating the text of the unrelated text paragraph set, the original translation text refers to the translated associated text paragraph, the source recognition vocabulary refers to the image recognition vocabulary in the same recognition vocabulary group, the multiple branch recognition vocabulary refers to the multiple same recognition vocabulary corresponding to the image recognition vocabulary in the same recognition vocabulary group, the branch associated vocabulary refers to the vocabulary that is the same as the branch recognition vocabulary in the original translation text, the associated word frequency refers to the artificially set word frequency constant, the independent associated vocabulary refers to the branch associated vocabulary whose word frequency appearing in the original translation text is less than the associated word frequency, the word meaning similarity refers to the semantic similarity between the independent associated vocabulary and the source recognition vocabulary, and its algorithm model for calculating it is, for example: bag of words model, BERT model, etc., the standard word meaning similarity refers to the artificially set similarity constant, and the replaceable associated vocabulary refers to the independent associated vocabulary whose word meaning similarity is greater than the standard word meaning similarity.

[0191] In detail, the step of obtaining a set of similar recognition vocabulary groups of the image recognition vocabulary group includes:

[0192] Acquire a target vocabulary library of the target translation language;

[0193] Extracting image recognition vocabulary in the image recognition vocabulary group in sequence, and identifying a plurality of similar recognition vocabulary of the image recognition vocabulary in the target vocabulary library;

[0194] Combining the image recognition vocabulary and a plurality of similar recognition vocabulary to obtain a similar recognition vocabulary group;

[0195] The same type of recognition vocabulary groups are summarized to obtain a same type of recognition vocabulary group set.

[0196] It should be explained that the target vocabulary library refers to the vocabulary library of the target translation language. The target vocabulary library covers a large number of words in the target translation language, including the definition, usage, etymology, etc. of different words. The target vocabulary library can be constructed by relevant language experts or through a dictionary of the target translation language.

[0197] The present invention solves the problem described in the background technology. First, a text data set and an image data set are respectively identified in the article to be translated. This process processes the text and image content in the article separately. Different processing strategies can be adopted for different types of data, thereby improving the accuracy of translation. Then, the image description text of the image data is identified to obtain the image description phrase. Extracting the image data and the description text is helpful to convert the image content into processable information. Meanwhile, the image description phrase can be associated with different paragraphs in the article, thereby improving the accuracy of text paragraph translation. Moreover, due to the intuitiveness of the image, the translation result is more in line with the context of the original article. By calculating the matching similarity of the text paragraph and the image description phrase, and judging whether the matching similarity is greater than the standard matching similarity, the correlation degree between the text paragraph and the image data is determined, so that images and texts with a high correlation degree can be integrated with each other, thereby improving the accuracy of the final translation result and also improving the efficiency of translation. The image data set is used to perform image context fusion translation on the associated text paragraph set to obtain the associated translation text. In this step, the image data and the text paragraph are contextually integrated, so that the translated associated translation text is more in line with the context, thereby improving the accuracy of cross-cultural translation. Meanwhile, the automated translation process also improves the efficiency of translation. Therefore, the present invention can improve the accuracy and efficiency of cross-cultural translation.

[0198] like Figure 2 , which is a functional module diagram of a cross-cultural translation system based on context-aware fusion provided by an embodiment of the present invention.

[0199] The cross-cultural translation system 100 based on context-aware fusion of the present invention can be installed in an electronic device. According to the functions implemented, the cross-cultural translation system 100 based on context-aware fusion can include a text paragraph division module 101, a phrase similarity evaluation module 102, an image text matching module 103 and an associated text translation module 104. The module of the present invention can also be called a unit, which refers to a series of computer program segments that can be executed by an electronic device processor and can complete fixed functions, which are stored in the memory of the electronic device.

[0200] The text paragraph division module 101 is used to receive an article to be translated, and start a data translation platform based on the article to be translated, wherein the article to be translated includes text data and image data, and the text data is divided into paragraphs, and the image data includes description text of related images, and the data translation platform includes intelligent software or a translation website. The data translation platform is used to identify a text data set and an image data set in the article to be translated, and the text data set is divided into paragraphs to obtain a text paragraph set;

[0201] The phrase similarity evaluation module 102 is used to sequentially extract image data from the image data set, identify image description text of the image data, segment the image description text to obtain image description phrases, sequentially extract text paragraphs from the text paragraph set, perform similarity determination on the text paragraphs and the image description phrases, and obtain matching similarity;

[0202] The image-text matching module 103 is used to determine whether the matching similarity is greater than a preset standard matching similarity. If the matching similarity is greater than the standard matching similarity, the matching similarity is recorded as a target matching similarity, the target matching similarities are summarized to obtain a target matching similarity set, an associated similarity group corresponding to the image data is identified in the target matching similarity set, and a plurality of associated text paragraphs corresponding to the associated similarity group are identified;

[0203] The associated text translation module 104 is used to summarize the multiple associated text paragraphs to obtain an associated text paragraph set, wherein each associated text paragraph in the associated text paragraph set corresponds to one or more image data, and there is no repeated text paragraph in the associated text paragraph set, and use the image data set to perform image context fusion translation on the associated text paragraph set to obtain associated translated text, and based on the associated text paragraph set, extract an unrelated text paragraph set from the text paragraph set, and perform text translation on the unrelated text paragraph set to obtain an unrelated translated text.

[0204] In detail, the modules in the cross-cultural translation system 100 based on context-aware fusion described in the embodiment of the present invention are used in the same manner as described above. Figure 1The same technical means are used as the cross-cultural translation method based on context-aware fusion described in , and can produce the same technical effects, so I will not go into details here.

[0205] like Figure 3 , which is a schematic diagram of the structure of an electronic device for implementing a cross-cultural translation method based on context-aware fusion provided by an embodiment of the present invention.

[0206] The electronic device 1 may include a processor 10, a memory 11 and a bus 12, and may also include a computer program stored in the memory 11 and executable on the processor 10, such as a cross-cultural translation method program based on context-aware fusion.

[0207] The memory 11 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, mobile hard disk, multimedia card, card-type memory (for example: SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. The memory 11 may be an internal storage unit of the electronic device 1 in some embodiments, such as a mobile hard disk of the electronic device 1. The memory 11 may also be an external storage device of the electronic device 1 in other embodiments, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device 1. Further, the memory 11 also includes an internal storage unit of the electronic device 1 and an external storage device. The memory 11 can not only be used to store application software and various types of data installed in the electronic device 1, such as the code of the cross-cultural translation method program based on context-aware fusion, but also be used to temporarily store data that has been output or is to be output.

[0208] The processor 10 may be composed of an integrated circuit in some embodiments, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and combinations of various control chips, etc. The processor 10 is the control core (Control Unit) of the electronic device, and uses various interfaces and lines to connect various components of the entire electronic device, and executes or executes programs or modules stored in the memory 11 (such as a cross-cultural translation method program based on context-aware fusion, etc.), and calls data stored in the memory 11 to execute various functions of the electronic device 1 and process data.

[0209] The bus 12 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus 12 may be divided into an address bus, a data bus, a control bus, etc. The bus 12 is configured to realize connection and communication between the memory 11 and at least one processor 10, etc.

[0210] Figure 3 Only an electronic device with components is shown, and those skilled in the art will understand that Figure 3 The structure shown does not constitute a limitation on the electronic device 1, and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0211] For example, although not shown, the electronic device 1 may also include a power source (such as a battery) for supplying power to each component. Preferably, the power source may be logically connected to the at least one processor 10 through a power management system, so that the power management system can realize functions such as charging management, discharging management, and power consumption management. The power source may also include any components such as one or more DC or AC power sources, recharging systems, power failure detection circuits, power converters or inverters, and power status indicators. The electronic device 1 may also include a variety of sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be repeated here.

[0212] Furthermore, the electronic device 1 may also include a network interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device 1 and other electronic devices.

[0213] Optionally, the electronic device 1 may further include a user interface, which may be a display, an input unit (such as a keyboard), or a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode) touch device. The display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device 1 and to display a visual user interface.

[0214] The cross-cultural translation method program based on context-aware fusion stored in the memory 11 of the electronic device 1 is a combination of multiple instructions. When running in the processor 10, it can achieve:

[0215] Receiving an article to be translated, and starting a data translation platform based on the article to be translated, wherein the article to be translated includes text data and image data, and the text data is separated by paragraphs, and the image data includes description text of related images, and the data translation platform includes intelligent software or a translation website;

[0216] Using the data translation platform, respectively identifying a text dataset and an image dataset in the article to be translated, dividing the text dataset into paragraphs, and obtaining a text paragraph set;

[0217] Extracting image data in the image data set in sequence, identifying image description text of the image data, and segmenting the image description text to obtain image description phrases;

[0218] Extracting text paragraphs from the text paragraph set in sequence, determining similarity between the text paragraphs and the image description phrases, and obtaining matching similarity;

[0219] Determining whether the matching similarity is greater than a preset standard matching similarity;

[0220] If the matching similarity is greater than the standard matching similarity, the matching similarity is recorded as the target matching similarity;

[0221] Summarizing the target matching similarities to obtain a target matching similarity set, identifying an associated similarity group corresponding to the image data in the target matching similarity set, and identifying a plurality of associated text paragraphs corresponding to the associated similarity group;

[0222] Summarizing the multiple associated text paragraphs to obtain an associated text paragraph set, wherein each associated text paragraph in the associated text paragraph set corresponds to one or more image data, and there is no duplicate text paragraph in the associated text paragraph set;

[0223] By using the image data set, the related text paragraph set is subjected to image context fusion translation to obtain related translated texts; based on the related text paragraph set, an unrelated text paragraph set is extracted from the text paragraph set; the unrelated text paragraph set is subjected to text translation to obtain unrelated translated texts; and based on the related translated texts and the unrelated translated texts, cross-cultural translation based on context-aware fusion is completed.

[0224] Specifically, the specific implementation method of the processor 10 for the above instructions can refer to Figures 1 to 3 The description of the relevant steps in the corresponding embodiments will not be repeated here.

[0225] Furthermore, if the module / unit integrated in the electronic device 1 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium can include: any entity or system capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, and a read-only memory (ROM).

[0226] The present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor of an electronic device, the computer program can implement:

[0227] Receiving an article to be translated, and starting a data translation platform based on the article to be translated, wherein the article to be translated includes text data and image data, and the text data is separated by paragraphs, and the image data includes description text of related images, and the data translation platform includes intelligent software or a translation website;

[0228] Using the data translation platform, respectively identifying a text dataset and an image dataset in the article to be translated, dividing the text dataset into paragraphs, and obtaining a text paragraph set;

[0229] Extracting image data in the image data set in sequence, identifying image description text of the image data, and segmenting the image description text to obtain image description phrases;

[0230] Extracting text paragraphs from the text paragraph set in sequence, determining similarity between the text paragraphs and the image description phrases, and obtaining matching similarity;

[0231] Determining whether the matching similarity is greater than a preset standard matching similarity;

[0232] If the matching similarity is greater than the standard matching similarity, the matching similarity is recorded as the target matching similarity;

[0233] Summarizing the target matching similarities to obtain a target matching similarity set, identifying an associated similarity group corresponding to the image data in the target matching similarity set, and identifying a plurality of associated text paragraphs corresponding to the associated similarity group;

[0234] Summarizing the multiple associated text paragraphs to obtain an associated text paragraph set, wherein each associated text paragraph in the associated text paragraph set corresponds to one or more image data, and there is no duplicate text paragraph in the associated text paragraph set;

[0235] By using the image data set, the related text paragraph set is subjected to image context fusion translation to obtain related translated texts; based on the related text paragraph set, an unrelated text paragraph set is extracted from the text paragraph set; the unrelated text paragraph set is subjected to text translation to obtain unrelated translated texts; and based on the related translated texts and the unrelated translated texts, cross-cultural translation based on context-aware fusion is completed.

[0236] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, systems and methods can be implemented in other ways. For example, the system embodiments described above are only illustrative, and actual implementation may have other division methods.

[0237] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0238] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0239] It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0240] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. A cross-cultural translation method based on context-aware fusion, characterized in that: The method comprises: Receiving an article to be translated, and starting a data translation platform based on the article to be translated, wherein the article to be translated includes text data and image data, and the text data is separated by paragraphs, and the image data includes description text of related images, and the data translation platform includes intelligent software or a translation website; Using the data translation platform, respectively identifying a text dataset and an image dataset in the article to be translated, dividing the text dataset into paragraphs, and obtaining a text paragraph set; Extracting image data in the image data set in sequence, identifying image description text of the image data, and segmenting the image description text to obtain image description phrases; Extracting text paragraphs from the text paragraph set in sequence, determining similarity between the text paragraphs and the image description phrases, and obtaining matching similarity; Determining whether the matching similarity is greater than a preset standard matching similarity; If the matching similarity is greater than the standard matching similarity, the matching similarity is recorded as the target matching similarity; Summarizing the target matching similarities to obtain a target matching similarity set, identifying an associated similarity group corresponding to the image data in the target matching similarity set, and identifying a plurality of associated text paragraphs corresponding to the associated similarity group; Summarizing the multiple associated text paragraphs to obtain an associated text paragraph set, wherein each associated text paragraph in the associated text paragraph set corresponds to one or more image data, and there is no duplicate text paragraph in the associated text paragraph set; By using the image data set, the related text paragraph set is subjected to image context fusion translation to obtain related translated texts; based on the related text paragraph set, an unrelated text paragraph set is extracted from the text paragraph set; the unrelated text paragraph set is subjected to text translation to obtain unrelated translated texts; and based on the related translated texts and the unrelated translated texts, cross-cultural translation based on context-aware fusion is completed.

2. The cross-cultural translation method based on context-aware fusion as claimed in claim 1, characterized in that: The step of segmenting the image description text to obtain image description phrases includes: Constructing a word segmentation dataset based on the text paragraph set, wherein the word segmentation dataset includes all words in the text paragraph set and their corresponding word frequencies; Segmenting the image description text to obtain original description phrases, wherein the original description phrases do not contain repeated original description words; Extracting original description words from the original description word group in sequence, and counting the original description frequency of the original description words in the image description text and the comparative description frequency in the word segmentation data set respectively; Identify the category to be translated of the article to be translated, obtain a text data set of the category to be translated, and count the overall description frequency of the original description words in the text data set; Calculate the description key value based on the original description frequency, the comparative description frequency and the overall description frequency; Determine whether the description key value is greater than a preset standard key value; If the description key value is greater than the standard key value, the original description word is recorded as the target description word, and the target description words are summarized to obtain an image description phrase.

3. The cross-cultural translation method based on context-aware fusion as claimed in claim 2, characterized in that: The calculation of description key values ​​according to the original description frequency, the comparative description frequency and the overall description frequency includes: Set the standard description frequency, and calculate the description key value using the following formula based on the standard description frequency, original description frequency, comparative description frequency, and overall description frequency: Among them, G represents the description key value, f y represents the original description frequency, e represents the natural constant, f z represents the overall description frequency, f d represents the comparison description frequency, and f0 represents the standard description frequency.

4. The cross-cultural translation method based on context-aware fusion as claimed in claim 3, characterized in that: The step of constructing a word segmentation dataset based on the text paragraph set includes: Performing word segmentation on each text paragraph in the text paragraph set to obtain a plurality of text paragraph word segmentation groups, wherein the number of the text paragraph word segmentation groups is equal to the number of text paragraphs in the text paragraph set; Merge all the text paragraph segmentations in multiple text paragraph segmentation groups to obtain a text paragraph segmentation set, remove duplicate words from the text paragraph segmentation set, and obtain a single paragraph segmentation set; Extracting single paragraph segmentation words in the single paragraph segmentation word set in turn, counting the frequency of occurrence of the single paragraph segmentation words in the single paragraph segmentation word set, performing key-value pairing of the paragraph occurrence frequency and the single paragraph segmentation word, and obtaining a single paragraph segmentation word group; The single paragraph word segmentation groups are aggregated to obtain a word segmentation dataset.

5. The cross-cultural translation method based on context-aware fusion as claimed in claim 4, characterized in that: The similarity determination of the text paragraph and the image description phrase to obtain the matching similarity includes: Identify the period symbol in the text paragraph, divide the text paragraph into sentences based on the period symbol to obtain paragraph short sentence groups, and extract paragraph short sentences in the paragraph short sentence groups in sequence; Segment the short sentences in the paragraph to obtain short sentence segmentation phrases, identify the overlapping words of the short sentence segmentation phrases and the image description phrases to obtain short sentence overlapping phrases, wherein there are no repeated words in the short sentence overlapping phrases, and the short sentence overlapping phrases may be an empty set; In the short sentence word groups, a short sentence overlap times array of the short sentence overlap words is counted, wherein the short sentence overlap times correspond to the short sentence overlap words one by one; According to the short sentence overlapping subarray and the image description phrase, constructing a phrase vector for the short sentence word segmentation group to obtain a short sentence word segmentation vector; Summarize the short sentence word segmentation vectors to obtain a short sentence word segmentation vector set; The image description phrase is vectorized to obtain the description phrase vector, where the description phrase vector is expressed as: in, represents the description word vector, K1 represents the description key value of the first image description word, K n represents the description key value of the nth image description word, where n represents the number of image description words in the image description phrase; According to the description phrase vector and the short sentence word vector set, the matching similarity is calculated using the following formula: Among them, S p represents the matching similarity, l represents the number of short sentence segmentation vectors in the short sentence segmentation vector set, represents the word segmentation vector of the a-th short sentence, and || represents the norm symbol of the vector.

6. The cross-cultural translation method based on context-aware fusion as claimed in claim 5, characterized in that: The constructing of a phrase vector for the short sentence word group according to the short sentence overlapping subarray and the image description phrase to obtain the short sentence word segmentation vector includes: Extracting image description words from the image description word group in sequence, and searching for the description overlap times corresponding to the image description words in the short sentence overlap times array, wherein the description overlap times may be 0; Summarizing the description overlap times according to the arrangement order of the image description words in the image description phrase group to obtain a description overlap times array; Based on the description overlap array, construct the short sentence segmentation vector, where the short sentence segmentation vector is expressed as: in, represents the short sentence segmentation vector, P1 represents the number of overlapping times of the first description, P n Indicates the number of times the nth description overlaps.

7. The cross-cultural translation method based on context-aware fusion as claimed in claim 6, characterized in that: The identifying the associated similarity group corresponding to the image data in the target matching similarity set includes: Identify a target matching paragraph set corresponding to the target matching similarity set; Extracting a nearest matching paragraph group that is closest to the image data from the target matching paragraph set, wherein the nearest matching paragraph group contains one or two nearest matching paragraphs; Sequentially extracting the nearest matching paragraphs from the nearest matching paragraph group, and identifying the number of associated lines between the nearest matching paragraphs and the image data; Determine whether the number of associated rows is less than a preset maximum number of rows; If the number of associated lines is not less than the maximum number of lines, the nearest similarity of the nearest matching paragraph is obtained, and the relevance of the nearest matching paragraph is evaluated according to the nearest similarity and the number of associated lines to obtain a relevance factor, wherein the relevance factor is expressed as: Among them, H represents the correlation factor, S z represents the nearest similarity, and L represents the number of associated rows; Determining whether the correlation factor is greater than a preset standard correlation factor; If the correlation factor is greater than the standard correlation factor, the nearest matching paragraph is recorded as a valid matching paragraph, and the valid matching paragraph is removed from the target matching paragraph set to obtain a correlated matching paragraph set; Using the associated matching paragraph set and the valid matching paragraph to update the target matching paragraph set and the image data respectively, and returning to the step of extracting the closest matching paragraph group closest to the image data in the target matching paragraph set until there is no closest matching paragraph group in the target matching paragraph set; If the number of associated lines is not less than the maximum number of lines, the nearest matching paragraph is recorded as a valid matching paragraph, and the valid matching paragraph is removed from the target matching paragraph set to obtain an associated matching paragraph set; Using the associated matching paragraph set and the valid matching paragraph to update the target matching paragraph set and the image data respectively, and returning to the step of extracting the closest matching paragraph group closest to the image data in the target matching paragraph set until there is no closest matching paragraph group in the target matching paragraph set; The valid matching paragraphs are summarized to obtain an associated similarity group.

8. The cross-cultural translation method based on context-aware fusion as claimed in claim 7, characterized in that: The method of using the image data set to perform image context fusion translation on the associated text paragraph set to obtain associated translated text includes: Extracting related text paragraphs in sequence from the related text paragraph set, identifying related image groups corresponding to the related text paragraphs in the image data set, wherein the related image group includes one or more related images; Extracting related images from the related image group in sequence, and using a pre-built deep learning network to perform object recognition on the related images to obtain an image recognition vocabulary group, wherein the deep learning network includes: a convolutional neural network, and the language of the image recognition vocabulary group is the target translation language; Obtain a similar recognition vocabulary group set of the image recognition vocabulary group, wherein the similar recognition vocabulary groups in the similar recognition vocabulary group set correspond one-to-one to the image recognition vocabulary in the image recognition vocabulary group; Translating the associated text paragraphs to obtain an original translation text, wherein the language category of the original translation text is the target translation language; Sequentially extracting similar recognition vocabulary groups from the similar recognition vocabulary group set, identifying a source recognition vocabulary and a plurality of branch recognition vocabulary of the similar recognition vocabulary group; Based on the multiple branch recognition words, multiple branch related words are searched in the original translation text, and independent related words are determined from the multiple branch related words according to the preset related word frequency, wherein the frequency of the independent related words appearing in the original translation text is less than the related word frequency; Calculating the semantic similarity between the independent associated words and the source identification words, and recording the independent associated words whose semantic similarity is greater than the preset standard semantic similarity as replaceable associated words; The source recognition vocabulary is used to replace the replaceable associated vocabulary in the original translation text to obtain the associated translation text.

9. The cross-cultural translation method based on context-aware fusion as claimed in claim 8, characterized in that: The step of obtaining a set of similar recognition vocabulary groups of the image recognition vocabulary group comprises: Acquire a target vocabulary library of the target translation language; Extracting image recognition vocabulary in the image recognition vocabulary group in sequence, and identifying a plurality of similar recognition vocabulary of the image recognition vocabulary in the target vocabulary library; Combining the image recognition vocabulary and a plurality of similar recognition vocabulary to obtain a similar recognition vocabulary group; The same type of recognition vocabulary groups are summarized to obtain a same type of recognition vocabulary group set.

10. A cross-cultural translation system based on context-aware fusion, characterized in that: The system comprises: A text paragraph division module is used to receive an article to be translated, and start a data translation platform based on the article to be translated, wherein the article to be translated includes text data and image data, and the text data is divided into paragraphs, and the image data includes description text of related images, and the data translation platform includes intelligent software or a translation website. The data translation platform is used to identify a text data set and an image data set in the article to be translated, and the text data set is divided into paragraphs to obtain a text paragraph set; A phrase similarity evaluation module is used to sequentially extract image data from the image data set, identify image description text of the image data, segment the image description text to obtain image description phrases, sequentially extract text paragraphs from the text paragraph set, perform similarity determination on the text paragraphs and the image description phrases, and obtain matching similarity; An image-text matching module, used to determine whether the matching similarity is greater than a preset standard matching similarity, if the matching similarity is greater than the standard matching similarity, the matching similarity is recorded as a target matching similarity, the target matching similarities are summarized to obtain a target matching similarity set, an associated similarity group corresponding to the image data is identified in the target matching similarity set, and a plurality of associated text paragraphs corresponding to the associated similarity group are identified; The associated text translation module is used to summarize the multiple associated text paragraphs to obtain an associated text paragraph set, wherein each associated text paragraph in the associated text paragraph set corresponds to one or more image data, and there is no repeated text paragraph in the associated text paragraph set, and the associated text paragraph set is subjected to image context fusion translation using the image data set to obtain associated translation text, and based on the associated text paragraph set, an unrelated text paragraph set is extracted from the text paragraph set, and text translation is performed on the unrelated text paragraph set to obtain an unrelated translation text.

Images