A language translation processing method, apparatus, device, medium, and product

Abstract

The application provides a language translation processing method, device, equipment, medium and product. The method comprises the following steps: obtaining source language data to be translated; translating the source language data into target language data by using a language translation model to obtain a target language translation result; wherein the language translation model is trained by using training samples of the source language and similar languages, the similar languages are languages that meet corresponding similarity requirements in each language feature dimension as compared with the source language, and each language feature dimension comprises at least one of a subword dimension, a phoneme dimension and a semantic dimension. The application can improve the translation quality of low-resource languages.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of machine translation, and in particular to a language translation processing method and device, equipment, medium and product. BACKGROUND

[0002] Machine translation refers to the use of computers to automatically translate one language into another language, thereby effectively facilitating information exchange and communication between different languages.

[0003] The currently widely used translation method is neural machine translation (NMT). Although this method performs well in translation quality, its performance mainly depends on the richness of corpus resources, so that the neural machine translation method only performs well in resource-rich languages, but often faces the problem of low translation accuracy in low-resource languages. SUMMARY

[0004] Based on the above technical status, the present application provides a language translation processing method, device, equipment, medium and product, which can improve the language translation accuracy of low-resource languages.

[0005] In order to achieve the above technical purpose, the present application specifically proposes the following technical solutions:

[0006] According to a first aspect of an embodiment of the present application, a language translation processing method is provided, comprising: obtaining source language data to be translated; translating the source language data into target language data through a language translation model to obtain a target language translation result; wherein the language translation model is trained through training samples of the source language and its similar language, the similar language being a language that meets the corresponding similarity requirement in each language feature dimension as the source language, and the each language feature dimension including at least one of subword dimension, phoneme dimension and semantic dimension.

[0007] In some implementations, the language translation model is trained by the following steps: determining the similar language of the source language; generating a training sample set based on the training sample corresponding to the source language and the training sample corresponding to the similar language; pre-training a preset neural network architecture according to the training sample set to obtain a pre-trained translation model; and performing transfer learning training on the pre-trained translation model based on the training sample corresponding to the source language to obtain the language translation model.

[0008] In some embodiments, the determining the similar language of the source language comprises: determining each related language of the source language; determining, for each language feature dimension in the language feature dimensions, a similarity between the source language and each related language in the language feature dimension; determining, as the similar language of the source language in the language feature dimension, the related language having a similarity greater than a preset similarity threshold in the language feature dimension; and determining, as the similar language of the source language, a union of the similar languages of the source language in the language feature dimensions.

[0009] In some embodiments, when the language feature dimensions comprise the subword dimension, the determining the similarity between the source language and each related language in the language feature dimension comprises: determining a coincidence degree of subwords in a subword dictionary of the source language and a subword dictionary corresponding to each related language, to obtain the similarity between the source language and each related language in the subword dimension; and / or, when the language feature dimensions comprise the phoneme dimension, the determining the similarity between the source language and each related language in the language feature dimension comprises: determining a similarity of phonemes in a phoneme dictionary of the source language and a phoneme dictionary corresponding to each related language, to obtain the similarity between the source language and each related language in the phoneme dimension; and / or, when the language feature dimensions comprise the semantic dimension, the determining the similarity between the source language and each related language in the language feature dimension comprises: determining a similarity between a training sample of the source language and a translation result of the source language corresponding to each related language, to obtain the similarity between the source language and each related language in the semantic dimension.

[0010] In some embodiments, the method further comprises: inputting the training sample of the similar language into the language translation model to obtain source language translation data; determining, as an optimized training sample, the source language translation data satisfying a preset scoring requirement; and performing optimized training on the language translation model based on the optimized training sample to obtain an optimized language translation model.

[0011] In some embodiments, the determining, as an optimized training sample, the source language translation data satisfying a preset scoring requirement comprises: determining a score of the source language translation data, the score comprising at least one of a language fluency score, a back-translation quality score, and a matching degree score with a preset source language term library; determining a comprehensive score of the source language translation data based on at least one of the language fluency score, the back-translation quality score, and the matching degree score; and determining that the source language translation data satisfies the preset scoring requirement and determining the source language translation data as the optimized training sample if the comprehensive score is greater than or equal to a preset score threshold.

[0012] In some implementations, when the score includes the language fluency score, determining the language fluency score of the source language translation data comprises: segmenting the source language translation data into individual n-grams; determining conditional probabilities of the individual n-grams by a language model; and determining the language fluency score of the source language translation data based on a multiplication result of the conditional probabilities of the individual n-grams and a weight coefficient corresponding to the language fluency score.

[0013] In some implementations, when the score includes the back translation quality score, determining the back translation quality score of the source language translation data comprises: translating the source language translation data into individual translation data in similar languages by the language translation model, and back translating the individual translation data in the similar languages into the source language to obtain individual source language back translation data; and obtaining the back translation quality score of the source language translation data based on a weighted sum of similarities between the individual source language back translation data and the source language translation data and a weight coefficient corresponding to the back translation quality score.

[0014] According to a second aspect of the embodiments of the present application, a language translation processing apparatus is provided, comprising: an acquisition unit configured to acquire source language data to be translated; and a translation processing unit configured to translate the source language data into target language data by a language translation model to obtain a target language translation result, wherein the language translation model is trained by training samples of the source language and similar languages of the source language, the similar languages being languages that satisfy corresponding similarity requirements with the source language in each language feature dimension, and the each language feature dimension comprising at least one of a subword dimension, a phoneme dimension, and a semantic dimension.

[0015] According to a third aspect of the embodiments of the present application, an electronic device is provided, comprising a memory and a processor; the memory is connected with the processor, and is configured to store a program; the processor is configured to realize the language translation processing method according to the first aspect by running the program in the memory.

[0016] According to a fourth aspect of the embodiments of the present application, a computer program product is provided, comprising computer program instructions, which, when executed by a processor, cause the processor to perform the language translation processing method according to the first aspect.

[0017] The language translation processing method, device, equipment, medium and product provided by the embodiments of the present application comprise the following steps: obtaining source language data to be translated; and translating the source language data into target language data by a language translation model to obtain a target language translation result, wherein the language translation model is trained by training samples of the source language and similar languages of the source language, the similar languages being languages that meet corresponding similarity requirements with the source language in each language feature dimension, and each language feature dimension comprising at least one of a subword dimension, a phoneme dimension and a semantic dimension. Since the language translation model is trained not only based on the training samples of the source language, but also by introducing the training samples of the similar languages that have similarities with the source language in multiple language feature dimensions, the model can identify and utilize the commonalities between different languages to enhance the understanding of the source language, thereby better capturing the language characteristics of the source language, enhancing the understanding and generation capabilities of the model for the source language, promoting positive transfer, and improving the translation quality of the translation processing model in a low-resource translation scenario. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings needed in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are only embodiments of the present application, and those skilled in the art can obtain other drawings according to the provided drawings without creative labor.

[0019] Figure 1 The flowchart of the language translation processing method provided by the embodiments of the present application.

[0020] Figure 2 The flowchart of the training process of the language translation model provided by the embodiments of the present application.

[0021] Figure 3 The principle diagram of determining the similar languages of the source language provided by the embodiments of the present application.

[0022] Figure 4 The flowchart of the model optimization training provided by the embodiments of the present application.

[0023] Figure 5 The flowchart of determining the optimization training samples provided by the embodiments of the present application.

[0024] Figure 6 The principle diagram of screening the optimization training samples provided by the embodiments of the present application.

[0025] Figure 7 The structural diagram of the language translation processing device provided by the embodiments of the present application.

[0026] Figure 8 A structural schematic diagram of an electronic device is provided for an embodiment of the present application. DETAILED DESCRIPTION

[0027] The technical solution provided by the embodiments of the present application is applicable to a cross-language real-time communication system, bilingual subtitle automatic generation of multimedia content, localization processing of commodity information of a cross-border e-commerce platform, and large-scale online translation services and other scenarios.

[0028] The technical solution provided by the embodiments of the present application can be exemplarily applied to a hardware device such as a processor, an electronic device, a server (including a cloud server), or packaged into a software program to be run. When the hardware device executes the processing process of the technical solution of the present application or the above-mentioned software program is run, the automatic splitting of the target task and the automatic calling of the application program interface required by the task can be realized, and the purpose of completing the target task can be achieved. The embodiments of the present application only exemplarily introduce the specific processing process of the technical solution of the present application, and do not limit the specific implementation form of the technical solution of the present application. Any technical implementation form that can execute the processing process of the technical solution of the present application can be adopted by the embodiments of the present application.

[0029] The technical solution in the embodiments of the present application will be described clearly and completely in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0030] Before introducing the scheme of the present application, first, the related art is introduced:

[0031] The neural machine translation system shows significant performance difference between Chinese, English, Korean and other resource-rich languages, and Uyghur, Tibetan and other low-resource languages. Because high-resource languages have massive parallel corpus and rich linguistic resources support, the translation quality can gradually approach the level of professional translators. In contrast, low-resource languages such as Uyghur and Tibetan often face higher error rates, weaker syntactic and semantic understanding ability, and poorer cross-domain generalization performance due to the lack of training data, lack of pre-training models, and difficult-to-catch language characteristics.

[0032] To improve the translation quality of low-resource languages, current methods mainly improve from the data level and the model construction level. At the data level, methods such as vocabulary replacement, synonymous syntax transformation, back-translation and its iterative variants are used to expand parallel corpora. At the model construction level, a general translation model based on a multilingual shared representation space is used, such as a Transformer-based multilingual encoder-decoder, which is adapted to low-resource languages through cross-language knowledge transfer and efficient parameter fine-tuning, and zero-shot / few-shot translation techniques are used to further improve the generalization ability of the model in resource-poor scenarios combined with temperature control decoding strategy.

[0033] Although the above methods can improve the translation quality of low-resource languages to some extent, there is still a significant gap in translation quality compared to mainstream languages. For example, at the data level, the target language text generated by back-translation may retain the syntactic features of the source language, resulting in insufficient domain adaptation. In addition, since back-translation relies on the effectiveness of existing translation models, if the model itself is poor, it may result in a decline in the quality of the back-translated data and contain more interference information.

[0034] At the model level, the efficiency of cross-language knowledge transfer is affected by the language family distance. For languages with complex morphological structure or large differences in linguistic features, the transfer effect is often limited. Therefore, when using a general multilingual translation model for translation tasks, since multiple languages share the same parameter space, it may cause negative transfer phenomenon, that is, high-resource languages tend to dominate the model learning process, thereby weakening the representation ability of low-resource languages.

[0035] Therefore, the embodiments of the present application aim to provide a language translation processing method, device, equipment, medium and product, which acquires a language that meets the corresponding similarity requirement of the source language at each language feature dimension as a similar language during model training, wherein each language feature dimension includes at least one of a subword dimension, a phoneme dimension and a semantic dimension, and then uses the source language and its similar language for model training. In this way, when the trained model is applied to a translation task, the translation quality can be improved. In the following embodiments, each will be described in detail.

[0036] Exemplary method

[0037] Figure 1 A flowchart of a language translation processing method provided by the embodiments of the present application. As shown in Figure 1 The language translation processing method provided by the embodiments of the present application includes steps S101-S102:

[0038] S101, acquiring source language data to be translated.

[0039] This step aims to collect the target content that the user needs to translate, usually including source language text data. Source language text can be obtained in various ways, such as the user directly inputting the text content to be translated in the input box of the translation application, translation APP or webpage interface; or through voice recognition technology, automatically transcribing the voice data entered by the user into corresponding text content. In addition, source language data can also come from document uploading, OCR of text in pictures, etc.

[0040] S102, translating the source language data into target language data through a language translation model to obtain a target language translation result.

[0041] The language translation model is trained through training samples of the source language and similar languages that meet the corresponding similarity requirements in each language feature dimension. Each language feature dimension includes at least one of the subword dimension, the phoneme dimension, and the semantic dimension.

[0042] The subword dimension is used to evaluate the similarity of different languages in vocabulary composition, such as whether they share similar word formation rules, prefixes, suffixes, or segmentation methods, etc.

[0043] The phoneme dimension is used to evaluate the similarity between languages from the perspective of the phonetic system, such as the degree of coincidence of the vowel and consonant system, the similarity of syllable structure, etc.

[0044] The semantic dimension is evaluated based on the similarity of lexical semantics and syntactic structure. Multilingual embedding space or pre-trained language models can be used to measure the semantic proximity of sentences or words between different languages.

[0045] This embodiment fuses training samples of the source language and similar languages, that is, the language translation model is not only trained based on training samples of the source language, but also introduces training samples of "similar languages" that have similarities with the source language in multiple language feature dimensions. Since similar languages often share some basic language building blocks (e.g., common subwords or phonemes), the model can identify and utilize these commonalities between different languages to enhance understanding of the source language. For example, in low-resource languages, by introducing similar language data, the model can learn a more extensive set of vocabulary representations, syntactic patterns, and semantic associations from more instances, so that even if two languages appear to be very different on the surface, they have significant commonalities in subword structure, phoneme system, or semantic expression, which can provide additional language information and contextual understanding for the language translation model, enabling it to better capture the language characteristics of the source language, enhance the model's understanding and generation capabilities of the source language, promote positive transfer, and improve the translation quality of the translation processing model in low-resource translation scenarios.

[0046] Figure 2 A flowchart of a training process of a language translation model is provided for an embodiment of the present application. As shown in Figure 2 the training process of the language translation model includes the following steps S201-S204:

[0047] S201, determine a similar language of a source language.

[0048] The similar language refers to a language that has commonality with the source language in at least one language feature dimension and meets a preset similarity threshold. For example, a language that has similar grammar and a large degree of coincidence of word groups with the source language. The source language can include low-resource languages such as Uyghur and Tibetan.

[0049] In some embodiments, the specific implementation of step S201 includes the following steps a1-a4:

[0050] Step a1, determine each related language of the source language.

[0051] Figure 3 A flowchart of determining a similar language of a source language is provided for an embodiment of the present application. As shown in Figure 3 , the languages similar to the source language in the language family or region can be preliminarily screened out as related languages from the dimensions of language family and / or region.

[0052] For example, taking Uyghur and Tibetan as examples, the following related languages can be screened out from the perspective of language family:

[0053] Further, the languages that meet the requirement of similarity are screened out as similar languages using screening rules. The specific screening process can be referred to the detailed introduction of steps a2-a4 as follows:

[0054] Step a2, for each language feature dimension in each language feature dimension, determine the similarity of the source language and each related language in the language feature dimension.

[0055] Each language feature dimension includes at least one of the subword dimension, the phoneme dimension, and the semantic dimension. For different language feature dimensions, step a2 corresponds to different similarity calculation methods, which are as follows:

[0056] Continuing to refer to Figure 3 In some examples, in the case where the language feature dimension includes the subword dimension, determining the similarity of the source language and each related language in the language feature dimension includes: determining the coincidence degree of the subwords in the subword dictionary of the source language and the subword dictionary corresponding to each related language, to obtain the similarity of the source language and each related language in the subword dimension.

[0057] In this example, first, based on the existing text data of the source language and each related language, a subword dictionary of the source language and a subword dictionary corresponding to each related language are respectively constructed by using the sentence piece modeling method.

[0058] Specifically, first, the size of the dictionary required for full coverage of the language characters is determined, and 1000 characters are added to the size as the final dictionary capacity; then, the subword dictionary of the source language is trained using this setting.

[0059] Similarly, for other languages related to the source language, a similar method can also be used for processing. That is: based on the existing text data, the sentence piece modeling is used, the size of the dictionary is set to the size required for full coverage of the characters plus 1000, and thus the subword dictionary of the related language is trained.

[0060] Next, the coincidence degree between each pair of subwords in the subword dictionary of the source language and the subword dictionary corresponding to each related language can be calculated, and the specific calculation formula is shown in formula (1) as follows:

[0061] (1)

[0062] In the formula, V 源语种 represents the set of subword dictionaries of the source language; V 相关语种 represents the set of subword dictionaries of the related language; ∩ is the intersection, ∪ is the union, and | · | represents the number of elements in the set.

[0063] The subword coincidence degree calculated by the above formula (1) can be used as the similarity of the source language and the related language in the subword dimension.

[0064] In some other examples, in the case of including the phoneme dimension in the language feature dimension, determining the similarity of the source language and each related language in the language feature dimension includes: determining the similarity of the phonemes in the phoneme dictionary of the source language and the phoneme dictionary corresponding to each related language, to obtain the similarity of the source language and each related language in the phoneme dimension.

[0065] Continuing to refer to Figure 3 In this example, first, the existing text data of the source language and each related language is preprocessed to construct the phoneme dictionary of the source language and the phoneme dictionary of each related language. The preprocessing includes: cleaning punctuation in the text data, processing Arabic numerals, etc.

[0066] Next, the similarity between each pair of phonemes in the phoneme dictionary of the source language and the phoneme dictionary of each related language can be calculated, and the specific calculation formula is shown in formula (2) as follows:

[0067] (1)

[0068] In the formula, U 源语种 denotes a set of phonemes in the source language; U 相关语种 denotes a set of phonemes in each related language; ∩ denotes an intersection, ∪ denotes a union, and | · | denotes the number of elements in a set.

[0069] The similarity between the source language and each related language in the phoneme dimension can be calculated by the above formula (2).

[0070] In some examples, when the language feature dimension includes a semantic dimension, determining the similarity between the source language and each related language in the language feature dimension includes: determining the similarity between the training sample of the source language and the source language translation result corresponding to each related language, to obtain the similarity between the source language and each related language in the semantic dimension.

[0071] Continuing to refer to Figure 3 In this example, parallel translation corpus between the source language and each related language needs to be obtained first. The parallel translation corpus refers to a pair of sentences in different languages that are translations of each other, for example, a sentence in Uyghur and its corresponding Chinese translation.

[0072] Subsequently, the parallel translation corpus is processed using a Word2vec technology model, so as to obtain the word vector representation of each word in different languages. The word vector is a numerical vector that maps words to a high-dimensional space and can reflect the semantic relationship between words.

[0073] Next, the cosine similarity between the word vector of each word in the source language and the word vector of the corresponding word in the related language is calculated, as the similarity between the source language and each related language in the semantic dimension. The cosine similarity is an index for measuring the similarity between two vectors in direction, and the value range is between -1 and 1. The closer the value is to 1, the more similar the semantics of the two words.

[0074] Step a3, determining the related language whose similarity with the source language in the language feature dimension is greater than a preset similarity threshold as the similar language of the source language in the language feature dimension.

[0075] Continuing to refer to Figure 3 When the language feature dimension includes a subword dimension, step a3 includes: selecting a related language whose subword overlap degree is greater than or equal to a preset overlap degree threshold as the similar language of the source language in the subword dimension, i.e., the similar language set 1 shown in the figure.

[0076] In the case that the language feature dimension comprises the phoneme dimension, the step a3 comprises: selecting the related language whose phoneme similarity is greater than or equal to the preset similarity threshold value as the similar language of the source language in the phoneme dimension, that is, the similar language set 2 shown in the figure.

[0077] In the case that the language feature dimension comprises the semantic dimension, the step a3 comprises: selecting the related language whose cosine similarity is greater than or equal to the preset similarity threshold value as the similar language of the source language in the semantic dimension, that is, the similar language set 3 shown in the figure.

[0078] The step a4 comprises: determining the union set of the similar languages of the source language in each language feature dimension as the similar language of the source language.

[0079] Continuing to refer to Figure 3 Since the language with high subword coincidence, high phoneme similarity or high semantic correlation with the source language is helpful for the subsequent training of the neural translation model, and in order to obtain more training data, the similar language set 1, the similar language set 2 and the similar language set 3 selected according to the above three language feature dimensions can be subjected to set operation and then de-duplication, and finally the similar language of the source language is obtained.

[0080] Continuing to refer to Figure 2 After the step S201, the following step S202 can also be included.

[0081] S202, generating a training sample set based on the training sample corresponding to the source language and the training sample corresponding to the similar language.

[0082] In the step S202, in order to obtain more training data and improve the model training effect, the text data corresponding to the standard language of the source language, the text data corresponding to the dialect of the source language and the text data corresponding to each similar language can be integrated, so as to construct the training sample set.

[0083] S203, pre-training the preset neural network architecture according to the training sample set to obtain a pre-training translation model.

[0084] In this step, by inputting the constructed training sample set into the preset neural network architecture for multilingual pre-training, a many-to-many translation model covering multiple languages and dialects, that is, a pre-training translation model, can be obtained.

[0085] The architecture can be a multilingual neural machine translation model based on the Transformer structure. The model has an encoder-decoder structure and supports joint modeling and parameter sharing of multiple languages.

[0086] Further, through multi-task learning and cross-language representation learning on large-scale, multi-lingual training sample sets, the model can automatically extract common language features between different languages and establish cross-language semantic mapping relationships. During the training process, the model not only learns the translation ability from the source language to the target language, but also simultaneously masters the mutual conversion ability between multiple languages, thereby forming a unified multi-lingual semantic space.

[0087] After sufficient training, the obtained pre-trained translation model has the ability to handle translation tasks between multiple languages, i.e., "many-to-many" translation ability, which not only enables high-quality translation between high-resource languages, but also supports translation reasoning for low-resource languages, dialects or unseen languages, thereby improving the generalization performance and adaptability of the model.

[0088] S204, performing transfer learning training on the pre-trained translation model based on the training sample corresponding to the source language to obtain a language translation model.

[0089] After obtaining the pre-trained translation model with multi-lingual understanding and translation ability, to further improve the translation performance of the model in the specific source language, especially in the low-resource language or dialect scenario, the embodiment can also use the transfer learning method to fine-tune the model.

[0090] Specifically, text data related to the source language can be used as training samples, including standard language text and dialect variant text of the language, to further fine-tune the pre-trained model, so that the model can more deeply learn the language structure, vocabulary usage, syntax features and expression habits of the source language, thereby significantly enhancing its language understanding and generation ability in the language.

[0091] During the transfer learning process, the model gradually adjusts its parameter distribution to better adapt to the language characteristics of the source language, while retaining the cross-language common knowledge learned in the early pre-training stage. Through this "general first, then specific" training strategy, not only can the adaptability of the model to low-resource languages be improved, but also the overfitting problem caused by limited data volume can be alleviated.

[0092] Finally, the language translation model obtained after transfer learning training can provide more accurate, natural and contextually appropriate translation results in translation tasks for the source language, especially its dialects or resource-scarce variants.

[0093] The embodiment introduces linguistic knowledge to screen related languages of the source language from multiple language feature dimensions, including subword overlap, phoneme similarity, and semantic relevance, etc. Through the multi-dimensional similarity screening, similar languages with commonalities with the source language at different language levels can be identified. On this basis, the text data of the similar languages and the dialect data of the source language are combined to be used in the pre-training process of the neural translation model. Since the multi-level language resources are fused, the model can learn more extensive language feature representations and cross-language semantic mapping relationships. Subsequently, the model pre-trained on the above multi-language and dialect data is migrated to a specific source language translation task to promote forward migration, enhance the understanding and generation ability of the model for low-resource languages, and improve the translation quality and model generalization ability of the model on low-resource languages.

[0094] To further improve the translation performance of the model on low-resource languages, the language translation model trained above can also be used to screen high-quality source language translation data, and the model can be trained again using the high-quality source language translation data to optimize the translation performance of the model on low-resource languages. The model optimization training process will be described in detail below with reference to the accompanying drawings:

[0095] Figure 4 A flowchart of the model optimization training provided by the embodiment of the present application is shown in FIG. 4. Figure 4 As shown in FIG. 4, the optimization training process of the model includes the following steps S401-S403:

[0096] S401, input the training samples of similar languages into the language translation model to obtain source language translation data.

[0097] The training samples of similar languages include text data. The text data of similar languages is translated into source language data by the language translation model, and the source language translation data can be obtained.

[0098] S402, determine the source language translation data meeting the preset score requirement as optimization training samples.

[0099] After obtaining the source language translation data, high-quality source language translation data can be screened out as the training samples for model optimization, so as to subsequently optimize the language translation model and further improve the translation performance of the model on low-resource languages. The specific implementation of the screening process will be described in detail below:

[0100] In some embodiments, as shown in FIG. 5, the specific implementation of step S402 includes the following steps S501-S503: Figure 5

[0101] ​S501. Determine the score of the source language translation data. The score includes at least one of the following: language fluency score, back-translation quality score, and matching score with the preset source language terminology database.

[0102] This step aims to determine the scores of the source language translation data under various filtering dimensions. These filtering dimensions include at least one of the following: n-grams filtering, back-translation quality filtering, and matching with a predefined source language terminology database. Specifically, the n-grams filtering dimension yields a language fluency score; the back-translation quality filtering dimension yields a back-translation quality score; and the matching with a predefined source language terminology database yields a matching score. The scores for each filtering dimension can be obtained using different implementation methods, as detailed below:

[0103] In some implementations, when the scoring includes a language fluency score, determining the language fluency score of the source language translation data includes: segmenting the source language translation data into individual n-gram phrases; determining the conditional probability of each n-gram phrase using a language model; and obtaining the fluency score of the source language translation data based on the cumulative product of the conditional probabilities of each n-gram phrase and the weight coefficient corresponding to the language fluency score.

[0104] like Figure 6 As shown, this example aims to filter high-quality source language translation data based on the n-grams selection dimension. Specifically, an n-gram Bi-Gram model can be used to calculate the log probability of each complete sentence in the source language translation data, thereby obtaining the fluency score 1 of the source language translation data under the n-grams selection dimension. This process includes: segmenting the source language translation text into multiple consecutive n-gram phrases, and calculating the conditional probability of each n-gram phrase using a pre-trained Bi-Gram model; then, by accumulating the conditional probabilities of all n-gram phrases and multiplying them by the weight coefficient w1 corresponding to the language fluency dimension, the fluency score 1 of the source language translation data is obtained. Here, n can take the value 2.

[0105] In other implementations, where the scoring includes a back-translation quality score, determining the back-translation quality score of the source language translation data involves: translating the source language translation data into translation data of various similar languages ​​using a language translation model, and back-translating the translation data of each similar language back into the source language to obtain back-translated data of each source language; and obtaining the back-translation score of the source language translation data based on the weighted sum of the similarity between each back-translated data of the source language and the source language translation data, and the weight coefficient corresponding to the back-translation quality score.

[0106] In the back-translation quality screening dimension, assuming that each similar language includes three similar languages, denoted as similar language 1, similar language 2 and similar language 3, the implementation manner includes:

[0107] Firstly, the source language translation data is translated into similar language 1, similar language 2 and similar language 3 through the language translation model, and the translation data corresponding to similar language 1, similar language 2 and similar language 3 is obtained.

[0108] Then, the translation data under similar language 1, similar language 2 and similar language 3 is translated back into the source language through the language translation model, and source language back-translation data 1, source language back-translation data 2 and source language back-translation data 3 are obtained.

[0109] Finally, the similarity algorithm is used to determine the similarity between the source language translation data and the source language back-translation data 1, source language back-translation data 2 and source language back-translation data 3, and the three similarities are weighted and summed and multiplied by the corresponding weight coefficient w2 to obtain the back-translation score 2 of the source language translation data. The similarity algorithm includes edit distance, cosine similarity, etc.

[0110] In some implementation manners, in the case where the score includes a matching degree score between the source language translation data and the preset source language term library, the matching degree score between the source language translation data and the preset source language term library is determined, including: determining the number of matched terms between the source language translation data and the terms in the source language term library; based on the number of matches, determining the matching degree score between the source language translation data and the preset source language term library.

[0111] Referring to Figure 6 , first, the source language term library needs to be constructed, specifically including: based on the existing high-quality source language text data, the source language terms are extracted from the high-quality source language text data, thereby obtaining the source language term library. Then, the source language translation data output by the language translation model is matched with the terms in the term library, and the number of matched terms is obtained, denoted as m, and the matching degree score 3 between the source language translation data and the preset source language term library can be represented as formula (3) as follows:

[0112] ; (3)

[0113] In the formula, is a preset parameter in the interval (1, 1.5); m is the number of completely matched terms, which is an integer greater than or equal to 0; w3 is the weight coefficient corresponding to the matching degree score.

[0114] S502, based on at least one of the language fluency score, the back-translation quality score and the matching degree score, determine the comprehensive score of the source language translation data.

[0115] Referring toFigure 6 The step S502 can have various implementations, and the details are as follows.

[0116] In some implementations, any one of the language fluency score, the back-translation quality score, and the matching degree score can be taken as the comprehensive score of the source language translation data.

[0117] In some other implementations, the sum of any two of the language fluency score, the back-translation quality score, and the matching degree score can be determined as the comprehensive score of the source language translation data.

[0118] In yet some other implementations, the sum of the language fluency score, the back-translation quality score, and the matching degree score can be determined as the comprehensive score of the source language translation data.

[0119] Continuing to refer to Figure 5 After the step S502, the step S503 can be further included.

[0120] S503, if the comprehensive score is greater than or equal to the preset score threshold, it is determined that the source language translation data meets the preset screening rule, and the source language translation data is determined as the optimized training sample.

[0121] After the comprehensive score is determined, high-quality source language translation data can be screened out as the optimized training sample based on the comparison result of the comprehensive score and the preset score threshold.

[0122] Continuing to refer to Figure 6 By judging whether the comprehensive score is greater than or equal to the preset score threshold, it is determined whether the source language translation data is high-quality translation data, if yes, the source language translation data is determined as the optimized training sample, otherwise, it is discarded.

[0123] Continuing to refer to Figure 4 After the step S402, the step S403 can be further included.

[0124] S403, the language translation model is optimized and trained based on the optimized training sample, and an optimized language translation model is obtained.

[0125] After the preferred training sample is obtained, the optimized training sample can be input into the language translation model for iterative training, so as to continuously optimize the model performance and continuously improve the translation quality, especially the translation quality of low-resource language.

[0126] In summary, the embodiments of the present application introduce linguistic knowledge to filter out languages similar to Uyghur and Tibetan, which are low-resource languages, from multiple dimensions such as subword overlap, phoneme similarity, and semantic relevance, and meanwhile, incorporate dialect data into training. The neural translation model is pre-trained by combining the data of Uyghur, Tibetan, and their dialects, as well as the data of similar languages, and is migrated to the translation task of Uyghur and Tibetan, thereby effectively promoting forward migration and improving the performance of the translation model.

[0127] Exemplary apparatus

[0128] Corresponding to the language translation processing method described above, the embodiments of the present application also provide a language translation processing apparatus. Figure 7 is a structural schematic diagram of a language translation processing apparatus provided by the embodiments of the present application. As shown in Figure 7 The language translation processing apparatus provided by the embodiments of the present application includes an acquisition unit 701 and a translation processing unit 702; the acquisition unit 701 is configured to acquire source language data to be translated; the translation processing unit 702 is configured to translate the source language data into target language data by a language translation model to obtain a target language translation result; the language translation model is trained by training samples of the source language and similar languages of the source language, the similar languages of the source language are languages that meet the corresponding similarity requirements in each language feature dimension, and the each language feature dimension includes at least one of a subword dimension, a phoneme dimension, and a semantic dimension.

[0129] In some embodiments, the language translation model is trained by a training unit 703, the training unit 703 is configured to perform the following steps: determining similar languages of the source language; generating a training sample set based on the training samples corresponding to the source language and the training samples corresponding to the similar languages; pre-training a preset neural network architecture according to the training sample set to obtain a pre-trained translation model; and performing transfer learning training on the pre-trained translation model based on the training samples corresponding to the source language to obtain the language translation model.

[0130] In some embodiments, the training unit 703 determines the similar languages of the source language, including: determining each related language of the source language; determining the similarity of the source language and the each related language in each language feature dimension; determining the related language whose similarity to the source language in the language feature dimension is greater than a preset similarity threshold as the similar language of the source language in the language feature dimension; and determining the union set of the similar languages of the source language in each language feature dimension as the similar languages of the source language.

[0131] In some embodiments, when the language feature dimension comprises the sub-word dimension, the training unit 703 determines the similarity of the source language and the each related language in the language feature dimension, comprising: determining the coincidence degree of the sub-words in the sub-word dictionary of the source language and the sub-word dictionary corresponding to the each related language, to obtain the similarity of the source language and the each related language in the sub-word dimension; and / or, when the language feature dimension comprises the phoneme dimension, the training unit 703 determines the similarity of the source language and the each related language in the language feature dimension, comprising: determining the similarity of the phonemes in the phoneme dictionary of the source language and the phoneme dictionary corresponding to the each related language, to obtain the similarity of the source language and the each related language in the phoneme dimension; and / or, when the language feature dimension comprises the semantic dimension, the training unit 703 determines the similarity of the source language and the each related language in the language feature dimension, comprising: determining the similarity between the training sample of the source language and the translation result of the source language corresponding to the each related language, to obtain the similarity of the source language and the each related language in the semantic dimension.

[0132] In some embodiments, the training unit 703 is further configured to perform the following steps: inputting the training sample of the similar language into the language translation model to obtain source language translation data; determining the source language translation data satisfying the preset score requirement as an optimized training sample; and performing optimized training on the language translation model based on the optimized training sample to obtain an optimized language translation model.

[0133] In some embodiments, the training unit 703 determines the source language translation data satisfying the preset score requirement as an optimized training sample, comprising: determining the score of the source language translation data, wherein the score comprises at least one of a language fluency score, a back-translation quality score, and a matching degree score with a preset source language term library; determining a comprehensive score of the source language translation data based on at least one of the language fluency score, the back-translation quality score, and the matching degree score; and determining that the source language translation data satisfies the preset score requirement and determining the source language translation data as the optimized training sample if the comprehensive score is greater than or equal to a preset score threshold.

[0134] In some embodiments, when the score comprises the language fluency score, the training unit 703 determines the language fluency score of the source language translation data, comprising: segmenting the source language translation data into each n-gram; determining the conditional probability of each n-gram through a language model; and determining the fluency score of the source language translation data based on the multiplication result of the conditional probability of each n-gram and the weight coefficient corresponding to the language fluency score.

[0135] In some embodiments, in the case that the score comprises the back translation quality score, the training unit 703 determines the back translation quality score of the source language translation data, comprising: translating the source language translation data into translation data in each similar language through the language translation model, and back translating the translation data in each similar language into source language to obtain each source language back translation data; obtaining the back translation quality score of the source language translation data based on the weighted sum of the similarity between each of the source language back translation data and the source language translation data, and the weight coefficient corresponding to the back translation quality score.

[0136] The language translation processing apparatus provided by the embodiments belongs to the same application concept as the language translation processing method provided by the embodiments of the present application, can execute the language translation processing method provided by any of the embodiments of the present application, and has the corresponding function modules and advantages of executing the language translation processing method. Technical details not described in detail in the embodiments can be found in the specific processing content of the language translation processing method provided by the embodiments of the present application, which will not be described here.

[0137] The functions implemented by the above acquisition unit 701, translation processing unit 702 and training unit 703 can be implemented by the same or different processors, which are not limited in the embodiments of the present application.

[0138] It should be understood that the units in the above apparatus can be implemented in the form of processor calling software. For example, the apparatus includes a processor connected with a memory, the memory stores instructions, and the processor calls the instructions stored in the memory to implement any of the above methods or to implement the functions of the units of the apparatus, wherein the processor can be a general processor such as CPU or microprocessor, and the memory can be an internal memory or an external memory of the apparatus. Alternatively, the units in the apparatus can be implemented in the form of hardware circuit. The functions of some or all units can be implemented by designing the hardware circuit. The hardware circuit can be understood as one or more processors. For example, in one implementation, the hardware circuit is ASIC, and the functions of some or all units are implemented by designing the logical relationship of elements in the circuit. For another example, in another implementation, the hardware circuit can be implemented by PLD. Taking FPGA as an example, it can include a large number of logic gate circuits, and the connection relationship between the logic gate circuits is configured by a configuration file, so as to implement the functions of some or all units. All units of the above apparatus can be implemented in the form of processor calling software, or implemented in the form of hardware circuit, or implemented in the form of processor calling software and the remaining part is implemented in the form of hardware circuit.

[0139] In the embodiments of the present application, the processor is a circuit with signal processing capability. In one implementation, the processor can be a circuit with instruction reading and running capability, such as a CPU, a microprocessor, a GPU, or a DSP, etc. In another implementation, the processor can implement certain functions through a logic relationship of a hardware circuit, which is fixed or can be reconfigured. For example, the processor is a hardware circuit implemented by an ASIC or a PLD, such as an FPGA, etc. In the reconfigurable hardware circuit, the processor loads a configuration document to implement the hardware circuit configuration. It can be understood that the processor loads instructions to implement the functions of the above units. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as an NPU, a TPU, a DPU, etc.

[0140] It can be seen that each unit in the above apparatus can be one or more processors (or processing circuits) configured to implement the above methods, such as a CPU, a GPU, an NPU, a TPU, a DPU, a microprocessor, a DSP, an ASIC, an FPGA, or a combination of at least two of these processor forms.

[0141] In addition, each unit in the above apparatus can be integrated together or can be independently implemented. In one implementation, these units are integrated together to implement a SOC. The SOC can include at least one processor for implementing any of the above methods or the functions of the units of the apparatus. The at least one processor can be different, such as including a CPU and an FPGA, a CPU and an artificial intelligence processor, a CPU and a GPU, etc.

[0142] Exemplary electronic device

[0143] The embodiments of the present application provide an electronic device, referring to Figure 8 The electronic device includes:

[0144] a memory 200 and a processor 210;

[0145] The memory 200 is connected with the processor 210, and is configured to store programs.

[0146] The processor 210 is configured to implement the language translation processing method disclosed in any of the above embodiments by running the programs stored in the memory 200.

[0147] Specifically, the above electronic device can further include a bus, a communication interface 220, an input device 230, and an output device 240.

[0148] The processor 210, the memory 200, the communication interface 220, the input device 230, and the output device 240 are connected with each other through the bus.

[0149] The bus can include a path for communicating information among the various components of the computer system.

[0150] The processor 210 can be a general purpose processor, such as a central processing unit (CPU), a microprocessor, etc., or can be an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of programs of the present application. It can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components.

[0151] The processor 210 can include a main processor, and can also include a baseband chip, a modem, etc.

[0152] The memory 200 stores programs for implementing the technical solutions of the present application, and can also store an operating system and other key services. Specifically, the programs can include program codes, and the program codes include computer operation instructions. More specifically, the memory 200 can include a read-only memory (ROM), other types of static storage devices that can store static information and instructions, a random access memory (RAM), other types of dynamic storage devices that can store information and instructions, a magnetic disk storage, a flash memory, etc.

[0153] The input device 230 can include a device that receives data and information input by a user, such as a keyboard, a mouse, a camera, a scanner, a light pen, a voice input device, a touch screen, a pedometer or a gravity sensor, etc.

[0154] The output device 240 can include a device that allows information to be output to a user, such as a display screen, a printer, a speaker, etc.

[0155] The communication interface 220 can include a device using any transceiver, such as an Ethernet, a radio access network (RAN), a wireless local area network (WLAN), etc., to communicate with other devices or communication networks.

[0156] The processor 210 executes the programs stored in the memory 200 and calls other devices, which can be used to implement each step of any one of the language translation processing methods provided by the above-mentioned embodiments of the present application.

[0157] The embodiment of the present application also provides a chip, which comprises a processor and a data interface, wherein the processor reads and runs a program stored on a memory through the data interface to execute the language translation processing method described in any of the above embodiments, and the specific processing process and advantages thereof can be referred to the above description of the language translation processing method.

[0158] Exemplary computer program product and storage medium

[0159] In addition to the above method and device, the embodiment of the present application can also be a computer program product, which comprises computer program instructions, and the computer program instructions make the processor execute the steps of the language translation processing method according to various embodiments of the present application described in any of the above embodiments of the present application when the processor runs the computer program instructions.

[0160] The computer program product can be written in any combination of one or more programming languages to perform the operations of the embodiments of the present application, including object-oriented programming languages, such as Java, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on a user computing device, partially on a user device, as an independent software package, partially on a user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0161] In addition, the embodiment of the present application can also be a storage medium, which stores a computer program, and the computer program makes the processor execute the steps of the language translation processing method according to various embodiments of the present application described in any of the above embodiments of the present application when the processor runs the computer program, and the following steps can be specifically implemented:

[0162] Step S101, obtaining facial video data of a target user.

[0163] Step S102, inputting the facial video data into a language translation processing model to obtain a target cognitive state corresponding to the target user, wherein the language translation processing model is obtained by optimizing and training a baseline language translation processing model based on artificially annotated video samples, and the accuracy of the baseline language translation processing model for predicting the cognitive state of the video samples is lower than a set accuracy threshold.

[0164] For each method embodiment described above, for the sake of simplicity, the method embodiments are described as a series of acts. But those skilled in the art will appreciate that the method embodiments are not limited by the order of acts, as some steps could occur in other orders or concurrently with each other. Moreover, those skilled in the art will appreciate that described acts could be implemented other ways, such as at least partially in hardware, and that the disclosure is not limited to the described or disclosed acts or to the order of such acts.

[0165] It should be noted that each of the above-described examples of the present disclosure are described in a progressive manner, and each example focuses on the differences from other examples. Therefore, the same or similar parts between examples can be mutually referred to. For the device examples, since they are basically similar to the method examples, the description is relatively simple, and the relevant parts can be referred to the description of the method examples.

[0166] The steps in the method embodiments of the present disclosure can be adjusted in order, combined, and reduced according to actual needs. The technical features described in the examples can be replaced or combined.

[0167] The modules and sub-modules in the device and terminal in the examples of the present disclosure can be combined, divided, and reduced according to actual needs.

[0168] In the several examples of the present disclosure, it should be understood that the disclosed terminal, device, and method can be implemented in other ways. For example, the terminal examples described above are only schematic. For example, the division of the modules or sub-modules is only a logical function division. In actual implementation, another division manner can be adopted, for example, a plurality of sub-modules or modules can be combined or integrated into another module, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the shown or discussed mutual elements can be indirect coupling or communication connection through some interface, device, or module, which can be electrical, mechanical, or other forms.

[0169] The modules or sub-modules described as separate components can or can not be physically separate, and the components of the modules or sub-modules can or can not be physical modules or sub-modules, i.e., can be located in one place or distributed on a plurality of network modules or sub-modules. Some or all of the modules or sub-modules can be selected according to actual needs to achieve the purpose of the present embodiment.

[0170] In addition, each functional module or sub-module in each embodiment of the present application can be integrated in one processing module, or each module or sub-module can exist physically alone, or two or more modules or sub-modules can be integrated in one module. The integrated module or sub-module can be realized in the form of hardware or in the form of a software functional module or sub-module.

[0171] Those skilled in the art will further appreciate that the units and algorithm steps of the examples described in connection with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or any combination thereof. To clearly illustrate the interchangeability of hardware and software, various examples have been described generally in terms of their functionality, without limitation to the corresponding description in any particular combination of hardware and software. Those skilled in the art will recognize many modifications and variations of this example that are apparent in light of the above teachings. It is therefore intended that such modifications and variations not depart from the scope of the application.

[0172] The steps of a method or algorithm described in connection with the embodiments disclosed herein can be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module can reside in random access memory (RAM), memory, read-only memory (ROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0173] Finally, it needs to be pointed out that, in this document, the relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or sequence between the entities or operations. Moreover, the terms "include", "contain" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or equipment. Without more limitations, the element defined by the statement "including a" does not exclude the presence of another identical element in the process, method, article or equipment including the element.

[0174] The above description of disclosed embodiments enables one of ordinary skill in the art to make and use the application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein can be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A language translation processing method characterized by comprising: The method comprises: obtaining source language data to be translated; translating the source language data into target language data through a language translation model to obtain a target language translation result; the training process of the language translation model comprises: training the language translation model through training samples of the source language and similar languages of the source language; the similar languages are languages that meet corresponding similarity requirements with the source language in each language feature dimension, and the each language feature dimension comprises at least one of a subword dimension, a phoneme dimension and a semantic dimension; optimizing and training the language translation model based on optimized training samples to obtain an optimized language translation model; wherein the optimized training samples are determined based on comprehensive scores of source language translation data corresponding to the training samples of the similar languages; the comprehensive scores of the source language translation data are determined according to a weighted sum of a language fluency score, a back translation quality score and a matching degree score; the determination process of the back translation quality score of the source language translation data comprises: translating the source language translation data into translation data of each similar language through the language translation model, and back translating the translation data of each similar language into source language to obtain each source language back translation data; determining a back translation score of the source language translation data based on a weighted sum of similarities between each of the source language back translation data and the source language translation data and a weight coefficient corresponding to the back translation quality score; the determination process of the language fluency score of the source language translation data comprises: segmenting the source language translation data into each n-gram; determining conditional probabilities of the each n-gram through a language model; determining a fluency score of the source language translation data based on a multiplication result of the conditional probabilities of the each n-gram and a weight coefficient corresponding to the language fluency score; the determination process of the matching degree score comprises: determining a matching number of words in the source language translation data and a source language terminology library; determining a matching degree score between the source language translation data and a preset source language terminology library based on the matching number.

2. The method of claim 1, wherein, the training of the language translation model through the training samples of the source language and the similar languages of the source language comprises: determining similar languages of the source language; generating a training sample set based on the training samples corresponding to the source language and the training samples corresponding to the similar languages; pre-training a preset neural network architecture according to the training sample set to obtain a pre-trained translation model; performing transfer learning training on the pre-trained translation model based on the training samples corresponding to the source language to obtain the language translation model.

3. The method of claim 2, wherein, the determination of the similar languages of the source language comprises: determining each related language of the source language; for each language feature dimension in the each language feature dimension, determining a similarity between the source language and the each related language in the language feature dimension; determining, as a similar language of the source language in the language feature dimension, a related language having a similarity greater than a preset similarity threshold with the source language in the language feature dimension. The union of the similar languages of the source language in each language feature dimension is determined as the similar languages of the source language.

4. The method of claim 3, wherein, In the case where the language feature dimension includes the sub-word dimension, the similarity of the source language and each related language in the language feature dimension is determined by: determining the coincidence degree of the sub-word dictionary of the source language and the sub-words in the respective corresponding sub-word dictionary of each related language, to obtain the similarity of the source language and each related language in the sub-word dimension; and / or, In the case where the language feature dimension includes the phoneme dimension, the similarity of the source language and each related language in the language feature dimension is determined by: determining the similarity of the phoneme dictionary of the source language and the phonemes in the respective corresponding phoneme dictionary of each related language, to obtain the similarity of the source language and each related language in the phoneme dimension; and / or, In the case where the language feature dimension includes the semantic dimension, the similarity of the source language and each related language in the language feature dimension is determined by: determining the similarity between the training samples of the source language and the source language translation results of each related language, to obtain the similarity of the source language and each related language in the semantic dimension.

5. The method of claim 2, wherein, The optimization training of the language translation model based on the optimization training samples includes: inputting the training samples of the similar languages into the language translation model to obtain source language translation data; determining the source language translation data meeting the preset scoring requirement as the optimization training samples; and optimizing training the language translation model based on the optimization training samples to obtain the optimized language translation model.

6. The method of claim 5, wherein, The determination of the source language translation data meeting the preset scoring requirement as the optimization training samples includes: determining the score of the source language translation data, the score including a language fluency score, a back-translation quality score, and a matching degree score with a preset source language term library; determining a comprehensive score of the source language translation data based on the weighted sum of the language fluency score, the back-translation quality score, and the matching degree score; if the comprehensive score is greater than or equal to a preset score threshold, it is determined that the source language translation data meets the preset scoring requirement, and the source language translation data is determined as the optimization training sample.

7. A language translation processing apparatus characterized by comprising: It includes: an acquisition unit configured to acquire source language data to be translated; a translation processing unit configured to translate the source language data into target language data through a language translation model to obtain a target language translation result; The training process of the language translation model includes: training the language translation model through the training samples of the source language and its similar languages; the similar languages are languages that meet the corresponding similarity requirements of the source language in each language feature dimension, and the each language feature dimension includes at least one of the sub-word dimension, the phoneme dimension, and the semantic dimension; optimization training the language translation model based on optimization training samples to obtain an optimized language translation model; The optimization training sample is determined based on a comprehensive score of source language translation data corresponding to the similar language training sample; and the comprehensive score of the source language translation data is determined according to a weighted sum of a language fluency score, a back translation quality score and a matching degree score. The back translation quality score of the source language translation data is determined by: The source language translation data is translated into translation data of each similar language by the language translation model, and the translation data of each similar language is back translated into source language to obtain each source language back translation data; The back translation quality score of the source language translation data is determined by: The source language translation data is cut into each n-gram; The conditional probability of each n-gram is determined by a language model; The fluency score of the source language translation data is determined based on a multiplication result of the conditional probability of each n-gram and a weight coefficient corresponding to the language fluency score; The matching degree score is determined by: The matching number of the source language translation data and the words in the source language term library is determined; The matching degree score between the source language translation data and the preset source language term library is determined based on the matching number. The memory and the processor are connected, and are used for storing programs; 8. An electronic device, comprising: The processor is used for realizing the method in any one of claims 1 to 6 by running the programs in the memory. The computer program instructions are used for realizing the method in any one of claims 1 to 6 when the computer program instructions are run by the processor. ​ 9. A computer program product, characterised in that, ​

Images