Translation quality evaluation method and device, equipment and storage medium

Abstract

The invention discloses a translation quality evaluation method and device, equipment and a storage medium, and the method comprises the steps: obtaining source texts of a plurality of fields and target translations obtained after the source texts are translated, and constructing a training set and a corpus; dividing the training set according to fields to obtain a plurality of groups of training subsets; respectively training a preset neural network through each group of training subsets, calculating a loss value according to a predicted quality score of the preset neural network and a corresponding actual quality score, updating network parameters of the preset neural network through the loss value until the preset neural network converges, and obtaining a translation quality evaluation model corresponding to each field; determining a field category to which the to-be-evaluated text belongs through a corpus; and according to the domain category, calling a corresponding translation quality evaluation model to carry out translation quality evaluation on the to-be-evaluated text to obtain a translation quality evaluation result of the to-be-evaluated text. According to the invention, the method improves the discrimination capability of the vocabularies in the professional field, and solves a problem that the evaluation of a general model in the professional field is not reliable.

Description

Technical Field

[0001] The present application relates to the field of machine translation technology, and in particular to a translation quality evaluation method, apparatus, device and storage medium. Background Art

[0002] With the rapid growth of true wireless stereo earphones, some high-end headphones are equipped with real-time translation features. Testing the translation functionality and quality of these headphones requires significant testing manpower. Furthermore, due to the limitations of testers' own skills, it is difficult to objectively and subjectively evaluate Chinese-to-English translations, or translations between Chinese and English. Unless more professional and experienced language experts are hired to provide objective scoring, this not only significantly delays testing progress but also incurs significant costs.

[0003] Existing translation quality assessment methods generally rely on a common dataset for unified training, and the resulting assessment models are equipped with a fixed set of universal parameters. However, when faced with specialized vocabulary in professional fields such as medicine, law, technology, and finance, these universal assessment models often exhibit limited representational capabilities, resulting in inaccurate evaluation results. Summary of the Invention

[0004] The present application provides a translation quality evaluation method, apparatus, device and storage medium for improving the existing translation quality evaluation method, which relies on a common data set for unified training. The resulting evaluation model is equipped with a set of fixed common parameters, resulting in a technical problem of low accuracy of translation quality evaluation results.

[0005] In view of this, the first aspect of this application provides a translation quality evaluation method, comprising:

[0006] Obtain source texts in multiple fields and target translated texts from the source texts to build training sets and corpora;

[0007] Dividing the training set according to fields to obtain multiple training subsets;

[0008] Training a preset neural network using each group of training subsets, calculating a loss value based on the predicted quality score of the preset neural network and the corresponding actual quality score, and updating the network parameters of the preset neural network using the loss value until the preset neural network converges, thereby obtaining a translation quality evaluation model corresponding to each field;

[0009] Determining the domain category to which the text to be evaluated belongs through the corpus; the text to be evaluated includes a source text to be evaluated and a target translation after translating the source text to be evaluated;

[0010] The corresponding translation quality evaluation model is called according to the field category to which the text to be evaluated belongs to perform translation quality evaluation on the text to be evaluated, and a translation quality evaluation result of the text to be evaluated is obtained.

[0011] Optionally, source texts in multiple fields and target translations of the source texts are obtained to construct a training set, including:

[0012] Acquire source texts in multiple fields and target translations of the source texts to obtain multiple sentence pairs;

[0013] Performing a quality score on the sentence pair to generate an actual quality score of the sentence pair;

[0014] Performing data cleaning on the sentence pairs, and performing word segmentation processing on the sentences in the cleaned sentence pairs;

[0015] Sentence alignment is performed on the sentence pairs after word segmentation.

[0016] Optionally, determining the field category to which the text to be evaluated belongs through the corpus includes:

[0017] Based on the keywords of each field marked in the corpus, identifying whether corresponding keywords exist in the text to be evaluated;

[0018] If so, determining the field category to which the text to be evaluated belongs based on the keywords in the text to be evaluated;

[0019] If not, semantic analysis and semantic matching are performed on the text to be evaluated and the corpus, and the field category to which the text to be evaluated belongs is determined according to the obtained semantic matching result.

[0020] Optionally, the preset neural network includes an encoder and a decoder; the encoder includes a multi-head attention layer, a residual and normalization layer, and a feedforward layer;

[0021] When the training subset belongs to a general domain, the decoder includes a parameter sharing layer, and the parameter sharing layer includes a linear layer and an activation layer;

[0022] When the training subset belongs to a non-general field, the decoder includes a parameter sharing layer and a feature parameter layer, and the feature parameter layer is a linear layer; wherein the non-general field includes the medical field, the legal field, the scientific and technological field, and the financial field.

[0023] Optionally, calling the corresponding translation quality evaluation model according to the field category to which the text to be evaluated belongs to perform translation quality evaluation on the text to be evaluated, and obtaining the translation quality evaluation result of the text to be evaluated includes:

[0024] Calling the corresponding translation quality evaluation model according to the field category to which the text to be evaluated belongs to perform translation quality evaluation on the text to be evaluated, and outputting a predicted quality score of the text to be evaluated;

[0025] The predicted quality score of the text to be evaluated is input into a trained linear regression model, the predicted quality score is mapped to an evaluation grade, and a translation quality evaluation result of the text to be evaluated is obtained.

[0026] Optionally, the training process of the linear regression model includes:

[0027] Translation experts in various fields evaluate the translation quality level based on the source texts in the training subsets of the corresponding fields and the target translations after translating the source texts, thereby obtaining the actual evaluation level of each group of the training subsets;

[0028] Initializing model parameters in the linear regression model;

[0029] Inputting the quality scores of the training subsets of each group into the linear regression model for linear mapping to obtain the predicted evaluation level of the training subsets of each group;

[0030] The grade evaluation loss value is calculated according to the actual evaluation grade and the predicted evaluation grade of each group of the training subsets, and the model parameters of the linear regression model are updated according to the grade evaluation loss value until the linear regression model converges to obtain a trained linear regression model.

[0031] Optionally, the method further includes:

[0032] Collecting feedback results from users when evaluating translation quality using the translation quality evaluation model and the linear regression model;

[0033] If the feedback result is disapproval of the translation quality evaluation result and the number of feedbacks exceeds the target number, the model parameters of the translation quality evaluation model and the linear regression model are updated.

[0034] A second aspect of the present application provides a translation quality evaluation device, comprising:

[0035] A data acquisition unit, configured to acquire source texts in multiple fields and target texts translated from the source texts, and construct training sets and corpora;

[0036] A data partitioning unit, configured to partition the training set according to fields to obtain multiple training subsets;

[0037] a training unit, configured to train a preset neural network using each group of the training subsets, calculate a loss value based on the predicted quality scores of the preset neural network and the corresponding actual quality scores, and update network parameters of the preset neural network using the loss value until the preset neural network converges, thereby obtaining a translation quality evaluation model corresponding to each field;

[0038] A domain classification unit is used to determine the domain category to which the text to be evaluated belongs through the corpus; the text to be evaluated includes a source text to be evaluated and a target translation of the source text to be evaluated;

[0039] The quality evaluation unit is used to call the corresponding translation quality evaluation model according to the field category to which the text to be evaluated belongs to perform translation quality evaluation on the text to be evaluated, and obtain a translation quality evaluation result of the text to be evaluated.

[0040] A third aspect of the present application provides an electronic device, the device comprising a processor and a memory;

[0041] The memory is used to store program code and transmit the program code to the processor;

[0042] The processor is configured to execute any one of the translation quality evaluation methods described in the first aspect according to instructions in the program code.

[0043] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store program code, and when the program code is executed by a processor, the translation quality evaluation method according to any one of the first aspects is implemented.

[0044] It can be seen from the above technical solutions that this application has the following advantages:

[0045] The translation quality evaluation method provided in this application trains model parameters for various fields to obtain a translation quality evaluation model for a specific field, and constructs a corpus covering multiple fields such as medicine, law, technology, finance, and general purposes. Based on the corpus, the input sentence is field-screened and the evaluation model of the corresponding field is selected to evaluate the translation quality, which effectively improves the ability to discriminate vocabulary in professional fields, solves the problem of unreliable vocabulary evaluation of traditional general models in professional fields, and improves the accuracy of translation quality evaluation results. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0047] Figure 1 A flowchart of a translation quality evaluation method provided in an embodiment of the present application;

[0048] Figure 2 A schematic diagram of the structure of a translation quality evaluation model provided in an embodiment of the present application;

[0049] Figure 3 Comparison results between the model of the present application and the existing large model provided in the embodiments of the present application;

[0050] Figure 4 A schematic diagram of the structure of a translation quality evaluation device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0051] In order to help those skilled in the art better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0052] With the rapid growth of true wireless stereo earphones, some high-end headphones are equipped with real-time translation features. Testing the translation functionality and quality of these headphones requires significant testing manpower. Furthermore, due to the limitations of testers' own abilities, it's difficult to objectively and subjectively evaluate Chinese-to-English translations. Unless more professional and experienced language experts are hired to provide objective scoring, this not only significantly delays testing progress but also incurs significant costs. Existing methods present the following problems:

[0053] 1. Existing translation quality assessment methods generally rely on a common dataset for unified training, and the resulting models are equipped with a fixed set of universal parameters. However, when faced with specialized vocabulary in fields such as medicine, law, technology, and finance, these general models often exhibit limited representational capabilities, significantly compromising the accuracy of the assessment results.

[0054] 2. Existing machine translation quality evaluation methods are often limited to providing a single subjective or objective score, which is fixed once established. This means that even if experts with deep expertise in a specific field disagree with the translation results, their opinions cannot positively impact the accuracy of the evaluation model. This one-way evaluation mechanism limits the potential for further model optimization.

[0055] 3. Among existing machine translation quality assessment methods, traditional deep learning models generally suffer from insufficient operational efficiency. While solutions based on pre-trained language models (such as BERT) offer high accuracy, their large number of parameters significantly increases inference latency, making real-time processing difficult, especially in scenarios with limited GPU memory. Some research uses ensemble learning methods to fuse predictions from multiple models, but parallelizing multiple models exponentially increases computing resource consumption. While statistical feature methods offer the advantage of being lightweight, their reliance on manually designed linguistic features (such as syntactic tree depth and semantic role matching) requires a complex feature extraction process. The feature calculation process involves multiple layers of loops and matrix operations, resulting in high CPU utilization. Furthermore, existing model compression techniques (such as knowledge distillation) often sacrifice accuracy while increasing speed. Unfine-tuned lightweight models struggle to balance accuracy and inference speed for translation quality assessment tasks. These limitations severely restrict the deployment and application of assessment systems on edge devices or in real-time interactive scenarios.

[0056] To improve the above problems, please refer to Figure 1 , the embodiment of the present application provides a translation quality evaluation method, comprising:

[0057] Step 110: Obtain source texts in multiple fields and target translated texts after the source texts are translated, and construct training sets and corpora;

[0058] The corpus is constructed by collecting source texts and target translations from multiple fields. It includes text from medicine, law, technology, and finance, four fields with the highest error rates in actual testing. Common idioms and slang are also collected to supplement the dialect library. In this embodiment of the application, the corpus may include a medical corpus, a legal corpus, a technology corpus, a financial corpus, and a general corpus (i.e., covering fields other than medicine, law, technology, and finance). The dialect library is categorized as part of the general corpus.

[0059] The training set construction process can be: obtaining source texts from multiple fields and target translations of the source texts to obtain multiple sentence pairs; scoring the quality of the sentence pairs to generate actual quality scores for the sentence pairs; cleaning the sentence pairs and performing word segmentation on the sentences in the cleaned sentence pairs; and aligning the sentence pairs after word segmentation.

[0060] Translation-enabled devices such as headphones and smart speakers can be used to translate the source text and obtain the target translation. Data sets are collected from source texts and corresponding target translations across multiple domains. Translation experts in various fields can assess the data quality of the datasets and generate actual quality scores. The data is then converted to a CSV format, with each row containing sentences and associated metadata (domain labels and actual quality scores). The dataset is then cleaned, including removing duplicate and incomplete sentences and deleting samples with missing values. Sentences in the source and target translations are segmented by word. For example, for English, the NLTK tokenizer is used; for Chinese, the Jieba tokenizer is used. This process converts sentences into word sequences, facilitating subsequent feature extraction. The dataset consists of sentence pairs from the source and target languages, ensuring accurate semantic alignment. The GIZA++ tool can be used to align bilingual sentences, calculating the probability of word alignment to find the most likely alignment.

[0061] The aligned dataset can be divided into training, test, and validation sets according to a certain ratio. The training set is used to train the model parameters, the test set is used to test the trained model parameters, and the validation set is used to verify the trained model performance.

[0062] Step 120: Divide the training set according to the field to obtain multiple training subsets;

[0063] Before training, the training data in the training set needs to be divided by domain to obtain multiple training subsets, each corresponding to a domain. Data division can be performed based on the domain label of each training data in the dataset.

[0064] Step 130: Train the preset neural network using each training subset, calculate a loss value based on the predicted quality scores of the preset neural network and the corresponding actual quality scores, and update the network parameters of the preset neural network using the loss value until the preset neural network converges, thereby obtaining a translation quality evaluation model corresponding to each field.

[0065] The structure of the preset neural network in the embodiment of the present application can be referred to Figure 2 The preset neural network includes an encoder and a decoder; the encoder includes a multi-head attention layer, a residual and normalization layer, and a feedforward layer; when the training subset belongs to a general field, the decoder includes a parameter sharing layer, and the parameter sharing layer includes a linear layer and an activation layer; when the training subset belongs to a non-general field, the decoder includes a parameter sharing layer and a feature parameter layer, and the feature parameter layer is a linear layer; among them, non-general fields include the medical field, the legal field, the scientific and technological field, and the financial field.

[0066] During training, the input sentences are divided into different domains. During testing and validation, the input sentences are only classified using a corpus; the model training process does not require a corpus. The encoder in this application includes 48 encoding blocks with 16-head attention layers and 32 encoding blocks with 24-head attention layers. These blocks work together to effectively improve the encoder's performance and feature extraction capabilities. A multi-head attention layer is followed by a regularization layer, which adds the residuals. This is then passed through a feed-forward layer, followed by another regularization layer and the addition of the residuals. A pre-trained transformer architecture can be used as the encoder, using the parameters of a pre-trained large model as the pre-trained input for the encoder. This avoids the long loading time of large model parameters and many efficiency and performance issues of traditional deep learning models. This improves training efficiency while ensuring reliable model evaluation, achieving a balance between accuracy and inference speed. The decoder primarily consists of a parameter sharing layer and a feature parameter layer. In the parameter sharing layer, an LN (linear regression) layer is followed by a ReLU (reduction-unit activation) layer. The LN layer performs a channel-wise transformation, with an input of 10 and an output of 20. The parameter sharing layer will share parameters with the words and sentences in the five fields, and all sentence features will be updated after passing the parameters. The feature parameter layer consists of four dynamic LN layers. Each LN layer corresponds to the parameters of a field. The general field directly uses the parameters of the parameter sharing layer and does not have the nature of feature parameters. The number of input channels of the feature parameter layer is 20, and the output is 1. During training, the parameters of the parameter sharing layer are updated uniformly, and the non-shared layer updates specific parameters based on the words in different fields. During the parameter loading stage, when saving the parameters, the parameters of the shared part and the specific part of each field are saved separately. The loss function used in the model training process can be:

[0067]

[0068] Where, Refers to the loss function; represents the actual quality score; represents the prediction quality score; Represents the training data, i.e., sentence pairs. The loss value is calculated using the actual and predicted quality scores of the training data. This loss value is then used to update the network parameters (including weights and biases) of the pre-trained neural network until the pre-trained neural network converges, resulting in a trained pre-trained neural network.

[0069] During the training phase, the training set is first divided into five training subsets based on domains. The first four subsets are for medicine, law, technology, and finance, respectively. The fifth subset is a general training subset, with dialects categorized as general domains. Training for each domain is performed step by step. For example, data from domain 1 is trained first. The training data in domain 1 undergoes feature extraction through the encoder and is then input into the decoder. The decoder passes through a parameter sharing layer and then a feature parameter layer. Domains 1 through 4 perform the same operations, differing only in the input data. After training, four model parameters are obtained. Finally, the general domain data, which includes dialects, is trained. The general domain training data undergoes feature extraction through the encoder and is then input into the decoder. The decoder only passes through the parameter sharing layer and directly outputs a predicted quality score. A loss value is calculated by comparing the predicted quality score with the actual quality score, and the model parameters are updated based on the loss value. After training, the model parameters for the general domain are obtained.

[0070] Test and verification phase: The verification sentences are first input into the corpus for domain division, and then the corresponding model parameters are loaded according to the divided domains for quality evaluation, and the quality scores are output.

[0071] Step 140: Determine the domain category to which the text to be evaluated belongs through the corpus;

[0072] The source text to be evaluated and the target translation obtained by translating the source text to be evaluated through a translation tool are collected to obtain the text to be evaluated. Based on the keywords of each field marked in the corpus, it is identified whether there are corresponding keywords in the text to be evaluated; if so, the field category to which the text to be evaluated belongs is determined based on the keywords in the text to be evaluated; if not, the text to be evaluated is subjected to semantic analysis and semantic matching with the corpus, and the field category to which the text to be evaluated belongs is determined based on the obtained semantic matching results. In the embodiment of the present application, keywords for each field are set based on the entries in each field in the prediction library (such as the legal field can be set to keywords such as law, lawyer, justice, etc.), and it is identified whether the text to be evaluated triggers specific keywords. If keywords in a certain field are triggered, it is divided into the corresponding field. If keywords are not triggered, the corresponding field is screened out through the semantic analysis and matching mechanism, and it can be determined whether the sentence contains dialect elements and the dialect to which it belongs.

[0073] Step 150 : Call the corresponding translation quality evaluation model according to the domain category to which the text to be evaluated belongs to perform translation quality evaluation on the text to be evaluated, and obtain a translation quality evaluation result of the text to be evaluated.

[0074] After determining the domain category of the text to be evaluated, the translation quality evaluation model corresponding to that domain category is used to evaluate the text's translation quality, outputting a predicted quality score for the text. This predicted quality score is then input into a trained linear regression model, which maps the predicted quality score to a rating, ultimately obtaining the translation quality evaluation result for the text.

[0075] After the translation quality assessment model outputs an objective score (i.e., a quality score), the results are quantitatively analyzed by experts and converted into a subjective evaluation grade. A linear regression model is constructed by regressing the subjective evaluation grade with the model's objective score. This linear regression model is then used to map the quality score to an evaluation grade. The training process for the linear regression model includes:

[0076] S1. Translation experts in various fields evaluate the translation quality of the source texts and target translations in the training subsets of the corresponding fields, and obtain the actual evaluation grade of each training subset;

[0077] S2, initialize the model parameters in the linear regression model;

[0078] The linear regression model in the embodiment of the present application can be:

[0079]

[0080] in, is the intermediate parameter, To predict the quality score, and are the parameters to be trained, that is, the model parameters that need to be initialized in this step.

[0081] S3. Inputting the quality score of each group of training subsets into a linear regression model for linear mapping to obtain a predicted evaluation level of each group of training subsets;

[0082] The quality scores of each training subset are input into the linear regression model for linear mapping, and the predicted probability of each training data is output. The predicted evaluation level of each training data can be determined based on the predicted probability. The evaluation level can include three levels: Bad, OK, and Good.

[0083] S4. Calculate the grade evaluation loss value based on the actual evaluation grade and the predicted evaluation grade of each group of training subsets, and update the model parameters of the linear regression model by the grade evaluation loss value until the linear regression model converges to obtain a trained linear regression model.

[0084] Among them, the calculation formula of the grade evaluation loss value can be:

[0085]

[0086] Where y i is the true label of the training data i, such as (0, 0.5, 1) corresponding to Bad, OK, and Good labels respectively. is the predicted probability corresponding to the training data i, when when When , corresponds to Bad.

[0087] Gradient descent method can be used to update model parameters and :

[0088] ; ; is the learning rate, which can be set to 0.01;

[0089] Iterative training is performed on the training subset until the linear regression model converges (such as reaching the maximum number of iterations), and a trained linear regression model is obtained.

[0090] Furthermore, after step 150 , the method further includes: step 160 , collecting feedback results from users when performing translation quality evaluation using the translation quality evaluation model and the linear regression model, and updating model parameters based on the feedback results.

[0091] Collect user feedback from translation quality evaluation models and linear regression models. This feedback can include whether users approve the evaluation results or not, and if they disapprove, a revised evaluation grade. If the feedback is disapproval and the number of feedback exceeds the target number, the model parameters of the translation quality evaluation model and linear regression model are updated based on the feedback grade. Subsequent translation quality evaluation is performed using the updated model.

[0092] To verify the effectiveness of the model, we randomly selected 10 sentence pairs from the validation set and fed them into the AI model. We required the AI to output corresponding scores. The AI models used included DeepSeek-v3, Gpt-4.0, and Doubao. Figure 3 The specific scores of 10 pairs of translation texts by different models and the MSE (Mean Squared Error) values of each model are shown. Figure 3From the data, we can see that our model (i.e. the model provided by this application) has the lowest MSE score compared to the other three models, proving that the stability of our model is better than the other three. In addition, in terms of subjective evaluation, the subjective evaluation level (Bad, OK, Good) of the model after the quantification table deviates from the expert evaluation level in only one sentence, while the remaining nine sentences are consistent with the expert evaluation level. Figure 3 The sentence pairs used for verification are all used in actual projects. It can also be verified that the evaluation method provided by the present invention can be directly used for translation quality evaluation of actual projects. In addition, the model supports translation evaluation in 100 languages. In terms of processing time, the time required for the method provided by the application is 50% of that of the listed models, which greatly saves evaluation time and improves evaluation speed.

[0093] This application constructs a corpus covering the four major fields of medicine, law, technology, and finance. In addition, additional dialect entries are collected as corpora for feature fields and general fields. Based on this corpus, a method for input sentence field screening and field-specific model parameter training is proposed, which effectively improves the ability to distinguish professional field vocabulary and solves the problem of unreliable evaluation of professional field vocabulary by traditional general models;

[0094] Furthermore, an evaluation system that selectively adjusts model parameters based on specific user feedback breaks the existing one-way fixed evaluation model. By collecting, analyzing, and integrating feedback, model parameters are optimized, greatly improving the accuracy and professionalism of the translation evaluation model.

[0095] The evaluation method provided in this application can be adapted to phrases in different fields and can also iteratively optimize parameters through a feedback mechanism. Furthermore, during the training process, the parameters of the large model are used as pre-training inputs for the small model, and the actual training adopts a hybrid model training strategy with lightweight small models. This successfully avoids the long loading time of large model parameters and many efficiency and performance issues of traditional deep learning models. While improving training efficiency, it ensures the reliability of model evaluation and achieves a balance between accuracy and inference speed.

[0096] Please refer to Figure 4 , the embodiment of the present application further provides a translation quality evaluation device, comprising:

[0097] The data acquisition unit 410 is used to acquire source texts in multiple fields and target translations of the source texts to construct training sets and corpora;

[0098] A data partitioning unit 420 is used to partition the training set according to fields to obtain multiple training subsets;

[0099] A training unit 430 is configured to train a preset neural network using each training subset, calculate a loss value based on the predicted quality scores of the preset neural network and the corresponding actual quality scores, and update the network parameters of the preset neural network using the loss value until the preset neural network converges, thereby obtaining a translation quality evaluation model corresponding to each field;

[0100] The domain classification unit 440 is used to obtain the text to be evaluated and determine the domain category to which the text to be evaluated belongs through the corpus; the text to be evaluated includes the source text to be evaluated and the target translation of the source text to be evaluated;

[0101] The quality evaluation unit 450 is configured to call a corresponding translation quality evaluation model according to the domain category to which the text to be evaluated belongs, perform translation quality evaluation on the text to be evaluated, and obtain a translation quality evaluation result of the text to be evaluated.

[0102] As a further improvement, the field division unit 440 is specifically configured to identify whether corresponding keywords exist in the text to be evaluated based on the keywords of each field marked in the corpus;

[0103] If so, determining the field category to which the text to be evaluated belongs based on the keywords in the text to be evaluated;

[0104] If not, semantic analysis and semantic matching are performed on the text to be evaluated and the corpus, and the field category to which the text to be evaluated belongs is determined according to the obtained semantic matching result.

[0105] As a further improvement, the quality evaluation unit 450 is specifically configured to call the corresponding translation quality evaluation model according to the field category to which the text to be evaluated belongs to perform translation quality evaluation on the text to be evaluated, and output a predicted quality score of the text to be evaluated;

[0106] The predicted quality score of the text to be evaluated is input into a trained linear regression model, the predicted quality score is mapped to an evaluation grade, and a translation quality evaluation result of the text to be evaluated is obtained.

[0107] The training process of the linear regression model includes:

[0108] Translation experts in various fields evaluate the translation quality level based on the source texts in the training subsets of the corresponding fields and the target translations after translating the source texts, thereby obtaining the actual evaluation level of each group of the training subsets;

[0109] Initializing model parameters in the linear regression model;

[0110] Inputting the quality scores of the training subsets of each group into the linear regression model for linear mapping to obtain the predicted evaluation level of the training subsets of each group;

[0111] The grade evaluation loss value is calculated according to the actual evaluation grade and the predicted evaluation grade of each group of the training subsets, and the model parameters of the linear regression model are updated according to the grade evaluation loss value until the linear regression model converges to obtain a trained linear regression model.

[0112] As a further improvement, the apparatus further includes: a feedback unit 460 for collecting feedback results from users when performing translation quality evaluation using the translation quality evaluation model and the linear regression model;

[0113] If the feedback result is disapproval of the translation quality evaluation result and the number of feedbacks exceeds the target number, the model parameters of the translation quality evaluation model and the linear regression model are updated.

[0114] An embodiment of the present application further provides an electronic device, the device including a processor and a memory;

[0115] The memory is used to store program codes and transmit the program codes to the processor;

[0116] The processor is configured to execute the translation quality evaluation method in the aforementioned method embodiment according to instructions in the program code.

[0117] An embodiment of the present application further provides a computer-readable storage medium, which is used to store program code. When the program code is executed by a processor, the translation quality evaluation method in the aforementioned method embodiment is implemented.

[0118] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0119] In the specification of this application and the above-mentioned drawings, the terms "first," "second," "third," "fourth," etc. (if any) are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. In addition, the terms "including" and "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus that includes a series of steps or elements is not necessarily limited to those steps or elements explicitly listed, but may include other steps or elements not explicitly listed or inherent to such process, method, product, or apparatus.

[0120] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships can exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or plural.

[0121] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0122] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0123] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0124] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for executing all or part of the steps of the method described in each embodiment of the present application through a computer device (which can be a personal computer, server, or network device, etc.). The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (full name: Read-Only Memory, English abbreviation: ROM), random access memory (full name: Random Access Memory, English abbreviation: RAM), disk or optical disk, and other media that can store program code.

[0125] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A translation quality evaluation method, characterized in that: include: Obtain source texts in multiple fields and target translated texts from the source texts to build training sets and corpora; Dividing the training set according to fields to obtain multiple training subsets; Training a preset neural network using each group of training subsets, calculating a loss value based on the predicted quality score of the preset neural network and the corresponding actual quality score, and updating the network parameters of the preset neural network using the loss value until the preset neural network converges, thereby obtaining a translation quality evaluation model corresponding to each field; Determining the domain category to which the text to be evaluated belongs through the corpus; The text to be evaluated includes a source text to be evaluated and a target translation after translating the source text to be evaluated; The corresponding translation quality evaluation model is called according to the field category to which the text to be evaluated belongs to perform translation quality evaluation on the text to be evaluated, and a translation quality evaluation result of the text to be evaluated is obtained.

2. The translation quality evaluation method according to claim 1, wherein: Obtain source texts from multiple fields and target texts translated from the source texts to construct a training set, including: Acquire source texts in multiple fields and target translations of the source texts to obtain multiple sentence pairs; Performing a quality score on the sentence pair to generate an actual quality score of the sentence pair; Performing data cleaning on the sentence pairs, and performing word segmentation processing on the sentences in the cleaned sentence pairs; Sentence alignment is performed on the sentence pairs after word segmentation.

3. The translation quality evaluation method according to claim 1, wherein: Determining the domain category to which the text to be evaluated belongs through the corpus includes: Based on the keywords of each field marked in the corpus, identifying whether corresponding keywords exist in the text to be evaluated; If so, determining the field category to which the text to be evaluated belongs based on the keywords in the text to be evaluated; If not, semantic analysis and semantic matching are performed on the text to be evaluated and the corpus, and the field category to which the text to be evaluated belongs is determined according to the obtained semantic matching result.

4. The translation quality evaluation method according to claim 1, wherein: The preset neural network includes an encoder and a decoder; the encoder includes a multi-head attention layer, a residual and normalization layer, and a feedforward layer; When the training subset belongs to a general domain, the decoder includes a parameter sharing layer, and the parameter sharing layer includes a linear layer and an activation layer; When the training subset belongs to a non-general field, the decoder includes a parameter sharing layer and a feature parameter layer, and the feature parameter layer is a linear layer; wherein the non-general field includes the medical field, the legal field, the scientific and technological field, and the financial field.

5. The translation quality evaluation method according to claim 1, wherein: The step of calling the corresponding translation quality evaluation model according to the field category to which the text to be evaluated belongs to perform translation quality evaluation on the text to be evaluated, and obtaining the translation quality evaluation result of the text to be evaluated, includes: Calling the corresponding translation quality evaluation model according to the field category to which the text to be evaluated belongs to perform translation quality evaluation on the text to be evaluated, and outputting a predicted quality score of the text to be evaluated; The predicted quality score of the text to be evaluated is input into a trained linear regression model, the predicted quality score is mapped to an evaluation grade, and a translation quality evaluation result of the text to be evaluated is obtained.

6. The translation quality evaluation method according to claim 5, characterized in that: The training process of the linear regression model includes: Translation experts in various fields evaluate the translation quality level based on the source texts in the training subsets of the corresponding fields and the target translations after translating the source texts, thereby obtaining the actual evaluation level of each group of the training subsets; Initializing model parameters in the linear regression model; Inputting the quality scores of the training subsets of each group into the linear regression model for linear mapping to obtain the predicted evaluation level of the training subsets of each group; The grade evaluation loss value is calculated according to the actual evaluation grade and the predicted evaluation grade of each group of the training subsets, and the model parameters of the linear regression model are updated according to the grade evaluation loss value until the linear regression model converges to obtain a trained linear regression model.

7. The translation quality evaluation method according to claim 5, characterized in that: The method further comprises: Collecting feedback results from users when evaluating translation quality using the translation quality evaluation model and the linear regression model; If the feedback result is disapproval of the translation quality evaluation result and the number of feedbacks exceeds the target number, the model parameters of the translation quality evaluation model and the linear regression model are updated.

8. A translation quality evaluation device, characterized in that: include: A data acquisition unit, configured to acquire source texts in multiple fields and target texts translated from the source texts, and construct training sets and corpora; A data partitioning unit, configured to partition the training set according to fields to obtain multiple training subsets; a training unit, configured to train a preset neural network using each group of the training subsets, calculate a loss value based on the predicted quality scores of the preset neural network and the corresponding actual quality scores, and update network parameters of the preset neural network using the loss value until the preset neural network converges, thereby obtaining a translation quality evaluation model corresponding to each field; A domain classification unit, configured to determine the domain category to which the text to be evaluated belongs through the corpus; The text to be evaluated includes a source text to be evaluated and a target translation after translating the source text to be evaluated; The quality evaluation unit is used to call the corresponding translation quality evaluation model according to the field category to which the text to be evaluated belongs to perform translation quality evaluation on the text to be evaluated, and obtain a translation quality evaluation result of the text to be evaluated.

9. An electronic device, characterized in that: The device includes a processor and a memory; The memory is used to store program code and transmit the program code to the processor; The processor is configured to execute the translation quality evaluation method according to any one of claims 1 to 7 according to instructions in the program code.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium is used to store program code, and when the program code is executed by a processor, the translation quality evaluation method according to any one of claims 1 to 7 is implemented.

Images