Low-resource language machine translation method based on active learning

Abstract

The invention provides a low-resource language machine translation method based on active learning, which belongs to the technical field of natural language processing, and comprises the following steps: extracting a token sequence corresponding to each sentence from an initial data set; inputting the token sequence corresponding to each sentence into a preset model, and outputting a feature vector corresponding to each sentence; constructing a feature vector set according to the feature vectors, and selecting a core set from the feature vector set; training a preset translation model by using the core set to obtain an initial translation model, and enabling the initial translation model to output the uncertainty ranking of each core sentence in the core set; acquiring initial data corresponding to each core sentence from the initial data set by utilizing a selection function according to the uncertainty ranking; labeling and serializing all the initial data to obtain a target training data set, and training the initial translation model by using the target training data set to obtain a target translation model. The accuracy of machine translation in a low-resource environment can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a low-resource language machine translation method based on active learning. Background Art

[0002] The task of low-resource language machine translation (MT) refers to the use of neural network technology to perform machine translation in the case of limited target language data to improve translation quality and efficiency. One of the main challenges facing machine translation in a low-resource environment is the lack of high-quality parallel corpora available for training. In traditional machine translation frameworks, a large amount of parallel corpora is essential to ensure that the model can learn an accurate mapping from the source language to the target language. However, for many language pairs, the scarcity of this resource greatly limits the performance of neural network models.

[0003] Although some existing methods have achieved remarkable results to a certain extent, they still cannot completely break through the limitation of sparse language data. For example, when the quality of the original data itself is low, the quality of the data generated by data augmentation may not be high, and the data augmentation method requires additional computing resources to generate or process this data. When using transfer learning, the model may experience catastrophic forgetting, that is, while adapting to low-resource language pairs, the model may forget the knowledge of the original high-resource language pairs. At the same time, its disadvantage is that the lower the similarity between languages, the worse the effect of transfer learning. The most obvious disadvantage of semi-supervised learning is that the noise of unlabeled data will cause the performance of the model to deteriorate. In low-resource language translation, active learning has obvious advantages over the above methods in improving data utilization efficiency, accelerating model convergence speed, and improving the generalization ability of the model. Existing active learning methods include Core-Set (CS), Discriminative Active Learning (DAL), and Round Trip Translation Likelihood (RTTL). The core idea of ​​the first two methods is to select a data subset to maximize the coverage of the entire data distribution. The RTTL method trains two opposite models. After the source sentence passes through the two models, the translated sentence is compared with the source sentence to analyze the value of the data. However, these current methods all have some problems. The problem with the DAL and CS methods is that although the selected data sets can guarantee the coverage (diversity) of the data to a certain extent, they cannot meet the density requirements of the model for some types of data. RTTL relies on the mutual translation of the two models. This method not only increases the amount of calculation, but also amplifies the noise in the data, interfering with the performance of the model.

[0004] Active Learning (AL) is a machine learning strategy that aims to actively query the most valuable samples from unlabeled data for annotation, so as to maximize the improvement of model performance while minimizing the annotation cost. Therefore, active learning happens to be an effective method for solving low-resource language machine translation tasks. Summary of the invention

[0005] In view of this, the purpose of the present invention is to overcome the deficiencies in the prior art and provide a low-resource language machine translation method based on active learning to improve the accuracy of machine translation in a low-resource environment.

[0006] In a first aspect, the present application provides a low-resource language machine translation method based on active learning, comprising:

[0007] Obtain an initial data set, and extract a token sequence corresponding to each sentence in the initial data set;

[0008] Input the token sequence corresponding to each sentence into the preset Doc2Vec model for training, and output the feature vector corresponding to each sentence;

[0009] Constructing a feature vector set according to the feature vector, and selecting a core set from the feature vector set using a Core-Set method;

[0010] Using the core set to train a preset translation model to obtain an initial translation model, and allowing the initial translation model to output an uncertainty ranking of each core sentence in the core set;

[0011] According to the uncertainty ranking, obtaining initial data corresponding to each of the core sentences from the initial data set using a selection function;

[0012] All the initial data are labeled and serialized to obtain a target training data set, and the initial translation model is trained using the target training data set to obtain a target translation model.

[0013] In one implementation, extracting a token sequence corresponding to each sentence in the initial data set includes:

[0014] Performing data cleaning and word segmentation operations on the initial data set to obtain a token for each sentence in the initial data set;

[0015] All tokens are deduplicated and a dictionary is generated. Each sentence in the initial data set is serialized according to the dictionary to obtain a token sequence for each sentence.

[0016] In one implementation, the training process of the preset Doc2Vec model includes:

[0017] The Doc2Vec model is used as a feature extraction model, and the feature extraction model is trained using the token sequence to obtain a trained feature extraction model, and the trained feature extraction model is made to output a feature vector corresponding to the token sequence;

[0018] Calculating the log-likelihood loss of the token sequence according to the feature vector, and determining whether the change amplitude of the log-likelihood loss in multiple consecutive training iterations is less than a preset threshold;

[0019] If it is greater than or equal to the preset threshold, adjusting the parameters of the initial model and continuing training;

[0020] If it is less than the preset threshold, the feature extraction model corresponding to the maximum value of the multiple log-likelihoods is used as the preset Doc2Vec model.

[0021] In one implementation, the log-likelihood calculation formula is:

[0022]

[0023] Among them, w t is the target word at time step t, w t-k ,…,w t+k is the token of the target word context, d is the unique document vector associated with the entire document, and p(w t |w t-k ,…,w t+k , d) is calculated by the Softmax function.

[0024] In one implementation, the selecting a core set from the feature vector set using a Core-Set method includes:

[0025] A Core-Set method is used to select multiple core sentences from the feature vector set, and an initial core set is constructed using the multiple core sentences; the data distribution of the initial core set is the same as or similar to the data distribution of the initial data set;

[0026] The core sentences in the initial core set are marked to obtain parallel sentence pairs, and the parallel sentence pairs are serialized to obtain the core set.

[0027] In one implementation, the step of causing the initial translation model to output the uncertainty ranking of each core sentence in the core set includes:

[0028] Using the initial translation model, obtaining the probability distribution of the token corresponding to each of the core sentences;

[0029] According to the entropy value calculation formula, the entropy value of each of the core sentences is calculated using the probability distribution;

[0030] According to the entropy value, the uncertainty ranking of each core sentence is determined.

[0031] In one embodiment, the entropy value calculation formula is:

[0032]

[0033] Among them, H(S) is the entropy value, S represents the sentence, n is the total number of tokens in the sentence, and w i Represents the token in the sentence, E[I(w i )] represents the average amount of information of each word in the sentence, E[I(w i )] is the amount of information I(w i ), P(w i ) is the probability of the i-th word predicted by the model.

[0034] In one implementation, obtaining initial data corresponding to each of the core sentences from the initial data set using a selection function according to the uncertainty ranking includes:

[0035] Determining a target number of initial data corresponding to each core sentence according to the uncertainty ranking, the initial data being the top N sentences in the initial data set with the smallest feature similarity to the core sentence;

[0036] The initial data is obtained from the initial data set according to the target quantity by using the selection function.

[0037] In one implementation, the formula for the selection function is:

[0038]

[0039] or

[0040]

[0041] Among them, i is the order of uncertainty ranking, starting from 0 and ending at M-1, N is the total number of data to be selected, and M is the size of the initial data set.

[0042] In one embodiment, the step of continuing to train the initial translation model using the target training data set to obtain a target translation model includes:

[0043] Convert the target training data set into an initial input sequence, and inject position information into each data in the initial input sequence through position encoding to obtain a target input sequence;

[0044] Inputting the target input sequence into the initial translation model, and outputting a target language sentence token sequence corresponding to the target input sequence;

[0045] Optimizing the parameters of the initial translation model by using a logarithmic probability evaluation function of the target sentence in the target language sentence token sequence;

[0046] If the continuous log-likelihood difference is greater than a preset threshold, the parameters of the initial translation model are continuously adjusted and training is continued, and the log-likelihood is re-evaluated after each training;

[0047] If multiple consecutive log-likelihood differences are all smaller than the preset threshold, the training is stopped and the target translation model is obtained.

[0048] The embodiments of the present invention have the following beneficial effects:

[0049] The present invention provides a low-resource language machine translation method based on active learning. Considering the coverage of an initial data set, a core set is selected from the feature vector set to ensure that the core data has the distribution characteristics of the initial data set. Then, the uncertainty of the data is obtained based on the output analysis of the model for the data. According to the uncertainty of the model for different data, different numbers of data are dynamically selected according to a selection function, so that the model can learn more focusedly. The whole process does not lose the distribution characteristics of the entire data set, and at the same time, the coverage and density of the data are balanced, thereby alleviating the data sparsity problem of the model on the most uncertain data and improving the accuracy of machine translation in a low-resource environment.

[0050] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and understandable, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0052] Figure 1 A flowchart of a low-resource language machine translation method based on active learning provided in an embodiment of the present application;

[0053] Figure 2A flowchart of a token sequence acquisition method provided in an embodiment of the present application;

[0054] Figure 3 A flowchart of a preset Doc2Vec model training method provided in an embodiment of the present application;

[0055] Figure 4 A schematic diagram of a core set data distribution effect provided in an embodiment of the present application;

[0056] Figure 5 A flowchart of an uncertainty ranking method provided in an embodiment of the present application;

[0057] Figure 6 A schematic diagram of a process for obtaining initial data provided in an embodiment of the present application;

[0058] Figure 7 A schematic diagram of an initial data distribution effect provided in an embodiment of the present application. DETAILED DESCRIPTION

[0059] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0060] The terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0061] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein in the specification of the template are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0062] Example 1

[0063] See also Figure 1 , Figure 1 A flowchart of a low-resource language machine translation method based on active learning provided in this embodiment, which can be used to improve the accuracy of machine translation in a low-resource environment, includes:

[0064] S101, obtaining an initial data set, and extracting a token sequence corresponding to each sentence in the initial data set.

[0065] The initial dataset D is a monolingual sentence set, that is, it only contains source language sentences.

[0066] See also Figure 2 , step S01 comprises:

[0067] S1011. Perform data cleaning and word segmentation operations on the initial data set to obtain a token for each sentence in the initial data set.

[0068] S1012, deduplicate all tokens and generate a dictionary, and serialize each sentence in the initial data set according to the dictionary to obtain a token sequence of each sentence.

[0069] First, the initial data set D is cleaned and segmented. The BPE algorithm can be used for segmentation, that is, each sentence in the initial data set is segmented at the word level. Each sentence will generate several tokens, that is, each sentence is a token sequence. A token is the smallest unit in text processing, which can be a word, a character, or a word.

[0070] All tokens obtained after word segmentation in all sentences in the dataset are deduplicated and a dictionary is generated, where each token corresponds to a unique numeric index. All sentences in the dataset are serialized according to the dictionary, that is, each token is replaced by its unique numeric index. This is done to prepare for subsequent model training and machine translation work.

[0071] S102: Input the token sequence corresponding to each sentence into a preset Doc2Vec model for training, and output a feature vector corresponding to each sentence.

[0072] See also Figure 3 , step S102 includes:

[0073] S1021. Use the Doc2Vec model as a feature extraction model, use the token sequence to train the feature extraction model to obtain a trained feature extraction model, and make the trained feature extraction model output a feature vector corresponding to the token sequence.

[0074] S1022. Calculate the log-likelihood loss of the token sequence according to the feature vector, and determine whether the change range of the log-likelihood loss in multiple consecutive training iterations is less than a preset threshold.

[0075] S1023: If it is greater than or equal to the preset threshold, adjust the parameters of the initial model and continue training.

[0076] S1024: If it is less than the preset threshold, the initial model corresponding to the maximum value among the multiple log-likelihoods is used as the preset Doc2Vec model.

[0077] First, based on the given initial data set D, a Doc2Vec model is trained as a feature extraction model. Usually, the Doc2Vec model method in the Gensim library is used, and the training is iterated 60 times to obtain a trained preset Doc2Vec model.

[0078] The goal of the whole process is to maximize the log-likelihood of the token sequence of the sentence given the source sentence token sequence. The log-likelihood function is shown in Formula 1.

[0079]

[0080] Among them, w t is the target word at time step t, w t-k ,…,w t+k is the context token of the target word, d is the unique document vector associated with the entire document, and p(w t |w t-k ,…,w t+k , d) is calculated by the Softmax function.

[0081] The core goal of model training is to adjust the parameters so that the log-likelihood L reaches the maximum value. In order to ensure that the log-likelihood has reached the maximum value, the Doc2Vec model iteration number parameter in the Gensim library is continuously adjusted during the training process. When the difference between consecutive log-likelihoods is less than the preset value, it means that the model training has stabilized and the cost of continuing training is too high. Finally, it is found that the Doc2Vec model tends to stabilize after about 60 iterations.

[0082] Then, the trained preset Doc2Vec model is used to convert the token sequence corresponding to each sentence, and each sentence is converted into a feature vector with a dimension of 100, that is, the feature vector corresponding to all sentences in the initial data set D.

[0083] S103 , constructing a feature vector set according to the feature vector, and selecting a core set from the feature vector set using a Core-Set method.

[0084] The feature vectors corresponding to all sentences in the initial data set D are constructed into a feature vector set, and then a plurality of core sentences are selected from the feature vector set using the Core-Set method, and an initial core set (Data_0) is constructed using the plurality of core sentences. The data distribution of the initial core set (Data_0) is the same as or similar to the data distribution of the initial data set, so that the initial core set (Data_0) can be used to represent the distribution of the entire initial data set D.

[0085] Then, the core sentences in the initial core set are annotated to obtain parallel sentence pairs, and the parallel sentence pairs are serialized to obtain the core set. The annotation can be performed manually or by machine to obtain a labeled core set D0. The data in the core set D0 are parallel sentence pairs, which refer to sentence pairs that are translated into each other on the bilingual parallel corpus.

[0086] The data selected by the Core-Set method is mapped to a 2D plane through principal component analysis (PCA). The distribution effect is as follows: Figure 4 shown.

[0087] S104, using the core set to train a preset translation model to obtain an initial translation model, and allowing the initial translation model to output an uncertainty ranking of each core sentence in the core set.

[0088] See also Figure 5 , step S104 includes:

[0089] S1041. Using the initial translation model, obtain the probability distribution of the token corresponding to each of the core sentences.

[0090] S1042. Calculate the entropy value of each of the core sentences using the probability distribution according to the entropy value calculation formula.

[0091] S1043. Determine the uncertainty ranking of each core sentence according to the entropy value.

[0092] Among them, the initial translation model M0 has a 6-layer encoder and a 6-layer decoder. After the core set D0 is input into the 6-layer encoder and the 6-layer decoder, the final output tensor is obtained, that is, the context-related representation vector. These representation vectors contain the context information of each token in the target sequence. The representation vector is further passed through a linear layer and a Softmax layer to obtain the probability distribution P(X) of each token in the sentence. The linear layer and the Softmax layer are not counted as independent layers of the model M0.

[0093] After obtaining the probability distribution, the entropy value of each sentence is calculated using the entropy value calculation formula, that is, Formula 2:

[0094]

[0095] Among them, H(S) is the entropy value, S represents the sentence, n is the total number of tokens in the sentence, and w i Represents the token in the sentence, E[I(w i )] represents the average amount of information of each word in the sentence, E[I(w i )] is the amount of information I(w i ), P(w i ) is the probability of the i-th word predicted by the model.

[0096] The entropy value represents the average uncertainty of the sentence. The uncertainty ranking is arranged from large to small. From Formula 2, we can see that the larger the entropy value, the higher the uncertainty of the sentence, which means that the model is more uncertain about the data, and the more valuable such data is for improving the model.

[0097] S105 . According to the uncertainty ranking, using a selection function, obtain initial data corresponding to each of the core sentences from the initial data set.

[0098] See also Figure 6 , step S105 includes:

[0099] S1051. Determine the target number of initial data corresponding to each core sentence according to the uncertainty ranking, where the initial data is the first N sentences in the initial data set that have the smallest feature similarity with the core sentence.

[0100] S1052: Utilize the selection function to obtain the initial data from the initial data set according to the target quantity.

[0101] After obtaining the uncertainty ranking of each sentence, a selection function may be used to select corresponding initial data from the initial data set D, where the selection function may be as shown in Formula 3 or 4:

[0102]

[0103] or

[0104]

[0105] Among them, i is the order of uncertainty ranking, starting from 0 and ending at M-1, N is the total number of data to be selected, and M is the size of the initial data set D.

[0106] From the initial data set D, according to the uncertainty ranking order of each sentence in the core set D0, select data similar to each data in the core set D0 in the initial data set D. The number of data selected in the initial data set D corresponding to each data in the core set D0 is f(i, N, M), and finally the initial data Data_1 with the number N is obtained.

[0107] The basis for selecting similar data can be determined based on the feature vector set.

[0108] Calculate the distance between each two data in the feature vector set. This problem is equivalent to copying this 100-dimensional set into two copies, one set is represented by A = {a1, a2, ..., a m}, B={b1,b2,…,b n}. Each of these a i and b i All R 100 A point in space.

[0109] Then use Formula 5 to calculate the Euclidean distance between every two points:

[0110]

[0111] Among them, a ik is the vector a i The kth component of ik Similarly, the calculated distance is finally saved in a matrix (array).

[0112] Select the data with the smallest Euclidean distance from this data f(i, N, M) as the data for subsequent model training, i.e., the initial data. The distribution effect of the selected data in the 2D plane is as follows: Figure 7 As shown in the figure. Since the data is too dense, although some data cannot be seen to be clearly clustered, after rigorous data science calculations, some data have been selected to cluster around the most uncertain data in the model.

[0113] The data selected according to this selection function not only retains the data distribution of the original data set, making the data coverage high, but also ensures the data density for the model, making the subsequent training of the model more focused.

[0114] S106: All the initial data are labeled and serialized to obtain a target training data set, and the initial translation model is further trained using the target training data set to obtain a target translation model.

[0115] The process of continuing to train the initial translation model includes:

[0116] Convert the target training data set into an initial input sequence, and inject position information into each data in the initial input sequence through position encoding to obtain a target input sequence;

[0117] Input the target input sequence into the initial translation model, and output the target language sentence toke corresponding to the target input sequence n sequence;

[0118] Optimizing the parameters of the initial translation model by using a logarithmic probability evaluation function of the target sentence in the target language sentence token sequence;

[0119] If the continuous log-likelihood difference is greater than a preset threshold, the parameters of the initial translation model are continuously adjusted and training is continued, and the log-likelihood is re-evaluated after each training;

[0120] If multiple consecutive log-likelihood differences are all smaller than the preset threshold, the training is stopped and the target translation model is obtained.

[0121] The initial data Data_1 is annotated manually or by machine to obtain the final training data D1, and then the final training data D1 is converted into the input target sequence x = [x1, x2, ..., x n ], each word x i is mapped to a dimension d model The embedding vector e i By embedding the matrix , where |V| is the size of the vocabulary.

[0122] The position information is then explicitly injected through position encoding. Position encoding is implemented by adding the input word embedding vector to the position encoding to obtain the final representation of the input z i =e i +PE i . These representations are then fed into the encoder, which is stacked with multiple layers, each containing a multi-head self-attention mechanism and a feedforward neural network (FNN). The self-attention mechanism captures the relationship between words in a sentence by calculating the query (Q), key (K), and value (V) matrices. The multi-head attention mechanism captures richer semantic information by calculating multiple attention heads in parallel and concatenating their results through formulas 9 and 10. Subsequently, the feedforward neural network performs a word-by-word nonlinear transformation on the output of the self-attention layer through formula 11 to enhance the expressive power of the model.

[0123] In the decoder part, an additional attention layer is added compared to the encoder, which is specifically used to focus on the output of the encoder. When the decoder generates the vocabulary of the target language, it first captures the dependencies between the generated vocabulary through the self-attention layer, and then focuses on the encoder output, that is, the representation of the input sentence, through the encoder-decoder attention layer. This process uses a mask mechanism to ensure that the decoder can only see previously generated vocabulary when generating vocabulary at each step. The output of the decoder is mapped to the vocabulary probability distribution of the target language through a linear layer, and finally the final vocabulary of the translation is generated through the softmax function. The final output sequence y = [y1, y2, ..., y m ] is generated by beam search, as shown in formulas 12 and 13, z dec is the output of the decoder, is the weight matrix for the output vocabulary size, b o is the bias vector.

[0124] During training, the model minimizes the cross entropy loss by maximizing the log probability of the target sentence, as shown in Formula 14, where y t is the vocabulary of the target language, and finally the target translation model M1 is obtained.

[0125] e i =W e ·x i (6)

[0126]

[0127]

[0128]

[0129] MultiHead(Q,K,V)=Concat(head1,...,head h )W O (10)

[0130] FFN(x)=max(0,xW1+b1)W2+b2 (11)

[0131] logits=z dec W o +b o (12)

[0132] P(y t |y ＜t , X) = softmax(logits) (13)

[0133]

[0134] This embodiment considers the coverage of the initial data set, selects a core set from the feature vector set, ensures that the core data has the distribution characteristics of the initial data set, and then derives the uncertainty of the data based on the output analysis of the model for this data. According to the uncertainty of the model for different data, different numbers of data are dynamically selected according to the selection function, so that the model can learn more focusedly. The whole process does not lose the distribution characteristics of the entire data set, but also balances the coverage and density of the data, alleviates the data sparsity problem of the model on the most uncertain data, and improves the accuracy of machine translation in low-resource environments.

[0135] This embodiment uses the data from English to Hebrew (he) in the WMT23 data set as training data. The method of this embodiment is compared with the Random, Core-Set, and DAL strategies. The model is trained on 10k data in total and verified on 2k data. Finally, the model trained by this method is tested on 3k sentences. The results are evaluated by using the BLEU↑, TER↓, and CHRF↑ indicators (the upward arrow represents that the higher the score of the indicator, the better the translation quality, and vice versa). The BLEU, TER, and CHRF indicators are directly calculated using the sacrebleu library. The results are shown in Table 1:

[0136] Table 1

[0137]

[0138] It can be seen from Table 1 that the method of this embodiment has obvious advantages in the BLEU↑ and TER↓ indicators, and is also very close to the best result in the CHRF↑ indicator, so the method has achieved good results in the three indicators.

[0139] In all examples shown and described herein, any specific values ​​should be interpreted as merely exemplary and not as limiting, and thus other examples of the exemplary embodiments may have different values.

[0140] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0141] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the present invention. It should be pointed out that for ordinary technicians in this field, several modifications and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention.

Claims

1. A low-resource language machine translation method based on active learning, characterized in that: include: Obtain an initial data set, and extract a token sequence corresponding to each sentence in the initial data set; Input the token sequence corresponding to each sentence into the preset Doc2Vec model for training, and output the feature vector corresponding to each sentence; Constructing a feature vector set according to the feature vector, and selecting a core set from the feature vector set using a Core-Set method; Using the core set to train a preset translation model to obtain an initial translation model, and allowing the initial translation model to output an uncertainty ranking of each core sentence in the core set; According to the uncertainty ranking, obtaining initial data corresponding to each of the core sentences from the initial data set using a selection function; All the initial data are labeled and serialized to obtain a target training data set, and the initial translation model is further trained using the target training data set to obtain a target translation model.

2. The low-resource language machine translation method based on active learning according to claim 1, characterized in that: The step of extracting a token sequence corresponding to each sentence in the initial data set includes: Performing data cleaning and word segmentation operations on the initial data set to obtain a token for each sentence in the initial data set; All tokens are deduplicated and a dictionary is generated. Each sentence in the initial data set is serialized according to the dictionary to obtain a token sequence for each sentence.

3. The low-resource language machine translation method based on active learning according to claim 1, characterized in that: The training process of the preset Doc2Vec model includes: The Doc2Vec model is used as a feature extraction model, and the feature extraction model is trained using the token sequence to obtain a trained feature extraction model, and the trained feature extraction model is made to output a feature vector corresponding to the token sequence; Calculating the log-likelihood loss of the token sequence according to the feature vector, and determining whether the change amplitude of the log-likelihood loss in multiple consecutive training iterations is less than a preset threshold; If it is greater than or equal to the preset threshold, adjusting the parameters of the feature extraction model and continuing training; If it is less than the preset threshold, the feature extraction model corresponding to the maximum value of the multiple log-likelihoods is used as the preset Doc2Vec model.

4. The low-resource language machine translation method based on active learning according to claim 3, characterized in that: The log-likelihood calculation formula is: Among them, w t is the target word at time step t, w t-k ,…,w t+k is the token of the target word context, d is the unique document vector associated with the entire document, and p(w t |w t-k ,…,w t+k , d) is calculated by the Softmax function.

5. The low-resource language machine translation method based on active learning according to claim 1, characterized in that: The method of selecting a core set from the feature vector set by using the Core-Set method includes: A Core-Set method is used to select multiple core sentences from the feature vector set, and an initial core set is constructed using the multiple core sentences; the data distribution of the initial core set is the same as or similar to the data distribution of the initial data set; The core sentences in the initial core set are marked to obtain parallel sentence pairs, and the parallel sentences are serialized to obtain the core set.

6. The low-resource language machine translation method based on active learning according to claim 5, characterized in that: The step of causing the initial translation model to output an uncertainty ranking of each core sentence in the core set includes: Using the initial translation model, obtaining the probability distribution of the token corresponding to each of the core sentences; According to the entropy value calculation formula, the entropy value of each of the core sentences is calculated using the probability distribution; According to the entropy value, the uncertainty ranking of each core sentence is determined.

7. The low-resource language machine translation method based on active learning according to claim 6, characterized in that: The entropy value calculation formula is: Among them, H(S) is the entropy value, S represents the sentence, n is the total number of tokens in the sentence, and w i Represents the token in the sentence, E[I(w i )] represents the average amount of information of each word in the sentence, E[I(w i )] is the amount of information I(w i ), P(w i ) is the probability of the i-th word predicted by the model.

8. The low-resource language machine translation method based on active learning according to claim 6, characterized in that: The step of obtaining initial data corresponding to each of the core sentences from the initial data set using a selection function according to the uncertainty ranking includes: Determining a target number of initial data corresponding to each core sentence according to the uncertainty ranking, the initial data being the top N sentences in the initial data set with the smallest feature similarity to the core sentence; The initial data is obtained from the initial data set according to the target quantity by using the selection function.

9. The low-resource language machine translation method based on active learning according to claim 8, characterized in that: The formula of the selection function is: or, Among them, i is the order of uncertainty ranking, starting from 0 and ending at M-1, N is the total number of data to be selected, and M is the size of the initial data set.

10. The low-resource language machine translation method based on active learning according to claim 1, characterized in that: The step of continuing to train the initial translation model using the target training data set to obtain a target translation model includes: Convert the target training data set into an initial input sequence, and inject position information into each data in the initial input sequence through position encoding to obtain a target input sequence; Inputting the target input sequence into the initial translation model, and outputting a target language sentence token sequence corresponding to the target input sequence; Optimizing the parameters of the initial translation model by using a logarithmic probability evaluation function of the target sentence in the target language sentence token sequence; If the continuous log-likelihood difference is greater than a preset threshold, the parameters of the initial translation model are continuously adjusted and training is continued, and the log-likelihood is re-evaluated after each training; If multiple consecutive log-likelihood differences are all smaller than the preset threshold, the training is stopped and the target translation model is obtained.

Images