A few-shot semantic alignment translation method for low-resource small languages

Abstract

The application discloses a low-resource small language oriented few-shot semantic alignment translation method, and belongs to the technical field of natural language processing. The method comprises the following steps: obtaining a source language small language sentence and a target language reference sample set; performing multi-granularity semantic decomposition on the source language sentence to generate a semantic segment sequence; mapping the semantic segments to a shared semantic space by using a cross-language semantic encoder to obtain a segment vector set; retrieving semantic neighbors from the reference sample set for each segment vector to construct a candidate alignment relationship graph; performing random walk on the candidate alignment relationship graph to calculate the steady-state probability of the alignment relationship, and selecting the alignment path with the highest probability as the semantic alignment result; combining target language segments based on the semantic alignment result to obtain a candidate translation sentence; and performing language model perplexity evaluation on the candidate translation sentence to select the sentence with the lowest perplexity as the final translation output.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, specifically to a few-sample semantic alignment translation method for low-resource minority languages. Background Technology

[0002] Machine translation for less commonly taught languages ​​faces a severe challenge due to the extreme scarcity of bilingual parallel data, making traditional neural machine translation models trained on large-scale parallel corpora unsuitable. Existing few-shot translation solutions typically rely on meta-learning or data augmentation techniques to improve the model's generalization ability under few-shot conditions by constructing different translation tasks or generating pseudo-parallel data. Meta-learning methods simulate few-shot scenarios for parameter optimization during the training phase, but their translation quality is highly dependent on the accuracy of the task distribution assumptions, and performance drops sharply when the language structure of the target less commonly taught language differs significantly from the training task. Data augmentation schemes utilize back-translation or word substitution strategies to expand parallel corpora; however, under extremely limited resources, the sparsity of the initial seed data leads to insufficient semantic diversity of augmented samples and is prone to introducing noise interference. Retrieval-based translation methods directly retrieve similar sentences or phrases from bilingual corpora for translation fragment reuse, but sentence-level retrieval has an extremely low hit rate in scenarios with only a few parallel sentence pairs, while word-level retrieval lacks effective modeling of contextual semantic relationships, often resulting in word silos and poor coherence among translated fragments. From the perspective of cross-language semantic alignment, existing solutions generally adopt a single-granularity word segmentation strategy to process source language text. For minority languages, due to their rich morphological variations and lack of mature word segmentation tools, single-granularity segmentation cannot simultaneously capture stem semantics, affix variations, and character-level word formation rules, resulting in the loss of semantic representation information. At the same time, segment-level alignment decisions are usually made in isolation, failing to fully utilize the potential semantic connections between segments and global consistency constraints, which can easily lead to the propagation of errors in local alignment deviations. Summary of the Invention

[0003] The purpose of this invention is to provide a few-sample semantic alignment translation method for low-resource minority languages, which solves the problem of incomplete capture of semantic information in minority languages ​​caused by single-granularity semantic decomposition, and the problem of poor translation coherence caused by isolated alignment decisions between segments.

[0004] To achieve the above objectives, this invention provides the following technical solution: This invention provides a few-sample semantic alignment translation method for low-resource minority languages. This method can achieve high-quality semantic alignment and translation using only a small number of reference samples, significantly reducing the dependence on large-scale parallel corpora of minority languages. The method includes the following: We obtain sentences in the source language and a reference sample set of the target language containing multiple aligned sentence pairs. By performing boundary recognition, sentence segmentation and length filtering on the text in the source language, and by automatically extracting high-confidence parallel sentence pairs from bilingual parallel corpora or seed word pairs, we construct a concise and high-quality reference sample set to provide a reliable basis for subsequent retrieval.

[0005] Multi-granularity semantic decomposition is performed on sentences in the source language (minority language) to generate a sequence of semantic fragments. Preferably, character-level segmentation, sub-word-level segmentation, and word-level segmentation are performed simultaneously. The resulting character sequences, sub-word sequences, and word sequences are merged in their original order into a unified sequence of semantic fragments, and each fragment is labeled with its granularity type. Fragments of different granularity types are weighted and fused according to preset weight coefficients during merging, thereby fully capturing the semantic information of the minor language at the glyph, word formation, and vocabulary levels, effectively mitigating the semantic loss problem caused by vocabulary gaps and out-of-vocabulary words.

[0006] A cross-lingual semantic encoder is used to map the semantic segment sequence to a shared semantic space, resulting in a set of segment vectors. Specifically, the semantic segment sequence is input into a transformer-based cross-lingual pre-trained model to obtain the context-related vector for each segment. These vectors are then subjected to layer normalization and dimensionality compression to form a fixed-dimensional set of segment vectors. Through cross-lingual semantic encoding, words in the source language segments and target language reference samples are mapped to a unified semantic space, eliminating inter-language distribution differences and making semantic similarity calculation more accurate.

[0007] For each fragment vector, semantic nearest neighbors are retrieved from the reference sample set to construct a candidate alignment graph. In a preferred embodiment of this invention, after L2 norm normalization of the fragment vector and the reference sample word vectors, the cosine similarity between them is calculated. The top K source language words with the highest similarity are selected as the semantic nearest neighbors of the fragment. Each nearest neighbor records the corresponding target language word and alignment weight, forming a candidate alignment graph with fragments and source language words as nodes and candidate alignment relationships as edges. This method utilizes a small number of alignment samples to quickly establish dense cross-linguistic candidate associations, providing a rich selection space for global alignment.

[0008] A random walk is performed on the candidate alignment graph to calculate the steady-state probability of each alignment relationship, and the alignment path with the highest probability is selected as the semantic alignment result. Preferably, a random walk strategy with restart is adopted. The node probability mass is initialized on the candidate alignment graph, and the probability distribution is updated iteratively multiple times according to a preset transition probability matrix. In each iteration, the probability mass is reset back to the initial distribution with a preset probability until the probability distribution converges to obtain the steady-state probability of each node. For each segment, the word with the highest steady-state probability is selected from the candidate target words connected to it as the alignment result. The candidate alignments corresponding to each segment are sorted in descending order of steady-state probability, and conflicts are checked and eliminated in turn to obtain the conflict-free globally optimal alignment path. Random walks can propagate semantic association information in the graph structure, avoiding alignment errors caused by local greed, and the restart mechanism further prevents the walk from getting stuck in local loops, significantly improving the accuracy and stability of alignment.

[0009] Based on the semantic alignment results, the corresponding target language segments are extracted from the reference sample set, and then concatenated sequentially according to the original order of the source language semantic segments. Repeated or conflicting words are merged and disambiguated to generate multiple candidate translation sentences, each corresponding to a different alignment selection combination.

[0010] The perplexity of candidate translation sentences is evaluated using a language model, and the sentence with the lowest perplexity is selected as the final translation output. Specifically, each candidate translation sentence is input into the target language's language model, the average negative log-likelihood is calculated, and the exponent is taken to obtain the perplexity value. The candidate translation sentence with the smallest perplexity value is selected as the final translation output. Through fluency optimization using the language model, the translation that best conforms to the natural expression habits of the target language is selected from multiple semantically reasonable candidates, ensuring the readability and naturalness of the translation results in low-resource scenarios where a large amount of training data is lacking.

[0011] The technical effects and advantages provided by the present invention in the above technical solution are as follows: This approach employs a multi-granularity semantic decomposition strategy, simultaneously performing character-level, sub-word-level, and word-level segmentation on sentences in the source language's minority languages. The segmentation results from different granularities are merged into a unified semantic fragment sequence, with each granularity type assigned a pre-defined weight coefficient for weighted fusion. Character-level segmentation captures morphological changes at the word formation level in minority languages, sub-word-level segmentation effectively handles out-of-vocabulary words and affix variations, and word-level segmentation preserves complete word meaning. The synergistic effect of these three granularities allows the semantic fragment sequence to more comprehensively cover the linguistic characteristics of minority languages. Compared to single-granularity word segmentation schemes, this method avoids excessive reliance on mature word segmentation tools. Even in minority language scenarios lacking rich corpora, it can still extract complete semantic information from multiple levels—morphology, affixes, and meaning—providing a higher information-density input representation for subsequent cross-language mapping.

[0012] A restarted random walk mechanism is employed to calculate the steady-state probability of alignment relationships on the candidate alignment graph. The entire alignment decision process no longer processes each semantic segment in isolation; instead, all candidate alignment relationships are constructed as an interconnected graph structure. Transitions between nodes are simulated through probability mass propagation, mimicking the diffusion of semantic associations. The restarted random walk strategy resets a portion of the probability mass to the initial distribution in each iteration with a preset probability, effectively preventing the random walk from getting trapped in local structures. This ensures that the alignment selection fully considers the global semantic coordination among candidates. Based on the converged steady-state probability distribution, a conflict-free high-probability alignment path is selected. The alignment results of each segment achieve semantic coherence constraints in a globally optimal sense, resolving the contextual breaks and word conflicts caused by greedy segment-by-segment alignment. The word order logic of the output translation sequence is more consistent with the expression habits of the target language, resulting in smoother semantic connections within sentences. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0014] Figure 1 This is a flowchart of a few-sample semantic alignment translation method for low-resource, less common languages; Figure 2 This is a flowchart of the process for obtaining sentences in less commonly spoken languages ​​and constructing a reference sample set for the target language; Figure 3 This is a flowchart of translation candidate generation and perplexity evaluation based on semantic alignment. Detailed Implementation

[0015] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0016] See Figure 1This invention provides a few-sample semantic alignment translation method for low-resource minority languages. The method includes: acquiring a source language sentence and a target language reference sample set containing multiple aligned sentence pairs; performing multi-granular semantic decomposition on the source language sentence to generate a semantic fragment sequence; mapping the semantic fragment sequence to a shared semantic space using a cross-language semantic encoder to obtain a fragment vector set; for each fragment vector, retrieving semantic nearest neighbors from the reference sample set to construct a candidate alignment graph; performing a random walk on the candidate alignment graph, calculating the steady-state probability of each alignment relationship, and selecting the alignment path with the highest probability as the semantic alignment result; based on the semantic alignment result, extracting the corresponding target language fragments from the reference sample set and combining them into candidate translation sentences; evaluating the perplexity of the candidate translation sentences using a language model, and selecting the sentence with the lowest perplexity as the final translation output.

[0017] In specific implementation, please refer to Figure 2 The method for obtaining source language sentences and a target language reference sample set containing multiple aligned sentence pairs is as follows. The method for identifying minority language text boundaries from the original corpus is to extract continuous text segments belonging to that minority language from the mixed corpus based on the Unicode encoding range of the minority language or a pre-trained minority language recognition classifier. For the identified minority language text, sentence segmentation is performed based on punctuation marks indicating sentence termination in the minority language, including periods, question marks, exclamation marks, and some stop words indicating semantic completion, to obtain the original sentence sequence. The segmented original sentence sequence is then length-filtered, retaining sentences within a preset length range as source language sentences. Optionally, the preset length range can be set with a lower limit of 3 words and an upper limit of 100 words. The lower limit is set based on the fact that text fragments shorter than 3 words are usually difficult to form complete semantic units; the upper limit is set based on the fact that sentences exceeding 100 words will significantly increase the computational cost of subsequent semantic decomposition and alignment, and that excessively long sentences in low-resource minority languages ​​are often caused by corpus segmentation errors. The retained sentences are those in the source language minority language.

[0018] When constructing the target language reference sample set, a preset number of bilingual aligned sentence pairs are randomly selected from the existing bilingual parallel corpus. This preset number can be configured to be 5000 pairs. Noise filtering is applied to the extracted sentence pairs. The noise filtering rules include: removing sentence pairs containing unrecognizable garbled characters in either the source or target language sentences; removing sentence pairs where the language identification results of the source and target language sentences are inconsistent with the expected language; and removing sentence pairs where the ratio of the source language sentence length to the target language sentence length exceeds the range [0.5, 2.0]. After noise filtering, the remaining sentence pairs undergo length normalization. The normalization method is as follows: if the length of any sentence in a sentence pair exceeds the preset maximum length of 100 words, that sentence is truncated to 100 words from the end, while the other sentence remains unprocessed, maintaining the alignment relationship. If the length of the source language sentence is less than the preset minimum length of 3 words, the sentence pair is discarded. The set of sentence pairs processed in the above way forms the target language reference sample set.

[0019] Optionally, in another implementation, a target language reference sample set is constructed through automatic seed word pair extraction. Seed word pairs from less commonly spoken languages ​​to the target language are collected from publicly available bilingual dictionaries, including Wikipedia and GlobalVoices bilingual vocabulary lists. The collected seed word pairs are then filtered by frequency, retaining only those pairs that appear more than a preset minimum frequency of 5 times in the monolingual corpus, thus obtaining a seed word pair set.

[0020] Parallel sentence pairs are automatically extracted from monolingual corpora using a word alignment algorithm based on seed word pairs. The word alignment algorithm takes as input a set of monolingual corpora of the target language, a set of monolingual corpora of the target language, and a set of seed word pairs. The output is parallel sentence pairs and a confidence score for each pair. The core architecture of the word alignment algorithm includes a word co-occurrence counting module, a bidirectional translation probability estimation module, and a sentence pair confidence scoring module. The word co-occurrence counting module performs word co-occurrence statistics on all sentence pairs in the monolingual corpus, counting the frequency of co-occurrence between source and target language words for each candidate sentence pair. The bidirectional translation probability estimation module estimates the translation probability from source to target language and from target to source language using an iterative expectation-maximization algorithm based on the co-occurrence frequency and seed word pairs. The sentence pair confidence scoring module calculates the confidence score for each candidate sentence pair based on the bidirectional translation probability.

[0021] The specific execution steps of the word alignment algorithm are as follows. Initially, the probability values ​​of the corresponding seed word pairs in the bidirectional translation probability table are initialized to a high value, set to 0.99. The probability values ​​of the remaining word pairs are initialized to a uniform distribution value, which is equal to 1 divided by the total number of words in the target language. In each iteration, the E-step is executed. For each sentence pair, the posterior alignment probability of each source language word and target language word is calculated based on the current translation probability table. The M-step is executed. The co-occurrence frequency is updated based on the alignment posterior probabilities obtained in the E-step, and the translation probability table is re-estimated. The number of iterations is pre-set to 5. After the iteration is completed, the bidirectional translation probability estimation module outputs the translation probability table from the source language to the target language, where each entry is denoted as... , indicating source language words Translate into target language words The probability of translation from the target language to the source language, and a translation probability table, wherein each entry is denoted as . , representing target language words Translated into source language words The probability of.

[0022] The confidence score of candidate parallel sentence pairs is scored using a sentence pair confidence scoring module. The confidence score for each candidate sentence pair is calculated using the following formula:

[0023] in, This represents the set of words contained in the source language sentence within a candidate sentence pair. This represents the number of words in the word set of the source language sentence. This represents the set of words contained in the target language sentence within a candidate sentence pair. This indicates the number of words in the target language sentence word set. Iterate through each word in the word set of the source language sentence. Iterate through every word in the target language sentence word set. Indicates source language words The maximum probability of being able to translate into all words in a sentence in the target language. Indicates taking target language words The maximum probability of translating into all words in the source language sentence. Confidence score. The value ranges from 0 to 1. This score integrates the alignment quality of the source language sentence to the target language sentence and the alignment quality of the target language sentence to the source language sentence. After scoring the automatically extracted parallel sentence pairs with confidence, sentence pairs with confidence scores higher than a threshold are retained as a reference sample set. Optionally, the threshold is set to 0.3. This threshold is obtained by adjusting the threshold on a manually annotated development set containing 1000 parallel sentence pairs so that the F1 score of the automatically extracted parallel sentence pairs reaches its maximum. The corresponding threshold is 0.3. All retained sentence pairs with confidence scores higher than 0.3 constitute the target language reference sample set.

[0024] In practice, the process of multi-granular semantic decomposition of sentences in the source language minority languages ​​and generating semantic fragment sequences is achieved through three parallel segmentation processes: character-level segmentation, sub-word-level segmentation, and word-level segmentation. The segmentation results are then merged in their original order, and the weights of fragments at different granularities are weighted and fused during the merging process.

[0025] The character-level segmentation is implemented by iterating through every Unicode code point in the sentence of the source language, treating each independent Unicode character as a character segment, and recording the starting and ending character indices of the segment within the sentence. All character segments are arranged in the order of their appearance in the sentence to form a character sequence.

[0026] Sub-word segmentation relies on a sub-word segmentation model pre-trained on a corpus of minority languages. The sub-word segmentation model employs a byte-pair encoding-based BPE model architecture. The training steps of the BPE model are as follows: Collect a corpus of minority languages, encompassing all available texts for that language. Standardize all texts using Unicode and add a special ending symbol to the end of each word. Count the frequency of all adjacent symbol pairs in the corpus, merge the most frequent pairs, update the vocabulary, and repeat the merging process until the preset number of merging operations is reached. The number of merging operations is set to 8000, based on the following: In translation evaluation experiments on a minority language development set, three settings were tested with 4000, 8000, and 12000 merging operations, and the highest subsequent segment alignment recall rate was achieved with 8000 merging operations. After training, the BPE model vocabulary consists of the initial characters and the merged sub-words. When segmenting sentences of a minor language source language using a trained BPE model, the sentence is fed into the BPE model. The BPE model outputs a sequence of sub-words according to the trained merging rules and records the starting and ending character indices of each sub-word segment in the sentence.

[0027] Word-level segmentation relies on a minor language dictionary and a forward maximum matching segmentation algorithm. The minor language dictionary is constructed by extracting all minor language sentences from the bilingual parallel corpus corresponding to the target language reference sample set, calculating word frequencies, retaining words with frequencies exceeding a preset threshold of 5, and adding all function words and closed word classes from the minor language to form the word-level segmentation dictionary. The forward maximum matching segmentation algorithm is executed as follows: The maximum word length parameter is set to 15 characters. The basis for setting the maximum word length to 15 is that, in the word length statistics of the minor language corpus, more than 99% of the words are no longer than 15 characters. Starting from the beginning of the minor language sentence in the source language, a string of length 15 characters is taken from the current pointer position. If this string matches a word in the word-level segmentation dictionary, the matched word is segmented, and the pointer is moved forward by the length of that word. If no match is found, the string length is decreased and matching continues until the length is reduced to 1 character. At this point, the current character is segmented as a single word, and the pointer is moved forward one position. Repeat the above process until the end of the sentence to obtain the word sequence, and record the starting character index and ending character index of each word segment in the sentence.

[0028] The method for merging character sequences, subword sequences, and word sequences into a unified semantic fragment sequence is as follows: A range mapping table indexed by character positions is constructed. Each fragment in the character, subword, and word sequences is traversed, and each fragment represents a continuous text range through its starting and ending character indices. For fragments with identical text ranges, fragment merging is performed. During merging, a fusion weight coefficient is assigned to the merged fragment. The formula for calculating the fusion weight coefficient is:

[0029] in, This indicates the preset base weight for character granularity. This indicates the preset base weight for the sub-word granularity. This indicates the preset base weight for word granularity. The range of values ​​for is the real number interval [0,1]. The range of values ​​for is the real number interval [0,1]. The range of values ​​is the real number interval [0,1], and the three satisfy the constraint conditions. . This is an indicator of the existence of character-level fragments within a text range. The value is 1 if it is set to 1, otherwise the value is 0. This is an indicator of the existence of sub-word-level fragments; it indicates the presence of sub-word-level fragments within the text range. The value is 1 if it is set to 1, otherwise the value is 0. This serves as an indicator of the presence of word-level segments within a text range. The value is 1 if it is not specified, and 0 otherwise. (Fusing weight coefficient) This reflects the importance weights of the merged segments in multi-granularity information fusion. Its calculation essentially involves normalizing and averaging the granularity weights that contribute to the fusion. Character granularity has preset base weights. Sub-word granularity preset basic weight And word granularity preset base weight The specific values ​​are determined through a grid search on a small language translation validation set. The grid search has a search step size of 0.1 and a constraint that the sum of the three weights is 1. The translation process is executed under each weight combination, and the BLEU score of the output translation is calculated. The weight combination with the highest BLEU score is selected as the final setting. The optimal combination obtained through the search is... , , .

[0030] For segments without overlapping intervals, they constitute a semantic segment independently. Their fusion weight coefficient is directly taken as the preset base weight for the corresponding granularity type, i.e., the character segment weight is... The weight of the sub-word fragment is The word fragment weight is All merged or individual fragments are arranged in ascending order according to the starting character index in the source language sentence, forming a semantic fragment sequence. Each fragment in the sequence carries its fusion weight coefficient and granularity type label.

[0031] In practice, a cross-lingual semantic encoder is used to map semantic segment sequences to a shared semantic space, resulting in a set of segment vectors. The core of the cross-lingual semantic encoder is a transformer-based cross-lingual pre-trained model.

[0032] The cross-lingual pre-trained model architecture consists of stacked Transformer encoders, comprising 12 Transformer layers. Each layer includes a multi-head self-attention mechanism sublayer and a feedforward neural network sublayer, both employing residual connections and layer normalization. The multi-head self-attention mechanism sublayer has 12 attention heads, each with a dimension of 64. The feedforward neural network sublayer has a hidden layer dimension of 3072, using GELU as the activation function. The embedding layer of the cross-lingual pre-trained model consists of three sublayers: word embedding, position embedding, and language embedding. The word embedding sublayer maps each input word symbol to a 768-dimensional vector; the position embedding sublayer generates a 768-dimensional position vector using sinusoidal position encoding; and the language embedding sublayer assigns a learnable 768-dimensional language embedding vector based on the language of the input sequence. Independent language embedding vectors are provided for both the source language (minority language) and the target language.

[0033] The pre-training phase of the cross-lingual pre-trained model utilizes a joint optimization of cross-lingual masking language modeling and translation language modeling tasks. The masking language modeling task randomly selects 15% of the input symbols from the monolingual corpus for masking, with 80% replaced by masking labels, 10% replaced by random words, and 10% left unchanged. The model is required to predict the masked symbols. The translation language modeling task is performed on parallel sentence pairs, exposing a portion of the symbols from the source language sentences to the model, requiring the model to generate target language symbols based on the bilingual context. The pre-training dataset consists of large-scale monolingual corpora of source languages ​​and minority languages ​​collected from CommonCrawl and Wikipedia, as well as target language monolingual corpora, and approximately 1 million bilingual parallel corpora collected from the OPUS parallel corpus. The training optimizer is Adam, with a learning rate set to [value missing]. The batch size is 8192 symbols, and the total training steps are 1 million. After the cross-language pre-trained model completes pre-training, its parameters are saved and loaded into the translation method for direct use, without requiring end-to-end fine-tuning for the translation task.

[0034] Before inputting the semantic segment sequence into the cross-lingual pre-trained model, the semantic segment sequence is segmented into sub-words. The text within each semantic segment is further segmented into sub-word symbols using the BPE segmentation model. In the vocabulary of the cross-lingual pre-trained model, each sub-word symbol corresponds to a unique identifier. Each position in the semantic segment sequence corresponds to a sub-word symbol, with a sequence start symbol added at the beginning of the sequence and a sequence end symbol added at the end. Simultaneously, based on the language category of the semantic segment sequence, a corresponding language embedding identifier is assigned to the entire input sequence. The cross-lingual pre-trained model performs forward computation on the input sequence, outputting a context-related vector for each sub-word symbol, with a dimension of 768.

[0035] For a semantic segment consisting of multiple sub-word symbols, average pooling is used to obtain the context-related vector of the semantic segment. Suppose a semantic segment covers the context from position... Arrive at the location Given a continuous sequence of words, the cross-lingual pre-trained model outputs vectors at these positions. Then the context-related vector of this semantic segment is:

[0036] in, The index variable is used for traversal, and its value range is an interval. Integers within, Indicates the cross-language pre-trained model in the 1st... The output vector is a 768-dimensional vector at each position. For a semantic segment consisting of only a single word symbol, its context-related vector is directly taken from the output vector of that word symbol.

[0037] After obtaining the context-related vector for each semantic segment, layer normalization and dimensionality compression are performed on the context-related vector. Dimensionality compression is achieved through a linear projection layer, which consists of a learnable projection matrix and a learnable bias vector. The formula for calculating the linear projection layer is:

[0038] in, The context-related vector representing a semantic segment has 768 dimensions; The layer normalization operation is represented by a normalization dimension of 768. The layer normalization process includes learnable scaling parameter vectors and offset parameter vectors, both of which have a dimension of 768. The initial value of the scaling parameter vector is all 1s, and the initial value of the offset parameter vector is all 0s. The normalized vector is then multiplied by the projection matrix. This represents the learnable projection matrix, with dimension 1. The parameters of the projection matrix are learned during the training process; This represents a learnable bias vector with a dimension of 256. This represents the obtained fixed-dimensional segment vector, with a dimension of 256. The 256-dimensional setting is based on the fact that in the evaluation of word alignment tasks from a source language to a target language, the retrieval hit rate was calculated by setting the semantic segment vector dimension to 128, 256, and 512 dimensions respectively. The retrieval hit rate under the 256-dimensional setting was the same as that under the 512-dimensional setting, and the encoding time per sentence was reduced by 40% compared to the 512-dimensional setting. Therefore, the fixed dimension was set to 256. The fixed-dimensional segment vectors corresponding to all semantic segments constitute the segment vector set, which is used for subsequent semantic nearest neighbor retrieval.

[0039] In practice, for each fragment vector, the process of retrieving semantic nearest neighbors from the reference sample set and constructing a candidate alignment graph is as follows.

[0040] Each source language word in the reference sample set is pre-mapped into a fixed-dimensional reference word vector using a cross-lingual semantic encoder. The dimension of the reference word vector is the same as that of the fragment vector, both being 256-dimensional. All reference word vectors are subjected to L2 norm normalization. The normalized reference word vectors are stored in a reference vector index library, which is constructed using the FAISS vector retrieval library. The index type is an exact inner product index, internally normalized using L2 normalization to make the inner product equivalent to cosine similarity. For each fragment vector in the fragment vector set, L2 norm normalization is also performed, dividing each element of the fragment vector by the L2 norm of the fragment vector to obtain a normalized fragment vector.

[0041] For each normalized segment vector, calculate the cosine similarity between the normalized segment vector and each normalized reference word vector. The formula for calculating cosine similarity is:

[0042] in, This represents a normalized segment vector with a dimension of 256, where each element takes values ​​in the real number field and the vector magnitude is 1. This represents the normalized reference word vector, which has a dimension of 256, with each element taking values ​​in the real number field and a vector magnitude of 1. This represents the dot product operation between the normalized segment vector and the normalized reference word vector. The result is equal to the cosine of the angle between these two 256-dimensional vectors, with a value range of [-1, 1].

[0043] After calculating the cosine similarity between the normalized fragment vector and all normalized reference word vectors, the source language words corresponding to the top K reference word vectors are selected as the semantic neighbors of the fragment vectors, sorted from high to low according to the cosine similarity. The value of K is set to 10. The basis for setting K to 10 is as follows: on a validation set of 500 manually annotated words from a minority language to the target language, the retrieval recall rate was evaluated by setting K to 3, 5, 10, 15, and 20 respectively. When K increases from 5 to 10, the proportion of true aligned words being hit in the candidate set increases from 86% to 93%, while when it increases from 10 to 15, it only increases to 94%. The computational cost and recall benefit brought by increasing K reach the optimal balance when K=10.

[0044] For each semantic nearest neighbor, obtain the target language words aligned with the source language words in the reference sample set, and record the correspondence between the source language words and the corresponding target language words. Each semantic nearest neighbor forms a candidate alignment relationship, which includes the source language semantic fragment, the nearest source language words, the nearest target language words, and the alignment weight, where the alignment weight is the cosine similarity value.

[0045] A candidate alignment graph is constructed, which is a bipartite graph structure. The node set of the candidate alignment graph consists of two parts: the first part is slice nodes, each slice node corresponding to a semantic segment in a sentence of the source language; the second part is word nodes, each word node corresponding to a source language word that is retrieved as a semantic nearest neighbor by at least one semantic segment. The edges of the candidate alignment graph are undirected, each edge connecting a slice node and a word node. An edge exists if the source language word corresponding to the word node is a semantic nearest neighbor of the semantic segment corresponding to the slice node. Each edge is assigned a weight, equal to the alignment weight recorded in the candidate alignment graph. In the candidate alignment graph, each word node also stores the target language word information corresponding to it.

[0046] The process of performing a random walk on the candidate alignment graph and calculating the steady-state probability of each alignment is as follows: Initialize the probability mass of each node on the candidate alignment graph. The initial probability distribution is set as follows: for each slice node, the initial probability mass is assigned as 1; for each word node, the initial probability mass is assigned as 0. Record the initialized probability mass distribution as an initial distribution vector, which is used for the random walk strategy with restart.

[0047] Construct the transition probability matrix of the candidate alignment graph. For any node, the transition probability is calculated as follows: if the node has neighboring nodes, the transition probability from the node to each neighboring node is equal to the edge weight of the edge between the node and its neighboring nodes divided by the sum of the edge weights between the node and all its neighboring nodes; if the node has no neighboring nodes, the transition probability from the node to itself is 1. This transition probability matrix satisfies that the sum of the elements of each row vector is 1.

[0048] The iterative update process of the random walk strategy with restart is as follows: at the first... In the next iteration, the first The probability distribution vector of the next iteration is multiplied by the transition probability matrix to obtain an intermediate probability distribution vector. Each element of the intermediate probability distribution vector is then multiplied by... Then multiply the initial distribution vector by Then, it is added element-wise to the scaled intermediate probability distribution vector to obtain the first... The probability distribution vector of the next iteration. Restart probability. The restart probability is set to 0.15. The rationale for setting the restart probability to 0.15 is that the general range for random walk restart probabilities is [0.1, 0.2]. In graph-based word alignment tasks, this is adopted... Three sets of values, 0.1, 0.15, and 0.2, were used to compare the alignment quality. The alignment accuracy is highest when the value is 0.15. The iteration termination condition is set as follows: when the difference in the L1 norm of the probability distribution vectors of two consecutive iterations is less than a preset convergence threshold, the probability distribution is considered to have converged, and the iteration stops. The convergence threshold is set to... The convergence threshold is set to The rationale is that, at this threshold, the steady-state probability remains stable to six decimal places, and further reducing the threshold will not affect the subsequent alignment selection results. The steady-state probability of each node is obtained after convergence.

[0049] For each slice node, obtain all word nodes connected to the slice node through edges, and read the values ​​from the steady-state probabilities of the word nodes. Among all word nodes connected to the slice node, select the word node with the highest steady-state probability as the alignment result of the slice node, and extract the target language words stored in the word nodes to form a single fragment alignment from the slice node to the target language words.

[0050] The process of selecting the alignment path with the highest probability as the semantic alignment result is as follows: Individual segments of all semantic fragments are aligned and combined into an initial alignment set. Different fragment nodes in the initial alignment set may correspond to the same target language word; such repeated alignment constitutes a conflict. The candidate alignment relationships corresponding to each semantic fragment are sorted in descending order of steady-state probability, resulting in a sorted candidate alignment list for each semantic fragment. A set of occupied target language words is maintained, and the processing order for all semantic fragments is iterated sequentially: For the sorted candidate alignment list of a semantic fragment, starting from the highest-ranked one, it is checked whether the target language word for the current candidate alignment already exists in the set of occupied target language words; if there is no conflict, the current candidate alignment is selected as the semantic fragment alignment result, and the target language word is added to the set of occupied target language words; if there is a conflict, the current candidate alignment is skipped, and the second-highest-ranked one is checked. The above operation is performed on all semantic fragments to obtain a conflict-free alignment path as the semantic alignment result.

[0051] In specific implementation, please refer to Figure 3 The process of extracting the corresponding target language fragments from the reference sample set based on semantic alignment results and combining them into candidate translation sentences includes three steps: initial translation sequence generation, merging and disambiguation of repeated and conflicting words, and generation of multiple candidate translation sentences.

[0052] The semantic alignment result contains the correspondence between each semantic segment in the source language sentence and its aligned target language word. Following the original order of the semantic segments in the source language sentence, the aligned target language words for each semantic segment are sequentially obtained. These target language words are then concatenated according to the semantic segment order to obtain a preliminary translation sequence. Each target language word occupies one position in the preliminary translation sequence.

[0053] The initial translation sequence performs merging and disambiguation processing on duplicate or conflicting words. The merging of duplicate words involves iterating through all adjacent target language words in the initial translation sequence. If two target language words have identical text, they are merged into one target language word, and the position of the merged target language word in the sequence is updated to the first position before the merge. Subsequent word indices are shifted forward. This merging process is repeated until no adjacent words with identical text remain in the sequence. The disambiguation of conflicting words addresses verb conjugation, noun singular / plural, and adjective / noun gender / number / case agreement conflicts in the target language. The disambiguation process involves detecting grammatical consistency constraints between adjacent target language words in the initial translation sequence. These constraints are determined by a pre-constructed target language grammatical consistency rule table. This table defines the collocation rules for any two parts of speech in the target language, including gender / number agreement rules for adjectives and nouns, number agreement rules for verbs and subjects, and gender / number agreement rules for articles and nouns. If two adjacent words in the initial translation sequence have grammatical consistency constraints and conflicting word forms, disambiguation is initiated: the form of the word in the preceding position remains unchanged, while the form of the word in the following position is adjusted to conform to the constraints specified in the grammatical consistency rule table. This morphological adjustment is performed by querying the target language's morphological change lexicon, which includes plural forms of nouns, personal conjugations of verbs, and gender and number forms of adjectives. If the conflict cannot be resolved through morphological adjustment, two branch sequences are generated at the conflicting position. One branch sequence retains the word in the preceding position and deletes the word in the following position, while the other branch sequence deletes the word in the preceding position and retains the word in the following position.

[0054] The method for generating multiple candidate translation sentences is as follows: For each semantic segment, retain the R candidate alignment words with the highest steady-state probability ranking in the candidate alignment graph, with R set to 3. The rationale for setting R to 3 is that when R increases from 1 to 3, the proportion of candidate translation sentences covering correctly translated sentences increases from 75% to 92%, while increasing from 3 to 5 only increases it to 93%. The computational overhead and coverage gain from increasing R reach an optimal balance when R=3. The R candidate alignment words for all semantic segments are combined and enumerated to generate... There are 10 candidate combinations, where M represents the total number of semantic segments. For each candidate combination generated by the enumeration, the target language words in the combination are concatenated according to the original order of the semantic segments. The concatenated results are then processed sequentially to merge duplicate words and disambiguate conflicting words, resulting in the corresponding candidate translation sentences. All candidate translation sentences are deduplicated, removing duplicate candidate translation sentences with identical text, resulting in a set of candidate translation sentences.

[0055] The process of evaluating the perplexity of candidate translation sentences using a language model and selecting the sentence with the lowest perplexity as the final translation output is as follows.

[0056] The target language language model employs an autoregressive language model based on a Transformer decoder. The language model architecture consists of 12 stacked Transformer decoder layers. Each decoder layer contains a masked multi-head self-attention mechanism sublayer, a cross-attention mechanism sublayer, and a feedforward neural network sublayer. In this language model, the cross-attention mechanism sublayer is not computationally involved and is set as an identity mapping. The masked multi-head self-attention mechanism sublayer has 12 attention heads, each with a dimension of 64. The hidden layer of the feedforward neural network sublayer has a dimension of 3072, and the activation function is GELU. The embedding layer has a dimension of 768, and learned positional encoding is used. The vocabulary size is the same as the target language's sub-vocabulary vocabulary, set to 32,000 sub-word types.

[0057] The language model training process is as follows: A large-scale monolingual corpus of the target language is collected, with a corpus size of no less than 10 million sentences. The corpus sources include news articles, encyclopedias, and transcribed spoken dialogues. Each sentence in the monolingual corpus is segmented into subwords, and the target language BPE model, which uses the same process as the translation, is used. The BPE model vocabulary size is 32,000 words. The training objective is to minimize the negative log-likelihood loss of the next word prediction. The optimizer used is Adam, and the learning rate is set to... The batch size was 128 sentences, the maximum sequence length was set to 256 sub-word symbols, and the number of training iterations was set to 10 rounds, traversing the entire monolingual corpus once in each round. After training, the language model parameters were saved and loaded into the language model evaluation stage of the translation method.

[0058] Each candidate translation sentence is input into the trained target language model. The language model receives the word sequence after segmentation of the candidate translation sentence as input and calculates the conditional probability at each step using autoregression. The perplexity value of the candidate translation sentence is calculated using the following formula:

[0059] in, This indicates the total number of sub-word symbols contained in the candidate translation sentence after it has been segmented into a sub-word sequence; For traversal indices, the value range is from 1 to... Integers; Indicates the position of the word in the sub-word sequence. Sub-word symbols at each position; Indicates the position of the word in the sub-word sequence. The context sequence consisting of all the sub-word symbols preceding each position; Representing a language model in a given context Predicting the first under the condition The sub-word symbol at each position is The conditional probability is output by the language model output layer after softmax normalization, and the output dimension is 32000. The natural logarithm represents the probability predicted by the language model. This represents the summation of the log-conditional probabilities over all positions in the entire word sequence; The average negative log-likelihood of the candidate translation sentences is represented by taking the negative value of the summation and dividing it by the length of the sub-word sequence. This represents an exponential function with the natural constant e as its base. The perplexity value is obtained by exponentially taking the mean negative log-likelihood. Confusion value The value of is a positive real number. The lower the perplexity value, the higher the fluency of the candidate translation sentence under the target language model.

[0060] Calculate the perplexity value for each candidate translation sentence, compare the perplexity values ​​of all candidate translation sentences, and select the candidate translation sentence with the smallest perplexity value as the final translation output.

[0061] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A few-sample semantic alignment translation method for low-resource minority languages, characterized in that, The method includes: Obtain sentences in the source language (minority language) and a target language reference sample set containing multiple aligned sentence pairs; Perform multi-granular semantic decomposition on sentences in the source language minority languages ​​to generate semantic fragment sequences; A cross-language semantic encoder is used to map a sequence of semantic fragments to a shared semantic space, resulting in a set of fragment vectors. For each fragment vector, semantic nearest neighbors are retrieved from the reference sample set to construct a candidate alignment graph; Perform a random walk on the candidate alignment graph, calculate the steady-state probability of each alignment relationship, and select the alignment path with the highest probability as the semantic alignment result; Based on the semantic alignment results, the corresponding target language segments are extracted from the reference sample set and combined into candidate translation sentences; The perplexity of the candidate translation sentences is evaluated using a language model, and the sentence with the lowest perplexity is selected as the final translation output. The steps of performing multi-granular semantic decomposition on sentences in the source language minority languages ​​to generate a sequence of semantic fragments specifically include: The source language minority language sentence is simultaneously segmented at the character level, sub-word level, and word level to obtain character sequences, sub-word sequences, and word sequences, respectively. The character sequence, sub-word sequence, and word sequence are merged into a unified semantic fragment sequence in their original order, with each fragment labeled with its granularity type; In the multi-granularity semantic decomposition, fragments of different granularity types are weighted and fused according to preset weight coefficients during merging.

2. The few-sample semantic alignment translation method for low-resource minority languages ​​according to claim 1, characterized in that, The steps of obtaining source language minority language sentences and target language reference sample sets containing multiple aligned sentence pairs specifically include: The text boundaries of minority languages ​​are identified from the original corpus. Sentences are segmented based on punctuation marks and stop words. The segmented sentences are then filtered by length, and sentences with a length within a preset range are retained as source language minority language sentences. A predetermined number of bilingual aligned sentence pairs are randomly selected from the bilingual parallel corpus. The selected sentence pairs are then subjected to noise filtering and length normalization to form a target language reference sample set.

3. The few-sample semantic alignment translation method for low-resource minority languages ​​according to claim 1, characterized in that, The step of mapping semantic fragment sequences to a shared semantic space using a cross-language semantic encoder to obtain a set of fragment vectors specifically includes: Semantic fragment sequences are input into a transformer-based cross-lingual pre-trained model, which outputs a context-related vector for each fragment. For each segment's vector, perform layer normalization and dimensional compression to obtain a set of segment vectors with fixed dimensions.

4. The few-sample semantic alignment translation method for low-resource minority languages ​​according to claim 1, characterized in that, The step of retrieving semantic nearest neighbors from the reference sample set and constructing a candidate alignment graph for each fragment vector specifically includes: For each fragment vector, calculate its cosine similarity with the vector of each source language word in the reference sample set; The top K source language words with the highest similarity are selected as the semantic nearest neighbors of the segment. Each nearest neighbor records the corresponding target language word and alignment weight, forming a candidate alignment relationship graph, where nodes are the segment and source language words, and edges are candidate alignment relationships.

5. A few-sample semantic alignment translation method for low-resource minority languages ​​according to claim 4, characterized in that, Before calculating the cosine similarity, the fragment vector and the reference sample word vector are subjected to L2 norm normalization.

6. The few-sample semantic alignment translation method for low-resource minority languages ​​according to claim 1, characterized in that, The step of performing a random walk on the candidate alignment graph and calculating the steady-state probability of each alignment relationship specifically includes: Initialize the probability mass of each node on the candidate alignment graph, and update the probability distribution multiple times according to the preset transition probability matrix. In each iteration, each node distributes its probability mass equally to its neighboring nodes and receives the probability mass from its neighboring nodes until the probability distribution converges to obtain the steady-state probability of each node. For each segment, the word with the highest steady-state probability is selected from the candidate target words connected to it as the alignment result for that segment.

7. A few-sample semantic alignment translation method for low-resource minority languages ​​according to claim 6, characterized in that, The random walk employs a random walk strategy with restart, in which the probability mass is reset to the initial distribution with a preset probability in each iteration.

8. A few-sample semantic alignment translation method for low-resource minority languages ​​according to claim 1, characterized in that, The step of extracting corresponding target language fragments from the reference sample set based on semantic alignment results and combining them into candidate translation sentences specifically includes: Following the original order of semantic segments in the source language sentences, the target language words obtained by aligning each segment are sequentially spliced ​​together to obtain a preliminary translation sequence; Repeated or conflicting words in the initial translation sequence are merged and disambiguated to generate multiple candidate translation sentences, each corresponding to a different alignment selection combination.

Images