Text translation method and system based on granular ball clustering

Abstract

The invention relates to a text translation method and system based on granular ball clustering. The method comprises the following steps: regularly acquiring a text data set by adopting a crawler technology and preprocessing the text data set; performing word segmentation processing on the obtained text data by utilizing a word segmentation tool, and updating a word bank by utilizing words obtained by the word segmentation processing; on the basis of a pre-trained large language model, coding words in the word bank to obtain word vector representation of the words; according to the word vector representation of the words in the word bank, adaptively generating a synonym set of the words in the word bank by using a granular ball clustering algorithm; using synonyms of words in the training text to replace the words to generate adversarial samples of the training text; constructing a training set according to the training text and the adversarial sample, and training a text translation model based on a large language model based on the constructed training set; and translating the to-be-translated text through the trained text translation model. According to the invention, the accuracy and reliability of translation can be improved.

Description

Technical Field

[0001] The invention belongs to the technical field of natural language processing, and in particular relates to a text translation method and system based on granular sphere clustering. Background Art

[0002] With the acceleration of globalization, the importance of machine translation systems in cross-language communication has become increasingly prominent. Traditional machine translation methods can only learn the mapping relationship between the source language and the target language to achieve translation functions. However, in practical applications, the diversity, complexity and structural changes of the input language often lead to unstable or erroneous translation results.

[0003] In the field of natural language processing, adversarial examples have been shown to improve the robustness of models. However, existing machine translation systems rarely incorporate adversarial training, and these models often have problems with translation stability when faced with diverse expressions such as vocabulary replacement and syntactic structure changes in the input. When synonym replacement or slightly different sentence structures appear in the input, the translation results output by the system may show obvious deviations, affecting the accuracy and reliability of the translation.

[0004] In this context, the generation of text adversarial samples requires adding imperceptible tiny perturbations without changing the original semantics of the sentence to successfully mislead the model to output incorrect results. However, the synonym sets used to generate adversarial samples in existing word-level attacks are limited. Word-level attacks rely on finding synonyms to replace words in the victim sample, which means that the quality of the synonym set directly affects the effectiveness of the attack. Most methods require pre-defined thresholds for the construction of synonym sets, such as the similarity distance between words, which limits the range of available synonyms. Summary of the invention

[0005] In order to solve the problems existing in the background technology, one aspect of the present invention provides a text translation method based on granular sphere clustering, comprising:

[0006] S1: Use crawler technology to regularly obtain text datasets and preprocess them;

[0007] S2: Use a word segmentation tool to perform word segmentation on the acquired text data, and use the words obtained by the word segmentation to update the word library;

[0008] S3: Encode the words in the vocabulary based on the pre-trained large language model to obtain the word vector representation;

[0009] S4: Adaptively generate synonym sets of words in the vocabulary using the granular sphere clustering algorithm according to the word vector representation of the words in the vocabulary;

[0010] S5: Generate adversarial samples of training text by replacing the words in the training text with their synonyms;

[0011] S6: Construct a training set based on the training text and the adversarial sample, and train a text translation model based on the large language model based on the constructed training set; translate the text to be translated by using the trained text translation model.

[0012] Preferably, the word segmentation processing of the acquired text data using a word segmentation tool includes:

[0013] For English text, the NLTK library in Python is used for word segmentation, and for Chinese text data, the jieba tool is used for word segmentation.

[0014] Preferably, the method of dynamically generating a synonym set of words in the vocabulary using a granular sphere clustering algorithm comprises:

[0015] S41: All words in the vocabulary are used as the initial spherulite set;

[0016] S42: Calculate the cosine distance matrix between word vectors of all words in the spheroidal set; select two words a and b with the largest difference in the cosine distance matrix; and put the two words a and b with the largest semantic difference into two new spheroidal sets respectively;

[0017] S43: traverse each word c in the original spherulite set, if the cosine distance between word c and word a is greater than the cosine distance between word c and word b, then add word c to the new spherulite set where word b is located, otherwise, add word c to the new spherulite set where word a is located;

[0018] S44: Repeat steps S42 to S44 for each new set of spheres until the density of the set of spheres reaches a set threshold, and output multiple sets of spheres, each of which represents a synonym set.

[0019] Preferably, the density of the pellet set includes:

[0020]

[0021] Where De(gb) represents the spheroidal set, n represents the number of words in the spheroidal set, and x i represents the word vector of the i-th word; c0 represents the center of the spherulite set; d() represents the cosine distance.

[0022] Preferably, step S5 comprises:

[0023] S51: Construct an objective function with the goal of introducing minimal perturbations to mislead the text translation model: Where P represents the objective function, X represents the training text; X adv represents the adversarial text after the perturbation; g(X,X adv ) represents text X and text Xadv The semantic similarity of ; m(X) represents the semantic confusion of text X;

[0024] S52: using a synonym set of words in the training text to randomly replace words in the training text to generate an initial population of the genetic algorithm;

[0025] S53: Based on the generated initial population of the genetic algorithm, the constructed objective function is solved using the genetic algorithm to obtain the optimal adversarial text.

[0026] Another aspect of the present invention provides a text translation system based on particle-sphere clustering, comprising a processor and a memory; the memory is used to store a computer program; the processor is connected to the memory and is used to execute the computer program stored in the memory, so that the text translation system based on particle-sphere clustering executes the text translation method based on particle-sphere clustering.

[0027] Another aspect of the present invention provides a computer-readable storage medium storing a program, wherein when the program is executed by a processor, the text translation method based on particle sphere clustering is implemented.

[0028] The present invention has at least the following beneficial effects

[0029] Compared with the prior art, the present invention introduces more diverse adversarial samples in model training through the synonym set generated by granular sphere clustering, so that the translation system can effectively cope with complex input situations such as synonym replacement or sentence structure changes, and the translation stability of the system under diversified expressions is significantly improved. By introducing the generation of adversarial samples, the present invention is more in line with the actual translation task requirements and improves the adaptability and accuracy of the translation model. By introducing adversarial samples in training, the model can better handle input changes, reduce translation errors caused by synonym replacement or input structure changes, and significantly improve the translation of professional terms and precise expressions. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a schematic diagram of the method flow of the present invention. DETAILED DESCRIPTION

[0031] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0032] Embodiment 1, in order to improve the accuracy of the translation model and solve the problem that the existing machine translation method is unstable when facing synonym replacement and input structure changes, the present invention proposes a text translation method based on sphere clustering, and the specific technical scheme is as follows:

[0033] Get word vectors:

[0034] Use crawler technology to collect data on the web, and then use LSTM or BERT model for training to obtain word vectors;

[0035] Generate dynamic synonym sets based on granular sphere clustering:

[0036] The pre-trained word vectors are clustered using the granular clustering algorithm. This method clusters semantically similar words into the same granular ball based on the similarity between word vectors through density adaptiveness to form a synonym set. Using the granular synonym set, there is no need to set a synonym number threshold, thus obtaining a more accurate multi-granular synonym set.

[0037] Adversarial sample generation:

[0038] Using the granular synonym set, we combined different strategies to generate candidate word sets, added perturbations to the words in the original training data by replacing synonyms, and generated adversarial samples containing synonym transformations. The generated adversarial samples were used together with the original data for translation model training, enabling the model to cope with diverse expressions and maintain translation stability.

[0039] Introducing adversarial samples for model training:

[0040] During the training process, the model calculates the translation loss of adversarial samples, and adversarial sample generation, model training, and loss calculation form a cycle that continuously improves model performance through multiple iterations. The system can autonomously adjust strategies and improve translation quality based on real-time training feedback.

[0041] Example 2, please refer to Figure 1 One aspect of the present invention provides a text translation method based on granular sphere clustering, comprising:

[0042] S1: Use crawler technology to regularly obtain text datasets and preprocess them;

[0043] In this embodiment, the preprocessing of the acquired text data set includes: using crawler technology to acquire the Internet text data set, and cleaning the data set, removing some illegal symbols, or irrelevant data such as URLs, email addresses, and numbers through regular matching.

[0044] S2: Use a word segmentation tool to perform word segmentation on the acquired text data, and use the words obtained by the word segmentation to update the word library;

[0045] Preferably, the word segmentation processing of the acquired text data using a word segmentation tool includes:

[0046] For English text, the NLTK library in Python is used for word segmentation, and for Chinese text data, the jieba tool is used for word segmentation.

[0047] S3: Encode the words in the vocabulary based on the pre-trained large language model to obtain the word vector representation;

[0048] In this embodiment, the Lstm or Bert model is used to pre-train the data (Lstm can be trained using classification tasks, and Bert can be trained using the initial pre-training tasks (filling in words and predicting the context)), and word vectors are extracted after the pre-training is completed.

[0049] S4: Generate a synonym set of the words in the vocabulary dynamically using the granular sphere clustering algorithm according to the word vector representation of the words in the vocabulary;

[0050] Preferably, the method of dynamically generating a synonym set of words in the vocabulary using a granular sphere clustering algorithm comprises:

[0051] S41: All words in the vocabulary are used as the initial spherulite set;

[0052] S42: Calculate the cosine distance matrix between word vectors of all words in the spheroidal set; select two words a and b with the largest difference in the cosine distance matrix; and put the two words a and b with the largest semantic difference into two new spheroidal sets respectively;

[0053] S43: traverse each word c in the original spherulite set, if the cosine distance between word c and word a is greater than the cosine distance between word c and word b, then add word c to the new spherulite set where word b is located, otherwise, add word c to the new spherulite set where word a is located;

[0054] S44: Repeat steps S42 to S44 for each new set of spheres until the density of the set of spheres reaches a set threshold, and output multiple sets of spheres, each of which represents a synonym set.

[0055] Preferably, the density of the pellet set includes:

[0056]

[0057] Where De(gb) represents the spheroidal set, n represents the number of words in the spheroidal set, and x i represents the word vector of the i-th word; c0 represents the center of the spherulite set; d() represents the cosine distance.

[0058] S5: Generate adversarial samples of training text by replacing the words in the training text with their synonyms;

[0059] Embodiment 3, traverses each word in the training text, finds the synonym set of the word, regards all other words in the synonym set as candidate words of the word, and obtains the candidate word set of the word; replaces the word based on the words in the candidate word set to obtain the adversarial text.

[0060] There are some defects in the above-mentioned Example 3. There is a lack of an effective synonym set intervention mechanism in the adversarial sample generation process, which makes the adversarial samples unable to effectively represent the real language changes, thereby affecting the training effect. . At the same time, these models often have the problem of insufficient translation stability when faced with diverse expressions such as vocabulary replacement and syntactic structure changes in the input. When synonym replacement or slightly different sentence structure occurs in the input, the translation result output by the system may have obvious deviations, affecting the accuracy and reliability of the translation. In this context, the generation of text adversarial samples requires adding imperceptible tiny disturbances without changing the original semantics of the sentence, so as to successfully mislead the model to output wrong results. Therefore, we provide a better adversarial sample generation method, as described in Example 4:

[0061] Embodiment 4, preferably, the step S5 comprises:

[0062] S51: Construct an objective function with the goal of introducing minimal perturbations to mislead the text translation model: Where P represents the objective function, X represents the training text; X adv represents the adversarial text after the perturbation; g(X,X adv ) represents text X and text X adv The semantic similarity of text X; m(X) represents the semantic perplexity of text X; m(X) represents the semantic perplexity of text X, which is obtained through the pre-trained large prediction model GPT-2.

[0063] S52: using a synonym set of words in the training text to randomly replace words in the training text to generate an initial population of the genetic algorithm;

[0064] S53: Based on the generated initial population of the genetic algorithm, the constructed objective function is solved using the genetic algorithm to obtain the optimal adversarial text.

[0065] In this embodiment, adversarial samples are generated by a genetic algorithm combined with a target function constructed with the goal of introducing minimal disturbance to mislead the text translation model, so that the translation system can effectively cope with complex input situations such as synonym replacement or sentence structure changes. The translation stability of the system under diversified expressions is significantly improved, making the synonym set more in line with the actual translation task requirements, and improving the adaptability and accuracy of the translation model. For example, in traditional synonym replacement, a large number of obviously different adversarial samples will be generated, that is, the semantics of the generated adversarial text is too different from the semantics of the original text. In order to avoid this situation, this embodiment provides a preferred implementation method, which refers to adding imperceptible tiny disturbances without changing the original semantics of the sentence, so as to train the model with adversarial samples that successfully mislead the model to output errors, so as to improve the robustness of the model.

[0066] S6: Construct a training set based on the training text and the adversarial sample, and train a text translation model based on the large language model based on the constructed training set; translate the text to be translated by using the trained text translation model.

[0067] Example 5: In the current batch, the training text and the training set generated by the adversarial sample are input into the text translation model for forward propagation to obtain the loss value loss, and the loss value loss is back-propagated to update the parameters of the text translation model. At the same time, if the loss of the current batch is greater than the loss of the previous batch, the influence coefficient δ is reduced by 0.05, otherwise if the loss of the current batch is less than the loss of the previous batch, the influence coefficient δ is increased by 0.05; the influence coefficient is used to adjust the population number of the genetic algorithm when generating adversarial samples. In the current batch, the population number of the genetic algorithm is equal to the population number of the previous batch × δ, and the calculation formula of the loss value loss is as follows:

[0068]

[0069] Where N is the sentence length, is the true probability distribution of the ith word, is the model predicted probability distribution of the ith word.

[0070] Beneficial effects of the regulation mechanism: by regulating the influence coefficient δ

[0071] 1. Ensure that the complexity of adversarial sample generation during training is consistent with the model training effect. By reducing the population size when the model loss is high, avoid excessive adversarial perturbations and reduce excessive interference to the translation model; when the loss decreases, increase the population size, enhance the ability to generate adversarial samples, and improve the model's ability to resist interference from input diversity.

[0072] 2. Reduce excessive invalid adversarial samples to help the model focus on core issues faster, thereby accelerating the convergence of the model.

[0073] In summary, the present invention generates a multi-granularity, size-adaptive synonym set through a granular sphere clustering algorithm, provides candidate words for subsequent adversarial sample attacks, and solves the problem that the traditional synonym set generation needs to set a threshold and cannot adapt to the size. At the same time, the introduction of the genetic algorithm makes it possible to change multiple words in a sentence at one time when generating adversarial samples, and to generate and optimize iterative populations, solving the problem that the traditional adversarial sample generation is prone to fall into local optimality. Through adversarial training, these generated adversarial samples are integrated into the training process of the translation model, effectively improving the robustness of the model in the face of complex inputs. Especially in the face of diversified expressions such as synonym replacement and syntactic structure changes, the translation results of the model are more stable and accurate, solving the translation deviation and inconsistency problems that are prone to occur in traditional translation models when dealing with input diversity.

[0074] Another aspect of the present invention provides a text translation system based on particle-sphere clustering, comprising a processor and a memory; the memory is used to store a computer program; the processor is connected to the memory and is used to execute the computer program stored in the memory, so that the text translation system based on particle-sphere clustering executes the text translation method based on particle-sphere clustering.

[0075] Another aspect of the present invention provides a computer-readable storage medium storing a program, wherein when the program is executed by a processor, the text translation method based on particle sphere clustering is implemented.

[0076] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0077] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A text translation method based on sphere clustering, characterized in that: include: S1: Use crawler technology to regularly obtain text datasets and preprocess them; S2: Use a word segmentation tool to perform word segmentation on the acquired text data, and use the words obtained by the word segmentation to update the word library; S3: Encode the words in the vocabulary based on the pre-trained large language model to obtain the word vector representation; S4: Adaptively generate synonym sets of words in the vocabulary using the granular sphere clustering algorithm according to the word vector representation of the words in the vocabulary; S5: Generate adversarial samples of training text by replacing the words in the training text with their synonyms; S6: Construct a training set based on the training text and the adversarial sample, and train a text translation model based on the large language model based on the constructed training set; translate the text to be translated by using the trained text translation model.

2. The text translation method based on granular sphere clustering according to claim 1, characterized in that: The word segmentation processing of the acquired text data using the word segmentation tool includes: For English text, the NLTK library in Python is used for word segmentation, and for Chinese text data, the jieba tool is used for word segmentation.

3. The text translation method based on granular sphere clustering according to claim 1, characterized in that: The method of dynamically generating a synonym set of words in the vocabulary using the granular sphere clustering algorithm includes: S41: All words in the vocabulary are used as the initial spherulite set; S42: Calculate the cosine distance matrix between word vectors of all words in the spheroidal set; select two words a and b with the largest difference in the cosine distance matrix; and put the two words a and b with the largest semantic difference into two new spheroidal sets respectively; S43: traverse each word c in the original spherulite set, if the cosine distance between word c and word a is greater than the cosine distance between word c and word b, then add word c to the new spherulite set where word b is located, otherwise, add word c to the new spherulite set where word a is located; S44: Repeat steps S42 to S44 for each new set of spheres until the density of the set of spheres reaches a set threshold, and output multiple sets of spheres, each of which represents a synonym set.

4. The text translation method based on granular sphere clustering according to claim 3, characterized in that: The density of the pellet set includes: Where De(gb) represents the spheroidal set, n represents the number of words in the spheroidal set, and x i represents the word vector of the i-th word; c0 represents the center of the spherulite set; d() represents the cosine distance.

5. The text translation method based on granular sphere clustering according to claim 1, characterized in that: The step S5 comprises: S51: Construct an objective function with the goal of introducing minimal perturbations to mislead the text translation model: Where P represents the objective function, X represents the training text; X adv represents the adversarial text after the perturbation; g(X,X adv ) represents text X and text X adv The semantic similarity of ; m(X) represents the semantic confusion of text X; S52: using a synonym set of words in the training text to randomly replace words in the training text to generate an initial population of the genetic algorithm; S53: Based on the generated initial population of the genetic algorithm, the constructed objective function is solved using the genetic algorithm to obtain the optimal adversarial text.

6. A text translation system based on sphere clustering, characterized in that: It comprises a processor and a memory; the memory is used to store a computer program; the processor is connected to the memory and is used to execute the computer program stored in the memory, so that the text translation system based on granular sphere clustering executes the text translation method based on granular sphere clustering described in any one of claims 1 to 5.

7. A computer-readable storage medium storing a program, characterized in that: When the program is executed by a processor, the text translation method based on particle sphere clustering described in any one of claims 1 to 5 is implemented.

Images