A text translation method and system based on granular clustering

Abstract

The application relates to a granular ball clustering-based text translation method and system, which comprises the following steps: periodically obtaining a text data set by using a crawler technology and performing pretreatment; performing word segmentation processing on the obtained text data by using a word segmentation tool, and updating a word library obtained by the word segmentation processing; encoding words in the word library based on a pre-trained large language model to obtain word vector representations of the words; adaptively generating a synonym set of the words in the word library by using a granular ball clustering algorithm according to the word vector representations of the words in the word library; replacing the words in the training text by using the synonyms of the words to generate adversarial samples of the training text; constructing a training set according to the training text and the adversarial samples, training a text translation model based on a large language model based on the constructed training set; and translating the text to be translated by using the trained text translation model, so that the accuracy and reliability of translation are improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of natural language processing, and particularly relates to a text translation method and system based on granular ball clustering. BACKGROUND

[0002] With the acceleration of globalization, the importance of machine translation systems in cross-language communication is increasingly prominent. Traditional machine translation methods can often only learn the mapping relationship between the source language and the target language to achieve translation function. However, in actual application, the diversity, complexity and structural changes of input languages often lead to unstable or incorrect translation results.

[0003] In the field of natural language processing, adversarial samples have been proven to improve the robustness of models. However, existing machine translation systems rarely combine adversarial training, and these models often have insufficient translation stability when facing diversified expressions such as input word substitution and syntactic structure changes. When there is synonym substitution or slight difference in sentence structure in the input, the translation result output by the system may deviate significantly, affecting the accuracy and reliability of translation.

[0004] Under this background, the generation of text adversarial samples needs to add subtle perturbations that are difficult to detect without changing the original semantics of the sentence to successfully mislead the model to output incorrect results. However, the existing word-level attack has a limited set of synonyms for generating adversarial samples. Word-level attack relies on finding synonyms to replace words in victim samples, which means that the quality of the synonym set directly affects the effectiveness of the attack. Most methods need to predefine the construction threshold of the synonym set, such as the similarity distance between words, which limits the range of available synonyms. SUMMARY

[0005] In order to solve the problems in the background art, one aspect of the present application provides a text translation method based on granular ball clustering, comprising:

[0006] S1: periodically obtaining a text data set by using a crawler technology and preprocessing;

[0007] S2: performing word segmentation processing on the obtained text data by using a word segmentation tool, and updating a word library obtained by the word segmentation processing;

[0008] S3: encoding the words in the word library based on a pre-trained large language model to obtain word vector representations of the words;

[0009] S4: generating a synonym set of the words in the word library adaptively according to the word vector representations of the words in the word library by using a granular ball clustering algorithm;

[0010] S5: generating adversarial samples of the training text by replacing the words in the training text with their synonyms.

[0011] S6: Construct a training set according to the training text and the adversarial sample, train the text translation model based on the large language model based on the constructed training set; and translate the text to be translated through the trained text translation model.

[0012] Preferably, the word segmentation processing of the obtained text data by using the word segmentation tool comprises:

[0013] The NLTK library in python is used for word segmentation of English text, and the jieba tool is used for word segmentation of Chinese text data.

[0014] Preferably, the dynamically generating a synonym set of words in the word library by using the particle clustering algorithm comprises:

[0015] S41: All words in the word library are taken as an initial particle set;

[0016] S42: Calculate the cosine distance matrix between the word vectors of all words in the particle set; select two words a and b with the largest difference in the cosine distance matrix; and put the two words a and b with the largest semantic difference into two new particle sets, respectively;

[0017] S43: Traverse each word c in the original particle set, if the cosine distance between word c and word a is greater than the cosine distance between word c and word b, then add word c to the new particle set where word b is located, otherwise add word c to the new particle set where word a is located;

[0018] S44: Repeat steps S42-S44 for each new particle set until the density of the particle set reaches a set threshold, and output multiple particle sets, each particle set representing a synonym set.

[0019] Preferably, the density of the particle set comprises:

[0020]

[0021] Wherein, De(gb) represents the particle set, n represents the number of words in the particle set, x i represents the word vector of the i-th word; c0 represents the center of the particle set; d() represents the cosine distance.

[0022] Preferably, the step S5 comprises:

[0023] S51: Construct a target function with the goal of introducing the least disturbance to mislead the text translation model: Wherein, P represents the target function, X represents the training text; X adv represents the adversarial text after introducing the disturbance; g(X,X adv ) represents the text X and the text Xadv semantic similarity of X and Y; m(X) represents the semantic perplexity of text X;

[0024] S52: generating an initial population of the genetic algorithm by randomly replacing the words in the training text with the synonym set of the words in the training text;

[0025] S53: obtaining the optimal adversarial text by solving the constructed objective function based on the generated initial population of the genetic algorithm.

[0026] Another aspect of the present application provides a text translation system based on granular ball clustering, comprising a processor and a memory; the memory is used to store a computer program; the processor is connected with the memory, and is used to execute the computer program stored in the memory, so that the text translation system based on granular ball clustering executes the text translation method based on granular ball clustering.

[0027] Still another aspect of the present application provides a computer readable storage medium, which stores a program; when the program is executed by a processor, the text translation method based on granular ball clustering is realized.

[0028] The present application has at least the following beneficial effects

[0029] Compared with the prior art, the synonym set generated by granular ball clustering introduces more diverse adversarial samples in model training, so that the translation system can effectively deal with complex input conditions such as synonym replacement or sentence structure change, and the translation stability of the system under diversified expression is significantly improved. The present application introduces the generation of adversarial samples, which is more suitable for the actual translation task requirements, and improves the adaptability and accuracy of the translation model. By introducing adversarial samples in training, the model can better handle input changes and reduce translation errors caused by synonym replacement or input structure changes, and the translation of professional terms and accurate expressions is significantly improved. BRIEF DESCRIPTION OF DRAWINGS

[0030] Figure 1 The present application is a method flowchart. DETAILED DESCRIPTION

[0031] The embodiments of the present application will be described in detail below with specific reference to the drawings. Those skilled in the art can easily understand other advantages and effects of the present application from the content disclosed in the specification. The present application can also be implemented or applied by different specific embodiments, and the details in the specification can be modified or changed based on different views and applications without departing from the spirit of the present application. It should be noted that the diagrams provided in the following examples only illustrate the basic concept of the present application in a schematic manner, and the following examples and features in the examples can be combined with each other without conflict.

[0032] In order to improve the accuracy of the translation model, and solve the problem that the existing machine translation method is unstable in the face of synonym replacement and input structure change, the present application provides a text translation method based on granular ball clustering, and the specific technical solutions are as follows:

[0033] Obtaining word vectors:

[0034] Using crawler technology to collect data on the web, and then using lstm or bert model for training to obtain word vectors;

[0035] Generating a dynamic synonym set based on granular ball clustering:

[0036] The pre-trained word vectors are clustered using the granular ball clustering algorithm. According to the similarity between the word vectors, the method adaptively clusters semantically similar words in the same granular ball, forming a synonym set. Using the granular ball synonym set, there is no need to set a synonym quantity threshold, so that a more accurate multi-granularity synonym set is obtained.

[0037] Adversarial sample generation:

[0038] Using the granular ball synonym set, combined with different strategies to generate a candidate word set, the words in the original training data are replaced with synonyms and added with disturbances to generate adversarial samples containing synonym transformations. The generated adversarial samples are used together with the original data for translation model training, so that the model can cope with diversified expression forms and maintain the stability of translation.

[0039] Introducing adversarial samples for model training:

[0040] During training, the translation loss of the model for the adversarial sample is calculated, and the adversarial sample generation, model training and loss calculation form a cycle, which continuously improves the model performance through multiple iterations. The system can automatically adjust the strategy according to real-time training feedback to improve the translation quality.

[0041] Embodiment 2, please refer to Figure 1 One aspect of the present application provides a text translation method based on granular ball clustering, comprising:

[0042] S1: Adopting crawler technology to regularly obtain text data set and pre-processing;

[0043] In this embodiment, the pre-processing of the obtained text data set includes: using crawler technology to obtain internet text data set, and cleaning the data set, removing some illegal symbols, or irrelevant data such as website, email, number, etc. through regular matching.

[0044] S2: performing word segmentation on the obtained text data using a word segmentation tool, and updating the word library using the words obtained through the word segmentation;

[0045] Preferably, the word segmentation on the obtained text data using a word segmentation tool comprises:

[0046] For English text, use the NLTK library in python for word segmentation, and for Chinese text data, use the jieba tool for word segmentation.

[0047] S3: encoding the words in the word library based on a pre-trained large language model to obtain word vector representations of the words;

[0048] In this embodiment, the data is pre-trained using an Lstm or Bert model (Lstm can be trained using a classification task, and Bert can be trained using the initial pre-training task (filling in words and predicting the next sentence)).

[0049] S4: dynamically generating a synonym set of words in the word library according to the word vector representations of the words in the word library using a particle ball clustering algorithm;

[0050] Preferably, the dynamically generating a synonym set of words in the word library using a particle ball clustering algorithm comprises:

[0051] S41: taking all the words in the word library as an initial ball particle set;

[0052] S42: calculating the cosine distance matrix between all the words in the ball particle set; selecting two words a and b with the largest difference in the cosine distance matrix; and placing the two words a and b with the largest semantic difference into two new ball particle sets, respectively;

[0053] S43: traversing each word c in the original ball particle set, if the cosine distance between word c and word a is greater than the cosine distance between word c and word b, then word c is added to the new ball particle set of word b, otherwise word c is added to the new ball particle set of word a;

[0054] S44: repeating steps S42-S44 for each new ball particle set until the density of the ball particle set reaches a set threshold, and outputting a plurality of ball particle sets, each ball particle set representing a synonym set.

[0055] Preferably, the density of the ball particle set comprises:

[0056]

[0057] where De(gb) represents the ball particle set, n represents the number of words in the ball particle set, x i represents the word vector of the i-th word; c0 represents the center of the ball particle set; d() represents the cosine distance.

[0058] S5: Replacing the words in the training text with their synonyms to generate the adversarial samples of the training text;

[0059] In embodiment 3, each word in the training text is traversed, the synonym set in which the word is located is found, all other words in the synonym set are regarded as candidate words of the word, and a candidate word set of the word is obtained; and the word in the candidate word set is replaced to obtain the adversarial text.

[0060] In the above embodiment 3, there are some defects, and there is a lack of effective synonym set intervention mechanism in the adversarial sample generation process, so that the adversarial sample cannot effectively represent the real language change, thereby affecting the training effect. At the same time, when facing the input of word replacement, syntactic structure change and other diversified expressions, these models often have the problem of insufficient translation stability. When the input appears synonym replacement or sentence structure is slightly different, the translation result output by the system may have obvious deviation, affecting the accuracy and reliability of translation. Under this background, the generation of text adversarial samples needs to add a small perturbation that is difficult to detect under the premise of not changing the original semantics of the sentence, in order to successfully mislead the model output error result. Therefore, we provide a more optimal adversarial sample generation method, as described in embodiment 4:

[0061] In embodiment 4, preferably, the step S5 comprises:

[0062] S51: A target function is constructed with the goal of introducing the smallest disturbance to mislead the text translation model: Wherein, P represents the target function, X represents the training text; X adv represents the adversarial text after introducing the disturbance; g(X, X adv ) represents the semantic similarity of the text X and the text X adv ; m(X) represents the semantic perplexity of the text X; m(X) represents the semantic perplexity of the text X, which is obtained by a pre-trained large prophecy model GPT-2.

[0063] S52: The words in the training text are randomly replaced with the synonym set of the words in the training text to generate the initial population of the genetic algorithm;

[0064] S53: Based on the generated initial population of the genetic algorithm, the genetic algorithm is used to solve the constructed target function to obtain the optimal adversarial text.

[0065] In this embodiment, the target function is generated by genetic algorithm, combined with introducing the minimum disturbance to mislead the text translation model, so that the translation system can effectively deal with complex input conditions such as synonym replacement or sentence structure change, and the stability of the system under diversified expression is significantly improved, so that the synonym set is more suitable for the actual translation task requirements, and the adaptability and accuracy of the translation model are improved. For example, in traditional synonym replacement, a large number of obviously different adversarial samples will be generated, that is, the generated adversarial semantics and the semantics of the original text are too large. In order to avoid this situation, the embodiment provides an optimal implementation manner, which is to add a small disturbance that is difficult to detect under the premise of not changing the original semantics of the sentence, so as to successfully mislead the model to output an error adversarial sample to train the model and improve the robustness of the model.

[0066] S6: Construct a training set according to the training text and the adversarial sample, and train the text translation model based on the large language model based on the constructed training set; and translate the text to be translated through the trained text translation model.

[0067] In embodiment 5, in the current batch, the training text and the training set generated by the adversarial sample are input into the text translation model to obtain a loss value loss through forward propagation, the loss value loss is back propagated to update the parameters of the text translation model, and at the same time, if the loss of the current batch > the loss of the last batch, the influence coefficient δ is reduced by 0.05, otherwise if the loss of the current batch < the loss of the last batch, the influence coefficient δ is increased by 0.05; the influence coefficient is used to adjust the population number of the genetic algorithm when generating the adversarial sample, and the population number of the genetic algorithm in the current batch is equal to the population number of the last batch x δ, and the calculation formula of the loss value loss is as follows:

[0068]

[0069] Wherein, N is the length of the sentence, is the true probability distribution of the i th word, is the model prediction probability distribution of the i th word.

[0070] The beneficial effect of the adjustment mechanism is that the influence coefficient δ is adjusted

[0071] 1. Ensure that the complexity of the adversarial sample generated in the training process is adapted to the model training effect. By reducing the population number when the model loss is high, the excessive interference of the adversarial disturbance on the translation model is avoided; and when the loss decreases, the population number is increased, the adversarial sample generation ability is enhanced, and the anti-interference ability of the model to input diversity is improved.

[0072] 2. Reduce excessive invalid adversarial samples, help the model focus on the core problem faster, and thus accelerate the convergence of the model.

[0073] In summary, the present application generates a multi-granularity, size-adaptive synonym set through the granular ball clustering algorithm, provides candidate words for subsequent adversarial sample attacks, and solves the problem of setting a threshold and the inability to adaptively set the size in the generation of a traditional synonym set. Meanwhile, the introduction of the genetic algorithm enables multiple words in a sentence to be changed at one time when generating adversarial samples, and generates and optimizes an iterative population, thus solving the problem of the traditional adversarial sample generation being prone to local optimization. Through adversarial training, these generated adversarial samples are integrated into the training process of the translation model, effectively improving the robustness of the model in the face of complex inputs. Especially in the face of synonym replacement, syntactic structure changes and other diversified expressions, the translation results of the model are more stable and accurate, and the problem of translation bias and inconsistency in the processing of input diversity by the traditional translation model is solved.

[0074] Another aspect of the present application provides a text translation system based on granular ball clustering, comprising a processor and a memory; the memory is used to store a computer program; the processor is connected with the memory, and is used to execute the computer program stored in the memory, so that the text translation system based on granular ball clustering executes the text translation method based on granular ball clustering.

[0075] Still another aspect of the present application provides a computer readable storage medium, which stores a program, and the program is executed by a processor to realize the text translation method based on granular ball clustering.

[0076] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The program can be stored in a non-volatile computer readable storage medium, and when the program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, storage, database or other medium used in the embodiments provided in the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0077] Finally, it should be pointed out that the above embodiments are only used to illustrate the technical solutions of the present application and not to limit it. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced equivalently without departing from the purpose and scope of the technical solutions, which should be covered in the scope of the claims of the present application.

Claims

1. A text translation method based on granular clustering, characterized in that, The method comprises the following steps: S1: periodically obtaining a text data set by using a crawler technology and preprocessing the text data set; S2: performing word segmentation processing on the obtained text data by using a word segmentation tool, and updating a word library by using the words obtained by the word segmentation processing; S3: encoding the words in the word library based on a pre-trained large language model to obtain word vector representations of the words; S4: adaptively generating a synonym set of the words in the word library by using a granular sphere clustering algorithm according to the word vector representations of the words in the word library; The granular sphere clustering algorithm dynamically generates the synonym set of the words in the word library, which comprises the following steps: S41: taking all the words in the word library as an initial sphere set; S42: calculating a cosine distance matrix between the word vectors of all the words in the sphere set; selecting two words a and b with the largest semantic difference in the cosine distance matrix; and placing the two words a and b with the largest semantic difference into two new sphere sets, respectively; S43: traversing each word c in the original sphere set, if the cosine distance between the word c and the word a is greater than the cosine distance between the word c and the word b, then the word c is added to the new sphere set in which the word b is located, otherwise the word c is added to the new sphere set in which the word a is located; S44: repeating steps S42-S44 for each new sphere set until the density of the sphere set reaches a set threshold, and outputting a plurality of sphere sets, each of which represents a synonym set; The density of the sphere set comprises: where De(gb) represents the bag of balls, n represents the number of words in the bag of balls, x i represents the word vector of the i-th word; c0represents the center of the bag of balls; d() represents the cosine distance; S5: generating an adversarial sample of the training text by replacing the words in the training text with the synonym set of the words; The step S5 comprises: S51: Construct a target function aiming at misleading the text translation model with the least disturbance introduced: Wherein, P represents the target function, X represents the training text; X adv represents the adversarial text after the disturbance is introduced; g(X, X adv ) represents the semantic similarity of the text X and the text X adv ; m(X) represents the semantic confusion degree of the text X; S52: generating an initial population of a genetic algorithm by randomly replacing the words in the training text with the synonym set of the words; S53: solving a constructed objective function by using a genetic algorithm based on the generated initial population of the genetic algorithm to obtain an optimal adversarial text; S6: constructing a training set according to the training text and the adversarial sample, training a text translation model based on a large language model based on the constructed training set, and translating the text to be translated by using the trained text translation model.

2. The text translation method based on granular clustering according to claim 1, wherein, The word segmentation processing on the obtained text data by using a word segmentation tool comprises: For English text, the NLTK library in python is used for word segmentation, and for Chinese text data, the jieba tool is used for word segmentation.

3. A text translation system based on granular clustering, characterized by, The system comprises a processor and a memory; the memory is used to store a computer program; the processor is connected with the memory and is used to execute the computer program stored in the memory, so that the granular sphere clustering-based text translation system executes the granular sphere clustering-based text translation method in any one of claims 1-2.

4. A computer-readable storage medium storing a program, characterized by comprising: The program is executed by the processor to implement the granular sphere clustering-based text translation method in any one of claims 1-2.

Images