A text detection method based on multi-granularity clustering
By employing a deep neural network model trained with multi-granularity clustering and adversarial training, the problem of synonym substitution attacks bypassing detection is solved, improving the accuracy of spam identification and the robustness of the model, while reducing human intervention.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHONGQING UNIV OF POSTS & TELECOMM
- Filing Date
- 2024-03-01
- Publication Date
- 2026-07-24
AI Technical Summary
Existing detection technologies are insufficient to effectively detect synonym substitution attacks, allowing spam to bypass detectors, and thus rely heavily on manual intervention.
A multi-granularity clustering method is used to obtain a set of synonyms. A deep neural network model is trained using adversarial text data to build a robust defense framework and generate adversarial samples to enhance the model's ability to resist synonym substitution.
It improves the accuracy of spam and junk message identification, reduces the workload of manual screening, enhances the robustness and adaptability of the model, and can better cope with synonym substitution attacks.
Smart Images
Figure CN118035451B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of natural language processing technology, specifically relating to a text detection method based on multi-granularity clustering. Background Technology
[0002] With the rapid development of the online virtual world, more and more internet users are sharing their life experiences, observations, and news in cyberspace. However, many cybercriminals are using the internet to spread illegal and spam information, creating an urgent need for detection methods to detect this spam. Existing detection methods struggle to detect synonym substitution attacks, where attackers bypass existing detection technologies by replacing words in the text with their synonyms. Currently, there is no effective detection technology specifically designed to counter this attack. Therefore, detecting this spam still primarily relies on manual intervention. Summary of the Invention
[0003] To overcome the problem of spam bypassing detectors due to the difficulty in detecting synonym substitution in current technologies, this invention provides a text detection method based on multi-granularity clustering, comprising the following steps:
[0004] S1. Obtain the text dataset and preprocess it to obtain the preprocessed text dataset. Then, perform word segmentation on the preprocessed text dataset to obtain the dictionary.
[0005] S2. Cluster all words in the vocabulary using the particle purity adaptive method to obtain a synonym set; the synonym set contains multiple pairs of synonyms;
[0006] S3. Perturb the preprocessed text dataset using a set of synonyms to obtain an adversarial text dataset;
[0007] S4. Construct an information detection model and conduct adversarial training on the information detection model using adversarial text data;
[0008] S5. Input the text data to be processed into the trained information detection model and output the detection results.
[0009] Technical effects of the present invention:
[0010] This invention primarily employs a multi-granularity clustering method to normalize synonyms, reducing the differences between them. Finally, it uses synonym sample enhancement combined with existing deep learning techniques to defend against synonym substitution attacks. Compared to existing technologies, this improves the accuracy of identifying spam and junk messages, further reducing the workload of manual screening.
[0011] This invention employs a purity-adaptive granular clustering method to innovatively address the complex semantic relationship shapes that may exist in word sense clustering. Compared to traditional purity clustering methods, it better adapts to non-spherical cluster shapes, and in terms of purity distribution among words, purity clustering can more accurately determine the shape and size of clusters, thus more precisely capturing the relationships between word senses. The advantage of this method is that it does not require pre-specifying the number of clusters; instead, it can automatically discover clusters present in the data and assign words to the corresponding clusters based on purity distribution. This characteristic of not requiring pre-specifying the number of clusters makes purity clustering more flexible and convenient in word sense clustering. Furthermore, by introducing random perturbations into datasets for different tasks based on the results generated by word sense clustering, adversarial samples are constructed. Finally, these adversarial samples are applied to traditional natural language deep neural networks. Networks trained through this process exhibit strong robustness and can be widely applied in various fields, such as spam detection. This technical solution has significant innovation and practicality in improving the adaptability of models and responding to adversarial attacks. Attached Figure Description
[0012] Figure 1 This is a flowchart of a text detection method based on multi-granularity clustering according to the present invention. Detailed Implementation
[0013] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0014] This invention provides a text detection method based on multi-granularity clustering, such as... Figure 1 As shown, it includes the following steps:
[0015] S1. Obtain the text dataset and preprocess it to obtain the preprocessed text dataset. Then, perform word segmentation on the preprocessed text dataset to obtain the dictionary.
[0016] Specifically, step S1 involves acquiring a text dataset and preprocessing it to obtain a preprocessed text dataset. The preprocessed text dataset is then segmented to obtain a lexicon, which includes:
[0017] S11. Use web crawlers and other methods to obtain text datasets;
[0018] The text dataset obtained at this time may contain a large number of illegal characters and emojis, which belong to noisy words in language processing; therefore, it is necessary to preprocess the text dataset.
[0019] S12. Construct regular expression rules, traverse the text dataset, match and replace according to the regular expression rules, and replace the noise words with spaces. The regular expression rules include using [0-9] to match numbers, using [\u4e00-\u9fa5] to match Chinese characters, and using [A-Za-z] to match English letters, etc.
[0020] S13. Perform word segmentation on the preprocessed text dataset to obtain multiple words, and label each word to form a vocabulary.
[0021] Specifically, a word segmentation tool is selected to segment the preprocessed text dataset. If the preprocessed text dataset is English text, NLTK in the Python library can be used for word segmentation; if the preprocessed text dataset is Chinese text, the jieba library can be used for word segmentation, and finally a dictionary is generated.
[0022] S2. The particle purity adaptive method is used to cluster all words in the lexicon to obtain a synonym set. The synonym set contains multiple similar word sets; each similar word set contains multiple semantically similar words.
[0023] Specifically, step S2 uses a particle purity adaptive method to cluster all words in the lexicon to obtain a set of synonyms, including:
[0024] S21. Vectorize each word in the vocabulary using methods such as TF-IDF to obtain the vector representation of each word;
[0025] S22. Combine the vector representations of all words in the dictionary into a single sphere, and then proceed to step S23;
[0026] S23. Calculate the purity of the sphere and determine whether the purity of the sphere is less than the purity threshold. If yes, proceed to step S24; otherwise, keep the sphere unchanged. Herein, determine the number of label types contained in the sphere and obtain the number of words for each label type in the sphere. Divide the maximum number of words for the label type by the total number of words in the sphere to obtain the purity of the sphere.
[0027] S24. Traverse each vector in the particle to represent the label of the word, determine the number of label types k in the particle, use the K-means algorithm to further divide the particle into k particles, and then execute step S23.
[0028] S25. Repeat steps S23-S24 until the purity of all spheres is not less than the purity threshold, and obtain multiple spheres; divide the words in the lexicon according to the sphere division results to obtain multiple similar word sets, and all similar word sets form a synonym set, where all words belonging to the same similar word set are synonyms.
[0029] The above method is more consistent with the data distribution and is more effective.
[0030] It is difficult to capture information between words themselves, so it is necessary to vectorize all words in the lexicon. There is a one-to-one correspondence between words and vector representations. After words are converted into vector representations, information such as distance can be captured. The object of particle sphere partitioning in this invention is also the vector representation of words. By using the particle sphere clustering algorithm, the vector representations of all words in the lexicon are clustered to obtain several particles. At this time, the entire lexicon is partitioned based on the partitioning results of the particles to obtain different sets of similar words. For example, suppose the vocabulary contains 10 words: {a, b, c, d, e, f, g, h, i, j}. Using a sphere clustering algorithm, these 10 words are divided into three spheres based on their vector representations. According to the one-to-one correspondence between words and their vector representations, three sets of similar words are ultimately obtained: {{a, b, c}, {d, e}, {f, g, h, i, j}}, i.e., res = {set A, set B, set C}. This entire set res is called a complete set of synonyms, where sets A, B, and C are the different sets of similar words within it. Therefore, each word corresponds to a set of similar words (a ∈ {a, b, c} => set A), and all words correspond to a complete set of synonyms (all words => res).
[0031] S3. Use a set of synonyms to perturb each preprocessed text in the preprocessed text dataset to obtain an adversarial sample set for each preprocessed text; all adversarial sample sets constitute the adversarial text dataset.
[0032] Specifically, step S3 involves perturbing any preprocessed text using a set of synonyms to obtain an adversarial example, including:
[0033] S31. Determine whether the i-th word (i=1,2,...,N) in the preprocessed text has a synonym in the synonym set (i.e., determine whether the word belongs to a similar word set in the synonym set, or whether the word has the same meaning as a similar word set in the synonym set). If yes, proceed to step S32; otherwise, the i-th word remains unchanged, and then proceed to step S33; where N represents the number of words in the preprocessed text.
[0034] S32. Determine whether the number of synonyms is greater than 1 (i.e., determine whether the number of words in the set of similar words corresponding to the word, excluding itself, is greater than 1). If yes, randomly select a synonym to replace it, and then proceed to step S33; if not, use the synonym to directly replace it, and then proceed to step S33.
[0035] S33. Determine if i is equal to N. If yes, obtain an adversarial sample. If not, set i = i + 1 and return to step S31.
[0036] Through steps S31-S33, multiple adversarial samples of a preprocessed text can be obtained, thus forming the adversarial sample set of the preprocessed text.
[0037] This process perturbs the text with a set of synonyms, introducing synonyms into the text. Through this process, we introduce synonyms into each preprocessed text in the preprocessed text dataset. Such perturbation helps improve the model's robustness, making it more generalizable to data with synonym substitutions.
[0038] S4. Construct an information detection model and train it adversarially using adversarial text data. This step aims to create a more robust defense framework that can effectively resist attacks such as synonym substitution.
[0039] Specifically, adversarial training of the information detection model using adversarial text data includes:
[0040] S41. Select any preprocessed data from the preprocessed text dataset, and select k-1 adversarial samples from its adversarial sample set;
[0041] S42. Input the preprocessed text and its k-1 adversarial samples into the information detection model to obtain the detection results, and calculate the training loss based on the detection results. The training loss includes a classification loss and a divergence loss between the preprocessed text and the k-1 adversarial samples.
[0042] S43. Train the model parameters by backpropagating the training loss until the model converges.
[0043] Specifically, deep neural network models suitable for natural language processing tasks, such as recurrent neural networks (RNNs), long short-term memory networks (LSTMs), convolutional neural networks (CNNs), or Transformers, can be selected to construct the information detection model. The information detection model includes an input layer, hidden layers, and an output layer.
[0044] Specifically, the formula for calculating the training loss is:
[0045]
[0046] Where L1 represents the classification loss, and α is a hyperparameter. Let represent the probability distribution of the j-th adversarial example, and x represent the probability distribution of the preprocessed text.
[0047] Specifically, the classification loss is calculated using the cross-entropy loss function.
[0048] Specifically, in adversarial training, adversarial losses, such as FGSM (Fast Gradient Sign Method) loss, can be introduced to increase the model's robustness to adversarial samples.
[0049] Through this process, we performed adversarial training on traditional deep neural networks for natural language processing, improving the model's robustness to adversarial attacks such as synonym substitution. This training method can provide a more reliable solution when facing challenges in real-world application scenarios.
[0050] S5. Input the text data to be processed into the trained information detection model and output the detection results.
[0051] Specifically, the text data to be processed is perturbed using a set of synonyms to obtain k-1 adversarial samples. The text data to be processed and the k-1 adversarial samples are input into the trained information detection model to obtain k probability distributions. The average probability distribution is calculated based on the k probability distributions, and the category corresponding to the maximum probability value in the average probability distribution is selected for output.
[0052] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "setting," "connection," "fixing," "rotation," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. Unless otherwise explicitly limited, those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.
[0053] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.
Claims
1. A text detection method based on multi-granularity clustering, characterized in that, Includes the following steps: S1. Obtain the text dataset and preprocess it to obtain the preprocessed text dataset. Then, perform word segmentation on the preprocessed text dataset to obtain the dictionary. S2. Cluster all words in the lexicon using the particle purity adaptive method to obtain a synonym set; the synonym set contains multiple similar word sets; Step S2 uses the particle purity adaptive method to cluster all words in the lexicon to obtain a set of synonyms, including: S21. Vectorize each word in the vocabulary to obtain the vector representation of each word; S22. Combine the vector representations of all words in the dictionary into a single sphere, and then proceed to step S23; S23. Calculate the purity of the particle and determine whether the purity of the particle is less than the purity threshold. If yes, proceed to step S24; otherwise, keep the particle unchanged. S24. Traverse each vector in the particle to represent the label of the word, determine the number of label types k in the particle, use the K-means algorithm to further divide the particle into k particles, and then execute step S23. S25. Repeat steps S23-S24 until the purity of all spheres is not less than the purity threshold, and obtain multiple spheres; divide the words in the lexicon according to the sphere division results to obtain multiple similar word sets, and all similar word sets form a synonym set, where all words belonging to the same similar word set are synonyms. S3. Use a set of synonyms to perturb each preprocessed text in the preprocessed text dataset to obtain an adversarial sample set for each preprocessed text; all adversarial sample sets constitute the adversarial text dataset. S4. Construct an information detection model and conduct adversarial training on the information detection model using adversarial text data; Step S4 involves adversarially training the information detection model using adversarial text data, including: S41. Select any preprocessed data from the preprocessed text dataset, and select k-1 adversarial samples from its adversarial sample set; S42. Input the preprocessed text and its k-1 adversarial samples into the information detection model to obtain the detection results, and calculate the training loss based on the detection results. The training loss includes a classification loss and a divergence loss between the preprocessed text and the k-1 adversarial samples. S43. Train the model parameters by backpropagating the training loss until the model converges; S5. Input the text data to be processed into the trained information detection model and output the detection results.
2. The text detection method based on multi-granularity clustering according to claim 1, characterized in that, Step S1: Obtain the text dataset and preprocess it to obtain the preprocessed text dataset. Then, perform word segmentation on the preprocessed text dataset to obtain the lexicon, which includes: S11. Use web crawling technology to obtain text datasets; S12. Construct regular expression rules to replace noisy words in the text dataset with spaces to obtain a preprocessed text dataset. The regular expression rules include using [0-9] to match numbers, using [\u4e00-\u9fa5] to match Chinese characters, and using [A-Za-z] to match English letters. S13. Perform word segmentation on the preprocessed text dataset to obtain multiple words, and label each word to form a vocabulary.
3. The text detection method based on multi-granularity clustering according to claim 1, characterized in that, Step S3 involves perturbing any preprocessed text using a set of synonyms to obtain an adversarial example, including: S31. Determine whether the i-th word (i=1,2,...,N) in the preprocessed text has a synonym in the synonym set. If yes, proceed to step S32. If not, the i-th word remains unchanged, and then proceed to step S33. Where N represents the number of words in the preprocessed text. S32. Determine if the number of synonyms is greater than 1. If yes, randomly select a synonym to replace it, and then proceed to step S33; otherwise, use the synonym to directly replace it, and then proceed to step S33. S33. Determine if i is equal to N. If yes, obtain an adversarial sample; otherwise, set i = i + 1 and return to step S31.
4. The text detection method based on multi-granularity clustering according to claim 1, characterized in that, The formula for calculating training loss is: Where L1 represents the classification loss, and α is a hyperparameter. Let represent the probability distribution of the j-th adversarial example, and x represent the probability distribution of the preprocessed text.
5. The text detection method based on multi-granularity clustering according to claim 1, characterized in that, Step S5: Obtain the text data to be processed, input it into the trained information detection model, and output the detection results, including: The text data to be processed is perturbed by a set of synonyms to obtain k-1 adversarial samples. The text data to be processed and the k-1 adversarial samples are input into the trained information detection model to obtain k probability distributions. The average probability distribution is calculated based on the k probability distributions, and the category corresponding to the maximum probability value in the average probability distribution is selected for output.