A text classification method based on improved particle swarm optimization algorithm

Abstract

The present invention belongs to the field of text classification, and specifically relates to a text classification method based on an improved particle swarm algorithm. The method comprises the following steps: obtaining an original sentiment sentence text data set, labeling the original sentiment sentence text in the data set with a category label; using the original sentiment sentence text data set as a first training sample to train a bidirectional long short-term memory neural network to generate a first text classification model; constructing an undirected graph based on the original sentiment sentence text and calculating an optimal adversarial sample through the improved particle swarm algorithm; using the original sentiment sentence text data set and the optimal adversarial sample as a second training sample, performing adversarial training on the first text classification model to obtain a second text classification model, inputting a target text into the second classification model for classification, and outputting a classification result. The method has strong scalability, and can append new nodes to the undirected graph according to newly discovered rules, thereby providing richer model training data and improving the accuracy of model classification.

Description

A text classification method based on improved particle swarm optimization algorithm

[0001] The present invention belongs to the field of artificial intelligence information security technology, and specifically relates to a text classification method based on an improved particle swarm algorithm. Background Art

[0002] With the rapid development of the Internet, more and more people are socializing through the Internet. Although this allows people to access more things, it also causes some network security issues. People should deeply realize that strengthening network security is urgent. Traditional maintenance of network security is mainly through manual review of data on social platforms. This method is now also used by major short video platforms. However, there is a certain degree of subjectivity in the manual review method, and there is a situation where the review standards are lowered during the review. In the era of big data, the massive amount of online text content brings huge pressure to manual review. Therefore, using artificial intelligence models to automatically review platform content is a development direction.

[0003] When using deep learning methods to complete tasks, including text classification, text is converted into word vectors (vectors representing words, abbreviated as word vectors). These vectors are used to represent the individual words in the text and fed into a neural network as input. The network is trained through a backpropagation algorithm to complete various downstream NLP tasks, including text classification, question-answer matching, and machine translation. Text classification is a fundamental NLP task and is further categorized into binary text classification and multi-class text classification, depending on the number of classification labels.

[0004] However, the recognition capabilities of existing text classification models are heavily dependent on the characteristics of existing training samples. When the characteristics of input samples deviate from the distribution of features in the training set, the model can easily fall into cognitive blind spots, resulting in incorrect classification results. The training samples for neural network classification models do not incorporate current internet jargon, and the incompleteness of the training samples leads to poor classification results, which seriously hinders the use and promotion of text classification methods. Summary of the Invention

[0005] In order to solve the above-mentioned problem that when the features of the input samples deviate from the feature distribution of the training set samples, the model is prone to fall into a cognitive blind spot, resulting in incorrect classification results, and the training samples of the neural network classification model do not combine the current network terms. The training samples are incomplete, resulting in poor classification effect of the text classification model, which seriously affects the use and promotion of the text classification method. The present invention provides a text classification method based on an improved particle swarm algorithm, comprising:

[0006] S1: Obtain a dataset of original sentiment sentence texts and assign category labels to the original sentiment sentence texts in the dataset;

[0007] S2: Use the original sentiment sentence text dataset as the first training sample to train the bidirectional long short-term memory neural network to generate the first text classification model;

[0008] S3: Construct an undirected graph based on the original sentiment sentence text, and calculate the optimal adversarial sample based on the undirected graph using the improved particle swarm algorithm;

[0009] S4: Use the original sentiment sentence text dataset and the optimal adversarial sample as the second training sample, perform adversarial training on the first text classification model, and obtain the second text classification model;

[0010] S5: Obtain the target text to be classified, input the target text into the second classification model for classification, and output the classification result.

[0011] The present invention has at least the following beneficial effects:

[0012] The speed update, position update and scoring functions of the particle swarm algorithm are modified to adapt to the discrete data space, and the modified particle swarm algorithm and the database composed of undirected graphs are used to generate optimal adversarial samples, and the model is trained twice using the optimal adversarial samples to improve the robustness of the text detection model; the improved particle swarm algorithm can be directly applied to text data types without mapping the text into vector form, avoiding the impact of the accuracy of the word vector mapping process on the semantics of the words, and also saving operational complexity; this method is independent of the model structure and does not require modification of the model structure, and is applicable to any text classification model; this method has strong scalability and can append new nodes to the undirected graph according to the newly discovered rules, providing richer model training data and improving the accuracy of model classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 is a flow chart of the present invention; DETAILED DESCRIPTION

[0014] The technical solutions in the embodiments of the present invention will be described clearly and in detail below in conjunction with the drawings of the embodiments of the present invention. The described embodiments are only part of the embodiments of the present invention and cannot be used to limit the scope of the present invention.

[0015] Referring to FIG1 , the present invention provides a text classification method based on an improved particle swarm optimization algorithm, comprising:

[0016] S1: Obtain a dataset of original sentiment sentence texts and assign category labels to the original sentiment sentence texts in the dataset, where the category labels include: positive sentiment category and negative sentiment category.

[0017] A specific implementation of a text classification method based on an improved particle swarm algorithm, wherein the original emotional sentence text is obtained by: obtaining comment data through social platforms, text packages provided by network platforms, and manually input text.

[0018] S2: The original sentiment sentence text dataset is used as the first training sample to train the bidirectional long short-term memory neural network to generate the first text classification model.

[0019] A specific implementation of a text classification method based on an improved particle swarm optimization algorithm, wherein the training of a bidirectional long short-term memory neural network comprises:

[0020] Construct a neural network model with a bidirectional long short-term memory network structure, which consists of a bidirectional long short-term memory layer, a dropout layer, a hidden layer, and an output layer; the bidirectional long short-term memory layer has 128 dimensions; the dropout rate of the dropout layer is 0.5; the activation function of the hidden layer is ReLU, the activation function of the output layer is Softmax; and the loss function is cross entropy loss;

[0021] The original sentiment sentence text dataset is used as a training set, and a word vector space of Chinese text is constructed based on word2vec. The samples of the training set are mapped into a vector form through the word vector space.

[0022] The Adam optimizer was used, the learning rate was set to 0.01, and after 3 rounds of training, a trained bidirectional long short-term memory neural network model was obtained;

[0023] S3: Construct an undirected graph based on the original sentiment sentence text, and calculate the optimal adversarial sample based on the undirected graph using the improved particle swarm algorithm;

[0024] A specific implementation of a text classification method based on an improved particle swarm algorithm, wherein constructing an undirected graph based on the original sentiment sentence text comprises the following steps:

[0025] S21: Perform word segmentation processing on the original sentiment sentence text to obtain the original word segmentation set of the original sentiment sentence text in, Represents the nth original word in the original word set, n∈[1,N], N represents the number of original words in the original word set;

[0026] S22: The original word segmentation As the initial node of an undirected graph;

[0027] S23: The original word segmentation Pinyin and original participles English translation and original participle of The similar words of are used as the adjacent nodes of the initial node;

[0028] Preferably, the original emotional statement text consists of Chinese characters. The pinyin of the original word segmentation is the Chinese pinyin corresponding to the original word segmentation; the English translation of the original word segmentation is the English translation corresponding to the original word segmentation; the acquisition method of the original word segmentation's similar-looking words includes:

[0029] Convert all common Chinese characters into RGB mode pictures with a resolution of 24*24. The common Chinese characters can be randomly selected by those skilled in the art through big data statistics. Each dimension represents a pixel point. The values of each pixel point form a 1*24*24-dimensional pixel value matrix. Here, to simplify the computational complexity, only the pixel values of a certain channel in the RGB channels need to be retained, rather than the pixel values of all channels. Their positions in the matrix are the same as their positions in the picture. By moving the second row of the matrix to the end of the first row, and successively moving all rows to the end of the first row to form a 1*576-dimensional matrix, the stretched 1*576-dimensional matrix is normalized to obtain a normalized matrix. The normalized matrices of all common Chinese characters form a data set. Using the cosine similarity calculation formula, calculate the similarity between the normalized matrix corresponding to the Chinese characters forming the original word segmentation in the data set and the normalized matrices of other common Chinese characters in the data set. Arrange the common Chinese characters in descending order according to the similarity, and take the first m common Chinese characters as the similar-looking words of the specific text. Here, m is set by those skilled in the art according to actual needs.

[0030] S24: Take the homophonic words of the original word segmentation as the adjacent nodes of the pinyin node; take the combination of pinyin letters of the original word segmentation as the adjacent nodes of the pinyin node;

[0031] Preferably, among them, the homophonic words of the original word segmentation are words with the same Chinese pinyin as the original word segmentation. For example, the Chinese pinyin of the original word segmentation "telephone" is "dianhua", then the words with the same Chinese pinyin as the original word segmentation are "point conversion", "electrification", etc. The words with the same Chinese pinyin as the original word segmentation can be determined by consulting the Chinese dictionary or using a Chinese character input method. Here, they are not listed one by one. At the same time, those skilled in the art can select several words with the same Chinese pinyin as the original word segmentation according to actual needs; the combination of the first letters of the pinyin of the original word segmentation is the abbreviation combination of the first letters of the Chinese pinyin of the original word segmentation. For example, the Chinese pinyin of the original word segmentation "telephone" is "dianhua", and the first letters of the pinyin of the original word segmentation are "dh".

[0032] S25: Take the English transliteration of the original word segmentation as the adjacent node of the English translation node, so as to obtain an undirected graph of the original word segmentation ;

[0033] Preferably, the English phonetic transcription of the original word segmentation is to match Chinese characters with the same pronunciation according to the pronunciation of the English translation of the original word segmentation as the English phonetic transcription of the original word segmentation. For example, the English translation of the original word segmentation "电话" is "telephone". According to the pronunciation of the English word "telephone", the English phonetic transcription of the original word segmentation can be "特里否嗯", "特例否嗯", etc. Those skilled in the art can selectively adopt several corresponding words as the English phonetic transcription of the original word segmentation according to actual needs;

[0034] S26: Mark the path information of the initial node of the original word segmentation as 0, and mark the path information of the pinyin node of the original word segmentation, the English translation node of the original word segmentation, and the similar word node of the original word segmentation as 1. Mark the path information of the pinyin node of the original word segmentation and the path information of the first letter node of the pinyin of the original word segmentation as 2. Mark the path information of the English phonetic transcription node of the original word segmentation as 3. of the pinyin node, the original word segmentation of the English translation node and the original word segmentation of the similar word node as 1, and mark the path information of the pinyin node of the original word segmentation and the original word segmentation of the pinyin node and the original word segmentation of the first letter node of the pinyin as 2, and mark the path information of the original word segmentation of the English phonetic transcription node as 3.

[0035] A specific implementation manner of a text classification method based on an improved particle swarm optimization algorithm. The specific steps of calculating an adversarial sample according to an undirected graph through the improved particle swarm optimization algorithm include:

[0036] S31: Take all nodes in the undirected graph of the original word segmentation as the replaceable words of the original word segmentation, and combine the replaceable words of all original word segmentations in the original word segmentation set x in sequence according to the position to obtain an adversarial sample set of the original sentiment statement text, and use the adversarial sample set as the adversarial sample search space. For example, for the original word segmentation set {今天, 天气, 好}, the replaceable words of the original word segmentation "今天" include "jintian", "今添", "today", etc., the replaceable words of the original word segmentation "天气" include "添汽", "tianqi", etc., and the replaceable words of the original word segmentation "不错" include "不钅昔", "bucuo", "补措", etc.; the replaceable words of all original word segmentations are combined in sequence according to the position to obtain several adversarial sample sets of the original sentiment statement text, such as "今添tianqi不钅昔", "jintian添汽补措", etc., and form an adversarial sample search space with all such combination results. of all original word segmentations in sequence according to the position to obtain an adversarial sample set of the original sentiment statement text, and use the adversarial sample set as the adversarial sample search space; for example, for the original word segmentation set {今天, 天气, 好}, the replaceable words of the original word segmentation "今天" include "jintian", "今添", "today", etc., the replaceable words of the original word segmentation "天气" include "添汽", "tianqi", etc., and the replaceable words of the original word segmentation "不错" include "不钅昔", "bucuo", "补措", etc.; the replaceable words of all original word segmentations are combined in sequence according to the position to obtain several adversarial sample sets of the original sentiment statement text, such as "今添tianqi不钅昔", "jintian添汽补措", etc., and form an adversarial sample search space with all such combination results. o in sequence according to the position to obtain an adversarial sample set of the original sentiment statement text, and use the adversarial sample set as the adversarial sample search space; for example, for the original word segmentation set {今天, 天气, 好}, the replaceable words of the original word segmentation "今天" include "jintian", "今添", "today", etc., the replaceable words of the original word segmentation "天气" include "添汽", "tianqi", etc., and the replaceable words of the original word segmentation "不错" include "不钅昔", "bucuo", "补措", etc.; the replaceable words of all original word segmentations are combined in sequence according to the position to obtain several adversarial sample sets of the original sentiment statement text, such as "今添tianqi不钅昔", "jintian添汽补措", etc., and form an adversarial sample search space with all such combination results. [[ID=…]]The content continues here, but the input seems to be cut off. Please provide the full text if needed.

[0037] S32: Input the adversarial samples in the adversarial sample search space into the first text classification model to obtain the confidence of the first text classification model in the category labels of the adversarial samples;

[0038] S33: Use the improved particle swarm optimization algorithm according to the original word segmentation The path information between the nodes in the undirected graph and the initial nodes, as well as the confidence of the adversarial sample category label, are used to calculate the optimal adversarial sample from the adversarial sample search space. The optimal adversarial sample is in the model's cognitive blind spot and can be used as new training data to enable the text classification model to obtain better classification capabilities.

[0039] A specific implementation of a text classification method based on an improved particle swarm algorithm, wherein the calculation process of the optimal adversarial sample specifically includes:

[0040] S321: Based on the original word segmentation The path information between the nodes in the undirected graph and the initial node, as well as the confidence of the adversarial sample category label, are used to construct the score function of the particle swarm algorithm:

[0041]

[0042]

[0043] Among them, x a represents an adversarial example in the adversarial example search space, Represents x a The alternative word for the nth original participle in , Represents the nth original word The set of all replaceable words; C(x a ) represents the first text classification model for x a The confidence of the category label, Represents x o and x a The sum of the path information of all the alternative words of the original word in the undirected graph;

[0044] S322: Randomly initialize the speed and position of particles in the particle swarm;

[0045] S323: Calculate the fitness value of the particle in the particle swarm by using the score function. When the current fitness value of the particle is greater than the historical best fitness value of the particle, replace the historical best fitness value of the particle with the current fitness value of the particle, and record the current position of the particle as the individual previous best position; when the historical best fitness value of the particle is greater than the historical best fitness value of the group particles, replace the historical best fitness value of the group particles with the historical best fitness value of the particle, and record the current position of the particle as the global previous best position;

[0046] S324: Update particle speed and position:

[0047]

[0048]

[0049] v = v1…v n …v N

[0050]

[0051] where v n represents the probability of being replaced by other words, represents the nth dimension of the global previous optimal position of the entire particle swarm, S(*) is the sigmoid function, and w is the non - negative inertia weight; and are acceleration coefficients that are random positive numbers; p represents the individual previous optimal position of the particle; p g represents the global previous optimal position of the entire particle swarm, r is a random number selected from [0.0, 1.0], and T is the set probability threshold, represents the original word segmentation in the undirected graph of a random adjacent node; for example, the current position of the particle is "今天tianqi不搓", the current velocity is [0.3, 0.4, 0.5], the global previous optimal position is "金天添汽不搓", the individual previous optimal position is "今天tianqi不搓", w takes the value of 0.5, takes the value takes the value of 0.8, and according to the velocity update formula, the first dimension of the new velocity can be calculated as follows:

[0052] v1 = S(0.5 * 0.3 + 0.2 * - 1 + 0.8 * 1) ≈ 0.679

[0053] The probability threshold T takes the value of 0.4, and assume that the random number r in this round of iteration takes the value of 0.3. According to the position update formula, since r < T and r < v1, the value of the first dimension of the new position can be calculated by replacing it with the first dimension of the global previous optimal position, that is, "今天" is replaced by "金天"; assume that the random number r in this round of iteration takes the value of 0.5. Since r > T and r < v1, the value of the first dimension of the new position can be calculated by replacing it with a random adjacent node of "今天" in the undirected graph, such as "jintian"; assume that the random number r in this round of iteration takes the value of 0.7. Since r > v1, "今天" is not replaced by any word in this round. The improved particle swarm algorithm can be directly applied to text data types, without mapping the text into a vector form, avoiding the impact of the accuracy of the word vector mapping process on the word semantics, and also saving the operation complexity.

[0054] S325: Repeat steps S323-S324 until the iteration ends, output the best fitness value of the current group of particles, and obtain the optimal adversarial sample in the adversarial sample space based on the solution corresponding to the best fitness value of the current group of particles.

[0055] S4: Use the original sentiment sentence text dataset and the optimal adversarial sample as the second training sample, perform adversarial training on the first text classification model, and obtain the second text classification model;

[0056] For example, the original sentiment sentence text dataset is {A, B, C}, the optimal adversarial sample is a, and the second training sample is represented as {A, B, C, a}.

[0057] Preferably, the present invention can repeatedly execute steps S3 and S4 to find multiple optimal adversarial samples, and use the multiple optimal adversarial samples and the original emotional sentence text data set as the second training samples to perform adversarial training on the first text classification model to obtain a second text classification model, wherein the adversarial training is expressed as re-training the bidirectional long short-term memory neural network with the second training samples.

[0058] S5: Obtain the target text to be classified, input the target text into the second classification model for classification, output the classification result, and determine whether the target text is a negative text or a positive text based on the classification result.

[0059] The above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Without departing from the spirit and essence of the method of the present invention, those skilled in the art can make various corresponding changes based on the method of the present invention, which all fall within the scope of protection of the claims of the method of the present invention.

Claims

1. A text classification method based on improved particle swarm optimization algorithm, characterized in that: The following steps are involved: S1: Obtain a dataset of original sentiment sentence texts and assign category labels to the original sentiment sentence texts in the dataset; S2: Use the original sentiment sentence text dataset as the first training sample to train the bidirectional long short-term memory neural network to generate the first text classification model; S3: Construct an undirected graph based on the original sentiment sentence text, and calculate the optimal adversarial sample based on the undirected graph using the improved particle swarm algorithm; The construction of an undirected graph based on the original sentiment sentence text comprises the following steps: S21: Perform word segmentation processing on the original sentiment sentence text to obtain the original word segmentation set of the original sentiment sentence text in, Represents the nth original word in the original word set, n∈[1,N], N represents the number of original words in the original word set; S22: The original word segmentation As the initial node of an undirected graph; S23: The original word segmentation Pinyin and original participles English translation and original participle of The similar words of are used as the adjacent nodes of the initial node; S24: The original word segmentation The homophone of the pinyin node is used as the adjacent node of the pinyin node; the original segmentation The pinyin letter combination is used as the adjacent node of the pinyin node; S25: The original word segmentation The English transliteration is used as the adjacent node of the English translation node to obtain the original word segmentation undirected graph of ; The specific steps of calculating the adversarial sample based on the undirected graph through the improved particle swarm algorithm include: S31: The original word segmentation All nodes in the undirected graph are used as original segmentations Replaceable words, the original word set x o All the alternative words of the original word segmentation are combined in turn according to their positions to obtain the adversarial sample set of the original sentiment sentence text, and the adversarial sample set is used as the adversarial sample search space; S32: Inputting the adversarial sample in the adversarial sample search space into the first text classification model to obtain the confidence of the first text classification model for the adversarial sample class label; The calculation process of the optimal adversarial example specifically includes: S321: Based on the original word segmentation The path information between the nodes in the undirected graph and the initial node, as well as the confidence of the adversarial sample class label, are used to construct the scoring function of the particle swarm algorithm; The score function of the particle swarm algorithm includes: Among them, x a represents an adversarial example in the adversarial example search space, Represents x a The alternative word for the nth original participle in , Represents the nth original word The set of all replaceable words; C(x a ) represents the first text classification model for x a The confidence of the category label, Represents x o and x a The sum of the path information of all the alternative words of the original word in the undirected graph; S322: Randomly initialize the speed and position of particles in the particle swarm; S323: Calculate the fitness value of the particle in the particle swarm by using the score function. When the current fitness value of the particle is greater than the historical best fitness value of the particle, replace the historical best fitness value of the particle with the current fitness value of the particle, and record the current position of the particle as the individual previous best position; when the historical best fitness value of the particle is greater than the historical best fitness value of the group particles, replace the historical best fitness value of the group particles with the historical best fitness value of the particle, and record the current position of the particle as the global previous best position; S324: Update the speed and position of particles; S325: Repeat steps S323-S324 until the iteration ends, output the best fitness value of the current group of particles, and obtain the optimal adversarial sample in the adversarial sample search space based on the best fitness value of the current group of particles; S33: Using the improved particle swarm algorithm to The path information between the nodes in the undirected graph and the initial node, as well as the confidence of the adversarial sample category label, are used to calculate the optimal adversarial sample from the adversarial sample search space; S4: Use the original sentiment sentence text dataset and the optimal adversarial sample as the second training sample, perform adversarial training on the first text classification model, and obtain the second text classification model; S5: Obtain the target text to be classified, input the target text into the second classification model for classification, and output the classification result.

2. A text classification method based on an improved particle swarm algorithm according to claim 1, characterized in that: The path information of the initial node of the original word is marked as 0. Pinyin nodes and original word segmentation English translation nodes and original word segmentation The path information of the similar word node is marked as 1, and the original word segmentation Pinyin nodes and original word segments The path information of the pinyin initial letter node is marked as 2, the original word segmentation The path information of the English transliteration node is marked as 3.

3. The text classification method based on the improved particle swarm algorithm according to claim 2 is characterized in that: The score function of the particle swarm algorithm includes: v=v1…v n …v N Among them, v n represent The probability of being replaced by other words, represents the nth dimension of the global previous optimal position of the entire particle swarm, S(*) is the sigmoid function, and w is the non-negative inertia weight; and is a random positive acceleration coefficient; p represents the individual previous optimal position of the particle; p g represents the global previous optimal position of the entire particle swarm, r refers to a random number selected from [0.0, 1.0], T sets the probability threshold, Represents the original participle In the undirected graph A random neighboring node of .

Images