Text classification method, device and storage medium based on tabu search algorithm

Abstract

The application discloses a text classification method and device based on a taboo search algorithm and a storage medium, and the method comprises the following steps: S1, collecting a text data set; S2, selecting an original text sample from the text data set, and generating a corresponding initial adversarial sample by using a Levy flight; S3, searching for an optimal adversarial sample in a search space by using the taboo search algorithm, and adjusting a search direction by using cross variation in the searching process; S4, training a deep learning model by using the original text sample and the optimal adversarial sample; and S5, inputting an actual text into the trained deep learning model to generate a text classification result. The deep learning model is trained by using the adversarial sample, so that the model remains stable and reliable under adversarial attacks, and the accuracy of text classification is improved.

Description

Technical Field

[0001] This invention relates to natural language processing technology, specifically to a text classification method, device, and storage medium based on a tabu search algorithm. Background Technology

[0002] In recent years, the rapid development of deep neural networks has led to their widespread application in the tourism industry. Through technologies such as intelligent recommendation systems, virtual tours, and automated customer service, tourists can receive more personalized and convenient services. For example, travel platforms can recommend suitable travel routes and hotels based on users' historical behavior and preferences, and intelligent assistants can answer various questions from tourists in real time. This not only enhances the user experience but also drives the digital transformation of the tourism industry.

[0003] However, research shows that the local linearity and high-dimensional data processing characteristics of deep neural networks make them vulnerable to adversarial attacks. These attacks can lead to incorrect travel recommendations, pricing errors, or itinerary confusion, impacting user experience and even causing financial losses. For example, malicious adversarial examples can mislead recommendation systems into providing irrelevant travel products or services through subtle changes in input.

[0004] Since the concept of adversarial examples was introduced, much research has focused on computer vision, exploring how to generate and defend against them. However, research in tourism-related natural language processing (NLP) applications, such as intelligent customer service and review analysis, remains relatively limited. The robustness and security of NLP models are equally critical and require further attention and research to ensure that intelligent systems in the tourism industry remain stable and reliable under adversarial attacks. By strengthening research on adversarial examples, the tourism industry can improve system security and user trust, promoting the further development of digital services. Summary of the Invention

[0005] This invention addresses the shortcomings of existing technologies by providing a text classification method, device, and storage medium based on a tabu search algorithm. First, the population is initialized using Lévy flight, aiming to ensure the initial population broadly covers the search space to find optimal adversarial examples. Second, during the tabu search process, the tabu list is dynamically adjusted adaptively. By adjusting the length of the tabu list, premature convergence and local optima are prevented. Then, crossover and mutation are employed, and the mutation probability is dynamically adjusted by considering population diversity and the variance of the fitness of the most recent k generations to improve search capability.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] A text classification method based on tabu search algorithm includes the following steps:

[0008] S1. Collect text datasets;

[0009] S2. Select original text samples from the text dataset and use Levi Flight to generate corresponding initial adversarial samples;

[0010] S3. Search for the optimal adversarial example in the search space using the tabu search algorithm, and adjust the search direction through crossover mutation during the search process;

[0011] S4. Train a deep learning model using original text samples and optimal adversarial samples;

[0012] S5. Input the actual text into the trained deep learning model to generate text classification results.

[0013] To optimize the above technical solution, the specific measures also include:

[0014] Furthermore, the text dataset includes original text samples and corresponding category labels.

[0015] Furthermore, the specific process of selecting original text samples from the text dataset and generating corresponding initial adversarial samples using Levi's Flight is as follows:

[0016] S2.1 Select original text samples X from the text dataset, denoted as X = (w1, w2, ..., w...). i , ...,w n ), where w i This represents the i-th word in the text, and at the same time, the true class label Y of the text sample is obtained as supervision information;

[0017] S2.2 Calculate w for each word i The influence weights I(w) on the prediction results of the target model i The formula is as follows:

[0018]

[0019] in, This means removing the word "w". i The text after that, f(X) is the prediction confidence of the target model for the original text sample. The influence weight is used to measure the importance of each word in the model decision and is arranged in descending order of weight. The top k words with the highest influence are selected as the attack targets.

[0020] S2.3. Generate a step size s based on the Lévy distribution. The step size s determines how many words to modify. Randomly sample from the set of synonyms of the target word to replace the selected word and generate the initial adversarial sample.

[0021] The formula for generating the step size based on the Lévy distribution is as follows:

[0022]

[0023] In the formula, μ is the scaling parameter, v is a random variable, μ, v ~ N(0, 1), and β is the stability index, β∈(1, 2).

[0024] Furthermore, step S3 specifically includes:

[0025] S3.1 Initialize the tabu list and use the initial adversarial sample as the current optimal solution;

[0026] S3.2 Calculate the influence weight of each word in the original text sample X on the prediction result of the target model. Select the top k words with the largest influence weight from the current original text sample X, and generate neighborhood solutions from the synonym set of these words and add them to the candidate solution set.

[0027] S3.3 Select two individuals X from the current optimal solution. A X B Calculate the difference between the two, and find the word with the largest cross-influence between the two individuals to obtain the offspring solution X′;

[0028] S3.4. Dynamically adjust the mutation probability and mutate the child solutions according to the mutation probability, and add them to the candidate solution set;

[0029] S3.5. Use the fitness function to evaluate the effect of each solution in the candidate solution set as an adversarial example, and select the solution with the largest fitness value from the candidate solution set as the new solution X′;

[0030] The fitness function is as follows:

[0031] F(X)=μ1·(1-f(X))+μ2·Sim(X,X0)-μ3·w(X) (1.3)

[0032] Where F(X) is the fitness, f(x) is the confidence of the target model in predicting the original class of the original text sample X, Sim(X, X0) is used to calculate the semantic similarity between the adversarial sample X0 and the original text sample X, w(X) is the proportion of modified words; μ1 is the first weight, μ2 is the second weight, and μ3 is the third weight.

[0033] S3.6 If the new solution X′ is not in the tabu list, then determine whether its fitness is better than the current best solution X. best If X′ is better, then update the optimal solution and add it to the tabu list;

[0034] S3.7 Dynamically adjust the size L of the taboo table, using the following formula:

[0035] L = max(L)min ,min(L max ,L+ΔL)) (1.4)

[0036] Among them, L min L is the minimum value of the taboo list. max ΔL is the maximum value of the tabu list. ΔL is a factor used to adjust the size of the tabu list. If there is no improvement for k consecutive rounds, then ΔL = -1, reducing the size of the tabu list. If a better solution is found in the current round, then ΔL = +1, increasing the size of the tabu list.

[0037] S3.8 Determine whether the maximum number of iterations has been reached. If so, output the optimal solution as the optimal adversarial sample; otherwise, return to step S3.2.

[0038] Furthermore, in S3.4, the formula for dynamically adjusting the mutation probability is specifically as follows:

[0039] P m =P m,base +λ·D+φ·Variance(fitness) (1.5)

[0040]

[0041] Among them, P m Let P be the mutation probability. m,base Let D be the basic variation rate, D be the population diversity (calculated as shown in Formula 1.6), Variance (fitness) be the variance of fitness over the most recent k generations, λ and φ be adjustment parameters, and X be the variation rate. best X is the current optimal solution. i Let be the i-th original text sample, and HammingDistance be the proportion of different words in the text.

[0042] Furthermore, in S4, the deep learning model selected is the BERT model, and the structure of the BERT model includes: an input layer, an encoder layer, and an output layer;

[0043] The input layer integrates word embedding, position embedding, and segment embedding to convert text into a vector sequence and processes text semantic boundaries through classification tags and delimiters.

[0044] The encoder layer consists of 12 Transformer encoders, each containing a multi-head self-attention mechanism and a feedforward neural network.

[0045] The output layer extracts the output vector labeled with the classification tag, and outputs the classification probability through a fully connected layer and a Softmax activation function.

[0046] Furthermore, in S4, the specific steps of training the deep learning model using the original text samples and the optimal adversarial samples are as follows:

[0047] The cross-entropy loss function with adversarial perturbation constraints is adopted, as shown in Equation 1.7:

[0048]

[0049] In the formula, L is the loss value, M is the total number of training samples, and yi is the true class of the i-th training sample. λ is the predicted class of the i-th training sample, λ is the regularization coefficient, Embedding(X0) is the embedding vector of the adversarial sample X0, and Embedding(X) is the embedding vector of the original sample X.

[0050] The original text samples and adversarial samples are mixed in a 1:1 ratio. The AdamW optimizer is used, with an initial learning rate set and dynamically decayed using a cosine annealing strategy. The original text samples and adversarial samples are input into the deep learning model, and the validation set loss is monitored. If the loss does not decrease for three consecutive rounds, the training is terminated.

[0051] Furthermore, S5 specifically refers to:

[0052] S5.1. Preprocess the actual text by using BERT's built-in WordPiece segmenter to divide the text into sub-word units, fix the sequence length to the model's preset value by truncation or padding, retain the classification label and delimiter, generate input ID, attention mask and segment ID, and finally obtain the preprocessed tensor.

[0053] S5.2 Input the preprocessed tensor into the trained BERT model, and output a logistic vector with dimensions (batch_size, C), where batch size is the batch size and C is the number of classes. Process the logistic vector through the Softmax activation function to calculate the class probability distribution.

[0054] S5.3 Select the category with the highest probability as the classification result, and mark the actual text with a probability lower than the threshold τ as "uncertain" to trigger the manual review mechanism.

[0055] The present invention also proposes an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the text classification method based on the tabu search algorithm as described above.

[0056] The present invention also proposes a computer-readable storage medium storing a computer program that enables a computer to perform the text classification method based on the tabu search algorithm as described above.

[0057] The beneficial effects of this invention are:

[0058] 1. Initializing the population using Levi's flight method enhances the exploratory nature of the search and avoids the initial solution getting trapped in local optima. When exploring the global context, introducing large-scale perturbations helps escape local optima. During local optimization, short-distance searches can improve semantic similarity, ensuring natural text flow.

[0059] 2. Through adaptive tabu search, the search rules can be dynamically adjusted to balance local and global searches, avoid getting trapped in local optima, guide the search more effectively, and accelerate convergence to the optimal adversarial example.

[0060] 3. The crossover operation is introduced after the iterative search to integrate the advantages of multiple adversarial examples and improve the diversity and quality of the population. Then, through adaptive mutation probability, the mutation rate can be dynamically adjusted according to the current population diversity and fitness to balance local optimization and global search. Attached Figure Description

[0061] Figure 1 This is an overall flowchart of the text classification method based on the tabu search algorithm of the present invention.

[0062] Figure 2 This is a flowchart of the improved tabu search algorithm of the present invention. Detailed Implementation

[0063] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0064] Example 1

[0065] This invention proposes a text classification method based on a tabu search algorithm. The process of this method is as follows: Figure 1 As shown, it includes the following steps:

[0066] S1. Collect a text dataset; the text dataset includes original text samples and corresponding category labels.

[0067] S2. Select original text samples from the text dataset and use Levi's Flight to generate corresponding initial adversarial samples; the specific process is as follows:

[0068] S2.1 Select original text samples X from the text dataset, denoted as X = (w1, w2, ..., w...). i , ...,w n ), where w iThis represents the i-th word in the text, and at the same time, the true class label Y of the text sample is obtained as supervision information;

[0069] S2.2 To determine the focus of the adversarial attack, calculate the w of each word. i The influence weights I(w) on the prediction results of the target model i The formula is as follows:

[0070]

[0071] in, This means removing the word "w". i The text after that, f(X) is the prediction confidence of the target model for the original text sample. The influence weight is used to measure the importance of each word in the model decision and is arranged in descending order of weight. The top k words with the highest influence are selected as the attack targets.

[0072] S2.3. Generate step size s based on the Lévy distribution. Step size s determines how many words to modify. Randomly sample from the set of synonyms of the target word to replace the selected word and generate the initial adversarial sample. Adversarial examples are samples that add small perturbations to the original sample. They have the following characteristics: (1) Humans cannot detect them; (2) They cause the model to make incorrect predictions.

[0073] The formula for generating the step size based on the Lévy distribution is as follows:

[0074]

[0075] In the formula, μ is the scaling parameter, v is a random variable, μ, v ~ N(0, 1), and β is the stability index, β∈(1, 2).

[0076] S3. The optimal adversarial example is searched in the search space using the tabu search algorithm, and the search direction is adjusted through crossover mutation during the search process. The original tabu search algorithm is prone to getting trapped in local optima; therefore, this invention optimizes the search direction through crossover mutation. For example... Figure 2 As shown.

[0077] S3 specifically includes:

[0078] S3.1 Initialize the tabu list and use the initial adversarial sample as the current optimal solution;

[0079] S3.2 Calculate the influence weight of each word in the original text sample X on the prediction result of the target model (the calculation method is the same as that in Formula 1.1). From the current original text sample X, select the top k words with the largest influence weights, and generate neighborhood solutions from the synonym set of these words and add them to the candidate solution set.

[0080] S3.3 Select two individuals X from the current optimal solution. A X B The difference between the two is calculated, and the word with the highest weight is cross-affected by the two individuals to obtain the offspring solution X′; only the word with the highest weight is cross-affected to avoid destroying the readability of the text.

[0081] S3.4. Dynamically adjust the mutation probability, mutating offspring solutions according to the mutation probability and adding them to the candidate solution set. By dynamically adjusting the mutation probability, mutations can be flexibly controlled based on population diversity and fitness variance, effectively preventing premature population convergence and maintaining population vitality. The specific formula for dynamically adjusting the mutation probability is:

[0082] P m =P m，base +λ·D+φ·Variance(fitness) (1.5)

[0083]

[0084] Among them, P m Let P be the mutation probability. m，base Let D be the basic variation rate, D be the population diversity (calculated as shown in Formula 1.6), Variance (fitness) be the variance of fitness over the most recent k generations, λ and φ be adjustment parameters, and X be the variation rate. best X is the current optimal solution. i Let be the i-th original text sample, and HammingDistance be the proportion of different words in the text. This approach aims to increase the mutation probability, escape stagnation regions, and avoid local optima when population diversity and fitness are low.

[0085] S3.5. Use the fitness function to evaluate the effect of each solution in the candidate solution set as an adversarial example, and select the solution with the largest fitness value from the candidate solution set as the new solution X′;

[0086] The fitness function is as follows:

[0087] F(X)=μ1·(1-f(X))+μ2·Sim(X,X0)-μ3·w(X) (1.3)

[0088] Where F(X) is the fitness, f(x) is the confidence of the target model in predicting the original class of the original text sample X, Sim(X, X0) is used to calculate the semantic similarity between the adversarial sample X0 and the original text sample X, w(X) is the proportion of modified words; μ1 is the first weight, μ2 is the second weight, and μ3 is the third weight.

[0089] S3.6 If the new solution X′ is not in the tabu list, then determine whether its fitness is better than the current best solution X. best If X′ is better, then update the optimal solution and add it to the tabu list;

[0090] S3.7 Dynamically adjust the size L of the tabu list to balance search depth and the ability to escape local optima, as shown in the following formula:

[0091] L = max(L) min ,min(L max ,L+ΔL)) (1.4)

[0092] Among them, L min L is the minimum value of the taboo list. max ΔL is the maximum value of the tabu list. ΔL is a factor used to adjust the size of the tabu list. If there is no improvement for k consecutive rounds, then ΔL = -1, reducing the size of the tabu list. If a better solution is found in the current round, then ΔL = +1, increasing the size of the tabu list to avoid jumping out of the potential optimal solution too quickly.

[0093] S3.8 Determine whether the maximum number of iterations has been reached. If so, output the optimal solution as the optimal adversarial sample; otherwise, return to step S3.2.

[0094] S4. Train a deep learning model using the original text samples and the optimal adversarial samples; In this embodiment, the deep learning model selected is the BERT (Bidirectional Encoder Representations from Transformers) model. The structure of the BERT model includes: an input layer, an encoder layer, and an output layer.

[0095] The input layer integrates token embedding, position embedding, and segment embedding to convert the text into a vector sequence, and processes the semantic boundaries of the text through classification tags [CLS] and delimiters [SEP].

[0096] The encoder layer consists of 12 Transformer encoder layers, each containing a multi-head self-attention mechanism (12 attention heads to capture long-range semantic dependencies) and a feedforward neural network (FFN to enhance nonlinear representation of features).

[0097] The output layer extracts the output vector labeled with the classification tag, and outputs the classification probability through a fully connected layer and a Softmax activation function.

[0098] The specific steps for training a deep learning model using original text samples and optimal adversarial examples are as follows:

[0099] The cross-entropy loss function with adversarial perturbation constraints is adopted, as shown in Equation 1.7:

[0100]

[0101] In the formula, L is the loss value, M is the total number of training samples, and yi is the true class of the i-th training sample. is the predicted class of the i-th training sample, λ is the regularization coefficient, Embedding(X0) is the embedding vector of the adversarial sample X0, and Embedding(X) is the embedding vector of the original sample X; Cross-entropy loss term (first term): optimizes the classification accuracy of the model for the original sample and the adversarial sample;

[0102] Adversarial perturbation constraint (second term): The difference between the embedding vectors of the adversarial sample X0 and the original sample X is limited by the regularization coefficient λ (suggested value 0.1) to avoid semantic distortion.

[0103] The original text samples and adversarial samples were mixed in a 1:1 ratio, and the AdamW optimizer was used with an initial learning rate set to 5×10 in this implementation. -5 The model employs a cosine annealing strategy to dynamically decay the loss. It inputs the original text samples and adversarial samples into the deep learning model and monitors the validation set loss. If the loss does not decrease for three consecutive rounds, the training is terminated. This early stopping mechanism can prevent overfitting.

[0104] S5. Input the actual text into the trained deep learning model to generate text classification results. S5 specifically involves:

[0105] S5.1. Preprocess the actual text by using BERT's built-in WordPiece segmenter to segment the text into sub-word units (e.g., "unhappiness" → "un" "happiness"). Fix the sequence length to the model's preset value (e.g., 128 or 512) by truncation or padding. Preserve the classification label and delimiter, and generate input IDs, attention masks (used to mark the positions of valid tokens), and segment IDs (all 0 in single-text scenarios). Finally, obtain the preprocessed tensor.

[0106] S5.2 Input the preprocessed tensor into the trained BERT model, and output a logistic vector with dimensions (batch_size, C), where batch_size is the batch size and C is the number of classes. Process the logistic vector through the Softmax activation function to calculate the class probability distribution.

[0107] S5.3 Select the category with the highest probability as the classification result, and mark the actual text with a probability lower than the threshold τ (set to 0.5 in this embodiment) as "uncertain" to trigger the manual review mechanism.

[0108] Example 2

[0109] This invention proposes an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the text classification method based on the tabu search algorithm as described in Embodiment 1.

[0110] Example 3

[0111] This invention proposes a computer-readable storage medium storing a computer program that enables a computer to execute a text classification method based on a tabu search algorithm as described in Embodiment 1.

[0112] In the embodiments disclosed in this application, a computer storage medium may be a tangible medium that may contain or store programs for use by or in conjunction with an instruction execution system, apparatus, or device. The computer storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of computer storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0113] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0114] The above are merely preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should be considered within the scope of protection of the present invention.

Claims

1. A text classification method based on a tabu search algorithm, characterized in that, Includes the following steps: S1. Collect text datasets; S2. Select original text samples from the text dataset and generate corresponding initial adversarial samples using Levy Flight; the specific process of selecting original text samples from the text dataset and generating corresponding initial adversarial samples using Levy Flight is as follows: S2.1 Selecting original text samples from the text dataset X , represented as ,in, This represents the i-th word in the text, and simultaneously, it retrieves the true class label of the text sample. As supervisory information; S2.2 Calculate each word Weights of the impact on the prediction results of the target model The formula is as follows: in, Indicates removing words The following text It is the prediction confidence of the target model for the original text sample. The influence weight is used to measure the importance of each word in the model decision and is arranged in descending order of weight. The top k words with the highest influence are selected as the attack targets. S2.

3. Generate a step size s based on the Lévy distribution. The step size s determines how many words to modify. Randomly sample from the set of synonyms of the target word to replace the selected word and generate the initial adversarial sample. The formula for generating the step size based on the Lévy distribution is as follows: In the formula, It is a scale parameter. v It is a random variable. , β It is a stability index. ; S3. Search for the optimal adversarial example in the search space using the tabu search algorithm, and adjust the search direction through crossover mutation during the search process; Step S3 specifically includes: S3.1 Initialize the tabu list and use the initial adversarial sample as the current optimal solution; S3.2 Calculate the influence weight of each word in the original text sample X on the prediction result of the target model, starting from the current original text sample. In the middle, select the top with the greatest influence weight. Each word is used to generate neighborhood solutions from the set of synonyms of these words, and these solutions are added to the candidate solution set. S3.3 Select two individuals from the current optimal solution. Calculate the difference between the two, and find the word with the greatest cross-influence weight between the two individuals to obtain the child solution. ; S3.

4. Dynamically adjust the mutation probability and mutate the child solutions according to the mutation probability, and add them to the candidate solution set; S3.

5. Use the fitness function to evaluate the effectiveness of each solution in the candidate solution set as an adversarial example, and select the solution with the largest fitness value from the candidate solution set as the new solution. ; The fitness function is as follows: in, For fitness, For the target model, the original text sample Predict the confidence level of the original category. It is to calculate adversarial examples and original text sample semantic similarity, To modify the proportion of words; As the first weight, As the second weight, It is the third weight; S3.6, If a new solution is found If a solution is not in the tabu list, then determine whether its fitness is better than the current optimal solution. ,like If a better solution is found, the optimal solution is updated and added to the tabu list. S3.7 Dynamically adjust the size of the taboo table The formula is as follows: in, The minimum value of the taboo list, The maximum value of the taboo list, It is a factor used to adjust the size of the taboo list; if continuous... If the wheel is not improved, then Reduce the size of the tabu list; if a better solution is found in the current round, then... Increase the size of the taboo list; S3.8 Determine if the maximum number of iterations has been reached. If yes, output the optimal solution as the optimal adversarial sample; otherwise, return to step S3.

2. S4. Train a deep learning model using original text samples and optimal adversarial samples; S5. Input the actual text into the trained deep learning model to generate text classification results.

2. The text classification method based on the tabu search algorithm as described in claim 1, characterized in that, The text dataset includes original text samples and corresponding category labels.

3. The text classification method based on the tabu search algorithm as described in claim 1, characterized in that, In S3.4, the formula for dynamically adjusting the mutation probability is as follows: in, The mutation probability, Based on the basic rate of variation, For population diversity, the calculation method is shown in Formula 1.

6. For the most recent The variance of generation fitness It's about adjusting parameters. This is the current optimal solution. For the i-th original text sample, This represents the proportion of different words in the text.

4. The text classification method based on the tabu search algorithm as described in claim 1, characterized in that, In S4, the deep learning model selected is the BERT model. The structure of the BERT model includes: an input layer, an encoder layer, and an output layer. The input layer integrates word embedding, position embedding, and segment embedding to convert text into a vector sequence and processes text semantic boundaries through classification tags and delimiters. The encoder layer consists of 12 Transformer encoder layers, each containing a multi-head self-attention mechanism and a feedforward neural network; The output layer extracts the output vector labeled with the classification tag, and outputs the classification probability through a fully connected layer and a Softmax activation function.

5. The text classification method based on the tabu search algorithm as described in claim 1, characterized in that, In S4, the specific steps for training the deep learning model using the original text samples and the optimal adversarial samples are as follows: The cross-entropy loss function with adversarial perturbation constraints is adopted, as shown in Equation 1.7: In the formula, This is the loss value, and M is the total number of training samples. It is the true class of the i-th training sample. It is the predicted category of the i-th training sample. λ It is the regularization coefficient. Adversarial examples Embedded vector, It is the original sample The embedding vector; The original text samples and adversarial samples are mixed in a 1:1 ratio. The AdamW optimizer is used, with an initial learning rate set and dynamically decayed using a cosine annealing strategy. The original text samples and adversarial samples are input into the deep learning model, and the validation set loss is monitored. If the loss does not decrease for three consecutive rounds, the training is terminated.

6. The text classification method based on the tabu search algorithm as described in claim 1, characterized in that, S5 specifically refers to: S5.

1. Preprocess the actual text by using BERT's built-in WordPiece segmenter to divide the text into sub-word units. Fix the sequence length to the model's preset value by truncating or padding, retain the classification label and delimiter, generate input ID, attention mask and segment ID, and finally obtain the preprocessed tensor. S5.2 Input the preprocessed tensor into the trained BERT model, and output a logistic vector with dimensions (batch_size, C), where batch_size is the batch size and C is the number of classes. Process the logistic vector through the Softmax activation function to calculate the class probability distribution. S5.3 Select the category with the highest probability as the classification result, and mark the actual text with a probability lower than the threshold τ as "uncertain" to trigger the manual review mechanism.

7. An electronic device, characterized in that, include: The present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the text classification method based on the tabu search algorithm as described in any one of claims 1-6.

8. A computer-readable storage medium storing a computer program, characterized in that, The computer program causes the computer to perform the text classification method based on the tabu search algorithm as described in any one of claims 1-6.

Images