An adversarial multi-label sensitive text classification method, system, device and medium

Abstract

The application discloses a kind of confrontation multi-label sensitive text classification method, system, equipment and medium, comprising: constructing self-adapting label co-occurrence graph dynamically mining inter-label association, obtaining label topology representation;Using pre-training encoder extracts original sample feature representation, and adds noise disturbance to construct confrontation sample feature representation;Through the class relevance of attention mechanism label is mapped into sample feature representation, obtains the label association feature representation of two kinds of samples, and calculates the loss of two kinds of samples to carry out confrontation training;Using the multi-label classification model of trained sensitive text classification is classified.The application identifies label topology relationship by dynamically updating label co-occurrence matrix, enhances the robustness of model to noise disturbance by combining confrontation training method, improves classification accuracy.

Description

Technical Field

[0001] This invention relates to the field of data privacy and security technology, and in particular to an adversarial multi-label sensitive text classification method, system, device and medium. Background Technology

[0002] Multi-label sensitive text classification, by assigning multiple category labels to text, can comprehensively reflect its attributes and support fine-grained security policy formulation. Existing mainstream methods are mainly divided into two categories: sample representation learning and label association modeling. The former utilizes deep learning models to enhance text semantic capture, but often ignores the broad associations between labels, leading to inaccurate mapping between samples and labels. The latter, while mining label correlations through graph neural networks, relies heavily on static statistical information to construct relationships, making it difficult to accurately model unseen samples and rare labels, and failing to capture complete and dynamic label topological features.

[0003] In summary, existing technologies face two main challenges in practical applications: first, uneven sample distribution leads to a bias towards high-frequency labels, reducing the prediction accuracy for minority class samples; second, insufficient label relationship mining, relying solely on co-occurrence counts, is insufficient to adapt to complex scenarios, resulting in weak recognition capabilities for unknown new samples. To address these issues, this invention proposes an adversarial multi-label sensitive text classification method based on an adaptive label co-occurrence graph. This method aims to improve the classification accuracy and robustness of the model under complex distributions by dynamically mining label associations and combining them with adversarial training. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides an adversarial multi-label sensitive text classification method, system, device, and medium to solve the problem that existing multi-label sensitive text classification methods are difficult to fully explore the deep correlation between labels and are easily affected by uneven sample distribution, resulting in weak prediction accuracy and generalization ability of the model for minority class samples and unknown new samples.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: In a first aspect, the present invention provides an adversarial multi-label sensitive text classification method, comprising: Obtain a set of historical sensitive samples and the corresponding set of historical classification labels. Construct an adaptive label co-occurrence graph based on the set of historical sensitive samples and the set of historical classification labels to dynamically mine the associations between labels and obtain the label topology representation. The original sample feature representation is extracted using a pre-trained encoder, and adversarial example feature representation is constructed by adding noise perturbation based on the original sample feature representation. The class correlation in the label topology representation is mapped to the original sample feature representation and the adversarial sample feature representation using an attention mechanism, respectively. The label association feature representation of the original sample and the label association feature representation of the adversarial sample are obtained. The joint loss is calculated based on the label association feature representation of the original sample and the label association feature representation of the adversarial sample to perform adversarial training, and the trained multi-label classification model is obtained. The trained multi-label classification model is used to classify sensitive texts to obtain multi-label classification results.

[0007] As a preferred embodiment of the adversarial multi-label sensitive text classification method of the present invention, the step of obtaining a historical sensitive sample set and a corresponding historical classification label set, and constructing an adaptive label co-occurrence graph based on the historical sensitive sample set and the historical classification label set to dynamically mine the associations between labels and obtain a label topological representation includes: Obtain the historical sensitive sample set and the historical classification label set; Input the historical classification label set into the pre-trained language model to obtain the label node representation vector; Based on the co-occurrence information of the historical classification label set, a learnable and adaptively updated dynamic co-occurrence matrix is ​​constructed. Based on the dynamic co-occurrence matrix and the label node representation vector, the association information between historical classification labels is aggregated through a graph attention network to obtain the label topology representation.

[0008] The beneficial effect of this preferred technical solution is that by combining the semantic prior of the pre-trained language model with the learnable dynamic co-occurrence matrix, and by using a graph attention network to adaptively aggregate label association information, it can effectively capture the complex semantic dependencies between labels, thereby obtaining a more discriminative label topological representation and improving the accuracy of multi-label classification.

[0009] As a preferred embodiment of the adversarial multi-label sensitive text classification method of the present invention, the step of extracting original sample feature representations using a pre-trained encoder and constructing adversarial sample feature representations by adding noise perturbations based on the original sample feature representations includes: The historical sensitive sample set is encoded using a pre-trained encoder to obtain the original sample feature representation; Calculate the gradient of the original classification loss with respect to the word embedding layer of the pre-trained encoder; Noise is generated based on the gradient and injected into the word embedding layer to construct the adversarial example feature representation.

[0010] The beneficial effect of this preferred technical solution is that by injecting noise perturbation based on the original classification loss gradient into the word embedding layer, it is possible to construct strong adversarial samples targeting the weak links of the model, forcing the model to learn more robust feature representations during training, thereby effectively improving the model's classification stability and generalization ability when facing perturbations or malicious attacks.

[0011] As a preferred embodiment of the adversarial multi-label sensitive text classification method of the present invention, wherein: the step of using an attention mechanism to map the category correlation in the label topology representation to the original sample feature representation and the adversarial sample feature representation, respectively, to obtain the label association feature representation of the original sample and the label association feature representation of the adversarial sample, includes: Calculate the attention weight between the label topological representation and the original sample feature representation, and aggregate them accordingly to obtain the label association feature representation of the original sample; Calculate the attention weight between the label topology representation and the adversarial example feature representation, and then aggregate them accordingly to obtain the label association feature representation of the adversarial example.

[0012] As a preferred embodiment of the adversarial multi-label sensitive text classification method of the present invention, the step of calculating the joint loss based on the label association feature representation of the original sample and the label association feature representation of the adversarial sample for adversarial training to obtain the trained multi-label classification model includes: The original classification loss is calculated based on the label association feature representation of the original samples; The adversarial classification loss is calculated based on the label association feature representation of the adversarial examples; The parameters of the initial model are jointly iteratively updated based on the original classification loss and the adversarial classification loss, and the trained multi-label classification model is obtained after convergence.

[0013] As a preferred embodiment of the adversarial multi-label sensitive text classification method of the present invention, the step of classifying sensitive text using the trained multi-label classification model to obtain multi-label classification results includes: Get the sensitive text to be classified; The sensitive text to be classified is input into the trained multi-label classification model to obtain the corresponding multi-label classification result.

[0014] As a preferred embodiment of the adversarial multi-label sensitive text classification method of the present invention, wherein: calculating the attention weight between the label topological representation and the original sample feature representation includes: The label topology representation is used as the key and value of the attention mechanism, and the original sample feature representation is used as the query of the attention mechanism. The attention weight is calculated through dot product attention to map the category relevance in the label topology representation to the original sample feature representation.

[0015] Secondly, the present invention provides an adversarial multi-label sensitive text classification system, comprising: The label topology representation acquisition module is used to acquire the historical sensitive sample set and the corresponding historical classification label set, and construct an adaptive label co-occurrence graph based on the historical sensitive sample set and the historical classification label set to dynamically mine the association between labels and obtain the label topology representation. An adversarial example construction module is used to extract original sample feature representations using a pre-trained encoder and to construct adversarial example feature representations by adding noise perturbations based on the original sample feature representations. The model adversarial training module is used to map the category correlation in the label topology representation to the original sample feature representation and the adversarial sample feature representation using an attention mechanism, respectively, to obtain the label association feature representation of the original sample and the label association feature representation of the adversarial sample, and to calculate the joint loss based on the label association feature representation of the original sample and the label association feature representation of the adversarial sample for adversarial training, so as to obtain the trained multi-label classification model. The sensitive text classification module is used to classify sensitive text using the trained multi-label classification model to obtain multi-label classification results.

[0016] Thirdly, the present invention provides a computer device, comprising: Memory, used to store programs; A processor is configured to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the adversarial multi-label sensitive text classification method.

[0017] Fourthly, the present invention provides a computer-readable storage medium comprising: when the program is executed by a processor, the step of implementing the adversarial multi-label sensitive text classification method.

[0018] The beneficial effects of this invention are as follows: By constructing a learnable dynamic co-occurrence matrix and combining it with a graph attention network, this invention achieves adaptive mining and aggregation of deep relationships between labels, thereby overcoming the limitations of traditional methods that rely solely on static statistical information and effectively improving the model's accuracy in recognizing rare labels and unknown samples. By injecting gradient-based adversarial noise into the word embedding layer, targeted enhancement of the model's weak points is achieved, forcing the model to learn more robust feature representations, thus significantly improving the stability and generalization ability of the classification model when facing perturbations or malicious attacks. By utilizing the attention mechanism to map the class correlation in the label topological representation to the original and adversarial sample features, and performing adversarial training based on joint loss, a deep fusion of sample semantics and label association information is achieved, thereby solving the problem of inaccurate sample-label mapping modeling and further improving the overall accuracy of multi-label sensitive text classification. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein: Figure 1 A basic flowchart of an adversarial multi-label sensitive text classification method provided in one embodiment of the present invention is shown below; Figure 2 A flowchart illustrating the training process of a classification model for an adversarial multi-label sensitive text classification method according to an embodiment of the present invention; Figure 3 A timing diagram of an adversarial multi-label sensitive text classification method provided in one embodiment of the present invention; Detailed Implementation

[0020] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0021] Example 1, referring to Figure 1 As an embodiment of the present invention, an adversarial multi-label sensitive text classification method is provided, comprising: S100: Obtain the historical sensitive sample set and the corresponding historical classification label set, and construct an adaptive label co-occurrence graph based on the historical sensitive sample set and the historical classification label set to dynamically mine the association between labels and obtain the label topology representation; S200: Use a pre-trained encoder to extract the original sample feature representation, and add noise perturbation based on the original sample feature representation to construct adversarial example feature representation; S300: Using an attention mechanism, the category correlation in the label topology representation is mapped to the original sample feature representation and the adversarial sample feature representation, respectively, to obtain the label association feature representation of the original sample and the label association feature representation of the adversarial sample. Based on the label association feature representation of the original sample and the label association feature representation of the adversarial sample, the joint loss is calculated to perform adversarial training, and the trained multi-label classification model is obtained. S400: Use the trained multi-label classification model to classify sensitive texts and obtain multi-label classification results.

[0022] It should be noted that in sensitive text multi-label classification tasks, text may belong to multiple categories simultaneously, and the sample distribution of different labels is extremely uneven. This causes the model to tend to predict high-frequency labels during training, while its ability to identify minority class samples is insufficient. Furthermore, existing methods often rely on fixed label co-occurrence statistics to model label relationships, making it difficult to fully explore the deep topological connections between labels, resulting in weak modeling capabilities for unknown new samples and rare labels. In addition, the model faces challenges in classification stability and generalization ability when dealing with perturbed text, making it difficult to meet the data security field's requirements for fine-grained and highly robust sensitive content identification.

[0023] Therefore, to address the problem that existing multi-label sensitive text classification methods suffer from weak prediction accuracy and generalization ability for minority class samples and unknown new samples due to the difficulty in fully exploring the deep correlations between labels and the susceptibility to uneven sample distribution, this paper proposes a method through steps S100-S400. This method constructs an adaptively updated dynamic label co-occurrence graph to fully explore the deep topological correlations between labels and combines it with an adversarial training mechanism to enhance the model's robustness to noise disturbances. This effectively solves the problem of low prediction accuracy for minority labels caused by insufficient label correlation mining and uneven sample distribution in traditional methods, thereby significantly improving the accuracy and generalization ability of multi-label sensitive text classification.

[0024] Example 2, refer to Figures 2-3 As an embodiment of the present invention, an adversarial multi-label sensitive text classification method is provided based on the previous embodiment, including: In this embodiment of the application, step S100 involves obtaining a historical sensitive sample set and the corresponding historical classification label set, constructing an adaptive label co-occurrence graph based on the historical sensitive sample set and the historical classification label set to dynamically mine the associations between labels and obtain a label topology representation, including: The specific process is as follows: Figure 2 As shown in the left branch, the historical sensitive sample set and the historical classification label set are obtained; Input the historical classification label set into the pre-trained language model to obtain the label node representation vector; Based on the co-occurrence information of the historical classification label set, a learnable and adaptively updated dynamic co-occurrence matrix is ​​constructed. Based on the dynamic co-occurrence matrix and the label node representation vector, the association information between historical classification labels is aggregated through a graph attention network to obtain the label topology representation.

[0025] In this embodiment, step S100 involves obtaining a set of historically sensitive samples and their corresponding historical classification labels, and performing word segmentation on the historically sensitive samples in the set to obtain corresponding text word sequences. These text word sequences are then input into a pre-set BERT model (i.e., the pre-trained encoder). The word embedding layer of the BERT model obtains the word embedding representations corresponding to each word in the text word sequence, and the encoding layer of the BERT model performs contextual semantic encoding on the word embedding representations to obtain the corresponding sample representation vectors. Specifically, given a set of historically sensitive samples… , Indicates the first For each sample, the first [CLS] vector of the hidden layer of the BERT model is used as the sample word embedding representation. The specific process is as follows: in, Represents the BERT encoder. Representing the Each sample represents a vector.

[0026] In this embodiment, step S100 involves constructing a learnable dynamic co-occurrence matrix. Based on the number of labels in the historical classification label set, a learnable parameter matrix of the corresponding dimension is constructed. Initial values ​​of the learnable parameter matrix are randomly sampled from a normal distribution and then processed through nonlinear activation to form a learnable and adaptively updated dynamic co-occurrence matrix.

[0027] In an optional implementation, the initialization and updating of the dynamic co-occurrence matrix in step S100 can also be based on the actual co-occurrence frequency statistics of the historical classification label set, directly constructing a fixed label co-occurrence matrix (such as TF-IDF or PMI matrix), and inputting it into the model as prior knowledge.

[0028] In an optional implementation, the initialization and updating of the dynamic co-occurrence matrix in step S100 can also utilize the Probabilistic Label Tree method, which clusters the label space based on the semantic similarity between labels and implicitly expresses the association between labels through the hierarchical division of the tree structure.

[0029] Specifically, the initial input to the dynamic co-occurrence matrix is ​​a two-dimensional matrix A randomly sampled from the standard normal distribution, and processed by the ReLU activation function, as shown in the following process: in, It is a learnable parameter whose initial value is a two-dimensional matrix randomly sampled from a normal distribution with a mean of 0 and a standard deviation of 1. It represents the standard normal distribution. Let be the dynamic co-occurrence matrix at the initial time. It is an activation function used to reduce weak connections that have little impact on the output.

[0030] In this embodiment, step S100 sets the weights of the dynamic co-occurrence matrix as updatable parameters, and updates these parameters during gradient backpropagation to allow the model to adaptively learn the fully interactive label node representation. The specific update process is as follows: in, express The loss of time, Indicates the learning rate. Indicates to The result of differentiation. This represents the label co-occurrence matrix at the next time step during the model training phase.

[0031] In this embodiment of the application, the specific process of obtaining the tag topology representation in step S100 is as follows: The SBERT encoder is used to encode each tag text in the historical classification tag set, generating corresponding tag node representations. Specifically, a tag text set L is defined. Given k labels, embed them using the SBERT model, selecting the [CLS] embedding from the last layer of the model as the label node representation. The specific process is as follows: in It is the original label text. Represents the SBERT encoder. This indicates the label node representation of the output.

[0032] Using the labeled nodes as graph nodes and the elements in the dynamic co-occurrence matrix as edge weights between nodes, a corresponding labeled graph is constructed. The attention coefficients between each labeled node and its neighboring labeled nodes in the labeled graph are calculated using a preset graph attention network (GAT). The features of the neighboring nodes of the same labeled node are aggregated according to the attention coefficients to obtain the aggregated features of each labeled node. The aggregated features of all labeled nodes are integrated to form a labeled topological representation.

[0033] In an alternative implementation, the generation of the label topological representation in step S100 can also utilize a pre-trained language model (such as BERT) to directly encode the label text, obtain the initial semantic embedding of the label, and then directly perform feature transformation through a fully connected layer (MLP), skipping the aggregation process of the graph structure.

[0034] In an optional implementation, the generation of the label topological representation in step S100 can also adopt the Hierarchical Contrastive Learning method, which constructs a tree-like hierarchical structure graph of labels and uses a contrastive loss function to learn the parent-child and sibling relationships between labels, thereby obtaining the label representation.

[0035] Specifically, the relationships between labels are mapped to graph-structured data, and GAT is used to assign weights to each node to weighted aggregate the features of its neighboring nodes. For a node... and its neighboring nodes Calculate the attention coefficient between the two. This is used as a semantic embedding of weights and labels. Multiplying, the features of all adjacent nodes are finally aggregated and sorted to obtain... Label topology representation The specific process is as follows: in, This represents a vector concatenation operation. It is a learnable parameter matrix. It is a node The set of adjacent nodes, and The activation function. The topological representation vector of the output label after GAT encoding. .

[0036] In this embodiment of the application, step S200 uses a pre-trained encoder to extract the original sample feature representation, and adds noise perturbation based on the original sample feature representation to construct an adversarial example feature representation, including: Combination Figure 2 right side and Figure 3 As shown in the process, a pre-trained encoder is used to encode the historical sensitive sample set to obtain the original sample feature representation; Calculate the gradient of the original classification loss with respect to the word embedding layer of the pre-trained encoder; Noise is generated based on the gradient and injected into the word embedding layer to construct the adversarial example feature representation.

[0037] In an optional implementation, the adversarial example feature representation in step S200 can also be constructed using the randomized smoothing technique, which directly adds isotropic Gaussian noise to the word embeddings of the original sample, thereby constructing adversarial examples through random perturbation.

[0038] In an optional implementation, the construction of adversarial sample feature representation in step S200 can also utilize a large language model (LLM) as a "creator" to perform minimal synonym substitution or word order adjustment on the original text, generating text-level adversarial samples while maintaining semantic integrity.

[0039] Specifically, such as Figure 2 As shown by the dashed arrow, gradient backpropagation is performed to calculate the gradient of the original classification loss with respect to the sample input embedding, which is used to generate adversarial noise. This gradient is denoted as... The specific process is as follows: The gradient is normalized to obtain a normalized gradient, which is then scaled by a preset perturbation coefficient to generate directional noise. A perturbation variable δ is constructed, initially set to 0. Within a preset number of iterations, the perturbation variable (i.e., directional noise) is calculated by progressively accumulating the normalized gradient of the original samples. The specific process is as follows: in, Represents gradient L2 norm, The perturbation coefficient is used to constrain the perturbation from becoming too large and causing semantic information loss. Directional noise is superimposed on the word embedding layer of historical sensitive samples to obtain adversarial word embedding representations, and adversarial examples are generated based on these representations. The perturbation variable is accumulated into the original samples to generate adversarial examples. The specific process is as follows: In this embodiment of the application, step S300 utilizes an attention mechanism to map the category correlation in the label topology representation to the original sample feature representation and the adversarial example feature representation, respectively, to obtain the label association feature representation of the original sample and the label association feature representation of the adversarial example, including: Calculate the attention weight between the label topological representation and the original sample feature representation, and aggregate them accordingly to obtain the label association feature representation of the original sample; Calculate the attention weight between the label topology representation and the adversarial example feature representation, and then aggregate them accordingly to obtain the label association feature representation of the adversarial example.

[0040] In this embodiment of the application, step S300, calculating the attention weight between the label topological representation and the original sample feature representation, includes: The label topology representation is used as the key and value of the attention mechanism, and the original sample feature representation is used as the query of the attention mechanism. The attention weight is calculated through dot product attention to map the category relevance in the label topology representation to the original sample feature representation.

[0041] In an optional implementation, the mapping mechanism from category relevance to sample features in step S300 can also adopt a two-stream interaction architecture, mapping text and labels to the same space, and using a cross-attention mechanism to interact with the text representation using the label embedding as a query to extract fine-grained dependencies.

[0042] In an optional implementation, the mapping mechanism from category relevance to sample features in step S300 can also employ a joint embedding module, which directly concatenates or adds element-wise the text feature vector to the label topological representation vector, and then performs nonlinear fusion through a multilayer perceptron (MLP).

[0043] Specifically, a dot product attention mechanism is used to calculate the similarity between the sample representation vector and each label topological representation, and the sample-label attention weight is obtained based on the similarity. All label topological representations are then weighted and aggregated according to the sample-label attention weight to obtain a label aggregate vector. This label aggregate vector is then fused with the sample representation vector to generate the corresponding sample feature representation. The original sample-label attention weight is calculated using the dot product attention mechanism, mapping the label topological representation to the sample representation vector to obtain the original sample feature representation related to the label. The specific process is as follows: in, Indicates text embedding The embedding vector dimension, express The transpose of .

[0044] In this embodiment of the application, step S300, which calculates the joint loss based on the label association feature representation of the original sample and the label association feature representation of the adversarial sample to perform adversarial training and obtain the trained multi-label classification model, includes: The original classification loss is calculated based on the label association feature representation of the original samples; The adversarial classification loss is calculated based on the label association feature representation of the adversarial examples; like Figure 2 As shown in the loss convergence node (⊕), the parameters of the initial model are jointly iteratively updated based on the original classification loss and the adversarial classification loss, and the trained multi-label classification model is obtained after convergence. Specifically, after obtaining the original sample feature representations related to the labels, the sigmoid function is used to obtain the probability of the original sample in each category, and the original classification loss is calculated by combining it with the cross-entropy loss function, denoted as the original classification loss. The specific process is as follows: in, Representing the The true label of each sample It is the sigmoid activation function.

[0045] For the adversarial examples, the same dot-product attention mechanism as the original samples is used to calculate the adversarial example-label attention weights, and the adversarial example feature representations are weighted and aggregated to obtain label-related adversarial example feature representations. The sigmoid function is used to obtain the probability of the adversarial example in each category, and the adversarial classification loss is calculated using the cross-entropy loss function. Based on this, the gradient of the adversarial example is obtained by backpropagation of the adversarial classification loss. The gradient calculation process for the adversarial examples is consistent with the gradient calculation process for the original examples.

[0046] The original classification loss and the adversarial classification loss are weighted and summed to obtain the joint loss. The joint loss is then backpropagated, and the gradients of the original samples and the adversarial samples are accumulated to update the model parameters. The specific process is as follows: like Figure 3 As shown by the self-looping arrow in step ⑤, the adversarial example construction and joint adversarial training are repeated until the joint loss converges or the preset number of iterations is reached, resulting in a trained multi-label classification model.

[0047] In this embodiment of the application, step S400 uses the trained multi-label classification model to classify sensitive text and obtain multi-label classification results, including: Get the sensitive text to be classified; The sensitive text to be classified is input into the trained multi-label classification model to obtain the corresponding multi-label classification result.

[0048] In this embodiment of the application, step S400 sets a sensitive text set to be classified. , This represents the i-th sample, which is a sample from the sensitive text set to be classified. Input the trained multi-label classification model to obtain the multi-label classification result set of the samples. ,in Indicates the first A tag.

[0049] In this embodiment of the application, step S400 uses at least one of the following metrics to evaluate the multi-label classification results: precision, normalized loss cumulative gain (nDCG), and propensity score precision (PSP).

[0050] Example 3, referring to Table 1, is an embodiment of the present invention. This embodiment provides an adversarial multi-label sensitive text classification method. In order to verify the beneficial effects of the present invention, specific implementation methods and implementation effects are used for scientific demonstration.

[0051] The specific details of this embodiment are as follows: To verify the effectiveness of the adversarial multi-label sensitive text classification method constructed by the adaptive label co-occurrence graph described in this invention, the following experiments were conducted.

[0052] I. Experimental Environment This experiment was conducted on two sets of hardware: Computer: CPU is Intel i7-7500 (2.70GHz), memory is 8GB, operating system is Windows 10 (64-bit).

[0053] Server: GPU is NVIDIA TESLA V100 (32G VRAM), operating system is Linux Ubuntu 20.04.

[0054] II. Experimental Procedure 1. Dataset Preparation: The experiment uses two widely used public datasets for multi-label text classification: AAPD and RCV1. Detailed statistical information of the datasets is shown in Table 1.

[0055] Table 1. Experimental Data of Multi-Label Text Classification (strips)

[0056] 2. Model Configuration: Pre-trained models: paraphrase-distilroberta-base-v1 was used as the SBERT model, and BERT-base-uncased was used as the BERT model.

[0057] Hyperparameter settings: Maximum input text length is set to 320, training batch size is 32, learning rate is 5e-5, and the number of layers in the graph attention network (GAT) is 1.

[0058] This experiment uses three metrics commonly used in multi-label classification tasks to evaluate the results. The specific calculation formulas are as follows: 1) The calculation method for precision (Precision@k, P@k) is as follows: in, Represents the predicted score vector. Represents a truth label vector. Indicates the index of the tag. Indicates will First in descending order Maximum value index.

[0059] 2) Normalized loss cumulative gain The calculation process is shown in equations (2), (3), and (4). Wherein, 1 and 1. Equal.

[0060] in, Indicates the first The true relevance score of each result This indicates that the results are sorted from highest to lowest according to their true relevance. The number of sets consisting of the results.

[0061] 3) Propensity score precision The calculation method is shown below.

[0062] in, This represents the bias score of a label, which is directly proportional to the frequency of the label's occurrence; that is, the more frequently a label appears, the higher its bias score.

[0063] III. Experimental Results The method of this invention was tested on two datasets, AAPD and RCV1, and the specific results are as follows: On the AAPD dataset, , , , and The indicators were 89.1%, 64.8%, 43.5%, 85.4%, and 88.5%, respectively.

[0064] On the RCV1 dataset, , , , and The indicators were 98.8%, 85.7%, 58.8%, 96.6%, and 96.8%, respectively.

[0065] Experimental results show that the method of the present invention achieves excellent results in the multi-label classification task of sensitive data, verifying its effectiveness and advanced nature.

[0066] Example 4 is an embodiment of the present invention. This embodiment differs from the first embodiment in that it provides an adversarial multi-label sensitive text classification system.

[0067] It should be noted that the technical solution of this adversarial multi-label sensitive text classification system and the technical solution of the aforementioned adversarial multi-label sensitive text classification method belong to the same concept. For details not described in detail in this embodiment, please refer to the description of the technical solution of the aforementioned adversarial multi-label sensitive text classification method.

[0068] This embodiment presents an adversarial multi-label sensitive text classification system, comprising: The label topology representation acquisition module is used to acquire the historical sensitive sample set and the corresponding historical classification label set, and construct an adaptive label co-occurrence graph based on the historical sensitive sample set and the historical classification label set to dynamically mine the association between labels and obtain the label topology representation. An adversarial example construction module is used to extract original sample feature representations using a pre-trained encoder and to construct adversarial example feature representations by adding noise perturbations based on the original sample feature representations. The model adversarial training module is used to map the category correlation in the label topology representation to the original sample feature representation and the adversarial sample feature representation using an attention mechanism, respectively, to obtain the label association feature representation of the original sample and the label association feature representation of the adversarial sample, and to calculate the joint loss based on the label association feature representation of the original sample and the label association feature representation of the adversarial sample for adversarial training, so as to obtain the trained multi-label classification model. The sensitive text classification module is used to classify sensitive text using the trained multi-label classification model to obtain multi-label classification results.

[0069] This embodiment also provides a computer device applicable to an adversarial multi-label sensitive text classification method, including: The system includes a memory and a processor. The memory stores computer-executable instructions, and the processor executes these instructions to implement an adversarial multi-label sensitive text classification method as proposed in the above embodiments.

[0070] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, it implements an adversarial multi-label sensitive text classification method as proposed in the above embodiments.

[0071] The storage medium proposed in this embodiment and the method for implementing an adversarial multi-label sensitive text classification proposed in the above embodiments belong to the same inventive concept. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0072] Based on the above description of the implementation methods, those skilled in the art can clearly understand that the present invention can be implemented using software and necessary general-purpose hardware, and of course, it can also be implemented using hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of the various embodiments of the present invention.

[0073] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. An adversarial multi-label sensitive text classification method, characterized in that, include: Obtain a set of historical sensitive samples and the corresponding set of historical classification labels. Construct an adaptive label co-occurrence graph based on the set of historical sensitive samples and the set of historical classification labels to dynamically mine the associations between labels and obtain the label topology representation. The original sample feature representation is extracted using a pre-trained encoder, and adversarial example feature representation is constructed by adding noise perturbation based on the original sample feature representation. The class correlation in the label topology representation is mapped to the original sample feature representation and the adversarial sample feature representation using an attention mechanism, respectively. The label association feature representation of the original sample and the label association feature representation of the adversarial sample are obtained. The joint loss is calculated based on the label association feature representation of the original sample and the label association feature representation of the adversarial sample to perform adversarial training, and the trained multi-label classification model is obtained. The trained multi-label classification model is used to classify sensitive texts to obtain multi-label classification results.

2. The adversarial multi-label sensitive text classification method as described in claim 1, characterized in that: The process of obtaining a historical sensitive sample set and its corresponding historical classification label set, constructing an adaptive label co-occurrence graph based on the historical sensitive sample set and the historical classification label set to dynamically mine the associations between labels and obtain a label topological representation includes: Obtain the historical sensitive sample set and the historical classification label set; Input the historical classification label set into the pre-trained language model to obtain the label node representation vector; Based on the co-occurrence information of the historical classification label set, a learnable and adaptively updated dynamic co-occurrence matrix is ​​constructed. Based on the dynamic co-occurrence matrix and the label node representation vector, the association information between historical classification labels is aggregated through a graph attention network to obtain the label topology representation.

3. The adversarial multi-label sensitive text classification method as described in claim 1 or 2, characterized in that: The step of extracting original sample feature representations using a pre-trained encoder and constructing adversarial example feature representations by adding noise perturbations based on the original sample feature representations includes: The historical sensitive sample set is encoded using a pre-trained encoder to obtain the original sample feature representation; Calculate the gradient of the original classification loss with respect to the word embedding layer of the pre-trained encoder; Noise is generated based on the gradient and injected into the word embedding layer to construct the adversarial example feature representation.

4. The adversarial multi-label sensitive text classification method as described in claim 3, characterized in that: The step of using an attention mechanism to map the category relevance in the label topology representation to the original sample feature representation and the adversarial example feature representation, respectively, to obtain the label association feature representation of the original sample and the label association feature representation of the adversarial example, includes: Calculate the attention weight between the label topological representation and the original sample feature representation, and aggregate them accordingly to obtain the label association feature representation of the original sample; Calculate the attention weight between the label topology representation and the adversarial example feature representation, and then aggregate them accordingly to obtain the label association feature representation of the adversarial example.

5. The adversarial multi-label sensitive text classification method as described in claim 4, characterized in that: The process of calculating a joint loss based on the label association feature representation of the original sample and the label association feature representation of the adversarial sample for adversarial training, resulting in a trained multi-label classification model, includes: The original classification loss is calculated based on the label association feature representation of the original samples; The adversarial classification loss is calculated based on the label association feature representation of the adversarial examples; The parameters of the initial model are jointly iteratively updated based on the original classification loss and the adversarial classification loss, and the trained multi-label classification model is obtained after convergence.

6. The adversarial multi-label sensitive text classification method as described in claim 5, characterized in that: The process of classifying sensitive text using the trained multi-label classification model to obtain multi-label classification results includes: Get the sensitive text to be classified; The sensitive text to be classified is input into the trained multi-label classification model to obtain the corresponding multi-label classification result.

7. The adversarial multi-label sensitive text classification method as described in claim 6, characterized in that: The calculation of the attention weights between the label topological representation and the original sample feature representation includes: The label topology representation is used as the key and value of the attention mechanism, and the original sample feature representation is used as the query of the attention mechanism. The attention weight is calculated through dot product attention to map the category relevance in the label topology representation to the original sample feature representation.

8. An adversarial multi-label sensitive text classification system, employing the method described in any one of claims 1-7, characterized in that, include: The label topology representation acquisition module is used to acquire the historical sensitive sample set and the corresponding historical classification label set, and construct an adaptive label co-occurrence graph based on the historical sensitive sample set and the historical classification label set to dynamically mine the association between labels and obtain the label topology representation. An adversarial example construction module is used to extract original sample feature representations using a pre-trained encoder and to construct adversarial example feature representations by adding noise perturbations based on the original sample feature representations. The model adversarial training module is used to map the category correlation in the label topology representation to the original sample feature representation and the adversarial sample feature representation using an attention mechanism, respectively, to obtain the label association feature representation of the original sample and the label association feature representation of the adversarial sample, and to calculate the joint loss based on the label association feature representation of the original sample and the label association feature representation of the adversarial sample for adversarial training, so as to obtain the trained multi-label classification model. The sensitive text classification module is used to classify sensitive text using the trained multi-label classification model to obtain multi-label classification results.

9. A computer device, characterized in that, include: Memory, used to store programs; A processor for loading the program to perform the steps of the method as claimed in any one of claims 1-7.

10. A computer-readable storage medium storing a program, characterized in that, When the program is executed by a processor, it implements the steps of the method as described in any one of claims 1-7.

Images