Text classification prediction method for multi-target joint optimization based on dynamic supervision driving
Through a dynamic supervision-driven multi-objective joint optimization method, combined with the weighted fusion of global and local structural features and the differential privacy mechanism, the shortcomings of the text classification model in performance, fairness and privacy protection are solved, a rich and robust representation of text features is achieved, and the accuracy and fairness of text classification are improved.
Patent Information
- Application Number
- CN202510827183.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-19
- Publication Date
- 2025-09-30
AI Technical Summary
Existing text classification models have shortcomings in performance optimization, fairness enhancement and privacy protection, and cannot take all of these into account at the same time. They also lack the ability to model local structures and complex semantics, which affects the generalization and robustness of the model.
A multi-objective joint optimization method based on dynamic supervision is adopted. Global structural features are extracted through singular value decomposition, and local structural features are extracted through structured generative adversarial networks. Combined with the attention mechanism and differential privacy mechanism, feature weighted fusion and noise processing are performed to construct a text representation space that takes into account both fairness and privacy protection.
It improves the structural integrity and discriminative ability of text feature expression, enhances the flexibility and practicality of privacy protection, improves the accuracy and fairness of the model, and is suitable for text intelligent classification tasks in diverse text scenarios.
Smart Images
Figure CN120723908A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the fields of artificial intelligence and natural language processing. Background Art
[0002] With the rapid development of artificial intelligence (AI), text classification, a fundamental task in natural language processing, has been widely applied in a variety of practical scenarios, including sentiment analysis, spam filtering, automatic news classification, and public opinion monitoring. In recent years, thanks to the widespread use of deep learning methods, particularly pre-trained language models (such as BERT and RoBERTa), text classification models have achieved significant improvements in accuracy and generalization.
[0003] However, despite continuous performance improvements, existing text classification models still face numerous challenges. On the one hand, due to attribute imbalance, sample bias, or leakage of sensitive features in training data, the model's prediction results for specific populations or labels may be unfair, leading to risks such as algorithmic discrimination. On the other hand, as models are deployed in real-world applications, they become highly sensitive to private information in input data (such as gender, age, and region), potentially leaking personal privacy and violating data compliance requirements. Therefore, while improving model performance, enhancing the model's ability to coordinate privacy protection and fairness has become a hot topic and a key challenge in current research.
[0004] Existing research largely focuses on optimizing a single objective: some methods introduce differential privacy mechanisms to reduce the risk of privacy leaks, but this trades off between maintaining model accuracy and semantic expression. Other studies have attempted to improve model fairness through adversarial training or reweighting, but these often overlook the model's ability to express diverse semantic structures and local details, impacting overall discriminative performance. Furthermore, the model's modeling of input features often focuses on global information, lacking deep understanding of local structure and complex semantics, limiting its generalization and robustness in diverse text scenarios.
[0005] Therefore, there is an urgent need for a multi-task adaptive optimization framework that simultaneously considers performance optimization, fairness enhancement, and privacy protection. Starting from the feature representation mechanism, it integrates multi-source structural information and noise perturbation strategies to construct a text representation space that takes into account expressiveness, security, and fairness, so as to comprehensively improve the practicality and controllability of the text classification system. Summary of the Invention
[0006] The purpose of this invention is to solve the problem that most existing classification methods focus on the optimization of a single objective and cannot simultaneously take into account performance optimization, fairness enhancement and privacy protection; the present invention provides a text classification prediction method based on dynamic supervision-driven multi-objective joint optimization.
[0007] A text classification prediction method based on multi-objective joint priority driven by dynamic supervision includes the following steps:
[0008] Step 1: Obtain the target text dataset and use the BERT model to convert each target text data in the target text dataset into a text feature vector;
[0009] Step 2: Extract global structural features from each text feature vector through singular value decomposition;
[0010] Step 3: Extract local structural features from each text feature vector through a structure-generated adversarial network;
[0011] Step 4: Calculate the reconstruction error between each text feature vector and its corresponding global structural feature and local structural feature;
[0012] Step 5: Perform weighted fusion of the corresponding global and local structural features according to the reconstruction errors corresponding to each text feature vector to obtain the fused feature vector corresponding to each text feature vector;
[0013] Step 6: Use the attention mechanism to dynamically add Laplace noise to each fused feature vector to obtain a feature vector containing noise;
[0014] Step 7: All feature vectors containing noise corresponding to all text feature vectors are formatted and encapsulated and then input into the feedforward neural network model to predict the category of the target text that takes into account both fairness and privacy protection.
[0015] Preferably, in step 4, the calculation of the reconstruction error between each text feature vector and its corresponding global structural feature and local structural feature is implemented as follows:
[0016] ;
[0017] ;
[0018] in, For the text feature vectors, for The global structural characteristics of for The local structural characteristics of for The reconstruction error between the corresponding global structural features, For the The reconstruction error between the text feature vector and the corresponding local structural feature, is the L2 norm.
[0019] Preferably, in step 5, the method for obtaining the fused feature vector corresponding to each text feature vector is as follows:
[0020] Definition The weight of the fused feature vector corresponding to the text feature vector The expression is:
[0021]
[0022] according to , get the The fused feature vector corresponding to the text feature vectors ;
[0023] in,
[0024] Preferably, ;
[0025] in, for The global structural characteristics of for local structural features.
[0026] Preferably, in step 6, the implementation method of obtaining the feature vector containing noise includes:
[0027] Calculate the attention score of each fused feature vector, use the Laplace algorithm to generate a random noise intensity value, multiply the generated random noise intensity value by the attention score corresponding to the fused feature vector to obtain the noise corresponding to the fused feature vector; add the noise to the corresponding fused feature vector to obtain a feature vector containing noise.
[0028] Preferably, the feedforward neural network model includes an encoder and a classifier;
[0029] The encoder consists of a fair fusion layer and two fully connected layers, and the classifier includes two fully connected layers;
[0030] After format conversion and encapsulation, each noisy feature vector is sequentially passed through the multi-view feature modeling module, label-guided attention weighted fusion module, label-aware perturbation generation module, contrastive representation learning module, and adversarial fairness training module in the fairness fusion layer to obtain a robust feature vector that is both discriminative and fair.
[0031] The robust feature vector with both discriminative and fair properties is then transmitted to the two fully connected layers of the encoder and the two fully connected layers of the classifier in sequence. After forward propagation of the received feature vector, each fully connected layer uses the ReLU activation function for gradient elimination and the Dropout function for regularization.
[0032] Preferably, the multi-view feature modeling module is used to perform multi-angle feature projection and fusion on each noise-containing feature vector received after format conversion and encapsulation;
[0033] The label-guided attention weighted fusion module is used to introduce label information to guide the attention mechanism to enhance the feature discriminability of feature projection and fused feature vectors;
[0034] The label-aware perturbation generation module is used to generate a perturbation signal based on the label information and inject the perturbation signal into the feature vector after enhancing the feature discriminability;
[0035] The contrastive representation learning module is used to combine the contrastive learning strategy to improve the robustness of the feature space structure of the feature vector after the perturbation signal is injected;
[0036] The adversarial fairness training module is used to explicitly suppress the leakage of sensitive attribute information in the feature vector after improving the robustness of the feature space structure through the adversarial training mechanism, thereby obtaining a robust feature vector that is both discriminative and fair.
[0037] Preferably, in step 1, the implementation method of using the BERT model to convert each target text data in the target text dataset into a text feature vector includes:
[0038] Step 1: Perform word segmentation on the target text data to obtain multiple phrases related to the text content, map the phrases into a high-dimensional vector space, and generate corresponding word vectors;
[0039] Step 12: Input the word vector into the pre-trained deep bidirectional transformer model to perform context encoding on the word vector;
[0040] Step 13: Aggregate the word vectors after context encoding to obtain the text feature vector.
[0041] Preferably, the structured generative adversarial network includes a generator and a discriminator;
[0042] The training phase of the structured generative adversarial network:
[0043] The generator learns to generate features with local structural information based on the received text feature vector;
[0044] The discriminator is used to determine whether the input features with local structural information have real local structural features, and guide the adversarial training through the judgment results to optimize the generation ability of the generator;
[0045] A local structure preservation loss function is introduced during adversarial training to constrain the consistency of the features generated by the generator in the local structure, thus completing the training of the generator.
[0046] Reasoning phase of the structured generative adversarial network:
[0047] The generator trained in the structured generative adversarial network extracts local structural features and outputs them based on the received text feature vector.
[0048] A text classification prediction device based on multi-objective joint optimization includes a storage device, a processor, and a computer program stored in the storage device and runnable on the processor. The processor executes the computer program to implement the text classification prediction method based on multi-objective joint optimization driven by dynamic supervision.
[0049] The beneficial effects brought by the present invention are:
[0050] The present invention provides a text classification prediction method based on dynamic supervision-driven multi-objective joint optimization. By introducing a weighted fusion mechanism of global and local structural features driven by reconstruction error, it can effectively improve the structural integrity and discrimination ability of text feature expression. It further combines the attention mechanism with the differential privacy mechanism, uses the attention score to dynamically adjust the intensity of Laplace noise addition, realizes differentiated treatment of different text samples in terms of privacy protection requirements, and enhances the flexibility and practicality of privacy protection. While ensuring the accuracy of text classification, it effectively suppresses the information loss problem caused by uniform noise addition, and improves the comprehensive performance of the model under multi-objective tasks.
[0051] In addition, local structural features are extracted through the structure-generative adversarial network, and the structure-preserving loss function is introduced for training, which enhances the model's ability to model local semantic structures, making the fused feature representation richer and more robust, and further improving the accuracy and fairness of text classification predictions.
[0052] The feedforward neural network model consists of a fair fusion layer and a multi-layer fully connected network. The fusion layer introduces a label-aware perturbation generator to construct a representation trajectory space with label preservation but structural perturbations. Trajectory contrast learning is used to enhance the intra-class structure modeling capability. Combined with a label-invariant adversarial training mechanism, the model effectively weakens the impact of sensitive attributes on model predictions, improving both fairness and robustness. The final output is the text classification result.
[0053] The present invention realizes multi-objective joint optimization of text classification performance, privacy protection and fairness modeling, and is suitable for text intelligent classification tasks in multiple scenarios. BRIEF DESCRIPTION OF THE DRAWINGS
[0054] Figure 1 It is a schematic diagram of the principle of the text classification prediction method based on dynamic supervision-driven multi-objective joint optimization described in the present invention. DETAILED DESCRIPTION
[0055] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.
[0056] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.
[0057] See also Figure 1 This embodiment describes a multi-objective joint priority text classification prediction method based on dynamic supervision driving, which includes the following steps:
[0058] Step 1: Obtain the target text dataset and use the BERT model to convert each target text data in the target text dataset into a text feature vector; specifically, the BERT model is a transformer represented by a bidirectional encoder;
[0059] Step 2: Extract global structural features from each text feature vector through singular value decomposition;
[0060] Step 3: Extract local structural features from each text feature vector through a structure-generated adversarial network;
[0061] Step 4: Calculate the reconstruction error between each text feature vector and its corresponding global structural feature and local structural feature;
[0062] Step 5: Perform weighted fusion of the corresponding global and local structural features according to the reconstruction errors corresponding to each text feature vector to obtain the fused feature vector corresponding to each text feature vector;
[0063] Step 6: Use the attention mechanism to dynamically add Laplace noise to each fused feature vector to obtain a feature vector containing noise;
[0064] Step 7: All feature vectors containing noise corresponding to all text feature vectors are formatted and encapsulated and then input into the feedforward neural network model to predict the category of the target text that takes into account both fairness and privacy protection.
[0065] This preferred embodiment can effectively improve the structural integrity and discrimination ability of text feature expression by introducing a weighted fusion mechanism of global and local structural features driven by reconstruction error; further combining the attention mechanism and the differential privacy mechanism, the attention score is used to dynamically adjust the intensity of Laplace noise addition, thereby achieving differentiated treatment of different text samples in terms of privacy protection needs, and enhancing the flexibility and practicality of privacy protection; while ensuring the accuracy of text classification, it effectively suppresses the information loss problem caused by uniformly adding noise, and improves the comprehensive performance of the model under multi-target tasks; in addition, by extracting local structural features through a structure-generating adversarial network and introducing a structure-preserving loss function for training, the model's ability to model local semantic structures is enhanced, making the fused feature representation richer and more robust, further improving the accuracy and fairness of text classification predictions. It is further given that in step one, the implementation method of using the BERT model to convert each target text data in the target text dataset into a text feature vector includes:
[0066] Step 1: Perform word segmentation on the target text data to obtain multiple phrases related to the text content, map the phrases into a high-dimensional vector space, and generate corresponding word vectors;
[0067] Step 12: Input the word vector into the pre-trained deep bidirectional transformer model to perform context encoding on the word vector;
[0068] Step 13: Aggregate the word vectors after context encoding to obtain the text feature vector.
[0069] It is further given that in step three, the structured generative adversarial network includes a generator and a discriminator;
[0070] The training phase of the structured generative adversarial network:
[0071] The generator learns to generate features with local structural information based on the received text feature vector;
[0072] The discriminator is used to determine whether the input features with local structural information have real local structural features, and guide the adversarial training through the judgment results to optimize the generation ability of the generator;
[0073] A local structure preservation loss function is introduced during adversarial training to constrain the consistency of the features generated by the generator in the local structure, thus completing the training of the generator.
[0074] Reasoning phase of the structured generative adversarial network:
[0075] The generator trained in the structured generative adversarial network extracts local structural features and outputs them based on the received text feature vector.
[0076] In this preferred embodiment, local structural features are extracted through a structure-generating adversarial network, and a structure-preserving loss function is introduced for training, which enhances the model's ability to model local semantic structures, making the fused feature representation richer and more robust, and further improving the accuracy and fairness of text classification predictions.
[0077] Furthermore, in step 4, the method for calculating the reconstruction error between each text feature vector and its corresponding global structural feature and local structural feature is as follows:
[0078] ;
[0079] ;
[0080] in, For the text feature vectors, for The global structural characteristics of for The local structural characteristics of is the L2 norm.
[0081] Furthermore, in step 5, the method for obtaining the fused feature vector corresponding to each text feature vector is as follows:
[0082] Definition The weight of the fused feature vector corresponding to the text feature vector The expression is:
[0083]
[0084] according to , get the The fused feature vector corresponding to the text feature vectors ;
[0085] ;
[0086] in, , For the text feature vectors, for The global structural characteristics of for local structural features.
[0087] Furthermore, in the embodiment, in step 6, the method for obtaining the feature vector containing noise includes:
[0088] Calculate the attention score of each fused feature vector, use the Laplace algorithm to generate a random noise intensity value, multiply the generated random noise intensity value by the attention score corresponding to the fused feature vector to obtain the noise corresponding to the fused feature vector; add the noise to the corresponding fused feature vector to obtain a feature vector containing noise.
[0089] In this preferred embodiment, in the process of obtaining the specific implementation of the feature vector containing noise, a dynamic Laplace noise injection strategy based on the attention mechanism is introduced to achieve differentiated control of text features in privacy protection processing. Specifically, the system first calculates the attention score of the fused feature vector to measure its importance in the text classification task, and adjusts the injection intensity of the Laplace noise accordingly, so that smaller noise is added to the key features to retain the discrimination ability, while larger noise is added to the non-key features to enhance the privacy protection effect, thereby effectively balancing the performance of the model and the privacy protection requirements. This method not only avoids the general weakening of the model performance by the traditional fixed noise mechanism, but also improves the robustness and generalization ability of the model under diversified inputs. At the same time, it has good scalability and adaptability, and is suitable for a variety of text classification and privacy protection scenarios. It has significant technical advantages and practical application value.
[0090] Furthermore, the feedforward neural network model includes an encoder and a classifier;
[0091] The encoder consists of a fair fusion layer and two fully connected layers, and the classifier includes two fully connected layers;
[0092] After format conversion and encapsulation, each noisy feature vector is sequentially passed through the multi-view feature modeling module, label-guided attention weighted fusion module, label-aware perturbation generation module, contrastive representation learning module, and adversarial fairness training module in the fairness fusion layer to obtain a robust feature vector that is both discriminative and fair.
[0093] The robust feature vector with both discriminative and fair properties is then transmitted to the two fully connected layers of the encoder and the two fully connected layers of the classifier in sequence. After forward propagation of the received feature vector, each fully connected layer uses the ReLU activation function for gradient elimination and the Dropout function for regularization.
[0094] This embodiment provides the specific composition of the feedforward neural network model. This combination, by organically integrating the fairness fusion layer with multiple functional modules into the encoder structure and combining it with the discriminative ability of the classifier, achieves the coordinated processing of the entire process of multi-perspective modeling, feature enhancement, perturbation injection, robustness improvement, and fairness optimization for the noisy text feature vector, significantly improving the overall performance of the model in complex text classification tasks. Specifically, the multi-perspective feature modeling module in the fairness fusion layer effectively captures the diversity of information in the features through multi-angle feature projection and fusion; the label-guided attention weighted fusion module introduces the label information to guide the attention mechanism, enhancing the discriminativeness of the features; the label-aware perturbation generation module further injects perturbation signals based on the label information, improving the model's sensitivity and adaptability to potential biases; the contrastive representation learning module optimizes the feature space structure through a contrastive learning strategy, enhancing the model's robustness to perturbations; the adversarial fairness training module explicitly suppresses the leakage of sensitive attribute information through adversarial training, ensuring that the model is fair while maintaining high discriminative performance. Subsequently, the fully connected layers in the encoder and classifier, combined with the ReLU activation function and Dropout regularization mechanism, further enhance the model's nonlinear expressiveness and generalization performance. Overall, this feedforward neural network model structure not only achieves a deep understanding and discrimination of text feature vectors, but also systematically introduces fairness and robustness optimization mechanisms into the feature processing process, resulting in greater adaptability, fairness, and practicality, making it suitable for text classification applications with high performance and fairness requirements.
[0095] Furthermore, the specific functions of each module in the fair fusion layer are given, specifically: the multi-view feature modeling module is used to perform multi-angle feature projection and fusion on the received feature vectors containing noise after format conversion and encapsulation;
[0096] The label-guided attention weighted fusion module is used to introduce label information to guide the attention mechanism to enhance the feature discriminability of feature projection and fused feature vectors;
[0097] The label-aware perturbation generation module is used to generate a perturbation signal based on the label information and inject the perturbation signal into the feature vector after enhancing the feature discriminability;
[0098] The contrastive representation learning module is used to combine the contrastive learning strategy to improve the robustness of the feature space structure of the feature vector after the perturbation signal is injected;
[0099] The adversarial fairness training module is used to explicitly suppress the leakage of sensitive attribute information in the feature vector after improving the robustness of the feature space structure through the adversarial training mechanism, thereby obtaining a robust feature vector that is both discriminative and fair.
[0100] In this preferred embodiment, the fair fusion layer is used to perform multi-angle feature projection and fusion on the received feature vector after format conversion and encapsulation, and then introduce label information to guide the attention mechanism to enhance the feature discriminability of the feature vector after feature projection and fusion. The comparative learning strategy is then combined to improve the robustness of the feature space structure of the feature vector after the enhanced feature discriminability, and the adversarial training mechanism is used to explicitly suppress the leakage of sensitive attribute information in the feature vector after the enhanced feature discriminability, thereby obtaining a robust feature vector with both discriminability and fairness.
[0101] Principle analysis: The present invention first obtains text data, encodes the text using the BERT model, and generates the corresponding feature vector; uses the singular value decomposition technology to extract the global structural features of the feature vector, and extracts the local structural features through the structure generation adversarial network; calculates the reconstruction error between the original features and the global and local structural features respectively, and performs weighted fusion of the global and local features accordingly to obtain the fused feature representation; uses the attention mechanism to score the fused features, and dynamically adjusts the Laplace noise intensity according to the attention score to achieve differential privacy protection; the feature vector containing noise is format-converted and input into the feedforward neural network model. The model consists of a fair fusion layer and a multi-layer fully connected network. The fusion layer introduces a label-aware perturbation generator to construct a representation trajectory space with label preservation but structural perturbation, and enhances the intra-class structure modeling capability through trajectory comparison learning. At the same time, combined with the label-invariant adversarial training mechanism, it effectively weakens the influence of sensitive attributes on model prediction and improves the fairness and robustness of the model; and finally outputs the text classification result.
[0102] The text classification prediction method based on dynamic supervision-driven multi-objective joint optimization described in this invention aims to improve text classification performance while achieving coordinated optimization of model fairness and privacy protection capabilities. The specific implementation is as follows:
[0103] First, obtain the original text data. In order to convert the text data into a processable numerical form, , The number index is converted into a feature vector using the pre-trained BERT model ;
[0104] Next, load these encoded feature vectors Global structural feature extraction and local structural feature extraction are performed on the feature vector respectively to obtain a more discriminative and structurally diverse representation.
[0105] Specifically, on the one hand, the singular value decomposition (SVD) method is used to decompose the eigenvectors Decompose it, extract its principal component information, and construct a global structural feature representation. This process can preserve the global distribution pattern of text data in high-dimensional space, thereby capturing the macro-semantic relationship between texts.
[0106] On the other hand, in order to enhance the model's ability to model local structures, the present invention designs a structure-generating adversarial network (Structure-GAN), which includes two modules: the generator and the discriminator. The discriminator is used as input to learn to generate representations with local structural features; the discriminator is used to determine whether the input features retain the true local structural information. By introducing the structure preservation loss function, the features output by the generator are constrained to be consistent with the original features in terms of adjacency, semantic similarity, etc., thereby improving the accuracy of local structure modeling. Finally, the global structural features are obtained respectively. and local structural features , as the input for subsequent reconstruction error calculation and fusion processing.
[0107] Next, the Euclidean distance between the original feature vector and the global and local reconstruction features is calculated as a measure of the reconstruction error. Global reconstruction error , local reconstruction error Then, the fusion weights of global and local features are dynamically adjusted according to the relative size of the reconstruction error. , The definition of The fused features are then expressed as In order to enhance the privacy protection capability of fused features, a differential privacy mechanism is introduced on the fused feature vector, and the attention mechanism is used to dynamically adjust the intensity of the added Laplace noise, thereby achieving effective protection of individual data. Specifically, the fused feature vector is , firstly, the importance of each data point is modeled through the attention mechanism. The feature vector of the input data point is linearly transformed with the learnable attention weight, and the hyperbolic tangent function is used. Perform nonlinear mapping to obtain the initial attention score, the formula is ,in and are learnable weight parameters and bias terms. Next, the attention scores are normalized by the softmax function to ensure that the sum of all attention scores is 1, that is, After obtaining the normalized attention weights, we dynamically adjust the intensity of the added Laplace noise according to the attention score of each sample. Let Laplace noise , where the noise scale parameter λ depends on the privacy budget and the sensitivity of the data , the specific formula is Finally, the fused feature vector will be perturbed by the noise modulated by the attention weight to obtain the privacy-preserving feature representation. Through the above method, the present invention realizes a differential privacy mechanism based on adaptive adjustment of data importance, which not only effectively protects the privacy security of user data, but also improves the flexibility of noise addition and the practicality of differential privacy while ensuring model performance.
[0108] After perturbation of the fused features, the obtained feature vector containing noise is fed into a feedforward neural network model to implement the text classification task. The feedforward neural network consists of an encoder module and a classifier module.
[0109] The encoder extracts feature representations of the input text and outputs representations of the intermediate hidden layers. To improve the discriminability and fairness of the feature representations, a fairness fusion layer is introduced into the encoder. This fusion layer consists of five submodules: a multi-view feature modeling module, a label-guided attention module, a label-aware perturbation generation module, a contrastive representation learning module, and an adversarial fairness training module.
[0110] The multi-view feature modeling module introduces multiple low-rank matrix decomposition branches to decompose the input feature vector Perform multi-angle nonlinear projection. Specifically, set the number of viewing angles to , for each perspective , construct a set of learnable parameter matrices 、 ,in is a low-rank dimension, is the output dimension. Through matrix multiplication The projection weight matrix at each perspective is obtained, the input features are linearly transformed and averaged to obtain the fused output prediction result.
[0111] The label-guided attention module introduces label information as an attention-guiding signal in each view. After concatenating the input features with the labels, it is fed into a lightweight attention network to generate the attention weights for each view. , used to adjust The mechanism dynamically adjusts the feature transformation direction in a label-aware manner, making samples of different categories more discriminative in the feature space.
[0112] Third, the label-aware perturbation generation module takes the concatenated multi-view representation and the label as input and generates a small perturbation vector , and add it to the original representation to obtain the perturbed feature representation. This perturbation is used to enhance the model's robustness to input changes and serves as positive and negative samples for contrastive learning.
[0113] Fourth, the contrastive representation learning module uses a projection head network to map the original and perturbed representations into a contrastive space. It then calculates a contrastive loss (e.g., NT-Xent) to bring different views of the same sample closer together and push different samples further apart, thereby optimizing the structure of the feature space. This module improves the model's generalization capabilities in scenarios with small sample sizes and label imbalance.
[0114] Fifth, the adversarial fairness training module introduces a sensitive attribute predictor and a gradient masking mechanism to attempt to predict sensitive attributes (such as gender and age) from the perturbed feature representations. The loss of this predictor is masked when backpropagated to the backbone network, forcing the backbone network to learn neutral representations unrelated to sensitive attributes, improving the model's fairness across different groups.
[0115] The main task classifier classifies the feature representation output by the encoder to obtain the classification result of the text. Specifically, the feature information passes through several fully connected layers, and ReLU activation function and Dropout are applied between each layer for processing. Assume that the input feature is , the output after the lth fully connected layer and activation function is ,in and is the parameter of the first layer. Finally, the network outputs the classification result, and the output of the main task classifier is the classification result of the text .
[0116] Specific embodiment 2: A text classification prediction device based on multi-objective joint optimization includes a storage device, a processor, and a computer program stored in the storage device and runnable on the processor, characterized in that the processor executes the computer program to implement the text classification prediction method based on multi-objective joint optimization driven by dynamic supervision.
[0117] Although the present invention is described herein with reference to specific embodiments, it should be understood that these embodiments are merely illustrative of the principles and applications of the invention. It should be understood that many modifications may be made to the illustrative embodiments, and that other arrangements may be devised, without departing from the spirit and scope of the invention as defined by the appended claims. It should be understood that the various dependent claims and features described herein may be combined in ways other than those described in the original claims. It should also be understood that features described in conjunction with individual embodiments may be employed in conjunction with other described embodiments.
Claims
1. A text classification prediction method based on multi-objective joint optimization driven by dynamic supervision, characterized by: The method comprises the following steps: Step 1: Obtain the target text dataset and use the BERT model to convert each target text data in the target text dataset into a text feature vector; Step 2: Extract global structural features from each text feature vector through singular value decomposition; Step 3: Extract local structural features from each text feature vector through a structure-generated adversarial network; Step 4: Calculate the reconstruction error between each text feature vector and its corresponding global structural feature and local structural feature; Step 5: Perform weighted fusion of the corresponding global and local structural features according to the reconstruction errors corresponding to each text feature vector to obtain the fused feature vector corresponding to each text feature vector; Step 6: Use the attention mechanism to dynamically add Laplace noise to each fused feature vector to obtain a feature vector containing noise; Step 7: All feature vectors containing noise corresponding to all text feature vectors are formatted and encapsulated and then input into the feedforward neural network model to predict the category of the target text that takes into account both fairness and privacy protection.
2. The text classification prediction method based on dynamic supervision-driven multi-objective joint optimization according to claim 1 is characterized in that: In step 4, the reconstruction error between each text feature vector and its corresponding global structural feature and local structural feature is calculated as follows: ； ； in, For the text feature vectors, for The global structural characteristics of for The local structural characteristics of for The reconstruction error between the corresponding global structural features, For the The reconstruction error between a text feature vector and its corresponding local structural feature, is the L2 norm.
3. The text classification prediction method based on dynamic supervision-driven multi-objective joint optimization according to claim 1 is characterized in that: In step 5, the method for obtaining the fused feature vector corresponding to each text feature vector is as follows: Definition The weight of the fused feature vector corresponding to the text feature vector The expression is: according to , get the The fused feature vector corresponding to the text feature vectors ; in, .
4. The text classification prediction method based on dynamic supervision-driven multi-objective joint optimization according to claim 3 is characterized in that: ； in, for The global structural characteristics of for local structural features.
5. The text classification prediction method based on dynamic supervision-driven multi-objective joint optimization according to claim 1 is characterized in that: In step 6, the implementation method of obtaining the feature vector containing noise includes: Calculate the attention score of each fused feature vector, use the Laplace algorithm to generate a random noise intensity value, multiply the generated random noise intensity value by the attention score corresponding to the fused feature vector to obtain the noise corresponding to the fused feature vector; add the noise to the corresponding fused feature vector to obtain a feature vector containing noise.
6. The text classification prediction method based on dynamic supervision-driven multi-objective joint optimization according to claim 1 is characterized in that: The feedforward neural network model consists of an encoder and a classifier; The encoder consists of a fair fusion layer and two fully connected layers, and the classifier includes two fully connected layers; After format conversion and encapsulation, each noisy feature vector is sequentially passed through the multi-view feature modeling module, label-guided attention weighted fusion module, label-aware perturbation generation module, contrastive representation learning module, and adversarial fairness training module in the fairness fusion layer to obtain a robust feature vector that is both discriminative and fair. The robust feature vector with both discriminative and fair properties is then transmitted to the two fully connected layers of the encoder and the two fully connected layers of the classifier in sequence. After forward propagation of the received feature vector, each fully connected layer uses the ReLU activation function for gradient elimination and the Dropout function for regularization.
7. The text classification prediction method based on dynamic supervision-driven multi-objective joint optimization according to claim 6 is characterized in that: The multi-view feature modeling module is used to perform multi-angle feature projection and fusion on the received feature vectors containing noise after format conversion and encapsulation; The label-guided attention weighted fusion module is used to introduce label information to guide the attention mechanism to enhance the feature discriminability of feature projection and fused feature vectors; The label-aware perturbation generation module is used to generate a perturbation signal based on the label information and inject the perturbation signal into the feature vector after enhancing the feature discriminability; The contrastive representation learning module is used to combine the contrastive learning strategy to improve the robustness of the feature space structure of the feature vector after the perturbation signal is injected; The adversarial fairness training module is used to explicitly suppress the leakage of sensitive attribute information in the feature vector after improving the robustness of the feature space structure through the adversarial training mechanism, thereby obtaining a robust feature vector that is both discriminative and fair.
8. The text classification prediction method based on dynamic supervision-driven multi-objective joint optimization according to claim 1 is characterized in that: In step 1, the implementation method of using the BERT model to convert each target text data in the target text dataset into a text feature vector includes: Step 1: Perform word segmentation on the target text data to obtain multiple phrases related to the text content, map the phrases into a high-dimensional vector space, and generate corresponding word vectors; Step 12: Input the word vector into the pre-trained deep bidirectional transformer model to perform context encoding on the word vector; Step 13: Aggregate the word vectors after context encoding to obtain the text feature vector.
9. The text classification prediction method based on dynamic supervision-driven multi-objective joint optimization according to claim 1 is characterized in that: The structured generative adversarial network includes a generator and a discriminator; The training phase of the structured generative adversarial network: The generator learns to generate features with local structural information based on the received text feature vector; The discriminator is used to determine whether the input features with local structural information have real local structural features, and guide the adversarial training through the judgment results to optimize the generation ability of the generator; A local structure preservation loss function is introduced during adversarial training to constrain the consistency of the features generated by the generator in the local structure, thus completing the training of the generator. Reasoning phase of the structured generative adversarial network: The generator trained in the structured generative adversarial network extracts local structural features and outputs them based on the received text feature vector.
10. A text classification prediction device based on multi-objective joint optimization, comprising a storage device, a processor, and a computer program stored in the storage device and executable on the processor, characterized in that: The processor executes the computer program to implement the text classification prediction method based on dynamic supervision-driven multi-objective joint optimization as described in any one of claims 1 to 9.