Depolarization method for multi-language text classification
By using deep prompt tuning and feature-weighted adversarial training methods in multilingual text classification, the problem that biased features are not effectively processed is solved, and more efficient debias training and resource utilization is achieved, while improving the fairness and classification performance of the model.
Patent Information
- Application Number
- CN202510229957.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-02-27
- Publication Date
- 2025-06-13
AI Technical Summary
There is a problem that biased features are not effectively processed in multilingual text classification, resulting in some sensitive attribute information still retained in the model output. At the same time, the method of fine-tuning of the pre-training language model with full parameters is inefficient and resource consumption is too large.
Deep prompt tuning method is adopted, a small number of learnable prompt vectors are added as prefix tokens to each layer of the pretrained language model, and the biased features are identified and eliminated by backpropagation-guided feature-weighted adversarial training.
A deeper debias effect is achieved, the fairness and classification performance of the model are improved, and the computing and memory resources are saved, and the efficiency of debias training is improved.
Smart Images
Figure CN120144760A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the technical field of data classification, and more specifically, to a method for debiasing multi-language text classification. Background Art
[0002] Fairness in text classification is an important research direction in the field of natural language processing (NLP). Its core goal is to address the bias problems existing in text data and models. Bias may stem from social stereotypes in the training data. If these biases are not addressed, they may be amplified in the model's predictions and decisions, leading to unfair or inaccurate results. With the acceleration of globalization, the importance of multi-language text classification has become increasingly prominent. The multi-language text classification task involves classifying text data in different languages according to their respective categories. Compared with monolingual models, multi-language models are more versatile in practical applications because they can use a single model in multi-language scenarios without having to train a separate model for each language. Therefore, exploring debiasing in multi-language text classification tasks has important theoretical and practical significance. However, the complex language environment of multi-language text classification also makes the debiasing work more challenging than monolingual text classification.
[0003] Currently, the research on debiasing multi-language text classification mainly focuses on the following two methods: One is the debiasing model for multi-language text classification based on domain adaptation. This method treats sensitive attributes as different domains (such as male and female), and by reducing the differences between different domains during training, it reduces the output bias of text classification caused by sensitive attributes, thereby achieving the effect of reducing bias. The other is the debiasing framework for multi-language text classification based on contrastive learning. This method uses contrastive learning to shorten the semantic distance between samples with different sensitive attribute values under the same target label, thereby achieving the confusion of sensitive attributes in the semantic space and preventing the model from identifying this sensitive attribute information.
[0004] Although the above methods have made some progress in debiasing multi-language text classification, they still have significant limitations. For example, the debiasing method for multi-language text classification based on domain adaptation fails to fully consider the importance of bias features. In text data, identity terms and language styles related to sensitive attributes are encoded as feature vectors in the text representation during training. These feature vectors make important contributions to the predictions of text classifiers but are also the root causes of bias. However, the existing debiasing methods do not effectively process these features and instead introduce a large number of noise features. This results in the fact that in debiasing adversarial training, the input of the feature vectors used cannot help the model deeply identify and eliminate bias, and some sensitive attribute information still remains in the model output.
[0005] In addition, both the domain adaptation-based and contrastive learning-based debiasing methods for multilingual text classification adopt the method of fine-tuning the pre-trained language model with all parameters. When dealing with large-scale multilingual datasets, this method consumes a large amount of computing and memory resources. As the number of parameters of the pre-trained language model continues to increase, the method of fine-tuning all parameters is no longer applicable because it is not only inefficient but also may lead to waste of resources.
[0006] Therefore, how to effectively identify and eliminate bias features in multilingual text classification, while improving the efficiency of debiasing training and reducing resource consumption, has become a technical problem to be solved. Summary of the Invention
[0007] In view of this, the present application provides a debiasing method for multilingual text classification to solve the problems of ignoring the importance of bias features, low efficiency, and excessive resource consumption during the debiasing process.
[0008] The technical solution provided by the present application is as follows:
[0009] A debiasing method for multilingual text classification includes:
[0010] Construct a debiasing framework for multilingual text classification, including a multilingual text representation module, a text classification module, and a text debiasing module;
[0011] Randomly initialize learnable prompt vectors based on deep prompt tuning, and add the learnable prompt vectors as prefix tokens to each layer of the multilingual text representation module to obtain the text representation of multilingual text;
[0012] Perform a linear transformation on the text representation to obtain a prediction output, and use the cross-entropy loss function to calculate the difference between the prediction output and the true label to determine the classification loss;
[0013] Through feature-weighted adversarial training guided by backpropagation, weight the text features in the text representation, and determine the sensitive attribute discrimination loss of the text;
[0014] Taking the weighted combination of minimizing the classification loss and maximizing the sensitive discrimination loss as the optimization objective of adversarial training, train the debiasing framework for multilingual text classification to generate a debiasing model for text classification, which is used to perform unbiased classification on the text to be classified.
[0015] In one possible implementation, the multilingual text representation module includes N (N>1) hierarchical prompt layers for capturing text features at different levels and obtaining the text representation.
[0016] In one possible implementation, adding the learnable prompt vector as a prefix token to each layer of the multilingual text representation module for obtaining the text representation of the multilingual text includes:
[0017] Following the P-tuning v2 method, inserting the learnable prompt vector as a prefix token into the input embedding sequence of different layers; where the learnable prompt vector is represented as:
[0018] P = {(p 1 , p 2 ,..., p m )
[0019] In the formula, P m represents the prompt token;
[0020] The input embedding sequence is represented as:
[0021] [h 1 , h 2 ,..., h m , e(x 1 ), e(x 2 ),..., e(x n )]
[0022] In the formula, the embedding vector (h 1 , h 2 ,..., h m ) = Embedding({(p 1 p 2 ,..., p m )}), Embedding represents the embedding layer that maps the prompt token to a continuous embedding; the input sequence X i = {(x 1 , x 2 ,..., x n ); e(x j )(1≤j≤n) is the embedding of x j obtained through the pre-trained embedding layer;
[0023] Obtaining the text based on the multilingual text representation model with learnable prompts, represented as:
[0024] v i = PLM([h 1 , h 2 ,..., h m , e(x 1 ), e(x 2 ),..., e(x n )])
[0025] In one possible implementation, the text classification module includes a classifier G c , which consists of a linear layer;
[0026] Performing a linear transformation on the text representation to obtain a predicted output, including:
[0027] Based on the linear layer, performing a linear transformation on the text representation to obtain a predicted unnormalized score, which is used to characterize the confidence of the text classification module in each category in the classification task, expressed as:
[0028] logits(v i ) = G c (v i )
[0029] Using the unnormalized score as the predicted output of the text classification module.
[0030] In one possible implementation, the cross-entropy loss function is used to calculate the difference between the predicted output and the true label to determine the classification loss, expressed as:
[0031] L c = CrossEntropy(logits(v i ), y i )
[0032] In the formula, y i is the target label of the i-th sample.
[0033] In one possible implementation, through feature-weighted adversarial training guided by backpropagation, the text features in the text representation are weighted, and the sensitive attribute discrimination loss of the text is determined, including:
[0034] Backpropagating the classification loss L c of the text classification module to the multilingual text representation module to determine the gradient of the multilingual text representation module, expressed as:
[0035]
[0036] In the formula, v i is the text representation;
[0037] Normalizing the gradient and setting importance weights for the text features in the text representation v i , expressed as:
[0038]
[0039] In the formula, λ is a scaling factor used to maintain the norm of the text representation space; is the gradient of the multi - language text representation module; v i is the weighted text representation;
[0040] Based on the weighted text representation, use the discriminator G d to determine the sensitive attribute discrimination loss of the current sample, expressed as:
[0041] L d = CrossEntropy(G d (v′ i ), g i )
[0042] In the formula, g i is the sensitive attribute label of the i - th sample, and G d (v i ′) is the unnormalized score generated by the discriminator G d according to the weighted text representation v′ i generated.
[0043] In one possible implementation, the optimization objective of the adversarial training is expressed as:
[0044]
[0045] In the formula, γ is the loss weight used to achieve the balance between the cross - entropy loss L c and the sensitive attribute discrimination loss L d ; represents the parameter of the learnable prompt; θ c represents the parameter of the classifier G c ; θ d represents the parameter of the discriminator G d .
[0046] In one possible implementation, use the generated text classification de - biasing model to perform unbiased classification on the text to be classified, including:
[0047] Obtain the text data to be classified, expressed as:
[0048] D ={(X 1 , y 1 , s 1 ), (X 2 , y 2 , s 2 ),..., (X K , y k , s k )}
[0049] In the formula, X i ={x 1 , x 2,..., x n}(1 ≤ i ≤ K) represents the text sequence of the i-th sample, and n represents the length of the i-th sample sequence; y i is the target label of the i-th sample, where y i ∈ Y, which is the set of categories of the classification task; s i is the sensitive attribute label, where s i ∈ S, and S represents the set Y of sensitive attribute values;
[0050] The text representation is obtained by using a multilingual pre-trained model with learnable prompts through the multilingual text representation module;
[0051] Based on the text classification module, the text representation is classified into predefined categories;
[0052] Based on the text debiasing module, the text features in the text representation are weighted using the gradient matrix obtained by backpropagating the classification loss L c of the text classification module;
[0053] Through adversarial training, the text classification debiasing model is guided to generate unbiased deep prompt embeddings for unbiased classification.
[0054] Compared with the prior art, the technical solution of this application has the following beneficial effects:
[0055] Through feature-weighted adversarial training guided by backpropagation, this application focuses on identifying and eliminating features that are prone to causing bias in text classification tasks. By weighting important features, the model pays more attention to these features that are crucial for the classification task but may also introduce bias, thereby achieving a deeper debiasing effect.
[0056] The deep prompt tuning method is adopted by adding a small number of learnable prompt vectors as prefix tokens in each layer of the pre-trained language model. It not only retains the original knowledge and language modeling ability of the pre-trained model but also significantly saves computational and memory resources, and is an efficient alternative to traditional full-parameter fine-tuning methods. By optimizing the model structure and training strategy, the classification performance of the model is effectively maintained during the debiasing training process, achieving a dual optimization of fairness and classification ability, and thus reaching the overall optimum of fairness and classification performance. Brief Description of the Drawings
[0057] Figure 1 It is a flowchart of a debiasing method for multilingual text classification based on feature-weighted adversarial training and deep prompt tuning provided in Embodiment 1 of this application.
[0058] Figure 2 It is a schematic structural diagram of a multilingual text classification debiasing framework provided in Embodiment 1 of this application.
[0059] Figure 3 This is a flowchart of a feature-weighted adversarial training guided by backpropagation provided in the first embodiment of this application.
[0060] Figure 4 This is a flowchart of a bias removal method for multi-language text classification provided in the second embodiment of this application. Detailed implementation manners
[0061] Next, in combination with the embodiments of this application, the technical solutions in the embodiments of this application will be clearly and completely described. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of this application.
[0062] Embodiment 1
[0063] The first embodiment of this application provides a bias removal method for multi-language text classification, which is implemented based on an improved pre-trained language model. Specifically, in this Embodiment 1, aiming at the problem that the existing bias removal methods easily ignore the importance of bias features, feature-weighted adversarial training guided by backpropagation is introduced to make the bias removal adversarial training process pay more attention to the features that are important for the classification task and are likely to cause bias, thereby deeply identifying and eliminating bias. Aiming at the problems of low efficiency and excessive resource consumption, a deep prompt tuning method is proposed to add a small number of randomly initialized learnable prompt vectors as prefix tokens in each layer of the pre-trained language model to save computing and memory resources, while retaining the original pre-trained knowledge and language modeling capabilities. In terms of fairness, combining deep prompt tuning with existing bias removal methods will bring better bias removal effects.
[0064] Next, the bias removal method for multi-language text classification based on feature-weighted adversarial training and deep prompt tuning provided in this Embodiment 1 will be introduced in detail in combination with the specific implementation manners.
[0065] See Figure 1 , which is a flowchart of the bias removal method for multi-language text classification provided in the first embodiment of this application. As Figure 1 shown in, the specific implementation steps of the above method include:
[0066] Step 101, construct a bias removal framework for multi-language text classification.
[0067] Specifically, as Figure 2 shown in, the bias removal framework for multi-language text classification mainly consists of three key modules, including a multi-language text representation module, a text classification module, and a text bias removal module.
[0068] The multilingual text representation module includes N hierarchical prompt layers (Layerl Prompts) for capturing text features at different levels, thereby obtaining a richer text representation v i . Among them, N is a positive integer and N > 1.
[0069] The text classification module (Text Classification module) includes a classifier G c . The classifier (Classifier) consists of a linear layer, which processes the text representation v i through a linear transformation to obtain the predicted unnormalized scores, expressed as:
[0070] logits(v i ) = G c (v i )
[0071] The unnormalized scores represent the confidence of the text classification module in each category for the classification task. To calculate the difference between the predicted output and the true label, this embodiment uses the cross-entropy loss function Lc as the training objective, expressed as:
[0072] L c = CrossEntropy(logits(v i ), y i )
[0073] where y i is the target label of the i-th sample.
[0074] The text debiasing module, including the domain classifier network introduced by DANN. Specifically, the text representation v i output by the multilingual text representation module will first pass through a gradient reversal layer (GRL) and then be input into the discriminator (Discriminator) Gd. During forward propagation, GRL does not have any impact on the propagation of data, but during backpropagation, GRL reverses the gradient by multiplying the gradient by a negative value, so that the multilingual representation module is trained to obtain text representations that are indistinguishable to the sensitive attribute discriminator, thereby generating unbiased text representations. However, not all features in the text representation are equally important for the classification task, and the features that are important for the classification task are the main reasons for generating biases. Therefore, more attention should be paid to the features that are important for the classification task during adversarial training. The text debiasing module weights the features based on their importance to the text classification module. During training, the text classification loss L c is backpropagated to the text representation layer, and the gradient of this layer is expressed as:
[0075]
[0076] The normalized gradient is used as the importance weight of the feature in the text representation, expressed as:
[0077]
[0078] Here, λ is a scaling factor used to maintain the norm of the text representation space. is the gradient of the text representation module. The weighted text representation can be better conveyed to the discriminator to determine the features that are important for the classification task. These important features are more likely to produce bias, so that adversarial training can better narrow the distribution of these biased features and achieve better debiasing effect than ordinary adversarial training. Discriminator G d The sensitive attribute identification loss of the current sample is calculated as:
[0079] L d =CrossEntropy(G d (v′ i ), g i )
[0080] Among them, g i is the sensitive attribute label of the i-th sample.
[0081] Step 102: randomly initialize a small number of learnable hint vectors based on deep hint tuning, and add the learnable hint vectors as prefix tokens to each layer of the multilingual text representation module to obtain text representation of the multilingual text.
[0082] Unlike ordinary soft hint tuning, the multilingual text representation module randomly initializes a small number of learnable hint vectors through deep hint tuning and adds them as prefix tokens to each layer of the pre-trained model, not just the input layer. This deep hint tuning framework saves computing and memory resources while retaining the original pre-trained knowledge and language modeling capabilities. In terms of fairness, deep hint tuning combined with existing debiasing methods will bring better debiasing effects. After being processed by the multilingual text representation module, it can have more learnable parameters than ordinary soft hints, improving the ability to handle complex tasks.
[0083] In a specific implementation, following the P-tuning v2 method, the learnable hint vector is inserted as a prefix tag into the input embedding sequence of different layers. Given an input sequence X i ={(x 1 , x 2 , ..., x n ), a series of prompt marks P = {(p 1 , p 2,..., p m ) is added before the input text. For each prompt token P m , it is mapped to the corresponding embedding vector through the embedding layer, denoted as:
[0084] (h 1 , h 2 ,..., h m ) = Embedding({(p 1 , p 2 ,..., p m )})
[0085] where Embedding is the embedding layer that maps discrete tokens to continuous embeddings. Therefore, the input embedding sequence can be represented as [h 1 , h 2 ,..., h m , e(x 1 ), e(x 2 ),..., e(x n ), where e(x j )(1 ≤ j ≤ n) is the embedding of x j obtained through the pre-trained embedding layer. To achieve more fine-grained control over the multilingual text representation module, in addition to the input layer, trainable prompts are added to each layer as prefix tokens. Finally, based on the multilingual text representation model with learnable prompts, the text representation is obtained, denoted as:
[0086] v i = PLM([h 1 , h 2 ,..., h m , e(x 1 ), e(x 2 ),..., e(x n )])
[0087] Step 103, perform a linear transformation on the above text representation to obtain the predicted unnormalized scores, and use the cross-entropy loss function to calculate the difference between the predicted output and the target label to determine the classification loss.
[0088] Specifically, in the text classification module, the text representation v i output by the multilingual text representation module is initially passed to the classifier G c to output the predicted unnormalized scores for determining the original predicted output of the text classification module for each class, which has not undergone any form of normalization, denoted as:
[0089] logits(v i ) = G c (vi )
[0090] The unnormalized fractional logits(v i ) are the prediction outputs of the classifier G c for each class, reflecting the confidence level of the classifier G c for each class before any probability transformation.
[0091] After determining the above prediction outputs, in order to calculate the difference between the prediction outputs and the target labels, the cross-entropy loss function is used as the training objective, and the classification loss is expressed as:
[0092] L c = trossEncropy(logits(v i ), y i )
[0093] where y i is the target label of the i-th sample.
[0094] During the training process, by minimizing the cross-entropy loss function Lc, the text classification module can learn more accurate classification boundaries, thereby improving the accuracy of text classification.
[0095] Step 104. Through feature-weighted adversarial training guided by backpropagation, the text features in the text representation v i are weighted to determine the discriminative loss of the sensitive attributes of the current text.
[0096] Specifically, considering that not all text features in the text representation v i output by the multilingual text representation module are equally important for the text classification task, and the text features important for the classification task are the main reasons for generating biases. Therefore, in adversarial training, more attention needs to be paid to the text features important for the classification task. In this embodiment, the text debiasing module weights the text features based on their importance to the text classification module, as shown in Figure 3 , and specifically includes the following steps:
[0097] Step 1041. Backpropagate the classification loss L c of the text classification module to the multilingual text representation module to determine the gradient of the multilingual text representation module, expressed as:
[0098]
[0099] Step 1042. Normalize the above gradient and use the normalization result as the importance weight of the text features in the text representation, expressed as:
[0100]
[0101] Among them, λ is a scaling factor, whose function is to maintain the norm of the text representation space. is the gradient of the multi - language text representation module. The weighted text representation v′ i can better convey the text features important for the classification task to the discriminator G d , and these important features are more likely to be biased, so that adversarial training can better narrow the distribution of biased features, achieving a better de - biasing effect compared to ordinary adversarial training.
[0102] Step 1043: Based on the weighted text representation, use the discriminator G d to determine the sensitive - attribute discrimination loss of the current sample, expressed as:
[0103] L d = CrossEntropy(G d (v′ i ), g i )
[0104] Among them, g i is the sensitive - attribute label of the i - th sample, and G d (v i ′) is the unnormalized score generated by the discriminator G d according to the weighted text representation v′ i .
[0105] Furthermore, during backpropagation, the Gradient Reversal Layer (GRL) reverses the gradient by multiplying the gradient by a negative value, so that the multi - language text representation module is trained to obtain text representations that are indistinguishable to the sensitive - attribute discriminator G d , thereby guiding the model to generate unbiased deep - prompt embeddings and reducing the bias generated in multi - language text classification.
[0106] Step 105: Using the weighted combination of minimizing the classification loss and maximizing the sensitive - attribute discrimination loss as the optimization objective of adversarial training, train the above - mentioned multi - language text classification de - biasing framework to generate an unbiased and accurate multi - language text classification de - biasing model.
[0107] Among them, the optimization objective of adversarial training is expressed as:
[0108]
[0109] In the formula, γ is the loss weight used to achieve the balance between the cross - entropy loss L c and the sensitive - attribute discrimination loss L d . represents the parameter of the learnable prompt. θ cDenote the parameters of classifier G c as θ d Denote the parameters of discriminator G d as. Solving this min-max optimization problem helps to align the feature distributions between different sensitive attributes, thus reducing bias.
[0110] Compared with the prior art, the technical solution provided in the first embodiment of this application has the following beneficial effects:
[0111] The multi-language text classification debiasing method provided in the embodiments of this application effectively improves the accuracy and fairness of text classification by constructing a comprehensive debiasing framework. Through the collaborative work of the multi-language text representation module, the text classification module, and the text debiasing module, this framework not only learns the features that can accurately classify text, but also reduces the model's bias towards sensitive attributes through adversarial training during the training process. The key to this method lies in using the deep prompt tuning technology to add learnable prompt vectors as prefix tokens to each layer of the text representation, thereby obtaining a richer text representation, which helps the model better understand the text content and improve the classification performance.
[0112] In addition, through feature-weighted adversarial training, this method can identify and emphasize the features important for the classification task while suppressing the features that may cause bias. This training strategy not only optimizes the classification loss but also maximizes the sensitive attribute discrimination loss, enabling the generated text classification debiasing model to achieve unbiased classification when processing multi-language text, significantly improving the fairness and reliability of the multi-language text classification task.
[0113] Embodiment 2
[0114] Refer to Figure 4 , which is a flowchart of the text classification debiasing operation based on the trained text classification debiasing model provided in the second embodiment of this application. As Figure 4 shown, the specific implementation steps of the above method include:
[0115] Step 201, obtain the text data to be classified.
[0116] Specifically, given the data sample D = {(X 1 , y 1 , s 1 ), (X 2 , y 2 , s 2 ),..., (X K , y k , s k )}
[0117] where X i = {x 1 , x 2,..., x n (1 ≤ i ≤ K) represents the text sequence of the i-th sample, and n represents the length of the i-th sample sequence. y i is the target label of the i-th sample, where y i ∈ Y, which is the set of categories of the classification task. s i is the sensitive attribute label, where s i ∈ S, and S represents the set Y of sensitive attribute values.
[0118] Step 202: Use a multilingual pre-trained model with learnable prompts through a multilingual text representation module to obtain text representations.
[0119] Step 203: Based on the text classification module, complete the basic task of multilingual text classification and classify the above text representations into predefined categories.
[0120] Step 204: Based on the text debiasing module, use the gradient matrix obtained by backpropagating the classification loss L c of the text classification module to weight important text features in the text representation.
[0121] Step 205: Through adversarial training, guide the text classification debiasing model to generate unbiased deep prompt embeddings, thereby reducing the bias generated in multilingual text classification and performing unbiased classification.
[0122] It should be noted that the specific implementation methods of the above steps can be combined with the description of Embodiment 1, and Embodiment 2 of this application will not be elaborated too much.
[0123] Compared with the prior art, the technical solution provided by Embodiment 2 of this application has the following beneficial effects:
[0124] This application proposes an innovative multilingual text classification debiasing method, the core of which is feature-weighted adversarial training. This training mechanism focuses on identifying and eliminating features that may cause bias. Through this mechanism, the model can focus more on the features that are crucial for the classification task. These features are beneficial for improving classification accuracy but are also the main causes of bias. At the same time, this application adopts the deep prompt tuning technology, introducing a small number of learnable prompt vectors in each layer of the pre-trained language model, which not only retains the original language understanding ability of the model but also significantly reduces the computational and memory requirements, providing a more efficient alternative to the traditional full-parameter fine-tuning method.
[0125] In addition, this application constructs a modular debiasing framework that integrates three major modules: multilingual text representation, text classification, and text debiasing, and efficiently processes bias problems in multilingual text classification in a structured manner. This framework not only improves the flexibility and scalability of the model in processing multilingual text, but also maintains the classification performance while debiasing by optimizing the model structure and training strategy, achieving a dual optimization of fairness and classification performance. This comprehensive optimization strategy ensures that the model effectively reduces bias while maintaining a high classification accuracy, achieving an overall optimal balance between fairness and classification performance.
[0126] Although the embodiments of this application have been shown and described, it will be understood by those of ordinary skill in the art that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of this application, and the scope of this application is defined by the appended claims and their equivalents.
Claims
1. A debiasing method for multilingual text classification, characterized in that: include: Construct a multilingual text classification and debiasing framework, including a multilingual text representation module, a text classification module, and a text debiasing module; Randomly initializing a learnable hint vector based on deep hint tuning, and adding the learnable hint vector as a prefix token to each layer of the multilingual text representation module to obtain a text representation of the multilingual text; Performing a linear transformation on the text representation to obtain a predicted output, and using a cross entropy loss function to calculate the difference between the predicted output and the true label to determine the classification loss; weighting text features in the text representation and determining a sensitive attribute discrimination loss of the text through back-propagation guided feature weighted adversarial training; Taking the weighted combination of minimizing the classification loss and maximizing the sensitive discrimination loss as the optimization goal of adversarial training, the multilingual text classification debiasing framework is trained to generate a text classification debiasing model for unbiased classification of the text to be classified.
2. A debiasing method for multilingual text classification according to claim 1, characterized in that: The multilingual text representation module includes N (N>1) hierarchical prompt layers, which are used to capture text features at different levels and obtain the text representation.
3. A debiasing method for multilingual text classification according to claim 2, characterized in that: The step of adding the learnable hint vector as a prefix token to each layer of the multilingual text representation module to obtain a text representation of the multilingual text includes: Following the P-tuning v2 method, the learnable hint vector is inserted as a prefix token into the input embedding sequence of different layers; where the learnable hint vector is represented as: P={(p1,p2,...,p m ) Where P m Indicates a prompt mark; The input embedding sequence is represented as: [h1,h2,...,h m ,e(x1),e(x2),…,e(x n )] In the formula, the embedding vector (h1, h2, ..., h m )=Embedding({(p1, p2,...,p m )}), E mbedding represents an embedding layer that maps prompt tokens to continuous embeddings; the input sequence X i ={(x1, x2, ..., x n ); e(x j )(1≤j≤n) is the x obtained by the pre-trained embedding layer j Embedding The text representation is obtained based on a multilingual text representation model with learnable hints, expressed as: v i =PLM([h1,h2,...,h m ,e(x1),e(x2),...,e(x n )])。 4. The debiasing method for multilingual text classification according to claim 1, characterized in that: The text classification module includes a classifier G c , which consists of a linear layer; Performing a linear transformation on the text representation to obtain a prediction output includes: The text representation is linearly transformed based on the linear layer to obtain a predicted unnormalized score, which is used to characterize the confidence of the text classification module for each category in the classification task, and is expressed as: logits(v i )=G c (v i ) The unnormalized score is used as the prediction output of the text classification module.
5. A debiasing method for multilingual text classification according to claim 4, characterized in that: The cross entropy loss function is used to calculate the difference between the predicted output and the true label to determine the classification loss, which is expressed as: L c =CrossEntropy(logits(v i ),y i ) In the formula, y i is the target label of the i-th sample.
6. The debiasing method for multilingual text classification according to claim 1, characterized in that: Through back-propagation guided feature weighted adversarial training, text features in the text representation are weighted and sensitive attribute discrimination loss of the text is determined, including: The classification loss L of the text classification module c Back propagates to the multilingual text representation module to determine the gradient of the multilingual text representation module, which is expressed as: In the formula, v i is said text representation; The gradient is normalized, and the normalized result is used as the text representation v i The importance weight of Chinese text features is expressed as: Where λ is a scaling factor used to maintain the norm of the text representation space; is the gradient of the multilingual text representation module; v′ i is the weighted text representation; Based on the weighted text representation, the discriminator Gd is used to determine the sensitive attribute identification loss of the current sample, which is expressed as: L d =CrossEntropy(G d (v′ i ),g i ) In the formula, g i is the sensitive attribute label of the i-th sample, G d (v i ′) is the discriminator G d According to the weighted text representation v′ i The resulting unnormalized score.
7. The debiasing method for multilingual text classification according to claim 1, characterized in that: The optimization goal of the adversarial training is expressed as: Where γ is used to implement the cross entropy loss L c and sensitive attribute identification loss L d The loss weights that are balanced between represents the parameters of the learnable cue; θ c Indicates the classification c Parameters of θ d Indicates the discriminant G d Parameters.
8. The debiasing method for multilingual text classification according to claim 1, characterized in that: Use the generated text classification debiasing model to perform unbiased classification on the text to be classified, including: Get the text data to be classified, expressed as: D={(X1,y1,s1),(X2,y2,s2),...,(X K ,y k ,s k )} Where, X i ={x1, x2, ..., x n }(1≤i≤K) represents the text sequence of the i-th sample, n represents the length of the i-th sample sequence; y i is the target label of the i-th sample, where y i ∈Y, is the set of classification task categories; s i is a sensitive attribute label, where s i ∈S, S represents the set Y of sensitive attribute values; Obtaining text representation using a multilingual pre-trained model with learnable hints through the multilingual text representation module; classifying the text representation into predefined categories based on the text classification module; Based on the text debiasing module, the classification loss L of the text classification module is used c The gradient matrix obtained by back-propagation weights the text features in the text representation; Adversarial training is used to guide the text classification debiasing model to generate unbiased deep cue embeddings for unbiased classification.