A multi-label text classification method based on contrastive loss

Abstract

The application provides a multi-label text classification method based on contrastive loss, and relates to the technical field of natural language processing. The method first extracts text features, and calculates Jaccard similarity based on the multi-label annotation vector of the sample to finely quantify the semantic similarity between samples; then generates a positive sample mask through a dynamic threshold and a minimum intersection constraint to guide the calculation of supervised contrastive loss, so as to learn a more discriminative feature representation; finally, a delayed start strategy is adopted to weight and fuse the supervised contrastive loss and the binary cross-entropy classification loss into a contrastive balanced loss, so as to cooperatively optimize feature learning and classification decision. The application effectively improves the classification ability of the model for tail classes and the overall generalization performance.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, specifically to a multi-label text classification method based on contrastive loss. Background Technology

[0002] In the field of natural language processing, multi-label text classification aims to assign multiple semantic category labels to a single text. The mainstream approach relies on deep learning models and a binary cross-entropy loss function for end-to-end optimization. Despite progress in this area, two interrelated core problems remain: First, existing technologies struggle to effectively utilize the rich semantic relationships between training samples. Especially when introducing contrastive learning mechanisms aimed at improving feature discriminative power, they typically only consider samples with identical labels as similar pairs, while samples with partially overlapping labels are simply categorized as dissimilar pairs. This coarse-grained approach ignores the complex semantic similarities between samples, failing to fully utilize the structural knowledge in the data and potentially introducing significant noise, hindering the model from learning high-quality, highly generalizable text representations. Second, in real-world data with severe class imbalance, on the one hand, loss functions designed to optimize label prediction are easily dominated by the head class; on the other hand, there is a lack of a training mechanism that can explicitly model and utilize the similarity relationships between samples to simultaneously enhance the learning ability for tail classes and the semantic structure of the feature space. Summary of the Invention

[0003] To achieve the above objectives, this invention proposes a multi-label text classification method based on contrastive loss, comprising: S1: Obtain the training batch sample set, in which each sample includes text data and its corresponding multi-label annotation vector; S2: Extract the feature representation of each text data in the training batch sample set to form a feature representation set; S3: Based on all multi-label annotation vectors in the training batch sample set, calculate the Jaccard similarity between sample pairs to construct a Jaccard similarity matrix; S4: Based on the Jaccard similarity matrix, generate a positive sample mask matrix for each sample by applying a dynamic threshold and minimum intersection constraint. S5: Calculate the supervised contrastive loss based on the feature representation set and the positive sample mask matrix; S6: Input the set of feature representations into the classifier for multi-label prediction and calculate the binary cross-entropy loss; S7: Using a delayed start strategy, the binary cross-entropy loss and the supervised contrast loss are weighted and fused together using weighting coefficients to form a contrast balance loss; S8: Based on the contrastive balance loss, update the model parameters through backpropagation, and iteratively execute the steps until the model converges to obtain a text classification model for performing multi-label text classification tasks.

[0004] As a further technical solution, obtaining the training batch sample set includes the specific operation of constructing each training batch from the original dataset; this operation first converts the label set of each sample into a fixed-dimensional binary label vector, the length of which is equal to the preset total number of categories C, and each position in the vector corresponds to a category. If the sample contains that category, the corresponding position y i,c Set it to the number 1, otherwise set it to the number 0, thus converting the multi-label annotations into a machine-processable numerical form, forming a label matrix. Where B is the batch size; to alleviate the class imbalance problem during training, a class balancing sampling strategy is adopted when sampling from the dataset to construct a batch in each training iteration. This ensures that the label distribution of the samples in the batch not only covers the common head categories but also the tail categories with low frequency of occurrence, and guarantees that each tail category has at least one sample in the batch, thereby enhancing the model's ability to learn tail categories; for each sample in the batch, its text content is normalized, including word segmentation and converting the word sequence into a word sequence acceptable to the model, and padding or truncation to reach a fixed preset length L, as standardized text data.

[0005] As a further technical solution, the process of forming a feature representation set involves: inputting the preprocessed batch word sequence into the main encoder of the text classification model, capturing the contextual dependencies between words through its deep self-attention mechanism, and obtaining the context vector representation corresponding to each word output by the last hidden layer; because information dilution may exist in long text sequences, all word vectors of each sample are structured, that is, grouping word vectors into multiple continuous, non-overlapping sub-sequence blocks according to the sequence order, with each sub-sequence block containing a fixed number of word vectors; for each sub-sequence block, calculating the arithmetic mean of all word vectors within the block, which is used as the local semantic representation vector of the text segment, effectively summarizing local contextual information; finally, concatenating all local semantic representation vectors generated by the same sample into a long vector in the original order, and then using a learnable linear transformation layer to reduce the dimensionality of the long vector and fuse global information, outputting a vector z of fixed dimension d. i As the overall feature representation of this sample, the overall feature representation of all samples constitutes the feature representation set z={z1,z2,...,z...} of this batch. B}

[0006] As a further technical solution, a Jaccard similarity matrix is ​​constructed. The calculation process involves, for any two samples i and j in the training batch, first calculating the number of label intersections between their multi-label annotation vectors. This is achieved by summing the element-wise multiplications of the two binary vectors. , where y i,c Let represent the label value of sample i in category c, which is 1 or 0, and C is the total number of categories. Next, calculate the union of the labels of these two samples, which is the sum of the total number of labels for sample i and the total number of labels for sample j, minus the intersection. ,in , representing the total number of labels for sample i; This is the number of labels for sample j; the Jaccard similarity between samples i and j is obtained by dividing the number of intersections by the number of unions and a very small constant e, i.e. The constant e is added for numerical stability to prevent division by zero errors when the union is empty. By traversing all sample pairs in the batch, a symmetric Jaccard similarity matrix can be obtained, which quantitatively describes the degree of overlap in the label composition of any two samples.

[0007] As a further technical solution, the generation of the positive sample mask matrix utilizes Jaccard similarity to screen reliable positive samples, which directly affects the effectiveness of supervised contrastive learning. This process involves, for any two different samples i and j in the batch, selecting reliable positive samples based on the calculated Jaccard similarity J... i,j and the number of their label intersections I i,j The system applies a dual-criteria system; the first condition is that the Jaccard similarity must be greater than a preset dynamic threshold, i.e., J... i,j The threshold of >0.2 ensures that only samples with labels that have a certain degree of similarity are considered positive sample pairs, avoiding noisy positive samples caused by slight overlap; the second condition is that the number of intersections between the two labels must be at least 1, i.e., I i,j ≥1 is the basic semantic prerequisite for forming a positive sample pair; only when both of these conditions are satisfied simultaneously can the corresponding position M in the positive sample mask matrix be considered positive. i,j The value is set to 1 only if the condition is met; otherwise, it is set to 0. This logic is expressed as follows: , where 1{} is the indicator function; furthermore, to ensure that each sample has at least one positive sample (i.e., itself) in the contrastive learning, post-processing is required, that is, for each sample i, check all M samples except itself. i,j The sum of the values ​​is used to determine if the mask remains unchanged. If the sum is greater than 0, it indicates that at least one positive sample exists, and the mask remains unchanged. If the sum is equal to 0, then M is set to... i,j Set it to 1, and set all M... i,jWhen j is not equal to i, it is set to 0; therefore, the generated positive sample mask matrix M is a binary matrix whose diagonal is not necessarily 1, which precisely defines the contrastive learning target of each sample in the batch.

[0008] As a further technical solution, the computation of supervised contrastive loss is a specific optimization objective based on feature representation and positive sample mask for representation learning; firstly, for each feature vector z in the feature representation set... i L2 norm normalization is performed, and the sample is projected onto a unit hypersphere so that subsequent similarity calculations depend only on the angle between vectors. For each sample i in the batch, the set of indices of all positive samples is determined based on the positive sample mask matrix M. Supervised contrastive loss L... SC The core idea is to encourage the feature representation of sample i to be as similar as possible to the feature representations of all its positive samples, while distinguishing it from the feature representations of other negative samples in the batch; the calculation process involves calculating the normalized feature vector z of sample i. i The normalized feature vector z of all other samples j in the batch j The dot product similarity between them, i.e., s i,j =z i ·z j Then, each similarity score is scaled by dividing it by a temperature parameter τ; to further enhance discriminative power, a sample-specific offset m is introduced. i The offset is defined as the maximum similarity between sample i and all other samples in the batch, excluding itself, i.e., m i =max k≠i (z i ·z k The final supervised contrast loss L SC It is calculated using the following formula: Where B is the batch size, N is the total number of samples in the batch, e2 is a constant to prevent the denominator from being zero, τ represents the temperature parameter, and τ0 is the base temperature parameter; the loss function is the negative of the expected log-likelihood of the normalized similarity of each sample to its positive samples, and is averaged over the entire batch.

[0009] As a further technical solution, the calculation of the binary cross-entropy loss serves as the fundamental supervisory signal for performing multi-label classification tasks; specifically, it involves inputting the feature representation set into an independent linear classifier, which is a fully connected layer without an activation function; for each sample's feature representation z... i The classifier outputs a raw score vector; an sigmoid function σ transform is applied independently to each score, compressing it to the interval between 0 and 1, thus obtaining the predicted probability p of the sample belonging to each class. i,c =σ(z i cBinary cross-entropy loss L BCE These predicted probabilities are then calculated by comparing them with the true binary label vectors; for each sample i and each class c in the batch, the predicted probability p is calculated separately. i,c With real label y i,c The cross-entropy between samples is calculated; finally, the cross-entropy losses of all samples across all classes in the batch are summed and then averaged to obtain the binary cross-entropy loss L. BCE The calculation formula is as follows: The binary cross-entropy loss directly measures the model's class-by-class classification accuracy in multi-label prediction.

[0010] As a further technical solution, a contrastive balanced loss is formed through a training strategy that organically integrates two loss functions using a delayed-start strategy. This strategy aims to prioritize stabilizing the learning of the classification task in the early stages of training. After the model acquires preliminary discriminative ability, contrastive learning is introduced to refine the feature representation. Let the index of the current training epoch be e, and preset a number of delayed-start epochs E. In the early stages of training, when e is less than E, the weight coefficient ω of the supervised contrastive loss is set to 0, at which point the total loss L... CB Completely determined by binary cross-entropy loss L BCE Composition, namely L CB =L BCE When the training cycle reaches or exceeds E (i.e., e is greater than or equal to E), the weight coefficient ω is set to a preset constant ω0 that is greater than 0. At this point, the total loss becomes a weighted sum of the two, i.e. Through this mechanism, the staged fusion of loss functions is realized; the value of the weight coefficient ω0 is a hyperparameter that needs to be tuned according to the task. It controls the relative strength of the contrastive learning signal in the overall optimization objective, balancing the trade-off between classification accuracy and representation quality of the model.

[0011] As a further technical solution, updating the model parameters and iterating until the model converges is the final step in completing the training loop of the entire method. In each training iteration, the current contrastive balance loss is first calculated based on all the aforementioned steps. Then, using the backpropagation algorithm, the gradients of all trainable parameters in the text classification model, including the feature encoder and classifier, with respect to the contrastive balance loss are automatically calculated. Next, stochastic gradient descent or its variants, such as the Adam optimizer, combined with optimizer hyperparameters such as the learning rate, are used to update all model parameters based on the calculated gradients. After completing one parameter update, one round of training iteration is completed. Subsequently, the process from obtaining a new training batch sample set to calculating the contrastive balance loss is repeated. After calculating all steps of the comparison and balance loss, proceed to the next iteration. Set a convergence criterion: after each training cycle (i.e., after traversing the training set once), evaluate the model performance on an independent validation set, using the macro-average F1 score or average precision as the evaluation metric. If the performance improvement on the validation set is less than a preset small threshold (e.g., 0.001) for multiple consecutive training cycles (e.g., 5 or 10), the model performance is considered to have converged and has no further significant improvement potential. At this point, stop the training process and save the model parameters that currently have the best validation set performance. Finally, obtain a text classification model with strong generalization ability that can be used to perform multi-label text classification tasks.

[0012] This invention provides a multi-label text classification method based on contrastive loss, which has the following beneficial effects: 1. This invention solves the problem of overly strict definition of positive samples and neglect of some semantic similarity in traditional contrastive learning scenarios by constructing dynamic positive samples based on label Jaccard similarity, thereby improving the ability of the feature space to distinguish between similar and dissimilar samples.

[0013] 2. This invention solves the problems of insufficient learning of tail labels under long-tailed distribution and multi-objective optimization conflicts by fusing supervised contrast loss and classification loss with a delayed start strategy. This improves the model's classification accuracy and recall for all categories, especially the tail category with few samples. Attached Figure Description

[0014] Figure 1 This is a schematic diagram of the process of the present invention. Detailed Implementation

[0015] The technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0016] like Figure 1 As shown, this invention provides a specific embodiment of a multi-label text classification method based on contrastive loss. This embodiment includes two complete stages: model training and model use. It is validated on a publicly available large-scale multi-label text dataset, EUR-Lex, which contains approximately 19,000 text documents involving approximately 4,000 categories. Its label distribution exhibits typical long-tail characteristics.

[0017] Phase 1: Model Training For example, the hardware platform for implementing this invention is a server equipped with an NVIDIA A100 GPU and 64GB of memory, and the software environment is Python 3.8 and PyTorch 1.12 framework; The specific training process is as follows: First, data preparation and batch construction are performed. The original text content of each sample and its corresponding multiple category labels are read from the training segment of the EUR-Lex dataset. The set of all category labels constitutes a preset global category list with a length C of 3950. For any sample, its label set is converted into a binary label vector of length 3950. If the sample belongs to a certain category, the value of the vector at the corresponding index position is set to 1, otherwise it is 0. In each training iteration, an improved batch sampling strategy is adopted to sample a batch set of samples with a batch size of B equal to 128 from the training set. This sampling strategy ensures that the label distribution of the samples in each batch covers the head and tail categories as much as possible, and that each tail category appears at least once in the batch. For each sample in the batch, its text content is preprocessed, for example, by using the vocabulary corresponding to the BERT pre-trained model for word segmentation, converting the text into a word sequence, and uniformly truncating or padding to a fixed length L equal to 512 to generate a uniform input sequence.

[0018] Subsequently, text feature extraction is performed. The 512-word sequence of each sample in the batch is input into a pre-trained language model based on the BERT architecture, specifically a 12-layer Transformer encoder based on BERT. The model outputs the context vector representation of the last hidden layer for all samples in the batch, with each word corresponding to a 768-dimensional vector. For each sample, its 512-word vector representation is sequentially divided into 32 consecutive blocks, each containing 16 consecutive word vectors. For each block, the arithmetic mean of the 16 word vectors within the block is calculated, resulting in 32 local semantic representation vectors, each still 768-dimensional. These 32 vectors are then concatenated along the feature dimension to obtain a 24,576-dimensional vector. This high-dimensional vector is then projected onto a 256-dimensional semantic space through a trainable fully connected linear transformation layer, ultimately generating the overall feature representation z of the sample.i The feature representations of the 128 samples in the entire batch constitute the feature representation set z.

[0019] Next, we calculate label similarity and construct positive sample pairs. Based on the true binary label vectors of all 128 samples in the current batch, we calculate the Jaccard similarity between sample pairs. For any two samples i and j in the batch, we first calculate the number of their label intersections, I. i,j This means summing two label vectors after element-wise multiplication; for example, if the label vector of sample i is y... i The label vector of sample j is y j ,but Next, calculate the number of label unions, U. i,j That is, the total number of labels for sample i plus the total number of labels for sample j, minus the number of intersections I. i,j The expression is To prevent calculation errors caused by the union number being zero, a very small constant e is introduced, for example, e = 1e-8; finally, the Jaccard similarity J between samples i and j is... i,j Through formula It was calculated that by traversing all sample pairs within a batch, a 128×128 Jaccard similarity matrix can be constructed.

[0020] Based on the Jaccard similarity matrix, a positive sample mask matrix is ​​generated for each sample using a dynamic threshold and a minimum intersection constraint. In this embodiment, the preset dynamic similarity threshold is set to 0.2, and the minimum intersection constraint value is set to 1. For any sample pair i and j, if their Jaccard similarity J i,j Greater than 0.2, and the number of tags I they share i,j If the value is greater than or equal to 1, then sample j is considered a positive sample of sample i, and the mask M is... i,j Set it to 1 if the value is 1, otherwise set it to 0. After completing the initial construction, it is necessary to check the number of positive samples for each sample. For sample i, count the number of values ​​of 1 in its mask row. If the number is greater than 1, keep the mask of that row unchanged; if the number is equal to 0, change the mask M. i,j Setting it to 1 indicates that the sample itself is considered its only positive sample, while ensuring that other masks M in that row are also positive. i,j It is 0.

[0021] Next, the supervised contrastive loss is calculated; first, for each feature vector z in the feature representation set z... i L2 norm normalization is performed to obtain unit vectors; for each sample i in the batch, according to its positive sample mask M... i,j Identify the indices of all positive samples; calculate the dot product similarity between sample i and the normalized features of all other samples j in the batch, i.e., z.i Multiply z j The specific formula for calculating the supervised comparison loss is as follows: Where B is the batch size (128), N is the total number of samples (128), the constant e2 to prevent the denominator from being zero is set to 1e-8, the basic temperature parameter τ0 is set to 0.07, and the temperature parameter τ is set to 0.1; m i It is an alignment term whose value is the maximum similarity between sample i and all samples in the batch.

[0022] Simultaneously, multi-label classification and binary cross-entropy loss calculation are performed; the feature representation set Z is input into an independent linear classifier, which is a linear layer with parameter matrix W, an input dimension of 256, and an output dimension equal to the total number of classes C, which equals 3950; for sample i, the classifier outputs a 3950-dimensional original predicted score vector s. i ; for s i Each fraction in the equation is applied using a sigmoid function, the specific expression of which is: This transformation maps each original score to a value between 0 and 1, yielding the predicted probability p of sample i belonging to each category. i Based on the predicted probability p i The true binary label vector y of sample i i Calculate the binary cross-entropy loss; the specific formula is as follows: .

[0023] Then, a delayed start strategy is implemented to fuse the losses; the contrastive balancing loss L is defined. CB This is the weighted sum of the binary cross-entropy loss and the supervised contrast loss, i.e. Where ω is the weight coefficient; this embodiment uses a delayed start strategy to dynamically adjust ω; the total training period is set to 50, and the preset number of delayed start periods E is 5; in the first 5 training periods, the weight coefficient ω is set to 0; starting from the 6th training period, the weight coefficient ω is set to a constant ω0 greater than 0, in this embodiment ω0 is 0.5; thereafter, the model uses the full contrastive balance loss L CB =L BCE +0.5×L SC Optimize.

[0024] Finally, model optimization and training convergence are performed; the calculated contrastive balance loss L is then applied. CB As the final optimization goal of this training iteration; the backpropagation algorithm is used to calculate all trainable parameters of the model relative to L. CBThe gradient is calculated; the AdamW optimizer is used, with an initial learning rate of 2e-5 and a weight decay of 0.01. All parameters are updated based on the calculated gradient; after one parameter update, the data preparation step is returned for the next training iteration; to determine whether the model has converged, the model performance is evaluated on a fixed validation set after each training cycle, mainly monitoring the P-precision index; the convergence condition is set as the optimal improvement of the P-precision index on the validation set being less than 0.001 within 10 consecutive training cycles; when this condition is met, the model is considered to have converged, the training process is stopped, and the model parameters at this time are saved, resulting in a trained text classification model; this model includes a BERT encoder, a feature fusion linear layer, and a linear classifier.

[0025] Phase Two: Model Use and Reasoning Once training is complete, a text classification model is obtained that can be used to perform specific multi-label text classification tasks; the following explains how to use this model to automatically classify an unseen text.

[0026] First, the new text to be classified undergoes the same preprocessing as during the training phase; the text is segmented using the same vocabulary as during training and converted into a sequence of words; then, this sequence is truncated or padded to the same fixed length L equal to 512 as during training to generate an input sequence acceptable to the model.

[0027] Next, the 512-character input sequence is fed into the pre-trained text classification model. The forward propagation process of the model is as follows: the text sequence first passes through the BERT encoder to obtain the context vector representation of the last hidden layer; then, following the same rules as during training, the vector representation of the 512 characters is divided into 32 consecutive blocks, each with 16 vectors, and the mean of all character vectors in each block is calculated to obtain 32 local semantic representation vectors; after concatenating these vectors, they are dimensionality reduced and fused through a pre-trained feature fusion linear layer to finally generate a 256-dimensional overall feature representation of the new text.

[0028] Then, this 256-dimensional overall feature representation is input into a pre-trained linear classifier; the classifier performs a linear transformation and outputs a raw predicted score vector with dimension C equal to 3950, where each score corresponds to a preset category.

[0029] Subsequently, a Sigmoid function transformation is applied to each score in the original predicted score vector; the Sigmoid function has the following form: After this transformation, each score is compressed into the range of 0 to 1, resulting in the predicted probability of the new text belonging to each category, forming a 3950-dimensional predicted probability vector.

[0030] Finally, based on actual application requirements, a probability threshold is set to determine whether to assign a certain category to the text. In this embodiment, a fixed threshold of 0.5 is used as an example. All 3950 probability values ​​in the predicted probability vector are traversed, and categories with a probability value greater than or equal to 0.5 are determined as the predicted labels for the text. For example, if the predicted probability of the 100th category is 0.8, which is greater than 0.5, then the text is determined to belong to the 100th category. If the predicted probability of the 200th category is 0.3, which is less than 0.5, then the text is determined not to belong to the 200th category. Finally, the set of all category labels that meet the conditions is output as the multi-label classification result for the new text.

[0031] Objective experimental results on the standard test set of the EUR-Lex dataset show that, compared with the baseline BERT model trained using only binary cross-entropy loss, the final model trained using the method of this invention achieves an absolute improvement in accuracy, verifying the effectiveness of the technical solution of this invention in improving the performance of multi-label text classification.

[0032] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A multi-label text classification method based on contrastive loss, characterized in that, Includes the following steps: S1: Obtain the training batch sample set, in which each sample includes text data and its corresponding multi-label annotation vector; S2: Extract the feature representation of each text data in the training batch sample set to form a feature representation set; S3: Based on all multi-label annotation vectors in the training batch sample set, calculate the Jaccard similarity between sample pairs to construct a Jaccard similarity matrix; S4: Based on the Jaccard similarity matrix, a positive sample mask matrix is ​​generated for each sample using a dynamic threshold and minimum intersection constraints. Specifically, for any two samples i and j in the training batch, a positive sample mask matrix is ​​generated based on their Jaccard similarity J. i,j The comparison result with the preset threshold, and the number of shared tags I. i,j Calculate the positive sample mask matrix by comparing the result with the minimum intersection value. ,in, It is an indicator function, which is 1 when the condition is met and 0 otherwise; for each sample If the number of positive samples is greater than 1, keep the mask unchanged; otherwise, set... =1, The value is 0, so that each sample has at least one positive sample, which is itself; S5: Calculate the supervised contrastive loss based on the feature representation set and the positive sample mask matrix; S6: Input the set of feature representations into the classifier for multi-label prediction and calculate the binary cross-entropy loss; S7: Using a delayed start strategy, the binary cross-entropy loss and the supervised contrast loss are weighted and fused together using weighting coefficients to form a contrast balance loss; S8: Based on the contrastive balance loss, update the model parameters through backpropagation, and iteratively execute the steps until the model converges to obtain a text classification model for performing multi-label text classification tasks.

2. The multi-label text classification method based on contrastive loss according to claim 1, characterized in that, Step S1 includes: S11: Obtain the text content of each sample and its corresponding multiple labels from the original text dataset, and convert the label set of each sample into a fixed-dimensional binary label vector, where each position of the binary label vector corresponds to a preset category. If the sample contains the category, the corresponding position is set to 1, otherwise it is 0. S12: In each training iteration, a batch containing multiple samples is sampled from the dataset, such that the label distribution of the samples in the batch covers both the head and tail categories, and that each tail category appears at least once in the batch; S13: Perform word segmentation and serialization on the text content of each sample in the batch to generate a fixed-length word sequence as input text data.

3. The multi-label text classification method based on contrastive loss according to claim 2, characterized in that, Step S2 includes: S21: Input the word sequence of each sample in the batch obtained in S1 into the text classification model, and obtain the context vector representation of each word output by the last hidden layer; S22: For each sample, group the context vector representations of all its words in sequence order to form multiple consecutive, fixed-length subsequence blocks; S23: For each sub-sequence block, calculate the mean of all word vectors within the block to obtain the local semantic representation vector of that sub-sequence block; S24: Concatenate all local semantic representation vectors of the same sample, and then perform dimensionality reduction and fusion through a linear transformation layer to generate the overall feature representation of the sample, thus forming the feature representation set.

4. The multi-label text classification method based on contrastive loss according to claim 1, characterized in that, Step S3 includes: S31: For any two samples i and j in the training batch, based on their multi-label annotation vectors, calculate the number of labels shared by the two samples, denoted as the label intersection of samples i and j. ,in It is a label matrix, where C is the number of categories; This indicates that sample i belongs to category c; otherwise, it is 0. This indicates the number of labels shared by samples i and j; S32: Calculate the sum of the total number of labels for sample i and the total number of labels for sample j, then subtract the number of labels that intersect to obtain the number of labels that are the union of the labels for samples i and j. ,in It is the number of labels for sample i; It is the number of labels for sample j; Representing sample i and The number of the union of tags; S33: Divide the number of label intersections by the sum of the number of label unions and a constant e to obtain the Jaccard similarity matrix between samples i and j. .

5. The multi-label text classification method based on contrastive loss according to claim 1, characterized in that, S5 includes: Based on the feature representation set and the positive sample mask matrix, a supervised contrast loss L is calculated SC ; The computation proceeds as follows: for each sample i in the training batch, first its feature representation z i L2 normalization is performed; According to the positive sample mask matrix M i,j , determine all positive samples corresponding to sample i; Compute the dot product similarity z between the normalized feature representation of sample i and all other samples j within the batch i • z j ; The supervision contrast loss L SC is calculated by the following equation: Where B is the batch size and N is the total number of samples. To prevent constants with zero denominators, τ represents the temperature parameter, and τ0 is the fundamental temperature parameter. This means that for each sample Maximum similarity with all samples.

6. The multi-label text classification method based on contrastive loss according to claim 1, characterized in that, S6 includes: inputting the feature representation set into a linear classifier, which outputs the original predicted score for each sample corresponding to all preset categories; applying an S-shaped function transformation to each original predicted score to compress it to between 0 and 1, obtaining the predicted probability of each sample belonging to each category; calculating the binary cross-entropy loss based on the predicted probability and the true multi-label annotation vector obtained from the training batch sample set; the calculation process is as follows: for each sample and each category in the batch, the cross-entropy between the predicted probability and the true label value is calculated respectively, and the cross-entropy results of all categories of all samples in the batch are summed and averaged to obtain the binary cross-entropy loss L. BCE The calculation formula is as follows: ,in, It is an S-shaped function.

7. The multi-label text classification method based on contrastive loss according to claim 1, characterized in that, The S7 includes: adopting a delayed start strategy, and taking the binary cross-entropy loss L BCE and the supervised contrast loss L SC and performing weighted fusion through a weight coefficient to form a contrast balance loss L CB ; specifically, the calculation formula of the contrast balance loss L CB is as follows: ; the delayed start strategy is implemented in the following way: let the current training cycle index be e, and the preset number of delayed start cycles be E. When e < E, set the weight coefficient ; when e ≥ E, set the weight coefficient to be a constant greater than 0 0.

8. The multi-label text classification method based on contrastive loss according to claim 1, characterized in that, S8 includes: using the calculated contrastive balance loss as the optimization objective, and using the backpropagation algorithm to calculate the gradient of all trainable parameters in the text classification model relative to the contrastive balance loss; updating the model parameters using stochastic gradient descent based on the gradient; repeating steps S1 to S7 after each parameter update for the next training iteration; setting a convergence judgment condition, determining that the model has converged when the performance improvement on the validation set for multiple consecutive training cycles is less than a preset threshold, stopping training, saving the final model parameters, and obtaining a text classification model for performing multi-label text classification tasks.

Images