Semi-supervised text data multi-label classification method, system and equipment and storage medium

Abstract

The invention discloses a semi-supervised text data multi-label classification method, system and device and a storage medium. According to the method, a double-branch model of a shared feature processing network is constructed, a pseudo label generator is utilized to automatically generate a pseudo label for an unlabeled sample on the basis of limited labeled data, and the pseudo label generator and a classifier are jointly trained to realize collaborative learning of labeled data and unlabeled data; by introducing an adaptive threshold mechanism and an improved loss function design, the recognition precision of minority class labels is effectively improved. Compared with a traditional full supervision model, the method has the advantages that the dependence on large-scale manual annotation is reduced, the data preparation cost is remarkably reduced, and the application performance of the classification model in multi-label scenes such as medical text analysis, public opinion monitoring and personalized recommendation is improved. The system, the device and the storage medium provided by the invention can realize the method, and have good expandability and engineering application value.

Description

Technical Field

[0001] This invention relates to the field of computer technology applications, specifically to a semi-supervised multi-label classification method and device for text data. Background Technology

[0002] With the deepening application of artificial intelligence in natural language processing, multi-label text classification has become a crucial task. Unlike traditional single-topic classification (such as determining whether a news article belongs to "sports" or "finance"), multi-label classification acknowledges the complex semantic structure of real-world text, allowing an instance to be assigned multiple labels. For example, a Weibo post about a new smartphone might simultaneously contain multiple labels such as "technology," "positive reviews," "complaints about battery life," and "price mentions"; a chief complaint in an electronic medical record might be associated with multiple diagnoses such as "hypertension," "diabetes," and "coronary heart disease." This capability makes multi-label classification indispensable in fields such as public opinion monitoring, medical diagnosis, content recommendation, and document management.

[0003] However, the superior performance of multi-label classification models heavily relies on a large amount of high-quality labeled data. Multi-label text annotation is an extremely challenging task: annotators need to carefully read the text, understand its multi-layered semantics, and accurately determine its relationship with all possible labels. This process is not only costly and time-consuming, but also demands a high level of domain expertise from the annotators (e.g., in the medical or legal fields), which greatly limits the widespread adoption and application of advanced multi-label learning techniques in real-world scenarios.

[0004] To overcome the bottleneck of labeled data, semi-supervised multi-label learning has emerged. This technique aims to train a model using a small amount of manually labeled text and a large amount of readily available unlabeled text. One of the core strategies is pseudo-labeling, which involves using the current model to predict unlabeled text and using the predictions with higher confidence as "pseudo-labels," mixing them with the original labeled data to expand the training set.

[0005] In multi-label text classification, the core challenge of pseudo-labeling techniques lies in setting a reliable confidence threshold for each potential label of a text sample. Traditional methods typically employ fixed thresholds or simple dynamic thresholds. However, the complexity of text makes single-threshold strategies problematic: for sparsely labeled text, such as a short product review mentioning only price, a threshold that is too high will miss true labels (false negatives). For densely labeled text, such as a detailed product review covering aspects like appearance, performance, system, and photography, a threshold that is too low will introduce a large number of noisy labels (false positives).

[0006] To address this issue, researchers proposed a class-level thresholding strategy, which sets an independent threshold for each category based on the frequency of each label's occurrence in the training set. While this method considers the overall distribution of different labels, it suffers from a fundamental flaw in text processing: it ignores the unique contextual semantic information of different text instances. Specifically:

[0007] Context bias is a problem. Models tend to overestimate the predictive power of labels appearing in specific contexts. For example, in sentiment analysis, once the word "price" appears in the text, the model might overestimate its confidence in predicting "complaints" among all texts containing "price" because of the strong correlation between this word and the "complaint" label, even if the text is actually praising the cost-effectiveness. Class-level thresholds cannot adaptively adjust based on the overall sentiment tone (context) of the current text.

[0008] Label co-occurrence is misleading. In medical texts, "diabetes" and "kidney disease" frequently co-occur. For a text that primarily describes the symptoms of "diabetes" but does not explicitly mention "kidney disease," the model might assign a moderate but spurious confidence score to "kidney disease" due to the strong association between the two. Class-level thresholds may rigidly treat all moderate-confidence "kidney disease" predictions as spurious labels, leading to error accumulation.

[0009] Text length and complexity interference. A lengthy, semantically complex technical document may generally have lower predicted confidence scores for its various tags than a short, clearly defined announcement. Class-level thresholds cannot detect this overall confidence shift caused by the inherent characteristics of the text itself.

[0010] Therefore, existing pseudo-labeling methods based on class-level thresholds still have significant shortcomings in text multi-label classification:

[0011] (1) Ignoring instance-level context leads to pseudo-tags being subject to contextual biases caused by lexical associations and tag co-occurrence.

[0012] (2) The threshold setting is rigid and cannot be flexibly adjusted according to the length, semantic complexity and other characteristics of the text itself.

[0013] (3) In the cold start scenario where labeled data is scarce, the above-mentioned biases and errors will be amplified in subsequent training iterations, eventually leading to a decline or even degradation of model performance.

[0014] Therefore, there is an urgent need for a pseudo-label generation method that can deeply understand the contextual information of text instances and dynamically adjust the threshold for each text and each label accordingly. This method can more accurately identify multiple labels that actually exist in the text, improve the quality and robustness of pseudo-labels, and thus push the performance of semi-supervised multi-label text classification to a new level. Summary of the Invention

[0015] Purpose of the invention: In existing text multi-label learning methods, pseudo-label generation relies on fixed thresholds or class-level thresholds, which cannot fully utilize the contextual information of different instances, thus leading to pseudo-label bias. This invention provides a semi-supervised text data multi-label classification method that improves the accuracy and robustness of pseudo-label generation under limited labeling resources, thereby enhancing the overall performance of semi-supervised text multi-label classification.

[0016] Another object of the present invention is to provide a semi-supervised text data multi-label classification system, as well as an electronic device, computer storage medium and computer program product capable of implementing the above method.

[0017] Technical solution: To achieve the above objectives, the present invention adopts the following technical solution:

[0018] A semi-supervised multi-label classification method for text data includes the following steps:

[0019] Step S1: Obtain the text data to be processed and initialize the semi-supervised multi-label text data classification task; the text data to be processed includes labeled text datasets. and unlabeled text datasets The initialization includes setting the number of model warm-up epochs (WARMUP_EPOCH) and the total number of training epochs (MAX_EPOCHS), as well as the loss weights. The model includes a feature processing network and a dual-branch network, wherein the dual-branch network includes a pseudo-label generator. and classifier Among them, pseudo-tag generator Used to generate pseudo-labels for unlabeled samples; classifier Used for multi-label prediction using labeled and pseudo-labeled samples;

[0020] Step S2: Use the labeled text dataset As pre-training data, the feature processing network and the dual-branch network are warmed up for 10 training epochs (WARMUP_EPOCH), with the pseudo-label generator branch... Based on the improved pairwise sorting loss function Optimize to learn the threshold function Classifier branch Based on asymmetric loss function Optimization is used to complete classification tasks; the sum of the two. The overall model parameters, which are composed of the feature processing network and the dual-branch network, are updated jointly as the total loss.

[0021] Step S3, in the Turn number In each training round, unlabeled samples will be... Input pseudo-label generator Get output Calculate its threshold And generate pseudo tags according to the following rules For each class k, when season ,otherwise Thus, a pseudo-annotated text dataset is obtained. ;

[0022] Step S4, in the Turn number In each training epoch, the feature processing network and the dual-branch network are semi-supervised, and the pseudo-label generator branch is trained. In labeled text datasets Continue to use Optimization, to optimize the threshold function; classifier branch Merged data from labeled text datasets and pseudo-labeled text datasets The above adopts Optimize to improve classification performance by utilizing pseudo-labels; update the parameters of the dual-branch network and the shared feature processing network;

[0023] Step S5: Repeat steps S3 and S4 until the iteration termination condition is met to obtain the final trained model. Use the final model for multi-label classification of unlabeled text data.

[0024] Furthermore, in step S1, there is a labeled text dataset. satisfy:

[0025]

[0026] in, Let represent the input feature vector of the i-th labeled text. This indicates the corresponding multi-label annotation, where C is the number of label categories and N is the total number of samples with labeled text;

[0027] Unlabeled text dataset satisfy:

[0028]

[0029] Where M is the total number of unlabeled text samples, .

[0030] Furthermore, in step S2, the pseudo-tag generator branch... Improved pairwise sorting loss function Defined as:

[0031]

[0032] in, For the sample The set of positive labels For the sample The set of negative labels; m is the interval parameter. The predicted value for label 0; Indicates sample The predicted score in the u-th category, Indicates sample The predicted score in the v-th category.

[0033] Furthermore, in step S2, the classifier branch... asymmetric loss function Defined as:

[0034]

[0035] in, This represents the loss term for positive samples. This represents the loss term for negative samples, while positive samples refer to the true labels. The category, negative samples refer to the true labels. Category and is the adjustment parameter for positive and negative samples, and C is the number of label categories.

[0036] Furthermore, in step S3,

[0037] Pseudo-tag generator The output dimension is C+1, where the 0th dimension comes from the added virtual category nodes, denoted as . The C dimension corresponds to the actual label categories 1 to C, denoted as The virtual category node serves as a mapping for the instance adaptive threshold function:

[0038]

[0039] For unlabeled samples The rules for generating pseudo-tags are as follows:

[0040]

[0041] By threshold function Calculate its threshold for:

[0042]

[0043] Therefore, we can deduce that:

[0044]

[0045] in, This indicates that the pseudo-label generator is applied to the samples. The predicted probability on the k-th class, This indicates the generated pseudo-tags;

[0046] The pseudo-annotated text dataset is defined as follows:

[0047]

[0048] in, This indicates a sample from an unlabeled text dataset. For pseudo-tag generator and instance adaptive threshold The generated multi-label pseudo-label vectors satisfy:

[0049] .

[0050] Furthermore, in step S5, the iteration termination condition includes one of the following: the number of training rounds reaches... The validation set metrics showed no significant improvement over P consecutive rounds, and the training / validation loss met the preset convergence threshold.

[0051] A semi-supervised multi-label classification system for text data includes:

[0052] The initialization module is used to acquire the text data to be processed and initialize the semi-supervised multi-label text data classification task; the text data to be processed includes labeled text datasets. and unlabeled text datasets The initialization includes setting the number of model warm-up epochs (WARMUP_EPOCH) and the total number of training epochs (MAX_EPOCHS), as well as the loss weights. The model includes a feature processing network and a dual-branch network, wherein the dual-branch network includes a pseudo-label generator. and classifier Among them, pseudo-tag generator Used to generate pseudo-labels for unlabeled samples; classifier Used for multi-label prediction using labeled and pseudo-labeled samples;

[0053] The warm-up training module is used with labeled text datasets. As pre-training data, the feature processing network and the dual-branch network are warmed up for 10 training epochs (WARMUP_EPOCH), with the pseudo-label generator branch... Based on the improved pairwise sorting loss function Optimize to learn the threshold function Classifier branch Based on asymmetric loss function Optimization is used to complete classification tasks; the sum of the two. The overall model parameters, which are composed of the feature processing network and the dual-branch network, are updated jointly as the total loss.

[0054] The pseudo-tag generation module is used in the first... Turn number In each training round, unlabeled samples will be... Input pseudo-label generator Get output Calculate its threshold And generate pseudo tags according to the following rules For each class k, when season ,otherwise Thus, a pseudo-annotated text dataset is obtained. ;

[0055] Joint training module, used in the second stage Turn number In each training epoch, the feature processing network and the dual-branch network are semi-supervised, and the pseudo-label generator branch is trained. In labeled text datasets Continue to use Optimization, to optimize the threshold function; classifier branch Merged data from labeled text datasets and pseudo-labeled text datasets The above adopts Optimize to improve classification performance by utilizing pseudo-labels; update the parameters of the dual-branch network and the shared feature processing network;

[0056] The termination and output block is used to repeatedly execute the training process of the pseudo-label generation module and the joint training module until the iteration termination condition is met, and obtain the final trained model. The final model is then used for multi-label classification tasks of unlabeled text data.

[0057] Furthermore, the termination and output module supports batch text inference, visualization of classification results, and online deployment interface calls, enabling the system not only to complete model training but also to be directly used in practical application scenarios of multi-label text classification.

[0058] The present invention also provides an electronic device, comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, wherein when the programs are executed by the processors, they implement the steps of the semi-supervised text data multi-label classification method as described above.

[0059] The present invention also provides a computer storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the semi-supervised text data multi-label classification method as described above.

[0060] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the semi-supervised text data multi-label classification method as described above.

[0061] Beneficial effects: (1) The semi-supervised text data multi-label classification method proposed in this invention no longer uses fixed or class-level thresholds, but uses the output of the virtual category node in the pseudo-label generator as the instance threshold function, so that the threshold can be adaptively adjusted according to the context of each sample, effectively alleviating the context bias problem in the pseudo-label generation process. (2) This invention trains the threshold function through an improved pairwise ranking loss function, ensuring that the predicted value of positive samples is greater than the threshold and the predicted value of negative samples is less than the threshold, thereby realizing the explicit learnable optimization of the threshold function and reducing the empirical and arbitrary nature of threshold selection. (3) This invention introduces two branches, a pseudo-label generator and a classifier, on the basis of a shared feature processing network, separating pseudo-label generation from model training, avoiding the direct impact of pseudo-label errors on classifier training, improving the independence of pseudo-labels and the robustness of training, significantly improving model performance under limited labeling resources, reducing manual labeling costs, and possessing good scalability and application value. Attached Figure Description

[0062] Figure 1 This is a flowchart of the semi-supervised text data multi-label classification method provided in this embodiment of the invention;

[0063] Figure 2 This is a schematic diagram illustrating the processing logic of the semi-supervised text data multi-label classification method provided in this embodiment of the invention for two types of input data.

[0064] Figure 3 This is a structural block diagram of a semi-supervised text data multi-label classification system provided in an embodiment of the present invention.

[0065] Figure 4 This is a hardware structure block diagram of the electronic device provided in the embodiments of the present invention. Detailed Implementation

[0066] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0067] Figure 1 This is a flowchart of a semi-supervised text data multi-label classification method provided in an embodiment of the present invention. Figure 1 As shown, the method in this embodiment includes steps S1 to S5: task initialization → warm-up training → pseudo-label generation → semi-supervised joint training → termination and output. This process is designed for scenarios where text-based labeled data is scarce but unlabeled data is sufficient. The goal is to improve multi-label classification performance under limited labeling conditions by using instance-level adaptive thresholding and dual-branch decoupled training.

[0068] Specifically, it includes the following steps:

[0069] Step S1: Initialize the semi-supervised multi-label classification task.

[0070] The semi-supervised multi-label classification scenario used in this method includes one labeled text dataset. 1 unlabeled text dataset One feature processing network, Net, and one two-branch network including a pseudo-label generator. and classifier The dual-branch network shares the same feature processing network Net, in which the pseudo-label generator... Used to generate pseudo-labels for unlabeled samples; classifier This model is used for multi-label prediction using labeled and pseudo-labeled samples. Model training consists of two parts: a warm-up phase and semi-supervised training. The number of warm-up rounds is set to WARMUP_EPOCH and the total number of training rounds is set to MAX_EPOCHS. The loss weights α and β are set to 0, the learning rate is set to 0, the batch size is set to 0, and the interval parameter m is set to 0.

[0071] Optional features: data augmentation strategies (such as random pruning, perturbation, MixUp / mean noise injection), optimizers can be SGD / Adam / AdamW, and learning rates can be cosine annealing or piecewise decay, none of which constitute a limitation on the present invention.

[0072] In this invention, there are labeled text datasets that satisfy:

[0073]

[0074] in, Let represent the input feature vector of the i-th sample. ,Right now , where represents the corresponding multi-label annotation vector, C is the number of label categories, and N is the total number of labeled samples.

[0075] In the multi-label text classification scenario of this invention This can be a complete text sample, such as a paragraph, an article, or a user comment. After text segmentation, encoding, or embedding (such as BERT encoding or word vector mapping), an input feature vector is formed. The corresponding label vector... Using 0 / 1 multi-hot encoding, the sample's affiliation to each category is represented. If it belongs to class c, then If it does not belong to this category, then For example, in a multi-label classification task for news text, if the category set is {"sports", "technology", "finance", "entertainment"}, then the number of label categories is... If a certain news article If it is related to both "technology" and "finance", the corresponding tags are:

[0076]

[0077] in, A multi-label vector of length C represents a sample. Label status in each category, for each dimension This corresponds to the c-th category.

[0078] The unlabeled text dataset satisfies:

[0079]

[0080] Where M is the total number of unlabeled samples, . This represents the input features of the j-th unlabeled text sample. Unlabeled data typically follows the same distribution as labeled data, but does not contain explicit label vectors. These samples will be used to predict pseudo-labels through a pseudo-label generator for subsequent semi-supervised joint training.

[0081] Step S2: Model warm-up training.

[0082] In the first WARMUP_EPOCH training epochs, the labeled text dataset was used. The feature processing network and the dual-branch network were pre-trained, with the pseudo-label generator branch being... Based on the improved pairwise sorting loss function Optimization to learn threshold function Classifier branch Based on asymmetric loss function Optimize to complete the classification task, using a weighted sum of the two. As the total loss, backpropagation simultaneously updates the overall model parameters, which include the feature processing network and the dual-branch network, to obtain a stable initial solution.

[0083] The pseudo-tag generator branch in this invention The improved pairwise sorting loss function is defined as:

[0084]

[0085] in, For example The set of positive labels For example The set of negative labels, where m is the interval parameter. The predicted value for label 0, which is the output of the virtual category node, can be used as a threshold by design. Its reasoning process is described in step S3. Based on consensus within the domain, a definition can be given: if label c belongs to sample... Authentic labels (i.e.) ),but This indicates that the category is a positive label; if label c does not belong to the sample... (Right now ),but This indicates that the category is a negative label.

[0086] The output of the pseudo-label generator branch is denoted as ,in Indicates sample The predicted score for the u-th class (usually the Sigmoid output) reflects the confidence that the sample belongs to the u-th class; similarly, This represents its predicted score in class v; the different signs (u and v) are to distinguish between positive and negative class attributes. Specifically, The predicted output of the "virtual category node (virtual label)" can be used as an estimate of the adaptive threshold based on the inference in step S3. This node does not correspond to an actual category but instead automatically learns an instance-level threshold through the training process to distinguish the positive and negative label confidence boundaries of a sample. Therefore, the improved pairwise ranking loss function... The intended constraints are: (1) the predicted score for each positive label. It should be higher than the virtual threshold. At least m; (2) the predicted score for each negative label Should be below the virtual threshold At least m.

[0087] Classifier branches The asymmetric loss function is defined as:

[0088]

[0089] in, This indicates that the positive sample ( The loss item, This indicates that the negative sample ( The loss term is calculated as follows: In multi-label classification tasks, a sample may belong to multiple categories simultaneously, so each category label k can be independently considered as a binary classification subtask. In this case, the positive sample refers to the true label. The category, negative samples refer to the true labels. The category. The asymmetric loss function introduces different adjustment factors for positive and negative samples respectively. , This approach enhances hard-to-classify samples and suppresses a large number of easily classified negative samples, thereby alleviating the problem of extreme imbalance between positive and negative samples in multi-label tasks.

[0090] The overall loss function is defined as:

[0091]

[0092] Used to jointly optimize the pseudo-label generator branch and the classifier branch, as well as the shared feature processing network.

[0093] Step S3: Generating pseudo-tags.

[0094] In the Turn In each training epoch, unlabeled samples will be... Input pseudo-label generator Get output Calculate its threshold And generate pseudo-tags according to the rules. For each class k, when season ,otherwise Thus, a pseudo-annotated text dataset is constructed. .

[0095] Specifically, pseudo-tag generator The output dimension is C+1, where the 0th dimension comes from the added virtual class node (i.e., the 0th neuron in the network output layer, used to represent the predicted output or confidence of the virtual class), denoted as . The C dimension corresponds to the actual label categories 1 to C, denoted as The virtual category node serves as a mapping for the instance adaptive threshold function:

[0096]

[0097] For unlabeled samples The rules for generating pseudo-tags are as follows:

[0098]

[0099] By threshold function Calculate its threshold for:

[0100]

[0101] Therefore, we can deduce that:

[0102]

[0103] in, This indicates that the pseudo-label generator is applied to the samples. The predicted probability on the k-th class, This indicates the generated pseudo-tag.

[0104] The pseudo-annotated text dataset is defined as follows:

[0105]

[0106] in, This indicates a sample from an unlabeled text dataset. For pseudo-tag generator and instance adaptive threshold The generated multi-label pseudo-label vectors satisfy:

[0107]

[0108] The processing procedures for labeled and unlabeled datasets are as follows: Figure 2 As shown.

[0109] Step S4: Semi-supervised dual-branch joint training.

[0110] In the Turn number In each training epoch, the feature processing network and the dual-branch network are semi-supervised, and the pseudo-label generator branch is trained. In the labeled text dataset Continue to use Optimization to refine the threshold function; classifier branch Merged data from labeled text datasets and pseudo-labeled text datasets The above adopts Optimize to improve classification performance by utilizing pseudo-labels; update the parameters of the dual-branch and shared feature processing networks.

[0111] Step S5: Iteration termination and model application.

[0112] Repeat the pseudo-label generation and joint training steps until the iteration termination condition is met to obtain the final trained model. The iteration termination condition includes one of the following: (1) The number of training rounds reaches (2) The validation set metrics did not improve significantly within P consecutive rounds; (3) The training / validation loss met the preset convergence threshold.

[0113] During iterative training, unlabeled text samples are predicted and assigned pseudo-labels by the model in each round of pseudo-label generation, for use in subsequent semi-supervised joint training. Therefore, at the end of the iteration, all unlabeled samples have obtained stable class prediction results through the final trained model. These prediction results can be considered as the model's final classification output for the unlabeled samples. After termination, the final model parameters are exported for online inference or downstream applications, including the feature processing network, the pseudo-label thresholding sub-network (i.e., the virtual class branch), and the classifier branch. In the model deployment phase, this model can be used for inference tasks in real-world application scenarios.

[0114] (1) Input stage: Receive the raw text data to be classified (such as articles, comments, Q&A content, etc.), and process it through the same feature extraction and encoding process;

[0115] (2) Inference stage: The feature vector is input into the trained model, and the model automatically outputs the prediction score corresponding to each category. and threshold branch output ;

[0116] (3) Output stage: According to the adaptive threshold rule, when When the text belongs to category k, the system determines that the text belongs to category k; otherwise, it does not, thus obtaining the final multi-label classification result.

[0117] This model can be deployed as a standalone inference service, or the threshold subnetwork and classifier subnetwork can be encapsulated as separate artifacts to adapt to different system topologies. For example, in a cloud-based inference system, the threshold subnetwork can run independently as a dynamic threshold module to enable flexible invocation for multi-task text classification.

[0118] To verify the performance of the method of this invention, a comparative experiment was conducted using this method and a neural network (CNN) model. The mean average precision (mAP) of different methods on a dataset with different label ratios P ​​is shown in Table 1.

[0119] Table 1. Comparison of mAP of different methods on different proportions of the dataset.

[0120] P=0.05 P=0.10 P=0.15 P=0.2 CNN+BCE 0.38 0.40 0.41 0.4 Method of the present invention 0.52 0.59 0.58 0.59

[0121] As can be seen from Table 1, the method proposed in this invention significantly improves the mAP index compared to the CNN model on different label ratios of a dataset, demonstrating the effectiveness and practicality of the method described in this invention.

[0122] Based on the same technical concept as the method embodiment, another embodiment of the present invention provides a semi-supervised text data multi-label classification system. For example... Figure 3 As shown, the system 300 of this embodiment includes components that are communicatively connected to each other:

[0123] Initialization module 310: Executes step S1 to complete data loading, network construction, and hyperparameter settings;

[0124] Preheating training module 320: Perform step S2, jointly minimize on the labeled set. ;

[0125] Pseudo-label generation module 330: Executes step S3 and outputs the text dataset of pseudo-labels. ;

[0126] Joint training module 340: Perform step S4 on the text data Iterative optimization;

[0127] Termination and Output Module 350: Executes step S5, determines the iteration termination condition, and exports the final trained model parameters. After model export, Termination and Output Module 350 also performs the model's practical application functions: receiving unlabeled text data to be classified, completing multi-label classification inference through the trained model, and outputting the corresponding category prediction results. Specifically, this module can call the model's feature processing network, pseudo-label threshold sub-network, and classifier sub-network to perform feature encoding and forward computation on the input text, obtaining prediction scores and instance-level thresholds for each category; and according to the rules, when the prediction score is greater than the threshold, the corresponding category is determined as a positive label, thereby achieving automatic label classification of the input text. This module can further support batch text inference, classification result visualization, and online deployment interface calls, enabling the system not only to complete model training but also to be directly used in practical application scenarios of multi-label text classification.

[0128] Preferably, the system 300 may also include a hardware acceleration interface 360 ​​for GPU / NPU / TPU acceleration of forward and backward propagation; it can support mixed precision training (such as FP16 / BF16) and tensor parallel / pipeline parallel strategies to adapt to large-scale data and model scenarios.

[0129] The above modules can be implemented as software functional modules, hardware circuits, or a combination of software and hardware, without being limited to a specific implementation form. Each module can be deployed on the same device or in a distributed manner, working collaboratively through network interconnection.

[0130] Another embodiment of the present invention provides an electronic device, including: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, wherein when the programs are executed by the processors, they implement the steps of the semi-supervised text data multi-label classification method as described above.

[0131] See Figure 4 , Figure 4 This is a hardware structure block diagram of the electronic device provided in an embodiment of the present invention. For example... Figure 4 As shown, the electronic device 400 of this embodiment includes: an input device 401, a processor 402, a memory 403, a storage device 404, an optional network interface 405, a display device 406, etc.

[0132] The memory 403 stores a computer program. When the processor 402 executes the program, it calls the method steps (S1–S5) of the above embodiments, including but not limited to: data loading, warm-up training, pseudo-label generation, joint training and termination output.

[0133] The processor 402 can be a CPU, DSP, ASIC, FPGA, or other programmable logic device; the memory 404 can be RAM / ROM / flash memory / SSD, etc. The network interface 401 is used to interact with external servers or storage resources to load data or synchronize models.

[0134] The above hardware configuration is merely an example, and the specific combination and topology do not constitute a limitation on the present invention.

[0135] The present invention also provides a computer storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the semi-supervised text data multi-label classification method as described above.

[0136] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, electronic devices, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0137] This invention is described with reference to a flowchart of a method according to embodiments of the invention. It should be understood that each step in the flowchart and combinations thereof can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing device, generate instructions for implementing the process. Figure 1 Means for a function specified in one or more processes. These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that are implemented in the process. Figure 1 The computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 Steps of a specified function in one or more processes.

Claims

1. A semi-supervised text data multi-label classification method, characterized in that, The method comprises the following steps: Step S1: Obtain the text data to be processed and initialize the semi-supervised multi-label text data classification task; the text data to be processed includes labeled text datasets. and unlabeled text datasets The initialization includes setting the number of model warm-up epochs (WARMUP_EPOCH) and the total number of training epochs (MAX_EPOCHS), as well as the loss weights. The model includes a feature processing network and a dual-branch network, wherein the dual-branch network includes a pseudo-label generator. and classifier Among them, pseudo-tag generator Used to generate pseudo-labels for unlabeled samples; classifier Used for multi-label prediction using labeled and pseudo-labeled samples; Step S2, using a labeled text dataset As pre-training data, the feature processing network and the double-branch network are pre-trained for WARMUP_EPOCH training rounds, where the pseudo-label generator branch Based on the improved pair-wise ranking loss function Optimization, so as to learn the threshold function ; the classifier branch Based on the asymmetric loss function Optimization, to complete the classification task; the sum of the two As the total loss, jointly update the overall model parameters including the feature processing network, the double-branch network Step S3, in the Turn number In each training round, unlabeled samples will be... Input pseudo-label generator Get output Calculate its threshold And generate pseudo tags according to the following rules For each class k, when season ,otherwise Thus, a pseudo-annotated text dataset is obtained. ; Step S4, in the first each training round, the feature processing network and the double-branch network are semi-supervised trained, the pseudo-label generator branch on the annotated text dataset continue to use optimization to optimize the threshold function; the classifier branch on the merged data of the annotated text dataset and the pseudo-annotated text dataset use optimization to improve the classification performance by using pseudo-labels; update the parameters of the double-branch network and the shared feature processing network;​ Step S5: repeatedly performing step S3 and step S4 until an iteration termination condition is reached, obtaining a final trained model, and using the final model for a multi-label classification task of unlabeled text data.

2. The method of claim 1, wherein, In the step S1, the annotated text dataset satisfies: ； wherein, represents the input feature vector of the i-th labeled text, represents the corresponding multi-label label, C is the number of label categories, and N is the total number of labeled text samples. Unlabeled text dataset satisfies: ； wherein M is the total number of samples of unlabeled text, .

3. The method of claim 1, wherein, In the step S2, the pseudo label generator branch An improved pairwise ranking loss function is defined as: ； wherein, is a positive label set for samples is a negative label set for samples is a negative label set for samples is an interval parameter, is a predicted value for the label 0; denotes a predicted score of a sample on the u-th class, denotes a predicted score of a sample on the v-th class.

4. The method of claim 1, wherein, In the step S2, the classifier branch of the asymmetric loss function is defined as: ； wherein, represents a loss term for positive samples, represents a loss term for negative samples, positive samples refer to the class of the real label , and negative samples refer to the class of the real label , and are adjustment parameters for positive and negative samples, and C is the number of label classes.

5. The method of claim 1, wherein, In the step S3, Pseudo label generator The output dimension is C + 1, where the 0-th dimension comes from the added virtual class node, denoted as , and the latter C dimensions correspond to the real label classes 1 ~ C, denoted as The virtual class node serves as a mapping of the instance-adaptive threshold function: ； For unlabeled samples The pseudo label generation rule is that ； By the threshold function Computing its threshold is: ； Thus, the inference can be obtained: ； wherein, denotes the pseudo-label generator on the sample the predicted probability on the k-th class, denotes the generated pseudo-label; The pseudo-labeled text data set is defined as: ； wherein, denotes a sample from an unlabelled text dataset, is generated according to a pseudo label generator and an instance adaptive threshold the generated multi-label pseudo-labeled vector satisfies: 。 6. A semi-supervised text data multi-label classification system, characterized in that, It comprises: An initialization module for obtaining text data to be processed and initializing a semi-supervised text data multi-label classification task; The to-be-processed text data includes an annotated text data set and an unannotated text data set ; the initialization includes setting a model warm-up round WARMUP_EPOCH and a total training round MAX_EPOCHS, a loss weight , the model includes a feature processing network and a double-branch network, the double-branch network includes a pseudo-label generator and a classifier , wherein the pseudo-label generator is used to generate a pseudo-label for an unannotated sample, and the classifier is used to perform multi-label prediction by using annotated samples and pseudo-annotated samples; a pre-training module for pre-training the feature processing network using a labeled text dataset as pre-training data, the feature processing network and the dual-branch network are pre-trained for WARMUP_EPOCH training epochs, wherein the pseudo-label generator branch based on an improved pair-wise ranking loss function optimization, so as to learn the threshold function ; the classifier branch based on an asymmetric loss function optimization, to complete the classification task; the sum of the two as the total loss, jointly update the overall model parameters including the feature processing network, the dual-branch network a pseudo-label generation module, configured to generate, for each training round of the first round to the nth round , a pseudo-label for each unlabeled sample input the pseudo-label generator to obtain an output , calculate a threshold value thereof , and generate a pseudo-label according to the following rules : for each class k, when , let , otherwise , so as to obtain a pseudo-labeled text dataset ;​ The joint training module is configured to perform semi-supervised training of the feature processing network and the dual-branch network in each training round of the first to the nth round. The joint training module is configured to perform semi-supervised training of the feature processing network and the dual-branch network in each training round of the first to the nth round. The joint training module is configured to perform semi-supervised training of the feature processing network and the dual-branch network in each training round of the first to the nth round. The joint training module is configured to perform semi-supervised training of the feature processing network and the dual-branch network in each training round of the first to the nth round. The joint training module is configured to perform semi-supervised training of the feature processing network and the dual-branch network in each training round of the first to the nth round. The joint training module is configured to perform semi-supervised training of the feature processing network and the dual-branch network in each training round of the first to the nth round. The joint training module is configured to perform semi-supervised training of the feature processing network and the dual-branch network in each training round of the first to the nth round. The joint training module is configured to perform semi-supervised training of the feature processing network and the dual-branch network in each training round of the first to the nth round. The joint training module is configured to perform semi-supervised training of the feature processing network and A termination and output module for repeatedly performing the training process of the pseudo-label generation module and the joint training module until an iteration termination condition is reached, obtaining a final trained model, and using the final model for a multi-label classification task of unlabeled text data.

7. The system of claim 6, wherein, The termination and output module supports batch text inference, classification result visualization, and online deployment interface calling.

8. An electronic device, comprising: It comprises: One or more processors; Memory; And one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the programs are executed by the processor to implement the steps of the semi-supervised text data multi-label classification method according to any one of claims 1 to 5.

9. A computer storage medium having stored thereon a computer program, characterized in that The computer program is executed by the processor to implement the steps of the semi-supervised text data multi-label classification method according to any one of claims 1 to 5.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the semi-supervised text data multi-label classification method according to any one of claims 1 to 5.

Images