Text classification method based on active semi-supervised learning and heterogeneous graph attention network

Abstract

The application discloses a text classification method based on active semi-supervised learning and a heterogeneous graph attention network, and the method comprises the following steps: training a heterogeneous graph attention network model based on labeled samples; performing multiple inference predictions on unlabeled samples based on the trained heterogeneous graph attention network model; marking hard label samples and soft label samples through expert judgment; performing multiple inference predictions on the unlabeled samples that are not marked by experts, and calculating a second prediction mean and a second prediction variance; adaptively calculating a mean threshold and a variance threshold of a current stage model to pseudo-label the unlabeled samples that are not marked by experts; mixing all samples, and further training the trained heterogeneous graph attention network model through the mixed samples; and repeating the above process until a set training termination condition is met. The application organically integrates active learning based on information gain and semi-supervised learning based on pseudo-labels, and improves the quality of training data and the generalization ability of the model.

Description

Technical Field

[0001] This invention belongs to the field of natural language processing technology, specifically relating to a text classification method based on active semi-supervised learning and heterogeneous graph attention networks. Background Technology

[0002] In recent years, methods based on fine-tuning large-scale pre-trained language models have been widely applied in text classification, named entity recognition, question answering, and other fields. Furthermore, these methods achieve state-of-the-art performance when large-scale, centralized training data is available. However, in some real-world scenarios, labeled data is scarce, and manual labeling is extremely time-consuming.

[0003] Active learning and semi-supervised learning are two commonly used techniques to address label scarcity. Active learning is an efficient labeling method that builds a high-performance classifier by selecting and labeling a small number of unlabeled samples containing rich information. Semi-supervised learning can fully utilize unlabeled data to learn more knowledge, enhancing models trained on labeled data and improving their generalization ability. Currently, text classification methods based on active learning have significantly reduced labeling costs by selecting a small number of the most valuable nodes. However, these active learning methods assume that hard labels (exact labels, i.e., labels for the exact class of a given node) can always be provided by experts. But in some domains or scenarios with too many categories, many categories are beyond the domain knowledge of experts, making it difficult for experts to provide exact class labels. Therefore, reducing the difficulty of expert labeling remains a challenge in the classification field.

[0004] Currently, semi-supervised learning methods based on uncertainty awareness for pseudo-labeling have achieved good results in the image processing domain. These methods alleviate the problem of poor neural network calibration leading to numerous incorrect pseudo-label predictions and a significant performance degradation by assigning positive and negative pseudo-labels to unlabeled data with high prediction mean and low uncertainty. However, these thresholds are fixed, requiring continuous trial and error to find the optimal threshold for different datasets. Setting the threshold too high can make it difficult for the model to select positive and negative pseudo-label samples in the early stages, while setting it too low can lead to the model selecting a large number of noisy positive and negative pseudo-label samples later. Therefore, in semi-supervised learning, how to adaptively generate confidence thresholds based on model performance is a pressing issue that needs to be addressed.

[0005] Although the paper "Heterogeneous Graph Attention Networks for Semi-supervised ShortText Classification" discloses a semi-supervised short text classification method using heterogeneous graph attention networks, it still cannot effectively solve the above problems.

[0006] In summary, how to economically and effectively utilize limited labeled data and massive amounts of unlabeled data to train models efficiently remains a challenge in the field of text classification. Summary of the Invention

[0007] In view of this, the present invention proposes a text classification method based on active semi-supervised learning and heterogeneous graph attention network to solve the problem of inefficient training of models based on limited labeled data.

[0008] In a first aspect, this invention discloses a text classification method based on active semi-supervised learning and heterogeneous graph attention networks, the method comprising:

[0009] S1. Construct a heterogeneous graph attention network model and train the heterogeneous graph attention network model based on labeled samples;

[0010] S2. Based on the trained heterogeneous graph attention network model, perform multiple inference predictions on unlabeled samples and calculate the mean of the first prediction.

[0011] S3. Calculate the information gain of all nodes in the heterogeneous graph attention network model using the first prediction mean. Select a batch of unlabeled samples with information gain greater than a preset threshold as the first subsample for expert judgment. The remaining unlabeled samples are the second subsample. Mark the correctly predicted samples in the first subsample as hard-labeled samples and the incorrectly predicted samples as soft-labeled samples.

[0012] S4. Based on the trained heterogeneous graph attention network model, perform multiple inference predictions on the unlabeled samples in the second subsample, and calculate the second prediction mean and the second prediction variance.

[0013] S5. Adaptively calculate the mean threshold and variance threshold of the model at the current stage based on the second predicted mean and the second predicted variance.

[0014] S6. Based on the second predicted mean, the second predicted variance, and the corresponding mean threshold and variance threshold, assign pseudo-labels to the unlabeled samples in the second sub-sample, and select positive and negative sample pseudo-labels.

[0015] S7. Mix labeled samples, hard-labeled samples, soft-labeled samples, and positive and negative pseudo-labeled samples to obtain mixed samples. Use the mixed samples to further train the trained heterogeneous graph attention network model.

[0016] S8. Repeat steps S2 to S7 until the set training termination conditions are met.

[0017] Based on the above technical solutions, preferably, the heterogeneous graph attention network model is defined as G = (N, R, f, g);

[0018] Where N represents a node and R represents a relation;

[0019] f:N→A represents the node type mapping relationship, that is, f(n)∈A means that the type of node n belongs to set A, and the node type set A includes topic type, entity type and text type;

[0020] g:R→B represents a relation type mapping relation, that is, g(r)∈B indicates that the type of relation r belongs to set B. The relation type set B includes the relationship between entities, the relationship between entities and text, and the relationship between text and topic.

[0021] Based on the above technical solutions, preferably, the goal of the heterogeneous graph attention network model is to aggregate topic nodes and entity nodes around text nodes, and to introduce a two-layer attention mechanism at the type level and node level to reduce noise in the text information fusion process.

[0022] Based on the above technical solution, preferably, the first prediction mean, the second prediction mean, and the second prediction variance are obtained by performing multiple forward propagations on the same input sample through a heterogeneous graph attention network model when dropout is always active during the prediction phase, thereby obtaining prediction values ​​of different network structures, and calculating the average and statistical variance of the multiple prediction values, wherein the average value is used as the corresponding prediction mean, and the statistical variance is used as the corresponding prediction variance.

[0023] Based on the above technical solutions, preferably, the step of calculating the information gain of all nodes in the heterogeneous graph attention network model using the first predicted mean specifically includes:

[0024] Define the information gain function:

[0025]

[0026] Where H represents the entropy function. P(v) represents the entropy before the expert's judgment. i -) indicates the probability that the model prediction is incorrect; After expert judgment, the model evaluates sample v. i Entropy in the case of prediction errors.

[0027] Based on the above technical solution, preferably, the adaptive calculation of the mean threshold and variance threshold of the model at the current stage according to the second predicted mean and the second predicted variance specifically includes:

[0028] The mean threshold and variance threshold are calculated adaptively using either positive or negative samples.

[0029] In the process of adaptively calculating the mean threshold and variance threshold using either positive or negative samples, the adaptive threshold at the t-th epoch of network training is defined as follows:

[0030]

[0031] T max This represents the total number of rounds of adaptive iterative learning required, where coefficients α and β are predefined values, and N represents the total number of rounds. A This is the number of samples that have already been labeled by experts, and K represents the budget that experts need to label.

[0032] It is a counting function used to estimate the learning state of the network in the t-th round, defined as follows:

[0033]

[0034] Where Max(P(v) i )) represents the maximum predicted score, N u This indicates the number of unlabeled samples. This means that when Max(P(v) is satisfied... i The value is 1 when the condition is greater than α+β, otherwise it is 0.

[0035] Based on the above technical solutions, preferably, the step of assigning pseudo-labels to unlabeled samples in the second sub-sample based on the second predicted mean, the second predicted variance, and the corresponding mean threshold and variance threshold, and selecting positive and negative sample pseudo-labels specifically includes:

[0036] Unlabeled samples that meet the following conditions will be selected as positive or false labels:

[0037] The second predicted mean corresponding to the largest category is greater than or equal to the mean threshold of the positive sample predictions, and the second predicted variance corresponding to the largest category is less than or equal to the variance threshold of the positive sample predictions.

[0038] Assuming the negative prediction mean threshold calculated by S160 is 0.1 and the prediction variance threshold is 0.005, unlabeled samples that meet the following conditions will be selected as negative pseudo-labels:

[0039] The second predicted mean corresponding to the smallest class is less than or equal to the mean threshold of the negative sample predictions, and the second predicted variance corresponding to the smallest class is less than or equal to the variance threshold of the negative sample predictions.

[0040] A second aspect of the present invention discloses a text classification system based on active semi-supervised learning and heterogeneous graph attention networks, characterized in that the system comprises:

[0041] The first training module is used to construct a heterogeneous graph attention network model and train the heterogeneous graph attention network model based on labeled samples.

[0042] First calculation module: used to perform multiple inference predictions on unlabeled samples based on the trained heterogeneous graph attention network model, and calculate the first prediction mean;

[0043] The expert judgment module is used to calculate the information gain of all nodes in the heterogeneous graph attention network model using the first prediction mean, select a batch of unlabeled samples with information gain greater than a preset threshold as the first subsample for expert judgment, and the remaining unlabeled samples as the second subsample; the samples that are correctly predicted in the first subsample are marked as hard-labeled samples, and the samples that are incorrectly predicted are marked as soft-labeled samples.

[0044] The second calculation module is used to perform multiple inference predictions on the unlabeled samples in the second subsample based on the trained heterogeneous graph attention network model, and to calculate the second prediction mean and the second prediction variance; and to adaptively calculate the mean threshold and variance threshold of the model at the current stage based on the second prediction mean and the second prediction variance.

[0045] The second training module is used to assign pseudo-labels to unlabeled samples in the second sub-sample based on the second predicted mean, the second predicted variance, and the corresponding mean threshold and variance threshold, and to select positive and negative pseudo-labels; it mixes labeled samples, hard-labeled samples, soft-labeled samples, and positive and negative pseudo-labeled samples to obtain mixed samples, and uses the mixed samples to further train the trained heterogeneous graph attention network model.

[0046] The loop calculation module is used to repeat the first calculation module, the expert judgment module, the second calculation module, and the second training module until the set training termination condition is met.

[0047] A third aspect of the present invention discloses an electronic device comprising: at least one processor, at least one memory, a communication interface, and a bus;

[0048] The processor, memory, and communication interface communicate with each other through the bus.

[0049] The memory stores program instructions that can be executed by the processor, which invokes the program instructions to implement the method as described in the first aspect of the present invention.

[0050] In a fourth aspect, the present invention discloses a computer-readable storage medium storing computer instructions that cause a computer to perform the method described in the first aspect of the present invention.

[0051] The present invention has the following advantages over the prior art:

[0052] 1) This invention smoothly integrates active learning based on information gain with semi-supervised learning based on pseudo-labels, thereby further improving the quality of training data and the generalization ability of the model. It also designs adaptive confidence mean threshold and variance threshold to solve the problems that the model has difficulty selecting positive and negative pseudo-label samples in the early stage due to the fixed threshold setting being too large, or the model selects a large number of noisy positive and negative pseudo-label samples in the later stage due to the threshold setting being too low.

[0053] 2) Active learning is transformed into a binary judgment problem, where a domain expert only judges the correctness of the predicted label, rather than identifying the exact class in a multi-class problem. The binary labeling strategy greatly reduces the difficulty of expert annotation. This invention can accurately classify common texts with low data sources, especially in scenarios with many classification categories, which can effectively reduce the difficulty of expert annotation.

[0054] 3) To prevent information loss, this invention retains both correct and incorrect model predictions as judged by experts. Correct predictions are used as hard labels, and incorrect predictions are used as soft labels. Finally, labeled samples, hard-labeled samples, soft-labeled samples, and positive and negative pseudo-labeled samples are mixed. The mixed samples are used to further train the trained heterogeneous graph attention network model and apply it to text classification. The trained model can integrate the topic features of the text and additional entity features. The pre-trained model is used to embed the sentence context of the text and integrates the contextual information features of a large amount of unlabeled data, which significantly improves the accuracy and generalization ability of the model classification. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0056] Figure 1 A schematic diagram illustrating the principle of a heterogeneous graph attention network model based on active semi-supervised learning, provided in one embodiment of the present invention;

[0057] Figure 2 This is a schematic diagram comparing a binary labeling strategy with a traditional strategy, provided as an embodiment of the present invention. Detailed Implementation

[0058] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0059] Please see Figure 1 A text classification method based on active semi-supervised learning and heterogeneous graph attention networks, the method comprising:

[0060] S1. Construct a heterogeneous graph attention network model and train the heterogeneous graph attention network model based on labeled samples.

[0061] Specifically, a heterogeneous information network is constructed by obtaining a small number of labeled samples and a large number of unlabeled samples. Based on the heterogeneous information network, a two-layer attention mechanism at the type level and node level is introduced to obtain a heterogeneous graph attention network model. This heterogeneous graph attention network model is then trained using a small number of labeled samples.

[0062] Figure 1 This is a schematic diagram of the principle of the constructed heterogeneous graph attention network model based on active semi-supervised learning, which mainly consists of two modules: the heterogeneous graph attention network model and the active semi-supervised strategy.

[0063] The heterogeneous graph attention network model is defined as: G = (N, R, f, g), where N represents nodes and R represents relations;

[0064] f:N→A represents the node type mapping relationship, that is, f(n)∈A means that the type of node n belongs to set A, and the node type set A includes topic type, entity type and text type;

[0065] g:R→B represents a relation type mapping relation, that is, g(r)∈B indicates that the type of relation r belongs to set B. The relation type set B includes the relationship between entities, the relationship between entities and text, and the relationship between text and topic.

[0066] The goal of heterogeneous graph attention networks is to aggregate topic nodes and entity nodes surrounding segment text nodes. Since heterogeneous graphs contain different types of nodes and different types of relationships, the introduced type-level and node-level two-layer attention mechanism can reduce noise during the text information fusion process. The heterogeneous information network model of this invention can effectively combine the topic distribution of text with entities embedded in the knowledge base, effectively alleviating the semantic sparsity problem of short texts.

[0067] S2. Based on the trained heterogeneous graph attention network model, perform multiple inference predictions on unlabeled samples and calculate the mean of the first prediction.

[0068] Specifically, when dropout is always active during the prediction phase (MC-Dropout is enabled), the heterogeneous graph attention network model performs multiple forward propagations on the same input sample to obtain prediction values ​​from different network structures. In this way, multiple inference predictions are performed on unlabeled samples, and the mean of multiple inference predictions is calculated as the first prediction mean. At the same time, the first prediction variance can also be calculated.

[0069] S3. Calculate the information gain of all nodes in the heterogeneous graph attention network model using the first prediction mean. Select a batch of unlabeled samples with information gain greater than a preset threshold as the first subsample for expert judgment, and the remaining unlabeled samples as the second subsample. Mark the correctly predicted samples in the first subsample as hard-labeled samples and the incorrectly predicted samples as soft-labeled samples.

[0070] Specifically, information gain refers to the expected reduction in entropy, i.e., the amount of entropy reduction in sample v. i Information gain Where H represents the entropy function. and Let represent the entropy before and after the expert judgment, respectively. The expert judgment has only two possible outcomes: either the expert judges the model's prediction is correct, or the expert judges the model's prediction is incorrect. Therefore, information gain is simply the weighted expectation of these two outcomes, without needing to know in advance whether the expert made the judgment. The information gain function is defined as follows:

[0071]

[0072] Where H represents the entropy function. and Let P(v) represent the entropy before and after the expert judgment, respectively. i +) indicates that the model applies to sample v. i The probability of making a correct prediction, P(v i -) indicates that the model applies to sample v. i The probability of prediction error; This represents the information gain assuming the expert judges the model's prediction to be correct. This represents the information gain when experts judge the model to be wrong.

[0073] When the model's prediction is correct, the entropy drops to 0 after expert judgment, i.e. Therefore, the information gain function simplifies to:

[0074]

[0075] in, These represent the entropy when the model predicts correctly based on expert judgment and the entropy when the model predicts incorrectly, respectively.

[0076] The proof is as follows:

[0077]

[0078] This invention calculates the information gain of each unlabeled sample in step S2, selects unlabeled samples with information gain greater than a preset threshold to form a first subsample for expert judgment, and uses the remaining unlabeled samples as a second subsample. After expert judgment, samples whose model prediction is correct are hard-labeled samples, and their sample labels can be represented by one-hot vectors. Samples whose model prediction is incorrect are soft-labeled samples, and their sample labels can be represented by vector distributions, such as vector [0.5, 0.25, 0.25]. If the model prediction is correct, it will be judged as a hard label [1, 0, 0], and if the model prediction is incorrect, it will be judged as a soft label [0, 0.5, 0.5] or [0, 0.6, 0.4].

[0079] The following presents two cases for calculating the information gain of a node:

[0080] Scenario 1: If the heterogeneous graph attention network model predicts correctly, the information will be converted from uncertain soft labels to one-hot labels. Suppose the model predicts [0.5, 0.3, 0.2], and after expert judgment, the label will become [1, 0, 0], with the corresponding information gain H([0.5, 0.3, 0.2]) - H([1, 0, 0]).

[0081] Scenario 2: If the heterogeneous graph attention network model makes an incorrect prediction, the information will be transformed from an uncertain soft label to another soft label, and its uncertainty will decrease. Suppose the model predicts [0.5, 0.3, 0.2], and after expert judgment, the label will become [0, 0.6, 0.4], with a corresponding information gain H([0.5, 0.3, 0.2]) - H([0, 0.6, 0.4]).

[0082] Combining the above two scenarios, the total information gain IG of the corresponding node is 0.5*(H([0.5,0.3,0.2])-H([1,0,0]))+0.6*(H([0.5,0.3,0.2])-H([0,0.5,0.5])).

[0083] This invention uses a binary labeling strategy to label the first subsample, transforming active learning into a binary judgment problem. That is, a domain expert only judges the correctness of the predicted label, rather than identifying the exact class in a multi-class problem. Figure 2This diagram illustrates a comparison between a binary labeling strategy and a traditional strategy, as provided in one embodiment of the present invention. The present invention significantly reduces the difficulty of expert annotation of data through a binary labeling strategy, enabling accurate classification of common texts even with limited data sources. This is particularly beneficial in scenarios with a large number of classification categories, effectively reducing the difficulty of expert annotation.

[0084] When making expert judgments, samples can be selected in batches. The number of samples selected in each batch should be set to a small number, such as 5 * the number of categories per batch. If the batch size is too large, redundant samples may be selected, wasting manpower labeling costs.

[0085] S4. Based on the trained heterogeneous graph attention network model, perform multiple inference predictions on the unlabeled samples in the second subsample, and calculate the second prediction mean and the second prediction variance.

[0086] Similar to step S2, the heterogeneous graph attention network model trained in step S1 enables MC-Dropout during the prediction phase and performs multiple inference predictions on the unlabeled samples in the second subsample. The multiple prediction values ​​are averaged and the statistical variance is calculated. The average value is used as the corresponding second prediction mean, and the statistical variance is used as the corresponding second prediction variance.

[0087] S5. Based on the second predicted mean and the second predicted variance, adaptively calculate the mean threshold and variance threshold of the model at the current stage.

[0088] In practice, the mean threshold and variance threshold can be calculated adaptively using either positive or negative samples.

[0089] Taking the adaptive prediction mean threshold for positive samples as an example, the adaptive threshold is defined as follows when the network is trained to the t-th round:

[0090]

[0091] T max This represents the total number of rounds of adaptive iterative learning required, where coefficients α and β are predefined values, and N represents the total number of rounds. A This is the number of samples that have already been labeled by experts, and K represents the budget that experts need to label.

[0092] It is a counting function used to estimate the learning state of the network in the t-th round, defined as follows:

[0093]

[0094] Where Max(P(v) i )) represents the maximum predicted score, N u This indicates the number of unlabeled samples. This means that when Max(P(v) is satisfied... i Its value is 1 when the condition is greater than α+β, otherwise its value is 0.

[0095] The learning effectiveness of an active semi-supervised strategy can be reflected by the number of samples whose predicted mean exceeds a certain threshold. Therefore, the adaptive calculation of the mean and variance thresholds can be dynamically adjusted based on the current learning state and the amount of active learning information selected, encouraging better utilization of unlabeled data. The principles and formulas for adaptively predicting the mean threshold with negative samples and the variance threshold for positive and negative samples are the same as those for adaptively predicting the mean threshold with positive samples, only the coefficient values ​​α and β are set differently, which will not be elaborated upon here.

[0096] The adaptive calculation of the prediction mean threshold and prediction variance threshold in this invention can solve the problems of the model having difficulty selecting positive and negative pseudo-label samples in the early stage due to the threshold setting being too large, or the model selecting a large number of noisy positive and negative pseudo-label samples in the later stage due to the threshold setting being too low, thereby improving the accuracy of the model prediction.

[0097] S6. Based on the second predicted mean, the second predicted variance, and the corresponding mean threshold and variance threshold, pseudo-labels are assigned to the unlabeled samples in the second subsample, and positive and negative pseudo-labels are selected.

[0098] Specifically, assuming that the mean threshold calculated in step S5 based on positive sample prediction is 0.7 and the variance threshold is 0.05, then unlabeled samples that meet the following conditions will be selected as positive pseudo-labels:

[0099] The mean of the second prediction for the largest category is ≥0.7, and the variance of the second prediction for the largest category is ≤0.05.

[0100] Assuming that the mean threshold calculated in step S5 based on negative sample prediction is 0.1 and the variance threshold is 0.005, then unlabeled samples that meet the following conditions will be selected as negative pseudo-labels:

[0101] The mean of the second prediction for the smallest category is ≤0.1 and the variance of the second prediction for the smallest category is ≤0.005.

[0102] S7. Mix labeled samples, hard-labeled samples, soft-labeled samples, and positive and negative pseudo-labeled samples to obtain mixed samples. Use the mixed samples to further train the trained heterogeneous graph attention network model.

[0103] When mixing samples, the proportion of samples and the proportion of each class should be balanced. Balancing the sample proportion aims to balance the number of hard-labeled, soft-labeled, and positive / negative pseudo-labeled samples, preventing the active learning samples from becoming ineffective due to an excessive number of positive / negative labeled samples. Balancing the class proportion is to alleviate the problem of model overfitting caused by an excessive number of positive / negative label or hard-labeled samples from one class.

[0104] When further training the heterogeneous graph attention network model using mixed samples, the loss value is calculated using the cross-entropy loss function for labeled samples, hard-labeled samples, and positive pseudo-labeled samples; the loss value is calculated using the negative sample cross-entropy for negative pseudo-labeled samples; and the loss value is calculated using KL divergence for soft labels.

[0105] S8. Repeat steps S2 to S7 until the set training termination conditions are met.

[0106] This invention transforms active learning into a binary judgment problem, where a domain expert (oracle) judges only the correctness of the predicted label, rather than identifying the exact class (a multi-class problem), significantly reducing the difficulty of expert data annotation. To prevent information loss, we retain both correctly and incorrectly predicted samples. Correctly predicted samples are treated as hard labels, and the loss is calculated using normal cross-entropy. Incorrectly predicted samples are treated as soft labels, and the loss is calculated using KL divergence, improving the model's generalization ability. This invention can accurately classify common text even with limited data sources, especially in scenarios with a large number of categories, effectively reducing the difficulty of expert annotation.

[0107] Furthermore, this invention organically integrates information gain-based active learning with pseudo-label-based semi-supervised learning under the MC-Dropout mechanism, thereby further improving the quality of training data and the generalization ability of the model. It also designs adaptive confidence mean threshold and variance threshold calculation to solve the problems that the fixed threshold setting is too large, which makes it difficult for the model to select positive and negative pseudo-label samples in the early stage, or the threshold setting is too low, which causes the model to select a large number of noisy positive and negative pseudo-label samples in the later stage.

[0108] This invention takes text classification as an example and further applies a heterogeneous information network structure to fuse the topic features and additional entity features of the text. It uses a pre-trained model to embed sentence context into the text, incorporating contextual information features from a large amount of unlabeled data, significantly improving the model's classification accuracy and generalization ability. This invention is the first to apply active semi-supervised learning methods to heterogeneous graph attention networks, and has broad application prospects and significant positive implications.

[0109] Corresponding to the above method embodiments, this invention also proposes a text classification system based on active semi-supervised learning and heterogeneous graph attention networks, the system comprising:

[0110] The first training module is used to construct a heterogeneous graph attention network model and train the heterogeneous graph attention network model based on labeled samples.

[0111] First calculation module: used to perform multiple inference predictions on unlabeled samples based on the trained heterogeneous graph attention network model, and calculate the first prediction mean;

[0112] The expert judgment module is used to calculate the information gain of all nodes in the heterogeneous graph attention network model using the first prediction mean, select a batch of unlabeled samples with information gain greater than a preset threshold as the first subsample for expert judgment, and the remaining unlabeled samples as the second subsample; the samples that are correctly predicted in the first subsample are marked as hard-labeled samples, and the samples that are incorrectly predicted are marked as soft-labeled samples.

[0113] The second calculation module is used to perform multiple inference predictions on the unlabeled samples in the second subsample based on the trained heterogeneous graph attention network model, and to calculate the second prediction mean and the second prediction variance; and to adaptively calculate the mean threshold and variance threshold of the model at the current stage based on the second prediction mean and the second prediction variance.

[0114] The second training module is used to assign pseudo-labels to unlabeled samples in the second sub-sample based on the second predicted mean, the second predicted variance, and the corresponding mean threshold and variance threshold, and to select positive and negative pseudo-labels; it mixes labeled samples, hard-labeled samples, soft-labeled samples, and positive and negative pseudo-labeled samples to obtain mixed samples, and uses the mixed samples to further train the trained heterogeneous graph attention network model.

[0115] The loop calculation module is used to repeat the first calculation module, the expert judgment module, the second calculation module, and the second training module until the set training termination condition is met.

[0116] The above system embodiments and method embodiments are one-to-one correspondences. For a brief description of the system embodiments, please refer to the method embodiments.

[0117] The present invention also discloses an electronic device, comprising: at least one processor, at least one memory, a communication interface, and a bus; wherein the processor, memory, and communication interface communicate with each other through the bus; the memory stores program instructions executable by the processor, and the processor calls the program instructions to implement the aforementioned method of the present invention.

[0118] The present invention also discloses a computer-readable storage medium that stores computer instructions, which cause the computer to implement all or part of the steps of the method described in the embodiments of the present invention. The storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0119] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, meaning they can be distributed across multiple network units. Those skilled in the art can select some or all of the modules to achieve the purpose of this embodiment without any inventive effort, based on actual needs.

[0120] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A text classification method based on active semi-supervised learning and heterogeneous graph attention networks, characterized in that, The method includes: S1. Construct a heterogeneous graph attention network model and train the heterogeneous graph attention network model based on labeled samples; S2. Based on the trained heterogeneous graph attention network model, perform multiple inference predictions on unlabeled samples and calculate the mean of the first prediction. S3. Calculate the information gain of all nodes in the heterogeneous graph attention network model using the first prediction mean. Select a batch of unlabeled samples with information gain greater than a preset threshold as the first subsample for expert judgment. The remaining unlabeled samples are the second subsample. Mark the correctly predicted samples in the first subsample as hard-labeled samples and the incorrectly predicted samples as soft-labeled samples. S4. Based on the trained heterogeneous graph attention network model, perform multiple inference predictions on the unlabeled samples in the second subsample, and calculate the second prediction mean and the second prediction variance. S5. Adaptively calculate the mean threshold and variance threshold of the model at the current stage based on the second predicted mean and the second predicted variance. S6. Based on the second predicted mean, the second predicted variance, and the corresponding mean threshold and variance threshold, assign pseudo-labels to the unlabeled samples in the second sub-sample, and select positive and negative sample pseudo-labels. S7. Mix labeled samples, hard-labeled samples, soft-labeled samples, and positive and negative pseudo-labeled samples to obtain mixed samples. Use the mixed samples to further train the trained heterogeneous graph attention network model. S8. Repeat steps S2 to S7 until the set training termination conditions are met. The heterogeneous graph attention network model is defined as G = (N, R, f, g); Where N represents a node and R represents a relation; f:N→A represents the node type mapping relationship, that is, f(n)∈A means that the type of node n belongs to set A, and the node type set A includes topic type, entity type and text type; g:R→B represents a relation type mapping relation, that is, g(r)∈B indicates that the type of relation r belongs to set B. The relation type set B includes the relationship between entities, the relationship between entities and text, and the relationship between text and topic.

2. The text classification method based on active semi-supervised learning and heterogeneous graph attention network according to claim 1, characterized in that, The goal of the heterogeneous graph attention network model is to aggregate topic nodes and entity nodes around text nodes, and to introduce a two-layer attention mechanism at the type level and node level to reduce noise in the text information fusion process.

3. The text classification method based on active semi-supervised learning and heterogeneous graph attention network according to claim 1, characterized in that, The first predicted mean, the second predicted mean, and the second predicted variance are obtained by performing multiple forward propagations on the same input sample using a heterogeneous graph attention network model while dropout remains active during the prediction phase. This allows for obtaining predicted values ​​from different network structures. The average and statistical variance of these multiple predicted values ​​are then calculated, with the average value serving as the corresponding predicted mean and the statistical variance serving as the corresponding predicted variance.

4. The text classification method based on active semi-supervised learning and heterogeneous graph attention network according to claim 3, characterized in that, The step of calculating the information gain of all nodes in the heterogeneous graph attention network model using the first predicted mean specifically includes: Define the information gain function: Where H represents the entropy function. P(v) represents the entropy before the expert's judgment. i -) indicates the probability that the model prediction is incorrect; After expert judgment, the model evaluates the sample v. i Entropy in the case of prediction errors.

5. The text classification method based on active semi-supervised learning and heterogeneous graph attention network according to claim 1, characterized in that, The adaptive calculation of the mean threshold and variance threshold of the model at the current stage based on the second predicted mean and the second predicted variance specifically includes: The mean threshold and variance threshold are calculated adaptively using either positive or negative samples. In the process of adaptively calculating the mean threshold and variance threshold using either positive or negative samples, the adaptive threshold at the t-th epoch of network training is defined as follows: T max This represents the total number of rounds of adaptive iterative learning required, where coefficients α and β are predefined values, and N... A This is the number of samples that have already been labeled by experts, and K represents the budget that experts need to label. It is a counting function used to estimate the learning state of the network in the t-th round, defined as follows: Where Max(P(v) i )) represents the maximum predicted score, N u This indicates the number of unlabeled samples. This means that when Max(P(v) is satisfied... i The value is 1 when the condition is greater than α+β, otherwise it is 0.

6. The text classification method based on active semi-supervised learning and heterogeneous graph attention network according to claim 5, characterized in that, The process of assigning pseudo-labels to unlabeled samples in the second sub-sample based on the second predicted mean, the second predicted variance, and the corresponding mean threshold and variance threshold, and selecting positive and negative pseudo-labels specifically includes: Unlabeled samples that meet the following conditions will be selected as positive or false labels: The second predicted mean corresponding to the largest category is greater than or equal to the mean threshold of the positive sample predictions, and the second predicted variance corresponding to the largest category is less than or equal to the variance threshold of the positive sample predictions. Unlabeled samples that meet the following conditions will be selected as negative pseudo-labels: The second predicted mean corresponding to the smallest class is less than or equal to the mean threshold of the negative sample predictions, and the second predicted variance corresponding to the smallest class is less than or equal to the variance threshold of the negative sample predictions.

7. A text classification system based on active semi-supervised learning and heterogeneous graph attention networks, characterized in that, The system is based on the method described in any one of claims 1 to 6, and comprises: The first training module is used to construct a heterogeneous graph attention network model and train the heterogeneous graph attention network model based on labeled samples. First calculation module: used to perform multiple inference predictions on unlabeled samples based on the trained heterogeneous graph attention network model, and calculate the first prediction mean; The expert judgment module is used to calculate the information gain of all nodes in the heterogeneous graph attention network model using the first prediction mean, select a batch of unlabeled samples with information gain greater than a preset threshold as the first subsample for expert judgment, and the remaining unlabeled samples as the second subsample; the samples that are correctly predicted in the first subsample are marked as hard-labeled samples, and the samples that are incorrectly predicted are marked as soft-labeled samples. The second calculation module is used to perform multiple inference predictions on the unlabeled samples in the second subsample based on the trained heterogeneous graph attention network model, and to calculate the second prediction mean and the second prediction variance; and to adaptively calculate the mean threshold and variance threshold of the model at the current stage based on the second prediction mean and the second prediction variance. The second training module is used to assign pseudo-labels to unlabeled samples in the second sub-sample based on the second predicted mean, the second predicted variance, and the corresponding mean threshold and variance threshold, and to select positive and negative pseudo-labels; it mixes labeled samples, hard-labeled samples, soft-labeled samples, and positive and negative pseudo-labeled samples to obtain mixed samples, and uses the mixed samples to further train the trained heterogeneous graph attention network model. The loop calculation module is used to repeat the first calculation module, the expert judgment module, the second calculation module, and the second training module until the set training termination condition is met.

8. An electronic device, characterized in that, include: At least one processor, at least one memory, a communication interface, and a bus; The processor, memory, and communication interface communicate with each other through the bus. The memory stores program instructions that can be executed by the processor, which invokes the program instructions to implement the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause the computer to perform the method as described in any one of claims 1 to 6.

Images