Graph contrast learning semi-supervised text classification method based on clustering improved negative sampling

Abstract

The invention relates to a graph contrast learning semi-supervised text classification method based on clustering improved negative sampling. According to the graph comparative learning semi-supervised text classification method based on clustering improved negative sampling, the complex relation between the text nodes and the word nodes is represented by constructing the heterogeneous text graph network model, and high-quality graph input is provided for subsequent comparative learning; diversified enhanced views are generated through a random edge discarding strategy, richer context information is obtained in the subsequent comparative learning process, and the generalization ability of node embedding is improved; performing text representation learning on the enhanced view through a clustering algorithm to ensure semantic diversity and high quality of a negative sample set so as to relieve a deviation problem in a negative sample selection process and avoid an intra-class separation phenomenon caused by wrong selection of negative samples; and through a joint optimization strategy of comparison loss and cross entropy loss, semantic consistency of text embedding is improved, and robustness of classification performance is ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of text classification, and in particular to a graph contrast learning semi-supervised text classification method based on clustering improved negative sampling. Background Art

[0002] Text classification is a core task in the field of Natural Language Processing (NLP), which aims to automatically classify text data according to predefined categories. The performance of text classification models mainly depends on their ability to learn text feature representations. In traditional methods, feature extraction mostly relies on hand-designed features or shallow learning models. However, in recent years, the rise of deep learning methods has brought breakthrough progress in text classification. For example, models based on Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN) can automatically extract high-dimensional semantic features from text. In particular, the emergence of the Transformer architecture and the application of its pre-trained models (such as BERT and RoBERTa) have enabled text classification models to demonstrate excellent performance in a wide range of tasks. These models have greatly improved the accuracy and generalization ability of text classification through a large amount of unsupervised pre-training and small-scale supervised fine-tuning.

[0003] Among the existing deep learning methods, graph-based text classification methods (GTC) have gradually attracted attention. Compared with sequence models, this type of method can effectively capture the complex semantic associations and structural information between documents by modeling text data as a graph structure. For example, text classification methods based on graph neural networks (GNNs) construct a global graph representation of the corpus, which not only analyzes the content of a single document, but also integrates the information propagated by adjacent nodes, thereby achieving more comprehensive text classification.

[0004] Although graph-based text classification methods have significant advantages in performance, they still face challenges in practical applications. Traditional methods often rely only on labeled nodes (i.e., training sets) and ignore the potential information value in unlabeled nodes. In recent years, Graph Contrastive Learning (GCL) has been introduced into the field of text classification, aiming to make better use of unlabeled data. GCL can enhance the generalization ability of the model without relying on labeled data by optimizing the contrast loss function, significantly expanding the range of data available for training.

[0005] However, GCL-based text classification methods still have significant problems in the negative sample selection process. Existing methods are usually based on a variant of the InfoNCE loss function, which considers the same node between views as a positive sample, while other nodes within or between views are considered negative samples. Since the selection of negative samples depends only on the structural information of the graph, nodes with the same label may be mistakenly classified as negative samples. This negative sample bias artificially distances nodes of the same type in the embedding space, resulting in reduced intra-class consistency of the embedded representation and even causing over-clustering problems (i.e., instances of the same category are divided into different subclasses). This bias destroys the semantic distribution structure of the embedding space and significantly reduces the model's ability to recognize the true category. Summary of the invention

[0006] Based on this, it is necessary to provide a graph contrast learning semi-supervised text classification method based on clustering improved negative sampling, which can effectively solve the bias problem in the negative sample selection process and improve the quality of text embedding representation, thereby enhancing the accuracy and robustness of text classification.

[0007] A graph contrast learning semi-supervised text classification method based on clustering improved negative sampling, comprising the following steps:

[0008] Acquire text data, and construct a heterogeneous text graph network model based on the text data; the text data includes document nodes and word nodes;

[0009] Calculating edge weights between different nodes in the text data according to the heterogeneous text graph network model to construct a weighted graph model; the weighted graph model includes a node feature matrix and a weighted adjacency matrix;

[0010] Using a random edge dropping strategy to perform random masking on the weighted adjacency matrix to generate multiple sets of enhanced views;

[0011] A clustering algorithm is used to perform text representation learning on multiple groups of enhanced views to select nodes across clusters as negative samples, and a self-correction mechanism is combined to supplement distant nodes within the cluster as negative samples to obtain a negative sample set;

[0012] Using a joint optimization strategy of contrast loss and cross entropy loss to perform node embedding representation learning and text classification training on the negative sample set to obtain an optimized classification model;

[0013] The optimized classification model is used to perform classification prediction on the text data.

[0014] The above-mentioned graph contrastive learning semi-supervised text classification method based on clustering improved negative sampling, by constructing a heterogeneous text graph network model to characterize the complex relationship between text nodes and word nodes, provides high-quality graph input for subsequent contrastive learning; generates diversified enhanced views through a random edge discarding strategy to obtain richer contextual information in the subsequent contrastive learning process, and improves the generalization ability of node embedding; performs text representation learning on the enhanced views through a clustering algorithm to give priority to cross-cluster nodes and distant nodes within clusters as negative samples, ensuring the semantic diversity and high quality of the negative sample set, effectively alleviating the bias problem in the negative sample selection process, and avoiding the intra-class separation phenomenon caused by the incorrect selection of negative samples; through the joint optimization strategy of contrast loss and cross entropy loss, not only the semantic consistency of text embedding is improved, but also the robustness of classification performance is ensured. In addition, the above-mentioned graph contrastive learning semi-supervised text classification method based on clustering improved negative sampling has been experimentally verified on multiple benchmark datasets (such as 20Newsgroups, R8, R52, etc.). The results show that this method is superior to the existing technology in classification accuracy and robustness, providing a better innovative solution for semi-supervised text classification tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 A schematic diagram of a flow chart of a graph contrast learning semi-supervised text classification method based on clustering improved negative sampling in a preferred embodiment of the present invention;

[0016] Figure 2 for Figure 1 The flowchart of step S20 in the graph contrast learning semi-supervised text classification method based on clustering improved negative sampling is shown;

[0017] Figure 3 for Figure 1 The flowchart of step S30 in the graph contrast learning semi-supervised text classification method based on clustering improved negative sampling is shown;

[0018] Figure 4 for Figure 1 The diagram shown is a partial flow chart of step S40 in the graph contrastive learning semi-supervised text classification method based on clustering improved negative sampling. DETAILED DESCRIPTION

[0019] In order to facilitate the understanding of the present invention, the present invention will be described more fully below with reference to the relevant drawings. Preferred embodiments of the present invention are provided in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present invention more thorough and comprehensive.

[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items.

[0021] When describing positional relationships, unless otherwise specified, when an element is referred to as being "on" another element, it can be directly on the other element or there can be an intermediate element. It is also understood that when an element is referred to as being "between" two elements, it can be the only one between the two elements, or there can be one or more intermediate elements.

[0022] In the case of using “including”, “having”, and “comprising” described herein, another component may be added unless a clear limiting term such as “only”, “consisting of”, etc. is used. Unless mentioned otherwise, a term in the singular form may include a plural form and should not be understood as being one in number.

[0023] See also Figure 1 The graph contrast learning semi-supervised text classification method based on clustering improved negative sampling in a preferred embodiment of the present invention includes steps S10 to S60.

[0024] Step S10, obtaining text data, and constructing a heterogeneous text graph network model based on the text data. The text data includes document nodes and word nodes.

[0025] Step S20, calculating the edge weights between different nodes in the text data according to the heterogeneous text graph network model to construct a weighted graph model. The weighted graph model includes a node feature matrix and a weighted adjacency matrix.

[0026] By executing steps S10 and S20, the text data is represented as a heterogeneous network including word nodes and document nodes, and the edge weights between different nodes are calculated to construct a weighted graph model containing semantic and structural information. That is, by constructing a heterogeneous text graph network model to characterize the complex relationship between text nodes and word nodes, high-quality graph input is provided for subsequent contrastive learning.

[0027] Step S30: using a random edge discarding strategy to perform random masking on the weighted adjacency matrix to generate multiple groups of enhanced views.

[0028] By executing step S30, a diverse enhanced view is generated through a random edge dropping strategy to obtain richer context information in a subsequent contrastive learning process and improve the generalization ability of node embedding.

[0029] Step S40, using a clustering algorithm to perform text representation learning on multiple groups of enhanced views, so as to select nodes across clusters as negative samples, and combining a self-correction mechanism to supplement distant nodes within the cluster as negative samples, so as to obtain a negative sample set.

[0030] By executing step S40, the nodes in the enhanced view are clustered, the similarity between the nodes is calculated, and the negative samples are screened according to the distance. That is, after eliminating the nodes with close distance in the cluster, the method preferentially selects the nodes across the cluster as negative samples, and combines the self-correction mechanism to select the distant nodes in the same cluster as the supplementary negative samples to ensure the semantic diversity and high quality of the negative sample set, thereby effectively alleviating the bias problem in the negative sample selection process and avoiding the intra-class separation phenomenon caused by the wrong selection of negative samples.

[0031] Step S50, using a joint optimization strategy of contrast loss and cross entropy loss to perform node embedding representation learning and text classification training on the negative sample set to obtain an optimized classification model.

[0032] By executing step S50, a joint optimization strategy of contrast loss and cross entropy loss is adopted to perform model training. In the contrast loss part, the discrimination ability of the embedded representation is further enhanced by maximizing the similarity between positive samples and minimizing the similarity between positive and negative samples. The cross entropy loss directly optimizes the accuracy of the classification results. The combination of the two losses not only improves the semantic consistency of the text, but also ensures the robustness of the classification performance.

[0033] Step S60: Use the optimized classification model to perform classification prediction on the text data.

[0034] By executing steps S10 to S60, accurate and reliable classification predictions are made for graph text samples. In existing graph contrast learning methods, negative sample selection is prone to misjudgment. When nodes with the same label are mistakenly grouped as negative samples, the nodes within the class are separated in the embedding space (transition clustering problem). This application generates pseudo labels through clustering, and preferentially selects nodes across clusters as negative samples. At the same time, a correction mechanism is combined to select distant nodes within the same cluster as supplementary negative samples, thereby effectively alleviating the problem of negative sampling bias and optimizing the distribution quality of the embedded representation.

[0035] By fusing contrast loss and cross entropy loss, the present invention achieves a balance between node representation learning and classification performance optimization in the classification model. Contrastive loss enhances the discriminability and robustness of embedded representation, while cross entropy loss directly optimizes the accuracy of classification tasks. In addition, experimental results show that the combination strategy of contrast loss and cross entropy loss significantly improves the classification performance on multiple data sets compared with the single use of cross entropy loss or contrast loss, indicating its superior performance in semi-supervised text classification tasks.

[0036] The present invention adopts the random edge discarding strategy in graph data enhancement to generate diversified comparative views, which improves the adaptability of the classification model to noise and distribution changes. In addition, through experimental verification, different clustering methods (such as K-means, Agglomerative Clustering, BIRCH) have a small fluctuation in the performance of the classification model (about 0.7% to 1.4%), indicating that the present application has strong adaptability and robustness in negative sampling optimization and can be applied to a variety of scenarios.

[0037] Experimental results on five benchmark datasets (20Newsgroups, R8, R52, Ohsumed, MR) show that the ClusterText method of the present invention outperforms the existing GCL method and baseline models (such as TextGCN, BERT, TextGCL) in classification accuracy, with an average performance improvement of 0.4%-1.3%. This significant performance improvement not only verifies the effectiveness of the method, but also demonstrates its wide applicability in different task areas, including sentiment analysis, topic classification and other application scenarios.

[0038] Please also read Figure 2 In some embodiments, step S20 includes steps S21 to S24.

[0039] Step S21, when both node i and node j are word nodes, the edge weight between the two nodes is calculated using point mutual information.

[0040] Step S22, when node i is a document node and node j is a word node, the edge weights between the nodes are calculated using word frequency-inverse document frequency.

[0041] Step S23: when node i and node j are the same node, the edge weight is set to 1.

[0042] In order to construct the complete graph structure, when two nodes i and j are the same, the edge weight A ij Set to 1 to form a self-connection to preserve the initial state characteristics of the node.

[0043] Step S24, when the node i and the node j are in other cases than the above cases, the edge weight is set to 0. In other cases except steps S21 to S23, the edge weight is set to zero to ensure the sparsity of the graph.

[0044] By executing steps S21 to S24, edge weights between nodes are calculated to obtain a weighted adjacency matrix in the weighted graph model.

[0045] Further, step S21 includes the following steps:

[0046] The PMI value is calculated using the following formula:

[0047]

[0048] When PMI(i,j)>0, the edge weight A between node i and node j ij =PMI(i,j);

[0049] When PMI(i,j)≤0, the edge weight A between node i and node j ij =0.

[0050] Among them, P(i) is the probability of word i appearing, P(j) is the probability of word j appearing, and P(i,j) is the probability of word i and word j appearing at the same time.

[0051] In this way, by retaining only the PMI (point mutual information) value, the interference of negatively correlated word relationships on the graph is avoided, and the sparsity of the graph is also improved, reducing the complexity of subsequent calculations.

[0052] Further, in some embodiments, step S22 includes the following steps:

[0053] The TF-IDF value is calculated using the following formula:

[0054]

[0055] When node i is a document node and node j is a word node, the edge weight A between node i and node j is ij =PMI(i,j).

[0056] Where TF(i,j) is the number of occurrences of word j in document i, N is the total number of documents, and DF(j) is the number of documents containing word j.

[0057] The TF-IDF (term frequency-inverse document frequency) value can effectively measure the importance of word j to document i, thereby building a connection between document and word nodes. It not only considers the local co-occurrence information of words, but also incorporates global statistical characteristics, thereby effectively avoiding the negative impact of high-frequency meaningless words (such as stop words) on the performance of the classification model.

[0058] In this way, the weighted adjacency matrix A is finally constructed ij It can be expressed as:

[0059]

[0060] In some embodiments, step S20 further includes the following steps:

[0061] Use the pre-trained model to generate the initial feature X for each document node doc ;

[0062] Set the initial features of word nodes to 0;

[0063] The node feature matrix X is calculated using the following formula:

[0064]

[0065] Among them, d is the dimension of the node representation, n doc is the representation of the document node, n word is the representation of a word node.

[0066] In this way, in order to improve the performance of the classification model, the pre-trained model Bert is used to initialize the representation of the document node. The pre-trained model Bert can learn context-sensitive features from a large-scale corpus. Therefore, the initial features of each document node are generated by the pre-trained model Bert, with the matrix X doc In order to allow the network to dynamically learn the features of word nodes, the initial features of word nodes are set to zero. Finally, through the above steps, the node feature matrix in the weighted graph model is obtained.

[0067] Please also read Figure 3 In some embodiments, step S30 includes step S31 and step S32.

[0068] Step S31, creating a random mask matrix. Each parameter in the random mask matrix is ​​a Bernoulli random variable.

[0069] Specifically, each element M of the mask matrix ij is an independent Bernoulli random variable, defined as follows: P(M ij =1) = 1-p,P(M ij =0)=p. Where p is the edge discard probability, which indicates the possibility of an edge being removed.

[0070] Step S32: performing an element-by-element product operation on the weighted adjacency matrix a preset number of times using a random mask matrix to generate a plurality of sets of enhanced views.

[0071] Specifically, the enhanced adjacency matrix Calculated by element-wise product operation: Wherein, ⊙ is an element-by-element product operation. Through step S32, it is ensured that the adjacency matrix A still retains the semantic structure information of the pipe fitting after discarding some edges.

[0072] Each enhancement operation is based on an independent random mask matrix M1, M2, ...Mk, thereby generating k enhanced adjacency matrices Each set of enhanced views can be represented as: Among them, X is the node feature matrix, which remains unchanged. This diversity design ensures that the enhanced views have significant structural differences while retaining global semantic consistency. The generated enhanced view G 1 ,G 2 ,...,G k It will be used in the contrastive learning phase to compute the representation consistency between enhanced views.

[0073] In some embodiments, the step of supplementing distant nodes in the cluster as negative samples in combination with the self-correction mechanism includes the following steps:

[0074] An unsupervised machine clustering method is used to cluster the document node embeddings in the text data to obtain the pseudo label C(v i ) = Cluster(Z);

[0075] The following formula is used to calculate the nodes v of the same cluster: i and node v j The Euclidean distance dist ij :

[0076] dist ij =EuclideanDist(v i ,v j );

[0077] According to the Euclidean distance dist ij Get an ascending collection

[0078] The hyperparameters are calculated according to the following formula

[0079]

[0080] Select the distance from the anchor node within the cluster to exceed the hyperparameter The text nodes are regarded as distant nodes in the cluster

[0081] The negative sample set N(v i ):

[0082]

[0083] Among them, Z is the fused text representation, n is the number of nodes in the cluster, and the distribution (v i ,d) is the distribution of d% of nodes farthest from the anchor node.

[0084] Based on the pseudo-label C, the present invention selects the remaining nodes from different clusters as the negative sample set of each anchor node. The nodes in the same cluster that tend to share the true label with the anchor point will be excluded from the negative samples. The present invention proposes a self-correction mechanism to expand the negative samples in the same cluster, that is, calculate the Euclidean distance within the nodes of the same cluster, and set a hyperparameter d to select the d% farthest from the anchor node as the negative sample, so as to solve the problem that the required negative candidate nodes in the same cluster (i.e., those nodes that share different true labels as anchor nodes) may be ignored when the pseudo-label and the true label are obviously different.

[0085] Please also read Figure 4 In some embodiments, the step of using a clustering algorithm to perform text representation learning on multiple groups of enhanced views includes step S40 - 1 and step S40 - 2 .

[0086] Step S40-1, obtaining a weighted fusion strategy based on the semantic features of Bert and the structural features of GCN.

[0087] Specifically, step S40-1 includes the following steps:

[0088] The weighted adjacency matrix A is normalized using the following formula to obtain the normalized adjacency matrix:

[0089]

[0090] The initial feature matrix X of the node is input into GCN for the first-layer feature aggregation and transformation to obtain the first-layer output feature matrix H (1) , the feature update formula is as follows:

[0091]

[0092] The first-layer output feature matrix is ​​aggregated and transformed in the second layer in GCN, and the matrix H is represented by the output node. The feature update formula is:

[0093]

[0094] Through the fully connected layer, the document representation H of GCN is doc and Bert's documentation indicates that X doc Perform linear mapping to obtain the linear document representation FC(H) of GCN doc ) and Bert's linear document representation FC(X doc );

[0095] The weighted fusion strategy Z is obtained according to the following formula:

[0096] Z = λ × FC (Hdoc )+(1-λ)×FC(X doc ).

[0097] Among them, W (0) is the trainable weight matrix of the first layer, σ is the nonlinear activation function, W (1) is the trainable weight matrix of the second layer, and λ is the weighting coefficient.

[0098] λ is used to control the contribution ratio of Bert (pre-trained language model) and GCN (graph convolutional neural network model) in the final document features. In practical applications, by adjusting the value of λ, dynamic feature fusion can be achieved according to the needs of specific tasks, thereby enhancing the adaptability and flexibility of the graph contrastive learning semi-supervised text classification method based on clustering improved negative sampling.

[0099] Step S40 - 2 , using a weighted fusion strategy to perform text representation learning on multiple groups of enhanced views, so as to select nodes across clusters as negative samples.

[0100] The fusion of the Bert model and the GCN model in the weighted fusion strategy can make full use of the complementary advantages between the two, while improving the robustness and generalization ability of the classification model, especially when processing noisy data and complex graph structures.

[0101] In some embodiments, the steps of the joint optimization strategy of contrast loss and cross entropy loss include:

[0102] For any node v i , the contrast loss function of data node 1 and data node 2 As shown in the following formula:

[0103]

[0104] For any node v i , the contrast loss function of data node 2 and data node 3 As shown in the following formula:

[0105]

[0106] The contrast loss function is calculated using the following formula:

[0107]

[0108] Use the softmax classifier to represent the text in the negative sample set Z 0 Perform label prediction to obtain text representation Z 0 The probability of occurrence P = softmax(Z 0 );

[0109] The cross entropy loss function is calculated using the following formula:

[0110]

[0111] The total loss function is calculated using the following formula:

[0112]

[0113] Using the total loss function The node embedding and classifier parameters are updated through the gradient descent algorithm to obtain the optimized classification model.

[0114] Among them, τ is the temperature parameter, is the node v in data set 1 i The node representation is is the node v in data set 2 i The node representation is is the node v in data k j The node representation of , the function Sim(·,·) represents the cosine similarity measure used to quantify the similarity between a pair of vectors.

[0115] In this way, through the above steps, the contrast loss function and the cross entropy loss function are calculated respectively, so as to finally obtain the total loss function in the joint optimization strategy of contrast loss and cross entropy loss.

[0116] In some embodiments, step S60 further includes the following steps:

[0117] The precision of the optimized classification model is calculated using the following formula.

[0118]

[0119] The recall rate Recall of the optimized classification model is calculated using the following formula:

[0120]

[0121] The F1 score is calculated using the following formula:

[0122]

[0123] The performance of the optimized classification model in the classification task of a single category is analyzed using precision, recall, and F1 score.

[0124] In this way, after executing step S60, the performance of the classification model is evaluated based on the classification prediction result obtained in step S60. The precision Precision calculates the sample ratio of the actual data samples of a certain category among the samples predicted by the model. The recall rate Recall measures the ratio of samples of a certain category correctly predicted by the classification model. The F1 score combines the precision Precision and the recall rate Recall, which can provide a balanced evaluation between the accuracy and coverage of the classification model. Therefore, the precision Precision, recall rate Recall and F1 score in the above steps can accurately evaluate the performance of the classification model in the classification task of a single category.

[0125] Furthermore, in some embodiments, after the step of calculating the F1 score, the method further includes:

[0126] In multi-category classification tasks, the macro-average calculation of the precision of each category is performed using the following formula:

[0127]

[0128] In multi-category classification tasks, the recall rate of each category is calculated macro-averaged using the following formula:

[0129]

[0130] In the classification task of multiple categories, the F1 score of each category is macro-averaged using the following formula:

[0131]

[0132] In the classification task of multiple categories, the weighted average calculation of the accuracy of each category is performed using the following formula:

[0133]

[0134] In the classification task of multiple categories, the recall rate of each category is calculated by weighted average using the following formula:

[0135]

[0136] In the classification task of multiple categories, the weighted average F1 score of each category is calculated using the following formula:

[0137]

[0138] The performance of the optimized classification model in classification tasks of multiple categories is analyzed using the macro-average precision, macro-average recall, macro-average F1 score, weighted average precision, weighted average recall, and weighted average F1 score.

[0139] Among them, C is the total number of categories, w i is the sample proportion of category i.

[0140] In multi-classification tasks, due to the differences in category distribution, the present invention adopts two strategies: macro-averaged metrics and weighted-averaged metrics for evaluation. Macro-averaged metrics focuses on the equality of all categories, while weighted-averaged metrics is more sensitive to the performance of mainstream categories. Therefore, through the above steps, the performance of the classification model in classification tasks of multiple categories can be accurately evaluated.

[0141] In some embodiments, after step S60, the method further includes the following steps:

[0142] In the two-category classification task, the false positive rate FPR in the ROC curve is calculated using the following formula:

[0143]

[0144] In the two-category classification task, the true proportion rate TPR in the ROC curve is calculated using the following formula:

[0145]

[0146] The area under the ROC curve (AUC) was used to analyze the performance of the optimized classification model in the binary classification task.

[0147] The ROC curve reflects the relationship between the false positive rate (FPR) and the true positive rate (TPR) of the model at different thresholds. The AUC represents the area under the ROC curve. In actual evaluation, the closer the AUC value is to 1, the stronger the classification ability of the classification model is, especially suitable for scenarios with severely unbalanced category distribution. Therefore, the performance of the classification model in the binary classification task is accurately evaluated by the area under the ROC curve AUC in the above steps.

[0148] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0149] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A graph contrast learning semi-supervised text classification method based on clustering improved negative sampling, characterized in that: Includes steps: Acquire text data, and construct a heterogeneous text graph network model based on the text data; the text data includes document nodes and word nodes; Calculating edge weights between different nodes in the text data according to the heterogeneous text graph network model to construct a weighted graph model; the weighted graph model includes a node feature matrix and a weighted adjacency matrix; Using a random edge dropping strategy to perform random masking on the weighted adjacency matrix to generate multiple sets of enhanced views; A clustering algorithm is used to perform text representation learning on multiple groups of enhanced views to select nodes across clusters as negative samples, and a self-correction mechanism is combined to supplement distant nodes within the cluster as negative samples to obtain a negative sample set; Using a joint optimization strategy of contrast loss and cross entropy loss to perform node embedding representation learning and text classification training on the negative sample set to obtain an optimized classification model; The optimized classification model is used to perform classification prediction on the text data.

2. The graph contrast learning semi-supervised text classification method based on clustering improved negative sampling according to claim 1 is characterized in that: The step of calculating edge weights between different nodes according to the heterogeneous text graph network includes: When both node i and node j are word nodes, the edge weight between the two nodes is calculated using point mutual information; When node i is a document node and node j is a word node, the edge weights between nodes are calculated using word frequency-inverse document frequency; When node i and node j are the same nodes, the edge weight is set to 1; When nodes i and j are in other cases, the edge weight is set to 0.

3. The graph contrast learning semi-supervised text classification method based on clustering improved negative sampling according to claim 2 is characterized in that: When both nodes are word nodes, the step of calculating the edge weight between the two nodes using the point-to-point mutual information includes: The PMI value is calculated using the following formula: When PMI(i,j)>0, the edge weight A between two nodes ij =PMI(i,j); When PMI(i,j)≤0, A ij =0; Among them, P(i) is the probability of word i appearing, P(j) is the probability of word j appearing, and P(i,j) is the probability of word i and word j appearing at the same time.

4. The graph contrast learning semi-supervised text classification method based on clustering improved negative sampling according to claim 3 is characterized in that: When node i is a document node and node j is a word node, the steps for calculating the edge weights between nodes using word frequency-inverse document frequency are: The TF-IDF value is calculated using the following formula: When node i is a document node and node j is a word node, the edge weight A between node i and node j is ij =PMI(i,j).

5. The graph contrast learning semi-supervised text classification method based on clustering improved negative sampling according to claim 1 is characterized in that: The step of calculating the edge weights between different nodes according to the heterogeneous text graph network model to construct a weighted graph model also includes: Use the pre-trained model to generate the initial feature X for each document node doc ; Set the initial features of word nodes to 0; The node feature matrix X is calculated using the following formula: Among them, d is the dimension of the node representation, n doc is the representation of the document node, n word is the representation of a word node.

6. The graph contrast learning semi-supervised text classification method based on clustering improved negative sampling according to claim 1, characterized in that: The steps of performing random masking processing on the weighted adjacency matrix by adopting a random edge discarding strategy to generate multiple groups of enhanced views include: Creating a random mask matrix; each parameter in the random mask matrix is ​​a Bernoulli random variable; The random mask matrix is ​​used to perform an element-by-element product operation on the weighted adjacency matrix a preset number of times to generate a plurality of sets of enhanced views.

7. The graph contrast learning semi-supervised text classification method based on clustering improved negative sampling according to claim 1 is characterized in that: The steps of combining the self-correction mechanism to supplement distant nodes in the cluster as negative samples include: The document node embeddings in the text data are clustered using an unsupervised machine clustering method to obtain a pseudo label C (v i ) = Cluster(Z); The following formula is used to calculate the nodes v of the same cluster: i and node v j The Euclidean distance dist ij : dist ij =EuclideanDist(v i ,v j ); According to the Euclidean distance dist ij Get an ascending collection The hyperparameters are calculated according to the following formula Select the distance from the anchor node within the cluster to exceed the hyperparameter The text nodes are regarded as distant nodes in the cluster The negative sample set N(v i ): Among them, Z is the fused text representation, n is the number of nodes in the cluster, and the distribution (v i ,d) is the distribution of d% of nodes farthest from the anchor node.

8. The graph contrast learning semi-supervised text classification method based on clustering improved negative sampling according to claim 1, characterized in that: The step of using a clustering algorithm to perform text representation learning on a plurality of groups of enhanced views comprises: A weighted fusion strategy is obtained based on the semantic features of Bert and the structural features of GCN; The weighted fusion strategy is used to perform text representation learning on multiple groups of enhanced views to select nodes across clusters as negative samples.

9. The graph contrast learning semi-supervised text classification method based on clustering improved negative sampling according to claim 8, characterized in that: The steps to obtain the weighted fusion strategy based on the semantic features of Bert and the structural features of GCN include: The weighted adjacency matrix A is normalized using the following formula to obtain the normalized adjacency matrix: The initial feature matrix X of the node is input into GCN for the first-layer feature aggregation and transformation to obtain the first-layer output feature matrix H (1) , the feature update formula is as follows: The first layer output feature matrix is ​​aggregated and transformed in the second layer in GCN, and the matrix H is represented by the output node. The feature update formula is: Through the fully connected layer, the document representation H of GCN is doc and Bert's documentation indicates that X doc Perform linear mapping to obtain the linear document representation FC(H) of GCN doc ) and Bert's linear document representation FC(X doc ); The weighted fusion strategy Z is obtained according to the following formula: Z=λ×FC(H doc )+(1-λ)×FC(X doc ); Among them, W (0) is the trainable weight matrix of the first layer, σ is the nonlinear activation function, W (1) is the trainable weight matrix of the second layer, and λ is the weighting coefficient.

10. The graph contrast learning semi-supervised text classification method based on clustering improved negative sampling according to claim 1, characterized in that: The steps of the joint optimization strategy of contrast loss and cross entropy loss include: For any node v i , the contrast loss function of data node 1 and data node 2 As shown in the following formula: For any node v i , the contrast loss function of data node 2 and data node 3 As shown in the following formula: The contrast loss function is calculated using the following formula: Using a softmax classifier to perform label prediction on the text representation Z0 in the negative sample set to obtain a probability P = softmax (Z0) of the text representation Z0 appearing; The cross entropy loss function is calculated using the following formula: The total loss function is calculated using the following formula: Using the total loss function Update the node embedding and classifier parameters through the gradient descent algorithm to obtain the optimized classification model; Among them, τ is the temperature parameter, is the node v in data set 1 i The node representation is is the node v in data set 2 i The node representation is is the node v in data k j The node representation of , the function Sim(·,·) represents the cosine similarity measure used to quantify the similarity between a pair of vectors.

11. The graph contrast learning semi-supervised text classification method based on clustering improved negative sampling according to claim 1, characterized in that: After the step of using the optimized classification model to perform classification prediction on the text data, the step further includes: The precision of the optimized classification model is calculated using the following formula: The recall rate Recall of the optimized classification model is calculated using the following formula: The F1 score is calculated using the following formula: The performance of the optimized classification model in the classification task of a single category is analyzed using the precision, the recall, and the F1 score.

12. The graph contrast learning semi-supervised text classification method based on clustering improved negative sampling according to claim 11, characterized in that: After the step of calculating the F1 score, it also includes: In multi-category classification tasks, the macro-average calculation of the precision of each category is performed using the following formula: In multi-category classification tasks, the recall rate of each category is calculated macro-averaged using the following formula: In the classification task of multiple categories, the F1 score of each category is macro-averaged using the following formula: In the classification task of multiple categories, the weighted average calculation of the accuracy of each category is performed using the following formula: In the classification task of multiple categories, the recall rate of each category is calculated by weighted average using the following formula: In the classification task of multiple categories, the weighted average F1 score of each category is calculated using the following formula: Analyzing the performance of the optimized classification model in classification tasks of multiple categories using the macro-average calculated precision, the macro-average calculated recall, the macro-average calculated F1 score, the weighted average calculated precision, the weighted average calculated recall, and the weighted average calculated F1 score; Among them, C is the total number of categories, w i is the sample proportion of category i.

13. The graph contrast learning semi-supervised text classification method based on clustering improved negative sampling according to claim 1, characterized in that: After the step of using the optimized classification model to perform classification prediction on the text data, the step further includes: In the two-category classification task, the false positive rate FPR in the ROC curve is calculated using the following formula: In the two-category classification task, the true proportion rate TPR in the ROC curve is calculated using the following formula: The area under the ROC curve (AUC-ROC) was used to analyze the performance of the optimized classification model in the binary classification task.

Images