Paper classification method based on low-rank self-expression contrast learning
Through the low-rank self-expression comparison learning method, the paper classification model is constructed using the self-expression matrix and contrast learning loss, which solves the problem of improper handling of complex citation relationships in the traditional method and improves the accuracy of paper classification.
Patent Information
- Application Number
- CN202510437969.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-09
- Publication Date
- 2025-07-25
AI Technical Summary
Traditional contrast learning methods cannot effectively deal with complex potential citation relationships in paper classification, resulting in the loss of key information in feature extraction stage, affecting the accuracy of classification.
The low-rank self-expression comparison learning method is used to cluster through the self-expression matrix, highly reliable positive and negative sample pairs are calculated, and the paper classification model is constructed, and the self-expression matrix and contrast learning loss are used for feature extraction and classification.
It improves the accuracy of paper classification, can discover indirectly connected potential positive samples, enhance feature expression, and improve classification effect.
Smart Images
Figure CN120372011A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a paper classification technology, specifically a paper classification method based on low-rank self-expression contrast learning. Background Art
[0002] In the research field of paper classification, how to efficiently and accurately classify a large number of papers has always been the core issue focused on by many scholars and researchers. Currently, a relatively common approach is to classify papers based on the relationships between papers. Traditional contrast learning, as a very effective means of feature expression learning, can be used to analyze the relationships between papers.
[0003] Traditional contrast learning focuses on extracting meaningful representations by comparing positive and negative examples. The assumption it uses is that in the learned embedding space, similar examples should be closer, while dissimilar examples should be farther apart. This method mainly involves the local relationship structure of sample pairs, that is, papers with direct citations between them are used as similar positive samples, and the rest are dissimilar negative samples, and contrast learning is carried out based on this. Finally, the learned feature expression is applied to the classification model. Specifically in the paper classification scenario: the traditional method often regards paper pairs with direct citation relationships as similar positive samples. In contrast, paper pairs without direct citation relationships are regarded as dissimilar negative samples. Researchers carry out contrast learning based on this division, hoping to continuously optimize the feature expression and apply it to the classification model to achieve the goal of accurate paper classification.
[0004] However, in the actual paper sample set, the internal relationships are far more complex than the theoretical assumptions. There are various complex potential citation situations among paper samples. Different from intuitive direct citations, potential citations have various forms: one situation is indirect citation. For example, if paper C cites paper D, and paper D cites paper E, there is an indirect citation relationship between paper C and paper E that is easily overlooked by the traditional direct citation judgment standard; in another situation, although there seems to be no citation relationship between papers on the surface, but from multiple dimensions such as theme, research direction, and professional field knowledge, they actually belong to the same category. For example, two papers study the treatment methods of the same disease from different perspectives. Although there is no direct citation, they essentially belong to the same research category. When dealing with these two situations, the traditional contrast learning method simply regards all paper pairs with non-direct citation relationships as negative samples for learning, resulting in a large amount of key information being lost in the feature extraction stage, seriously affecting the accuracy of paper classification, unable to truly reflect the complex real relationships between papers, and ultimately having a great negative impact on the classification effect. Summary of the Invention
[0005] Aiming at the deficiencies in the prior art, the present invention provides a paper classification method based on low-rank self-expression contrast learning.
[0006] The present invention achieves the above technical objectives through the following technical means.
[0007] A method for classifying papers based on low-rank self-expression contrast learning, comprising: First, prepare n papers, construct an original sample dataset X ∈ R d×n , where d is the number of papers after removing duplicates of the papers cited by the n papers, and preprocess the original sample dataset; Second, construct a paper classification model through self-expression theory and contrast learning; Finally, use the evaluated paper classification model for paper classification.
[0008] Furthermore, the preprocessing process of the original sample dataset is as follows:
[0009] When the citation relationship between the i-th paper among the n papers and the j-th paper among the d papers is a direct citation, set the element in the i-th row and j-th column of the original sample dataset X to 1, otherwise set it to 0, thereby updating the original sample dataset X, and then normalize the original sample dataset X through the Laplacian matrix.
[0010] Even further, the process of constructing the paper classification model is as follows:
[0011] First, obtain a self-expression matrix using self-expression theory; Second, based on the self-expression matrix, combine the original samples and self-expression learning samples in the original sample dataset to construct a new sample dataset; Then, construct the final contrast learning loss through the self-expression regularization term and the contrast learning loss; Finally, construct a contrast learning network based on the final contrast learning loss to obtain the classification result.
[0012] Even further, the formula for obtaining the self-expression matrix S is:
[0013]
[0014] s.t. w T w = I, diag(S) = 0
[0015] where λ is a hyperparameter set manually, w is a low-rank projection matrix, I is the identity matrix, and ||·|| F represents the Frobenius norm.
[0016] Even further, the new sample dataset The formula is:
[0017]
[0018] s.t. w T w = I, rank(S) = r
[0019] where r is the total number of paper sample categories.
[0020] Furthermore, the formula for the final contrastive learning loss L is as follows:
[0021]
[0022] where sim is the similarity calculation, is the i-th sample of the new sample dataset and is the positive sample of and is the negative sample of, t is the temperature parameter, and exp() is the exponential of the value in the parentheses to the constant e.
[0023] Furthermore, the process of constructing the contrastive learning network is as follows:
[0024] First, learn the original sample dataset X through the final contrastive learning loss L, and use backpropagation to learn the self-expression matrix S; second, perform Kmeans clustering on S, select positive and negative samples for contrastive learning based on the clustering results of S, and continue to learn using L until L converges to obtain the optimal S; finally, replace X with the optimal S and perform Kmeans clustering on the replaced samples to obtain the clustering classification results.
[0025] Furthermore, the formula for backpropagation to learn the self-expression matrix S is as follows:
[0026]
[0027] where λ1 is a manually set hyperparameter, and γ and β are penalty parameters.
[0028] Furthermore, the process of evaluating the paper classification model is as follows: Use the paper classification model and CCGC, AutoSSL, AGE, MVGRL, DAEGC, ARGA clustering methods to cluster the Cora and Citeseer datasets to obtain classification results, and then evaluate each classification result through evaluation criteria.
[0029] Furthermore, the evaluation criteria include accuracy and F1 score.
[0030] The beneficial effects of the present invention are as follows:
[0031] The present invention does not rely on the direct citation between local papers to obtain positive and negative sample pairs, but introduces a self-expression matrix for clustering, thereby calculating highly credible positive and negative sample pairs. This method can discover potential positive samples that are not directly connected between samples and perform contrastive learning based on this, thus differing from the selection of positive samples in traditional contrastive learning and improving the classification accuracy. Description of the Drawings
[0032] Figure 1 This is the flowchart of the paper classification method based on low-rank self-expression contrast learning described in the present invention. Specific embodiments
[0033] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, but the protection scope of the present invention is not limited thereto.
[0034] Combined with the attached Figure 1 drawings, a paper classification method based on low-rank self-expression contrast learning proposed in this example can be roughly divided into the following steps.
[0035] Step 1: First, prepare n papers and construct an original sample dataset X ∈ R d×n , where d is the number of papers after removing duplicates of the papers cited by the n papers. Secondly, preprocess the original sample dataset: when the citation relationship between the i-th paper in the n papers and the j-th paper in the d papers is a direct citation, set the element in the i-th row and j-th column of the original sample dataset X to 1, otherwise set it to 0, so as to update the original sample dataset X, and then normalize the original sample dataset X through the Laplacian matrix.
[0036] Step 2: Construct the self-expression contrast learning paper classification model
[0037] Step 2-1: Assume that all samples within the subspace are independent of each other, that is, samples belonging to the same category are independent of each other, and all samples of the same category can be represented by each other through a certain linear combination, which is called self-expression. In this embodiment, the original samples in the dataset are learned through self-expression, and the self-expression theory can be formally expressed as:
[0038] min‖X - XS‖ F (1)
[0039] where, ||·|| F represents the Frobenius norm, and S is the self-expression matrix.
[0040] However, the original samples usually contain high-dimensional noise. To solve this problem, a low-rank projection matrix w is introduced to map the samples to a low-dimensional subspace to reduce the influence of noise. Therefore, the self-expression theory is redefined using the following formula:
[0041]
[0042] s.t. w T w = I, diag(S) = 0
[0043] Among them, λ is a hyperparameter set manually, and I is the identity matrix. Formula (2) is used to measure the difference between the original samples and the self-expressive learning samples. When the value of formula (2) is small, the similarity between the self-expressive learning samples and the original samples is higher, which also means that the self-expressive matrix S can more accurately describe the internal structure and correlation between the original sample data at this time.
[0044] Based on the self-expression theory, the self-expression matrix S is obtained, and the original samples and the self-expressive learning samples are combined to construct new samples, and the new sample dataset is expressed as:
[0045]
[0046] Step 2-2: The self-expression theory can only capture the linear combination relationship between sample data. When there are some non-linear relationships between sample data, introducing contrastive learning can weaken the strict linear relationship between sample data, obtain enhanced sample features that are more in line with the samples, and reduce overfitting.
[0047] Contrastive learning constructs positive and negative sample features for contrastive learning, calculates the minimized contrastive learning loss, so that the features between positive samples are closer in the feature space, and the features between negative samples are farther in the feature space. Among them, the contrastive learning loss L con The formula is as follows:
[0048]
[0049] Among them, sim is the similarity calculation, generally the cosine similarity, x i is the i-th sample in X, is the positive sample of the same class as x i , is the negative sample of the same class as x i , and t is the temperature parameter, and exp() is the exponential of the value in the parentheses to the constant e.
[0050] Furthermore, the contrastive learning loss of the new samples is calculated, and the calculation formula is:
[0051]
[0052] Among them, is the i-th sample of , is the positive sample of , is the negative sample of
[0053] Even further, a self-expression regularization term is added during the calculation of the contrastive learning loss to obtain the final calculation formula of the contrastive learning loss L:
[0054]
[0055] Step 2-3: Construct a contrastive learning network: Learn X through L and learn S using backpropagation; perform Kmeans clustering on S, select positive and negative samples for contrastive learning based on the clustering results of S, and continue to learn using L; until L converges, obtain the optimal S, use the optimal S to replace X, and perform Kmeans clustering on the replaced samples to obtain the clustering classification results.
[0056] In addition, the formula for backpropagation learning of the self-expression matrix S is:
[0057]
[0058] where λ1 is a hyperparameter set manually, and γ, β are penalty parameters.
[0059] Step 3: Use the self-expression contrastive learning paper classification model to classify papers.
[0060] First, verify the reliability of the self-expression contrastive learning paper classification model constructed in this embodiment. Use the method of this embodiment and other advanced clustering methods (such as CCGC, AutoSSL, AGE, MVGRL, DAEGC, ARGA) to cluster data sets such as Cora and Citeseer to obtain classification results; secondly, evaluate each classification result through evaluation criteria (accuracy, F1 score); finally, use the evaluated self-expression contrastive learning paper classification model to classify papers.
[0061] The above embodiment is a preferred embodiment of the present invention, but the present invention is not limited to the above embodiment. Without departing from the essence of the present invention, any obvious improvements, substitutions or variations that those skilled in the art can make all fall within the protection scope of the present invention.
Claims
1. A paper classification method based on low-rank self-expression contrast learning, characterized in that: First, prepare n papers and construct the original sample dataset X ∈ R d×n , where d is the number of papers after removing duplicates of the papers cited by the n papers, and preprocess the original sample dataset; secondly, construct a paper classification model through self-expression theory and contrastive learning; finally, use the evaluated paper classification model to classify papers.
2. The method for classifying papers based on low-rank self-expression contrast learning according to claim 1, wherein The preprocessing process of the original sample dataset is as follows: When the citation relationship of the j-th paper in d papers by the i-th paper in n papers is a direct citation, the element in the i-th row and j-th column of the original sample dataset X is set to 1, otherwise it is set to 0, so as to update the original sample dataset X, and then the original sample dataset X is normalized by the Laplacian matrix.
3. The method for classifying papers based on low-rank self-expression contrast learning according to claim 2, wherein The process of constructing the paper classification model is as follows: First, the self-expression matrix is obtained using the self-expression theory; second, based on the self-expression matrix, the original samples in the original sample dataset and the self-expression learning samples are combined to construct a new sample dataset; then, the final contrast learning loss is constructed through the self-expression regularization term and the contrast learning loss; finally, a contrast learning network is constructed based on the final contrast learning loss to obtain the classification result.
4. The method for classifying papers based on low-rank self-expression contrast learning according to claim 3, wherein, The formula for obtaining the self-expression matrix S is: s.t.w T w = I, diag(S) = 0 Among them, λ is a hyperparameter set manually, w is a low-rank projection matrix, I is an identity matrix, and ||·|| F represents the Frobenius norm.
5. The method for classifying papers based on low-rank self-expression contrast learning according to claim 4, characterized in that The new sample data set The formula is: s.t.w T w = I, rank(S) = r where r is the total number of paper sample categories.
6. The method for classifying papers based on low-rank self-expression contrastive learning according to claim 5, wherein The formula for the final contrast learning loss L is: Among them, sim is the similarity calculation, is the i-th sample of the new sample data set , is the positive sample of , is the negative sample of , and t is the temperature parameter. exp() is to calculate the exponential of the constant e for the value inside the parentheses.
7. The method for classifying papers based on low-rank self-expression contrast learning according to claim 6, wherein The process of constructing the contrast learning network is as follows: First, learn the original sample dataset X through the final contrast learning loss L, and use backpropagation to learn the self-expression matrix S; second, perform Kmeans clustering on S, select positive and negative samples for contrast learning based on the clustering results of S, and continue to learn using L until L converges to obtain the optimal S; finally, replace X with the optimal S, and perform Kmeans clustering on the replaced samples to obtain the clustering classification result.
8. The method for classifying papers based on low-rank self-expression contrast learning according to claim 7, wherein The formula for learning the self-expression matrix S by backpropagation is: where λ1 is a hyperparameter set manually, and γ and β are penalty parameters.
9. The method for classifying papers based on low-rank self-expression contrast learning according to claim 8, wherein The evaluation process of the paper classification model is: use the paper classification model and clustering methods such as CCGC, AutoSSL, AGE, MVGRL, DAEGC, and ARGA to cluster the Cora and Citeseer datasets to obtain the classification results, and then evaluate each classification result through the evaluation criteria.
10. The method for classifying papers based on low-rank self-expression contrast learning according to claim 9, wherein, The evaluation criteria include accuracy and F1 score.