A literature classification method and system based on an elastic graph neural network

Abstract

This invention discloses a document classification method and system based on elastic graph neural networks (GNNs), comprising: extracting feature information from documents to construct a feature matrix and an adjacency matrix representing the relationships between documents; inputting the feature matrix and adjacency matrix into two parallel branches for processing; the first branch passing through a first multilayer perceptron, a first activation function, and a second multilayer perceptron; the second branch passing through a first multi-head perceptron attention module, a third multilayer perceptron, a second activation function, a second multi-head perceptron attention module, and weighted summation; aggregating and connecting the results of the first and second branches to obtain a first node representation, processing it through an elastic graph neural network with a low-pass filter to obtain a second node representation; and inputting the second node representation into a softmax function to obtain the final classification result of the node. This invention enhances the local smoothing and adaptive ability of GNNs and solves the problem of fixed neighbor scores.

Description

Technical Field

[0001] This invention belongs to the field of machine learning, and in particular relates to a document classification method and system based on elastic graph neural networks. Background Technology

[0002] Currently, document classification plays a crucial role in quickly recommending relevant documents to a specific audience. There are many document classification methods. For example, GNN (Graph Neural Network) is a branch of neural networks used to process graph-structured data, capable of uniformly aggregating features between different nodes. Meanwhile, GAT (Graph Attention Networks) is another effective method that has emerged in recent years for processing graph-structured data. It aggregates and classifies nodes by calculating the attention coefficient between each node and its surrounding nodes. GAT can fully utilize the relationships between nodes and consider the different importance of each node, thereby improving classification accuracy. However, existing technologies do not fully learn the features within nodes and do not adequately consider the interactions between nodes. For example, existing GNN methods may experience oversmoothing when learning node features, and existing GAT methods give less consideration to the spatial relationships between nodes. Summary of the Invention

[0003] The technical problem to be solved by this invention is to provide a document classification method and system based on elastic graph neural networks. By extracting the feature information of documents, a feature matrix and an adjacency matrix representing the relationship between documents are constructed. The feature matrix and adjacency matrix are processed by elastic graph neural networks, and the final classification result of the nodes is obtained by using the softmax function. This enhances the local smoothness and adaptive ability of GNN and solves the problem of fixed neighbor scores for any query node.

[0004] To address the aforementioned technical problems, this invention provides the following technical solution: a document classification method based on elastic graph neural networks, comprising nodes based on various label features, including:

[0005] S1. Extract the feature information of the documents, construct the feature matrix and the adjacency matrix representing the relationship between the documents.

[0006] S2. Input the feature matrix and adjacency matrix into two parallel branches for processing.

[0007] (1) The first branch passes through the first multilayer perceptron, the first activation function ReLU, and the second multilayer perceptron.

[0008] (2) The second branch passes through the first multi-head perception attention module, the third multi-layer perceptron, the second activation function, and the second multi-head perception attention module respectively.

[0009] S3. Aggregate and connect the results obtained from the first branch and the second branch to obtain the representation of the first node.

[0010] S4. The representation of the first node is processed by an elastic graph neural network with a low-pass filter to obtain the representation of the second node.

[0011] S5. Input the representation of the second node into the softmax function to obtain the final classification result of the node.

[0012] Furthermore, step S1 specifically involves: assuming a document has m nodes and n features, the resulting feature matrix corresponding to the document is X∈R. m×n The adjacency matrix corresponding to the literature is A∈R n×n .

[0013] Furthermore, the specific steps of the processing flow in step S2 are as follows:

[0014] (1) The specific content of the first branch data processing flow is as follows:

[0015] After processing by the first multilayer perceptron and the first activation function, the specific formula is as follows:

[0016] H = ReLU(XW) h +b h );

[0017] Where H is the output of the hidden layer, ReLU is the rectified linear function, and W... h ∈R n×h b h ∈R m×h denoted as the weights and biases of the hidden layer, respectively, and h is the number of neurons in the hidden layer.

[0018] After processing by the second layer of the perceptron, the specific formula is as follows:

[0019] O = HW y +b y

[0020] Where O is the obtained output vector, W y ∈R h×i b y ∈R m×i denoted as the weights and biases of the output layer, respectively, and i represents the feature dimension of the output Y.

[0021] (2) The specific content of the second branch data processing flow is as follows:

[0022] The feature matrix X and the adjacency matrix A are input into the first multi-head perception attention module to obtain the output H′. The first multi-head perception attention module is an 8-head perception attention module, and the specific formula is as follows:

[0023] H′=H′1||H′2||H′3||H′4||H′5||H′6||H′7||H′8

[0024] Among them, H′ j (j=1~8) is the output of the feature matrix X and the adjacency matrix A through the input of the j-th head perception attention module, and | is the parallel operation.

[0025] H′ j The calculation formula is:

[0026] H′ j =α j W j AX

[0027] Where, α j =SPEattention(X), SPEattention(·) uses a self-attention mechanism with spatial embedding, W j These are learnable weights.

[0028] Self-attention values ​​with spatial location embedding introduce spatial location information into the self-attention mechanism, and their expression is as follows:

[0029] SPE attention(X)=(softmax(FC1(X)FC2(X))+ReLU(CNN(E)))FC3(X)

[0030] Where Softmax is the normalized exponential function, FC1 is the first fully connected function, FC2 is the second fully connected function, FC3 is the third fully connected function, ReLU is the activation function in the self-attention mechanism with spatial location embedding, CNN is a one-dimensional convolution function, and E is a learnable vector.

[0031] Input H′ sequentially into the third multilayer perceptron and the second activation function to obtain the output H″, as shown in the specific formula:

[0032] H″=ReLU(H′W′+b′)

[0033] Where W′ is the learnable weight and b′ is the learnable bias.

[0034] Input H″ into the second multi-head perception attention module to obtain the output H″′. The second multi-head perception attention module is an 8-head perception attention module, and the specific formula is as follows:

[0035] H″′=H′1′||H′2′||H′3′||H′4′||H′5′||H′6′||H′7′||H′8′

[0036] Among them, H′ j ′(j=1~8) is the output of the j-th head perception attention module, with H″ and adjacency matrix A as input.

[0037] H′ j The formula for calculating ′ is:

[0038] H′ j ′=α′ j W j 'AH'

[0039] Where, α′ j =SPEattention(H″), W j ′ represents the learnable weight.

[0040] Furthermore, step S3 specifically involves: aggregating and connecting the O obtained from the first branch and the H″′ obtained from the second branch to obtain the first node representation X. in The specific formula is: X in =O||H″′.

[0041] Furthermore, in step S4, the first node is represented as X. in The second node representation is obtained by processing the elastic graph neural network with a low-pass filter. The specific steps are as follows:

[0042] S401, with X in As input, based on l 21 Regularization yields the graph signal smoothing formula, the specific formula is as follows:

[0043]

[0044] Where n is the number of nodes; d is the value of X. in The characteristic dimension of the signal; F is the final output signal; λ1 and λ2 are hyperparameters controlling the balance of graph smoothness; L represents the graph Laplacian matrix. This represents the regularized graph Laplacian matrix. I represents the identity matrix. This represents the regularized graph adjacency matrix. Let A be the degree matrix of the graph and A be the adjacency matrix of the graph. yes l 21 Norm, express The trace, where ε represents the equilibrium parameter. V iV represents the i-th vertex in the graph showing the relationship between vertices for each document. j This indicates that each document corresponds to the j-th vertex in the graph of the relationships between vertices, and d i d represents the degree of the i-th vertex in the graph of the relationship between vertices corresponding to each document. j F represents the degree of the j-th vertex in the graph of the relationship between vertices corresponding to each document. i and F j These are the i-th and j-th output signals of F, respectively. The regularization term section introduces low-pass filtering functionality.

[0045] S402. After 10 iterations, the final output signal F is used to obtain the final document classification result. The result is then input into the normalized exponential function Softmax to obtain the final document classification category.

[0046] Furthermore, this invention also proposes a document classification system based on elastic graph neural networks, comprising:

[0047] The matrix construction module is used to extract feature information from documents and construct feature matrices and adjacency matrices that represent the relationships between documents.

[0048] The matrix processing module is used to process the feature matrix and the adjacency matrix by inputting them into two parallel branches.

[0049] The first node representation acquisition module is used to aggregate and connect the results obtained from the processing of the first branch and the second branch to obtain the first node representation.

[0050] The second node representation acquisition module is used to process the first node representation through an elastic graph neural network with a low-pass filter to obtain the second node representation.

[0051] The classification results module is used to input the second node representation into the softmax function to obtain the final classification result of the node.

[0052] Furthermore, the specific steps of the processing flow in the matrix processing module are as follows:

[0053] (1) The specific data processing flow of the first branch is as follows:

[0054] After processing by the first multilayer perceptron and the first activation function, the specific formula is as follows:

[0055] H = ReLU(XW) h +b h );

[0056] Where X is the feature matrix corresponding to the literature, H is the output of the hidden layer, and ReLU is the linear rectified function.

[0057] W h ∈R n×h b h ∈R m×h denoted as the weights and biases of the hidden layer, respectively; m is the number of document nodes; n is the number of features; and h is the number of neurons in the hidden layer.

[0058] After processing by the second layer of the perceptron, the specific formula is as follows:

[0059] O = HW y +b y

[0060] Where O is the obtained output vector, W y ∈R h×i b y ∈R m×i denoted as the weights and biases of the output layer, respectively, and i represents the feature dimension of the output Y.

[0061] (2) The specific data processing flow of the second branch is as follows:

[0062] The feature matrix X and the adjacency matrix A are input into the first multi-head perception attention module to obtain the output H′. The first multi-head perception attention module is an 8-head perception attention module, and the specific formula is as follows:

[0063] H′=H′1||H′2||H′3||H′4||H′5||H′6||H′7||H′8

[0064] Among them, H′ j It is the output of the feature matrix X and the adjacency matrix A through input to the j-th head perception attention module, j = 1 to 8, and | is a parallel operation.

[0065] H′ j The calculation formula is:

[0066] H′ j =α j W j AX

[0067] Where, α j =SPEattention(X), FPEattention(·) uses a self-attention mechanism with spatial embedding, W j These are learnable weights.

[0068] Self-attention values ​​with spatial location embedding incorporate spatial location information into the self-attention mechanism, as shown in the following expression:

[0069] SPEattention(X)=(softmax(FC1(X)FC2(X))+ReLU(CNN(E)))FC3(X)

[0070] Where Softmax is the normalized exponential function, FC1 is the first fully connected function, FC2 is the second fully connected function, FC3 is the third fully connected function, ReLU is the activation function in the self-attention mechanism with spatial location embedding, CNN is a one-dimensional convolution function, and E is a learnable vector.

[0071] Input H′ sequentially into the third multilayer perceptron and the second activation function to obtain the output H″, as shown in the specific formula:

[0072] H″=ReLU(H′W′+b′)

[0073] Where W′ is the learnable weight and b′ is the learnable bias.

[0074] Input H″ into the second multi-head perception attention module to obtain the output H″′. The second multi-head perception attention module is an 8-head perception attention module, and the specific formula is as follows:

[0075] H″′=H′1′||H′2′||H′3′||H′4′||H′5′||H′6′||H′7′||H′8′

[0076] Among them, H′ j ′ is the output of the j-th head perception attention module, with H″ and adjacency matrix A as input.

[0077] H′ j The formula for calculating ′ is:

[0078] H′ j ′=α′ j W j 'AH'

[0079] Where, α′ j =SPEattention(H″), W j ′ represents the learnable weight.

[0080] Furthermore, in the first node representation acquisition module, the specific content is as follows: the O obtained from the first branch and the H″′ obtained from the second branch are aggregated and connected to obtain the first node representation X. in The specific formula is: X in =O||H″′.

[0081] Furthermore, in the second node representation acquisition module, the specific steps are as follows:

[0082] Step 1, with X in As input, based on l21 Regularization yields the graph signal smoothing formula, the specific formula is as follows:

[0083]

[0084] Where n is the number of nodes; d is the value of X. in The characteristic dimension of the signal; F is the final output signal; λ1 and λ2 are hyperparameters controlling the balance of graph smoothness; L represents the graph Laplacian matrix. This represents the regularized graph Laplacian matrix. I represents the identity matrix. This represents the regularized graph adjacency matrix. Let A be the degree matrix of the graph and A be the adjacency matrix of the graph. yes l 21 Norm; express The trace; ε represents the equilibrium parameter; V i V represents the i-th vertex in the graph showing the relationship between vertices for each document. j This indicates that each document corresponds to the j-th vertex in the graph of the relationships between vertices, and d i d represents the degree of the i-th vertex in the graph of the relationship between vertices corresponding to each document. j F represents the degree of the j-th vertex in the graph of the relationship between vertices corresponding to each document. i and F j These are the i-th and j-th output signals of F, respectively; The regularization term section introduces low-pass filtering functionality.

[0085] Step 2: Iterate the final output signal F 10 times to obtain the final document classification result, and input it into the normalized exponential function Softmax to obtain the final document classification category.

[0086] Furthermore, the present invention also proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the document classification method based on an elastic graph neural network described above.

[0087] Compared with the prior art, the present invention, employing the above technical solution, has the following significant technical advantages:

[0088] This invention proposes a document node classification method based on elastic graph neural networks. This method introduces two parallel paths to obtain the features of the corresponding document nodes. One path consists of two concatenated MLPs; the other consists of two concatenated multi-head attention networks. The features obtained from both paths are simultaneously input into an elastic neural network with a low-pass filter based on the L21 norm, thereby completing the document classification task. This method achieves excellent results in both accuracy and recall.

[0089] Meanwhile, this invention not only further enhances the local smoothness and adaptive capability of GNN, but also solves the problem that the attention function of any query node always has a fixed neighbor score in the traditional attention mechanism, and enhances spatial awareness by introducing spatial location information into the self-attention mechanism. Attached Figure Description

[0090] Figure 1 This is a schematic diagram of the overall implementation process of the present invention. Detailed Implementation

[0091] The technical solution of the present invention will now be clearly and completely described in conjunction with the accompanying drawings and specific embodiments. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention are within the scope of protection of the present invention.

[0092] Literature recommendation systems recommend different documents to different people. Therefore, document classification categorizes existing documents, such as into categories like science and technology, sports, literature, and history, and then recommends these categories to different groups of people, greatly improving the efficiency of document recommendation. Taking science and technology documents as an example, the main processing steps for document classification are as follows: First, we obtain the feature vector representing the content of the document, the label representing the category (science and technology), and the adjacency matrix representing the relationships between different documents. Second, we input thousands of data points into the model. Then, after training, we obtain a better model. Finally, we can input any feature vector representing the content of a document into the model, which can accurately output its corresponding document category, such as science and technology, sports, literature, and history.

[0093] This invention proposes a document node classification method based on graph neural networks, such as... Figure 1 As shown, it includes:

[0094] S1. Extract the feature information of the documents, such as the title, author, publisher or research institution, source, report number, patent number, subject terms, keywords, classification number, and characteristic words, and construct a feature matrix and an adjacency matrix that represents the relationship between the documents.

[0095] S2. Input the feature matrix and adjacency matrix into two parallel branches for processing.

[0096] (1) The first branch passes through the first multilayer perceptron, the first activation function ReLU, and the second multilayer perceptron, respectively. The specific content is as follows:

[0097] After processing by the first multilayer perceptron and the first activation function, the specific formula is as follows:

[0098] H = ReLU(XW) h +b h );

[0099] Where H is the output of the hidden layer, ReLU is the rectified linear function, and W... h ∈R n×h b h ∈R m×h denoted as the weights and biases of the hidden layer, respectively, and h is the number of neurons in the hidden layer.

[0100] After processing by the second layer of the perceptron, the specific formula is as follows:

[0101] O = HW y +b y

[0102] Where O is the obtained output vector, W y ∈R h×i b y ∈R m×i denoted as the weights and biases of the output layer, respectively, and i represents the feature dimension of the output Y.

[0103] (2) The second branch passes through the first multi-head perception attention module, the third multilayer perceptron, the second activation function, and the second multi-head perception attention module, respectively. The specific content is as follows:

[0104] The feature matrix X and the adjacency matrix A are input into the first multi-head perception attention module to obtain the output H′. The first multi-head perception attention module is an 8-head perception attention module, and the specific formula is as follows:

[0105] H′=H′1||H′2||H′3||H′4||H′5||H′6||H′7||H′8

[0106] Among them, H′ j (j = 1~8) is the output of the feature matrix X and the adjacency matrix A through input to the j-th head perception attention module, |

[0107] This is a parallel operation.

[0108] H′ j The calculation formula is:

[0109] H′j =α j W j AX

[0110] Where, α j =SPEattention(X), SPEattention(·) uses a self-attention mechanism with spatial embedding, W j These are learnable weights.

[0111] Self-attention values ​​with spatial location embedding incorporate spatial location information into the self-attention mechanism, as shown in the following expression:

[0112] SPEattention(X)=(softmax(FC1(X)FC2(X))+ReLU(CNN(E)))FC3(X)

[0113] Where Softmax is the normalized exponential function, FC1 is the first fully connected function, FC2 is the second fully connected function, FC3 is the third fully connected function, ReLU is the activation function in the self-attention mechanism with spatial location embedding, CNN is a one-dimensional convolution function, and E is a learnable vector.

[0114] Input H′ sequentially into the third multilayer perceptron and the second activation function to obtain the output H″, as shown in the specific formula:

[0115] H″=ReLU(H′W′+b′)

[0116] Where W′ is the learnable weight and b′ is the learnable bias.

[0117] Input H″ into the second multi-head perception attention module to obtain the output H″′. The second multi-head perception attention module is an 8-head perception attention module, and the specific formula is as follows:

[0118] H″′=H′1′||H′2′||H′3′||H′4′||H′5′||H′6′||H′7′||H′8′

[0119] Among them, H′ j ′(j=1~8) is the output of the j-th head perception attention module, with H″ and adjacency matrix A as input.

[0120] H′ j The formula for calculating ′ is:

[0121] H′ j ′=α′ j W j 'AH'

[0122] Where, α′ j=SPEattention(H″), W j ′ represents the learnable weight.

[0123] S3. Aggregate and connect the results obtained from the first branch and the second branch to obtain the representation of the first node. The specific content is as follows:

[0124] The first node representation X is obtained by aggregating and connecting the O obtained from the first branch and the H″′ obtained from the second branch. in The specific formula is: X in =O||H″′.

[0125] S4. The representation of the first node is processed by an elastic graph neural network with a low-pass filter to obtain the representation of the second node. The specific steps are as follows:

[0126] S401, with X in As input, based on l 21 Regularization yields the graph signal smoothing formula, the specific formula is as follows:

[0127]

[0128] Where n is the number of nodes; d is the value of X. in The characteristic dimension of the signal; F is the final output signal; λ1 and λ2 are hyperparameters controlling the balance of graph smoothness; L represents the graph Laplacian matrix. Let L = I - A, where I represents the identity matrix. This represents the regularized graph adjacency matrix. Let A represent the degree matrix of the graph, and let A be the adjacency matrix of the graph. yes l 21 Norm; express The trace; ε represents the equilibrium parameter; V i V represents the i-th vertex in the graph showing the relationship between vertices for each document. j This indicates that each document corresponds to the j-th vertex in the graph of the relationships between vertices, and d i d represents the degree of the i-th vertex in the graph of the relationship between vertices corresponding to each document. j F represents the degree of the j-th vertex in the graph of the relationship between vertices corresponding to each document. i and F j These are the i-th and j-th output signals of F, respectively; The regularization term section introduces low-pass filtering functionality.

[0129] S402. After 10 iterations, the final output signal F is used to obtain the final document classification result. The result is then input into the normalized exponential function Softmax to obtain the final document classification category.

[0130] S5. Input the representation of the second node into the softmax function to obtain the final classification result of the node.

[0131] Low-pass filters are typically used to smooth the eigenvectors of adjacency matrices to better reflect the local information of document nodes. Specifically, for each document node, each element in the eigenvector of the adjacency matrix corresponds to the relationship between that node and other nodes. The low-pass filter reduces noise by smoothing these elements, thus better reflecting the similarity between nodes.

[0132] Compared with L2-based methods, L2-based methods 21 The method penalizes large values ​​less, thus better preserving discontinuous or non-smooth signals. Theoretically, based on L... 21 The approach tends to promote signal sparsity to improve discontinuity.

[0133] For L2-based graph smoothing, the message-passing scheme can be derived from the gradient descent iteration of the graph signal denoising problem. However, due to the addition of L2... 21 The non-smoothness of the smoothed graph makes computational estimation more challenging. The elastic message passing mechanism proposes a computationally efficient primal-dual algorithm to solve the equivalent saddle point problem.

[0134] This embodiment uses the PyTorch library in Python to experiment with the proposed EDGNN model. In the experiment, all models were initially set to three layers. The datasets used were Cora, CiteSeer, PubMed, CS, and Physics. The first three datasets came from citation networks, with nodes representing papers. The latter two datasets came from co-author datasets, with nodes representing authors; co-authors were connected by edges. This invention aims to classify documents according to certain relationships, achieving a node classification task (document retrieval).

[0135] This embodiment selects seven methods for comparison to evaluate the effectiveness of the representations learned by EDGNN in a dual-path structure. As shown in Table 1, the accuracy of node classification is used as a quantitative indicator to evaluate various models, with the best performance indicated in bold. Different models perform differently on different datasets. Generally speaking, ElasticGNN represents the latest research in this field and outperforms the other six models in all aspects.

[0136] Table 1 Quantitative evaluation of each method

[0137] Model Cora CiteSeer PubMed CS Physics ChebNet 76.3±1.5 67.4±1.5 75.0±2.0 91.8±0.4 OMM GCN 79,6±1.1 68.9±1.2 77.6±2.3 91.6±0.6 93.3±0.8 GAT 80.1±1.2 68.9±1.8 77.6±2.2 91.1±0.5 93.3±0.7 SGC 80.2±1.5 68.9±1.3 75.5±2.9 90.1±1.3 93.1±0.6 APPNP 82.2±1.3 70.4±1.2 78.9±2.2 92.5±0.3 93.7±0.7 GraphSAGE 79.0±1.1 67.5±2.0 77.6±2.0 91.7±0.5 92.5±0.8 ElasticGNN 82.7±1.0 70.9±1.4 79.4±1.8 92.5±0.3 94.2±0.5 EDGNN 83.2±0.2 70.7±0.7 80.0±1.3 93.5±0.2 94.7±0.4

[0138] The following specific conclusions can be drawn from Table 1:

[0139] (1) The EDGNN proposed in this invention consistently outperforms all baselines on most datasets, demonstrating the high efficiency of EDGNN in node classification tasks.

[0140] (2) Compared with the above six models, EDGNN outperforms all six models (including GAT and ElasticGNN, which we use as the model base), only slightly underperforming ElasticGNN on the CiteSeer dataset. GATv2 uses a dynamic attention mechanism as its base model, while ElasticGNN uses a resilient message passing mechanism on top of MLP. The EDGNN model proposed in this invention uses a dual-path network combining GATv2 and MLP as its base model, and adds a resilient message passing mechanism with an added low-pass filter. From the above results, it can be seen that our network performs well on all five datasets in the graph node classification task, and the stability of classification is also improved.

[0141] Through the above comparison, it can be determined that the model proposed in this invention outperforms other baseline models in all performance indicators, achieving the best node classification effect.

[0142] In summary, this invention learns the features of nodes in a graph by employing a first multilayer perceptron, a first activation function ReLU, a first branch of the second multilayer perceptron, and a second branch of the second multilayer perceptron, comprising a first multi-head attention module, a third multilayer perceptron, a second activation function, and a second multi-head attention module. The results from the first and second branches are then aggregated and connected to enhance the interactivity of the nodes. Furthermore, based on the classic elastic graph neural network, this invention introduces low-pass filtering and the l21 norm to further enhance the learning ability of the graph neural network, incorporating spatial location information into the self-attention mechanism to enhance its spatial awareness.

[0143] This invention also proposes a document classification system based on an elastic graph neural network, including a matrix construction module, a matrix processing module, a first node representation acquisition module, a second node representation acquisition module, a classification result module, and a computer program executable on a processor. It should be noted that each module in the above system corresponds to a specific step of the method provided in this invention, possessing the corresponding functional modules and beneficial effects for executing the method. Technical details not described in detail in this embodiment can be found in the method provided in this invention.

[0144] This invention also proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. It should be noted that each module in the above system corresponds to a specific step of the method provided in this invention, possessing the corresponding functional modules and beneficial effects for executing the method. Technical details not described in detail in this embodiment can be found in the method provided in this invention.

[0145] The above description is merely a specific embodiment of the present invention and is only used to help understand the principles and core ideas of the present invention. It is not intended to limit the present invention. Within the spirit and principles of the present invention, any modifications or equivalent substitutions made by those skilled in the art to the technical solutions described in the present invention should be included within the protection scope of the present invention.

Claims

1. A document classification method based on an elastic graph neural network, characterized in that, Comprise: S1, extract the feature information of the literature, construct a feature matrix and an adjacency matrix representing the relationship between each literature; S2, input the feature matrix and the adjacency matrix into two parallel branches for processing, wherein the first branch passes through a first multi-layer perception, a first activation function and a second multi-layer perception; The second branch passes through a first multi-head perception attention module, a third multi-layer perception, a second activation function, a second multi-head perception attention module; S3, aggregate the results obtained by processing the first branch and the second branch to obtain a first node representation; S4, process the first node representation through an elastic graph neural network with a low-pass filter function to obtain a second node representation; Specifically: S401、with X in based on l 21 regularization to obtain a graph signal smoothing formula, and the specific formula is: where n is the number of nodes; d is X in dimension of the signal; F is the final output signal; λ1 and λ2 are the hyperparameters that control the balance of the smoothness of the graph; L represents the graph Laplacian matrix, denotes the regularized graph Laplacian matrix, I represents the unit matrix, denotes the regularized graph adjacency matrix, denotes the degree matrix of the graph, A is the graph adjacency matrix; is the l 21 norm of denotes the trace of V i denotes the ith vertex of the inter-document association graph corresponding to each document, V j denotes the jth vertex of the inter-document association graph corresponding to each document, d i denotes the degree of the ith vertex in the inter-document association graph corresponding to each document, d j denotes the degree of the jth vertex in the inter-document association graph corresponding to each document, F i and F j are the ith and jth output signals of F, respectively; is the regularization term part for introducing the low-pass filtering function; S402, iterate the final output signal F for 10 times to obtain the final literature classification result, and input it into the normalized exponential function Softmax to obtain the final literature classification category; S5, input the second node representation into the softmax function to obtain the final classification result of the node. 2.The document classification method based on the elastic graph neural network according to claim 1, characterized in that, In step S1: assuming that the literature has m nodes and n features, the obtained literature corresponding feature matrix is X ∈ R m×n The literature corresponding adjacency matrix is A ∈ R n×n . 3.The method of claim 1, wherein, The specific steps of the processing flow in step S2 are: (1) The specific data processing flow of the first branch is: After processing by the first multi-layer perception and the first activation function, the specific formula is: H = ReLU(XW h + b h ); where X is the corresponding feature matrix of the literature, H is the output of the hidden layer, ReLU is the linear rectification function, W h ∈R n×h , b h ∈R m×h are the weights and biases of the hidden layer, respectively, m is the number of literature nodes, n is the number of features, and h is the number of neurons in the hidden layer. After processing by the second multi-layer perception, the specific formula is: O = HW y + b y where O is the output vector obtained, W y ∈R h×i , b y ∈R m×i are the weights and biases of the output layer, respectively, and i is the feature dimension of the output Y. (2) The specific data processing flow of the second branch is: Input the feature matrix X and the adjacency matrix A into the first multi-head perception attention module to obtain the output H', the first multi-head perception attention module is an 8-head perception attention module, and the specific formula is: H' = H'1||H'2||H'3||H'4||H'5||H'6||H'7||H'8 where H' = H + A j is the feature matrix X and the adjacency matrix A through the input of the first j head perception attention module output, j = 1 ~ 8, | is a parallel operation; H' j The calculation formula is: H' j = a j W j AX wherein, a j = SPE attention (X), SPE attention (·) selects a self-attention mechanism with spatial position embedding, W j is a learnable weight; The self-attention value with spatial position embedding introduces spatial position information into the self-attention mechanism, and the expression is as follows: SPEattention(X) = (softmax(FC1(X)FC2(X))+ReLU(CNN(E)))FC3(X) Where, Softmax is a normalized exponential function, FC1 is a first full connection function, FC2 is a second full connection function, FC3 is a third full connection function, ReLU is an activation function in the self-attention mechanism with spatial position embedding, CNN is a one-dimensional convolution function, and E is a learnable vector; Input H' into the third multi-layer perception and the second activation function in turn to obtain the output H'', and the specific formula is: H'' = ReLU(H'W'+b') Where, W' is a learnable weight, and b' is a learnable bias; Input H'' into the second multi-head perception attention module to obtain the output H''', and the second multi-head perception attention module is an 8-head perception attention module, and the specific formula is: H''' = H''1||H''2||H''3||H''4||H''5||H''6||H''7||H''8 where H" = H' - A j is the output of the jth head attention module through inputting H" and the adjacent matrix A. H" j The calculation formula is: H" j = a' j W' j AH" where a' = a + a', and j = SPEattention(H"), W' = W + W', and j are learnable weights. 4.The method of claim 3, wherein, The specific content of step S3 is: the O obtained by the first branch and the H''' obtained by the second branch are connected to obtain the first node representation X in , and the specific formula is: X in = O||H''' 5. A document classification system based on elastic graph neural networks, characterized in that, Comprise: A matrix construction module for extracting feature information of the literature, constructing a feature matrix and an adjacency matrix representing the relationship between each literature; A matrix processing module for inputting the feature matrix and the adjacency matrix into two parallel branches for processing; The first node representation obtaining module is configured to aggregate and connect the results obtained by the first branch and the second branch to obtain a first node representation; The second node representation obtaining module is configured to process the first node representation through an elastic graph neural network with a low-pass filter function to obtain a second node representation; specifically, Step 1, with X in as input, based on l 21 regularization to obtain a graph signal smoothing formula, the specific formula is: where n is the number of nodes; d is X in dimension of the signal; F is the final output signal; λ1 and λ2 are hyperparameters that control the balance of the smoothness of the graph; L represents the graph Laplacian matrix, regularized graph Laplacian matrix, I represents the unit matrix, regularized graph adjacency matrix, D represents the degree matrix of the graph, and A represents the graph adjacency matrix; is the l 21 norm of ; represents the trace of ; ε represents the balance parameter; V i represents the ith vertex of the inter-vertex association graph corresponding to each document, respectively, V j represents the jth vertex of the inter-vertex association graph corresponding to each document, respectively, d i represents the degree of the ith vertex in the inter-vertex association graph corresponding to each document, respectively, d j represents the degree of the jth vertex in the inter-vertex association graph corresponding to each document, respectively, F i and F j are the ith and jth output signals of F, respectively; is a regularization term part for introducing a low-pass filtering function; Step 2: The final output signal F is iterated for 10 times to obtain a final literature classification result, which is input into a normalization exponential function Softmax to obtain a final literature classification category; The classification result module is configured to input the second node representation into a softmax function to obtain a final classification result of the node.

6. The elastic graph neural network-based document classification system of claim 5, wherein, In the matrix processing module, the specific steps of the processing procedure are as follows: (1) The specific steps of the first branch data processing procedure are as follows: After being processed by the first multi-layer perception and the first activation function, the specific formula is as follows: H = ReLU(XW h + b h ); where X is the corresponding feature matrix of the literature, H is the output of the hidden layer, ReLU is the linear rectification function, W h ∈R n×h , b h ∈R m×h are the weights and biases of the hidden layer, respectively, m is the number of literature nodes, n is the number of features, and h is the number of neurons in the hidden layer. After being processed by the second multi-layer perception, the specific formula is as follows: O = HW y + b y where O is the output vector obtained, W y ∈R h×i , b y ∈R m×i are the weight and bias of the output layer, respectively, and i is the feature dimension of the output Y. (2) The specific steps of the second branch data processing procedure are as follows: The feature matrix X and the adjacency matrix A are input into the first multi-head perception attention module to obtain an output H', the first multi-head perception attention module is an 8-head perception attention module, and the specific formula is as follows: H' = H'1||H'2||H'3||H'4||H'5||H'6||H'7||H'8 where H' = H + A j is the output of the input jth head perceptual attention module, j = 1 ~ 8, || is a parallel operation. H' j The calculation formula is: H' j = a j W j AX wherein, a j = SPEattention(X), SPEattention(·) selects a self-attention mechanism with spatial position embedding, W j is a learnable weight; The self-attention value with spatial position embedding introduces spatial position information into the self-attention mechanism, and the expression is as follows: SPEattention(X) = (softmax(FC1(X)FC2(X))+ReLU(CNN(E)))FC3(X) Wherein, Softmax is a normalization exponential function, FC1 is a first full connection function, FC2 is a second full connection function, FC3 is a third full connection function, ReLU is an activation function in the self-attention mechanism with spatial position embedding, CNN is a one-dimensional convolution function, and E is a learnable vector; H' is input into the third multi-layer perception and the second activation function in sequence to obtain an output H'', and the specific formula is as follows: H'' = ReLU(H'W'+b') Wherein, W' is a learnable weight, and b' is a learnable bias; H'' is input into the second multi-head perception attention module to obtain an output H''', the second multi-head perception attention module is an 8-head perception attention module, and the specific formula is as follows: H''' = H''1||H''2||H''3||H''4||H''5||H''6||H''7||H''8 where H" = H' * A j is the output of the jth head attention module through inputting H" and the adjacent matrix A. H" j The calculation formula is: H" j = a' j W j 'AH" where a' = a + a', and j = SPEattention(H"), W' = W + W', and j are learnable weights.

7. The elastic graph neural network-based document classification system of claim 5, wherein, In the first node representation obtaining module, the specific content is: the O obtained by the first branch and the H'" obtained by the second branch are connected to obtain the first node representation X in , and the specific formula is: X in =O||H'" 8. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the steps of the method in any one of claims 1 to 4.

Images