A hierarchical graph enhancement based graph classification method and device

Abstract

The application provides a kind of based on hierarchical graph enhancement's graph classification method, this method includes: obtaining original graph sample;Random edge deletion method and node feature enhancement method based on mask mechanism are successively used, and structure and node feature enhanced graph are obtained;For the same original graph, structure and node feature enhancement are executed twice, and two structure and node feature enhanced graphs are obtained as two hierarchical graph enhancement views;Two hierarchical graph enhancement views are input into graph encoder, and node representation, global graph representation and negative sample node representation are obtained;Local-global contrast learning loss is constructed;Two node representations are spliced, and graph-level aggregation is performed on all node representations;Graph classification is completed, and classification results are output.Compared with the prior art, the application can simultaneously improve the adaptability of the model to topological changes and feature missing, and construct different training views for graph contrast learning, thereby improving the stability and classification ability of graph representation.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence, machine learning, graph neural networks and graph data processing technology, and specifically to a graph classification method based on hierarchical graph augmentation. Background Technology

[0002] Graph-structured data is widely used in scenarios such as social networks, citation networks, knowledge graphs, bioinformatics networks, and recommender systems. Graph neural networks obtain node representations or graph representations by aggregating information about nodes and their neighborhoods, and are used for tasks such as classification, prediction, and clustering.

[0003] Traditional graph neural networks typically rely on labeled data, but labeling graph data is costly. Graph contrastive learning enhances the original graph to construct semantically consistent and differentiated graph views, and then uses the contrastive relationships between these views to train a graph encoder. Existing enhancement methods mainly include structural enhancement by changing topological relationships and feature enhancement by changing node attributes.

[0004] Existing methods typically apply structural enhancement and feature enhancement independently or in parallel to the original graph, lacking a sequential, staged approach. This can easily lead to insufficient view differentiation or loss of effective semantics. Therefore, a hierarchical graph enhancement method is needed, which first enhances structure and then enhances features, to improve the stability and classification ability of graph representations.

[0005] In summary, there is an urgent need to propose a graph classification method based on hierarchical graph augmentation to solve the above problems. Summary of the Invention

[0006] In view of this, the main objective of the present invention is to provide a vehicle detection method based on associated target boxes, which can achieve both accuracy and speed in vehicle detection.

[0007] To achieve the above objectives, according to a first aspect of the present invention, a graph classification method based on hierarchical graph enhancement is provided, the method comprising: The first step is to obtain the original image sample; The second step involves using a random edge deletion method to enhance the structure of the original graph, resulting in a structure-enhanced graph. Then, a node feature enhancement method based on a masking mechanism is used to enhance the node features of the structure-enhanced graph, yielding both structure and node feature-enhanced graphs. The third step involves performing the second step twice on the same original graph to obtain two enhanced graphs of structure and node features, which serve as the first and second hierarchical graph enhancement views. The fourth step involves inputting the two hierarchical graph augmentation views into the graph encoder to obtain node representations, global graph representations, and negative sample node representations; and then constructing a local-global contrastive learning loss. The fifth step is to concatenate the two node representations and perform graph-level aggregation on all node representations; complete the graph classification and output the classification results.

[0008] In the first step, the original image sample is represented as: G = (A, X). This is an adjacency matrix used to represent the connection relationships between nodes; is the node feature matrix; N represents the number of nodes; F represents the node feature dimension.

[0009] Furthermore, the second step includes: The structure enhancement step obtains the edge set of the original graph G: M represents the number of edges in the original graph; for nodes in the original graph With nodes The edge between Randomly sampled Bernoulli variables: ,in Construct an edge mask matrix from all Bernoulli variables: The adjacency matrix after structural enhancement is represented as: This yields the structural enhancement diagram: ; wherein This represents the probability of edge deletion. The value range is 0 to 1; Represents a node With nodes The edges between them retain mask variables; The node feature enhancement step assumes the original graph's node set is: Randomly select a subset of nodes from the node set V as feature occlusion nodes to obtain the occlusion node set: The number of shading nodes is represented as: ,in Indicates the occlusion ratio of node features; for any node The enhanced node features are represented as follows: ,in Represents a node The original feature vector, Representing the mask feature vector, the feature enhancement matrix is ​​formed by assembling all the enhanced node features. Obtain the enhanced feature map of structure and nodes: ;in .

[0010] The third step includes: applying the same edge deletion probability to the same original graph G=(A,X). and node feature occlusion ratio Using independent random sampling results, the second step is performed twice to obtain two structure and node feature enhancement maps: and Each serves as an enhanced view of the first-level graph. Second-level graph enhancement view .

[0011] The fourth step includes: The nodes represent the steps to obtain the view, enhancing the first-level graph. Second-level graph enhancement view Input the encoder of the first image respectively Second Figure Encoder ,in and These represent the node representations corresponding to the first-level and second-level graph augmented views, respectively. and The parameters are respectively and The image encoder, and These represent the learnable parameters of the first encoder and the second encoder, respectively. The steps for obtaining global graph representation and negative sample node representation are as follows: Construct global graph representation and negative sample representation; perform average aggregation on the node representations of the first-level and second-level graph augmented views respectively; and obtain the first global graph representation through a projection layer. Second global graph representation ,in and Representing nodes respectively The nodes are represented in both views, where N represents the number of nodes. This represents a projection module consisting of a linear transformation and a nonlinear activation function. While keeping the adjacency matrices of the two hierarchical graph enhancement views unchanged, the node arrangement order of their node feature matrices is randomly permuted. Let P be the node random permutation matrix, then the negative sample node matrices are as follows: and ,in and These represent the node feature matrices corresponding to the two hierarchical graph augmentation views, respectively. The negative sample feature matrix and its corresponding augmentation adjacency matrix are input into the respective graph encoders to obtain the negative sample node representations. and ,in and These represent the adjacency matrices corresponding to the two hierarchical graph enhancement views; The local-global contrastive learning step constructs the local-global contrastive learning loss based on the node representations of the augmented view, the global graph representation, and the negative sample node representations from two hierarchical graphs: The loss is a contrastive learning loss based on Jensen–Shannon divergence.

[0012] The first and second graph encoders both employ two-layer graph convolutional networks with PReLU activation function, and the hidden layer dimensions are manually set. This represents a local-global contrastive loss function built based on Jensen–Shannon divergence, used to measure the difference between positive and negative sample relationships; This represents the contrastive learning loss value obtained by inputting the node representation, global graph representation, and negative sample node representation into the loss function. During model training, this loss value is minimized. Optimize the parameters of the graph encoder.

[0013] The fifth step includes: The two nodes represent the concatenation step, and the first and second node representations are concatenated along the feature dimension: , This indicates a feature dimension concatenation operation, where Z represents the final node representation. In the image aggregation step, for any image to be classified, graph-level aggregation is performed on the final node representation Z to obtain a graph-level representation. , This represents the graph-level representation of the graph to be classified. This represents a graph-level aggregation function, where the graph to be classified is the original graph sample whose graph-level category needs to be determined; The graph classification and result output steps involve inputting the graph-level representation into the graph classifier. The predicted category of the image to be classified is obtained and output, where A graph classifier is used to classify graphs based on their graph-level representations. Calculate the prediction results for each category. This represents the final predicted category label of the image to be classified.

[0014] According to another aspect of the present invention, a graph classification apparatus based on hierarchical graph enhancement is provided, the apparatus comprising: The original image sample acquisition module is used to acquire original image samples; The module for obtaining structure and node feature enhancement graphs uses a random edge deletion method to enhance the structure of the original graph and obtain a structure-enhanced graph; it also uses a node feature enhancement method based on a masking mechanism to enhance the node features of the structure-enhanced graph and obtain both structure and node feature enhancement graphs. Two hierarchical graph enhancement view acquisition modules are used to perform the second step twice for the same original graph to obtain two enhanced graphs of structure and node features, which serve as the first and second hierarchical graph enhancement views. The graph encoding and local-global contrastive learning module is used to input two hierarchical graph augmentation views into the graph encoder to obtain node representations, global graph representations, and negative sample node representations; and to construct the local-global contrastive learning loss. The graph aggregation and graph classification modules are used to concatenate two node representations, perform graph-level aggregation of all node representations, complete graph classification, and output the classification results.

[0015] In the original image sample acquisition module, the original image sample is represented as: G = (A, X). This is an adjacency matrix used to represent the connection relationships between nodes; is the node feature matrix; N represents the number of nodes; F represents the node feature dimension.

[0016] Furthermore, the structure and node feature enhancement graph acquisition module includes: The structure enhancement module is used to obtain the edge set of the original graph G: M represents the number of edges in the original graph; for nodes in the original graph With nodes The edge between Randomly sampled Bernoulli variables: ,in Construct an edge mask matrix from all Bernoulli variables: The adjacency matrix after structural enhancement is represented as: This yields the structural enhancement diagram: ; wherein This represents the probability of edge deletion. The value range is 0 to 1; Represents a node With nodes The edges between them retain mask variables; The node feature enhancement module is used to define the node set of the original graph as follows: Randomly select a subset of nodes from the node set V as feature occlusion nodes to obtain the occlusion node set: The number of shading nodes is represented as: ,in Indicates the occlusion ratio of node features; for any node The enhanced node features are represented as follows: ,in Represents a node The original feature vector, Representing the mask feature vector, the feature enhancement matrix is ​​formed by assembling all the enhanced node features. Obtain enhanced feature maps of structure and nodes: ;in .

[0017] The two hierarchical graph enhancement view acquisition modules include: modules for applying the same edge deletion probability to the same original graph G=(A,X). and node feature occlusion ratio Using independent random sampling results, the structure and node feature enhancement map acquisition module is executed twice to obtain two structure and node feature enhancement maps: and Each serves as an enhanced view of the first-level graph. Second-level graph enhancement view .

[0018] The graph encoding and local-global contrastive learning module includes: The node represents the acquisition module, used to enhance the view of the first-level graph. Second-level graph enhancement view Input the encoder of the first image respectively Second Figure Encoder ,in and These represent the node representations corresponding to the first-level and second-level graph augmented views, respectively. and The parameters are respectively and The image encoder, and These represent the learnable parameters of the first encoder and the second encoder, respectively. The global graph representation and negative sample node representation acquisition module is used to construct the global graph representation and negative sample representation. It performs average aggregation on the node representations of the first-level and second-level graph augmented views, respectively, and obtains the first global graph representation through the projection layer. Second global graph representation ,in and Representing nodes respectively The nodes are represented in both views, where N represents the number of nodes. This represents a projection module consisting of a linear transformation and a nonlinear activation function. While keeping the adjacency matrices of the two hierarchical graph enhancement views unchanged, the node arrangement order of their node feature matrices is randomly permuted. Let P be the node random permutation matrix, then the negative sample node matrices are as follows: and ,in and These represent the node feature matrices corresponding to the two hierarchical graph augmentation views, respectively. The negative sample feature matrix and its corresponding augmentation adjacency matrix are input into the respective graph encoders to obtain the negative sample node representations. and ,in and These represent the adjacency matrices corresponding to the two hierarchical graph enhancement views; The local-global contrastive learning module is used to construct the local-global contrastive learning loss based on the node representations of the enhanced view, the global graph representation, and the negative sample node representations from two hierarchical graphs. The loss is a contrastive learning loss based on Jensen–Shannon divergence.

[0019] The first and second graph encoders both employ two-layer graph convolutional networks with PReLU activation function, and the hidden layer dimensions are manually set. This represents a local-global contrastive loss function built based on Jensen–Shannon divergence, used to measure the difference between positive and negative sample relationships; This represents the contrastive learning loss value obtained by inputting the node representation, global graph representation, and negative sample node representation into the loss function. During model training, this loss value is minimized. Optimize the parameters of the graph encoder.

[0020] The graph aggregation and graph classification module includes: The two nodes represent the concatenation module, used to concatenate the first and second node representations along the feature dimension: , This indicates a feature dimension concatenation operation, where Z represents the final node representation. The image aggregation module is used to perform graph-level aggregation on the final node representation Z for any image to be classified, to obtain a graph-level representation. , This represents the graph-level representation of the graph to be classified. This represents a graph-level aggregation function, where the graph to be classified is the original graph sample whose graph-level category needs to be determined; The graph classification and result output module is used to input the graph-level representation into the graph classifier. The predicted category of the image to be classified is obtained and output, where A graph classifier is used to classify graphs based on their graph-level representations. Calculate the prediction results for each category. This represents the final predicted category label of the image to be classified.

[0021] Compared with existing graph classification techniques, the graph classification method based on hierarchical graph augmentation of the present invention has the following advantages: it processes graph topology and node features in stages according to the order of first structural augmentation and then feature augmentation, which can simultaneously improve the model's adaptability to topological changes and feature loss, and construct a differentiated training view for graph contrastive learning, thereby improving the stability of graph representation and classification ability. Attached Figure Description

[0022] Figure 1 A flowchart of a graph classification method based on hierarchical graph enhancement according to the present invention is shown.

[0023] Figure 2 A framework diagram of a graph classification apparatus based on hierarchical graph enhancement according to the present invention is shown. Detailed Implementation

[0024] To enable those skilled in the art to further understand the structure, features and other objectives of the present invention, the following detailed description is provided in conjunction with the accompanying preferred embodiments. The preferred embodiments described are only for illustrating the technical solutions of the present invention and are not intended to limit the present invention.

[0025] Figure 1 A flowchart of a graph classification method based on hierarchical graph enhancement according to the present invention is provided. Figure 1 As shown, a graph classification method based on hierarchical graph enhancement according to the present invention includes: The first step, S1, is to obtain the original image sample; In the second step S2, a random edge deletion method is used to enhance the structure of the original graph to obtain a structure-enhanced graph; a node feature enhancement method based on a masking mechanism is used to enhance the node features of the structure-enhanced graph to obtain a structure and node feature-enhanced graph. The third step, S3, involves performing the second step twice for the same original graph to obtain two enhanced graphs of structure and node features, which serve as the first and second hierarchical graph enhanced views. In the fourth step S4, the two hierarchical graph augmentation views are input into the graph encoder to obtain node representations, global graph representations, and negative sample node representations; a local-global contrastive learning loss is then constructed. In step S5, the two node representations are concatenated, and all node representations are aggregated at the graph level; the graph classification is completed, and the classification results are output.

[0026] In the first step S1, the original image sample is represented as: G = (A, X). This is an adjacency matrix used to represent the connection relationships between nodes; Let N be the node feature matrix; N represents the number of nodes; and F represents the node feature dimension. Both the number of nodes N and the node feature dimension F are positive integers and can be determined based on the graph data to be processed. Further, the value of N ranges from 2708 to 19717, and the value of F ranges from 300 to 3703.

[0027] For example, the original graph sample contains 2708 nodes, i.e., N is 2708, and each node contains 1433-dimensional features, i.e., F is 1433.

[0028] Furthermore, the first step S1 may also include: preprocessing the original image sample.

[0029] The DropEdge method, a graph neural network data augmentation technique, generates a structurally perturbed view by randomly removing some edges during the training phase with probability. The GraphMAE-style node feature augmentation method, based on a masking mechanism, randomly or adaptively masks some elements in the node feature matrix, forcing the model to reconstruct or infer missing values ​​using contextual and structural information, thereby learning a more robust and generalizable node representation.

[0030] Further, the second step S2 includes: S21, the structure enhancement step, obtains the edge set of the original graph G: M represents the number of edges in the original graph; for nodes in the original graph With nodes The edge between Randomly sampled Bernoulli variables: ,in Construct an edge mask matrix from all Bernoulli variables: The adjacency matrix after structural enhancement is represented as: This yields the structural enhancement diagram: ; wherein This represents the probability of edge deletion. The value range is 0 to 1; Represents a node With nodes The edges between them retain mask variables; In node feature enhancement step S22, let the set of nodes in the original graph be: Randomly select a subset of nodes from the node set V as feature occlusion nodes to obtain the occlusion node set: The number of shading nodes is represented as: ,in Indicates the occlusion ratio of node features; for any node The enhanced node features are represented as follows: ,in Represents a node The original feature vector, Representing the mask feature vector, the feature enhancement matrix is ​​formed by assembling all the enhanced node features. Obtain the enhanced feature map of structure and nodes: ;in .

[0031] The third step S3 includes: applying the same edge deletion probability to the same original graph G=(A,X). and node feature occlusion ratio Using mutually independent random sampling results, the second step S2 is performed twice to obtain two structure and node feature enhancement maps: and Each serves as an enhanced view of the first-level graph. Second-level graph enhancement view .

[0032] For example, the Set to 0.1, the Set it to 0.2.

[0033] The fourth step S4 includes: Node representation acquisition step S41, enhancing the view of the first-level graph. Second-level graph enhancement view Input the encoder of the first image respectively Second Figure Encoder ,in and These represent the node representations corresponding to the first-level and second-level graph augmented views, respectively. and The parameters are respectively and The graph encoder, and These represent the learnable parameters of the first encoder and the second encoder, respectively. Step S42 involves obtaining the global graph representation and negative sample node representation. The global graph representation and negative sample representation are constructed by averaging and aggregating the node representations of the first-level and second-level graph augmented views, respectively, and then obtaining the first global graph representation through a projection layer. Second global graph representation ,in and Representing nodes respectively The nodes are represented in both views, where N represents the number of nodes. This represents a projection module consisting of a linear transformation and a nonlinear activation function. While keeping the adjacency matrices of the two hierarchical graph enhancement views unchanged, the node arrangement order of their node feature matrices is randomly permuted. Let P be the node random permutation matrix, then the negative sample node matrices are as follows: and ,in and These represent the node feature matrices corresponding to the two hierarchical graph augmentation views, respectively. The negative sample feature matrix and its corresponding augmentation adjacency matrix are input into the respective graph encoders to obtain the negative sample node representations. and ,in and These represent the adjacency matrices corresponding to the two hierarchical graph enhancement views; In the local-global contrastive learning step S43, the local-global contrastive learning loss is constructed based on the node representations of the augmented view, the global graph representation, and the negative sample node representations from the two hierarchical graphs. The loss is a contrastive learning loss based on Jensen–Shannon divergence.

[0034] The first and second graph encoders both employ two-layer graph convolutional networks with PReLU activation function, and the hidden layer dimensions are manually set. This represents a local-global contrastive loss function built based on Jensen–Shannon divergence, used to measure the difference between positive and negative sample relationships; This represents the contrastive learning loss value obtained by inputting the node representation, global graph representation, and negative sample node representation into the loss function. During model training, this loss value is minimized. Optimize the parameters of the graph encoder.

[0035] The first and second graph encoders both employ two-layer graph convolutional networks with the PReLU activation function, and the hidden layer dimension is manually set. and The initial values ​​are automatically generated during the network initialization process and automatically updated during model training via backpropagation and the Adam optimizer.

[0036] For example, the hidden layer dimension is set to 512. The local-global contrastive learning step S33 uses backpropagation to update the parameters of the graph encoder and projection module, employs the Adam optimizer, sets the learning rate to 0.0001, and trains for 200 epochs.

[0037] The fifth step S5 includes: The two node representations are joined in step S51, where the first and second node representations are joined along the feature dimension: , This indicates a feature dimension concatenation operation, where Z represents the final node representation. Image aggregation step S52: For any image to be classified, perform graph-level aggregation on the final node representation Z to obtain a graph-level representation. , This represents the graph-level representation of the graph to be classified. This represents a graph-level aggregation function, where the graph to be classified is the original graph sample whose graph-level category needs to be determined; In the graph classification and result output step S53, the graph-level representation is input into the graph classifier. The predicted category of the image to be classified is obtained and output, where A graph classifier is used to classify graphs based on their graph-level representations. Calculate the prediction results for each category. This represents the final predicted category label of the image to be classified.

[0038] Furthermore, when the graph-level aggregation function employs average pooling, the graph-level representation can be expressed as follows: N represents the number of nodes in the graph to be classified. Represents a node The corresponding fusion node representation.

[0039] For example, the graph classification and result output step S53 includes: the graph classifier maps the graph atlas representation to the predicted scores of each category through a fully connected layer, then obtains the predicted probabilities of each category through the Softmax function, and takes the category with the highest predicted probability as the final predicted category, and outputs the final predicted category.

[0040] Figure 2 A framework diagram of a graph classification apparatus based on hierarchical graph enhancement according to the present invention is given. For example... Figure 2 As shown, a graph classification apparatus based on hierarchical graph enhancement according to the present invention includes: Original image sample acquisition module 1 is used to acquire original image samples; The structure and node feature enhancement graph acquisition module 2 is used to perform structure enhancement on the original graph using a random edge deletion method to obtain a structure enhancement graph; and to perform node feature enhancement on the structure enhancement graph using a mask-based node feature enhancement method to obtain a structure and node feature enhancement graph. The two hierarchical graph enhancement view acquisition module 3 is used to perform the second step twice for the same original graph to obtain two structure and node feature enhancement graphs, which serve as the first hierarchical graph enhancement view and the second hierarchical graph enhancement view. Graph encoding and local-global contrastive learning module 4 is used to input two hierarchical graph augmentation views into the graph encoder to obtain node representations, global graph representations and negative sample node representations; and to construct the local-global contrastive learning loss. The graph aggregation and graph classification module 5 is used to concatenate two node representations, perform graph-level aggregation on all node representations, complete graph classification, and output the classification results.

[0041] In the original image sample acquisition module 1, the original image sample is represented as: G = (A, X). This is an adjacency matrix used to represent the connection relationships between nodes; Let N be the node feature matrix; N represents the number of nodes; and F represents the node feature dimension. Both the number of nodes N and the node feature dimension F are positive integers and can be determined based on the graph data to be processed. Further, the value of N ranges from 2708 to 19717, and the value of F ranges from 300 to 3703.

[0042] Furthermore, the structure and node feature enhancement graph acquisition module 2 includes: Structure enhancement module 21 is used to obtain the edge set of the original graph G: M represents the number of edges in the original graph; for nodes in the original graph With nodes The edge between Randomly sampled Bernoulli variables: ,in Construct an edge mask matrix from all Bernoulli variables: The adjacency matrix after structural enhancement is represented as: This yields the structural enhancement diagram: ; wherein This represents the probability of edge deletion. The value range is 0 to 1; Represents a node With nodes The edges between them retain mask variables; Node feature enhancement module 22 is used to set the node set of the original graph as follows: Randomly select a subset of nodes from the node set V as feature occlusion nodes to obtain the occlusion node set: The number of shading nodes is represented as: ,in Indicates the occlusion ratio of node features; for any node The enhanced node features are represented as follows: ,in Represents a node The original feature vector, Representing the mask feature vector, the feature enhancement matrix is ​​formed by assembling all the enhanced node features. Obtain the enhanced feature map of structure and nodes: ;in .

[0043] The two hierarchical graph enhancement view acquisition modules 3 include: modules for applying the same edge deletion probability to the same original graph G=(A,X). and node feature occlusion ratio Using independent random sampling results, the structure and node feature enhancement map acquisition module 2 is executed twice to obtain two structure and node feature enhancement maps: and Each serves as an enhanced view of the first-level graph. Second-level graph enhancement view .

[0044] The graph encoding and local-global contrastive learning module 4 includes: Node representation acquisition module 41 is used to enhance the view of the first hierarchical graph. Second-level graph enhancement view Input the encoder of the first image respectively Second Figure Encoder ,in and These represent the node representations corresponding to the first-level and second-level graph augmented views, respectively. and The parameters are respectively and The image encoder, and These represent the learnable parameters of the first encoder and the second encoder, respectively. The global graph representation and negative sample node representation acquisition module 42 is used to construct the global graph representation and negative sample representation, respectively, to perform average aggregation on the node representations of the first hierarchical and second hierarchical graph augmentation views, and to obtain the first global graph representation through the projection layer. Second global graph representation ,in and Representing nodes respectively The nodes are represented in both views, where N represents the number of nodes. This represents a projection module consisting of a linear transformation and a nonlinear activation function. While keeping the adjacency matrices of the two hierarchical graph enhancement views unchanged, the node arrangement order of their node feature matrices is randomly permuted. Let P be the node random permutation matrix, then the negative sample node matrices are as follows: and ,in and These represent the node feature matrices corresponding to the two hierarchical graph augmentation views, respectively. The negative sample feature matrix and its corresponding augmentation adjacency matrix are input into the respective graph encoders to obtain the negative sample node representations. and ,in and These represent the adjacency matrices corresponding to the two hierarchical graph enhancement views; Local-Global Contrastive Learning Module 43 is used to construct a local-global contrastive learning loss based on the node representations of the enhanced view, the global graph representation, and the negative sample node representations from two hierarchical graphs. The loss is a contrastive learning loss based on Jensen–Shannon divergence.

[0045] The first and second graph encoders both employ two-layer graph convolutional networks with PReLU activation function, and the hidden layer dimensions are manually set. This represents a local-global contrastive loss function built based on Jensen–Shannon divergence, used to measure the difference between positive and negative sample relationships; This represents the contrastive learning loss value obtained by inputting the node representation, global graph representation, and negative sample node representation into the loss function. During model training, this loss value is minimized. Optimize the parameters of the graph encoder.

[0046] The graph aggregation and graph classification module 5 includes: The two node representations are concatenated by module 51, which concatenates the first and second node representations along the feature dimension: , This indicates a feature dimension concatenation operation, where Z represents the final node representation. Image aggregation module 52 is used to perform graph-level aggregation on the final node representation Z for any image to be classified, to obtain a graph-level representation. , This represents the graph-level representation of the graph to be classified. This represents a graph-level aggregation function, where the graph to be classified is the original graph sample whose graph-level category needs to be determined; The graph classification and result output module 53 is used to input the graph-level representation into the graph classifier. The predicted category of the image to be classified is obtained and output, where A graph classifier is used to classify graphs based on their graph-level representations. Calculate the prediction results for each category. This represents the final predicted category label of the image to be classified.

[0047] Furthermore, when the graph-level aggregation function employs average pooling, the graph-level representation can be expressed as follows: N represents the number of nodes in the graph to be classified. Represents a node The corresponding fusion node representation.

[0048] Compared with existing graph classification techniques, the graph classification method based on hierarchical graph augmentation of the present invention has the following advantages: it processes graph topology and node features in stages according to the order of first structural augmentation and then feature augmentation, which can simultaneously improve the model's adaptability to topological changes and feature loss, and construct a differentiated training view for graph contrastive learning, thereby improving the stability of graph representation and classification ability.

[0049] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. It should be understood that the present invention is not limited to the implementations described herein. These implementations are described to help those skilled in the art practice the present invention. Any person skilled in the art can easily make further improvements and modifications without departing from the spirit and scope of the present invention. Therefore, the present invention is only limited by the content and scope of the claims, and is intended to cover all alternatives and equivalents included within the spirit and scope of the present invention as defined by the appended claims.

Claims

1. A graph classification method based on hierarchical graph augmentation, characterized in that, The method includes: The first step is to obtain the original image sample; The second step involves using a random edge deletion method to enhance the structure of the original graph, resulting in a structure-enhanced graph. Then, a node feature enhancement method based on a masking mechanism is used to enhance the node features of the structure-enhanced graph, yielding both structure and node feature-enhanced graphs. The third step involves performing the second step twice on the same original graph to obtain two enhanced graphs of structure and node features, which serve as the first and second hierarchical graph enhancement views. The fourth step involves inputting the two hierarchical graph augmentation views into the graph encoder to obtain node representations, global graph representations, and negative sample node representations; and then constructing a local-global contrastive learning loss. The fifth step is to concatenate the representations of the two nodes and perform graph-level aggregation on all node representations; complete the graph classification and output the classification results. In the first step, the original image sample is represented as: G = (A, X). This is an adjacency matrix used to represent the connection relationships between nodes; Here, F represents the node feature matrix; N represents the number of nodes; F represents the dimension of the node features. Furthermore, the second step includes: The structure enhancement step obtains the edge set of the original graph G: M represents the number of edges in the original graph; for nodes in the original graph With nodes The edge between Randomly sampled Bernoulli variables: ,in Construct an edge mask matrix from all Bernoulli variables: M e = m i j The adjacency matrix after structural enhancement is represented as: This yields the structural enhancement diagram: ; wherein This represents the probability of edge deletion. The value range is 0 to 1; Represents a node With nodes The edges between them retain mask variables; Node feature enhancement steps, assuming the original graph's node set is: Randomly select a subset of nodes from the node set V as feature occlusion nodes to obtain the occlusion node set: The number of shading nodes is represented as: ,in Indicates the occlusion ratio of node features; for any node The enhanced node features are represented as follows: ,in Represents a node The original feature vector, Representing the mask feature vector, the feature enhancement matrix is ​​formed by assembling all the enhanced node features. Obtain the enhanced feature map of structure and nodes: ;in .

2. The method as described in claim 1, characterized in that, The third step includes: applying the same edge deletion probability to the same original graph G=(A,X). and node feature occlusion ratio Using independent random sampling results, the second step is performed twice to obtain two structure and node feature enhancement maps: and Each serves as an enhanced view of the first-level graph. Second-level graph enhancement view ; The fourth step includes: The nodes represent the steps to obtain the view, enhancing the first-level graph. Second-level graph enhancement view Input the encoder of the first image respectively Second Figure Encoder ,in and These represent the node representations corresponding to the first-level and second-level graph augmented views, respectively. and The parameters are respectively and The graph encoder, and These represent the learnable parameters of the first encoder and the second encoder, respectively. The steps for obtaining global graph representation and negative sample node representation are as follows: Construct global graph representation and negative sample representation; perform average aggregation on the node representations of the first-level and second-level graph augmented views respectively; and obtain the first global graph representation through a projection layer. Second global graph representation ,in and Representing nodes respectively The nodes are represented in both views, where N represents the number of nodes. This represents a projection module consisting of a linear transformation and a nonlinear activation function. While keeping the adjacency matrices of the two hierarchical graph enhancement views unchanged, the node arrangement order of their node feature matrices is randomly permuted. Let P be the node random permutation matrix, then the negative sample node matrices are as follows: and ,in and These represent the node feature matrices corresponding to the two hierarchical graph augmentation views, respectively. The negative sample feature matrix and its corresponding augmentation adjacency matrix are input into the respective graph encoders to obtain the negative sample node representations. and ,in and These represent the adjacency matrices corresponding to the two hierarchical graph enhancement views; The local-global contrastive learning step constructs the local-global contrastive learning loss based on the node representations of the augmented view, the global graph representation, and the negative sample node representations from two hierarchical graphs: The loss is a contrastive learning loss based on Jensen–Shannon divergence; The first and second graph encoders both employ two-layer graph convolutional networks with PReLU activation function, and the hidden layer dimensions are manually set. This represents a local-global contrastive loss function built based on Jensen–Shannon divergence, used to measure the difference between positive and negative sample relationships; This represents the contrastive learning loss value obtained by inputting the node representation, global graph representation, and negative sample node representation into the loss function. During model training, this loss value is minimized. Optimize the parameters of the graph encoder.

3. The method as described in claim 2, characterized in that, The fifth step includes: The two nodes represent the concatenation step, and the first and second node representations are concatenated along the feature dimension: , This indicates a feature dimension concatenation operation, where Z represents the final node representation. In the image aggregation step, for any image to be classified, graph-level aggregation is performed on the final node representation Z to obtain a graph-level representation. , This represents the graph-level representation of the graph to be classified. This represents a graph-level aggregation function, where the graph to be classified is the original graph sample whose graph-level category needs to be determined; The graph classification and result output steps involve inputting the graph-level representation into the graph classifier. The predicted category of the image to be classified is obtained and output, where A graph classifier is used to classify graphs based on their graph-level representations. Calculate the prediction results for each category. This represents the final predicted category label of the image to be classified.

4. The method as described in claim 3, characterized in that, When the graph-level aggregation function uses average pooling, the graph-level representation can be expressed as follows: N represents the number of nodes in the graph to be classified. Represents a node The corresponding fusion node representation.

5. The method as described in claim 1, characterized in that, The value of N ranges from 2708 to 19717, and the value of F ranges from 300 to 3703.

6. A graph classification device based on hierarchical graph enhancement, characterized in that, The device includes: The original image sample acquisition module is used to acquire original image samples; The module for obtaining structure and node feature enhancement graphs uses a random edge deletion method to perform structural enhancement on the original graph to obtain a structure-enhanced graph; and uses a node feature enhancement method based on a masking mechanism to perform node feature enhancement on the structure-enhanced graph to obtain both structure and node feature enhancement graphs. Two hierarchical graph enhancement view acquisition modules are used to perform the second step twice for the same original graph to obtain two enhanced graphs of structure and node features, which serve as the first and second hierarchical graph enhancement views. The graph encoding and local-global contrastive learning module is used to input two hierarchical graph augmentation views into the graph encoder to obtain node representations, global graph representations, and negative sample node representations; and to construct the local-global contrastive learning loss. The graph aggregation and graph classification modules are used to concatenate two node representations, perform graph-level aggregation of all node representations, complete graph classification, and output the classification results. In the original image sample acquisition module, the original image sample is represented as: G = (A, X). This is an adjacency matrix used to represent the connection relationships between nodes; Here, F represents the node feature matrix; N represents the number of nodes; F represents the dimension of the node features. Furthermore, the structure and node feature enhancement graph acquisition module includes: The structure enhancement module is used to obtain the edge set of the original graph G: M represents the number of edges in the original graph; for nodes in the original graph With nodes The edge between Randomly sampled Bernoulli variables: ,in Construct an edge mask matrix from all Bernoulli variables: M e = m i j The adjacency matrix after structural enhancement is represented as: This yields the structural enhancement diagram: ; wherein This represents the probability of edge deletion. The value range is 0 to 1; Represents a node With nodes The edges between them retain mask variables; The node feature enhancement module is used to define the node set of the original graph as follows: Randomly select a subset of nodes from the node set V as feature occlusion nodes to obtain the occlusion node set: The number of shading nodes is represented as: ,in Indicates the occlusion ratio of node features; for any node The enhanced node features are represented as follows: ,in Represents a node The original feature vector, Representing the mask feature vector, the feature enhancement matrix is ​​formed by assembling all the enhanced node features. Obtain the enhanced feature map of structure and nodes: ;in .

7. The apparatus as claimed in claim 6, characterized in that, The two hierarchical graph enhancement view acquisition modules include: modules for applying the same edge deletion probability to the same original graph G=(A,X). and node feature occlusion ratio Using independent random sampling results, the structure and node feature enhancement map acquisition module is executed twice to obtain two structure and node feature enhancement maps: and Each serves as an enhanced view of the first-level graph. Second-level graph enhancement view ; The graph encoding and local-global contrastive learning module includes: The node represents the acquisition module, used to enhance the view of the first-level graph. Second-level graph enhancement view Input the encoder of the first image respectively Second Figure Encoder ,in and These represent the node representations corresponding to the first-level and second-level graph augmented views, respectively. and The parameters are respectively and The graph encoder, and These represent the learnable parameters of the first encoder and the second encoder, respectively. The global graph representation and negative sample node representation acquisition module is used to construct the global graph representation and negative sample representation. It performs average aggregation on the node representations of the first-level and second-level graph augmented views, respectively, and obtains the first global graph representation through the projection layer. Second global graph representation ,in and Representing nodes respectively The nodes are represented in both views, where N represents the number of nodes. This represents a projection module consisting of a linear transformation and a nonlinear activation function. While keeping the adjacency matrices of the two hierarchical graph enhancement views unchanged, the node arrangement order of their node feature matrices is randomly permuted. Let P be the node random permutation matrix, then the negative sample node matrices are as follows: and ,in and These represent the node feature matrices corresponding to the two hierarchical graph augmentation views, respectively. The negative sample feature matrix and its corresponding augmentation adjacency matrix are input into the respective graph encoders to obtain the negative sample node representations. and ,in and These represent the adjacency matrices corresponding to the two hierarchical graph enhancement views; The local-global contrastive learning module is used to construct the local-global contrastive learning loss based on the node representations of the enhanced view, the global graph representation, and the negative sample node representations from two hierarchical graphs. The loss is a contrastive learning loss based on Jensen–Shannon divergence; The first and second graph encoders both employ two-layer graph convolutional networks with PReLU activation function, and the hidden layer dimensions are manually set. This represents a local-global contrastive loss function built based on Jensen–Shannon divergence, used to measure the difference between positive and negative sample relationships; This represents the contrastive learning loss value obtained by inputting the node representation, global graph representation, and negative sample node representation into the loss function. During model training, this loss value is minimized. Optimize the parameters of the graph encoder.

8. The apparatus as claimed in claim 7, characterized in that, The graph aggregation and graph classification module includes: The two nodes represent the concatenation module, used to concatenate the first and second node representations along the feature dimension: , This indicates a feature dimension concatenation operation, where Z represents the final node representation. The image aggregation module is used to perform graph-level aggregation on the final node representation Z for any image to be classified, to obtain a graph-level representation. , This represents the graph-level representation of the graph to be classified. This represents a graph-level aggregation function, where the graph to be classified is the original graph sample whose graph-level category needs to be determined; The graph classification and result output module is used to input the graph-level representation into the graph classifier. The predicted category of the image to be classified is obtained and output, where A graph classifier is used to classify graphs based on their graph-level representations. Calculate the prediction results for each category. This represents the final predicted category label of the image to be classified.

9. The apparatus as claimed in claim 8, characterized in that, When the graph-level aggregation function uses average pooling, the graph-level representation can be expressed as follows: N represents the number of nodes in the graph to be classified. Represents a node The corresponding fusion node representation.

10. The apparatus as claimed in claim 6, characterized in that, The value of N ranges from 2708 to 19717, and the value of F ranges from 300 to 3703.

Images