Coder disturbance diagram-based comparative learning method, system and equipment
Through the encoder perturbation graph comparison learning method, graph comparison learning is automatically implemented, which solves the problem of graph neural network dependence on annotation data and enhancement of complex data, and realizes efficient and concise graph representation learning, which is suitable for a variety of graph data tasks.
Patent Information
- Application Number
- CN202510340284.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-21
- Publication Date
- 2025-08-01
AI Technical Summary
The existing graph neural networks require a large amount of real labeled data for supervision and training in graph data tasks, and the data enhancement methods of existing graph comparison learning methods are complex and rely on professional field knowledge, which limits their universality and practicality.
The encoder-based perturbation graph comparison learning method is adopted, and graph neural network encoder and perturbation encoder are constructed, graph representation is generated using random noise, and encoder parameters are optimized through comparison loss, and graph comparison learning is automatically realized, avoiding manual design data enhancement methods.
Reduces operational complexity, reduces computational costs, improves the applicability and accuracy of the method, and can achieve higher or near-optimal classification accuracy on different graph data without relying on domain knowledge.
Smart Images
Figure CN120409599A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to a contrastive learning method, in particular to an encoder perturbation graph contrastive learning method, system and device, belonging to the technical field of contrastive learning. Background Art
[0002] In recent years, graph neural networks have demonstrated excellent performance in various graph-based data tasks, such as node classification, graph classification or graph generation. However, most existing graph neural networks require a large amount of real labeled data for supervised training, and the labeled data available in the knowledge base is often limited. To solve this problem, researchers have begun to explore self-supervised representation learning methods for graphs, among which graph contrastive learning (GCL) has become one of the mainstream methods. The basic idea of GCL is: perform data augmentation on each graph twice to make the two augmented subgraphs semantically related, and then maximize the mutual information between the representations of these two subgraphs. However, due to the large structural information and semantic cross-domain differences of graph data, existing general data augmentation methods are difficult to well maintain the semantic relevance between the subgraph and the original graph.
[0003] There are mainly three imperfect implementation methods for data augmentation in current GCL methods: one is that a large amount of manual parameter tuning is required for different datasets; the second is that complex searches are needed to find suitable augmentation methods; the third is that expensive domain knowledge needs to be introduced to guide data augmentation. These methods all limit the generality and practicality of existing GCL methods. Summary of the Invention
[0004] In view of this, the present application provides an encoder perturbation graph contrastive learning method, system and device to solve or alleviate the technical problems existing in the prior art, and at least provide a beneficial option.
[0005] The technical solution of the embodiment of the present application is implemented as follows. There is provided an encoder perturbation graph contrastive learning method, including:
[0006] S100: Construct a graph neural network encoder f(·; θ) based on preset encoder parameters θ, and construct a perturbation encoder f(·; θ′) based on the graph neural network encoder f(·; θ).
[0007] S200: Obtain the original graph data G, and input the original graph data into the graph neural network encoder f(·; θ) and the perturbation encoder f(·; θ′) respectively to obtain a first initial graph representation h and a second initial graph representation h′.
[0008] S300: Project the first initial graph representation h and the second initial graph representation h′ to obtain a first target graph representation z and a second target graph representation z′.
[0009] S400: Obtain a contrastive loss based on the first target graph representation z and the second target graph representation z', and determine whether the current graph neural network encoder f(·; θ) meets the graph representation requirements according to the contrastive loss. If it meets the requirements, obtain the final graph neural network encoder f(·; θ); otherwise, update the encoder parameter θ and return to S100.
[0010] Further preferably: In S100, constructing the perturbed encoder f(·; θ) includes:
[0011] Adding a perturbation operation θ' l to the graph neural network encoder f(·; θ) to obtain the perturbed encoder f(·; θ').
[0012] where θ' l = θ l + η·Δθ l ; Δθ l ~ N(0, σ l 2 ), θ l and θ' l are the weight tensors of the l-th layer of the graph neural network encoder f(·; θ) and the corresponding perturbation operation respectively, η is a coefficient measuring the perturbation operation, and Δθ l is a perturbation term sampled from a Gaussian distribution with mean and variance of zero .
[0013] Further preferably: S200 includes:
[0014] Encode the original graph data G through the graph neural network encoder f(·; θ) and the perturbed encoder f(·; θ') respectively to obtain the first initial graph representation h = f(G; θ) and the second initial graph representation h' = f(G; θ').
[0015] Further preferably: In S300, obtaining the first target graph representation z and the second target graph representation z' includes:
[0016] Obtain a non-linear mapping g, and based on the non-linear mapping g and the first initial graph representation h, obtain the first target graph representation z = g(h), and at the same time, based on the non-linear mapping g and the second initial graph representation h', obtain the second target graph representation z' = g(h').
[0017] Further preferably: In S300, obtaining the contrastive loss includes:
[0018] Extract N graphs from the original graph data G as positive samples; at the same time, based on the original graph data G and the perturbation operation θ' lExtract N - 1 graphs as negative samples, and repeat S100 - S300 to obtain N groups of first target graph representations z and second target graph representations Z′.
[0019] Further preferably: In S300, to obtain the contrastive loss, it further includes:
[0020] Obtain the contrastive loss according to N groups of first target graph representations z and second target graph representations z′ where sim(z, z′) = z Τ z′ / (||z|| ||z′||), exp represents the exponential function with the natural constant e as the base, τ represents the temperature hyperparameter, and ||·|| represents the norm of the vector.
[0021] Further preferably: The original graph data G at least includes a node set and an edge set.
[0022] Further preferably: This application also provides a system for a graph contrastive learning method based on encoder perturbation, and the system includes:
[0023] A construction module: used to construct a graph neural network encoder f(·; θ) based on preset encoder parameters θ, and construct a perturbation encoder f(·; θ′) based on the graph neural network encoder f(·; θ).
[0024] An acquisition module: used to acquire the original graph data G, and input the original graph data into the graph neural network encoder f(·; θ) and the perturbation encoder f(·; θ′) respectively to obtain a first initial graph representation h and a second initial graph representation h′.
[0025] A projection module: used to project the first initial graph representation h and the second initial graph representation h′ to obtain a first target graph representation z and a second target graph representation z′.
[0026] An output module: used to obtain the contrastive loss based on the first target graph representation z and the second target graph representation z′, and judge whether the current graph neural network encoder f(·; θ) meets the graph representation requirements according to the contrastive loss. If it meets the requirements, obtain the final graph neural network encoder f(·; θ); otherwise, update the encoder parameters θ and return to the construction module.
[0027] Further preferably: This application also provides a device, the device includes a processor and a memory, and at least one instruction, at least one program, a code set or an instruction set is stored in the memory, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the graph contrastive learning method based on encoder perturbation.
[0028] Due to the above technical solutions adopted in the embodiments of the present application, it has the following advantages:
[0029] 1. The implementation of automated graph contrastive learning eliminates the need for manual design and selection of data augmentation methods, reducing the operational complexity.
[0030] 2. It does not rely on professional domain knowledge, making the method easier to use. By generating graph representations that preserve semantics through encoder perturbations, the concern about semantic changes is eliminated. The method is simple and efficient, only requiring the introduction of a small amount of random noise, thus significantly reducing the computational cost.
[0031] 3. It has wide applicability and can be used for various types of graph data. The learned graph representations are stable to perturbations and can be easily transferred to downstream tasks.
[0032] The above summary is only for the purpose of the specification and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features of the present application will be readily apparent by reference to the drawings and the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.
[0034] Figure 1 It is a flowchart of the graph contrastive learning method based on encoder perturbation described in the present application;
[0035] Figure 2 For Figure 1 the system framework diagram of the graph contrastive learning method based on encoder perturbation described. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0036] In the following, only some exemplary embodiments are briefly described. As those skilled in the art can recognize, the described embodiments can be modified in various different ways without departing from the spirit or scope of the present application. Therefore, the drawings and the description are considered to be exemplary in nature rather than restrictive.
[0037] The embodiments of the present application will be described in detail below with reference to the drawings.
[0038] As Figure 1 shown, the embodiments of the present application provide a graph contrastive learning method based on encoder perturbation, including:
[0039] S100: Construct a graph neural network encoder \(f(\cdot;\theta)\) based on the preset encoder parameters \(\theta\), and construct a perturbation encoder \(f(\cdot;\theta')\) based on the graph neural network encoder \(f(\cdot;\theta)\).
[0040] More preferably: The original graph data \(G\) at least includes a node set and an edge set.
[0041] In some embodiments, the original graph data \(G\) can be represented as \((V, E)\); \(V\) represents the node set in the graph, and each node \(v\in V\) has a feature vector to represent its attributes; \(E\) represents the edge set in the graph. Each edge \(e\in E\) represents the relationship between nodes.
[0042] Among them, the original graph data \(G\) includes but is not limited to social network graphs, molecular graphs, protein interaction networks, and knowledge graphs, etc.
[0043] More preferably: In S100, constructing the perturbation encoder \(f(\cdot;\theta)\) includes:
[0044] Adding a perturbation operation \(\theta\) l ' to the graph neural network encoder \(f(\cdot;\theta)\) to obtain the perturbation encoder \(f(\cdot;\theta')\).
[0045] Among them, \(\theta\) l '=\(\theta\) l +\(\eta\cdot\Delta\theta\) l ;\(\Delta\theta\) l ~\(N(0,\sigma\) l 2 ), \(\theta\) l and \(\theta\) l ' are the weight tensors of the \(l\)th layer of the graph neural network encoder \(f(\cdot;\theta)\) and the corresponding perturbation operation respectively, \(\eta\) is a coefficient measuring the perturbation operation, and \(\Delta\theta\) l is a perturbation term sampled from a Gaussian distribution with a mean and variance of zero .
[0046] S200: Obtain the original graph data \(G\), and input the original graph data into the graph neural network encoder \(f(\cdot;\theta)\) and the perturbation encoder \(f(\cdot;\theta')\) respectively to obtain a first initial graph representation \(h\) and a second initial graph representation \(h'\).
[0047] It should be noted that the first initial graph representation \(h\) and the second initial graph representation \(h'\) are a set of initial graph representations corresponding to the same original graph data \(G\); according to the following iterative optimization, there will be \(N\) sets of first initial graph representations \(h\) and second initial graph representations \(h'\).
[0048] More preferably: S200 includes:
[0049] Encode the original graph data G through the graph neural network encoder f(·; θ) and the perturbation encoder f(·; θ′) respectively to obtain the first initial graph representation h = f(G; θ) and the second initial graph representation h′ = f(G; θ′).
[0050] Further preferably: in the S300, obtaining the first target graph representation z and the second target graph representation z′ includes:
[0051] Obtain the non-linear mapping g, and based on the non-linear mapping g and the first initial graph representation h, obtain the first target graph representation z = g(h), and at the same time, based on the non-linear mapping g and the second initial graph representation h′, obtain the second target graph representation z′ = g(h′).
[0052] Among them, the non-linear mapping g is implemented by a two-layer perceptron, mapping h and h′ to another latent space to obtain z and z′.
[0053] S300: Project the first initial graph representation h and the second initial graph representation h′ to obtain the first target graph representation z and the second target graph representation z′.
[0054] Further preferably: in the S300, obtaining the contrastive loss includes:
[0055] Extract N graphs from the original graph data G as positive samples; at the same time, based on the original graph data G and the perturbation operation θ′ l Extract N - 1 graphs as negative samples, and repeat S100 - S300 to obtain N groups of the first target graph representations z and the second target graph representations Z′.
[0056] In some embodiments, when defining the contrastive loss between z and z′, maximize the similarity of positive samples and minimize the similarity of negative samples, and then obtain the loss function
[0057] Randomly sample a subgraph from the graph dataset each time, including N graphs {G1, G2,..., G N}, and repeat the process of the above framework to help optimize and update the encoder parameters through the contrastive loss later.
[0058] Further preferably: in the S3, obtaining the contrastive loss further includes:
[0059] Obtain the contrastive loss according to N groups of the first target graph representations z and the second target graph representations z′ Among them, sim(z, z′) = z Τ z′ / ||z||||z′||, exp represents the exponential function with the natural constant e as the base, τ represents the temperature hyperparameter, and ||·|| represents the norm of the vector.
[0060] S400: Obtain a contrastive loss based on the first target graph representation z and the second target graph representation z', and determine whether the current graph neural network encoder f(·; θ) meets the graph representation requirements according to the contrastive loss. If it meets the requirements, obtain the final graph neural network encoder f(·; θ); otherwise, update the encoder parameter θ and return to S100.
[0061] In some embodiments, those skilled in the art determine the graph representation requirements according to actual application needs, such as a graph representation that is stable to encoder perturbations. How to define stability specifically is set by those skilled in the art and is not restricted herein.
[0062] In operation: This application selects four datasets from the publicly available benchmark TUDataset, including NCI1, PROTEINS, DD, and MUTAG. These datasets cover various types such as social networks and biochemical molecules. For each dataset, 5 experiments are conducted, and each time a 10-fold cross-validation strategy is adopted. Finally, the average accuracy and its standard deviation of all experiments are calculated. The invention achieves an accuracy of 79.12% on the NCI 1 dataset, 75.35% on PROTEINS, 77.44% on DD, and 89.01% on the MUTAG dataset. Without relying on data augmentation techniques, the invention achieves a classification accuracy higher than or close to the best level, highlighting its characteristic of achieving excellent performance without manual adjustment or consuming domain expertise.
[0063] Further preferably: Please refer to Figure 2 , this application also provides a system for a graph contrastive learning method based on encoder perturbation, and the system includes:
[0064] A construction module: used to construct a graph neural network encoder f(·; θ) based on a preset encoder parameter θ, and construct a perturbed encoder f(·; θ') based on the graph neural network encoder f(·; θ).
[0065] An acquisition module: used to acquire the original graph data G, and input the original graph data into the graph neural network encoder f(·; θ) and the perturbed encoder f(·; θ') respectively to obtain a first initial graph representation h and a second initial graph representation h'.
[0066] A projection module: used to project the first initial graph representation h and the second initial graph representation h' to obtain a first target graph representation z and a second target graph representation z'.
[0067] Output module: It is used to obtain a contrastive loss based on the first target graph representation z and the second target graph representation z', and determine whether the current graph neural network encoder f(·; θ) meets the graph representation requirements according to the contrastive loss. If it meets the requirements, the final graph neural network encoder f(·; θ) is obtained; otherwise, the encoder parameter θ is updated and returned to the construction module.
[0068] Further preferably: The present application also provides a device, which includes a processor and a memory. At least one instruction, at least one program, a code set or an instruction set is stored in the memory, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the graph contrastive learning method based on encoder perturbation.
[0069] As mentioned above, the above is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of various changes or substitutions thereof, and these should all be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.
Claims
1. A graph contrastive learning method based on encoder perturbation, characterized in that It includes the following steps: S100: Construct a graph neural network encoder f(·; θ) based on preset encoder parameters θ, and construct a perturbation encoder f(·; θ′) based on the graph neural network encoder f(·; θ); S200: Obtain the original graph data G, and input the original graph data into the graph neural network encoder f(·; θ) and the perturbation encoder f(·; θ′) respectively to obtain a first initial graph representation h and a second initial graph representation h′; S300: Project the first initial graph representation h and the second initial graph representation h′ to obtain a first target graph representation z and a second target graph representation z′; S400: Obtain a contrastive loss based on the first target graph representation z and the second target graph representation z′, and determine whether the current graph neural network encoder f(·; θ) meets the graph representation requirements according to the contrastive loss. If it meets the requirements, obtain the final graph neural network encoder f(·; θ); otherwise, update the encoder parameters θ and return to S100.
2. The graph contrastive learning method based on encoder perturbation according to claim 1, wherein In S100, constructing the perturbation encoder f(·; θ) includes: Add the perturbation operation θ′ l to the graph neural network encoder f(·; θ) to obtain a perturbed encoder f(·; θ′); where, θ′ l = θ l + η·Δθ l ; Δθ l ~ N(0, σ l 2 ), θ l and θ′ l are the weight tensors of the l-th layer of the graph neural network encoder f(·; θ) and the corresponding perturbation operation respectively, η is the coefficient measuring the perturbation operation, and Δθ l is the perturbation term sampled from the Gaussian distribution with mean and variance of zero .
3. The graph contrastive learning method based on encoder perturbation according to claim 2, wherein S200 includes: Encode the original graph data G through the graph neural network encoder f(·; θ) and the perturbation encoder f(·; θ′) respectively to obtain a first initial graph representation h = f(G; θ) and a second initial graph representation h′ = f(G; θ′).
4. The graph contrastive learning method based on encoder perturbation according to claim 3, wherein In S300, obtaining the first target graph representation z and the second target graph representation z′ includes: Obtain a non-linear mapping g, and obtain the first target graph representation based on the non-linear mapping g and the first initial graph representation; z = g(h), and at the same time obtain the second target graph representation z′ = g(h′) based on the non-linear mapping g and the second initial graph representation h′.
5. The graph contrastive learning method based on encoder perturbation according to claim 4, wherein Extract N graphs as positive samples based on the original graph data G; at the same time, based on the original graph data G and the perturbation operation θ l ′ extract N - 1 graphs as negative samples, and repeat S100 - S300 to obtain N sets of first target graph representations z and second target graph representations z′.
6. The graph contrastive learning method based on encoder perturbation according to claim 5, wherein Obtain a contrastive loss based on N groups of first target graph representations z and second target graph representations z'. where sim(z, z') = z Τ z' / (||z|| ||z'||), exp represents the exponential function with the natural constant e as the base, τ represents the temperature hyperparameter, and ||·|| represents the norm of a vector.
7. The graph contrastive learning method based on encoder perturbation according to claim 1, wherein 8. A system adopting the graph contrastive learning method based on encoder perturbation as described in any one of claims 1-7, characterized in that, 9. A device, characterized in that, The device includes a processor and a memory, and at least one instruction, at least one program, a code set or an instruction set is stored in the memory. The at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the graph contrastive learning method based on encoder perturbation according to any one of claims 1-8.