GNN and Transform collaborative graph contrast learning method, system and equipment

Abstract

The invention discloses a GNN (Global Navigation Network) and Transform collaborative graph contrast learning method, a GNN and Transform collaborative graph contrast learning system and GNN and Transform collaborative graph contrast learning equipment. According to the scheme, the method comprises the steps of determining node features and a first feature matrix of a GNN coding graph, and determining a second feature matrix of a Transform coding graph; obtaining a positive and negative sample pair set of each node based on the first feature matrix, the second feature matrix and the graph topology structure; performing contrast loss training on the positive and negative sample pair set, and training node features of the first feature matrix and the second feature matrix through a minimization loss function to obtain target features; and processing the target features to obtain an output result. According to the method, the characteristic relation of the nodes in the graph can be more accurately captured, the performance of the model in comparison learning tasks of various graphs is improved, and the method has high generalization performance and practical value.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a GNN and Transformer collaborative graph comparison learning method, system and device. Background Art

[0002] Graph contrastive learning is a self-supervised learning method that does not rely on labels and has been widely used in many fields such as biochemistry in recent years. It uses an enhancement strategy to generate multiple views and create positive and negative pairs for node or graph representations. By maximizing the similarity between positive pairs and minimizing the similarity between negative pairs, it generates highly expressive representations for downstream tasks.

[0003] The performance of graph contrastive learning mainly depends on the graph augmentation strategy, the selection of encoders, and the design of loss functions. Most graph contrastive methods use graph augmentation, which generates new views by randomly transforming the graph (such as node dropping, edge sampling, feature perturbation, etc.) and learns the representation of the graph by comparing the views. However, graph augmentation may change the underlying semantics of the graph, which may affect the performance of downstream tasks. At the same time, existing graph contrastive learning methods mostly use GNN as a single encoder for graph representation learning. Although GNN is good at capturing local topological structures and dependencies between neighbors, it is not good at dealing with long-range dependencies and global structures. In addition, most graph contrastive learning methods use the InfoNCE loss function to train the model. Although InfoNCE performs well in practice, it only trains the corresponding nodes between the enhanced graphs as positive sample pairs, ignoring other potential positive sample pairs, which limits the model's learning of rich semantic representations of nodes.

[0004] Based on this, there is an urgent need to provide a graph contrastive learning solution that collaborates with GNN and Transformer to solve the problems of using graph enhancement and relying only on a single encoder for graph representation learning in the existing technology. Summary of the invention

[0005] In view of this, the present invention provides a GNN and Transformer collaborative graph contrast learning method, system and device, which are used to solve the problems of using graph enhancement and relying only on a single encoder for graph representation learning in the prior art.

[0006] In order to solve the above technical problems, the present invention provides a graph contrast learning method based on GNN and Transformer collaboration, the method comprising:

[0007] Determine the node features and the first feature matrix of the GNN encoding graph, and determine the second feature matrix of the Transformer encoding graph;

[0008] Obtaining a set of positive and negative sample pairs for each node based on the first feature matrix, the second feature matrix and the graph topology structure;

[0009] Perform contrast loss training on the set of positive and negative sample pairs, and train the node features of the first feature matrix and the second feature matrix by minimizing the loss function to obtain target features;

[0010] The target features are processed to obtain output results.

[0011] Preferably, determining the node features and the first feature matrix of the GNN encoding graph, and determining the second feature matrix of the Transformer encoding graph, includes:

[0012] Inputting the adjacency matrix, degree matrix and node features of the graph into a GNN encoder to obtain a first feature matrix; the GNN encoder adopts a graph convolutional network;

[0013] The adjacency matrix and the node features are input into a Transformer encoder to obtain a second feature matrix; the Transformer encoder adopts NodeFormer.

[0014] Preferably, obtaining a set of positive and negative sample pairs for each node based on the first feature matrix, the second feature matrix and the graph topology structure includes:

[0015] Calculating the cosine similarity of the first feature matrix and the second feature matrix;

[0016] Arrange the cosine similarities in descending order to obtain a first characteristic neighbor matrix and a second characteristic neighbor matrix;

[0017] Determine the topological neighbor order of each node based on the adjacency matrix to obtain a topological neighbor matrix;

[0018] Respectively extracting a first node set and a second node set of a preset node in the first characteristic neighbor matrix and the second characteristic neighbor matrix;

[0019] Taking out a third node set of a preset node in the topological neighbor matrix;

[0020] The intersection of the first node set, the second node set and the third node set is taken to obtain a positive sample pair set of the preset node, and all other nodes are used as negative sample pairs of the preset node to obtain a negative sample pair set of the preset node.

[0021] Preferably, the set of positive sample pairs of the preset nodes is expressed as:

[0022]

[0023] Among them, B θ represents the first characteristic neighbor matrix, represents the second characteristic neighbor matrix, v i represents a preset node, the preset node is the i-th node, represents the first node set, represents the second node set, T i represents the third node set, P i Represents a set of positive sample pairs of preset nodes;

[0024] The negative sample pair set of the preset node is expressed as:

[0025] N i =V\(P i ∪{v i})

[0026] Among them, V represents the set of all nodes in the graph, N i Represents a set of negative sample pairs of preset nodes.

[0027] Preferably, node v i The characteristic is expressed as The loss function is:

[0028]

[0029] The loss function of all nodes can be expressed as:

[0030]

[0031] Among them, H θ is the first characteristic matrix, is the second characteristic matrix, in the first characteristic matrix H θ The features in are expressed as In the second characteristic matrix The features in are expressed as represents the loss function of all nodes, Represents node v i Features The loss function is Represents node v i Features The loss function of .

[0032] Preferably, the processing of the target feature to obtain an output result includes:

[0033] Batch normalize the first feature matrix and the second feature matrix;

[0034] The two feature matrices after batch normalization are added according to the preset ratio to obtain the output matrix.

[0035] Preferably, the batch normalizing the first feature matrix and the second feature matrix includes:

[0036] Using the formula:

[0037] H″ θ =BatchNorm(H′ θ )

[0038] Normalize the first feature matrix corresponding to the trained model; where H' θ The first feature matrix corresponding to the trained model;

[0039] Using the formula:

[0040]

[0041] Normalize the second feature matrix corresponding to the trained model; where: The second feature matrix corresponding to the trained model;

[0042] The two feature matrices after batch normalization are added according to a preset ratio to obtain an output matrix, including:

[0043] Using the formula:

[0044]

[0045] The two feature matrices after batch normalization are added according to the preset ratio to obtain the output matrix H.

[0046] Beneficial effects: The present invention provides a graph contrast learning method that collaborates with GNN and Transformer, by determining the node features and the first feature matrix of the GNN encoding graph, and determining the second feature matrix of the Transformer encoding graph; obtaining a set of positive and negative sample pairs for each node based on the first feature matrix, the second feature matrix and the graph topology; performing contrast loss training on the set of positive and negative sample pairs, and training the node features of the first feature matrix and the second feature matrix by minimizing the loss function to obtain the target features; processing the target features to obtain output results. The present invention combines GNN and Transformer, and integrates the topological structure information of the graph; using a linear complexity Transformer, while capturing global dependencies, reduces computational complexity, and integrates the global topological information of the graph. At the same time, a non-enhancement strategy is adopted to construct a new loss function while keeping the semantics of the graph unchanged, and the intersection of the node representation view and the topological structure view is used as a positive sample pair, so that the learned graph representation is more comprehensive and expressive.

[0047] The present invention also provides a GNN and Transformer collaborative graph contrast learning system, the system comprising:

[0048] Feature encoding module, positive and negative sample pair construction module, cross-view comparison module and normalization module;

[0049] The feature encoding module is used to determine the node features and the first feature matrix of the GNN encoding graph; and is used to determine the second feature matrix of the Transformer encoding graph;

[0050] The positive and negative sample pair construction module is used to obtain a set of positive and negative sample pairs for each node based on the first feature matrix, the second feature matrix and the graph topology structure;

[0051] The cross-view comparison module is used to perform contrast loss training on the positive and negative sample pairs, and train the node features of the first feature matrix and the second feature matrix by minimizing the loss function to obtain the target features;

[0052] The normalization module is used to process the target features to obtain an output result.

[0053] The present invention also provides a GNN and Transformer collaborative graph comparison learning device, the device comprising:

[0054] A memory, a processor, and a communication interface coupled to the processor; the memory stores a computer program that can be executed by the processor; when the processor runs the computer program, it executes the aforementioned GNN and Transformer collaborative graph comparison learning method.

[0055] The present invention also provides a computer storage medium, in which instructions are stored. When the instructions are executed, the aforementioned GNN and Transformer collaborative graph comparison learning method is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The drawings described herein are used to provide a further understanding of one or more embodiments of this specification and constitute a part of one or more embodiments of this specification. The exemplary embodiments of this specification and their descriptions are used to explain one or more embodiments of this specification and do not constitute an improper limitation on one or more embodiments of this specification. In the drawings:

[0057] Figure 1 A schematic diagram of a flow chart of a GNN and Transformer collaborative graph comparison learning method provided by the present invention;

[0058] Figure 2 A structural diagram of the model provided by the present invention;

[0059] Figure 3 A schematic diagram of a GNN and Transformer collaborative graph comparison learning system provided by the present invention;

[0060] Figure 4 A schematic diagram of a graph comparison learning device for collaborative GNN and Transformer provided by the present invention. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solutions and advantages of one or more embodiments of this specification clearer, the technical solutions of one or more embodiments of this specification will be clearly and completely described below in combination with the specific embodiments of this specification and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this description, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of one or more embodiments of this specification.

[0062] The technical solutions provided by the embodiments of this specification are described in detail below in conjunction with the accompanying drawings.

[0063] The present invention aims to overcome the limitations of existing graph contrastive learning methods. In order to solve the problems in the prior art, a graph contrastive learning method, system and device that collaborates with GNN and Transformer are proposed. The linear complexity Transformer is used to capture global dependencies while reducing the computational complexity and integrating the global topological information of the graph. At the same time, a non-enhancement strategy is adopted to construct a new loss function while keeping the semantics of the graph unchanged, and the intersection of the node representation view and the topological structure view is used as a positive sample pair, so that the learned graph representation is more comprehensive and expressive.

[0064] In order to solve the defects in the prior art, this solution provides the following embodiments:

[0065] Next, the solution provided by the embodiments of this specification is described in conjunction with the accompanying drawings:

[0066] like Figure 1 As shown, the process may include the following steps:

[0067] S1: Determine the node features and the first feature matrix of the GNN encoding graph, and determine the second feature matrix of the Transformer encoding graph.

[0068] This step can be performed in the feature encoding module, that is, in the feature encoding module, the node features and adjacency matrix of the GNN encoding graph are used, and the adjacency matrix of the Transformer encoding graph is used at the same time to obtain two feature matrices.

[0069] S2: Obtain a set of positive and negative sample pairs for each node based on the first feature matrix, the second feature matrix and the graph topology structure;

[0070] This step can be performed in the positive and negative sample pair construction module. The two feature matrices and the adjacency matrix are passed into the positive and negative sample pair construction module, and the positive and negative sample pair sets of each node are obtained in combination with the graph topology.

[0071] S3: Perform contrast loss training on the positive and negative sample pairs, and train the node features of the first feature matrix and the second feature matrix by minimizing the loss function to obtain target features;

[0072] This step can be performed in the cross-view comparison module. The positive and negative sample pairs are passed into the cross-view comparison module for contrast loss training. The node features in the two feature matrices are trained by minimizing the loss function.

[0073] S4: Process the target features to obtain output results.

[0074] This step can be performed in the normalization module. The two trained feature matrices are passed into the normalization module, and the corresponding node features are added in a certain ratio to obtain the final output.

[0075] based on Figure 1 The present specification also provides some specific implementation methods of the method. Figure 2 Provide explanation.

[0076] Step S1: Determine the node features and the first feature matrix of the GNN encoding graph, and determine the second feature matrix of the Transformer encoding graph. The specific steps are:

[0077] Since two feature encoders, GNN and Transformer, are used for graph contrastive learning, the node features and adjacency matrix of the graph need to be input into the GNN encoder, and the adjacency matrix of the graph needs to be input into the Transformer.

[0078] The present invention uses a graph convolutional network (GCN) as the GNN encoder and a NodeFormer as the Transformer encoder. The adjacency matrix and the node feature matrix are input into the two encoders respectively to obtain two feature representation matrices. The two graph encoders are introduced as follows:

[0079] 1) Graph Convolutional Network (GCN): Given an adjacency matrix A of size n×n, a degree matrix D and node features X, GCN extends the traditional convolution operation to the graph structure to achieve node representation learning of the graph.

[0080] Specifically, the core operation of GCN is to aggregate and update information based on the features of the nodes and their neighbors in the graph. The standard GCN model can be expressed by formula (1):

[0081]

[0082] in, It is the adjacency matrix A plus the self-loop, that is, the identity matrix I, which ensures that each node contains its own characteristics; yes The corresponding degree matrix; H (l) is the node feature matrix of the lth layer, initially H (0) is the input node feature X; W (l) is the learnable weight matrix of layer l; σ is the activation function (such as ReLU).

[0083] The node representation is updated layer by layer by performing a standardized weighted summation of the adjacent node features and combining the learnable weights W. The convolution operation of each layer gradually enriches the feature expression of the node and captures the local graph structure information by aggregating the information of each node and its neighboring nodes.

[0084] 2) NodeFormer: NodeFormer improves the traditional Transformer encoder by using the kernel function approximation method to reduce the complexity from the original O(N 2 ) becomes O(N), greatly improving the computational efficiency. For a given adjacency matrix A of size n×n and node feature X, for the update of each node feature, see formula (2):

[0085]

[0086] in, represents the kernel function, The query Q, key K and value V matrices are obtained by linear projection, that is, Q = XW Q , K = XW K and V=XW V ;

[0087] For the above two encoders, the graph adjacency matrix A, degree matrix D and node features X are input into the GCN encoder to obtain the feature matrix H θ ; Input the adjacency matrix A and node feature x into the NodeFormer encoder to obtain the feature matrix

[0088] Step S2: Based on the first feature matrix, the second feature matrix and the graph topology structure, a set of positive and negative sample pairs of each node is obtained. The specific steps of step S2 are:

[0089] S2.1: First, the two feature matrices H θ and Perform cosine similarity measurement to obtain the cosine similarity between each node and all other nodes, and sort them from large to small to obtain two feature neighbor matrices B respectively. θ and

[0090] S2.2: Obtain the topological neighbor sorting of each node through the adjacency matrix A, that is, sort the node from small to large according to the number of hops between the node and all other nodes, and obtain the topological neighbor matrix T;

[0091] S2.3: Take out the i-th node v respectively i The set of k-NN nodes in two feature neighbor matrices and And take out v iThe set of k-NN nodes T in the topological neighbor matrix T i , and take the intersection of the three sets to get v i The positive sample pair set P i , and treat all other nodes as v i Negative sample pairs, get v i The negative sample pair set N i , where v i The positive sample pair set P i And the negative sample set N i The construction of is shown in formula (3)-(4):

[0092]

[0093] N i =V\(P i ∪{v i}) (4)

[0094] Where V represents the set of all nodes in the graph.

[0095] Step S3: performing contrast loss training on the positive and negative sample pairs, and training the node features of the first feature matrix and the second feature matrix by minimizing the loss function to obtain target features;

[0096] Further, the specific steps of step S3 are:

[0097] Take node v i For example, in the feature matrix H θ and The features in are expressed as and Node v i The characteristic is expressed as The loss function is formula (5):

[0098]

[0099] The loss function of all nodes can be expressed as formula (6):

[0100]

[0101] Step S4: Process the target features to obtain output results. Further, the two trained feature matrices are passed into the normalization module, and the corresponding node features are added in a certain ratio to obtain the final output;

[0102] The specific steps are:

[0103] S4.1: Save the trained model, then input the graph adjacency matrix A, degree matrix D and node feature X into the saved GCN encoder and NodeFormer encoder respectively, and obtain the final trained two feature matrices H' θ and The two feature matrices are then normalized using the batch normalization method BatchNorm, which is expressed as follows:

[0104] H″ θ =BatchNorm(H' θ ) (7)

[0105]

[0106] S4.2: Add the two feature matrices in a certain ratio to obtain the final feature output matrix H;

[0107] It is expressed as formula (9):

[0108]

[0109] Where λ is the weight ratio parameter.

[0110] The method of the present invention can more accurately capture the characteristic relationship between nodes in the graph, improve the performance of the model in various graph comparison learning tasks, and has high generalizability and practical value.

[0111] Furthermore, the present invention aims to combine GNN and Transformer and integrate the topological structure information of the graph. Through this method, the model can obtain more accurate positive and negative sample pairs of nodes, so as to cope with downstream tasks more effectively.

[0112] The present invention improves the limitation of existing methods that rely only on a single encoder for graph contrast learning, and enables the model to learn more accurate node representations. The present invention has broad application potential and is applicable to a variety of network structures and application scenarios. Since graph contrast learning has applications in multiple fields, it is expected to promote interdisciplinary research and cooperation, combining expertise and methods from different fields. In short, this invention is of great significance for graph deep learning and other fields.

[0113] Based on the same idea, the present invention also provides a graph contrast learning system that combines GNN and Transformer. Figure 3 As shown, the system may include:

[0114] Feature encoding module 310, positive and negative sample pair construction module 320, cross-view comparison module 330 and normalization module 340;

[0115] The feature encoding module 310 is used to determine the node features and the first feature matrix of the GNN encoding graph; and to determine the second feature matrix of the Transformer encoding graph;

[0116] The positive and negative sample pair construction module 320 is used to obtain a set of positive and negative sample pairs for each node based on the first feature matrix, the second feature matrix and the graph topology structure;

[0117] The cross-view comparison module 330 is used to perform contrast loss training on the positive and negative sample pairs, and train the node features of the first feature matrix and the second feature matrix by minimizing the loss function to obtain target features;

[0118] The normalization module 340 is used to process the target features to obtain an output result.

[0119] based on Figure 3 The system in the example may also include some specific implementation units:

[0120] Optionally, the feature encoding module 310 includes:

[0121] A first feature matrix determination unit is used to input the adjacency matrix, degree matrix and node features of the graph into a GNN encoder to obtain a first feature matrix; the GNN encoder adopts a graph convolutional network;

[0122] The second feature matrix determination unit is used to input the adjacency matrix and the node features into the Transformer encoder to obtain the second feature matrix; the Transformer encoder adopts NodeFormer.

[0123] Optionally, the positive and negative sample pair construction module 320 includes:

[0124] Calculating the cosine similarity of the first feature matrix and the second feature matrix;

[0125] A cosine similarity calculation unit, used to arrange in descending order based on the cosine similarities to obtain a first characteristic neighbor matrix and a second characteristic neighbor matrix;

[0126] A topological neighbor matrix determining unit, used to determine the topological neighbor order of each node based on the adjacency matrix to obtain a topological neighbor matrix;

[0127] A first node selection unit, used to respectively extract a first node set and a second node set of a preset node in the first characteristic neighbor matrix and the second characteristic neighbor matrix;

[0128] A second node selection unit, used for taking out a third node set of a preset node in the topological neighbor matrix;

[0129] An intersection unit is used to take the intersection of the first node set, the second node set and the third node set to obtain a positive sample pair set of the preset node, and use all other nodes as negative sample pairs of the preset node to obtain a negative sample pair set of the preset node.

[0130] Optionally, in the positive and negative sample pair construction module 320, the positive sample pair set of the preset node is expressed as:

[0131]

[0132] Among them, B θ represents the first characteristic neighbor matrix, represents the second characteristic neighbor matrix, v i represents a preset node, the preset node is the i-th node, represents the first node set, represents the second node set, T i represents the third node set, P i Represents a set of positive sample pairs of preset nodes;

[0133] In the positive and negative sample pair construction module 320, the negative sample pair set of the preset node is expressed as:

[0134] N i =V\(P i ∪{v i})

[0135] Among them, V represents the set of all nodes in the graph, N i Represents a set of negative sample pairs of preset nodes.

[0136] Optionally, in the cross-view comparison module 330, node v i The characteristic is expressed as The loss function is:

[0137]

[0138] In the cross-view comparison module 330, the loss function of all nodes can be expressed as:

[0139]

[0140] Among them, H θ is the first characteristic matrix, is the second characteristic matrix, in the first characteristic matrix H θ The features in are expressed as In the second characteristic matrix The features in are expressed as represents the loss function of all nodes, Represents node v i Features The loss function is Represents node v i Features The loss function of .

[0141] Optionally, the normalization module 340 includes:

[0142] A normalization unit, used for batch normalization of the first feature matrix and the second feature matrix;

[0143] The output matrix determination unit is used to add the two feature matrices after batch normalization according to a preset ratio to obtain an output matrix.

[0144] Optionally, normalization units are used to:

[0145] Using the formula:

[0146] H″ θ =BatchNorm(H′ θ )

[0147] Normalize the first feature matrix corresponding to the trained model; where H' θ The first feature matrix corresponding to the trained model;

[0148] Using the formula:

[0149]

[0150] Normalize the second feature matrix corresponding to the trained model; where: The second feature matrix corresponding to the trained model;

[0151] The output matrix determines the unit used to:

[0152] Using the formula:

[0153]

[0154] The two feature matrices after batch normalization are added according to the preset ratio to obtain the output matrix H.

[0155] Based on the same idea, the embodiment of this specification also provides a graph comparison learning device that collaborates with GNN and Transformer. Figure 4 As shown, the device includes:

[0156] A memory, a processor, and a communication interface coupled to the processor; the memory stores a computer program that can be executed by the processor; when the processor runs the computer program, it executes the aforementioned GNN and Transformer collaborative graph comparison learning method.

[0157] A computer storage medium is also provided, in which instructions are stored. When the instructions are executed, the aforementioned GNN and Transformer collaborative graph comparison learning method is implemented.

[0158] One or more embodiments of the present specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. One or more embodiments of the present specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0159] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0160] The above description is only an embodiment of the present specification and is not intended to limit one or more embodiments of the present specification. For those skilled in the art, one or more embodiments of the present specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of one or more embodiments of the present specification shall be included in the scope of the claims of one or more embodiments of the present specification.

Claims

1. A graph contrast learning method based on GNN and Transformer, characterized in that: Methods include: Determine the node features and the first feature matrix of the GNN encoding graph, and determine the second feature matrix of the Transformer encoding graph; Obtaining a set of positive and negative sample pairs for each node based on the first feature matrix, the second feature matrix and the graph topology structure; Perform contrast loss training on the set of positive and negative sample pairs, and train the node features of the first feature matrix and the second feature matrix by minimizing the loss function to obtain target features; The target features are processed to obtain output results.

2. According to claim 1, a graph contrast learning method for collaborative GNN and Transformer is characterized in that: The determining of the node features and the first feature matrix of the GNN encoding graph, and the determining of the second feature matrix of the Transformer encoding graph, include: Inputting the adjacency matrix, degree matrix and node features of the graph into a GNN encoder to obtain a first feature matrix; the GNN encoder adopts a graph convolutional network; The adjacency matrix and the node features are input into a Transformer encoder to obtain a second feature matrix; the Transformer encoder adopts NodeFormer.

3. According to claim 1, a graph contrast learning method based on GNN and Transformer collaboration is characterized in that: The step of obtaining a set of positive and negative sample pairs for each node based on the first feature matrix, the second feature matrix and a graph topology structure includes: Calculating the cosine similarity of the first feature matrix and the second feature matrix; Arrange the cosine similarities in descending order to obtain a first characteristic neighbor matrix and a second characteristic neighbor matrix; Determine the topological neighbor order of each node based on the adjacency matrix to obtain a topological neighbor matrix; Respectively extracting a first node set and a second node set of a preset node in the first characteristic neighbor matrix and the second characteristic neighbor matrix; Taking out a third node set of a preset node in the topological neighbor matrix; The intersection of the first node set, the second node set and the third node set is taken to obtain a positive sample pair set of the preset node, and all other nodes are used as negative sample pairs of the preset node to obtain a negative sample pair set of the preset node.

4. According to claim 3, a GNN and Transformer collaborative graph contrast learning method is characterized in that: The positive sample pair set of the preset node is expressed as: Among them, B θ represents the first characteristic neighbor matrix, represents the second characteristic neighbor matrix, v i represents a preset node, the preset node is the i-th node, represents the first node set, represents the second node set, T i represents the third node set, P i Represents a set of positive sample pairs of preset nodes; The negative sample pair set of the preset node is expressed as: N i =V\(P i ∪{v i }) Among them, V represents the set of all nodes in the graph, N i Represents a set of negative sample pairs of preset nodes.

5. According to claim 4, a graph contrast learning method for collaborative GNN and Transformer is characterized in that: Node v i The characteristic is expressed as The loss function is: The loss function of all nodes can be expressed as: Among them, H θ is the first characteristic matrix, is the second characteristic matrix, in the first characteristic matrix H θ The features in are expressed as In the second characteristic matrix The features in are expressed as represents the loss function of all nodes, Represents node v i Features The loss function is Represents node v i Features The loss function of .

6. The graph contrast learning method of GNN and Transformer collaboration according to claim 1, characterized in that: The processing of the target feature to obtain an output result includes: Batch normalize the first feature matrix and the second feature matrix; The two feature matrices after batch normalization are added according to the preset ratio to obtain the output matrix.

7. The graph contrast learning method of GNN and Transformer collaboration according to claim 6, characterized in that: The batch normalizing the first feature matrix and the second feature matrix includes: Using the formula: H″ θ =BatchNorm(H′ θ ) Normalize the first feature matrix corresponding to the trained model; where H' θ The first feature matrix corresponding to the trained model; Using the formula: Normalize the second feature matrix corresponding to the trained model; where: The second feature matrix corresponding to the trained model; The two feature matrices after batch normalization are added according to a preset ratio to obtain an output matrix, including: Using the formula: The two feature matrices after batch normalization are added according to the preset ratio to obtain the output matrix H.

8. A GNN and Transformer collaborative graph contrast learning system, characterized in that: The system includes: Feature encoding module, positive and negative sample pair construction module, cross-view comparison module and normalization module; The feature encoding module is used to determine the node features and the first feature matrix of the GNN encoding graph; and is used to determine the second feature matrix of the Transformer encoding graph; The positive and negative sample pair construction module is used to obtain a set of positive and negative sample pairs for each node based on the first feature matrix, the second feature matrix and the graph topology structure; The cross-view comparison module is used to perform contrast loss training on the positive and negative sample pairs, and train the node features of the first feature matrix and the second feature matrix by minimizing the loss function to obtain the target features; The normalization module is used to process the target features to obtain an output result.

9. A graph contrast learning device for collaborative GNN and Transformer, characterized in that the device include: A memory, a processor, and a communication interface coupled to the processor; The memory stores a computer program executable by the processor; When the processor runs the computer program, it executes the GNN and Transformer collaborative graph comparison learning method as described in any one of claims 1 to 7.

10. A computer storage medium, characterized in that: The computer storage medium stores instructions, and when the instructions are executed, the GNN and Transformer collaborative graph comparison learning method described in any one of claims 1 to 7 is implemented.

Images