Causal subgraph extraction method and device based on information bottleneck and causal learning
The causal subgraph extraction method using information bottleneck and causal learning addresses the interference of noise and pseudo-correlation substructures, enhancing the credibility and accuracy of graph classification by iteratively training with specific loss functions to eliminate these interferences.
Patent Information
- Application Number
- CN202410825090.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-25
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2044-06-25
AI Technical Summary
Existing graph classification methods using graph neural networks struggle to accurately extract interpretable subgraphs due to the interference of noise and pseudo-correlation substructures, which compromises the credibility of prediction results, particularly in fields like biomedicine where causal explanations are crucial.
A causal subgraph extraction method utilizing information bottleneck and causal learning, involving iterative training with causal learning, classification, and contrast loss functions to eliminate pseudo-correlation and noise subgraphs, ensuring the extracted subgraphs are credible.
The method effectively removes noise and pseudo-correlation subgraphs, enhancing the credibility and accuracy of causal subgraph extraction, improving the interpretability and reliability of graph classification tasks.
Smart Images

Figure CN118379565B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of computer systems based on specific computing models, and in particular to a method and device for extracting a causal subgraph based on information bottleneck and causal learning. Background Art
[0002] In recent years, graph classification technology based on graph neural networks has developed rapidly and has been widely used in fields such as biochemistry, social networks, and computer vision. However, simply obtaining a neural network with good classification performance may not be enough for its practical application in some fields, because many scenarios also have high requirements for the credibility of the prediction results, that is, the ability to give an explanation for the prediction results. For example, in the field of biomedicine, discovering key substructures that can lead to fixed characteristics is more helpful for the exploration and development of new drugs than the characteristic prediction results. Therefore, identifying key subgraph structures used to explain and further improve classification results has a strong practical significance.
[0003] In order to extract interpretable subgraph structures, it is necessary to explore the causality between subgraphs and labels in essence and extract substructures with causal effects as interpretable results. However, accurately identifying causal subgraphs in actual scenarios is very challenging. First, the components of the input graph are complex and unknown. In addition to the causal part, the remaining interference parts are also likely to be incorrectly included in the extracted causal subgraph. Since there is no corresponding causal relationship between the interference structure and the label, incorporating the interference structure into the extracted interpretable subgraph will greatly reduce the credibility of the interpretation result.
[0004] Interference structures usually include noise substructures and pseudo-correlation substructures, where the noise substructure is irrelevant to the label of the graph, while the pseudo-correlation substructure is usually pseudo-correlated with the label during the data generation process due to a certain probabilistic dependence on the causal structure. At present, some methods limit the mutual information between the extracted subgraph and the original graph to eliminate noise while maintaining the predictive ability of the extracted subgraph, but ignore the possible pseudo-correlation substructure that is also related to the label and has a certain predictive ability, resulting in the inclusion of some pseudo-correlation structures in the results. Another type of method focuses on causal relationships to remove the pseudo-correlation part, but there is no clear constraint on the noise subgraph, resulting in the causal subgraph being mixed with noise. Therefore, in the face of the problem that noise and pseudo-correlation structures may exist at the same time in more general scenarios, if the real interpretable substructure is to be obtained, the model must have the ability to eliminate both noise and pseudo-correlation substructures. Summary of the invention
[0005] The present invention provides a causal subgraph extraction method and device based on information bottleneck and causal learning, which are used to solve the defect that the prior art is easily interfered by noise subgraphs and pseudo-correlation subgraphs when extracting interpretable subgraphs from graph structure data, and achieves targeted elimination of noise subgraphs and pseudo-correlation subgraphs, thereby effectively improving the credibility of the extracted causal subgraph.
[0006] The present invention provides a causal subgraph extraction method based on information bottleneck and causal learning, comprising the following steps.
[0007] Acquire graph data to be processed, wherein the graph data to be processed includes a causal subgraph, and the graph data to be processed also includes at least one of a noise subgraph and a pseudo-correlation subgraph;
[0008] Inputting the graph data to be processed into a causal subgraph extraction model to perform subgraph extraction processing to obtain the causal subgraph;
[0009] Among them, the causal subgraph extraction model is obtained by iterative training based on causal learning loss function, classification loss function and contrast loss function. The causal learning loss function is used to eliminate pseudo-correlated subgraphs in the graph data to be processed, and the classification loss function and contrast loss function are used to eliminate noise subgraphs in the graph data to be processed.
[0010] According to a causal subgraph extraction method based on information bottleneck and causal learning provided by the present invention, the training method of the causal subgraph extraction model includes:
[0011] Constructing a sample graph data set and an initial causal subgraph extraction model, wherein the sample graph data set includes at least one batch of sample graphs;
[0012] Performing feature extraction processing on the sample graph to obtain a whole-graph representation, a causal subgraph representation, and a pseudo-correlation subgraph representation of the sample graph, wherein the causal subgraph representation is a feature representation of the causal subgraph of the sample graph, and the pseudo-correlation subgraph representation is a feature representation of the pseudo-correlation subgraph of the sample graph;
[0013] For the sample graphs of the same batch, feature fusion processing is performed based on the causal subgraph representation and the pseudo-correlation subgraph representation to obtain intervention sample representation;
[0014] Inputting the causal subgraph representation, the pseudo-correlation subgraph representation, and the intervention sample representation into the initial causal subgraph extraction model for prediction processing, respectively, to obtain a first predicted label value, and determining a causal learning loss function based on the first predicted label value and the true label value;
[0015] Determining a sample pair based on the whole graph representation and the causal subgraph representation, and determining a contrast loss function based on the sample pair;
[0016] Inputting the whole graph representation into the initial causal subgraph extraction model for prediction processing to obtain a second predicted label value, and determining a classification loss function based on the second predicted label value and the true label value;
[0017] Based on the causal learning loss function, the contrast loss function and the classification loss function, the parameters of the initial causal subgraph extraction model are iteratively updated to obtain the causal subgraph extraction model.
[0018] According to a causal subgraph extraction method based on information bottleneck and causal learning provided by the present invention, the feature fusion processing is performed based on the causal subgraph representation and the pseudo-correlation subgraph representation to obtain the intervention sample representation, including:
[0019] Randomly shuffle the causal subgraph representation to obtain a shuffled causal subgraph representation;
[0020] The disordered causal subgraph representation and the pseudo-correlation subgraph representation are feature concatenated to obtain the intervention sample representation.
[0021] According to a causal subgraph extraction method based on information bottleneck and causal learning provided by the present invention, the sample pairs include positive sample pairs and negative sample pairs, the sample pairs are determined based on the whole graph representation and the causal subgraph representation, and the contrast loss function is determined based on the sample pairs, including:
[0022] Inputting the causal subgraph representation and the whole graph representation into a mapper for feature mapping processing to obtain the causal subgraph representation and the whole graph representation in the same feature space;
[0023] For the causal subgraph representation and the whole graph representation of the same feature space, the causal subgraph representation and the whole graph representation respectively corresponding to the same sample graph are combined into the positive sample pair;
[0024] Combining the causal subgraph representations corresponding to different sample graphs with the whole graph representations into the negative sample pairs;
[0025] The contrast loss function is obtained by minimizing the similarity score between the positive sample pairs and maximizing the similarity score between the negative sample pairs as optimization goals.
[0026] According to a causal subgraph extraction method based on information bottleneck and causal learning provided by the present invention, the parameters of the initial causal subgraph extraction model are iteratively updated based on the causal learning loss function, the contrast loss function and the classification loss function to obtain a causal subgraph extraction model, including:
[0027] Fixing other model parameters in the initial causal subgraph extraction model except the parameters of the mapper, and iteratively updating the parameters of the mapper with the goal of maximizing the contrast loss function;
[0028] The parameters of the iteratively updated mapper are fixed, and with the goal of minimizing the causal learning loss function and the classification loss function, the parameters of the initial causal subgraph extraction model are iteratively updated through back propagation and gradient descent algorithms to obtain the causal subgraph extraction model.
[0029] According to a causal subgraph extraction method based on information bottleneck and causal learning provided by the present invention, the feature extraction process is performed on the sample graph to obtain the whole graph representation, causal subgraph representation and pseudo-correlation subgraph representation of the sample graph, including:
[0030] Inputting the sample graph into a graph neural network for feature encoding processing to obtain a node representation corresponding to each node of the sample graph;
[0031] Performing pooling processing on the node representation to obtain a whole-graph representation of the sample graph;
[0032] Performing feature concatenation processing on the node representation to obtain the representation of the edge corresponding to the node;
[0033] Inputting the representation of the edge into the feature discriminator for weight determination to obtain the importance weight of each edge, and taking the edge whose importance weight is greater than or equal to the weight threshold as the causal subgraph, and taking the edge whose importance weight is less than the weight threshold as the pseudo-correlation subgraph;
[0034] Feature extraction is performed on the causal subgraph and the pseudo-correlation subgraph respectively to obtain the causal subgraph representation and the pseudo-correlation subgraph representation.
[0035] According to a causal subgraph extraction method based on information bottleneck and causal learning provided by the present invention, the causal subgraph and the pseudo-correlation subgraph are subjected to feature extraction processing respectively to obtain the causal subgraph representation and the pseudo-correlation subgraph representation, including:
[0036] Inputting the causal subgraph and the pseudo-correlation subgraph into the graph neural network for feature encoding processing respectively, to obtain causal node representations corresponding to each node of the causal subgraph, and pseudo-correlation node representations corresponding to each node of the pseudo-correlation subgraph;
[0037] The causal node representation and the pseudo-correlation node representation are respectively subjected to pooling processing to obtain the causal subgraph representation and the pseudo-correlation subgraph representation.
[0038] The present invention also provides a causal subgraph extraction device based on information bottleneck and causal learning, comprising the following modules:
[0039] An acquisition module, used for acquiring graph data to be processed, wherein the graph data to be processed includes a causal subgraph, and the graph data to be processed also includes at least one of a noise subgraph and a pseudo-correlation subgraph;
[0040] A processing module, used for inputting the graph data to be processed into a causal subgraph extraction model to perform subgraph extraction processing to obtain the causal subgraph;
[0041] Among them, the causal subgraph extraction model is obtained by iterative training based on causal learning loss function, classification loss function and contrast loss function. The causal learning loss function is used to eliminate pseudo-correlated subgraphs in the graph data to be processed, and the classification loss function and contrast loss function are used to eliminate noise subgraphs in the graph data to be processed.
[0042] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a causal subgraph extraction method based on information bottleneck and causal learning as described above is implemented.
[0043] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the causal subgraph extraction methods based on information bottleneck and causal learning as described above.
[0044] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements any of the above-mentioned causal subgraph extraction methods based on information bottleneck and causal learning.
[0045] The present invention provides a causal subgraph extraction method based on information bottleneck and causal learning. Since the causal subgraph extraction model is obtained by iterative training based on causal learning loss function, classification loss function and contrast loss function, and the causal learning loss function is used to eliminate pseudo-correlated subgraphs in the graph data to be processed, and the classification loss function and contrast loss function are used to eliminate noise subgraphs in the graph data to be processed, the causal subgraph extraction model provided by the present invention can effectively eliminate the influence of pseudo-correlated subgraphs and noise subgraphs in the graph data, and then when the graph data to be processed is input into the causal subgraph extraction model for subgraph extraction processing, the credibility of the obtained causal subgraph is higher. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0047] Figure 1 It is one of the flow charts of the causal subgraph extraction method based on information bottleneck and causal learning provided by the present invention.
[0048] Figure 2 It is a schematic diagram of the training process of the causal subgraph extraction model provided by the present invention.
[0049] Figure 3 This is the second flow chart of the causal subgraph extraction method based on information bottleneck and causal learning provided by the present invention.
[0050] Figure 4 It is a schematic diagram of a causal graph in the graph data generation process provided by the present invention.
[0051] Figure 5 It is a schematic diagram of a causal subgraph in the test results provided by the present invention.
[0052] Figure 6 It is a structural schematic diagram of a causal subgraph extraction device based on information bottleneck and causal learning provided by the present invention.
[0053] Figure 7 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION
[0054] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0055] Figure 1 is one of the flow charts of the causal subgraph extraction method based on information bottleneck and causal learning provided by the present invention, such as Figure 1 As shown, the method includes the following steps 101 to 102.
[0056] Step 101: Obtain graph data to be processed.
[0057] The graph data to be processed includes a causal subgraph, and the graph data to be processed also includes at least one of a noise subgraph and a pseudo-correlation subgraph. Graph data is an unstructured data, including a number of nodes, and the nodes are connected by edges. A noise subgraph refers to a data point in the graph data that is inconsistent or irrelevant with other data. These noises may come from errors in the data collection process, interference in the transmission process, or errors in the data processing process; a pseudo-correlation subgraph refers to a false correlation between certain nodes or subgraphs and target labels in graph data analysis. In the actual data collection process, some irrelevant features may be associated with the target label due to the subjective bias of the collector. For example, in the analysis of molecular properties, the -OH functional group actually determines the properties of the molecule, but due to the bias when collecting data, there may also be features in other parts of the molecule that are correlated with the properties annotated with it; a causal subgraph refers to a subgraph extracted by decoupling and identifying causal relationships in a graph neural network (GNN). The causal subgraph can help the model better understand and utilize the causal information in the graph data, thereby improving the accuracy and interpretability of the prediction.
[0058] See also Figure 4 , Figure 4 is a schematic diagram of a causal graph in the graph data generation process provided by the present invention, Figure 4 In the above formula, G represents graph data, S represents pseudo-correlation substructure (i.e., pseudo-correlation subgraph), C represents causal structure (i.e., causal subgraph), N represents noise substructure (i.e., noise subgraph), and Y represents label. Under normal circumstances, the label Y of the graph is directly determined only by its causal part C, while the pseudo-correlation substructure S has some pseudo-correlation with Y due to some probabilistic association with C during the data generation process.
[0059] Step 102: Input the graph data to be processed into a causal subgraph extraction model to perform subgraph extraction processing to obtain the causal subgraph.
[0060] Among them, the causal subgraph extraction model is obtained by iterative training based on causal learning loss function, classification loss function and contrast loss function. The causal learning loss function is used to eliminate pseudo-correlated subgraphs in the graph data to be processed, and the classification loss function and contrast loss function are used to eliminate noise subgraphs in the graph data to be processed.
[0061] In some embodiments, see Figure 2 , Figure 2 is a schematic diagram of the training process of the causal subgraph extraction model provided by the present invention, such as Figure 2 As shown, the training method of the causal subgraph extraction model includes the following steps 201 to 207.
[0062] Step 201: construct a sample graph dataset and an initial causal subgraph extraction model.
[0063] Specifically, we first need to construct a sample graph dataset, including a training set and a test set. The training set contains a graph dataset of pseudo-correlation substructures and noise substructures. For each graph in the dataset, its initial features are represented as a matrix consisting of all node features, and each graph is labeled. The true causal interpretable subgraphs (i.e., causal subgraphs) in the graph dataset are labeled for subsequent model performance evaluation. In addition, all graphs in the graph dataset need to be divided into multiple batches, and the multiple graphs in each batch need to be preprocessed into a large graph so that the data in a batch can be processed as a whole during the training process, that is, the sample graph dataset includes at least one batch of sample graphs.
[0064] Step 202: Perform feature extraction processing on the sample graph to obtain a whole-graph representation, a causal subgraph representation, and a pseudo-correlation subgraph representation of the sample graph.
[0065] In some embodiments, the causal subgraph representation is a feature representation of the causal subgraph of the sample graph, and the pseudo-correlation subgraph representation is a feature representation of the pseudo-correlation subgraph of the sample graph.
[0066] In some embodiments, step 202 can be implemented in the following manner: input the sample graph into a graph neural network for feature encoding processing to obtain node representations corresponding to each node of the sample graph; perform pooling processing on the node representations to obtain a whole-graph representation of the sample graph; perform feature splicing processing on the node representations to obtain edge representations corresponding to the nodes; input the edge representations into a feature discriminator for weight determination to obtain the importance weights of each edge, and use the edges whose importance weights are greater than or equal to the weight threshold as the causal subgraph, and use the edges whose importance weights are less than the weight threshold as the pseudo-correlation subgraph; perform feature extraction processing on the causal subgraph and the pseudo-correlation subgraph, respectively, to obtain the causal subgraph representation and the pseudo-correlation subgraph representation.
[0067] In some embodiments, feature extraction is performed on the causal subgraph and the pseudo-correlation subgraph respectively to obtain the causal subgraph representation and the pseudo-correlation subgraph representation, which can be achieved in the following ways: the causal subgraph and the pseudo-correlation subgraph are respectively input into the graph neural network for feature encoding processing to obtain a causal node representation corresponding to each node of the causal subgraph, and a pseudo-correlation node representation corresponding to each node of the pseudo-correlation subgraph; and the causal node representation and the pseudo-correlation node representation are respectively pooled to obtain the causal subgraph representation and the pseudo-correlation subgraph representation.
[0068] Specifically, for an input sample graph in a batch , assuming that its node set is represented as , the edge set is represented as First, through the graph neural network For the sample graph Perform feature encoding processing to obtain a sample graph The node representation corresponding to each node ,in , d represents the feature dimension; then the node representation The representation of the edge corresponding to the node is generated through the following feature splicing processing .
[0069]
[0070] in, and Represent the features and adjacency matrix of the input sample graph respectively, and represents any node, Represented by the node and Consisting of edges and nodes Used to generate a mask matrix to complete the subsequent sub-image screening process.
[0071] And the node characterization Perform pooling to obtain the whole image representation of the sample image ,in .
[0072] Then the edge representation Input to a feature discriminator composed of a multi-layer perceptron The feature importance is judged in and the edge mask matrix is output , used to extract causal substructure and generate causal subgraph The adjacency matrix of ,in ,picture The remaining part is the pseudo-correlation subgraph , whose adjacency matrix is expressed as ,in , the specific calculation method is as follows:
[0073]
[0074] in, , Represents the entire subgraph structure extractor.
[0075] Get the extracted causal subgraph and pseudo-correlation subgraph Finally, two graph neural network encoders are introduced and The extracted and Perform feature coding calculation and processing to obtain the pseudo-correlation node representation corresponding to each node of the pseudo-correlation subgraph ,in , and the causal node representation corresponding to each node of the causal subgraph ,in , the calculation formula is as follows:
[0076]
[0077] Then, the causal node representation and pseudo-correlation node representation are pooled to read the node representation information and obtain the causal subgraph representation respectively. and pseudo-correlation subgraph representation ,in, , , the calculation formula is as follows:
[0078]
[0079] Among them, the Readout() function represents a pooling function that compresses the information represented by the node to obtain the full graph representation.
[0080] Since graph representation can comprehensively represent the semantic information of a graph, and actual operations on a graph are usually difficult, subsequent operations on the graph are replaced by graph representation, making subsequent operations on the graph more convenient and quick.
[0081] Step 203: For the sample graphs of the same batch, feature fusion processing is performed based on the causal subgraph representation and the pseudo-correlation subgraph representation to obtain intervention sample representation.
[0082] In some embodiments, step 203 can be implemented by: randomly shuffling the causal subgraph representation to obtain a shuffled causal subgraph representation; and performing feature concatenation of the shuffled causal subgraph representation and the pseudo-correlation subgraph representation to obtain the intervention sample representation.
[0083] In order to optimize the model parameters so that the extracted causal subgraph can completely contain the causal part and completely separate it from the pseudo-correlation subgraph part, the present invention adopts a causal intervention method. Figure 4 As shown in the causal diagram on the left, Part of the label The only mother node of , so we need to maximize the causal subgraph with label In addition, since the pseudo-correlation subgraph also It has a certain predictive ability. Therefore, in order to prevent the pseudo-correlated subgraph from partially entering the extracted causal substructure, it is necessary to maximize the pseudo-correlated subgraph and the label The mutual information between them can be used to completely separate the pseudo-correlation part from the causal part after optimization.
[0084] Maximizing the causal subgraph and label simultaneously between the pseudo-correlated subgraphs and labels After the mutual information between them is obtained, it is necessary to further ensure that the extracted causal subgraph and pseudo-correlation subgraph will not be mixed with each other and are close to the real causal subgraph and pseudo-correlation subgraph. , that is, Figure 4 As shown in the right side, through intervention Operation, change The distribution of and The relationship between them is destroyed, which and The pseudo-correlation between them is cut off, and the pseudo-correlation part after being cut off is irrelevant to the label, so it loses the ability to predict the original label. Minimize the intervention sample prediction The conditional probability formula is as follows:
[0085]
[0086] This operation minimizes the intervention sample and the label The mutual information between them, and the above two optimization objectives (maximizing the causal sub-label Mutual information between, maximize pseudo-correlation subgraph and The mutual information between the two can be combined to make the extracted causal subgraph and pseudo-correlation subgraph fully reflect the and properties, so that the extracted causal subgraph converges to the true causal substructure.
[0087] Specifically, for all samples in a batch, the causal subgraph parts of all samples need to be randomly shuffled, and then the shuffled causal subgraph parts are spliced with the unchanged pseudo-correlation subgraph to complete the above-mentioned intervention operation on the pseudo-correlation part. Since splicing subgraphs is more difficult, the causal subgraph representation calculated in step 202 is used here for shuffling and splicing operations. , and then shuffle them to get ,in Indicates the batch size. and unchanged Combined , the newly generated sample destroys the distribution of the causal part in the original data, so it can be regarded as the above-mentioned intervention sample representation.
[0088] Step 204: input the causal subgraph representation, the pseudo-correlation subgraph representation and the intervention sample representation into the initial causal subgraph extraction model for prediction processing, obtain a first predicted label value, and determine a causal learning loss function based on the first predicted label value and the true label value.
[0089] Specifically, since the intervention samples obtained in step 203 are irrelevant to the labels, they should be trained through classification tasks to make them lose the ability to predict the original labels. In addition, for maximizing the mutual information between the extracted causal subgraph and the labels, and maximizing the mutual information between the extracted pseudo-correlation subgraph and the labels, since the mutual information is not easy to optimize directly, the approximate operation is to make the extracted causal subgraph represent Pseudo-correlation subgraph representation For the classification task, the mutual information between the two is maximized by minimizing its classification loss. Therefore, the causal learning loss function of the causal learning part is as follows:
[0090]
[0091] in , and is the loss coefficient, which is used to control the role of each loss function. represents the first predicted label, represents the true label, The labels are in the same random order as the causal subgraph. represents the cross entropy loss.
[0092] Step 205: Determine sample pairs based on the whole graph representation and the causal subgraph representation, and determine a contrast loss function based on the sample pairs.
[0093] In some embodiments, the sample pairs include positive sample pairs and negative sample pairs, and step 205 can be implemented in the following manner: input the causal subgraph representation and the whole graph representation into a mapper for feature mapping processing to obtain the causal subgraph representation and the whole graph representation of the same feature space; for the causal subgraph representation and the whole graph representation of the same feature space, the causal subgraph representation corresponding to the same sample graph and the whole graph representation are combined into the positive sample pair; the causal subgraph representation corresponding to different sample graphs and the whole graph representation are combined into the negative sample pair; and the contrast loss function is obtained by minimizing the similarity score between the positive sample pairs and maximizing the similarity score between the negative sample pairs as the optimization goals.
[0094] After removing the pseudo-correlation subgraph, it is still not guaranteed that the model can obtain the true causal subgraph, because there may still be Figure 4 The noise substructure shown in Therefore, it is necessary to introduce the information bottleneck principle, that is, to maximize the causal substructure and label While minimizing the mutual information between the extracted causal substructure and the original graph, it also minimizes the mutual information between the extracted causal substructure and the original graph to prevent the noise substructure irrelevant to the label from entering the final extracted interpretable causal subgraph.
[0095] Specifically, step 204 already includes the optimization goal of maximizing the mutual information between the causal subgraph and the label, so this step mainly achieves minimizing the mutual information between the extracted causal subgraph and the full graph. Since the mutual information between graphs is not easy to calculate, the mutual information between the subgraph representation and the original graph representation is still used here to approximate the mutual information between graphs. And since the mutual information is not easy to optimize directly, InfoNCE-based contrastive learning is introduced to estimate the mutual information. This contrastive learning method can provide an effective lower bound of the mutual information as an optimization target.
[0096] Contrastive learning represents causal subgraphs from the same graph within a batch With full-image representation As positive sample pairs, the causal subgraphs from different graphs are represented and full-image representation As negative sample pairs, in order to calculate the similarity between positive sample pairs and negative sample pairs, a mapper composed of multi-layer perceptrons is introduced The full-graph representation and the causal subgraph representation are mapped into the same space, and then the contrast loss function is obtained by minimizing the similarity between positive sample pairs and maximizing the similarity between negative sample pairs. , thus approximately estimating the minimization of the mutual information between the two. The calculation formula of the contrast loss function is as follows:
[0097]
[0098] in, is a function that calculates the cosine similarity of two vectors. represents the temperature coefficient required for contrast learning, Represents a mapping process of a mapper.
[0099] Step 206: input the whole graph representation into the initial causal subgraph extraction model for prediction processing to obtain a second predicted label value, and determine a classification loss function based on the second predicted label value and the true label value.
[0100] In some embodiments, in order to ensure the quality of the full image representation and thus ensure the approximation of contrastive learning for mutual information optimization, the full image representation is Perform graph classification and optimize its classification loss , and its calculation formula is as follows:
[0101]
[0102] in, represents the second predicted label, represents the true label, represents the cross entropy loss.
[0103] Therefore, in order to eliminate the noise substructure, the overall loss as follows:
[0104]
[0105] in, and is the loss coefficient.
[0106] Step 207: Based on the causal learning loss function, the contrast loss function and the classification loss function, iteratively update the parameters of the initial causal subgraph extraction model to obtain the causal subgraph extraction model.
[0107] In some embodiments, step 207 can be implemented in the following manner: fixing the other model parameters in the initial causal subgraph extraction model except the parameters of the mapper, and iteratively updating the parameters of the mapper with the goal of maximizing the contrast loss function; fixing the parameters of the iteratively updated mapper, and iteratively updating the parameters of the initial causal subgraph extraction model through back propagation and gradient descent algorithms with the goal of minimizing the causal learning loss function and the classification loss function, to obtain the causal subgraph extraction model.
[0108] Specifically, since the mapper in InfoNCE The optimization needs to maximize the mutual information between the estimated causal subgraph and the full graph representation, and the overall optimization goal is The mutual information needs to be minimized, so the present invention adopts a bi-level optimization strategy. First, all parameters except the mapper parameters are fixed, and then the contrast loss is maximized. To optimize the mapper so that it can map the two into a space suitable for contrastive learning to calculate similarity; then fix the mapper parameters and minimize the overall loss To optimize the parameters of the subgraph neural network encoder, subgraph extractor, and classifier in the model, the optimization strategy can be expressed as follows:
[0109]
[0110] in, represents the parameters of all subgraph neural network encoders and classifiers, Represents a parameter of a mapper.
[0111] Then, the model parameters are iteratively updated through back propagation and gradient descent algorithms. After the model converges, a causal subgraph extraction model can be obtained. This model can extract interpretable causal subgraphs for unseen test graph data through a subgraph extractor, and can be used to complete classification tasks. Figure 5 As shown, Figure 5 is a schematic diagram of a causal subgraph in the test results provided by the present invention, Figure 5 An example of a graph classification dataset for whether a molecule is mutagenic is given, where the "-NO2" structure is a true causal interpretable subgraph.
[0112] In a specific embodiment of the present invention, see Figure 3 , Figure 3 This is the second flow chart of the causal subgraph extraction method based on information bottleneck and causal learning provided by the present invention, such as Figure 3 As shown in the figure, we first construct a graph classification dataset for general scenarios so that the graphs in the dataset contain both noise and pseudo-correlation substructures as much as possible. At the same time, for datasets where real interpretable causal substructures can be obtained in advance, the causal substructures are annotated to facilitate the subsequent measurement and evaluation of interpretable results.
[0113] For the input graph data, the graph neural network is used as an encoder to obtain the latent space representation of the node, which contains the semantic features of the node. The node representation is then used to generate the representation of the edge corresponding to the node by splicing. Similarly, the edge representation can represent the semantics of the edge and can be used for subsequent calculation and screening of the importance of the edge in the graph.
[0114] Then, a feature importance discriminator composed of a multi-layer perceptron is introduced, which takes the representation of the edge as input, obtains the importance weight of each edge in the graph, and then selects the edges with high weight as the extracted causal substructure. At the same time, the remaining part is used as a pseudo-correlation substructure for calculation in the subsequent training process.
[0115] According to the extracted causal substructure and pseudo-correlation substructure, a new graph neural network encoder is introduced to calculate the subgraph node representation, and the node representation is further used to obtain the full graph representation of the entire subgraph using the pooling method. Since the subgraph representation contains rich semantics in the subgraph, it can approximately replace the subgraph for subsequent intervention operations.
[0116] For the pseudo-correlation substructure that may be contained in the graph data, a causal intervention operation is used to intervene in the pseudo-correlation substructure. First, the pseudo-correlation subgraph representation of all graph data in a training batch is kept unchanged, all causal subgraph representations in the batch are disrupted, and randomly combined with the unchanged pseudo-correlation subgraph representation. This operation destroys the association between the causal and pseudo-correlation parts in the data generation process. After that, the pseudo-correlation subgraph representation and the recombined intervention data representation are input into the classifier. By controlling the pseudo-correlation samples to have classification ability while ensuring that the intervention samples have no classification ability, the separation of the causal part and the pseudo-correlation part is effectively promoted, thereby promoting the extracted causal subgraph to be closer to the real interpretable results.
[0117] In view of the noise substructure that may be contained in the graph data, the information bottleneck principle is introduced to reduce the mutual information between the extracted causal subgraph and the whole graph while maintaining the classification ability of the causal subgraph. Specifically, the generated causal subgraph is input into the classifier to improve the classification ability of the causal subgraph. At the same time, the node representation is used to generate the full graph representation, and then the mutual information is approximately estimated by contrastive learning. First, a mapper is introduced to map the causal subgraph and the full graph representation to the same space, and then the causal subgraph representation and the full graph representation from the same graph are regarded as positive sample pairs, and the causal subgraph and the full graph representation of different graphs are regarded as negative sample pairs. By minimizing the similarity between positive sample pairs and maximizing the similarity between negative sample pairs, the contrast loss is minimized, and the mutual information between the causal subgraph and the original graph is minimized.
[0118] The classification loss and contrast loss generated by optimizing the classification ability are used to construct a joint loss function to jointly optimize the model. Since the optimization goal of the mapper introduced in the contrastive learning process is opposite to the overall optimization goal, the present invention adopts a two-layer optimization strategy. First, other parameters are fixed unchanged, and the mapper is trained by maximizing the contrast loss. After that, the mapper parameters are fixed and the joint loss function is minimized. Through back propagation and gradient descent algorithms, the parameters of the graph neural network encoder, subgraph extractor and classifier in the entire model are updated to improve the model's ability to extract true causal subgraphs.
[0119] The present invention is tested on real-world datasets and artificially synthesized datasets. Its graph classification performance is measured by accuracy, and its subgraph interpretability performance is measured by Precision@5, which measures the matching degree between the top 5 edges in the confidence ranking and the real interpretable causal subgraph. The comparison results of the present invention and other key subgraph extraction methods are shown in Tables 1 and 2, respectively.
[0120] It can be seen from Table 1 that the method of the present invention has achieved the best graph classification performance on all data sets, and according to Table 2, the causal subgraph extracted by the present invention can more accurately match the actual causal subgraph compared with the comparison method. This verifies that when facing more complex and general scenarios, the present invention can simultaneously remove the pseudo-correlation part and the noise part, has significant advantages in obtaining accurate interpretable subgraphs, and can better promote the understanding of graph classification tasks and improve performance.
[0121] Table 1: Graph classification results of the present invention and other comparative methods on the test set
[0122]
[0123] Table 2: Precision@5 results of the present invention and other comparison methods on the test set
[0124]
[0125] The causal subgraph extraction device based on information bottleneck and causal learning provided by the present invention is described below. The causal subgraph extraction device based on information bottleneck and causal learning described below and the causal subgraph extraction method based on information bottleneck and causal learning described above can be referenced to each other.
[0126] See also Figure 6 , Figure 6 is a schematic diagram of the structure of a causal subgraph extraction device based on information bottleneck and causal learning provided by the present invention, such as Figure 6 As shown, the causal subgraph extraction device 600 based on information bottleneck and causal learning provided by the present invention includes an acquisition module 601 and a processing module 602, wherein the acquisition module 601 is used to obtain graph data to be processed, the graph data to be processed includes a causal subgraph, and the graph data to be processed also includes at least one of a noise subgraph and a pseudo-correlation subgraph; the processing module 602 is used to input the graph data to be processed into a causal subgraph extraction model for subgraph extraction processing to obtain the causal subgraph; wherein the causal subgraph extraction model is obtained by iterative training based on a causal learning loss function, a classification loss function and a contrast loss function, the causal learning loss function is used to eliminate the pseudo-correlation subgraph in the graph data to be processed, and the classification loss function and the contrast loss function are used to eliminate the noise subgraph in the graph data to be processed.
[0127] Figure 7 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 7As shown, the electronic device may include: a processor 710, a communication interface 720, a memory 730 and a communication bus 740, wherein the processor 710, the communication interface 720 and the memory 730 communicate with each other through the communication bus 740. The processor 710 may call the logic instructions in the memory 730 to execute a causal subgraph extraction method based on information bottleneck and causal learning, the method comprising: obtaining graph data to be processed, wherein the graph data to be processed includes a causal subgraph, and the graph data to be processed also includes at least one of a noise subgraph and a pseudo-correlation subgraph; inputting the graph data to be processed into a causal subgraph extraction model for subgraph extraction processing to obtain the causal subgraph; wherein the causal subgraph extraction model is obtained by iterative training based on a causal learning loss function, a classification loss function and a contrast loss function, wherein the causal learning loss function is used to eliminate the pseudo-correlation subgraph in the graph data to be processed, and the classification loss function and the contrast loss function are used to eliminate the noise subgraph in the graph data to be processed.
[0128] In addition, the logic instructions in the above-mentioned memory 730 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.
[0129] On the other hand, the present invention also provides a computer program product, which includes a computer program, and the computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the causal subgraph extraction method based on information bottleneck and causal learning provided by the above-mentioned methods, and the method includes: obtaining graph data to be processed, wherein the graph data to be processed includes a causal subgraph, and the graph data to be processed also includes at least one of a noise subgraph and a pseudo-correlation subgraph; inputting the graph data to be processed into a causal subgraph extraction model for subgraph extraction processing to obtain the causal subgraph; wherein the causal subgraph extraction model is obtained by iterative training based on a causal learning loss function, a classification loss function and a contrast loss function, and the causal learning loss function is used to eliminate the pseudo-correlation subgraph in the graph data to be processed, and the classification loss function and the contrast loss function are used to eliminate the noise subgraph in the graph data to be processed.
[0130] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the causal subgraph extraction method based on information bottleneck and causal learning provided by the above-mentioned methods, the method comprising: obtaining graph data to be processed, the graph data to be processed including a causal subgraph, and the graph data to be processed also including at least one of a noise subgraph and a pseudo-correlation subgraph; inputting the graph data to be processed into a causal subgraph extraction model for subgraph extraction processing to obtain the causal subgraph; wherein the causal subgraph extraction model is obtained by iterative training based on a causal learning loss function, a classification loss function and a contrast loss function, the causal learning loss function is used to eliminate the pseudo-correlation subgraph in the graph data to be processed, and the classification loss function and the contrast loss function are used to eliminate the noise subgraph in the graph data to be processed.
[0131] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.
[0132] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.
[0133] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A causal subgraph extraction method based on information bottleneck and causal learning, characterized in that: include: Acquire graph data to be processed, wherein the graph data to be processed includes a causal subgraph, and the graph data to be processed also includes at least one of a noise subgraph and a pseudo-correlation subgraph, and the causal subgraph is probabilistically associated with the pseudo-correlation subgraph; Inputting the graph data to be processed into a causal subgraph extraction model to perform subgraph extraction processing to obtain the causal subgraph; The causal subgraph extraction model is obtained by iterative training based on a causal learning loss function, a classification loss function and a contrast loss function. The causal learning loss function is used to eliminate pseudo-correlated subgraphs in the graph data to be processed. The classification loss function and the contrast loss function are used to eliminate noise subgraphs in the graph data to be processed. The training method of the causal subgraph extraction model includes: Constructing a sample graph data set and an initial causal subgraph extraction model, wherein the sample graph data set includes at least one batch of sample graphs; Performing feature extraction processing on the sample graph to obtain a whole-graph representation, a causal subgraph representation, and a pseudo-correlation subgraph representation of the sample graph, wherein the causal subgraph representation is a feature representation of the causal subgraph of the sample graph, and the pseudo-correlation subgraph representation is a feature representation of the pseudo-correlation subgraph of the sample graph; For the sample graphs of the same batch, the causal subgraph representation is randomly shuffled to obtain a shuffled causal subgraph representation; the shuffled causal subgraph representation is feature-concatenated with the pseudo-correlation subgraph representation to obtain an intervention sample representation; Inputting the causal subgraph representation, the pseudo-correlation subgraph representation, and the intervention sample representation into the initial causal subgraph extraction model for prediction processing, respectively, to obtain a first predicted label value, and determining the causal learning loss function based on the first predicted label value and the true label value; Determining a sample pair based on the whole graph representation and the causal subgraph representation, and determining the contrast loss function based on the sample pair; Inputting the whole graph representation into the initial causal subgraph extraction model for prediction processing to obtain a second predicted label value, and determining the classification loss function based on the second predicted label value and the true label value; Based on the causal learning loss function, the contrast loss function and the classification loss function, the parameters of the initial causal subgraph extraction model are iteratively updated to obtain the causal subgraph extraction model.
2. The causal subgraph extraction method based on information bottleneck and causal learning according to claim 1 is characterized in that: The sample pairs include positive sample pairs and negative sample pairs, the sample pairs are determined based on the whole graph representation and the causal subgraph representation, and the contrast loss function is determined based on the sample pairs, including: Inputting the causal subgraph representation and the whole graph representation into a mapper for feature mapping processing to obtain the causal subgraph representation and the whole graph representation in the same feature space; For the causal subgraph representation and the whole graph representation of the same feature space, the causal subgraph representation and the whole graph representation respectively corresponding to the same sample graph are combined into the positive sample pair; Combining the causal subgraph representations corresponding to different sample graphs with the whole graph representations into the negative sample pairs; The contrast loss function is obtained by minimizing the similarity score between the positive sample pairs and maximizing the similarity score between the negative sample pairs as optimization goals.
3. The causal subgraph extraction method based on information bottleneck and causal learning according to claim 1 is characterized in that: The iterative updating of the parameters of the initial causal subgraph extraction model based on the causal learning loss function, the contrast loss function and the classification loss function to obtain the causal subgraph extraction model includes: Fixing other model parameters in the initial causal subgraph extraction model except the parameters of the mapper, and iteratively updating the parameters of the mapper with the goal of maximizing the contrast loss function; The parameters of the iteratively updated mapper are fixed, and with the goal of minimizing the causal learning loss function and the classification loss function, the parameters of the initial causal subgraph extraction model are iteratively updated through back propagation and gradient descent algorithms to obtain the causal subgraph extraction model.
4. The causal subgraph extraction method based on information bottleneck and causal learning according to claim 1 is characterized in that: The performing feature extraction processing on the sample graph to obtain the whole graph representation, causal subgraph representation and pseudo-correlation subgraph representation of the sample graph includes: Inputting the sample graph into a graph neural network for feature encoding processing to obtain a node representation corresponding to each node of the sample graph; Performing pooling processing on the node representation to obtain a whole-graph representation of the sample graph; Performing feature concatenation processing on the node representation to obtain the representation of the edge corresponding to the node; Inputting the representation of the edge into the feature discriminator for weight determination to obtain the importance weight of each edge, and taking the edge whose importance weight is greater than or equal to the weight threshold as the causal subgraph, and taking the edge whose importance weight is less than the weight threshold as the pseudo-correlation subgraph; Feature extraction is performed on the causal subgraph and the pseudo-correlation subgraph respectively to obtain the causal subgraph representation and the pseudo-correlation subgraph representation.
5. The causal subgraph extraction method based on information bottleneck and causal learning according to claim 4 is characterized in that: The performing feature extraction processing on the causal subgraph and the pseudo-correlation subgraph respectively to obtain the causal subgraph representation and the pseudo-correlation subgraph representation comprises: Inputting the causal subgraph and the pseudo-correlation subgraph into the graph neural network for feature encoding processing respectively, to obtain causal node representations corresponding to each node of the causal subgraph, and pseudo-correlation node representations corresponding to each node of the pseudo-correlation subgraph; The causal node representation and the pseudo-correlation node representation are respectively subjected to pooling processing to obtain the causal subgraph representation and the pseudo-correlation subgraph representation.
6. A causal subgraph extraction device based on information bottleneck and causal learning, characterized in that: include: An acquisition module, used for acquiring graph data to be processed, wherein the graph data to be processed includes a causal subgraph, and the graph data to be processed also includes at least one of a noise subgraph and a pseudo-correlation subgraph; A processing module, used for inputting the graph data to be processed into a causal subgraph extraction model to perform subgraph extraction processing to obtain the causal subgraph; The causal subgraph extraction model is obtained by iterative training based on a causal learning loss function, a classification loss function and a contrast loss function. The causal learning loss function is used to eliminate pseudo-correlated subgraphs in the graph data to be processed. The classification loss function and the contrast loss function are used to eliminate noise subgraphs in the graph data to be processed. The training method of the causal subgraph extraction model includes: Constructing a sample graph data set and an initial causal subgraph extraction model, wherein the sample graph data set includes at least one batch of sample graphs; Performing feature extraction processing on the sample graph to obtain a whole-graph representation, a causal subgraph representation, and a pseudo-correlation subgraph representation of the sample graph, wherein the causal subgraph representation is a feature representation of the causal subgraph of the sample graph, and the pseudo-correlation subgraph representation is a feature representation of the pseudo-correlation subgraph of the sample graph; For the sample graphs of the same batch, the causal subgraph representation is randomly shuffled to obtain a shuffled causal subgraph representation; the shuffled causal subgraph representation is feature-concatenated with the pseudo-correlation subgraph representation to obtain an intervention sample representation; Inputting the causal subgraph representation, the pseudo-correlation subgraph representation, and the intervention sample representation into the initial causal subgraph extraction model for prediction processing, respectively, to obtain a first predicted label value, and determining the causal learning loss function based on the first predicted label value and the true label value; Determining a sample pair based on the whole graph representation and the causal subgraph representation, and determining the contrast loss function based on the sample pair; Inputting the whole graph representation into the initial causal subgraph extraction model for prediction processing to obtain a second predicted label value, and determining the classification loss function based on the second predicted label value and the true label value; Based on the causal learning loss function, the contrast loss function and the classification loss function, the parameters of the initial causal subgraph extraction model are iteratively updated to obtain the causal subgraph extraction model.
7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the causal subgraph extraction method based on information bottleneck and causal learning as described in any one of claims 1 to 5 is implemented.
8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the causal subgraph extraction method based on information bottleneck and causal learning as described in any one of claims 1 to 5 is implemented.
