Global interpretation of graph learning explainability based on cumulative denoising and clustered search

Abstract

To provide: a method for global interpretation of graph learning explainability based on cumulative denoising and clustered search; a storage medium; and an electronic device.SOLUTION: A method includes: receiving a dataset including a set of graphs; applying a graph explainer model on the dataset to determine explain scores; filtering graph elements of each graph based on the explain scores to determine a first set of graph elements; applying the graph explainer model to another dataset of randomized graphs to determine noise scores; determining denoised explain scores; filtering graph elements of each graph based on the denoised explain scores to determine a second set of graph elements; determining feature vectors for the first set of graph elements and the second set of graph elements; determining prediction driving graph elements based on the feature vectors; and rendering explainability information associated with the set of graphs based on the prediction driving graph elements.SELECTED DRAWING: Figure 13

Description

[Technical Field]

[0001] The embodiments described in this disclosure relate to a global interpretation of graph learning explainability based on cumulative denoising and clustering search. [Background technology]

[0002] Advances in the field of explainable graph machine learning have led to the development of explainable graph machine learning models that can generate explanations for predictions made by graph neural networks on input graph data. Explanations can ensure that generated predictions are transparent, trustworthy, ethical, fair, and human-interpretable. Explanations can also allow end users to understand or infer the reasoning behind the predictions. In some scenarios, explanations can allow expert users to determine whether the generated predictions are affected by induced bias, overfitting, underfitting, or data drift in the neural network-based graph machine learning model and whether the generated predictions should be accepted or rejected. Explanations for predictions can be crucial, especially for predictions related to application areas such as financial predictions, transaction fraud detection, entertainment, and product recommendations.

[0003] Typically, opaque models (such as neural network-based explainable graph machine learning models) can generate explanations for generated predictions as a post-processing step. Explanations may be generated after the predictions are generated. Such post-processing (i.e., generating explanations after predictions) can result in explanations that may be inconclusive. This may be due to inaccurate or partial detection of elements (e.g., nodes, edges, or graph structure) in the input graph data that may affect the output prediction. Furthermore, neural network-based explainable graph machine learning models can generate explanations locally, i.e., explanations may be generated for each prediction within a silo. Localized interpretations of explanations may be subject to confirmation bias, thereby affecting the transparency of the predictions. Furthermore, explanations for one prediction may differ significantly from explanations for other predictions, even if the predictions are generated based on the same input graph data.

[0004] The subject matter claimed in this disclosure is not limited to embodiments that solve any drawbacks or that operate only in environments such as those described above. Rather, this background is provided only to illustrate one example technology area where some embodiments described in this disclosure may be practiced. Summary of the Invention [Means for solving the problem]

[0005] According to an aspect of an embodiment, a method may include a set of operations that may include receiving a first dataset including a first set of graphs. The set of operations may further include applying a graph explainer model to the received first dataset to determine a set of explanation scores. The set of operations may further include filtering graph elements of each graph in the first set of graphs to determine a first set of graph elements based on the determined set of explanation scores. The set of operations may further include generating a second dataset including a set of randomized graphs based on the first set of graphs. The set of operations may further include applying a graph explainer model to the generated second dataset to determine a set of noise scores. The set of operations may further include denoising the determined set of explanation scores based on the determined set of noise scores to determine a set of denoised explanation scores. The set of operations may further include filtering graph elements of each graph in the first set of graphs based on the determined set of denoised explanation scores to determine a second set of graph elements. The set of operations may further include determining a first set of feature vectors for the determined first set of graph elements. The set of operations may further include determining a second set of feature vectors for the determined second set of graph elements. The set of operations may further include determining a set of predicted-driven graph elements associated with the first set of graphs based on the first set of feature vectors and the second set of feature vectors. The set of operations may further include rendering graph data including explainability information associated with the first set of graphs based on the determined set of predicted-driven graph elements.

[0006] The object and advantages of the embodiments will be realized and achieved at least by the elements, features, and combinations particularly pointed out in the claims.

[0007] Both the foregoing general description and the following detailed description are provided by way of example and explanation, and are not restrictive of the invention as claimed. [Brief explanation of the drawings]

[0008] Example embodiments will be described and explained with additional specificity and detail through the use of the accompanying drawings.

[0009] [Figure 1] FIG. 1 depicts an example network environment related to global interpretation of explainability in graph learning based on cumulative denoising and clustering search.

[0010] [Figure 2] FIG. 1 is a block diagram illustrating an exemplary electronic device for global interpretation of explainability in graph learning based on cumulative denoising and clustering search.

[0011] [Figure 3] FIG. 1 illustrates an exemplary execution pipeline for global interpretation of explainability in graph learning based on cumulative denoising and clustering search.

[0012] [Figure 4] FIG. 1 illustrates a flowchart of an exemplary method for determining a set of prediction-driven graph elements in a graph dataset based on explanation scores.

[0013] [Figure 5A] FIG. 1 is the first half of a diagram illustrating a flowchart of an exemplary method for determining graph elements in a graph dataset that may not be driving a prediction. [Figure 5B] FIG. 11 is the second half of a diagram illustrating a flowchart of an exemplary method for determining graph elements within a graph dataset that may not be driving a prediction.

[0014] [Figure 6]FIG. 10 illustrates a flowchart of an exemplary method for determining a denoised explanation score based on a set of noise scores.

[0015] [Figure 7] FIG. 1 illustrates a flowchart of an exemplary method for determining a set of prediction-driven graph elements based on feature vectors of the graph elements.

[0016] [Figure 8] FIG. 1 illustrates a flowchart of an exemplary method for rendering explainability information associated with a graph dataset.

[0017] [Figure 9] FIG. 1 illustrates a flowchart of an exemplary method for determination of a global explainability metric for a global interpretation of explainability information associated with predictions on a graph dataset.

[0018] [Figure 10A] FIG. 16, together with B, illustrates an example scenario for rendering explainability information associated with a set of graphs. [Figure 10B] FIG. 10, along with A, illustrates an example scenario for rendering explainability information associated with a set of graphs.

[0019] [Figure 11A] 1 is a diagram illustrating the first half of a flowchart of an exemplary method for global interpretation and visualization of graph explainability results. [Figure 11B] 1 is the second half of a diagram illustrating a flowchart of an exemplary method for global interpretation and visualization of graph explainability results.

[0020] [Figure 12] FIG. 1 illustrates an exemplary scenario for rendering explainability information associated with graph elements of a graph dataset.

[0021] [Figure 13] FIG. 1 illustrates a flowchart of an exemplary method for global interpretation of graph learning explainability based on cumulative denoising and clustering search.

[0022] All according to at least one embodiment described in this disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0023] Some embodiments described herein relate to methods and systems for global interpretation of graph learning explainability based on cumulative denoising and clustering search. The global interpretation of graph learning explainability herein may involve receiving a first dataset including a first set of graphs. A graph explainer model may be applied to the received first dataset to determine a set of explanation scores. Further, graph elements of each graph in the first set of graphs may be filtered based on the determined set of explanation scores to determine a first set of graph elements. A second dataset including a randomized set of graphs may then be generated based on the first set of graphs. Further, the graph explainer model may be applied to the generated second dataset to determine a set of noise scores. Further, the determined set of explanation scores may be denoised based on the determined set of noise scores to determine a denoised set of explanation scores. Further, graph elements of each graph in the first set of graphs may be filtered based on the determined set of denoised explanation scores to determine a second set of graph elements. Further, a first set of feature vectors may be determined for the determined first set of graph elements. Further, a second set of feature vectors may be determined for the determined second set of graph elements. Further, a set of predicted-driven graph elements associated with the first set of graphs may be determined based on the first set of feature vectors and the second set of feature vectors. Finally, graph data including explainability information associated with the first set of graphs may be rendered based on the determined set of predicted-driven graph elements.

[0024] The explainable graph machine learning workflow may include generating a predictive model based on training graph data. The predictive model may be trained by a graph neural network based on the training graph data. The explainable graph machine learning workflow may further include generating a prediction by the predictive model based on application of the predictive model to inferred graph data and generating an explanation for the prediction by the explainable graph machine learning model. The generated explanation may indicate graph elements (e.g., nodes, edges, node features, edge features, graph properties, substructures, motifs, etc.) that may be identified as prediction drivers. The explanation may be generated by an explainer model that can obtain information from the predictive model or modify the inferred graph data or the predictive model (to generate the explanation). The explanation can ensure that the prediction is transparent, reliable, accurate, unbiased, and human-interpretable. The explanation may further allow a user to understand or infer the reasoning behind the prediction.

[0025] Typically, explanations for predictions can be generated as a posteriori step, i.e., after the predictions are generated. Furthermore, traditionally, explanations can be generated locally, i.e., an explanation can be generated for each prediction. Even though explanations can facilitate building a user's trust and confidence in the generated predictions, the explanations can be subject to confirmation bias due to localized interpretations of the explanations. Furthermore, generating explanations after the fact for predictions can result in a large proportion of inconclusive explanations. For example, in some cases, graph elements (such as nodes or edges) may be incorrectly identified as driving or influencing the predictions. However, in other cases, the graph elements that drive the predictions may be partially identified. In such cases, a subset of graph elements from the set of graph elements that drive the predictions may be identified, and other prediction-driving graph elements from the set of graph elements may not be detected (missed).

[0026] According to one or more embodiments of the present disclosure, the technical field of explainable graph machine learning may be improved by configuring a computing system (e.g., an electronic device) to process information (e.g., explanation scores) generated by an explainable graph machine learning model and detect noise in the resulting explanation scores. The processing may include global statistical analysis of cumulative explanation scores, which may be determined based on multiple applications of a predictive or explanation model to input graph data (which may include a set of input graphs, such as a production graph dataset). The cumulative explanation score may represent the contribution of graph elements in each input graph to predictions regarding different input graph data characteristics. Determining the cumulative explanation score may enable the detection and isolation of noise in the explanation scores. Through global statistical analysis, the computing system may identify graph elements (i.e., nodes, edges, or structures of the graph data) in each input graph of the set of input graphs that may be the most relevant drivers of the predictions.

[0027] The computing system may create a randomized graph dataset based on modifying features of graph elements of graphs in the production graph dataset. For the modification, graph elements in the production graph dataset may be identified based on automated machine learning. The automated machine learning may improve the efficiency of identifying features and graph elements that may introduce noise into the explanation score. The identified graph elements may not drive predictions and thus may be modified for creation of the randomized graph dataset. Creation of the randomized graph dataset may allow for the artificial creation of explanation noise. The computing system may further use the identified graph elements to determine baseline noise that may affect the cumulative explanation score. The baseline noise may be removed from the cumulative explanation score to determine a denoised explanation score.

[0028] The computing system may perform a clustering search based on the cumulative explanation score and the denoised explanation score to facilitate global interpretation of the explanation results for the prediction. The clustering search may include determining similarities between features of graph elements (within the production graph dataset), which may be determined as prediction driving factors based on the cumulative explanation score and the denoised explanation score. The clustering search may further include clustering the graph elements determined as prediction driving factors into clusters based on the features of the graph elements. The clustering search may allow determination of actual prediction-driving graph elements in the production graph dataset based on the features of the graph elements. The clustering search may further resolve conflicts between explanation results obtained based on the cumulative explanation score and the denoised explanation score and may also improve the quality of explanations generated by the explainable graph machine learning model. The computing system may further facilitate accurate visualization of explainability information corresponding to actual graph elements determined based on the clustering search.

[0029] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0030] 1 is a diagram depicting an example network environment for global interpretation of graph learning explainability based on cumulative denoising and clustering search, according to at least one embodiment described in this disclosure. Referring to FIG. 1, a network environment 100 is shown. The network environment 100 may include an electronic device 102, a user end device 104, and a server 106 (which may host a database 108). The electronic device 102 may include a graph explainer model 110 and a first dataset 114 (which may include a first set of graphs 114A...114N). The electronic device 102, the user end device 104, and the server 106 may be communicatively coupled to each other via a communication network (such as communication network 112).

[0031] The electronic device 102 may include appropriate logic, circuitry, interfaces, and / or code that may be configured to receive a first dataset 114 including a first set of graphs 114A...114N. The electronic device 102 may further be configured to generate a second dataset 116 including a set of randomized graphs based on the first set of graphs 114A...114N. The electronic device 102 may apply the graph explainer model 110 to the received first dataset 114 to determine a set of explanation scores and may apply the graph explainer model 110 to the second dataset 116 to determine a set of noise scores. The electronic device 102 may further filter graph elements of each graph in the first set of graphs 114A...114N based on the determined set of explanation scores to determine a first set of graph elements. The electronic device 102 may also denoise the determined set of explanation scores based on the determined set of noise scores to determine a set of denoised explanation scores. Further, the electronic device 102 may filter graph elements of each graph in the first set of graphs 114A...114N based on the determined set of denoised explanation scores to determine a second set of graph elements. The electronic device 102 may then determine a first set of feature vectors for the first set of graph elements and a second set of feature vectors for the second set of graph elements. The electronic device 102 may be further configured to determine a set of predicted-driven graph elements associated with each graph in the first set of graphs 114A...114N based on the first set of feature vectors and the second set of feature vectors. The electronic device 102 may be further configured to render graph data, which may include explainability information associated with the first set of graphs 114A...114N, based on the determined set of predicted-driven graph elements.Examples of electronic device 102 may include, but are not limited to, a computing device, a smartphone, a mainframe machine, a server, a computer workstation, a consumer electronics (CE) device, and / or any device with graph processing capabilities (such as a device with a set of graphics processing units (GPUs)).

[0032] The user end device 104 may include appropriate logic, circuitry, interfaces, and / or code that may be configured to receive control instructions from the electronic device 102. The user end device 104 may be configured to render an electronic user interface (UI) including one or more UI elements corresponding to graph data, including explainability information associated with a plurality of graphs, based on the received control instructions. The user end device 104 may further be configured to receive, via the electronic UI, user input indicating a user selection of a first set of graphs (e.g., first set of graphs 114A...114N) from the plurality of graphs. In FIG. 1 , the electronic device 102 and the user end device 104 are shown as two separate devices by way of example only. In some embodiments, the entire functionality of the user end device 104 may be incorporated into the electronic device 102 without departing from the scope of the present disclosure. Examples of user end devices 104 may include, but are not limited to, computing devices, tablets, smartphones, smart wearable devices, game consoles, virtual reality (VR) devices, augmented reality (AR) devices, mixed reality (MR) devices, mainframe machines, computer workstations, Internet of Things (IoT) devices, and / or any consumer electronics (CE) device.

[0033] The server 106 may include appropriate logic, circuitry, and interfaces, and / or code that may be configured to receive a request for the first dataset 114 (i.e., the first set of graphs 114A...114N) from the electronic device 102. The server 106 may be further configured to retrieve the first dataset 114 from the database 108 and transmit the first dataset 114 to the electronic device 102. In at least one embodiment, the server 106 may receive the first dataset 114 from the electronic device 102, generate a second dataset 116 (i.e., a set of randomized graphs) based on the received first dataset, and transmit the second dataset 116 to the electronic device 102. In some embodiments, the server 106 applies the graph explainer model 110 to the received first dataset 114 to determine a set of explanation scores and transmit the set of explanation scores to the electronic device 102. The server 106 may be configured to apply the graph explainer model 110 to the second dataset 116 to determine a set of noise scores and transmit the set of explanation scores to the electronic device 102. The server 106 may be implemented as a cloud server and may perform operations through web applications, cloud applications, Hypertext Transfer Protocol (HTTP) requests, repository operations, file transfers, etc. Other example implementations of the server 106 may include, but are not limited to, a database server, a file server, a web server, a media server, an application server, a mainframe server, a cloud computing server, and / or any device with graph processing capabilities (such as a device with a set of graphics processing units (GPUs)).

[0034] In at least one embodiment, the server 106 may be implemented as multiple distributed cloud-based resources through the use of several techniques that may be known to those skilled in the art. Those skilled in the art will understand that the scope of the present disclosure may not be limited to the implementation of the server 106 and the electronic device 102 as two separate entities. In certain embodiments, the functionality of the server 106 may be incorporated in whole or at least in part into the electronic device 102 without departing from the scope of the present disclosure.

[0035] The database 108 may include appropriate logic, circuitry, interfaces, and / or code that may be configured to store a first dataset 114 (i.e., a first set of graphs 114A...114N) representing various domains (e.g., a social network domain, a financial domain dataset, a molecular chemistry domain dataset, an e-commerce domain dataset, or a genome sequence domain dataset). The database 108 may also store a second dataset 116. In some embodiments, the database 108 may be further configured to store a graph explainer model 110. The database 108 may be derived from data from a relational or non-relational database, or a set of comma-separated value (csv) files in traditional storage or big data storage. In certain scenarios, the database 108 may correspond to a file system that may be mounted on a storage device. For example, the first dataset 114, the second dataset 116, and / or the graph explainer model 110 may be stored in a file system mounted on a storage device that may host the database 108. In one embodiment, the database 108 may be stored or cached on a device such as the server 106 or the electronic device 102. The device storing the database 108 may be configured to receive a query for the first data set 114. In response, the device storing the database 108 may be configured to retrieve and transmit the first data set 114 to the electronic device 102. According to one embodiment, the database 108 may be hosted on multiple servers stored in the same or different locations. The operations of the database 108 may be performed using hardware, including a processor, a microprocessor (e.g., to perform or control the execution of one or more operations), a field programmable gate array (FPGA), or an application-specific integrated circuit (ASIC). In some other cases, the database 108 may be implemented using software.

[0036] The graph explainer model 110 may include appropriate logic, circuitry, interfaces, and / or code that may be configured to analyze predictions regarding input graph data (e.g., the first dataset 114, the second dataset 116, or a combination of graph elements in the first dataset 114). The predictions regarding the input graph data may be analyzed to generate output results (e.g., explanation scores) for each graph element in each graph of the first set of graphs 114A...114N or the set of randomized graphs. The explain score for a graph element may indicate the graph element's contribution to predictions that may be generated by the graph neural network based on the input graph data. The graph explainer model 110 may be a trained graph neural network model that may recognize different types of nodes in the input graph data or edges between each pair of nodes. The edges may correspond to different connections or relationships between each node in the input graph data. Based on the recognized nodes and edges, the graph explainer model 110 may generate explanations that can be used to classify or predict the input graph data into different labels or classes (e.g., using a node classification task) and to understand, explain, or provide the reason(s) for the classification (i.e., prediction). In one example, a particular node in the input graph data may have a set of features associated with it. Furthermore, each edge may connect to different nodes that have a similar set of features. The electronic device 102 may be configured to encode the set of features to generate a feature vector using the graph explainer model 110. After encoding, information may be passed between the particular node and neighboring nodes connected through the edges. Based on the information passed to the neighboring nodes, a final vector may be generated for each node. Such a final vector may include information associated with the set of features for the particular node as well as the neighboring nodes, thereby providing reliable and accurate information associated with the particular node.As a result, the graph explainer model 110 can analyze information represented as input graph data and provide the reasons behind certain predicted outcomes regarding the input graph data. In one embodiment, the graph explainer model 110 may be stored on the electronic device 102, the server 106, or the database 108. The graph explainer model 110 may be implemented using hardware including a processor, a microprocessor (e.g., for performing or controlling the execution of one or more operations), a field programmable gate array (FPGA), or an application specific integrated circuit (ASIC). In some other cases, the graph explainer model 110 may be a set of code, program, or software instructions, or may be implemented using a combination of hardware and software.

[0037] In some embodiments, the graph explainer model 110 may correspond to a machine learning model (e.g., a neural network model) having multiple classification layers for classification of different nodes in the input graph data, where each successive layer may use the output of the previous layer as input. Each classification layer may be associated with multiple edges, each of which may be further associated with multiple weights. During training, the graph explainer model 110 may be configured to filter or remove edges or nodes based on the input graph data and further provide an output result (i.e., a graph representation). Examples of the graph explainer model 110 may include, but are not limited to, a graph convolutional network (GCN), a graph spatiotemporal network with a GCN, a recurrent neural network (RNN), a deep Bayesian neural network, and / or a combination of such networks.

[0038] The communication network 112 may include a communication medium through which the electronic device 102, the server 106, and the database 108 can communicate with each other. The communication network 112 may be one of a wired connection or a wireless connection. Examples of the communication network 112 may include, but are not limited to, the Internet, a cloud network, a cellular or wireless mobile network (such as Long Term Evolution and 5G New Radio), a satellite network (such as a network of a set of low-earth orbit satellites), a Wireless Fidelity (Wi-Fi) network, a personal area network (PAN), a local area network (LAN), or a metropolitan area network (MAN). The various devices in the network environment 100 may be configured to connect to the communication network 112 according to various wired and wireless communication protocols. Examples of wired and wireless communication protocols may include, but are not limited to, at least one of Transmission Control Protocol and Internet Protocol (TCP / IP), User Datagram Protocol (UDP), Hypertext Transfer Protocol (HTTP), File Transfer Protocol (FTP), ZigBee, EDGE, IEEE 802.11, Light Fidelity (Li-Fi), 802.16, IEEE 802.11s, IEEE 802.11g, multi-hop communication, wireless access point (AP), device-to-device communication, cellular communication protocols, and Bluetooth (BT) communication protocols.

[0039] During operation, the electronic device 102 may be configured to receive a first dataset 114, which may include a first set of graphs 114A...114N. Each graph in the first set of graphs 114A...114N in the first dataset 114A may represent a domain dataset (e.g., a genome sequence domain dataset, an e-commerce domain dataset, or a social network domain dataset). In some embodiments, the first dataset 114 may be received from the server 106 or the database 108 (via the server 106). Each graph in the first set of graphs 114A...114N may include graph elements (e.g., node features, edge features, or graph structures). Each node feature of the graph may represent an entity associated with the domain, and each edge between any two nodes in the set of nodes may indicate a relationship between the two entities represented by the two nodes. For example, the first dataset 114 may include gene expression data, and each graph in the first set of graphs 114A...114N may represent a genome sequence domain. The first set of graphs 114A...114N may include gene expression data that may be obtained based on wet lab experiments. The graph elements of each graph in the first set of graphs 114A...114N may represent genes (nodes) and relationships or correlations between genes (edges). Details of receiving the first dataset 114 are further provided, for example, in FIG. 3.

[0040] The electronic device 102 may be further configured to apply the graph explainer model 110 to the received first data set 114 to determine a set of explanation scores. The graph explainer model 110 may be applied to the first data set 114 to determine graph elements of each graph of the first set of graphs 114A...114N that may be driving predictions generated by the graph neural network. The electronic device 102 may calculate explanation scores associated with the graph elements of each graph based on application of the graph explainer model 110 to the generated predictions to determine the prediction-driving graph elements.

[0041] According to one embodiment, the electronic device 102 may include an explainable graph artificial intelligence (GXAI) module. The GXAI module may include a graph explainer model 110 and a graph neural network (i.e., a predictive model). The predictive model may be trained to perform node-level, edge-level, and graph-level prediction tasks. The predictive model may generate predictions for the graph, and the graph explainer model 110 may determine the contribution of each graph element of the graph to the prediction. The predictive model may be applied multiple times to each graph (i.e., input graph data) in the first set of graphs 114A...114N to generate a subset of predictions in the set of predictions. For example, the predictive model may be applied multiple times to graph 114A to generate a first subset of predictions in the set of predictions. At each instance of application of the predictive model to graph 114A, a prediction in the first subset of predictions may be generated for graph 114A. Further, the graph explainer model 110 may be applied to each prediction of the first subset of predictions. Based on these applications, a first subset of explanation scores of the set of explanation scores for the graph 114A may be determined. The application of each prediction of the first subset of predictions may result in the determination of a subset of initial explanation scores. Thus, based on the application of the graph explainer model 110 to the first subset of predictions, multiple subsets of initial explanation scores may be determined. The first subset of explanation scores may be determined based on an element-by-element accumulation of the initial explanation scores in each set of initial explanation scores.

[0042] A first subset of explanatory scores may be associated with graph elements of graph 114A. Each explanatory score in the first subset of explanatory scores may indicate a contribution of the graph element of graph 114A to the first subset of predictions for graph 114A. A higher explanatory score associated with a graph element of graph 114A may indicate a greater contribution of the graph element to the first subset of predictions for graph 114A. However, a lower explanatory score associated with a graph element of graph 114A may indicate a lower contribution of the graph element to the first subset of predictions for graph 114A.

[0043] Similarly, other subsets of the set of explanation scores associated with graph elements of other graphs in the first set of graphs 114A...114N may be determined. Such subsets of explanation scores may indicate the contribution of graph elements of other graphs to other subsets of predictions. Details of the application of the graph explainer model 110 to determine sets of explanation scores are provided further in, for example, Figures 3 and 4.

[0044] The electronic device 102 may be further configured to filter graph elements of each graph in the first set of graphs 114A...114N based on the determined set of explanation scores to determine a first set of graph elements. The electronic device 102 may determine the contribution of the graph elements to a prediction for the corresponding graph (based on the explanation scores (of the set of explanation scores) associated with the graph elements of each graph). Based on the contribution, graph elements that do not drive the prediction may be determined and then filtered or removed from the corresponding graph. According to an embodiment, the electronic device 102 may apply the predictive model multiple times to each graph in the first set of graphs 114A...114N to generate multiple predictions for the same input graph data. The contribution of each graph element of each graph in the first set of graphs 114A...114N (e.g., graph 114A) to each prediction for graph 114A generated in each instance of application of the predictive model to graph 114A may be different.

[0045] For example, the electronic device 102 may apply the predictive model to the graph 114A four times to generate four predictions. The contribution of each graph element to each of the four predictions may be different. The contribution may be determined based on application of the graph explainer model 110 to each of the four predictions. Thus, multiple explanation scores (e.g., four explanation scores) may be obtained that indicate the contribution of each graph element to the multiple predictions (e.g., four predictions) for each graph in the first set of graphs 114A...114N. The multiple explanation scores associated with each graph element in each graph in the first set of graphs 114A...114N may be normalized and accumulated to determine a final explanation score for each graph element. Based on the final explanation score, the electronic device 102 may then filter the corresponding graph element from the corresponding graph. The unfiltered graph elements of the corresponding graph (or the graph elements remaining after filtering) may be determined as a subset of the graph elements of the first set of graph elements (for the corresponding graph). Details of the filtering of the first set of graphs 114A...114N are provided further in, for example, FIGS.

[0046] The electronic device 102 may be further configured to generate a second data set 116 including a set of randomized graphs based on the first set of graphs 114A...114N. Each randomized graph in the set of randomized graphs may correspond to a graph in the first set of graphs 114A...114N and may be obtained based on modifying or randomizing characteristics of the graph elements belonging to the graph. According to an embodiment, the electronic device 102 may identify a set of combinations of graph elements belonging to each graph in the first set of graphs 114A...114N. The identification may be based on principal component analysis and similarity analysis of each of the graph elements. The graph elements in each identified combination in the set of combinations may be obtained from the set of graph elements belonging to the corresponding graph. The electronic device 102 may use automated machine learning to select the set of graph elements belonging to each graph in the first set of graphs 114A...114N. Thus, the graph elements in each identified combination in the set of combinations may be obtained from the selected set of graph elements.

[0047] For example, the electronic device 102 may identify five combinations of graph elements belonging to graph 114A. The graph elements in each of the five combinations may be identified from a set of graph elements that may be selected from graph 114A based on automated machine learning (or AutoML). For each combination of the five combinations, the electronic device 102 may be configured to determine a randomized graph from the set of randomized graphs. The electronic device 102 may select a first combination of graph elements belonging to graph 114A and randomize characteristics of each graph element of the first combination to generate a randomized graph corresponding to graph 114A. Similarly, four additional randomized graphs corresponding to graph 114A may be generated. Furthermore, randomized graphs corresponding to each of the other graphs in the first set of graphs 114A...114N may be generated. Details of the generation of the set of randomized graphs are further provided, for example, in FIGS. 3, 5A, and 5B.

[0048] The electronic device 102 may be further configured to apply the graph explainer model 110 to the generated second data set 116 to determine a set of noise scores. The electronic device 102 may apply a predictive model to each randomized graph (of the second data set 116) corresponding to each graph of the first set of graphs 114A...114N. Based on the application of the predictive model, a prediction may be generated for each randomized graph. The electronic device 102 may further apply the graph explainer model 110 to the prediction to determine an explanatory score associated with the graph element of the corresponding randomized graph. Thus, an explanatory score for each randomized graph of the second data set 116 may be determined. The electronic device 102 may then determine randomized graphs having graph elements associated with explanatory scores having an even distribution. The electronic device 102 may determine at least one randomized graph corresponding to each graph of the first set of graphs 114A...114N, which may include graph elements associated with explanatory scores having an even distribution.

[0049] According to an embodiment, the electronic device 102 can determine a combination of graph elements with randomized features for generating a randomized graph (including graph elements associated with explanatory scores having a uniform distribution) corresponding to a graph in the first set of graphs 114A...114N. If multiple randomized graphs (including graph elements associated with explanatory scores having a uniform distribution) corresponding to a particular graph in the first set of graphs 114A...114N (e.g., graph 114A) are generated, a randomized graph can be selected from the multiple randomized graphs. The combination of graph elements with modified features for generating the selected randomized graph can be the longest among the combinations of graph elements with modified features for generating other determined randomized graphs corresponding to graph 114A. Thus, a combination of graph elements can be determined for each graph in the first set of graphs 114A...114N, and can be used to generate a subset of noise scores in the set of noise scores.

[0050] According to one embodiment, the electronic device 102 can apply a prediction model to the determined combination of graph elements for each graph in the first set of graphs 114A...114N. For example, the prediction model may be applied to the determined combination of graph elements for graph 114A to generate a prediction. The graph explainer model 110 may then be applied to the prediction to determine an explanation score. The determined explanation score may correspond to a first subset of noise scores from the set of noise scores for graph 114A. Similarly, other subsets of noise scores for other graphs in the first set of graphs 114A...114N may be determined for determining the set of noise scores. Each subset of noise scores for a graph in the first set of graphs 114A...114N may represent baseline explanatory noise that may need to be eliminated to obtain an accurate explanation score that indicates the actual contribution of that graph's graph elements to the prediction for that graph. Details of determining the set of noise scores are further provided, for example, in FIGS. 3, 5A, 5B, and 6.

[0051] The electronic device 102 may be configured to denoise the determined set of explanation scores based on the determined set of noise scores to determine a set of denoised explanation scores. According to an embodiment, the electronic device 102 may denoise a subset of the explanation scores of the set of explanation scores for each graph of the first set of graphs 114A...114N based on a subset of the noise scores of the set of noise scores for the corresponding graph. For example, a first subset of explanation scores (i.e., explanation scores associated with graph elements of graph 114A) for graph 114A may be denoised based on the first subset of noise scores for graph 114A. The electronic device 102 may calculate a subtraction, per graph element, of the noise scores of the first subset of noise scores from the explanation scores of the first subset of explanation scores. Based on the subtraction, a first subset of denoised explanation scores (of the set of denoised explanation scores) for graph 114A may be determined. Each denoised explanation score of the first subset of denoised explanation scores may be associated with a respective graph element of graph 114A. Similarly, other subsets of denoised explanation scores (of the set of denoised explanation scores) for other graphs of the first set of graphs 114A...114N may be determined. Details of denoising sets of explanation scores are further provided in, for example, Figures 3 and 6.

[0052] The electronic device 102 may be further configured to filter graph elements of each graph of the first set of graphs 114A...114N based on the determined set of denoised explanation scores to determine a second set of graph elements. Each subset of denoised explanation scores of the set of denoised explanation scores determined for each graph of the first set of graphs 114A...114N may indicate the contribution of the graph elements of the corresponding graph to the prediction for the corresponding graph. Based on the contribution, the electronic device 102 may determine graph elements that may not drive the prediction for the corresponding graph and then filter those graph elements. For example, based on the first subset of denoised explanation scores associated with each graph element of graph 114A, graph elements that may not drive the prediction on graph 114A may be determined and filtered. The unfiltered graph elements of graph 114A (or the graph elements remaining after filtering) may be determined as a first subset of graph elements of the second set of graph elements. Similarly, other subsets of graph elements in the second set of graph elements may be determined based on subsets of denoised explanation scores associated with graph elements of other graphs in the first set of graphs 114A...114N. Details of determining the second set of graph elements are provided further in, for example, Figures 3, 5A, and 5B.

[0053] The electronic device 102 may be further configured to determine a first set of feature vectors for the first set of graph elements. Each feature vector in the first set of feature vectors may correspond to a graph element of the first set of graph elements. The first set of feature vectors may correspond to a predicted driving graph element of the first set of graphs 114A...114N determined based on an explanation score that may be associated with each graph element of the first set of graph elements.

[0054] The electronic device 102 may be further configured to determine a second set of feature vectors for the second set of graph elements. Each feature vector in the second set of feature vectors may correspond to a graph element of the second set of graph elements. The second set of feature vectors may correspond to a predicted driving graph element of the first set of graphs 114A...114N determined based on a denoised explanation score that may be associated with each graph element of the second set of graph elements. Details of determining the first set of feature vectors and the second set of feature vectors are further provided in, for example, Figures 3 and 7.

[0055] The electronic device 102 may be further configured to determine a set of prediction-driven graph elements associated with the first set of graphs 114A...114N based on the first set of feature vectors and the second set of feature vectors. According to an embodiment, the electronic device 102 may determine a similarity between each pair of feature vectors belonging to the first set of feature vectors, the second set of feature vectors, or the first set of feature vectors and the second set of feature vectors. The similarity between each pair of feature vectors may be determined. The determination of the similarity between each pair of feature vectors may be based on unsupervised learning. The determined similarity may correspond to a similarity in the distribution of feature values ​​of the feature vectors in each pair of feature vectors. Based on the determined similarity, the electronic device 102 may group the feature vectors of the first set of feature vectors and the feature vectors of the second set of feature vectors into multiple clusters for clustering search. According to an embodiment, the electronic device 102 may determine, as a prediction-driven graph element, a graph element of each graph of the first set of graphs 114A...114N whose feature vectors are grouped into different clusters among the multiple clusters. Details of determining the set of prediction-driven graph elements are provided further in, for example, FIGS.

[0056] The electronic device 102 may be configured to render graph data that may include explainability information associated with the first set of graphs 114A...114N based on the determined set of prediction-driven elements. According to an embodiment, the rendered explainability information may include the determined set of prediction-driven graph elements, an explanation score associated with each prediction-driven graph element of the set of prediction-driven graph elements, or graphs of the first set of graphs 114A...114N annotated with the graph elements of the graphs determined as prediction-driven graph elements. In some embodiments, the electronic device 102 may control the user end device 104 to render the graph data that may include the explainability information. Details of rendering the explainability information associated with the first set of graphs 114A...114N are further provided, for example, in FIGS. 3, 8, 9, 10A, 10B, 11A, 11B, and 12.

[0057] Modifications, additions, or omissions may be made to Figure 1 without departing from the scope of the present disclosure. For example, network environment 100 may include more or fewer elements than those shown and described in this disclosure. In some embodiments, the functionality of each of user end device 104, server 106, and database 108 may be incorporated into electronic device 102 without departing from the scope of the present disclosure.

[0058] FIG. 2 is a block diagram illustrating an exemplary electronic device for global interpretation of graph learning explainability based on cumulative denoising and clustering search, according to at least one embodiment described in this disclosure. FIG. 2 is described in conjunction with elements of FIG. 1. Referring to FIG. 2, a block diagram 200 of a system 202 including an electronic device 102 is shown. The electronic device 102 may include a processor 204, a memory 206, persistent data storage 208, an input / output (I / O) device 210, and a network interface 212. In at least one embodiment, the memory 206 may store the graph explainer model 110. In at least one embodiment, the I / O device 210 may include a display device 210A.

[0059] The processor 204 may include appropriate logic, circuits, and interfaces that may be configured to execute a set of instructions stored in the memory 206. The processor 204 may be configured to execute program instructions associated with different operations performed by the electronic device 102. The processor 204 may be configured to receive a first dataset 114 including a first set of graphs 114A...114N. The processor 204 may be further configured to apply the graph explainer model 110 to the received first dataset 114 to determine a set of explanation scores. The processor 204 may be further configured to filter graph elements of each graph in the first set of graphs 114A...114N based on the determined set of explanation scores to determine a first set of graph elements. The processor 204 may be further configured to generate a second dataset 116 including a set of randomized graphs. The processor 204 may be further configured to apply the graph explainer model 110 to the received second dataset 116 to determine a set of noise scores. The processor 204 may be further configured to denoise the determined set of explanation scores based on the determined set of noise scores to determine a denoised set of explanation scores. The processor 204 may be further configured to filter the graph elements of each graph in the first set of graphs 114A...114N based on the determined set of denoised explanation scores to determine a second set of graph elements. The processor 204 may be further configured to determine a first set of feature vectors for the determined first set of graph elements. The processor 204 may be further configured to determine a second set of feature vectors for the determined second set of graph elements. The processor 204 may be further configured to determine a set of prediction-driven graph elements associated with the first set of graphs 114A...114N based on the first set of feature vectors and the second set of feature vectors.The processor 204 may be further configured to render graph data including explainability information associated with the first set of graphs 114A...114N based on the determined set of prediction drivers. The processor 204 may be implemented based on several processor technologies known in the art. Examples of processor technologies may include, but are not limited to, a central processing unit (CPU), an X86-based processor, a reduced instruction set computing (RISC) processor, an application specific integrated circuit (ASIC) processor, a complex instruction set computing (CISC) processor, a graphical processing unit (GPU), a coprocessor, or a combination thereof.

[0060] 2 as a single processor, processor 204 may include any number of processors configured to individually or collectively perform or direct the performance of any number of operations of electronic device 102 as described in this disclosure. Additionally, one or more of the processors may reside on one or more different electronic devices, such as different servers. In at least one embodiment, processor 204 may be configured to interpret and / or execute program instructions or process data, which may be stored in memory 206 or persistent data storage 208. In some embodiments, processor 204 may be configured to fetch program instructions from persistent data storage 208 and load the program instructions into memory 206. After the program instructions are loaded into memory 206, processor 204 may execute the program instructions.

[0061] The memory 206 may include appropriate logic, circuitry, and interfaces that may be configured to store one or more instructions executed by the processor 204. The one or more instructions stored in the memory 206 may be executed by the processor 204 to perform different operations of the processor 204 (and the electronic device 102). The memory 206 may store the received first dataset 114, the generated second dataset 116, a set of randomized graphs, a first set of graph elements of the first set of graphs 114A...114N, a second set of graph elements of the first set of graphs 114A...114N, a first set of feature vectors, a second set of feature vectors, a set of prediction-driven graph elements, and explainability information associated with the first set of graphs 114A...114N. The memory 206 may further store the graph explainer model 110. Examples of implementations of memory 206 may include, but are not limited to, a CPU cache, a hard disk drive (HDD), a solid state drive (SSD), random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), and / or a secure digital (SD) card.

[0062] The persistent data storage device 208 may include appropriate logic, circuitry, and / or interfaces that may be configured to store program instructions executable by the processor 204. The persistent data storage device 208 may include computer-readable storage media that carry or have computer-executable instructions or data structures stored thereon. Such computer-readable storage media may include any available medium that can be accessed by a general-purpose or special-purpose computer, such as the processor 204. By way of example, and not limitation, such computer-readable storage media may include tangible or non-transitory computer-readable storage media, including compact disc read-only memory (CD-ROM) or other optical disk storage, magnetic disk storage or other magnetic storage devices (e.g., hard disk drives (HDD)), flash memory devices (e.g., solid-state drives (SSD), secure digital (SD) cards, other solid-state memory devices), or any other storage medium that can be used to carry or store specific program code in the form of computer-executable instructions or data structures and that can be accessed by a general-purpose or special-purpose computer. Combinations of the above may also be included within the scope of computer-readable storage media. Computer-executable instructions may include, for example, instructions and data configured to cause processor 204 to perform a certain operation or group of operations associated with electronic device 102 .

[0063] The I / O device 210 may include appropriate logic, circuitry, and interfaces that may be configured to receive input and render output based on the received input. For example, the I / O device 210 may receive an input that may trigger receipt of the first dataset 114. The I / O device 210 may further receive a first user input indicating a user selection of a first set of graphs 114A...114N from the plurality of graphs included in the first dataset 114. Furthermore, the I / O device 210 may render output such as a set of prediction-driving graph elements associated with the first set of graphs 114A...114N, each set of reduced subgraphs (such as the set of reduced subgraph-1 118A), or explainability information associated with the first set of graphs 114A...114N. The I / O device 210, which may include various input and output devices, may be configured to communicate with the processor 204. Examples of I / O devices 210 may include, but are not limited to, a touch screen, a keyboard, a mouse, a joystick, a display device (e.g., display device 210A), a microphone, and a speaker.

[0064] The display device 210A may include appropriate logic, circuitry, and interfaces that may be configured to render graph data that may include explainability information associated with the first set of graphs 114A...114N determined by the electronic device 102 based on the determined set of predictive driving elements. The graph data may be rendered on the display device 210A via an electronic user interface (UI). The display device 210A may be a touchscreen that may allow a user to provide user input via the display device 210A. The touchscreen may be at least one of a resistive touchscreen, a capacitive touchscreen, or a thermal touchscreen. The display device 210A may be implemented through several known technologies, such as, but not limited to, at least one of a liquid crystal display (LCD) display, a light emitting diode (LED) display, a plasma display, or an organic LED (OLED) display technology, or other display devices. According to some embodiments, the display device 210A may refer to a display screen of a head-mounted device (HMD), a smart glasses device, a see-through display, a projection-based display, an electrochromic display, or a transparent display.

[0065] The network interface 212 may include appropriate logic, circuits, and interfaces that may be configured to facilitate communication between the processor 204 (i.e., the electronic device 102) and the server 106 over the communications network 112. The network interface 212 may be implemented using various known technologies to support wired or wireless communication of the electronic device 102 with the communications network 112. The network interface 212 may include, but is not limited to, an antenna, a radio frequency (RF) transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a coder-decoder (codec) chipset, a subscriber identity module (SIM) card, or local buffer circuitry. The network interface 212 may be configured to communicate via wireless communication with networks such as the Internet, an intranet, or a wireless network, e.g., a cellular network, a wireless local area network (LAN), and a metropolitan area network (MAN). The wireless communication may be configured to use one or more of a number of communications standards, protocols, and technologies. For example, Enhanced Data GSM Environment (EDGE), Wideband Code Division Multiple Access (W-CDMA), Long Term Evolution (LTE), Fifth Generation (5G) New Radio (NR), Global System for Mobile Communications (GSM), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth, Wireless Fidelity (Wi-Fi) (such as IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, or IEEE 802.11n), Voice over Internet Protocol (VoIP), Light Fidelity (Li-Fi), Worldwide Interoperability for Microwave Access (Wi-MAX), protocols for email, instant messaging, and short message service (SMS).

[0066] Modifications, additions, or omissions may be made to the example electronic device 102 without departing from the scope of the present disclosure. For example, in some embodiments, the example electronic device 102 may include any number of other components that may not be explicitly shown or described for the sake of brevity.

[0067] FIG. 3 illustrates an exemplary execution pipeline for global interpretation of graph learning explainability based on cumulative denoising and clustering search, according to an embodiment of the present disclosure. FIG. 3 will be described in conjunction with elements of FIGS. 1 and 2. Referring to FIG. 3, an execution pipeline 300 is shown. The exemplary execution pipeline 300 may include a sequence of operations that may be performed by the processor 204 of the electronic device 102 of FIG. 1 for global interpretation of graph learning explainability based on cumulative denoising and clustering search. A series of operations is shown in the execution pipeline 300, which may begin at 302 and end at 318.

[0068] At 302, a first dataset may be received. In at least one embodiment, the processor 204 may be configured to receive the first dataset, which may include a first set of graphs 302A...302N. The first dataset (i.e., the first set of graphs 302A...302N) may be received as input graph data (or production graph data), which may represent a domain dataset. Each graph in the first set of graphs 302A...302N may include graph elements, such as nodes, edges, or graph structures. Furthermore, each graph element may be associated with a feature vector.

[0069] For example, the first dataset may correspond to a financial domain dataset, an e-commerce domain dataset, a social network domain dataset, a molecular chemistry domain dataset, or a genome sequence domain dataset. If the graphs in the first set of graphs 302A...302N correspond to a molecular chemistry domain dataset, the graphs in the first set of graphs 302A...302N may represent chemical molecules, the nodes of the graphs may represent atoms of the chemical molecules, and each edge between a pair of nodes (i.e., atoms) may represent a chemical bond (or the nature of the chemical bond) between the pair of atoms.

[0070] At 304, a set of explanation scores may be determined. In at least one embodiment, the processor 204 may be configured to determine the set of explanation scores based on application of the graph explainer model 110 to the first dataset 114 (i.e., the first set of graphs 302A...302N). The processor 204 may be configured to apply the predictive model a predetermined number of times to each graph in the first set of graphs 302A...302N to generate a subset of predictions for the corresponding graph. Thus, a set of predictions corresponding to the first set of graphs 302A...302N may be generated. The processor 204 may then be configured to apply the graph explainer model 110 to each subset of predictions generated for each graph in the first set of graphs 302A...302N to determine a subset of explanation scores of the set of explanation scores associated with the graph element of the corresponding graph in the first set of graphs 302A...302N. For example, a first subset of explanatory scores may be determined based on application of the graph explainer model 110 to a first subset of predictions of a set of predictions generated for a first graph (e.g., graph 302A). Similarly, an Nth subset of explanatory scores may be determined based on application of the graph explainer model 110 to an Nth subset of predictions of a set of predictions that may be generated for an Nth graph (e.g., graph 302N).

[0071] The number of explanation scores in each subset of explanation scores corresponding to each graph in the first set of graphs 302A...302N may be equal to the number of graph elements in the corresponding graph. For example, the first graph 302A may include 19 graph elements. For each graph element of the 19 graph elements, an explanation score in the first subset of explanation scores may be determined based on application of the graph explainer model 110 to the first subset of predictions. Thus, the first subset of explanation scores may include 19 explanation scores. Thus, each graph element in each graph in the first set of graphs 302A...302N may be associated with one explanation score in the set of explanation scores. Furthermore, an explanation score in the set of explanation scores associated with a graph element may indicate the contribution of the graph element to a prediction for a graph in the first set of graphs 302A...302N to which the graph element may belong.

[0072] At 306, a first set of graph elements 306A may be determined based on the determined set of explanatory scores. In at least one embodiment, the processor 204 may be configured to determine the first set of graph elements 306A based on the determined set of explanatory scores. The determination of the first set of graph elements 306A may be based on filtering the graph elements of each graph in the first set of graphs 302A...302N. The processor 204 may filter the graph elements of each graph in the first set of graphs 302A...302N based on the explanatory score (of the set of explanatory scores) associated with the graph elements of the corresponding graph. The unfiltered graph elements of each graph in the first set of graphs 302A...302N (i.e., the graph elements remaining after filtering) constitute the first set of graph elements 306A and may be determined as prediction-driving graph elements.

[0073] According to one embodiment, the processor 204 may apply the predictive model multiple times to each graph in the first set of graphs 302A...302N to generate multiple predictions for the corresponding graph. For example, the processor 204 may apply the predictive model four times to the first graph 302A to generate four predictions. The processor 204 may then apply the graph explainer model 110 to each of the four predictions to determine four subsets of initial explanation scores for the first graph 302A. If the first graph 302A includes 19 graph elements, each of the four subsets of initial explanation scores may include 19 initial explanation scores that may be associated with the 19 graph elements of the first graph 302A. Four predictions may be generated, and four subsets of initial explanation scores (corresponding to the four predictions) may be determined to overcome noise that may corrupt the explanation scores associated with the graph elements during each application of the predictive model and the graph explainer model 110. The noise may result in an erroneous identification of a graph element (of the first graph 302A) as a prediction-driving graph element, or a partial identification of a graph element (of the first graph 302A) that is driving the prediction. The processor 204 may normalize the initial explanation scores associated with the graph elements of each of the four subsets of initial explanation scores. The processor 204 may then perform an accumulation of the normalized explanation scores associated with each graph element in the four subsets of initial explanation scores to determine a first subset of explanation scores of the set of explanation scores.

[0074] Thus, multiple subsets of initial explanation scores may be determined corresponding to multiple predictions generated for each graph in the first set of graphs 302A...302N. The initial explanation scores in each of the multiple subsets of initial explanation scores may be normalized. The processor 204 may then perform an accumulation of the normalized initial explanation scores associated with each graph element of the corresponding graph in the multiple subsets of initial explanation scores to determine each subset of explanation scores.

[0075] The processor 204 may further sort the explanation scores of each subset of explanation scores in ascending order and determine a slope associated with the corresponding subset of sorted explanation scores. For example, the explanation scores of a first subset of explanation scores may be sorted, and a slope associated with the first subset of sorted explanation scores may be determined. The processor 204 may further identify an inflection point associated with the first subset of sorted explanation scores, where the determined slope exceeds a positive threshold slope. The processor 204 may further determine graph elements associated with the explanation scores of the first subset of explanation scores that exceed the identified inflection point. The determined graph elements of the first graph 302A may form a first subset of graph elements of the first set of graph elements 306A corresponding to the first graph 302A. The graph elements of the first graph 302A associated with explanation scores that do not exceed the inflection point may be filtered. The graph elements of the first subset of graph elements may be determined as contributing factors to the prediction for the first graph 302A or as prediction-driving graph elements for the first graph 302A. Similarly, the processor 204 may determine subsets of graph elements of the first set of graph elements 306A that correspond to each of the other graphs (such as the Nth graph 302N) in the first set of graphs 302A...302N to determine the first set of graph elements 306A.

[0076] At 308, a second dataset including a set of randomized graphs may be generated based on the first set of graphs 302A...302N. In at least one embodiment, the processor 204 may be configured to generate the second dataset, which may include the set of randomized graphs 308A, based on the first set of graphs 302A...302N. To generate the set of randomized graphs, the processor 204 may first select a set of graph elements from each graph in the first set of graphs 302A...302N using automated machine learning (AutoML). According to an embodiment, the processor 204 may be configured to perform principal component analysis (PCA) and similarity analysis on each graph element within each set of graph elements selected from each graph in the first set of graphs 302A...302N. The principal component analysis may be performed to identify significant graph elements in each set of graph elements selected from each graph (e.g., the first graph 302A). The similarity analysis may be performed to identify insignificant graph elements within each set of graph elements selected from each graph.

[0077] The processor 204 may then identify combinations of graph elements from the set of graph elements selected from each graph in the first set of graphs 302A...302N. For each graph (e.g., the first graph 302A), a set of combinations of graph elements may be identified. For example, five combinations of graph elements may be identified for the first graph 302A (i.e., the set of combinations identified for the first graph 302A may include five combinations). The graph elements in each of the five combinations may be obtained from the set of graph elements selected from the first graph 302A using AutoML.

[0078] According to an embodiment, the processor 204 may further store each combination identified for each graph in the first set of graphs 302A...302N in a decision tree. The processor 204 may then sort the combinations in the decision tree based on the importance and dependency of each graph element in each combination (determined based on principal component analysis and similarity analysis). The processor 204 may further select each combination identified for each graph in the first set of graphs 302A...302N. For example, a first combination for the first graph 302A may be selected. Then, feature values ​​of the graph elements of the first graph 302A included in the first combination may be randomized to generate a randomized graph corresponding to the graph 302A. Similarly, other randomized graphs corresponding to the graph 302A may be generated based on randomizing the feature values ​​of the graph elements included in the other combinations identified for the first graph 302A. The generated randomized graph corresponding to the graph 302A may constitute a first subset of randomized graphs in the set of randomized graphs 308A. Similarly, randomized graphs may be generated that correspond to each of the other graphs in the first set of graphs 302A...302N, which may constitute other subsets of randomized graphs in the set of randomized graphs 308A.

[0079] At 310, a set of noise scores may be determined. In at least one embodiment, the processor 204 may be configured to determine the set of noise scores based on application of the graph explainer model 110 to the second dataset 308 (i.e., the set of randomized graphs 308A). The processor 204 may apply the prediction model to each randomized graph in the set of randomized graphs 308A to generate a prediction in the set of predictions. The processor 204 may then apply the graph explainer model 110 to each prediction in the set of predictions. A set of explanation scores may be determined based on application of the graph explainer model 110 to each prediction in the set of predictions. The processor 204 may then determine whether the distribution of the set of explanation scores is even.

[0080] For example, the set of explanation scores may be determined based on application of the graph explainer model 110 to predictions that may be generated based on application of the predictive model to a randomized graph (of the first subset of randomized graphs) corresponding to the first graph 302A. If the distribution of the set of explanation scores is determined to be even, the processor 204 may determine combinations that include graph elements of the first graph 302A identified for the first graph 302A that may not be driving the predictions for the first graph 302A. The determined combinations may include graph elements of the first graph 302A that were randomized for generation of the randomized graph corresponding to the first graph 302A.

[0081] The processor 204 may similarly determine other combinations of the set of combinations identified for the first graph 302A, which may include graph elements of the first graph 302A whose features were randomized to generate another randomized graph corresponding to the first graph 302A. Such a combination may be identified for the first graph 302A if the set of explanatory scores associated with the graph elements of the randomized graph corresponding to the first graph 302A is determined to be evenly distributed. Among the determined combinations, the processor 204 may select the longest combination. The selected longest combination for the first graph 302A may be determined as a first combination for the first graph 302A. The first combination may be used to determine a first subset of noise scores associated with the first graph 302A. Conversely, if the distribution of the set of explanation scores associated with the graph elements of the randomized graph corresponding to the first graph 302A is uneven (odd), the corresponding combination identified for the first graph 302A (and including the graph elements whose features were randomized to generate the randomized graph) may be removed from the decision tree.

[0082] Similarly, combinations may be determined for each of the other graphs in the first set of graphs 302A...302N to determine each subset of noise scores in the set of noise scores. For example, an Nth combination may be determined for an Nth graph 302N, such that the Nth combination may include graph elements of the Nth graph 302N that may not be driving the prediction for the Nth graph 302N.

[0083] The processor 204 may be further configured to apply a predictive model to the graph elements in each determined combination identified for each graph in the first set of graphs 302A...302N to generate a set of predictions. The processor 204 may then apply the graph explainer model 110 to each prediction in the set of predictions to generate a respective subset of noise scores in the set of noise scores. For example, a first prediction may be generated based on application of the predictive model to the graph elements in the first combination determined for the first graph 302A. The graph explainer model 110 may be applied to the first prediction to determine a first subset of noise scores for the first graph 302A. Similarly, an Nth subset of noise scores in the set of noise scores associated with the graph elements in the Nth combination (determined for the Nth graph 302N) may be determined for the Nth graph 302N. The determination of the Nth subset of noise scores may be based on application of the graph explainer model 110 to the Nth prediction for the graph elements in the Nth combination.

[0084] At 312, a set of denoised explanation scores may be determined based on the set of noise scores. In at least one embodiment, the processor 204 may be configured to determine the set of denoised explanation scores based on the set of noise scores. The processor 204 may calculate a graph element-wise subtraction of the noise scores of each subset of noise scores for each graph in the first set of graphs 302A...302N from the explanation scores of each subset of explanation scores in the set of explanation scores associated with the graph elements of the corresponding graph. Based on such subtraction, a subset of denoised explanation scores of the set of denoised explanation scores for the corresponding graph may be determined. For example, a first subset of denoised explanation scores (for the first graph 302A) may be determined based on a graph element-wise subtraction of the noise scores of the first subset of noise scores (for the first graph 302A) from the explanation scores of the first subset of explanation scores (associated with the graph elements of the first graph 302A). Similarly, an Nth subset of denoised explanatory scores for the Nth graph 302N may be determined based on the Nth subset of noise scores for the Nth graph 302N and the Nth subset of explanatory scores associated with the graph elements of the Nth graph 302N.

[0085] In some embodiments, the processor 204 may normalize and calculate the average of the denoised explanation scores for each subset of the denoised explanation scores associated with the graph elements of the graphs in the first set of graphs 302A...302N.

[0086] At 314, a second set of graph elements 314A may be determined based on the determined set of denoised scores. In at least one embodiment, the processor 204 may be configured to determine the second set of graph elements 314A based on the determined set of denoised scores. This determination may be based on filtering the graph elements of each graph in the first set of graphs 302A...302N based on a subset of the denoised explanatory scores for the corresponding graph. The denoised explanatory scores in each subset of the denoised explanatory scores for the graph may indicate the contribution of the graph element to the prediction for the graph. A higher denoised explanatory score associated with a graph element indicates a greater contribution, causing the graph element to be determined as a prediction-driving graph element for the corresponding graph. Conversely, a lower denoised explanatory score associated with a graph element indicates a lower contribution, causing the graph element to be determined as a graph element that may not be driving the prediction for the corresponding graph.

[0087] The processor 204 may be configured to filter graph elements from each graph that may not be driving predictions for the corresponding graph. Such graph elements may be determined based on a subset of denoised explanatory scores associated with the graph elements of the corresponding graph. The unfiltered graph elements (i.e., the graph elements that remain after filtering) of the corresponding graphs of the first set of graphs 302A...302N may constitute a subset of the graph elements of the second set of graph elements 314A. For example, the processor 204 may determine a first subset of the graph elements of the second set of graph elements 314A based on filtering the graph elements that may not be driving predictions for the first graph 302A. The unfiltered graph elements of the first graph 302A may constitute the first subset of the graph elements of the second set of graph elements 314A. The graph elements selected for filtering may be determined based on the denoised explanatory scores associated with the graph elements. The denoised explanatory scores may belong to the first subset of the denoised explanatory scores for the first graph 302A. Similarly, the Nth subset of graph elements of the second set of graph elements 314A may be determined by filtering graph elements that may not be driving the prediction for the Nth graph 302N. The unfiltered graph elements of the Nth graph 302N may constitute the Nth subset of graph elements of the second set of graph elements 314A. The graph elements selected for filtering may be determined based on denoised explanation scores associated with the graph elements. The denoised explanation scores may belong to the Nth subset of denoised explanation scores for the Nth graph 302N.

[0088] At 316, a set of predicted driven graph elements associated with the first set of graphs 302A...302N may be determined based on the first set of graph elements 306A and the second set of graph elements 314A. In at least one embodiment, the processor 204 may be configured to determine the set of predicted driven graph elements associated with the first set of graphs 302A...302N based on the first set of graph elements 306A and the second set of graph elements 314A. The processor 204 may determine a first set of feature vectors for the first set of graph elements 306A. Each feature vector in the first set of feature vectors may correspond to a graph element in the first set of graph elements 306A. The processor 204 may further determine a second set of feature vectors for the second set of graph elements 314A. Each feature vector in the second set of feature vectors may correspond to a graph element in the second set of graph elements 314A.

[0089] According to an embodiment, the processor 204 may be configured to apply unsupervised learning to the first set of feature vectors and the second set of feature vectors using a machine learning model. The unsupervised learning may be applied to determine similarity between the feature vectors of the first set of graph elements 306A (i.e., the first set of feature vectors) and the feature vectors of the second set of graph elements 314A (i.e., the second set of feature vectors). Based on the unsupervised learning, the feature vectors of the first set of feature vectors and the feature vectors of the second set of feature vectors may be grouped into multiple clusters for clustering search. Each cluster of the multiple clusters may include the first set of feature vectors, the second set of feature vectors, or the first set of feature vectors and the second set of feature vectors. The feature vectors grouped into one cluster of the multiple clusters may be determined to be similar based on the unsupervised learning. For each graph element included in the first set of graph elements 306A and the second set of graph elements 314A, the feature vector corresponding to the graph element may be included in the first set of feature vectors and the second set of feature vectors. Additionally, the processor 204 may group feature vectors corresponding to different graph elements of the same graph in the first set of graphs 302A...302N into clusters based on similarities between the feature vectors determined based on unsupervised learning.

[0090] For example, the first set of graphs 302A...302N may include four graphs: first graph 302A, second graph 302B, third graph 302C, and fourth graph 302D. The first set of feature vectors may include 55 feature vectors that may correspond to 55 graph elements that may belong to different graphs in the first set of graphs 302A...302N. The second set of feature vectors may include 37 feature vectors that may correspond to 37 graph elements that may belong to different graphs in the first set of graphs 302A...302N. The 82 feature vectors may be grouped into 11 clusters, and the feature vectors grouped in each cluster may be similar.

[0091] The number of feature vectors grouped into each cluster may vary. The processor 204 may retain or reject one of the multiple clusters based on the number of feature vectors grouped into the cluster. If the number of feature vectors grouped into a cluster is less than a threshold number of feature vectors, the cluster may be rejected. However, if the number of feature vectors grouped into a cluster is equal to or greater than the threshold number of feature vectors, the cluster may be retained. For example, of 11 clusters, the processor 204 may reject five clusters and retain six clusters. In one example, the six retained clusters may contain 60 feature vectors.

[0092] The processor 204 may further determine a feature vector corresponding to a graph element of each graph in the first set of graphs 302A...302N that may be grouped into the retained cluster of the plurality of clusters. The determined feature vectors corresponding to the graph elements of each graph may be sorted into buckets. Thus, if the number of graphs included in the first set of graphs 302A...302N is "N," the determined feature vectors may be sorted into "N" buckets. The graph elements corresponding to the determined feature vectors may constitute a set of prediction-driving graph elements associated with the first set of graphs 302A...302N.

[0093] At 318, graph data including explainability information associated with the first set of graphs 302A...302N may be rendered based on the determined set of prediction-driven elements. In at least one embodiment, the processor 204 may be configured to render graph data including explainability information associated with the first set of graphs 302A...302N based on the determined set of prediction-driven elements. The explainability information may include user-interpretable information and a visualization. The user-interpretable information may include graph elements of the determined set of prediction-driven graph elements and explanation scores associated with the graph elements. The visualization may include graphical user interface elements that may correspond to graphs of the first set of graphs 302A...302N rendered on the user end device 104 or the display device 210A. Graph elements included in the determined set of prediction-driven graph elements may be annotated in the rendered graph to which the graph elements belong.

[0094] If the first dataset, i.e., the first set of graphs 302A...302N, corresponds to a financial domain dataset, the explainability information associated with the first set of graphs 302A...302N can facilitate users in making wise financial decisions, such as approving loans, identifying transaction fraud, or creating transparency in financial decisions. If the first dataset corresponds to an e-commerce domain dataset or a social network domain dataset, the explainability information can facilitate accurate predictions, increase the fidelity of user pattern detection in product purchase histories, and generate correct recommendations. If the first dataset corresponds to a molecular chemistry domain dataset, the explainability information can allow for the precise identification of atoms or bonds that drive the chemical properties of molecules, or for the collective comparison of sets of molecules to ascertain the causes of the chemical properties of the sets of molecules. If the first dataset corresponds to a genome sequence domain dataset, the explainability information can allow for the determination of correlations and previously unknown relationships between genes and provide explanations behind the relationships between genes.

[0095] Embodiments of the present disclosure may enable detection of noise in explanation results (i.e., explanation scores). Noise may be introduced into explanation results due to incorrect identification of graph elements (i.e., node features, edge features, or graph structure) that drive predictions on a graph. Because the explanation scores associated with incorrectly identified graph elements may change during each application of the predictive model and graph explainer model 110, incorrectly identified graph elements may be detected. Because graph elements identified as prediction drivers may change during each application of the predictive model or graph explainer model 110, embodiments of the present disclosure may further allow for determination of partial identification of prediction-driving graph elements. Embodiments of the present disclosure may further facilitate global analysis of explanation results instead of local interpretation (i.e., generating explanation results for each prediction). The global analysis may be automatic and may not require the involvement of a subject matter expert. The global analysis may also enable mitigation of confirmation bias. Embodiments of the present disclosure may incorporate multiple explainability techniques (such as explanation score accumulation and explanation score denoising), which may target different graph elements (nodes, edges, or graph structures) as explanations for predictions. Furthermore, embodiments of the present disclosure may enable the integration of graph analysis and visualization frameworks, which in turn may allow for sorting, plotting, and grouping of explanations, as well as annotating graph elements for all graphs in a graph dataset. Embodiments of the present disclosure may further provide explainability metrics to allow users to sort explanation results into different ranges. Embodiments of the present disclosure may provide a user interface for annotating explanation results against the original graph dataset (i.e., graphs in the graph dataset).

[0096] FIG. 4 illustrates a flowchart of an exemplary method for determining a set of prediction-driving graph elements in a graph dataset based on explanation scores, according to certain embodiments of the present disclosure. FIG. 4 is described in conjunction with elements of FIGS. 1, 2, and 3. Referring to FIG. 4, a flowchart 400 is shown. The method illustrated in flowchart 400 may begin at 402 and may be performed by any suitable system, apparatus, or device, such as the example electronic device 102 of FIG. 1 or the processor 204 of FIG. 2. Although illustrated with discrete blocks, steps and operations associated with one or more of the blocks of flowchart 400 may be divided into additional blocks, combined into fewer blocks, or eliminated, depending on the particular implementation.

[0097] At block 402, a first graph may be selected from a first set of graphs (e.g., first set of graphs 302A...302N). In an embodiment, the processor 204 may be configured to select the first graph from the first set of graphs 302A...302N. The first set of graphs 302A...302N may be included in a received first data set (see 302 in FIG. 3). As an example, graph 302A may be selected as the first graph.

[0098] At block 404, a first set of operations may be performed on the selected first graph (e.g., graph 302A) to determine a first subset of graph elements of a first set of graph elements (e.g., first set of graph elements 306A) that may belong to the selected first graph (e.g., graph 302A) and a first subset of explanation scores. In an embodiment, processor 204 may be configured to perform the first set of operations on the selected first graph (e.g., graph 302A) to determine a first subset of explanation scores and a first subset of graph elements of the first set of graph elements (e.g., first set of graph elements 306A) that may belong to the selected first graph (e.g., graph 302A). The first set of graph elements 306A may include graph elements that may drive predictions for the first set of graphs 302A...302N. The subset of graph elements of the first set of graph elements 306A may include graph elements of graph 302A that may be driving predictions for graph 302A. A first set of operations may be performed on the selected first graph (now graph 302A) to determine the graph elements that drive the predictions for graph 302A. This determination may be based on a set of explanation scores. The set of explanation scores may be determined based on application of the graph explainer model 110 to predictions generated by the predictive model for the first set of graphs 302A...302N. The first set of operations may include block 404A, block 404B, block 404C, block 404D, block 404E, block 404F, block 404G, block 404H, and block 404I.

[0099] At block 404A, a predetermined number of subsets of initial explanation scores may be determined based on application of the graph explainer model 110 to predictions generated for the selected first graph. In an embodiment, the processor 204 may be configured to determine the predetermined number of subsets of initial explanation scores based on application of the graph explainer model 110 to predictions generated for the selected first graph (at this stage, graph 302A). Each initial explanation score in each subset of initial explanation scores among the predetermined number of subsets of initial explanation scores may be associated with a graph element of the selected first graph (e.g., graph 302A).

[0100] Each subset of initial explanation scores of the predetermined number of subsets of initial explanation scores may be determined based on applying a predictive model to the graph 302A to generate a prediction, and applying the graph explainer model 110 to the generated prediction. The processor 204 may generate multiple predictions (i.e., a predetermined number of predictions) for the graph 302A based on the multiple applications of the predictive model to the graph 302A. Further, a set of initial explanation scores may be determined based on applying the graph explainer model 110 to each prediction. Thus, multiple sets of initial explanation scores corresponding to multiple predictions may be determined.

[0101] At block 404B, the initial explanation scores for each subset of the predetermined number of subsets of the initial explanation scores may be normalized. In an embodiment, processor 204 may be configured to normalize the initial explanation scores for each subset of the predetermined number of subsets of the initial explanation scores. For example, processor 204 may apply a min-max normalization technique to normalize the initial explanation scores to a range of values ​​between 0 and 1.

[0102] At block 404C, an accumulation of normalized initial explanation scores associated with each graph element in the predetermined number of subsets of initial explanation scores may be performed to obtain a first subset of explanation scores of the set of explanation scores. In an embodiment, processor 204 may be configured to perform an accumulation of normalized initial explanation scores associated with each graph element in the predetermined number of subsets of initial explanation scores to obtain a first subset of explanation scores of the set of explanation scores. The normalized initial explanation scores associated with each graph element in the multiple sets of normalized initial explanation scores may be accumulated. Based on such accumulation, a consolidated set of normalized initial explanation scores may be obtained. The consolidated set may correspond to a first subset of explanation scores of the set of explanation scores associated with the graph elements of the first graph (e.g., graph 302A). The first subset of explanation scores may be referred to as a normalized subset of explanation scores because the explanation scores in the first subset of explanation scores may be obtained based on normalization of the initial explanation scores.

[0103] At block 404D, the explanation scores in the first subset of explanation scores may be sorted. In an embodiment, the processor 204 may be configured to sort the explanation scores in the first subset of explanation scores. The explanation scores may be sorted in ascending order. The first explanation score in the first subset of explanation scores may be the smallest, and the last explanation score in the first subset of explanation scores may be the largest. Graphic elements of graph 302A associated with larger explanation scores may be driving predictions about graph 302A.

[0104] At block 404E, a slope associated with the sorted explanation scores may be determined. In an embodiment, the processor 204 may be configured to determine a slope associated with the sorted explanation scores. The slope may be determined to identify graph elements that may be driving predictions regarding the first graph (e.g., graph 302A).

[0105] At block 404F, an inflection point associated with the sorted explanation scores may be identified, where the determined slope may exceed a positive threshold slope. In an embodiment, the processor 204 may be configured to identify an inflection point associated with the sorted explanation scores, where the determined slope may exceed a positive threshold slope. Identifying the inflection point may allow identifying a first subset of graph elements of the first set of graph elements 306A that drives a prediction for the first graph (e.g., graph 302A). The inflection point may be identified based on a slope associated with the sorted explanation scores of the first subset of explanation scores.

[0106] At block 404G, graph elements beyond the identified inflection point may be identified as a first subset of graph elements of the first set of graph elements 306A. In one embodiment, the processor 204 may be configured to identify graph elements beyond the identified inflection point as the first subset of graph elements of the first set of graph elements 306A. The processor 204 may determine a first transition from a low derivative to a high derivative based on the slope. Graph elements of the first graph (e.g., graph 302A) associated with the sorted explanation scores (of the first subset of explanation scores) to the right of the inflection point may be identified as prediction-driving graph elements. The identified graph elements may constitute a first subset of graph elements of the first set of graph elements 306A. Graph elements to the left of the inflection point may be filtered, and these graph elements may not contribute to the prediction for the first graph (now graph 302A).

[0107] At block 404H, it may be determined whether the first set of graphs 302A...302N includes at least one unprocessed graph. In an embodiment, the processor 204 may be configured to determine whether the first set of graphs 302A...302N includes at least one unprocessed graph. For example, in a first instance of execution of the first set of operations (i.e., a first iteration), processing of graph 302A may be completed, and "N-1" graphs, i.e., graphs 302B...302N of the first set of graphs 302A...302N, may be unprocessed.

[0108] At block 404I, a second graph (e.g., graph 302B) from the first set of graphs 302A...302N may be reselected as the selected first graph based on the first set of graphs 302A...302N including at least one unprocessed graph. In an embodiment, the processor 204 may be configured to reselect a second graph (i.e., graph 302B) from the first set of graphs 302A...302N as the selected first graph based on the first set of graphs 302A...302N including at least one unprocessed graph. Thus, at the current stage (i.e., at the end of the first iteration of the execution of the first set of operations), graph 302B may be selected as the first graph, and the first set of operations may be executed in the second iteration.

[0109] At block 406, the set of explanation scores and the first set of graph elements 306A may be obtained based on iterative execution of the first set of operations until the first set of graphs 302A...302N contains no unprocessed graphs. In an embodiment, the processor 204 may be configured to obtain the set of explanation scores and the first set of graph elements 306A based on iterative execution of the first set of operations until the first set of graphs 302A...302N contains no unprocessed graphs. In each iteration, a graph (e.g., the Nth graph 302N) from the set of graphs 302A...302N may be selected as the first graph, and a subset (e.g., the Nth subset) of graph elements from the first set of graph elements 306A may be identified. The first set of graph elements 306A may be obtained when a subset of graph elements from the first set of graphs 302A...302N is identified for each graph in the first set of graphs 302A...302N. Control can be passed to the end.

[0110] Although flowchart 400 is depicted as discrete operations such as 402, 404 (including 404A-404I), and 406, the disclosure is not so limited. However, in certain embodiments, such discrete operations may be further divided into additional operations, combined into fewer operations, or eliminated, depending on the particular implementation, without detracting from the essence of the disclosed embodiments.

[0111] 5A and 5B collectively illustrate a flowchart of an exemplary method for determining graph elements in a graph dataset that may not be driving a prediction, in accordance with certain embodiments of the present disclosure. FIGS. 5A and 5B are described in conjunction with elements from FIGS. 1, 2, 3, and 4. With reference to FIGS. 5A and 5B, a flowchart 500 is shown. The method illustrated in flowchart 500 begins at 502 and may be performed by any suitable system, apparatus, or device, such as the exemplary electronic device 102 of FIG. 1 or the processor 204 of FIG. 2. While shown in individual blocks, steps and operations associated with one or more of the blocks of flowchart 500 may be divided into additional blocks, combined into fewer blocks, or eliminated, depending on the particular implementation.

[0112] At block 502, the importance of a set of graph elements in each graph in the first dataset (i.e., the first set of graphs 302A...302N) may be determined based on a dimensionality reduction model and a similarity analysis model. In an embodiment, the processor 204 may be configured to determine the importance of a set of graph elements in each graph in the first dataset (i.e., the first set of graphs 302A...302N) based on the dimensionality reduction model and the similarity analysis model. The processor 204 may use automated ML to select a set of graph elements from each graph in the first dataset and determine the importance of each graph element in the selected set of graph elements. The importance determination may be based on principal component analysis and similarity analysis for each graph element in the selected set of graph elements. In an embodiment, the dimensionality reduction model may correspond to a principal component analysis (PCA) model.

[0113] At block 504, combinations of graph elements may be identified from the selected set of graph elements based on the determined importance. In an embodiment, the processor 204 may be configured to identify combinations of graph elements from the selected set of graph elements based on the determined importance. From the set of graph elements selected from each graph, a set of combinations of graph elements for the corresponding graph may be identified. For example, for a set of graph elements selected from the first graph 302A using automated ML, a set of combinations of graph elements may be identified based on the importance of each graph element in the set of graph elements (determined based on principal component analysis and similarity analysis of each graph element in the set of graph elements). Thus, for the first set of graphs 302A...302N (i.e., the first data set), "N" sets of combinations may be identified. Each set of combinations in the "N" sets of combinations may include at least one identified combination of graph elements.

[0114] At block 506, the identified combinations may be sorted into a decision tree based on the determined importance and dependency of each graph element in each of the identified combinations. In an embodiment, the processor 204 may be configured to sort the identified combinations into a decision tree based on the importance and dependency of each graph element in each of the identified combinations. The identified combinations sorted into the decision tree may be sorted based on the importance and dependency of each graph element.

[0115] At block 508, a first combination of graph elements may be selected from the decision tree. In an embodiment, the processor 204 may be configured to select the first combination of graph elements from the decision tree. The graph elements in the selected first combination of graph elements may belong to a particular graph (e.g., graph 302A) of the first set of graphs 302A...302N. The first combination may be an identified combination of the set of identified combinations for graph 302A.

[0116] A second set of operations may be performed on the selected first combination of graph elements at block 510. In an embodiment, processor 204 may be configured to perform the second set of operations on the selected first combination of graph elements. The second set of operations may include block 510A, block 510B, block 510C, block 510D, block 510E, block 510F, block 510G, and block 510H.

[0117] At block 510A, an artificial dataset may be generated by randomizing the graph elements in the selected first combination of graph elements. In an embodiment, the processor 204 may be configured to generate the artificial dataset by randomizing the graph elements in the selected first combination of graph elements. The second dataset (see 308 in FIG. 3 ) may correspond to the artificial dataset. The processor 204 randomizes the feature values ​​of each graph element in the selected first combination of graph elements. Based on the randomization of the features of the graph elements in the first combination, a randomized graph corresponding to graph 302A may be generated. The randomized graph may be part of the artificial dataset.

[0118] At block 510B, a distribution of explanation scores for the generated artificial dataset (including only the randomized graphs corresponding to graph 302A) may be determined. In an embodiment, processor 204 may be configured to determine the distribution of explanation scores for the generated artificial dataset. Processor 204 may apply a predictive model to the randomized graphs to generate predictions. Processor 204 may then apply graph explainer model 110 to the generated predictions to determine explanation scores associated with the explanation scores associated with graph elements in the randomized graphs.

[0119] At block 510C, it may be determined whether the determined distribution of the explanation scores is an even distribution. In an embodiment, the processor 204 may be configured to determine whether the determined distribution of the explanation scores is an even distribution.

[0120] At block 510D, the first combination of graph elements may be identified as a predicted non-driven graph element based on the determined distribution of explanation scores being an even distribution. In one embodiment, the processor 204 may be configured to identify the first combination of graph elements as a predicted non-driven graph element based on the determined distribution of explanation scores being an even distribution. The explanation scores associated with the predicted non-driven graph elements may contribute noise in a first subset of explanation scores (of the set of explanation scores) that may be determined based on application of the graph explainer model 110 to predictions generated for the graph 302A (because the graph elements in the first combination belong to the graph 302A). Thus, the first combination of graph elements may be used to generate a first subset of noise scores of the set of noise scores associated with the first graph 302A (see 310 in FIG. 3).

[0121] At block 510E, a set of branches may be removed from the decision tree based on the determined distribution of explanation scores not being an even distribution. In an embodiment, the processor 204 may be configured to remove a set of branches from the decision tree based on the determined distribution of explanation scores not being an even distribution. The removed set of branches may include a first combination. The processor 204 may determine that the graph elements in the first combination may be driving the prediction for the first graph 302A or that the determined distribution of explanation scores associated with the graph elements in the first combination may be uneven.

[0122] At block 510F, it may be determined whether the decision tree includes at least one unprocessed combination of graph elements. In an embodiment, processor 204 may be configured to determine that the decision tree includes at least one unprocessed combination of graph elements. For example, after the first iteration of the second set of operations, only the first combination of graph 302A may have been processed.

[0123] At block 510G, a second combination of graph elements may be reselected from the decision tree as the selected first combination based on the decision tree including at least one unprocessed combination. In an embodiment, processor 204 may be configured to reselect a second combination of graph elements from the decision tree as the selected first combination based on the decision tree including at least one unprocessed combination. At this stage (i.e., at the end of the first iteration of the second set of operations), only the first combination of graph 302A may be a processed combination, so the second combination of graph elements may be reselected.

[0124] At block 512, the identified combinations of graph elements may be determined as predicted non-driving graph elements based on the iterative execution of the second set of operations until the decision tree contains no unprocessed combinations of graph elements. In one embodiment, the processor 204 may be configured to determine the identified combinations of graph elements as predicted non-driving graph elements based on the iterative execution of the second set of operations until the decision tree contains no unprocessed combinations of graph elements. Thus, the processor 204 may determine, for the first set of graphs 302A...302N, combinations of graph elements that may not be driving the prediction, i.e., predicted non-driving graph elements. The determined combinations may be used to determine baseline noise that may be affecting the explanation scores associated with the graph elements of each graph in the first set of graphs 302A...302N. The distribution of explanation scores that may be determined based on randomized graphs (generated based on randomizing the features of the determined combinations of graph elements) may be uniform. Control may then pass to an end.

[0125] Although flowchart 500 is shown as discrete operations such as 502, 504, 506, 508, 510 (510A-510G), and 512, the disclosure is not so limited. However, in certain embodiments, such discrete operations may be further divided into additional operations, combined into fewer operations, or eliminated, depending on the particular implementation, without detracting from the essence of the disclosed embodiments.

[0126] FIG. 6 illustrates a flowchart of an exemplary method for determining a denoised description score based on a set of noise scores, according to certain embodiments of the present disclosure. FIG. 6 will be described in conjunction with elements from FIGS. 1, 2, 3, 4, 5A, and 5B. Referring to FIG. 6, a flowchart 600 is shown. The method illustrated in flowchart 600 may begin at 602 and may be performed by any suitable system, apparatus, or device, such as the exemplary electronic device 102 of FIG. 1 or the processor 204 of FIG. 2. Although illustrated with discrete blocks, steps and operations associated with one or more of the blocks of flowchart 600 may be divided into additional blocks, combined into fewer blocks, or eliminated, depending on the particular implementation.

[0127] At block 602, a set of baseline noise scores may be determined based on application of the graph explainer model to the identified combinations of graph elements determined as predicted non-driven graph elements. In one embodiment, the processor 204 may be configured to determine the set of baseline noise scores based on application of the graph explainer model 110 to the identified combinations of graph elements that may be determined as predicted non-driven graph elements. The set of baseline noise scores may correspond to a set of noise scores. The set of baseline noise scores may include "N" subsets of the baseline scores. Each subset of baseline scores (i.e., each subset of noise scores) may be determined based on application of the predictive model to each combination of graph elements (determined as predicted non-driven graph elements) and application of the graph explainer model 110 to predictions generated by the predictive model based on the corresponding combinations.

[0128] At block 604, a noise component may be removed from the set of explanatory scores associated with the received first dataset based on the determined set of baseline noise scores to determine a set of denoised scores. In an embodiment, the processor 204 may be configured to remove the noise component from the set of explanatory scores associated with the received first dataset based on the determined set of baseline noise scores to determine a set of denoised scores. The noise component may be introduced into the set of explanatory scores due to predicted non-driving graph elements in the received first dataset. The processor 204 may calculate a subtraction of a baseline noise score in the set of baseline noise scores associated with each graph element of the first dataset from an explanatory score in the set of explanatory scores associated with a corresponding graph element of the first dataset to determine each denoised score in the set of denoised scores.

[0129] At block 606, the determined set of denoised scores may be normalised and averaged. In an embodiment, the processor 204 may be configured to normalise and average the determined set of denoised scores. A set of denoised explanation scores may be determined further based on the normalisation and averaging of the determined set of denoised scores. Control may pass to an end.

[0130] Although flowchart 600 is depicted as discrete operations such as 602, 604, and 606, the disclosure is not so limited. However, in certain embodiments, such discrete operations may be further divided into additional operations, combined into fewer operations, or eliminated, depending on the particular implementation, without detracting from the essence of the disclosed embodiments.

[0131] FIG. 7 illustrates a flowchart of an exemplary method for determining a set of predictively driven graph elements based on feature vectors of the graph elements, according to an embodiment of the present disclosure. FIG. 7 is described in conjunction with elements from FIGS. 1, 2, 3, 4, 5A, 5B, and 6. Referring to FIG. 7, a flowchart 700 is shown. The method illustrated in flowchart 700 may begin at 702 and may be performed by any suitable system, apparatus, or device, such as the exemplary electronic device 102 of FIG. 1 or the processor 204 of FIG. 2. Although illustrated with discrete blocks, steps and operations associated with one or more of the blocks of flowchart 700 may be divided into additional blocks, combined into fewer blocks, or eliminated, depending on the particular implementation.

[0132] At block 702, a set of feature vector clusters may be determined, including the feature vectors of the first set of feature vectors and the second set of feature vectors. In one embodiment, the processor 204 may be configured to determine the set of feature vector clusters, including the feature vectors of the first set of feature vectors and the second set of feature vectors. The processor may cluster the set of feature vectors of the first set of feature vectors and the second set of feature vectors based on similarity between each pair of the feature vectors of the first set of feature vectors, the second set of feature vectors, and the feature vectors of the first set of feature vectors or the second set of feature vectors. The similarity may be determined through application of unsupervised learning (e.g., clustering using a machine learning model (e.g., using K-means clustering)) to the first set of feature vectors and the second set of feature vectors.

[0133] At block 704, a set of vector components common to a first feature vector cluster of the set of feature vector clusters may be determined. In an embodiment, the processor 204 may be configured to determine a set of vector components (from the first set of feature vectors and the second set of feature vectors) that may be common to a first feature vector cluster of the set of feature vector clusters. A set of prediction-driven graph elements may be determined based on the determined set of vector components.

[0134] For example, each cluster may have certain similar feature vectors, but the clusters may not all be exactly the same. For each graph (e.g., "Graph 1" and "Graph 10"), the feature vector of each node may be evaluated for similarity to the feature vectors in each cluster (e.g., Cluster "A" and Cluster "B") using a similarity metric such as Pearson correlation. If the feature vector of each node in a graph (e.g., "Graph 1") is highly correlated to a particular cluster (e.g., "Cluster A"), the graph may be assigned that particular cluster (i.e., "Graph 1" may belong to "Cluster A"). Using dimensionality reduction techniques (such as principal component analysis (PCA)), common feature vectors within the clusters may be identified as feature values ​​that drive predictions for the corresponding graphs. For example, a PCA of all feature vectors in "Cluster A" may drive the prediction of "Graph 1". Details of determining the set of prediction-driving graph elements are provided further, for example, in FIG. 3 (316). Control may then pass to End.

[0135] Although flowchart 700 is depicted as discrete operations such as 702 and 704, the disclosure is not so limited. However, in certain embodiments, such discrete operations may be further divided into additional operations, combined into fewer operations, or eliminated, depending on the particular implementation, without detracting from the essence of the disclosed embodiments.

[0136] FIG. 8 illustrates a flowchart of an exemplary method for rendering explainability information associated with a graph dataset, according to certain embodiments of the present disclosure. FIG. 8 will be described in conjunction with elements from FIGS. 1, 2, 3, 4, 5A, 5B, 6, and 7. Referring to FIG. 8, a flowchart 800 is shown. The method illustrated in flowchart 800 may begin at 802 and may be performed by any suitable system, apparatus, or device, such as the example electronic device 102 of FIG. 1 or the processor 204 of FIG. 2. Although illustrated with discrete blocks, steps and operations associated with one or more of the blocks of flowchart 800 may be divided into additional blocks, combined into fewer blocks, or eliminated, depending on the particular implementation.

[0137] At block 802, each prediction-driven graph element of the determined set of prediction-driven elements associated with each of the first set of graphs (e.g., first set of graphs 302A...302N) may be annotated. In an embodiment, the processor 204 may be configured to annotate each prediction-driven graph element (e.g., a node, an edge, or a graph structure) of the determined set of prediction-driven graph elements associated with each graph of the first set of graphs 302A...302N. The annotation for each prediction-driven graph element of the set of prediction-driven graph elements may correspond to explainability information associated with the first set of graphs 302A...302N.

[0138] At block 804, graph data including explainability information associated with the first set of graphs 302A...302N may be rendered. In an embodiment, the processor 204 may be configured to render the graph data including the explainability information associated with the first set of graphs 302A...302N. The processor 204 may control the user end device 104 or the display device 210A to render the explainability information. For example, if a node in a graph of the first set of graphs 302A...302N is determined as a prediction-driving graph element, the processor 204 may control the user end device 104 or the display device 210A to annotate the node and render the graph. Control may pass to an end.

[0139] Although flowchart 800 is depicted as discrete operations such as 802 and 804, the disclosure is not so limited. However, in certain embodiments, such discrete operations may be further divided into additional operations, combined into fewer operations, or eliminated, depending on the particular implementation, without detracting from the essence of the disclosed embodiments.

[0140] FIG. 9 illustrates a flowchart of an exemplary method for determining a global explainability metric for a global interpretation of explainability information associated with predictions on a graph dataset, according to certain embodiments of the present disclosure. FIG. 9 will be described in conjunction with elements from FIGS. 1, 2, 3, 4, 5A, 5B, 6, 7, and 8. Referring to FIG. 9, a flowchart 900 is shown. The method illustrated in flowchart 900 may begin at 902 and may be performed by any suitable system, apparatus, or device, such as the exemplary electronic device 102 of FIG. 1 or the processor 204 of FIG. 2. Although illustrated with discrete blocks, steps and operations associated with one or more of the blocks of flowchart 900 may be divided into additional blocks, combined into fewer blocks, or eliminated, depending on the specific implementation.

[0141] At block 902, a global explainability metric associated with each graph in the first set of graphs (e.g., the first set of graphs 302A...302N) may be determined. In an embodiment, the processor 204 may be configured to determine a global explainability metric associated with each graph in the first set of graphs 302A...302N. The global explainability metric may be determined to mitigate confirmation bias to which explainability information associated with the first set of graphs 302A...302N may be subject. The global explainability metric may be determined based on a cosine similarity metric. The processor 204 may be configured to determine a cosine similarity between each pair of feature vectors corresponding to prediction-driven graph elements in a set of prediction-driven graph elements associated with each graph in the first set of graphs 302A...302N. Thus, cosine similarities may be determined between a feature vector in the first set of feature vectors, a feature vector in the second set of feature vectors, and between each feature vector in the first set of feature vectors and another feature vector in the second set of feature vectors.

[0142] At block 904, a group of similar graphs of the first set of graphs 302A...302N may be generated based on the global explainability metric. In one embodiment, the processor 204 may be configured to generate a group of similar graphs of the first set of graphs 302A...302N based on the determined global explainability metric. Graph elements (or prediction non-driving graph elements) that may not be contributors to predictions for the first set of graphs 302A...302N based on the global explainability metric may be classified into a first group. Feature vectors corresponding to the graph elements classified into the first group may be determined to be similar based on cosine similarity between the feature vectors. Similarly, graph elements (or prediction driving graph elements) that contribute to predictions for the first set of graphs 302A...302N may be classified into a second group. Feature vectors corresponding to the graph elements classified into the second group may be determined to be similar based on cosine similarity between the feature vectors. The processor 204 may be configured to render graph data based on the generated group of similar graphs. The processor 204 may control the user end device 104 or the display device 210A to render the graph data. The graph data may show the first group of graph elements and the second group of graph elements. Control may pass to an end.

[0143] Although flowchart 900 is depicted as discrete operations such as 902 and 904, the disclosure is not so limited. However, in certain embodiments, such discrete operations may be further divided into additional operations, combined into fewer operations, or eliminated, depending on the particular implementation, without detracting from the essence of the disclosed embodiments.

[0144] 10A and 10B collectively illustrate an example scenario for rendering explainability information associated with a set of graphs, according to at least one embodiment described in this disclosure. FIGS. 10A and 10B are described in conjunction with elements from FIGS. 1, 2, 3, 4, 5A, 5B, 6, 7, 8, and 9. With reference to FIGS. 10A and 10B , an example scenario 1000 is shown. The example scenario 1000 may include an electronic device 1002, which may be an example implementation of the electronic device 102 of FIG. 1. The processor 204 may render an electronic UI 1004 on a display (e.g., display device 210A) of the electronic device 1002. The electronic UI 1004 may include four UI elements: a first UI element 1006, a second UI element 1008, a third UI element 1010, and a fourth UI element 1012.

[0145] As shown in FIG. 10A , at time instance T−1, the processor 204 may control the electronic UI 1004 to render explainability information associated with the set of graphs. The rendered explainability information may include graph data of a graph dataset, which may include the set of graphs. A first UI element 1006 may correspond to the graph data. An X-axis of the graph data may represent a graph element of the graph dataset, and a Y-axis of the graph data may represent a contribution of the graph element of the graph to a prediction for the set of graphs of the graph dataset. The contribution may be represented using an explanation score associated with the graph element of the graph dataset. Feature vectors of the graph elements of the set of graphs may be grouped based on similarity between the feature vectors. The similarity may be determined based on a global explainability metric. Determining the global explainability metric is further described, for example, in FIG. 9 (902).

[0146] The processor 204 may receive a first user input via the second UI element 1008. The second UI element 1008 may allow a user to select a set of graph elements of the graph dataset associated with possible explanation scores within a certain range. Based on the received first user input, the processor 204 may control the position of the third UI element 1010 across the graph data. For example, the received first user input may indicate a selection of a set of graph elements that may have a very small contribution to predictions for a set of graphs in the graph dataset. The set of graph elements may be associated with a lower explanation score. Based on the user input, the processor 204 may control the position of the third UI element 1010 to the left edge of the x-axis of the graph data, as shown in FIG. 10A . The processor 204 may further control the fourth UI element 1012 to render annotations for the set of graph elements (e.g., nodes) based on the selection indicated in the received first user input. A fourth UI element 1012 may show the annotated nodes in each graph of the set of graphs in the graph dataset.

[0147] As shown in FIG. 10B , at time instance T−2 after T−1, processor 204 may receive a second user input via second UI element 1008. Based on the received second user input, processor 204 may control a position of third UI element 1010 across the graph data. For example, the received second user input may indicate a selection of a set of graph elements that may have a significant contribution to predictions for a set of graphs in the graph dataset. The set of graph elements may be associated with a higher explanation score. Based on the user input, processor 204 may adjust the position of third UI element 1010 to the right edge of the x-axis of the graph data, as shown in FIG. 10B . Processor 204 may further control fourth UI element 1012 to render annotations for the set of graph elements (e.g., nodes) based on the selection indicated in the received second user input.

[0148] It should be noted that the scenario 1000 of FIGS. 10A and 10B is for illustrative purposes and should not be construed as limiting the scope of the present disclosure.

[0149] 11A and 11B collectively illustrate a flowchart of an exemplary method for global interpretation and visualization of graph explainability results, in accordance with certain embodiments of the present disclosure. FIGS. 11A and 11B are described in conjunction with elements from FIGS. 1, 2, 3, 4, 5A, 5B, 6, 7, 8, 9, 10A, and 10B. With reference to FIGS. 11A and 11B , a flowchart 1100 is shown. The method illustrated in flowchart 1100 may begin at 1102 and may be performed by any suitable system, apparatus, or device, such as the exemplary electronic device 102 of FIG. 1 or the processor 204 of FIG. 2. While illustrated with discrete blocks, steps and operations associated with one or more of the blocks of flowchart 1100 may be divided into additional blocks, combined into fewer blocks, or eliminated, depending on the particular implementation.

[0150] At block 1102, an electronic user interface (UI) may be displayed that includes one or more UI elements corresponding to graph data, including explainability information associated with a plurality of graphs. In one embodiment, the processor 204 may be configured to display an electronic UI that includes one or more UI elements corresponding to graph data, including explainability information associated with a plurality of graphs. For example, the one or more UI elements may include a first set of UI elements through which the processor 204 may receive user input that may indicate a selection of a graph dataset. The one or more UI elements may further include a second set of UI elements that may render graph explainability information associated with graphs that may be included in the selected graph dataset.

[0151] At block 1104, a first user input indicating a user selection of a first set of graphs (e.g., first set of graphs 114A...114N) from the plurality of graphs may be received via the displayed electronic UI. In an embodiment, the processor 204 may be configured to receive the first user input, which may indicate a user selection of the first set of graphs 114A...114N from the plurality of graphs, via the displayed electronic UI. For example, the processor 204 may control the displayed electronic UI to render three graph datasets. The electronic UI may receive the user input, indicating a selection of a graph dataset (out of the three datasets), via a UI element rendered on the displayed electronic UI. Details of receiving the user input indicating a user selection of a set of graphs are further provided, for example, in FIG. 12 .

[0152] At block 1106, a first dataset (e.g., first dataset 114) including a first set of graphs (e.g., first set of graphs 114A...114N) may be received. In an embodiment, the processor 204 may be configured to receive the first dataset 114 including the first set of graphs 114A...114N. The first dataset 114 may be received in response to a user input indicating a user selection of the first set of graphs 114A...114N (i.e., first dataset 114).

[0153] At block 1108, a graph explainer model (e.g., graph explainer model 110) may be applied to the received first data set 114 to determine a set of explanation scores. In an embodiment, the processor 204 may be configured to apply the graph explainer model 110 to the received first data set 114 to determine a set of explanation scores. Details of the application of the graph explainer model 110 to determine a set of explanation scores are further provided in, for example, FIGS. 1, 3, and 4.

[0154] At block 1110, the graph elements of each graph in the first set of graphs 114A...114N may be filtered based on the determined set of explanatory scores to determine the first set of graph elements. In an embodiment, the processor 204 may be configured to filter the graph elements of each graph in the first set of graphs 114A...114N based on the determined set of explanatory scores to determine the first set of graph elements. Details of the filtering of the first set of graphs 114A...114N are further provided in, for example, Figures 1, 3, and 4.

[0155] At block 1112, a second data set (e.g., second data set 116), which may include a set of randomized graphs, may be generated based on the first set of graphs 114A...114N. In an embodiment, the processor 204 may be configured to generate the second data set 116, which may include a set of randomized graphs, based on the first set of graphs 114A...114N. Details of the generation of the set of randomized graphs are further provided in, for example, Figures 1, 3, 5A, and 5B.

[0156] At block 1114, the graph explainer model 110 may be applied to the generated second data set 116 to determine a set of noise scores. In an embodiment, the processor 204 may be configured to apply the graph explainer model 110 to the generated second data set 116 to determine a set of noise scores. Details of determining the set of noise scores are provided further in, for example, FIGS. 1, 3, and 6.

[0157] At block 1116, the determined set of explanation scores may be denoised to determine a set of denoised explanation scores based on the determined set of noise scores. In an embodiment, the processor 204 may be configured to denoise the determined set of explanation scores to determine a set of denoised explanation scores based on the determined set of noise scores. Details of denoising the set of explanation scores are further provided in, for example, Figures 1, 3, and 6.

[0158] At block 1118, the graph elements of each graph in the first set of graphs 114A...114N may be filtered to determine a second set of graph elements based on the determined set of denoised explanation scores. In an embodiment, the processor 204 may be configured to filter the graph elements of each graph in the first set of graphs 114A...114N based on the determined set of denoised explanation scores to determine the second set of graph elements. Details of determining the second set of graph elements are further provided in, for example, Figures 1, 3, 5A, and 5B.

[0159] At block 1120, a first set of feature vectors may be determined for the determined first set of graph elements. In an embodiment, the processor 204 may be configured to determine the first set of feature vectors for the determined first set of graph elements. Details of determining the first set of feature vectors are further provided in, for example, Figures 1, 3, and 7.

[0160] At block 1122, a second set of feature vectors may be determined for the determined second set of graph elements. In an embodiment, the processor 204 may be configured to determine the second set of feature vectors for the determined second set of graph elements. Details of determining the second set of feature vectors are further provided in, for example, Figures 1, 3, and 7.

[0161] At block 1124, a set of prediction-driven graph elements associated with the first set of graphs 114A...114N may be determined based on the first set of feature vectors and the second set of feature vectors. In an embodiment, the processor 204 may be configured to determine the set of prediction-driven graph elements associated with the first set of graphs 114A...114N based on the first set of feature vectors and the second set of feature vectors. The explainability information associated with the first set of graphs 114A...114N may correspond to the determined set of prediction-driven graph elements. Details of determining the set of prediction-driven graph elements are further provided in, for example, Figures 1, 3, and 7. Control may pass to an end.

[0162] Although flowchart 1100 is shown as discrete operations such as 1102, 1104, 1106, 1108, 1110, 1112, 1114, 1116, 1118, 1120, 1122, and 1124, the disclosure is not so limited. However, in certain embodiments, such discrete operations may be further divided into additional operations, combined into fewer operations, or eliminated, depending on the particular implementation, without detracting from the essence of the disclosed embodiments.

[0163] FIG. 12 illustrates an example scenario for rendering explainability information associated with graph elements of a graph dataset, according to at least one embodiment described in this disclosure.

[0034] Referring to FIG. 12, an example scenario 1200 is shown. The example scenario 1200 may include an electronic device 1202, which may be an example implementation of the electronic device 102 of FIG. 1. The processor 204 may render an electronic UI 1204 on a display (e.g., display device 210A) of the electronic device 1202. The electronic UI 1204 may include multiple UI elements, i.e., UI elements 1206, 1208, and 1210, that enable selection of a graph dataset. The electronic UI 1204 may further include a UI element 1212 corresponding to graph data including explainability information associated with multiple graphs. The multiple graphs may include three graph datasets: graph dataset-1 (GD1), graph dataset-2 (GD2), and graph dataset-3 (GD3).

[0164] According to an embodiment, at time instance T-1, processor 204 may receive user input via UI element 1206. The user input may indicate a user selection of a first set of graphs (e.g., first set of graphs 114A...114N) from the plurality of graphs. The first set of graphs 114A...114N may be included in GD1. Processor 204 may be configured to determine a set of prediction-driven graph elements in each graph of the selected first set of graphs 114A...114N. According to an embodiment, at time instance T-2, later than T-1, processor 204 may control electronic UI 1212 to render the determined set of prediction-driven graph elements associated with first set of graphs 114A...114N as explainability information. UI element 1212 may include annotations for the determined set of prediction-driven graph elements (e.g., nodes) in the selected first set of graphs 114A...114N.

[0165] It should be noted that the scenario 1200 of FIG. 12 is for illustrative purposes and should not be construed as limiting the scope of the present disclosure.

[0166] FIG. 13 illustrates a flowchart of an exemplary method for global interpretation of graph learning explainability based on cumulative denoising and clustering search, according to an embodiment of the present disclosure. FIG. 13 is described in conjunction with elements from FIGS. 1, 2, 3, 4, 5A, 5B, 6, 7, 8, 9, 10A, 10B, 11A, 11B, and 12. Referring to FIG. 13, a flowchart 1300 is shown. The method illustrated in flowchart 1300 may begin at 1302 and may be performed by any suitable system, apparatus, or device, such as the exemplary electronic device 102 of FIG. 1 or the processor 204 of FIG. 2. Although illustrated with discrete blocks, steps and operations associated with one or more of the blocks of flowchart 1300 may be divided into additional blocks, combined into fewer blocks, or eliminated, depending on the specific implementation.

[0167] At block 1302, a first dataset (e.g., first dataset 114) including a first set of graphs (e.g., first set of graphs 114A...114N) may be received. In an embodiment, the processor 204 may be configured to receive the first dataset 114 including the first set of graphs 114A...114N. The first dataset 114 may be received in response to a user input indicating a user selection of the first set of graphs 114A...114N (i.e., first dataset 114) from the plurality of graphs.

[0168] At block 1304, a graph explainer model (e.g., graph explainer model 110) may be applied to the received first data set 114 to determine a set of explanation scores. In an embodiment, the processor 204 may be configured to apply the graph explainer model 110 to the received first data set 114 to determine a set of explanation scores. Details of the application of the graph explainer model 110 to determine a set of explanation scores are further provided in, for example, FIGS. 1, 3, and 4.

[0169] At block 1306, the graph elements of each graph in the first set of graphs 114A...114N may be filtered based on the determined set of explanatory scores to determine the first set of graph elements. In an embodiment, the processor 204 may be configured to filter the graph elements of each graph in the first set of graphs 114A...114N based on the determined set of explanatory scores to determine the first set of graph elements. Details of filtering the first set of graphs 114A...114N based on the determined set of explanatory scores are further provided in, for example, FIGS. 1, 3, and 4.

[0170] At block 1308, a second data set (e.g., second data set 116), which may include a set of randomized graphs, may be generated based on the first set of graphs 114A...114N. In an embodiment, the processor 204 may be configured to generate the second data set 116, which may include a set of randomized graphs, based on the first set of graphs 114A...114N. Details of the generation of the set of randomized graphs are further provided in, for example, Figures 1, 3, 5A, and 5B.

[0171] At block 1310, the graph explainer model 110 may be applied to the generated second data set 116 to determine a set of noise scores. In an embodiment, the processor 204 may be configured to apply the graph explainer model 110 to the generated second data set 116 to determine a set of noise scores. Details of determining the set of noise scores are provided further in, for example, FIGS. 1, 3, and 6.

[0172] At block 1312, the determined set of explanation scores may be denoised to determine a set of denoised explanation scores based on the determined set of noise scores. In an embodiment, the processor 204 may be configured to denoise the determined set of explanation scores to determine a set of denoised explanation scores based on the determined set of noise scores. Details of denoising the set of explanation scores are further provided in, for example, Figures 1, 3, and 6.

[0173] At block 1314, the graph elements of each graph in the first set of graphs 114A...114N may be filtered based on the determined set of denoised explanation scores to determine a second set of graph elements. In an embodiment, the processor 204 may be configured to filter the graph elements of each graph in the first set of graphs 114A...114N based on the determined set of denoised explanation scores to determine the second set of graph elements. Details of determining the second set of graph elements are further provided in, for example, Figures 1, 3, 5A, and 5B.

[0174] At block 1316, a first set of feature vectors may be determined for the determined first set of graph elements. In an embodiment, the processor 204 may be configured to determine the first set of feature vectors for the determined first set of graph elements. Details of determining the first set of feature vectors are further provided in, for example, Figures 1, 3, and 7.

[0175] At block 1318, a second set of feature vectors may be determined for the determined second set of graph elements. In an embodiment, the processor 204 may be configured to determine the second set of feature vectors for the determined second set of graph elements. Details of determining the second set of feature vectors are further provided in, for example, Figures 1, 3, and 7.

[0176] At block 1320, a set of predicted-driven graph elements associated with the first set of graphs 114A...114N may be determined based on the first set of feature vectors and the second set of feature vectors. In an embodiment, the processor 204 may be configured to determine the set of predicted-driven graph elements associated with the first set of graphs 114A...114N based on the first set of feature vectors and the second set of feature vectors. Details of determining the set of predicted-driven graph elements are further provided in, for example, Figures 1, 3, and 7.

[0177] At block 1322, graph data including explainability information associated with the first set of graphs 114A...114N may be rendered based on the determined set of prediction-driven graph elements. In an embodiment, the processor 204 may be configured to render graph data including explainability information associated with the first set of graphs 114A...114N based on the determined set of prediction-driven graph elements. Details of rendering the explainability information associated with the first set of graphs 114A...114N are further provided in, for example, Figures 1, 3, 8, 9, 10A, 10B, 11A, 11B, and 12. Control may pass to an end.

[0178] Although flowchart 1300 is shown as discrete operations such as 1302, 1304, 1306, 1308, 1310, 1312, 1314, 1316, 1318, 1320, and 1322, the disclosure is not so limited. However, in certain embodiments, such discrete operations may be further divided into additional operations, combined into fewer operations, or eliminated, depending on the particular implementation, without detracting from the essence of the disclosed embodiments.

[0179] Various embodiments of the present disclosure may provide one or more non-transitory computer-readable storage media configured to store instructions that, in response to being executed, cause a system (such as the exemplary electronic device 102) to perform operations. The operations may include receiving a first dataset (e.g., the first dataset 114) including a first set of graphs (e.g., the first set of graphs 114A...114N). The operations may further include applying a graph explainer model (e.g., the graph explainer model 110) to the received first dataset 114 to determine a set of explanation scores. The operations may further include filtering graph elements of each graph in the first set of graphs 114A...114N based on the determined set of explanation scores to determine a first set of graph elements. The operations may further include generating a second dataset 116 including a set of randomized graphs based on the first set of graphs 114A...114N. The operations may further include applying the graph explainer model 110 to the generated second data set 116 to determine a set of noise scores. The operations may further include denoising the determined set of explanation scores based on the determined set of noise scores to determine a set of denoised explanation scores. The operations may further include filtering the graph elements of each graph in the first set of graphs 114A...114N based on the determined set of denoised explanation scores to determine a second set of graph elements. The operations may further include determining a first set of feature vectors for the determined first set of graph elements. The operations may further include determining a second set of feature vectors for the determined second set of graph elements. The operations may further include determining a set of predicted driving graph elements associated with the first set of graphs 114A...114N based on the first set of feature vectors and the second set of feature vectors.The operations may further include rendering graph data including explainability information associated with the first set of graphs 114A...114N based on the determined set of predictive driving graph elements.

[0180] As used in this disclosure, the term "module" or "component" can refer to a specific hardware implementation configured to perform the actions of the module or component, and / or to a software object or software routine that may be stored on and / or executed on general-purpose hardware (e.g., computer-readable media, processing device, etc.) of a computing system. In some embodiments, different components, modules, engines, and services described in this disclosure may be implemented as objects or processes (e.g., as separate threads) running on a computing system. While some of the systems and methods described in this disclosure are generally described as being implemented in software (stored on and / or executed on general-purpose hardware), specific hardware implementations or combinations of software and specific hardware implementations are also possible and contemplated. In this description, a "computing entity" can be any computing system, as defined earlier in this disclosure, or any combination of modules or sub-modules running on a computing system.

[0181] The terms used in this disclosure, and particularly in the appended claims (e.g., the body of the appended claims), are generally intended as "open" terms (e.g., the term "including" should be interpreted as "including, but not limited to," the term "having" should be interpreted as "having at least," the term "including" should be interpreted as "including, but not limited to," etc.).

[0182] Furthermore, where a specific number of introduced claim recitations is intended, such intention will be expressly recited in the claim; absent such recitation, no such intention exists. For example, as an aid to understanding, the following appended claims may include the use of the introductory phrases "at least one" and "one or more" to introduce claim recitations. However, the use of such phrases should not be construed as implying that introducing a claim recitation with the indefinite article "a" or "an" limits any particular claim containing such introduced claim recitation to embodiments containing only one such recitation. This is true even if the same claim includes the introductory phrases "one or more" or "at least one" and an indefinite article such as "a" or "an" (e.g., "a" and / or "an" should be construed to mean "at least one" or "one or more"). The same applies to the use of definite articles used to introduce claim recitations.

[0183] Additionally, even when a particular number of introduced claim recitations is explicitly recited, those skilled in the art will recognize that such recitation should be interpreted to mean at least the recited number (e.g., the recitation "two recitations" without other modifiers means at least two recitations, or more than two recitations). Furthermore, when idiomatic expressions similar to "such as at least one of A, B, and C" or "such as one or more of A, B, and C" are used, such constructions are generally intended to include A only, B only, C only, A and B, A and C, B and C, or A, B, and C, etc.

[0184] Furthermore, any disjunctive phrase presenting two or more alternative terms, whether in the present text, claims, or drawings, should be understood to contemplate the possibility of including one of those terms, either one of those terms, or both terms. For example, the phrase "A or B" should be understood to include the possibilities of "A" or "B" or "A and B."

[0185] All examples and conditional language set forth in this disclosure are intended for educational purposes to aid the reader in understanding the disclosure and the concepts contributed by the inventor to further the art, and should be construed without limitation to such specifically set forth examples and conditions. Although embodiments of the present disclosure have been described in detail, various changes, substitutions, and alterations can be made thereto without departing from the spirit and scope of the present disclosure.

[0186] The following additional notes are provided regarding the embodiments including the above examples. (Appendix 1) 1. A processor-implemented method comprising: receiving a first dataset including a first set of graphs; applying a graph explainer model to the received first dataset to determine a set of explanation scores; filtering the graph elements of each graph of the first set of graphs based on the determined set of explanatory scores to determine a first set of graph elements; generating a second data set comprising a set of randomized graphs based on the first set of graphs; applying the graph interpreter model to the generated second data set to determine a set of noise scores; denoising the determined set of explanation scores based on the determined set of noise scores to determine a set of denoised explanation scores; filtering the graph elements of each graph of the first set of graphs based on the determined set of denoised explanatory scores to determine a second set of graph elements; determining a first set of feature vectors for the determined first set of graph elements; determining a second set of feature vectors for a second set of determined graph elements; determining a set of prediction-driven graph elements associated with the first set of graphs based on the first set of feature vectors and the second set of feature vectors; and rendering graph data including explainability information associated with the first set of graphs based on the determined set of prediction-driven graph elements. method. (Appendix 2) selecting a first graph from the first set of graphs; performing a first set of operations on a selected first graph to determine a first subset of explanation scores and a first subset of graph elements of the first set of graph elements belonging to the selected first graph, the first set of operations comprising: determining a predetermined number of subsets of initial explanation scores based on application of the graph explainer model to predictions generated for the selected first graph; each initial explanation score in each subset of initial explanation scores among the predetermined number of subsets of initial explanation scores is associated with a graph element of the selected first graph; normalizing the initial explanation scores for each subset of the initial explanation scores of the predetermined number of subsets of initial explanation scores; performing an accumulation of normalized initial explanation scores associated with each graph element in the predetermined number of subsets of initial explanation scores to obtain the first subset of explanation scores of the set of explanation scores; sorting the explanation scores in the first subset of explanation scores; determining the slope associated with the sorted explanatory scores; and identifying an inflection point associated with said sorted explanation scores where a determined slope exceeds a positive threshold slope; identifying graph elements that exceed the identified inflection point as the first subset of graph elements of the first set of graph elements; determining whether the first set of graphs includes at least one unprocessed graph; reselecting a second graph from the first set of graphs as the selected first graph based on the first set of graphs including the at least one unprocessed graph. and, including, stages and; obtaining the set of explanation scores and the first set of graph elements based on sequential, iterative execution of the first set of operations until the first set of graphs does not contain any unprocessed graphs. The method described in Appendix 1. (Appendix 3) 2. The method of claim 1, wherein the first set of graph elements includes graph elements that drive predictions regarding the first set of graphs. (Appendix 4) determining importance of a set of graph elements in each graph in the first dataset based on a dimensionality reduction model and a similarity analysis model; identifying a combination of graph elements from said set of graph elements based on the determined importance; storing the identified combinations in a decision tree based on the determined importance and the dependency of each graph element in each of the identified combinations; selecting a first combination of graph elements from the decision tree; performing a second set of operations on the selected first combination of graph elements, said second set of operations comprising: generating an artificial data set by randomizing the graphical elements in the selected first combination of graphical elements, wherein the received second data set corresponds to the generated artificial data set; determining a distribution of explanation scores for the generated artificial dataset; determining whether the distribution of the determined explanatory scores is an even distribution; determining the first combination of graph elements as predicted non-driven graph elements based on the determined distribution of explanation scores being the even distribution; removing a set of branches from the decision tree based on the determined distribution of explanation scores not being the even distribution; determining whether the decision tree includes at least one unprocessed combination of graph elements; reselecting a second combination of graph elements from the decision tree as the selected first combination based on the decision tree including the at least one unprocessed combination. and, including, stages and; determining identified combinations of graph elements as predicted non-driven graph elements based on iterative execution of the second set of operations until the decision tree contains no unprocessed combinations. The method described in Appendix 1. (Appendix 5) 5. The method of claim 4, wherein the dimensionality reduction model corresponds to a principal component analysis (PCA) model. (Appendix 6) 5. The method of claim 4, wherein the determined combination of graph elements associated with an even explanation score distribution corresponds to a predicted non-driving graph element. (Appendix 7) determining a set of baseline noise scores based on application of the graph explainer model to identified combinations of graph elements determined as predicted non-driving graph elements; wherein said set of baseline noise scores corresponds to said set of noise scores; removing noise components from the set of explanatory scores associated with the received first data set based on the determined set of baseline noise scores to determine a set of denoised scores; normalizing and averaging the determined set of denoised scores, wherein the set of denoised explanation scores is determined further based on the normalisation and averaging of the determined set of denoised scores. The method described in Appendix 4. (Appendix 8) 2. The method of claim 1, wherein one or more feature vectors of the first set of determined feature vectors and the second set of determined feature vectors correspond to the set of predicted driven graph elements. (Appendix 9) 2. The method of claim 1, wherein the set of prediction-driven graph elements is determined via unsupervised machine learning on the determined first set of feature vectors and the determined second set of feature vectors. (Appendix 10) determining a set of feature vector clusters comprising feature vectors of the first set of feature vectors and the second set of feature vectors; determining a set of vector components common to a first feature vector cluster of the set of feature vector clusters; the set of predicted driven graph elements is determined further based on the determined set of vector components. The method described in Appendix 1. (Appendix 11) annotating each prediction-driven graph element of the determined set of prediction-driven elements associated with each graph of the first set of graphs; the explainability information corresponds to the annotations associated with the determined set of prediction-driven graph elements. The method described in Appendix 1. (Appendix 12) determining a global explainability metric associated with each graph in the first set of graphs; generating groups of similar graphs from the first set of graphs based on the determined global explainability metric; The graph data is rendered further based on a group of generated similar graphs; The method described in Appendix 1. (Appendix 13) determining a cosine similarity metric associated with each pair of feature vectors corresponding to predicted-driven graph elements of the determined set of predicted-driven graphs associated with each graph of the first set of graphs; determining the global explainability metric based on a determined cosine similarity metric; 12. The method described in Appendix 12. (Appendix 14) 2. The method of claim 1, wherein the received first dataset corresponds to at least one of a financial domain dataset, an e-commerce domain dataset, a social network domain dataset, a molecular chemistry domain dataset, or a genome sequence domain dataset. (Appendix 15) One or more non-transitory computer-readable storage media configured to store instructions that, in response to being executed, cause an electronic device to perform operations, the operations including: receiving a first dataset including a first set of graphs; applying a graph explainer model to the received first dataset to determine a set of explanation scores; filtering the graph elements of each graph of the first set of graphs based on the determined set of explanatory scores to determine a first set of graph elements; generating a second data set comprising a set of randomized graphs based on the first set of graphs; applying the graph interpreter model to the generated second data set to determine a set of noise scores; denoising the determined set of explanation scores based on the determined set of noise scores to determine a set of denoised explanation scores; filtering the graph elements of each graph of the first set of graphs based on the determined set of denoised explanatory scores to determine a second set of graph elements; determining a first set of feature vectors for the determined first set of graph elements; determining a second set of feature vectors for a second set of determined graph elements; determining a set of prediction-driven graph elements associated with the first set of graphs based on the first set of feature vectors and the second set of feature vectors; and rendering graph data including explainability information associated with the first set of graphs based on the determined set of prediction-driven graph elements. One or more non-transitory computer-readable storage media. (Appendix 16) The action further comprises: selecting a first graph from the first set of graphs; performing a first set of operations on a selected first graph to determine a first subset of explanation scores and a first subset of graph elements of the first set of graph elements belonging to the selected first graph, the first set of operations comprising: determining a predetermined number of subsets of initial explanation scores based on application of the graph explainer model to predictions generated for the selected first graph; each initial explanation score in each subset of initial explanation scores among the predetermined number of subsets of initial explanation scores is associated with a graph element of the selected first graph; normalizing the initial explanation scores for each subset of the initial explanation scores of the predetermined number of subsets of initial explanation scores; performing an accumulation of normalized initial explanation scores associated with each graph element in the predetermined number of subsets of initial explanation scores to obtain the first subset of explanation scores of the set of explanation scores; sorting the explanation scores in the first subset of explanation scores; determining the slope associated with the sorted explanatory scores; and identifying an inflection point associated with said sorted explanation scores where a determined slope exceeds a positive threshold slope; identifying graph elements that exceed the identified inflection point as the first subset of graph elements of the first set of graph elements; determining whether the first set of graphs includes at least one unprocessed graph; reselecting a second graph from the first set of graphs as the selected first graph based on the first set of graphs including the at least one unprocessed graph. and, including, stages and; obtaining the set of explanation scores and the first set of graph elements based on sequential, iterative execution of the first set of operations until the first set of graphs does not contain any unprocessed graphs. 16. One or more non-transitory computer-readable storage media as described in Clause 15. (Appendix 17) The action further comprises: determining importance of a set of graph elements in each graph in the first dataset based on a dimensionality reduction model and a similarity analysis model; identifying a combination of graph elements from said set of graph elements based on the determined importance; storing the identified combinations in a decision tree based on the determined importance and the dependency of each graph element in each of the identified combinations; selecting a first combination of graph elements from the decision tree; performing a second set of operations on the selected first combination of graph elements, said second set of operations comprising: generating an artificial data set by randomizing the graphical elements in the selected first combination of graphical elements, wherein the received second data set corresponds to the generated artificial data set; determining a distribution of explanation scores for the generated artificial dataset; determining whether the distribution of the determined explanatory scores is an even distribution; determining the first combination of graph elements as predicted non-driven graph elements based on the determined distribution of explanation scores being the even distribution; removing a set of branches from the decision tree based on the determined distribution of explanation scores not being the even distribution; determining whether the decision tree includes at least one unprocessed combination of graph elements; reselecting a second combination of graph elements from the decision tree as the selected first combination based on the decision tree including the at least one unprocessed combination. and, including, stages and; determining identified combinations of graph elements as predicted non-driven graph elements based on iterative execution of the second set of operations until the decision tree contains no unprocessed combinations. 16. One or more non-transitory computer-readable storage media as described in Clause 15. (Appendix 18) 18. The one or more non-transitory computer-readable storage media of claim 17, wherein the determined combination of graph elements associated with an even explanation score distribution corresponds to a predictive non-driving graph element. (Appendix 19) The action further comprises: determining a set of baseline noise scores based on application of the graph explainer model to identified combinations of graph elements determined as predicted non-driving graph elements; wherein said set of baseline noise scores corresponds to said set of noise scores; removing noise components from the set of explanatory scores associated with the received first data set based on the determined set of baseline noise scores to determine a set of denoised scores; normalizing and averaging the determined set of denoised scores, wherein the set of denoised explanation scores is determined further based on the normalisation and averaging of the determined set of denoised scores. 18. One or more non-transitory computer-readable storage media as described in Clause 17. (Appendix 20) a memory configured to store instructions; a processor, coupled to the memory, configured to execute the instructions to perform a process; 1. An electronic device comprising: displaying an electronic user interface (UI) including one or more UI elements corresponding to graph data including explainability information associated with a plurality of graphs; receiving a first user input via a displayed electronic UI indicating a user selection of a first set of graphs from the plurality of graphs; receiving a first dataset comprising said first set of graphs; applying a graph explainer model to the received first dataset to determine a set of explanation scores; filtering the graph elements of each graph of the first set of graphs based on the determined set of explanatory scores to determine a first set of graph elements; generating a second data set comprising a set of randomized graphs based on the first set of graphs; applying the graph interpreter model to the generated second data set to determine a set of noise scores; denoising the determined set of explanation scores based on the determined set of noise scores to determine a set of denoised explanation scores; filtering the graph elements of each graph of the first set of graphs based on the determined set of denoised explanatory scores to determine a second set of graph elements; determining a first set of feature vectors for the determined first set of graph elements; determining a second set of feature vectors for a second set of determined graph elements; determining a set of predicted driven graph elements associated with the first set of graphs based on the first set of feature vectors and the second set of feature vectors; the explainability information corresponds to a determined set of prediction-driven graph elements. Electronic devices. [Explanation of symbols]

[0187] 102 Electronic Devices 104 User End Devices 110 Graph Explainer Model 112 Communication Network 114 First Data Set 114A...114N First set of graphs 116 Second Data Set 202 System 204 processors 206 memory 208 Persistent Data Storage 210 I / O devices 210A Display Device 212 Network Interface 110 Graph Explainer Model 302 First dataset received 302A…302N First set of graphs 304 Determine the set of explanatory scores 306 Determine the first set of graph elements 306A First set of graph elements 308 Generate a second dataset 308A Randomized set of graphs 310 Determine a set of noise scores 312 Determine the set of denoised scores 314 Determine the second set of graph elements 314A Second set of graph elements 316 Determine the set of prediction-driven graph elements Rendering 318 graph data and explainability information 402 Select the first graph from the first set of graphs 404. Perform a first set of operations on the selected first graph to determine a first subset of explanation scores and a first subset of graph elements of the first set of graph elements belonging to the selected first graph. 404A. Determine a predetermined number of subsets of initial explanation scores based on application of the graph explainer model to predictions generated for the selected first graph, wherein each initial explanation score in each subset of initial explanation scores among the predetermined number of subsets of initial explanation scores is associated with a graph element of the selected first graph. 404B Normalizing the initial explanation scores for each subset of the initial explanation scores for a predetermined number of subsets of the initial explanation scores. 404C performs an accumulation of the normalized initial explanation scores associated with each graph element in the predetermined number of subsets of the initial explanation scores to obtain a first subset of explanation scores of the set of explanation scores. 404D Sort the explanation scores within the first subset of explanation scores 404E Determine the slope associated with the sorted explanation scores 404F Identify inflection points associated with sorted explanatory scores where the determined slope exceeds a positive threshold slope. 404G Identifying the graph elements beyond the identified inflection point as a first subset of graph elements of the first set of graph elements. 404H Determine if the first set of graphs contains at least one unprocessed graph 404I Reselect a second graph from a first set of graphs as the selected first graph based on a first set of graphs that includes at least one unprocessed graph 406. Obtaining a set of explanation scores and a first set of graph elements based on sequential, iterative execution of the first set of operations until the first set of graphs does not contain any unprocessed graphs. 502 determining the importance of a set of graph elements in each graph in the first dataset based on the dimensionality reduction model and the similarity analysis model; 504 Identifying combinations of graph elements from a set of graph elements based on their determined importance 506 The identified combinations are stored in a decision tree based on the importance and dependency of each graph element in each of the identified combinations. 508 Select the first combination of graph elements from the decision tree 510 performing a second set of actions on the selected first combination of graph elements 510A. Generating an artificial data set by randomizing the graph elements in a selected first combination of the graph elements. The received second data set corresponds to the generated artificial data set. 510B Determine the distribution of explanation scores for the generated artificial dataset 510C Determine whether the determined distribution of explanatory scores is a uniform distribution 510D. Identifying a first combination of graph elements as a predicted non-driven graph element based on the determined distribution of the explanation scores being a uniform distribution. 510E Remove a set of branches from a decision tree based on the determined distribution of explanation scores not being a uniform distribution 510F Determine whether the decision tree contains at least one unprocessed combination of graph elements 510G. Reselecting a second combination of graph elements from the decision tree as the selected first combination based on the decision tree containing at least one unprocessed combination. 512. Determine the identified combination of graph elements as a predicted non-driven graph element based on sequentially and iteratively performing the second set of operations until the decision tree contains no unprocessed combinations. 602 determines a set of baseline noise scores based on application of the graph explainer model to the identified combinations of graph elements that can be determined as predicted non-driving graph elements. The set of baseline noise scores corresponds to the set of noise scores 604. Based on the determined set of baseline noise scores, remove noise components from the set of explanatory scores associated with the received first data set to determine a set of denoised scores. 606. Normalizing and averaging the determined set of denoised scores. The set of denoised explanatory scores is determined further based on normalising and averaging the determined set of denoised scores. 702 determine a set of feature vector clusters including the feature vectors of the first set of feature vectors and the feature vectors of the second set of feature vectors. 704. Determine a set of vector components common to a first feature vector cluster of the set of feature vector clusters. A set of prediction-driven graph elements is determined based on the determined set of vector components. 802 annotating each prediction-driven graph element of the determined set of prediction-driven elements associated with each graph of the first set of graphs. 804 Rendering graph data including explainability information associated with a first set of graphs 902. Determine a global explainability metric associated with each graph in the first set of graphs. The determination of the global explainability metric is based on a cosine similarity metric. 904. Generate a group of similar graphs for the first set of graphs based on the determined global explainability metric. The graph data is further rendered based on the generated group of similar graphs. 1004 Electronic User Interface 1102 displays an electronic user interface (UI) including one or more UI elements corresponding to graph data including explainability information associated with a plurality of graphs. 1104 receives a first user input via the displayed electronic UI, the first user input may indicate a user selection of a first set of graphs from the plurality of graphs. 1106 receiving a first data set including a first set of graphs 1108 Applying the graph explainer model to the received first data set to determine a set of explanation scores. 1110. Filtering the graph elements of each graph in the first set of graphs based on the determined set of explanation scores to determine a first set of graph elements. 1112. Generate a second data set based on the first set of graphs, which may include a randomized set of graphs. 1114 Applying the graph explainer model to the generated second dataset to determine a set of noise scores. 1116 Denoise the determined set of explanatory scores based on the determined set of noise scores. 1116 (Second) Determine a second set of graph elements by filtering the graph elements of each graph in the first set of graphs based on the determined set of denoised explanatory scores. 1118. Determining a first set of feature vectors for the determined first set of graph elements. 1120 determining a second set of feature vectors for the second set of determined graph elements. 1122. Determine a set of prediction-driven graph elements associated with the first set of graphs based on the first set of feature vectors and the second set of feature vectors. The explainability information corresponds to the determined set of prediction-driven graph elements. 1302 receiving a first data set including a first set of graphs; 1304 applies a graph explainer model to the received first data set to determine a set of explanation scores. 1306. Filtering the graph elements of each graph in the first set of graphs based on the determined set of explanation scores to determine a first set of graph elements. 1308 generating a second data set based on the first set of graphs, which may include a randomized set of graphs; 1310 Applying the graph explainer model to the generated second data set to determine a set of noise scores. 1312 Denoise the determined set of explanatory scores based on the determined set of noise scores. 1314. Filtering the graph elements of each graph in the first set of graphs based on the determined set of denoised explanatory scores to determine a second set of graph elements. 1316 determining a first set of feature vectors for the determined first set of graph elements; 1318 determining a second set of feature vectors for the determined second set of graph elements; 1320 determining a set of prediction-driven graph elements associated with the first set of graphs based on the first set of feature vectors and the second set of feature vectors. 1322. Rendering graph data including explainability information associated with a first set of graphs based on the determined set of prediction-driven graph elements.

Claims

1. 1. A processor-implemented method comprising: receiving a first dataset comprising a first set of graphs; applying a graph explainer model to the received first dataset to determine a set of explanation scores; filtering the graph elements of each graph of the first set of graphs based on the determined set of explanatory scores to determine a first set of graph elements; generating a second data set comprising a set of randomized graphs based on the first set of graphs; applying the graph interpreter model to the generated second data set to determine a set of noise scores; denoising the determined set of explanation scores based on the determined set of noise scores to determine a set of denoised explanation scores; filtering the graph elements of each graph of the first set of graphs based on the determined set of denoised explanatory scores to determine a second set of graph elements; determining a first set of feature vectors for the determined first set of graph elements; determining a second set of feature vectors for a second set of determined graph elements; determining a set of predicted driven graph elements associated with the first set of graphs based on the first set of feature vectors and the second set of feature vectors; and rendering graph data including explainability information associated with the first set of graphs based on the determined set of prediction-driven graph elements. method.

2. selecting a first graph from the first set of graphs; performing a first set of operations on the selected first graph to determine a first subset of explanation scores and a first subset of graph elements of the first set of graph elements belonging to the selected first graph, the first set of operations comprising: determining a predetermined number of subsets of initial explanation scores based on application of the graph explainer model to predictions generated for the selected first graph; each initial explanation score in each subset of initial explanation scores among the predetermined number of subsets of initial explanation scores is associated with a graph element of the selected first graph; normalizing the initial explanation scores for each subset of the initial explanation scores for the predetermined number of subsets of initial explanation scores; performing an accumulation of normalized initial explanation scores associated with each graph element in the predetermined number of subsets of initial explanation scores to obtain the first subset of explanation scores of the set of explanation scores; sorting the explanation scores in the first subset of explanation scores; determining a slope associated with the sorted explanatory scores; identifying an inflection point associated with the sorted explanation scores where a determined slope exceeds a positive threshold slope; identifying graph elements that exceed the identified inflection point as the first subset of graph elements of the first set of graph elements; determining whether the first set of graphs includes at least one unprocessed graph; reselecting a second graph from the first set of graphs as the selected first graph based on the first set of graphs including the at least one unprocessed graph. and obtaining the set of explanation scores and the first set of graph elements based on sequential, iterative execution of the first set of operations until the first set of graphs does not contain any unprocessed graphs. The method of claim 1.

3. The method of claim 1 , wherein the first set of graph elements includes graph elements that drive predictions for the first set of graphs.

4. determining importance of a set of graph elements in each graph in the first dataset based on a dimensionality reduction model and a similarity analysis model; identifying a combination of graph elements from said set of graph elements based on the determined importance; sorting the identified combinations into a decision tree based on the determined importance and the dependency of each graph element in each of the identified combinations; selecting a first combination of graph elements from the decision tree; performing a second set of operations on the selected first combination of graph elements, the second set of operations comprising: generating an artificial data set by randomizing the graphical elements in the selected first combination of graphical elements, wherein the received second data set corresponds to the generated artificial data set; determining a distribution of explanation scores for the generated artificial dataset; determining whether the distribution of the determined explanatory scores is an even distribution; determining the first combination of graph elements as predicted non-driven graph elements based on the determined distribution of explanation scores being the even distribution; removing a set of branches from the decision tree based on the determined distribution of explanation scores not being the even distribution; determining whether the decision tree includes at least one unprocessed combination of graph elements; reselecting a second combination of graph elements from the decision tree as the selected first combination based on the decision tree including the at least one unprocessed combination. and determining identified combinations of graph elements as predicted non-driven graph elements based on iterative execution of the second set of operations until the decision tree contains no unprocessed combinations. The method of claim 1.

5. The method of claim 4 , wherein the dimensionality reduced model corresponds to a principal component analysis (PCA) model.

6. The method of claim 4 , wherein the determined combination of graph elements associated with an even explanation score distribution corresponds to a predicted non-driving graph element.

7. determining a set of baseline noise scores based on application of the graph explainer model to identified combinations of graph elements determined as predicted non-driving graph elements; the set of baseline noise scores corresponds to the set of noise scores; removing noise components from the set of explanatory scores associated with the received first data set based on the determined set of baseline noise scores to determine a set of denoised scores; normalizing and averaging the determined set of denoised scores, wherein the set of denoised explanation scores is determined further based on the normalisation and averaging of the determined set of denoised scores. The method of claim 4.

8. The method of claim 1 , wherein one or more feature vectors of the first set of determined feature vectors and the second set of determined feature vectors correspond to the set of prediction-driven graph elements.

9. 2. The method of claim 1 , wherein the set of prediction-driven graph elements is determined via unsupervised machine learning on the determined first set of feature vectors and the determined second set of feature vectors.

10. determining a set of feature vector clusters comprising feature vectors of the first set of feature vectors and the second set of feature vectors; determining a set of vector components common to a first feature vector cluster of said set of feature vector clusters; the set of predicted driven graph elements is determined further based on the determined set of vector components. The method of claim 1.

11. annotating each prediction-driven graph element of the determined set of prediction-driven elements associated with each graph of the first set of graphs; the explainability information corresponds to the annotations associated with the determined set of prediction-driven graph elements. The method of claim 1.

12. determining a global explainability metric associated with each graph in the first set of graphs; generating groups of similar graphs from the first set of graphs based on the determined global explainability metric; the graph data is rendered further based on the generated group of similar graphs; The method of claim 1.

13. determining a cosine similarity metric associated with each pair of feature vectors corresponding to predicted-driven graph elements of the determined set of predicted-driven graphs associated with each graph of the first set of graphs; determining the global explainability metric based on a determined cosine similarity metric; The method of claim 12.

14. 10. The method of claim 1, wherein the received first dataset corresponds to at least one of a financial domain dataset, an e-commerce domain dataset, a social network domain dataset, a molecular chemistry domain dataset, or a genome sequence domain dataset.

15. One or more non-transitory computer-readable storage media configured to store instructions that, in response to being executed, cause an electronic device to perform operations, the operations including: receiving a first dataset comprising a first set of graphs; applying a graph explainer model to the received first dataset to determine a set of explanation scores; filtering the graph elements of each graph of the first set of graphs based on the determined set of explanatory scores to determine a first set of graph elements; generating a second data set comprising a set of randomized graphs based on the first set of graphs; applying the graph interpreter model to the generated second data set to determine a set of noise scores; denoising the determined set of explanation scores based on the determined set of noise scores to determine a set of denoised explanation scores; filtering the graph elements of each graph of the first set of graphs based on the determined set of denoised explanatory scores to determine a second set of graph elements; determining a first set of feature vectors for the determined first set of graph elements; determining a second set of feature vectors for a second set of determined graph elements; determining a set of predicted driven graph elements associated with the first set of graphs based on the first set of feature vectors and the second set of feature vectors; and rendering graph data including explainability information associated with the first set of graphs based on the determined set of prediction-driven graph elements. One or more non-transitory computer-readable storage media.

16. The operations further comprise: selecting a first graph from the first set of graphs; performing a first set of operations on the selected first graph to determine a first subset of explanation scores and a first subset of graph elements of the first set of graph elements belonging to the selected first graph, the first set of operations comprising: determining a predetermined number of subsets of initial explanation scores based on application of the graph explainer model to predictions generated for the selected first graph; each initial explanation score in each subset of initial explanation scores among the predetermined number of subsets of initial explanation scores is associated with a graph element of the selected first graph; normalizing the initial explanation scores for each subset of the initial explanation scores for the predetermined number of subsets of initial explanation scores; performing an accumulation of normalized initial explanation scores associated with each graph element in the predetermined number of subsets of initial explanation scores to obtain the first subset of explanation scores of the set of explanation scores; sorting the explanation scores in the first subset of explanation scores; determining a slope associated with the sorted explanatory scores; identifying an inflection point associated with the sorted explanation scores where a determined slope exceeds a positive threshold slope; identifying graph elements that exceed the identified inflection point as the first subset of graph elements of the first set of graph elements; determining whether the first set of graphs includes at least one unprocessed graph; reselecting a second graph from the first set of graphs as the selected first graph based on the first set of graphs including the at least one unprocessed graph. and obtaining the set of explanation scores and the first set of graph elements based on sequential, iterative execution of the first set of operations until the first set of graphs does not contain any unprocessed graphs.

16. One or more non-transitory computer-readable storage media according to claim 15.

17. The operations further comprise: determining importance of a set of graph elements in each graph in the first dataset based on a dimensionality reduction model and a similarity analysis model; identifying a combination of graph elements from said set of graph elements based on the determined importance; sorting the identified combinations into a decision tree based on the determined importance and the dependency of each graph element in each of the identified combinations; selecting a first combination of graph elements from the decision tree; performing a second set of operations on the selected first combination of graph elements, the second set of operations comprising: generating an artificial data set by randomizing the graphical elements in the selected first combination of graphical elements, wherein the received second data set corresponds to the generated artificial data set; determining a distribution of explanation scores for the generated artificial dataset; determining whether the distribution of the determined explanatory scores is an even distribution; determining the first combination of graph elements as predicted non-driven graph elements based on the determined distribution of explanation scores being the even distribution; removing a set of branches from the decision tree based on the determined distribution of explanation scores not being the even distribution; determining whether the decision tree includes at least one unprocessed combination of graph elements; reselecting a second combination of graph elements from the decision tree as the selected first combination based on the decision tree including the at least one unprocessed combination. and determining identified combinations of graph elements as predicted non-driven graph elements based on iterative execution of the second set of operations until the decision tree contains no unprocessed combinations.

16. One or more non-transitory computer-readable storage media according to claim 15.

18. 20. The one or more non-transitory computer-readable storage media of claim 17, wherein the determined combination of graph elements associated with an even explanation score distribution corresponds to a predictive non-driving graph element.

19. The operations further comprise: determining a set of baseline noise scores based on application of the graph explainer model to identified combinations of graph elements determined as predicted non-driving graph elements; the set of baseline noise scores corresponds to the set of noise scores; removing noise components from the set of explanatory scores associated with the received first data set based on the determined set of baseline noise scores to determine a set of denoised scores; normalizing and averaging the determined set of denoised scores, wherein the set of denoised explanation scores is determined further based on the normalisation and averaging of the determined set of denoised scores.

20. One or more non-transitory computer-readable storage media as recited in claim 17.

20. a memory configured to store instructions; a processor, coupled to the memory, configured to execute the instructions to perform a process; 1. An electronic device comprising: displaying an electronic user interface (UI) including one or more UI elements corresponding to graph data including explainability information associated with a plurality of graphs; receiving a first user input via a displayed electronic UI indicating a user selection of a first set of graphs from the plurality of graphs; receiving a first data set including the first set of graphs; applying a graph explainer model to the received first dataset to determine a set of explanation scores; filtering the graph elements of each graph of the first set of graphs based on the determined set of explanatory scores to determine a first set of graph elements; generating a second data set comprising a set of randomized graphs based on the first set of graphs; applying the graph interpreter model to the generated second data set to determine a set of noise scores; denoising the determined set of explanation scores based on the determined set of noise scores to determine a set of denoised explanation scores; filtering the graph elements of each graph of the first set of graphs based on the determined set of denoised explanation scores to determine a second set of graph elements; determining a first set of feature vectors for the determined first set of graph elements; determining a second set of feature vectors for a second set of determined graph elements; determining a set of predicted driven graph elements associated with the first set of graphs based on the first set of feature vectors and the second set of feature vectors; the explainability information corresponds to a determined set of prediction-driven graph elements. Electronic devices.

Images