Graph neural network visualization method and apparatus, and computing device cluster

Abstract

The graph neural network visualization method comprises the steps that first data of a hidden space of a first GNN and second data of the first space are acquired, and the first space is any one of a topological space, a feature space or a prediction space of the first GNN; the first node instrument panel is displayed, the first node instrument panel comprises a first view and a second view, the first view is obtained by visualizing the hidden space of the first GNN based on the first data, and the second view is obtained by visualizing the first space based on the second data. Thus, the node relation in the two spaces can be accurately compared through the visualized hidden space and the first space, the graph structure which a user needs to pay attention to is found out, and an efficient and accurate interpretable basis is provided for debugging and optimizing of GNN. In addition, analysis with finer granularity can be carried out through the hidden space, and the analysis efficiency and accuracy are improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence (AI) technology, and in particular to a graph neural network visualization method, device and computing device cluster. Background Art

[0002] Graph data has a wide range of applications in many fields, including models of atoms, molecules, proteins, etc. in physics, chemistry, and biology, as well as entity relationships in finance and business. In order to better process graph data, graph neural networks (GNNs) were introduced, and their main purpose is to convert the relationships between entities into vector representations. However, neural network models, especially GNNs, are often regarded as black box models, and their prediction results and reasons are difficult to understand. In some high-risk areas, such as credit risk control in finance, decision makers need to understand why the model makes specific decisions, so the interpretability of the model becomes the key. The training, evaluation, and application of GNNs all require interpretability, but compared with visual and textual neural network models, the interpretability research on GNNs is relatively scarce. Therefore, how to achieve the interpretability of GNNs is a technical problem that needs to be solved urgently. Summary of the invention

[0003] The present application provides a graph neural network visualization method, apparatus, computing device cluster, computer storage medium and computer product, which can provide efficient and accurate interpretable basis for GNN.

[0004] In a first aspect, the present application provides a graph neural network (GNN) visualization method, comprising: obtaining first data of a latent space of a first GNN and second data of the first space, the first space being any one of a topological space, a feature space or a prediction space of the first GNN; displaying a first node dashboard, the first node dashboard comprising: a first view and a second view, wherein the first view is obtained by visualizing the latent space of the first GNN based on the first data, and the second view is obtained by visualizing the first space based on the second data.

[0005] In this way, by visualizing the latent space and the first space, we can accurately compare the node relationships in the two spaces, find out the graph structure that users need to focus on, and provide efficient and accurate interpretable basis for GNN debugging and tuning. In addition, more fine-grained analysis can also be performed through the latent space, which improves the efficiency and accuracy of analysis.

[0006] In a possible implementation, it also includes: obtaining third data of the second space of the first GNN, the second space is any one of the topological space, feature space or prediction space of the first GNN, and the second space is different from the first space; displaying a third view in the first node dashboard, wherein the third view is obtained by visualizing the second space based on the third data. In this way, the GNN can be analyzed through other spaces, thereby providing efficient and accurate interpretable basis.

[0007] In a possible implementation, it also includes: obtaining fourth data of the third space of the first GNN, the third space is any one of the topological space, feature space or prediction space of the first GNN, and the first space and the second space are different from the third space; displaying a fourth view in the first node dashboard, wherein the fourth view is obtained by visualizing the third space based on the fourth data. In this way, the GNN can be analyzed through other spaces, thereby providing efficient and accurate interpretable basis.

[0008] In a possible implementation, after displaying the first node dashboard, the method further includes: scaling the first view or the second view in response to the acquired first operation, so that the user can view the details of the node he is interested in, thereby improving the accuracy of analysis.

[0009] In a possible implementation, after displaying the first node dashboard, it also includes: in response to the filtering operation on the nodes in the first view or the second view, displaying the second node dashboard, the second node dashboard includes a first sub-view and a second sub-view, and the first sub-view and the second sub-view are both used to display the nodes filtered from the first view or the second view, wherein the first sub-view is related to the first view, and the second sub-view is related to the second view. In this way, by iteratively filtering the nodes in the node dashboard, the node relationships in each space can be accurately compared, and the graph structure that the user needs to focus on can be found, providing efficient and accurate interpretable basis for GNN debugging and tuning.

[0010] In a possible implementation, after displaying the second node dashboard, it also includes: in response to the filtering operation on the nodes in the first subview or the second subview, displaying the third node dashboard, the third node dashboard includes the first grandchild view and the second grandchild view, the first grandchild view and the second grandchild view are both used to display the nodes filtered from the first subview or the second subview, wherein the first grandchild view is related to the first subview, and the second grandchild view is related to the second subview. In this way, by iteratively filtering the nodes in the node dashboard, the node relationships in each space can be accurately compared, and the graph structure that the user needs to focus on can be found, providing efficient and accurate interpretable basis for GNN debugging and tuning.

[0011] In a possible implementation, it further includes: when multiple node dashboards are displayed, one of the multiple node dashboards is fixed, and at least one view in an unfixed dashboard among the multiple dashboards is compared with the view in the fixed dashboard. In this way, the views in different dashboards can be compared to find out the graph structure that the user needs to focus on, providing efficient and accurate interpretable basis for GNN debugging and tuning.

[0012] In a possible implementation, after displaying the first node dashboard, it further includes: in response to the obtained benchmark selection operation, displaying the distribution statistics of the nodes in each local area in the first view in a pie chart in the first view, wherein the size of each local area is obtained through the benchmark; or, in the case where the first space is a topological space, in response to the obtained benchmark selection operation, displaying the distribution statistics of the nodes in each local area in the second view in a pie chart in the second view, wherein the size of each local area is obtained through the benchmark. In this way, the node distribution in different areas can be intuitively observed, and which types of nodes are very similar and difficult to separate can be quickly analyzed.

[0013] In a possible implementation, the method further includes: obtaining fifth data of the latent space of the second GNN; and displaying a fifth view in the first node dashboard, wherein the fifth view is obtained by visualizing the latent space of the second GNN based on the fifth data. In this way, the two GNNs can be compared and analyzed.

[0014] In a possible implementation, it also includes: obtaining a selection operation of at least one node in the latent space of the first GNN or the second GNN; in response to a viewing operation of the distance sorting relationship between the unselected nodes and the selected nodes in the latent space of the first GNN and the second GNN, visually displaying the distance sorting relationship between the unselected nodes and the selected nodes in the latent space of the first GNN and the latent space of the second GNN in a coordinate system, the horizontal axis of the coordinate system represents the distance distribution between the unselected nodes and the selected nodes in the latent space of the first GNN, and the vertical axis of the coordinate system represents the distance distribution between the unselected nodes and the selected nodes in the latent space of the second GNN. In this way, through the distance sorting between different nodes and the user-selected interest points in the distance sorting relationship diagram between the unselected nodes and the selected nodes, the similarity and difference points of the two GNNs in the latent space can be seen.

[0015] In the second aspect, the present application provides a graph neural network (GNN) visualization device, including: an acquisition module and a display module. The acquisition module is used to acquire first data of a hidden space of a first GNN and second data of a first space, and the first space is any one of a topological space, a feature space, or a prediction space of the first GNN; the display module is used to display a first node dashboard, and the first node dashboard includes: a first view and a second view, wherein the first view is obtained by visualizing the hidden space of the first GNN based on the first data, and the second view is obtained by visualizing the first space based on the second data.

[0016] In a possible implementation, the acquisition module is also used to obtain third data of the second space of the first GNN, where the second space is any one of the topological space, feature space or prediction space of the first GNN, and the second space is different from the first space; the display module is also used to display a third view in the first node dashboard, wherein the third view is obtained by visualizing the second space based on the third data.

[0017] In a possible implementation, the acquisition module is also used to obtain fourth data of the third space of the first GNN, where the third space is any one of the topological space, feature space or prediction space of the first GNN, and the first space and the second space are different from the third space; the display module is also used to display a fourth view in the first node dashboard, wherein the fourth view is obtained by visualizing the third space based on the fourth data.

[0018] In a possible implementation, after displaying the first node dashboard, the display module is further configured to: scale the first view or the second view in response to the acquired first operation.

[0019] In one possible implementation, after displaying the first node dashboard, the display module is further used to: display a second node dashboard in response to a filtering operation on the nodes in the first view or the second view, the second node dashboard including a first sub-view and a second sub-view, both of which are used to display nodes filtered from the first view or the second view, wherein the first sub-view is related to the first view, and the second sub-view is related to the second view.

[0020] In one possible implementation, after displaying the second node dashboard, the display module further includes: in response to a filtering operation on the nodes in the first sub-view or the second sub-view, displaying a third node dashboard, the third node dashboard including a first grandchild view and a second grandchild view, both of which are used to display nodes filtered from the first sub-view or the second sub-view, wherein the first grandchild view is related to the first sub-view, and the second grandchild view is related to the second sub-view.

[0021] In a possible implementation, the display module is further used to: when multiple node dashboards are displayed, fix one of the multiple node dashboards, and compare at least one view in an unfixed dashboard among the multiple dashboards with the view in the fixed dashboard.

[0022] In one possible implementation, after displaying the first node dashboard, the display module is further used to: in response to an acquired benchmark selection operation, display the distribution statistics of the nodes in each local area in the first view in a pie chart in the first view, wherein the size of each local area is obtained through the benchmark; or, when the first space is a topological space, in response to an acquired benchmark selection operation, display the distribution statistics of the nodes in each local area in the second view in a pie chart in the second view, wherein the size of each local area is obtained through the benchmark.

[0023] In a possible implementation, the acquisition module is further used to acquire fifth data of the latent space of the second GNN; the display module is further used to display a fifth view in the first node dashboard, and the fifth view is obtained by visualizing the latent space of the second GNN based on the fifth data.

[0024] In one possible implementation, the display module is also used to: obtain a selection operation on at least one node in the latent space of the first GNN or the second GNN; in response to a viewing operation on the distance sorting relationship between unselected nodes and selected nodes in the latent space of the first GNN and the second GNN, visualize the distance sorting relationship between the unselected nodes and the selected nodes in the latent space of the first GNN and the second GNN in a coordinate system, where the horizontal axis of the coordinate system represents the distance distribution between the unselected nodes and the selected nodes in the latent space of the first GNN, and the vertical axis of the coordinate system represents the distance distribution between the unselected nodes and the selected nodes in the latent space of the second GNN.

[0025] In a third aspect, the present application provides a computing device cluster, comprising at least one computing device, each computing device comprising a processor and a memory; the processor of at least one computing device is used to execute instructions stored in the memory of at least one computing device, so that the computing device cluster performs the method described in the first aspect or any possible implementation of the first aspect.

[0026] In a fourth aspect, the present application provides a computer-readable storage medium, including computer program instructions, when the computer program instructions are executed by a computing device cluster, the computing device cluster executes the method described in the first aspect or any possible implementation of the first aspect, or executes the method described in the second aspect or any possible implementation of the second aspect. The computing device cluster may include one or more computing devices.

[0027] In a fifth aspect, the present application provides a computer program product including instructions, which, when executed by a computing device cluster, enables the computing device cluster to perform the method described in the first aspect or any possible implementation of the first aspect. The computing device cluster may include one or more computing devices.

[0028] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a flowchart of a GNN visualization method provided in an embodiment of the present application;

[0030] Figure 2 is a schematic diagram of a node dashboard provided in an embodiment of the present application;

[0031] Figure 3 This is a schematic diagram of a pie chart showing the statistical distribution results of nodes in different regions provided in an embodiment of the present application;

[0032] Figure 4 It is a schematic diagram of the distribution of nodes in the latent space of two different GNNs provided in an embodiment of the present application;

[0033] Figure 5 It is a schematic diagram of the distance sorting relationship between unselected nodes and selected nodes in the latent space of two different GNNs provided in an embodiment of the present application;

[0034] Figure 6 It is a schematic diagram of a topological space, latent space and prediction space provided in an embodiment of the present application;

[0035] Figure 7 is a structural schematic diagram of a GNN visualization device provided in an embodiment of the present application;

[0036] Figure 8 is a schematic diagram of the structure of a computing device provided in an embodiment of the present application;

[0037] Fig. 9 is a schematic diagram of the structure of a computing device cluster provided in an embodiment of the present application;

[0038] Fig.10 It is a structural diagram of another computing device cluster provided in an embodiment of the present application. DETAILED DESCRIPTION

[0039] The term "and / or" in this article is a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The symbol " / " in this article indicates that the associated objects are in an or relationship, for example, A / B means A or B.

[0040] The terms "first" and "second" in the specification and claims herein are used to distinguish different objects rather than to describe a specific order of the objects. For example, a first response message and a second response message are used to distinguish different response messages rather than to describe a specific order of the response messages.

[0041] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0042] In the description of the embodiments of the present application, unless otherwise specified, "multiple" means two or more than two. For example, multiple processing units refer to two or more processing units, etc.; multiple elements refer to two or more elements, etc.

[0043] First, some technical terms involved in this application are introduced.

[0044] (1) Graph data

[0045] Graph data refers to data that can present the association relationship between each node in the form of a "graph" data structure, which can be used to represent entities and complex relationships between entities. A "graph" can be an undirected graph or a directed graph. When a "graph" is an undirected graph, the undirected graph contains a finite (possibly variable) set as a node set, and a set of unordered pairs (corresponding to an undirected graph) as an edge set. Exemplarily, graph data can be a representation form of recommendation data, which uses the form of a "graph" to present the association relationship between the recommender, the purchaser, and the recommended item. Graph data can also be social network data, which uses the form of a "graph" to present the association relationship between each registered user, for example, whether two registered users are in a relationship of mutual concern, whether two registered users have the same preferences, etc. Graph data can also be an organic chemical formula, which uses the form of a "graph" to present the association relationship between each atom, and the present disclosure is not limited to this.

[0046] (2) Node

[0047] A node is also called a vertex, which is the basic unit of a data structure such as a "graph". Each node may have node data. Node data may be a part of the data associated with the node in the graph data, or an external entity represented by an integer subscript or reference. In addition to node data, the data structure of the graph data may also contain edges, each of which may be associated with a value (edge ​​value), such as a label or a value (i.e., weight). For example, in the above-mentioned example of recommendation data, the node set in the graph data may include any one or more items representing the recommendation node or the recommendation node. The edge connecting the recommender node and the recommendation node indicates that the recommender recommended the recommendation, and the weight of the edge indicates the transaction volume or transaction amount generated by the recommender's recommendation of the recommendation, etc. In addition, the node set in the graph data may also include a buyer node, and the edge connecting the buyer and the recommendation indicates that the buyer purchased the recommendation. The edge connecting the buyer and the recommender indicates that the buyer follows the recommender.

[0048] (3) Topological space (TS)

[0049] Topological space refers to the topological structure of the graph in GNN, that is, the connection relationship between nodes or the distribution of edges in the graph. Topological space describes the adjacency relationship between nodes.

[0050] (4) Feature space (FS)

[0051] Feature space refers to the feature representation space of the nodes in the graph in GNN. Each node is assigned at least one feature vector, which contains information related to the node.

[0052] (5) Latent space (LS)

[0053] Hidden space refers to a potential, low-dimensional vector space in GNN. The representation of the node in the hidden space is mapped to a lower-dimensional vector for encoding.

[0054] (6) Prediction space (PS)

[0055] The prediction space refers to the output space generated by GNN, that is, the space of the target results used by GNN for prediction or classification. This space includes the prediction results.

[0056] (7) Neighborhood aggregation (NA)

[0057] Domain aggregation is mainly used to aggregate or summarize the information of neighbor nodes of a node in the GNN graph into the representation of the node. This process helps nodes propagate information in the graph structure and learn the characteristics of neighbor nodes, thereby improving the performance of various tasks on graph data, such as node classification, graph classification, link prediction, etc.

[0058] Next, the technical solution provided by this application is introduced.

[0059] For example, Figure 1 The flowchart of a GNN visualization method provided by an embodiment of the present application is shown. It can be understood that the method can be executed by any device, equipment, platform, or device cluster with computing and processing capabilities. For ease of description, the following is an introduction using client execution as an example. The execution processes of other devices or equipment can refer to the execution process of the client, which will not be described one by one here. Among them, the client can be a desktop application, a mobile application, a Web application, or a Web-based application, etc. Figure 1 As shown, the GNN visualization method may include the following steps:

[0060] S101. Obtain first data of a latent space of a first GNN and second data of a first space, wherein the first space is any one of a topological space, a feature space, or a prediction space of the first GNN.

[0061] In this embodiment, after the user uses the first GNN to process the graph data, the user can upload both the data in the latent space of the first GNN and the data in the first space to the client. Among them, the first space can be any one of the topological space, feature space or prediction space of the first GNN. Exemplarily, the data in the latent space can be a vector representation of the nodes in the graph data in the latent space. The data in the topological space can be data describing the adjacency relationship between nodes in the graph data. The data in the prediction space can be data describing the prediction results. The data in the feature space can be a vector representation of the nodes in the graph data.

[0062] In some embodiments, the client may also obtain third data of the second space of the first GNN. The second space is any one of the topological space, feature space or prediction space of the first GNN, and the second space is different from the first space. In addition, the client may also obtain fourth data of the third space of the first GNN. The third space is any one of the topological space, feature space or prediction space of the first GNN, and both the first space and the second space are different from the third space.

[0063] S102. Display a first node dashboard, wherein the first node dashboard includes: a first view and a second view, wherein the first view is obtained by visualizing a latent space of the first GNN based on the first data, and the second view is obtained by visualizing the first space based on the second data.

[0064] In this embodiment, after obtaining the first data and the second data, the client can display the first node dashboard on its display interface. The first node dashboard may include: a first view and a second view. Among them, the first view is obtained by visualizing the latent space of the first GNN based on the first data, and the second view is obtained by visualizing the first space based on the second data. Exemplarily, for the data of the visualized latent space, the data of the latent space can be reduced in dimension so that the latent space of the first GNN can be visualized in a two-dimensional graph. For the data of the visualized topological space, the data of the topological space can be reduced in dimension so that the topological space can be visualized in a two-dimensional graph. For the data of the visualized prediction space, the data of the prediction space can be reduced in dimension so that the prediction space can be visualized in a two-dimensional graph. For the data of the visualized feature space, after the user selects a part of the nodes, the nodes selected by the user and the unselected nodes can be added in the feature space respectively, and rescaled to the set range, so as to complete the data visualization of the feature space. Through the visualization view of the feature space, the value size distribution of the corresponding positions of the nodes selected by the user and the unselected nodes in the feature space can be compared.

[0065] In some embodiments, when the client also obtains the second data of the second space, a third view may be further displayed in the first node dashboard. The third view is obtained by visualizing the second space based on the third data. In addition, when the client also obtains the third data of the third space, a fourth view may be further displayed in the first node dashboard. The fourth view is obtained by visualizing the third space based on the fourth data. For example, Figure 2 As shown, area 21 is a visualization of the latent space, area 22 is a visualization of the prediction space, area 23 is a visualization of the topological space, and area 24 is a visualization of the feature space. Figure 2 It can be seen that through these visualizations, users can intuitively understand the information related to the graph data, so that they can perform visual analysis on the graph data. For example, the visualization of the prediction space can include a confusion matrix. The confusion matrix can be used to characterize the relationship between the prediction results of the GNN and the labels. For example, continue to refer to Figure 2In the confusion matrix shown in area 22, the vertical axis is the real classification label information of the data set, and the horizontal axis is the classification information predicted by GNN. The content on the diagonal represents the data that is predicted completely correctly, and the numbers in other positions represent the number of corresponding data that are predicted incorrectly. For example, the number 5 in area 221 means that 5 data in ss0 are predicted to be in the 7th category.

[0066] In this way, by visualizing the latent space and the first space, we can accurately compare the node relationships in the two spaces, find out the graph structure that users need to focus on, and provide efficient and accurate interpretable basis for GNN debugging and tuning. In addition, more fine-grained analysis can also be performed through the latent space, which improves the efficiency and accuracy of analysis.

[0067] In some embodiments, after S101, the user can also perform a zoom operation on the view (such as the first view or the second view, etc.) displayed on the first node dashboard. After the user performs the zoom operation, the client can zoom the view on the view operated by the user. For example, continue to refer to Figure 2 , the user can zoom in on the visualization view of the latent space in area 21 , and at this time, the client can zoom in on the view in area 21 .

[0068] In some embodiments, after S101, the user can also filter the nodes in a certain view (such as the first view or the second view, etc.) displayed on the first node dashboard on the first node dashboard. For example, the user can circle a part of the area in a certain view, or select a part of the nodes, etc. After the user completes the filtering operation, the client can display the second node dashboard. The second node dashboard contains multiple views, and each view is associated with a view in the first node dashboard. Each view in the second node dashboard is used to display the nodes filtered out from a certain view on the first node dashboard. Exemplarily, a view in the second node dashboard can be understood as a subview of a certain view in the first node dashboard. For example, when the first node dashboard contains a visualization view of the latent space and a visualization view of the topological space, the second node dashboard also contains a visualization view of the latent space and a visualization view of the topological space, and the nodes displayed in the second node dashboard are only the nodes filtered out by the user. For example, when 100 nodes are displayed in the visualization view of the latent space in the first node dashboard, if the user selects 40 nodes therefrom, 40 nodes will be displayed in the visualization view of the latent space in the second node dashboard, and these 40 nodes will also be displayed in other views in the second node dashboard.

[0069] Furthermore, after displaying the second node dashboard, the user can also filter the nodes in a certain view displayed on the second node dashboard on the second node dashboard. After the user completes the filtering operation, the client can display the third node dashboard. The third node dashboard includes multiple views, and each view is associated with a view in the second node dashboard. Each view in the third node dashboard is used to display the nodes filtered out from a certain view on the second node dashboard. Exemplarily, a view in the third node dashboard can be understood as a subview of a certain view in the second node dashboard. For example, when the second node dashboard includes a visualization view of the latent space and a visualization view of the topological space, the third node dashboard also includes a visualization view of the latent space and a visualization view of the topological space, and the nodes displayed in the third node dashboard are only the nodes filtered out by the user. For example, when 40 nodes are displayed in the visualization view of the latent space in the second node dashboard, if the user selects 10 nodes from them, 10 nodes will be displayed in the visualization view of the latent space in the third node dashboard, and these 10 nodes will also be displayed in other views in the third node dashboard. In this way, by iteratively screening the nodes in the node dashboard, the node relationships in each space can be accurately compared, and the graph structure that the user needs to focus on can be found, providing efficient and accurate interpretable basis for GNN debugging and tuning.

[0070] In some embodiments, after S101, the user can also select a benchmark in the first node dashboard, such as a radius or an N-gon. Then, the client can display the distribution statistics of the nodes in each local area in the visualization view of the latent space in a pie chart in the visualization view of the latent space. The size of each local area is obtained by the benchmark. For example, after the user selects the benchmark, Figure 3 The distribution statistics of nodes in each local area are shown in the form of pie charts. The statistical results shown in the pie charts can intuitively show which types of nodes are very similar and difficult to separate. For example, in Figure 3 In the figure, the area indicated by the color 31 and the area indicated by the color 32 are often mixed together, indicating that the nodes of these two colors are very similar and difficult to separate.

[0071] In addition, when the visualization view of the topological space is displayed in the first node dashboard, the client can also display the distribution statistics of the nodes in each local area in the visualization view of the topological space in a pie chart in the visualization view of the topological space, wherein the size of each local area is obtained through the benchmark.

[0072] In some embodiments, in addition to analyzing only the first GNN, the user can also compare and analyze the first GNN with other GNNs (hereinafter referred to as "second GNN") to determine the pros and cons of the two under different tasks. At this time, the client can obtain the latent space data of the second GNN and the data of at least one other space (such as: topological space, feature space or prediction space, etc.). Then, the client can display the visualization views of each space of the second GNN on the first node dashboard, such as the visualization view of the latent space. Exemplarily, each visualization view of the second GNN is obtained through data related to the visualization view, such as: the visualization view of the latent space is based on the data of the latent space. In this way, the user can compare and analyze the first GNN and the second GNN in the first node dashboard. For example, Figure 4 As shown, area 41 shows a visualization view of the latent space of the first GNN, and area 42 shows a visualization view of the latent space of the second GNN. By comparing the views shown in area 41 and area 42, the distribution of nodes in the latent spaces of the two GNNs can be seen.

[0073] Furthermore, the user can select at least one node from the latent space of the first GNN or the second GNN. Then, a viewing operation is issued for the distance ranking relationship between the unselected nodes and the selected nodes in the latent space of the first GNN and the second GNN. Then, the client can visualize the distance ranking relationship between the unselected nodes and the selected nodes in the latent space of the first GNN and the latent space of the second GNN in a coordinate system. The horizontal axis of the coordinate system represents the distance distribution between the unselected nodes and the selected nodes in the latent space of the first GNN, and the vertical axis of the coordinate system represents the distance distribution between the unselected nodes and the selected nodes in the latent space of the second GNN. Exemplarily, the distance relationship between the unselected nodes and the selected nodes in the latent space of the first GNN and the latent space of the second GNN can be as follows. Figure 5 By sorting the distances between different nodes in the relationship graph and the points of interest selected by the user, we can see the similarities and differences between the two GNNs in the latent space.

[0074] The above is an introduction to the GNN visualization solution provided in the embodiment of the present application. For ease of understanding, the following example illustrates that the visualization solution can provide efficient and accurate interpretable evidence for GNN.

[0075] For example, Figure 6As shown in (A), the visualization view of the latent space shows that there are black nodes mixed in the white nodes. The visualization view of the topological space shows that the black node 62 is connected to the white node 61. In addition, most of the white nodes are connected to the white node 61, which shows that the white node 61 is a node with obvious aggregation characteristics, while the black node 62 is a misclassified node, which is connected to the white node 61, which means that the prediction of the black node 62 is wrong due to the influence of the white node 61. Through the analysis results, we can consider adjusting the data corresponding to the black node 62, such as adjusting the reference relationship, so as to achieve the tuning of the GNN. In addition, the analysis results can also explain why the black node 62 is predicted incorrectly. In this way, it is achieved to provide efficient and accurate interpretable basis for GNN.

[0076] like Figure 6 As shown in (B), from the visualization view of the prediction space, it can be seen that the third row and second column (i.e., area 71) in the confusion matrix contains the number of data items that mistakenly classified class2 as class1. At this time, in order to learn the cause of the classification error, the user can click on the number 17 shown in area 71 to further observe and analyze these 17 nodes. After the user clicks on the number 17, a new node dashboard can be created, and the 17 nodes are displayed in the visualization view in different spaces in the new node dashboard. In this way, the user can further observe and analyze these 17 nodes. For example, Figure 6 Observe and analyze in the way shown. In this way, users can explore problems and dig out causes through the GNN node dashboard.

[0077] It is understandable that the order of execution of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. In addition, the various embodiments described above can be combined according to actual conditions, and the combined solutions are still within the scope of protection of the present application.

[0078] Based on the method in the above embodiment, an embodiment of the present application provides a GNN visualization device.

[0079] For example, Figure 7 FIG. 1 is a schematic diagram showing the structure of a GNN visualization device provided in an embodiment of the present application. Figure 7As shown, the GNN visualization device 700 includes: an acquisition module 701 and a display module 702. The acquisition module 701 is used to acquire the first data of the latent space of the first GNN and the second data of the first space, where the first space is any one of the topological space, feature space or prediction space of the first GNN; the display module 702 is used to display the first node dashboard, where the first node dashboard includes: a first view and a second view, wherein the first view is obtained by visualizing the latent space of the first GNN based on the first data, and the second view is obtained by visualizing the first space based on the second data.

[0080] In some embodiments, the acquisition module 701 is also used to obtain third data of the second space of the first GNN, where the second space is any one of the topological space, feature space or prediction space of the first GNN, and the second space is different from the first space; the display module 702 is also used to display a third view in the first node dashboard, wherein the third view is obtained by visualizing the second space based on the third data.

[0081] In some embodiments, the acquisition module 701 is also used to obtain fourth data of the third space of the first GNN, where the third space is any one of the topological space, feature space or prediction space of the first GNN, and the first space and the second space are different from the third space; the display module 702 is also used to display a fourth view in the first node dashboard, wherein the fourth view is obtained by visualizing the third space based on the fourth data.

[0082] In some embodiments, after displaying the first node dashboard, the display module 702 is further configured to: scale the first view or the second view in response to the acquired first operation.

[0083] In some embodiments, after displaying the first node dashboard, the display module 702 is further used to: display a second node dashboard in response to a filtering operation on the nodes in the first view or the second view, the second node dashboard including a first sub-view and a second sub-view, both of which are used to display the nodes filtered from the first view or the second view, wherein the first sub-view is related to the first view, and the second sub-view is related to the second view.

[0084] In some embodiments, after displaying the second node dashboard, the display module 702 is further used to: display a third node dashboard in response to a filtering operation on the nodes in the first sub-view or the second sub-view, the third node dashboard including a first grandchild view and a second grandchild view, both of which are used to display nodes filtered from the first sub-view or the second sub-view, wherein the first grandchild view is related to the first sub-view, and the second grandchild view is related to the second sub-view.

[0085] In some embodiments, the display module 702 is further used to: when multiple node dashboards are displayed, fix one of the multiple node dashboards, and compare at least one view in an unfixed dashboard among the multiple dashboards with the view in the fixed dashboard.

[0086] In some embodiments, after displaying the first node dashboard, the display module 702 is further used to: in response to the obtained benchmark selection operation, display the distribution statistics of the nodes in each local area in the first view in a pie chart in the first view, wherein the size of each local area is obtained through the benchmark; or, when the first space is a topological space, in response to the obtained benchmark selection operation, display the distribution statistics of the nodes in each local area in the second view in a pie chart in the second view, wherein the size of each local area is obtained through the benchmark.

[0087] In some embodiments, the acquisition module 701 is also used to obtain fifth data of the latent space of the second GNN; the display module 702 is also used to display a fifth view in the first node dashboard, and the fifth view is obtained by visualizing the latent space of the second GNN based on the fifth data.

[0088] In some embodiments, the display module 702 is also used to obtain a selection operation on at least one node in the latent space of the first GNN or the second GNN; in response to a viewing operation on the distance ranking relationship between unselected nodes and selected nodes in the latent space of the first GNN and the second GNN, the distance ranking relationship between the unselected nodes and the selected nodes in the latent space of the first GNN and the second GNN is visually displayed in a coordinate system, the horizontal axis of the coordinate system represents the distance distribution between the unselected nodes and the selected nodes in the latent space of the first GNN, and the vertical axis of the coordinate system represents the distance distribution between the unselected nodes and the selected nodes in the latent space of the second GNN.

[0089] In some embodiments, Figure 7 The acquisition module 701 and the display module 702 shown in the figure can be implemented by software or by hardware. For example, the implementation of the acquisition module 701 is described below by taking the acquisition module 701 as an example. Similarly, the implementation of the display module 702 can refer to the implementation of the acquisition module 701.

[0090] As an example of a software functional unit, the acquisition module 701 may include code running on a computing instance. Among them, the computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Further, the above-mentioned computing instance may be one or more. For example, the acquisition module 701 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region (region) or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including a data center or multiple data centers with close geographical locations. Among them, usually a region may include multiple AZs.

[0091] Similarly, multiple hosts / virtual machines / containers used to run the code can be distributed in the same virtual private cloud (VPC) or in multiple VPCs. Usually, a VPC is set up in a region. For cross-region communication between two VPCs in the same region and between VPCs in different regions, a communication gateway needs to be set up in each VPC to achieve interconnection between VPCs through the communication gateway.

[0092] As an example of a hardware functional unit, the acquisition module 701 may include at least one computing device, such as a server, etc. Alternatively, the acquisition module 701 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.

[0093] The multiple computing devices included in the acquisition module 701 can be distributed in the same region or in different regions. The multiple computing devices included in the acquisition module 701 can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the acquisition module 701 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0094] It should be noted that, in other embodiments, the acquisition module 701 can be used to execute any step in the GNN visualization method described in the above embodiment, and one or more of the display modules 702 can be used to execute any step in the GNN visualization method described in the above embodiment. The steps that the acquisition module 701 and the display module 702 are responsible for implementing can be specified as needed, and the acquisition module 701 and the display module 702 respectively implement different steps in the GNN visualization method described in the above embodiment to achieve Figure 7 The overall functionality of the GNN visualization device 700 is shown.

[0095] The present application also provides a computing device 800. Figure 8 As shown, the computing device 800 includes: a bus 802, a processor 804, a memory 806, and a communication interface 808. The processor 804, the memory 806, and the communication interface 808 communicate through the bus 802. The computing device 800 can be a server or a terminal device. It should be understood that the present application does not limit the number of processors and memories in the computing device 800.

[0096] The bus 802 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 8 The bus 804 is represented by only one line, but does not mean that there is only one bus or one type of bus. The bus 804 may include a path for transmitting information between various components of the computing device 800 (eg, the memory 806, the processor 804, and the communication interface 808).

[0097] The processor 804 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0098] The memory 806 may include a volatile memory, such as a random access memory (RAM). The processor 804 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0099] The memory 806 stores executable program codes, and the processor 804 executes the executable program codes to respectively implement the aforementioned Figure 7 The functions of the acquisition module 701 and the display module 702 shown in the embodiment are implemented to realize the GNN visualization method described in the above embodiment. That is, the memory 806 stores instructions for executing the GNN visualization method described in the above embodiment.

[0100] Alternatively, the memory 806 stores executable codes, and the processor 804 executes the executable codes to respectively implement the aforementioned Figure 7 The functions of the GNN visualization device 700 shown in the figure are implemented to realize the GNN visualization method described in the above embodiment. That is, the memory 806 stores instructions for executing the GNN visualization method described in the above embodiment.

[0101] The communication interface 803 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 800 and other devices or a communication network.

[0102] The embodiment of the present application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smart phone.

[0103] like Fig. 9As shown, the computing device cluster includes at least one computing device 800. The memory 806 in one or more computing devices 800 in the computing device cluster may store the same instructions for executing the GNN visualization method described in the above embodiment.

[0104] In some possible implementations, the memory 806 of one or more computing devices 800 in the computing device cluster may also respectively store some instructions for executing the GNN visualization method described in the above embodiment. In other words, the combination of one or more computing devices 800 can jointly execute the instructions for executing the GNN visualization method described in the above embodiment.

[0105] It should be noted that the memory 806 in different computing devices 800 in the computing device cluster may store different instructions, which are respectively used to execute the aforementioned Figure 7 Part of the functions of the GNN visualization device 700 shown. That is, the instructions stored in the memory 806 in different computing devices 800 can implement the functions of one or more modules in the acquisition module 701 and the display module 702.

[0106] In some possible implementations, one or more computing devices in the computing device cluster may be connected via a network, which may be a wide area network or a local area network. Fig.10 A possible implementation is shown. Fig.10 As shown, two computing devices 800A and 800B are connected via a network. Specifically, the network is connected via a communication interface in each computing device. In this type of possible implementation, the memory 806 in the computing device 800A stores instructions for executing the functions of the acquisition module 701. At the same time, the memory 806 in the computing device 800B stores instructions for executing the functions of the display module 702.

[0107] It should be understood that Fig.10 The functions of the computing device 800A shown in FIG. 8 may also be completed by multiple computing devices 800. Similarly, the functions of the computing device 800B may also be completed by multiple computing devices 800.

[0108] The present application embodiment also provides another computing device cluster. The connection relationship between the computing devices in the computing device cluster can be similar to that of Fig. 9 and Fig.10 The connection mode of the computing device cluster is different in that the memory 806 in one or more computing devices 800 in the computing device cluster may store the same instructions for executing the method in the above embodiment.

[0109] In some possible implementations, the memory 806 of one or more computing devices 800 in the computing device cluster may also respectively store some instructions for executing the aforementioned GNN visualization method. In other words, the combination of one or more computing devices 800 can jointly execute instructions for executing the aforementioned GNN visualization method.

[0110] Based on the method in the above embodiment, the embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a computing device cluster including at least one computing device, the computing device cluster executes the method described in the above embodiment. Exemplarily, the computer-readable storage medium can be any available medium that can be stored by the computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk), etc.

[0111] Based on the method in the above embodiment, an embodiment of the present application provides a computer program product including instructions. When the computer program product is run on a computing device cluster including at least one computing device, the computing device cluster executes the method in the above embodiment.

[0112] It is understood that the processor in the embodiments of the present application may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0113] The method steps in the embodiments of the present application can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC.

[0114] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions may be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state disk (SSD)), etc.

[0115] It should be understood that the various numerical numbers involved in the embodiments of the present application are only used for the convenience of description and are not used to limit the scope of the embodiments of the present application.

[0116] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present application.

Claims

1. A graph neural network (GNN) visualization method, characterized in that: include: Obtaining first data of a hidden space of a first GNN and second data of a first space, where the first space is any one of a topological space, a feature space, or a prediction space of the first GNN; A first node dashboard is displayed, wherein the first node dashboard includes: a first view and a second view, wherein the first view is obtained by visualizing a latent space of the first GNN based on the first data, and the second view is obtained by visualizing the first space based on the second data.

2. The method according to claim 1, characterized in that Also includes: Obtain third data of a second space of the first GNN, where the second space is any one of a topological space, a feature space, or a prediction space of the first GNN, and the second space is different from the first space; A third view is displayed in the first node dashboard, wherein the third view is obtained by visualizing the second space based on the third data.

3. The method according to claim 2, characterized in that Also includes: Acquire fourth data of a third space of the first GNN, where the third space is any one of a topological space, a feature space, or a prediction space of the first GNN, and both the first space and the second space are different from the third space; A fourth view is displayed in the first node dashboard, wherein the fourth view is obtained by visualizing the third space based on the fourth data.

4. The method according to any one of claims 1 to 3, characterized in that: After displaying the first node dashboard, it also includes: In response to the acquired first operation, scaling the first view or the second view.

5. The method according to any one of claims 1 to 4, characterized in that: After displaying the first node dashboard, it also includes: In response to a filtering operation on nodes in the first view or the second view, a second node dashboard is displayed, wherein the second node dashboard includes a first sub-view and a second sub-view, and the first sub-view and the second sub-view are both used to display nodes filtered from the first view or the second view, wherein the first sub-view is related to the first view, and the second sub-view is related to the second view.

6. The method according to claim 5, characterized in that After displaying the second node dashboard, it also includes: In response to a filtering operation on nodes in the first sub-view or the second sub-view, a third node dashboard is displayed, wherein the third node dashboard includes a first grandchild view and a second grandchild view, wherein the first grandchild view and the second grandchild view are both used to display nodes filtered from the first sub-view or the second sub-view, wherein the first grandchild view is related to the first sub-view, and the second grandchild view is related to the second sub-view.

7. The method according to any one of claims 5 or 6, characterized in that: Also includes: In the case where multiple node dashboards are displayed, one of the multiple node dashboards is fixed, and at least one view in an unfixed dashboard among the multiple dashboards is compared with a view in the fixed dashboard.

8. The method according to any one of claims 1 to 7, characterized in that: After displaying the first node dashboard, it also includes: In response to the acquired benchmark selection operation, displaying the distribution statistics of nodes in each local area in the first view in a pie chart in the first view, wherein the size of each local area is obtained according to the benchmark; Alternatively, when the first space is a topological space, in response to the obtained benchmark selection operation, the distribution statistics of the nodes in each local area in the second view are displayed in a pie chart in the second view, wherein the size of each local area is obtained through the benchmark.

9. The method according to any one of claims 1 to 8, characterized in that: Also includes: Obtain fifth data of the latent space of the second GNN; A fifth view is displayed in the first node dashboard, where the fifth view is obtained by visualizing the latent space of the second GNN based on the fifth data.

10. The method according to claim 9, characterized in that Also includes: Obtaining a selection operation of at least one node in the latent space of the first GNN or the second GNN; In response to a viewing operation on the distance ranking relationship between unselected nodes and selected nodes in the latent space of the first GNN and the second GNN, the distance ranking relationship between the unselected nodes and the selected nodes in the latent space of the first GNN and the second GNN is visualized in a coordinate system, the horizontal axis of the coordinate system represents the distance distribution between the unselected nodes and the selected nodes in the latent space of the first GNN, and the vertical axis of the coordinate system represents the distance distribution between the unselected nodes and the selected nodes in the latent space of the second GNN.

11. A graph neural network (GNN) visualization device, characterized in that: include: An acquisition module, used to acquire first data of a hidden space of a first GNN and second data of a first space, where the first space is any one of a topological space, a feature space, or a prediction space of the first GNN; A display module is used to display a first node dashboard, wherein the first node dashboard includes: a first view and a second view, wherein the first view is obtained by visualizing the latent space of the first GNN based on the first data, and the second view is obtained by visualizing the first space based on the second data.

12. The device according to claim 11, characterized in that The acquisition module is further used to acquire third data of a second space of the first GNN, where the second space is any one of a topological space, a feature space, or a prediction space of the first GNN, and the second space is different from the first space; The display module is further used to display a third view in the first node dashboard, wherein the third view is obtained by visualizing the second space based on the third data.

13. The device according to claim 12, characterized in that The acquisition module is further used to acquire fourth data of a third space of the first GNN, where the third space is any one of a topological space, a feature space, or a prediction space of the first GNN, and both the first space and the second space are different from the third space; The display module is further used to display a fourth view in the first node dashboard, wherein the fourth view is obtained by visualizing the third space based on the fourth data.

14. The device according to any one of claims 11 to 13, characterized in that: After displaying the first node dashboard, the display module is further used to: In response to the acquired first operation, scaling the first view or the second view.

15. The device according to any one of claims 11 to 14, characterized in that: After displaying the first node dashboard, the display module is further used to: In response to a filtering operation on nodes in the first view or the second view, a second node dashboard is displayed, wherein the second node dashboard includes a first sub-view and a second sub-view, and the first sub-view and the second sub-view are both used to display nodes filtered from the first view or the second view, wherein the first sub-view is related to the first view, and the second sub-view is related to the second view.

16. The device according to claim 15, characterized in that After displaying the second node dashboard, the display module is further used to: In response to a filtering operation on nodes in the first sub-view or the second sub-view, a third node dashboard is displayed, wherein the third node dashboard includes a first grandchild view and a second grandchild view, wherein the first grandchild view and the second grandchild view are both used to display nodes filtered from the first sub-view or the second sub-view, wherein the first grandchild view is related to the first sub-view, and the second grandchild view is related to the second sub-view.

17. The device according to any one of claims 15 or 16, characterized in that: The display module is also used for: In the case where multiple node dashboards are displayed, one of the multiple node dashboards is fixed, and at least one view in an unfixed dashboard among the multiple dashboards is compared with a view in the fixed dashboard.

18. The device according to any one of claims 11 to 17, characterized in that: After displaying the first node dashboard, the display module is further used to: In response to the acquired benchmark selection operation, displaying the distribution statistics of nodes in each local area in the first view in a pie chart in the first view, wherein the size of each local area is obtained according to the benchmark; Alternatively, when the first space is a topological space, in response to the obtained benchmark selection operation, the distribution statistics of the nodes in each local area in the second view are displayed in a pie chart in the second view, wherein the size of each local area is obtained through the benchmark.

19. The device according to any one of claims 11 to 18, characterized in that: The acquisition module is further used to acquire fifth data of the latent space of the second GNN; The display module is further used to display a fifth view in the first node dashboard, where the fifth view is obtained by visualizing the latent space of the second GNN based on the fifth data.

20. The device according to claim 19, characterized in that The display module is also used for: Obtaining a selection operation of at least one node in the latent space of the first GNN or the second GNN; In response to a viewing operation on the distance ranking relationship between unselected nodes and selected nodes in the latent space of the first GNN and the second GNN, the distance ranking relationship between the unselected nodes and the selected nodes in the latent space of the first GNN and the second GNN is visualized in a coordinate system, the horizontal axis of the coordinate system represents the distance distribution between the unselected nodes and the selected nodes in the latent space of the first GNN, and the vertical axis of the coordinate system represents the distance distribution between the unselected nodes and the selected nodes in the latent space of the second GNN.

21. A computing device cluster, characterized in that: comprising at least one computing device, each computing device comprising a processor and a memory; The processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method according to any one of claims 1-10.

22. A computer-readable storage medium storing a computer program, which, when executed on a computing device cluster comprising at least one computing device, enables the computing device cluster to execute the method according to any one of claims 1 to 10.

23. A computer program product, characterized in that When the computer program product is run on a computing device cluster including at least one computing device, the computing device cluster is enabled to execute the method according to any one of claims 1 to 10.

Images