Interactive generation method of controllable layout of graph visualization based on deep diffusion model

Abstract

The application discloses a graph visualization controllable layout interactive generation method based on a deep diffusion model, and belongs to the field of visualization and human-computer interaction; the method comprises the following steps: (1) graph data enhancement; (2) deep diffusion model structure design for graph layout; (3) deep diffusion model pre-training for graph layout; (4) user uploading of graph data in JSON format; (5) user interaction for constructing expected layout constraints; (6) system analysis of the layout constraints and generation of graph visualization layout based on the pre-trained model; (7) system display of the graph visualization layout result and support for user exploration of the output of different models; and (8) user iteration for creating layout constraints to construct a data streaming visualization exploration process. The method discloses a graph layout method based on a deep diffusion model and an interactive graph visualization generation system, supports an analyst without a programming background to efficiently construct a graph visualization result meeting user requirements, and reduces the difficulty of user screening of a suitable graph visualization layout form.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of visualization and human-computer interaction, and specifically proposes an interactive generation method of controllable layout of graph visualization based on a deep diffusion model for the node-link graph visualization form commonly used in practical applications. BACKGROUND

[0002] Graph data is widely used in various fields in practical applications, including social networks, molecular structures, and citation networks. Graph visualization supports users in intuitively understanding the internal connections and topological structures of data. Among various methods of graph visualization, the node-link graph is most widely used, which supports users in understanding the topological structure and connection relationship of graph data, wherein data objects are mapped to nodes, and the connection relationship between data objects is represented as a connection line between nodes. Many algorithms have been proposed in the field of visualization research to layout node-link graphs. Different algorithms are suitable for different types of graph data and analysis tasks. The layout form of the same graph presents diversity according to the selected algorithm and its specific configuration.

[0003] In the process of determining the visualization form of the node-link graph, users often need to combine their own preferences and their understanding of the data to select the most suitable graph visualization form, and then assist users in analyzing the graph data. Therefore, it is an important task to support users to efficiently obtain the graph visualization layout that meets their needs. Currently, the graph layout form that meets the user's needs is mainly obtained by writing code and interactive selection templates. However, the method of writing code supports users to explore the layout that meets their expectations by adjusting the parameters of the layout algorithm, which has a high learning difficulty and a heavy programming burden, and is not suitable for most non-expert users. Although the interactive selection template method can improve the efficiency of user layout selection, this process requires users to traverse all graph layout algorithms, which is tedious and time-consuming, and cannot fully utilize the user's expectations of the graph layout result. Currently, there is no method that can efficiently and accurately generate the graph visualization layout form according to the user's expectations. SUMMARY

[0004] In view of the defects in the prior art, the purpose of the present application is to provide a controllable layout interactive generation method for graph visualization based on a deep diffusion model for node link graph data. This method supports users to efficiently obtain graph visualization results that meet user expectations. Users specify the positions of specific nodes as constraints for graph layout based on their expectations for graph layout. These nodes can form a connected subgraph or multiple disconnected parts. The system includes a front end and a back end. Users load the graph data to be processed through the system front end, and users can add layout constraints for this data. Then the system front end transmits these data and possible constraints to the system back end. The system back end analyzes the layout constraints and generates a graph visualization layout based on a pre-trained model, and transmits the generated data to the front end. The front end displays the graph visualization layout result according to these data. The entire system automatically generates a graph visualization layout form according to the constraints provided by the user, and ensures that the generated layout result matches the user's constraints and the diversity of the output graph layout result, thereby directly generating a graph visualization result that meets the user's expectations, reducing the user's difficulty in exploring different graph visualization forms.

[0005] To achieve the above-mentioned purpose, the technical solution adopted by the present application for node link graph visualization generation is:

[0006] The controllable layout interactive generation method for graph visualization based on a deep diffusion model includes the following eight steps: (1) graph data enhancement; (2) graph layout model structure design; (3) graph layout model pre-training; (4) users upload JSON format graph data through the system front end; (5) users interactively construct expected layout constraints through the system front end; (6) the system back end analyzes the layout constraints and generates a graph visualization layout based on a pre-trained model; (7) the system front end displays the graph visualization layout result and supports users to explore different model outputs; (8) users iteratively create layout constraints to build a data stream visualization exploration process.

[0007] Specifically:

[0008] (1) graph data enhancement;

[0009] Graph data augmentation involves the augmentation of node features and the augmentation of graph structure. Among them, the augmentation of node features is crucial for generating the layout of non-attribute graphs. Since the graph layout depends largely on the topology of the graph, the choice of node encoding method must accurately represent the topological features of the graph, so the present application adopts the Laplacian position encoding method as the initial node feature. Laplacian position encoding is a spectral technique that embeds a topological graph in a Euclidean space, and the node encoding result combines position and structure information, establishing a meaningful local coordinate system while preserving the global graph structure. Nodes that are far apart in the graph have different encodings, while nodes that are close should have similar encodings. Laplacian position encoding L is defined by factorizing the graph Laplacian matrix:

[0010]

[0011] Here, I is an N x N identity matrix, D is a node degree matrix, B represents an adjacency matrix, and Λ and U represent eigenvalues and eigenvectors, respectively. The present application uses the k smallest non-trivial eigenvectors as the initial node feature, and k is a hyperparameter. The eigenvectors encode the smooth position coordinates of neighboring nodes, enabling the nodes to effectively capture the structure information within their perception domain during the message passing process, thereby helping the model better capture the topological relationship between nodes.

[0012] In addition, the present application enhances the original graph topology to improve the efficiency of message passing between nodes. Graph structure augmentation involves establishing connections between nodes, which is divided into three types: original edges, local edges, and global edges.

[0013] (a) Original edges represent edges from the original graph, preserving the inherent connections in a given graph and directly reflecting its topology.

[0014] (b) Local edges, as an extension of the original graph topology, connect each node to its n-order neighbor, expanding the node's perception range and helping the model effectively generate the local structure around each node, where n is determined by the hyperparameter adjustment of the grid search.

[0015] (c) Both original edges and local edges originate from the topology of the original graph, facilitating information exchange between topologically adjacent nodes. However, each node in the graph layout is at a different location in a two-dimensional space, and nodes that are physically close can significantly influence the graph layout, but they may have considerable topological distance, limiting their exchange of information. Therefore, the present application introduces global edges to dynamically connect nodes within a distance threshold τ range, and the radius cutoff value is determined according to the specific scale of the layout. Global edges help to achieve more consistent and comprehensive global structure.

[0016] (2) Deep diffusion model structure design for graph layout

[0017] The graph layout model structure design mainly includes two processes: a forward process and a backward process. The present invention defines a graph layout as a system consisting of multiple nodes and edges in a two-dimensional space. The forward process involves transforming the system from a balanced state to a chaotic state, while the backward process optimizes the system from the chaotic state to restore to the balanced state. The balanced state represents an orderly layout, while the chaotic state represents a chaotic and noisy layout. The present invention defines X0 as the graph layout in the balanced state. t Xt is the graph layout in the chaotic state (t = 1,..., T), where t represents the index of the diffusion step. The overall goal of the model is to learn how to restore X0 from X T For a given graph G, the forward process gradually introduces a slight Gaussian noise (sampled from a standard Gaussian distribution) into the layout X0. Through several steps of perturbation, it is transformed into a completely noisy layout X T while ensuring that X T follows the prior distribution N(0, 1). In each step, the signal-to-noise ratio is determined by a fixed variance pattern β1,..., β T The present invention represents the forward process as a Markov chain:

[0018]

[0019] That is, for a given graph G, the state X t is subject to a Gaussian distribution t-1 given the previous state X where represents the mean of the Gaussian distribution, β t I represents the covariance matrix, where I is an N x N identity matrix;

[0020] For t independent and identically distributed Gaussian noise N(μ, σ 2 , its superposition is still a Gaussian noise; therefore, define α t = 1 - β t and then it is able to sample X t :

[0021]

[0022] That is, for a given graph G, the state X t is subject to a Gaussian distribution given the initial state X0. This means that for any state X t of the graph G, the corresponding Gaussian noise can be added to the initial state X0 to obtain;

[0023] The inverse process is an iterative optimization process that samples a completely noisy layout X from the prior distribution N(0, 1) T This process involves T denoising steps that gradually optimize the layout system from a chaotic state to an equilibrium state; the inverse process is also a Markov chain:

[0024]

[0025] That is, for a given X t-1 , the state at X t , follows a Gaussian distribution where μ θ is a function used to estimate the mean of the distribution, and σ θ represents the arbitrary variance; μ θ is calculated as follows:

[0026]

[0027] ∈ t is a trainable function used to predict the noise on the layout X t , and is also the training objective:

[0028]

[0029] This function is a loss function that measures the difference between the predicted noise ∈ θ at the t-th step and the true noise ∈ θ ; by minimizing this loss function, a model that accurately predicts future noise ∈ θ can be trained;

[0030] For the task of graph layout generation, it is crucial to ensure the invariance of the graph layout under rotation and translation. Therefore, the function ∈ θ must ensure the invariance of rotation and translation. The present invention

[0031] uses an equivariant graph neural network to parameterize ∈ θ ; the equivariant graph neural network is composed of multiple layers of equivariant convolution layers:

[0032]

[0033] The first formula represents the message passing mechanism in the equivariant graph neural network: each node i sends a message m ij to its neighbor node j; the content of the message includes the hidden state of nodes i and j at the current layer and the square of the distance between them and the edge feature e ij ; is a fully connected neural network that aggregates the edge endpoint embeddings and node distances;

[0034] The second formula represents the mechanism of updating the hidden state of the isometric graph neural network: the next-layer digital state of node i is obtained by summing the messages from all neighbors and then passing through a fully connected neural network ; here N(i) represents the neighbor set of node i;

[0035] The third formula represents the mechanism of updating the coordinates of the isometric graph neural network: the next-layer digital coordinates of node i are obtained by adding a correction term to the current coordinates ; the correction term takes into account the relative position information between nodes and the updated hidden state; is a fully connected neural network responsible for projecting the updated endpoint embedding and edge embedding into a two-dimensional space;

[0036] where l is the index of the layer; represents the node embedding vector, and the initial value is obtained from the Laplace eigenvalue position encoding; corresponds to the node coordinates, quantifies the Euclidean distance between node i and node j; e ij represents the type of edge, including original, local and global edge; N(i) represents the neighborhood of node i; is a fully connected neural network used to aggregate the endpoint embedding of the edge and the node distance; by utilizing the information of the neighbor nodes to update the node embedding, and using the graph isomorphism neural network for parameterization; the node coordinates are updated by the weighted difference of the invariant features ; ∈ θ ensure the rotation and translation invariance of the graph layout;

[0037] The centroid of the X0 layout is transformed to the origin to obtain the graph layout The centroid represents the weighted average of all node positions x i in the layout, and the specific transformation operation is as follows:

[0038]

[0039] (3) Deep diffusion model pre-training for graph layout

[0040] For the pre-training of the graph layout model, the present application collects the layout results of several graphs, including different types of graphs and layout results obtained by using different layout algorithms for the same graph. In the training process, the present application selects a random time step t, adds noise ∈ sampled from the prior distribution N(0, 1) to the equilibrium state graph layout X0, and the signal-to-noise ratio is determined by t and the pre-defined variance mode β1, …, βT determines that the perturbation results in a chaotic layout X t . Then, the perturbed layout X t is input into the denoising network ∈ θ and the network is trained to predict the layout noise using ∈ as the supervisory signal. When ∈ θ can accurately predict the noise on X t , the model can recover the chaotic graph layout to a high-quality graph layout in the equilibrium state through the inverse process defined above.

[0041] (4) The user uploads the graph data in JSON format through the system front end;

[0042] The user loads and uploads the graph data in JSON format to the system back end through the system front end. After the system back end accepts it, it automatically parses the data and enhances the original graph data, enriches the node attributes in the graph, and enhances the topology of the graph. The system will use a pre-trained model to generate an initial layout without any constraints for the uploaded graph data. The output result of the model will be automatically visualized in the system.

[0043] (5) The user interacts to build the desired layout constraints through the system front end;

[0044] This system supports users to quickly build layout constraints according to their expectations of the graph layout to guide the generation of the graph layout. The user can select specific nodes to move to their desired positions in the visualized results on the system front end, or collectively move multiple nodes by marqueeing and dragging. The system control panel is used for the user to add the desired layout constraint information for this graph. In order to help the user efficiently build the graph layout constraints, the system front end provides four common interactive operations: horizontal flip, vertical flip, scaling, and rotation, as well as four commonly used layout modes to arrange the selected nodes into corresponding shapes: straight line, spiral, circle, and regular polygon. By building layout constraints, the user can control the graph layout generation process and guide the model to generate a layout result that matches the layout constraints, thereby generating a graph layout visualization form that meets the user's expectations.

[0045] (6) The system back end parses the layout constraints and generates the graph visualization layout based on the pre-trained model;

[0046] The system back end parses the layout constraints built by the user in step (5) into inputs acceptable to the model and fuses them into each step of the layout generation process. In each time step t in the inverse process in step (2), the present invention combines the constraints provided by the user with the generation results of the model. Considering the rotation and translation invariance of the graph layout, the layout constraints and the model generation results usually have deviations in the overall position in the two-dimensional space. In order to facilitate the effective coordination between the layout constraints and the model generation results, the present invention resamples at each time step t, and adjusts the position of the model generation results to match the layout constraints. X t-1 Back to X t The process is repeated n times to ensure that the model can align the layout constraints and the generated results. Through the constrained generation process, the model realizes the seamless integration of user layout constraints into the model generation results.

[0047] (7) The system front end displays the graph visualization layout results and supports user exploration of different model outputs;

[0048] The system front end visualization module visualizes the graph layout results generated by the system back end model. The visualization part of the system is completed based on the visualization toolkit D3. The graph visualization form draws the results, which can be automatically adjusted according to the window size. After the graph visualization form is displayed on the computer screen, the user can interact with the graph visualization form, rotate, zoom or interactively select a part of the data to view the details, so as to explore the graph layout results. At the same time, the user can select pre-trained models of different data sets and adjust the layout hyperparameters to explore the graph layout results of different models.

[0049] (8) User iteratively creates layout constraints to build a data stream visual exploration process;

[0050] The system supports the user to iteratively create layout constraints to build a data stream visual exploration process. The system draws lessons from the classic data stream model, and the user defines the layout constraints in each module of the data stream according to step (5). The system generates the graph layout results based on the provided constraints, and the layout results are used as the input of the next module. When the data in a module is updated, the input and output of all subsequent modules connected thereto will be automatically updated. At the same time, in order to support the reuse of graph layout constraints, the system also provides a module duplication function, and the user can duplicate part of the modules in the data stream. The input of the duplicated sub-data stream is determined by the new connection relationship. The user can iteratively create layout constraints to adjust the graph layout results until the graph layout visualization form that meets the user's needs is obtained.

[0051] The beneficial effects of the present application are as follows:

[0052] The method provided by the present application supports user interaction to build layout constraints to guide the graph visualization generation process, and then directly obtains the graph visualization layout form that meets the user's expectations. On the basis of the pre-trained graph layout model, the system introduces an interactive layout constraint construction method and a data stream model to realize a complete exploratory graph visualization form generation system, which supports the user to efficiently generate the graph visualization form according to the needs.

[0053] Specifically, the effects of the present application include:

[0054] The ability of generating node-link diagram visualization forms is provided: the diffusion model-based graph visualization generation model is designed and pre-trained in steps (2) and (3), and suitable node-link visualization layout forms of the graph data are supported.

[0055] The user-guided graph layout generation process is supported, and the difficulty of obtaining the expected layout is reduced: the user constructs the layout constraint reflecting the expectation in step (5), and the layout result adapted to the layout constraint provided by the user is generated in step (6). The generation process does not require the user to screen the layout result, and does not require the user to have programming basis, so that the method has a low learning curve. BRIEF DESCRIPTION OF DRAWINGS

[0056] Figure 1 The operation flow of the user in the embodiment of the application is shown.

[0057] Figure 2 The schematic diagram of the graph topology enhancement in the embodiment of the application is shown.

[0058] Figure 3 The flowchart of the graph layout constraint generation in the embodiment of the application is shown.

[0059] Figure 4 The graph interface diagram of the graph visualization exploration and the layout constraint interaction in the embodiment of the application is shown.

[0060] Figure 5 The graph of the graph layout generation result and the reference layout in the embodiment of the application is shown.

[0061] Figure 6 The graph of the tree graph generation result and the reference layout in the embodiment of the application is shown.

[0062] Figure 7 The graph of the multi-class graph data generation result and the reference layout in the embodiment of the application is shown.

[0063] Figure 8 The schematic diagram of the constraint generation effect in the embodiment of the application is shown. DETAILED DESCRIPTION

[0064] The application will be further described in detail below in combination with the drawings and specific embodiments in the specification.

[0065] Figure 1 The operation flow of the user in the embodiment of the application is shown.

[0066] The method for controllable layout interactive generation of graph visualization based on a deep diffusion model comprises the following eight steps: (1) graph data enhancement; (2) graph layout model structure design; (3) graph layout model pre-training; (4) a user uploads graph data in JSON format through a system front end; (5) the user constructs desired layout constraints through the system front end; (6) the system back end analyzes the layout constraints and generates graph visualization layout based on the pre-trained model; (7) the system front end displays the graph visualization layout result and supports user exploration of the output of different models; and (8) the user iteratively creates layout constraints to construct a data stream visualization exploration process.

[0067] Specifically,

[0068] (1) graph data enhancement;

[0069] Graph data enhancement involves enhancement of node features and enhancement of graph structure. Enhancement of node features is crucial for generating layouts of non-attribute graphs. Since graph layout largely depends on the topology of the graph, the selection of the node encoding method must accurately represent the topological features of the graph, and therefore the present application adopts a Laplacian position encoding method as the initial node feature. Laplacian position encoding is a spectral technique that embeds a topological graph in a Euclidean space, and the node encoding result combines position and structure information, establishes a meaningful local coordinate system, and at the same time preserves the global graph structure. Nodes that are far apart in the graph have different encodings, while nodes that are close together should have similar encodings. The Laplacian position encoding L is defined by factorizing the graph Laplacian matrix:

[0070]

[0071] Here, I is an N x N identity matrix, D is a node degree matrix, B represents an adjacency matrix, and Lambda and U represent eigenvalues and eigenvectors, respectively. The present application uses the k smallest non-trivial eigenvectors as the initial node features, and k is a hyperparameter. The eigenvectors encode the smooth position coordinates of neighboring nodes, enabling the nodes to effectively capture the structure information within their perception fields during the message passing process, thereby helping the model better obtain the topological relationship between nodes.

[0072] In addition, the present application enhances the original graph topology to improve the efficiency of message passing between nodes. Graph structure enhancement involves establishing connections between nodes, which is divided into three types: original edges, local edges and global edges.

[0073] (a) Original edges represent edges from the original graph, preserving the inherent connections in a given graph and directly reflecting its topological structure.

[0074] (b) Local edges as extensions of the original graph topology connect each node to its n-order neighbors, extending the node's perception range, helping the model to effectively generate the local structure around each node, where n is determined by hyperparameter tuning through grid search.

[0075] (c) Both original and local edges are derived from the topology of the original graph, facilitating information exchange between topologically adjacent nodes, while in the graph layout each node is at a different location in two-dimensional space, nodes that are close in physical location can significantly affect the graph layout, but they can have considerable topological distance, limiting their exchange of information. Therefore, the present invention introduces global edges to dynamically connect nodes within a distance threshold τ range, and the radius cutoff is determined according to the specific scale of the layout. Global edges help to generate more consistent and comprehensive global structure.

[0076] (2) Structure design of deep diffusion model for graph layout

[0077] The structure design of the graph layout model mainly includes two processes: the forward process and the inverse process. The present invention defines the graph layout as a system composed of multiple nodes and edges in two-dimensional space. The forward process involves transforming the system from a balanced state to a chaotic state, while the inverse process optimizes the system from a chaotic state to restore it to a balanced state. The balanced state represents an orderly layout, while the chaotic state represents a chaotic and noisy layout. The present invention defines X0 as the graph layout in the balanced state. X t is the graph layout in the chaotic state (t = 1,..., T), where t represents the index of the diffusion step. The overall goal of the model is to learn how to restore X0 from X T . For a given graph G, the forward process gradually introduces slight Gaussian noise (sampled from a standard Gaussian distribution) into the layout X0. Through several steps of perturbation, it is transformed into a completely noisy layout X T , while ensuring that X T follows the prior distribution N(0, 1). In each step, the signal-to-noise ratio is determined by a fixed variance pattern β1,..., β T . The present invention represents the forward process as a Markov chain:

[0078]

[0079] That is, for a given graph G, the state X t is conditioned on the previous state X t-1 , and follows a Gaussian distribution where represents the mean of the Gaussian distribution, β t I represents the covariance matrix, where I is an N x N identity matrix;

[0080] for t independent and identically distributed Gaussian noise N(μ, σ2 ), which is still a Gaussian noise after superposition; therefore, define α t = 1-β t and then given X0, sample X t :

[0081]

[0082] that is, for a given graph G, the state X t at the initial state X0, obeys a Gaussian distribution This means that for any state X t of the graph G, the corresponding Gaussian noise can be added on the initial state X0

[0083] The inverse process is an iterative optimization process, which samples a completely noisy layout X T from the prior distribution N(0, 1); this process involves T denoising steps, which gradually optimize the layout system from a chaotic state to an equilibrium state; the inverse process is also a Markov chain:

[0084]

[0085] that is, for a given X t-1 , the state at X t obeys a Gaussian distribution where μ θ is a function used to estimate the mean of the distribution, represents any variance; μ θ is calculated as follows:

[0086]

[0087] ∈ θ is a trainable function used to predict the noise on the layout X t , which is also the training objective:

[0088]

[0089] This function is a loss function, which measures the difference between the predicted noise ∈ θ at the t-th step and the true noise ∈; by minimizing this loss function, a model that can accurately predict future noise ∈ θ can be trained;

[0090] For the task of graph layout generation, it is crucial to ensure the invariance of the graph layout under rotation and translation. Therefore, the function ∈ θ must ensure the invariance of rotation and translation.

[0091] Using an equivariant graph neural network to parameterize ∈ θ Equivariant graph neural networks consist of multiple equivariant convolutional layers:

[0092]

[0093] The first formula represents the message passing mechanism in an equivariant graph neural network: each node i sends message m to its neighbor node j. ij The message content includes the hidden states of nodes i and j at the current level. and The square of the distance between them and edge feature e ij ; It is a fully connected neural network used to aggregate edge endpoint embeddings and node distances;

[0094] The second formula describes the mechanism by which an equivariant graph neural network updates the hidden state of a node: the next layer digital state of node i. It works by summing the messages from all neighbors and then passing them through a fully connected neural network. The update yields N(i), where N(i) represents the set of neighbors of node i.

[0095] The third formula describes the mechanism by which the equivariant graph neural network updates the coordinates of nodes: the next layer's numerical coordinates of node i. Through current coordinates This is obtained by adding a correction term; the correction term takes into account the relative positions between nodes and the updated hidden state; It is a fully connected neural network responsible for projecting updated endpoint embeddings and edge embeddings into a two-dimensional space;

[0096] Where: l is the layer index; This represents the node embedding vector, with initial values ​​obtained from the Laplacian feature vector position encoding. Corresponding to the node coordinates, The Euclidean distance between node i and node j was quantified; e ij The edge type is indicated, including original, local, and global edges; N(i) represents the neighborhood of node i; It is a fully connected neural network used to aggregate edge endpoint embeddings and node distances; Node embeddings are updated using information from neighboring nodes and parameterized using a graph isomorphic neural network; node coordinates are obtained by weighting differences based on invariant features. Update; ∈ θ This ensures the rotation and translation invariance of the graph layout;

[0097] Transform the centroid of the X0 layout to the origin to get the graph layout The centroid represents the weighted average of all node positions x in the layout i The specific transformation operation is as follows:

[0098]

[0099] (3) Depth diffusion model pre-training for graph layout

[0100] For the pre-training of the graph layout model, the present application collects the layout results of several graphs, including different types of graphs and the layout results obtained by using different layout algorithms for the same graph. In the training process, the present application selects a random time step t, adds noise ∈ sampled from the prior distribution N(0, 1) to the equilibrium state graph layout X0, and the signal-to-noise ratio is determined by t and the predefined variance mode β1, …, β T , and the disturbance obtains the chaotic state layout X t . Then, the disturbed layout X t is input into the denoising network ∈ θ , and the ability of the network to predict the layout noise is trained using ∈ as the supervision signal. When ∈ θ can accurately predict the noise on X t , the model can restore the chaotic state graph layout to the equilibrium state high-quality graph layout through the reverse process defined above.

[0101] (4) The user uploads the graph data in JSON format through the system front end;

[0102] The user loads and uploads the graph data in JSON format to the system back end through the system front end, and the system back end automatically parses the data after acceptance, and enhances the original graph data, enriches the node attributes in the graph, and enhances the topology of the graph. The system will use the pre-trained model to generate an initial layout without any constraints for the uploaded graph data. The output result of the model will be automatically visualized in the system.

[0103] (5) The user constructs the expected layout constraint through the system front end interaction;

[0104] The system supports users to quickly build layout constraints to guide the generation of graph layout according to their expectations of graph layout. Users can select specific nodes in the system front-end visualization result to move them to their desired positions, or collectively move multiple nodes by marqueeing and dragging. The system control panel is used for users to add desired layout constraint information for this graph. In order to help users efficiently build graph layout constraints, the system front-end provides four common interactive operations: horizontal flip, vertical flip, zoom, and rotate, as well as four commonly used layout modes to arrange selected nodes into corresponding shapes: straight line, spiral, circle, and regular polygon. By building layout constraints, users can control the graph layout generation process, guide the model to generate layout results that match the layout constraints, and thus generate graph layout visual forms that meet user expectations.

[0105] (6) The system backend parses the layout constraints and generates graph visualization layout based on the pre-trained model;

[0106] The system backend parses the layout constraints built by the user in step (5) into model-acceptable inputs and integrates them into each step of the layout generation process. At each time step t in the reverse process in step (2), the invention combines the user-provided constraints with the model's generation results. Considering the rotational and translational invariance of graph layout, the layout constraints and the model's generation results usually have deviations in the overall position in two-dimensional space. In order to facilitate the effective coordination between the layout constraints and the model's generation results, the invention resamples at each time step t, aligns the layout constraints and the model's generation results by X t-1 X t This process is repeated n times to ensure that the model can align the layout constraints and the generation results. Through the constrained generation process, the model seamlessly integrates the user's layout constraints into the model's generation results.

[0107] (7) The system front-end displays the graph visualization layout results and supports users to explore the outputs of different models;

[0108] The system front-end visualization module visualizes the graph layout results generated by the system backend model. The visualization part of the system is completed based on the visualization toolkit D3. The graph visualization form drawing result can be automatically adjusted according to the window size. After displaying the graph visualization form on the computer screen, users can interact with the graph visualization form, rotate, zoom, or interactively select a part of the data to view the details, and thus explore the graph layout results. At the same time, users can select pre-trained models for different data sets and adjust layout hyperparameters to explore different model graph layout results.

[0109] (8) Users iteratively create layout constraints to build data streaming visual exploration processes;

[0110] The system supports users to iteratively create layout constraints to build a dataflow visual exploration process. The system draws lessons from the classic dataflow model. Users define layout constraints in each module of the dataflow according to step (5). The system generates a graph layout result based on the provided constraints. The layout result serves as the input of the next module. When the data in a module is updated, the input and output of all subsequent modules connected to it will be automatically updated. Meanwhile, in order to support the reuse of graph layout constraints, the system also provides a module replication function. Users can copy part of the modules in the dataflow. The input of the copied sub-dataflow is determined by the new connection relationship. Users can iteratively create layout constraints to adjust the graph layout result until a graph layout visualization form that meets their needs is obtained.

[0111] Figure 2 The way of graph topology enhancement in the embodiment of the application is shown, taking the yellow node as an example. The left graph highlights the original edges connected to it. The middle graph highlights the 2-order local edges, and the right graph highlights the global face variation, which is connected according to the predefined distance threshold τ.

[0112] Figure 3 The constraint generation process of the graph layout in the embodiment of the application is shown, taking a layout X T with noise as an example. The layout is combined with the constraints derived from the user's layout expectations, and after repositioning the centroid of the layout, the tensor X representing the node position is obtained. Further graph enhancement generates the tensors E and A representing the edge type and the adjacency matrix of the graph. The Laplacian eigenvectors are used for position encoding to obtain the tensor H representing the initial node embedding. The above tensors are input into the noise predictor for denoising. In order to effectively coordinate the given constraints and model generation, n resamplings are performed at each time step. The complete generation process is iterated T times, and finally X0 is generated as the output of the model.

[0113] Figure 4 The system front-end graph visual exploration and layout constraint interaction construction interface in the embodiment of the application is shown. Each module of the exploration dataflow corresponds to a layout constraint and graph data. The model generation result will be directly displayed in the form of graph visualization. As shown in FIG. 8, the system graph visualization panel is used to visualize the generation result of the back-end model. The yellow nodes in the graph are the constraint information added by the user. Figure 4

[0114] Figure 5 The comparison between the graph layout generation result and the reference layout in the embodiment of the application is shown.

[0115] Figure 6 ​The generation effect of the tree graph in the embodiment of the application is shown in comparison with the reference layout, wherein the model used is trained on the benchmark graph dataset, and the generalization ability of the model is illustrated.

[0116] Figure 7 The generation of a plurality of different types of graph layouts in the embodiment of the application is shown in comparison with the reference layout, all graph layouts are generated by one model, and the ability of the model to simultaneously learn different types of graphs and the corresponding visualization layout results is verified.

[0117] Figure 8 The layout effect obtained by using the layout constraint in the embodiment of the application is shown, and the yellow nodes represent the layout constraints provided to the model. The two layouts in each row depict the same graph, all graph layouts are generated by one model, and the constraint generation ability of the model is verified.

Claims

1. A method for controllable layout interactive generation of graph visualization based on a depth diffusion model, characterized in that, The method comprises the following eight steps: (1) graph data augmentation; (2) graph layout model structure design; (3) graph layout model pre-training; (4) a user uploads graph data in JSON format through a system front end; (5) the user constructs desired layout constraints through system front end interaction; (6) a system back end analyzes the layout constraints and generates a graph visualization layout based on a pre-trained model; the pre-trained model is a graph visualization generation model based on a diffusion model; at each time step t in the reverse process of the diffusion model, the constraints constructed by the user are combined with the generation results of the model; Sampling a layout X with noise T Starting with ~N(0, 1), the layout X T is merged with user-built constraints, after repositioning the centroid of the layout, resulting in a tensor X representing node positions; through graph enhancement, tensors E and A representing edge types and the adjacency matrix of the graph are generated; using Laplacian eigenvectors for position encoding, a tensor H representing initial node embeddings is obtained; tensors X, E, A, and H are input into a noise predictor to predict noise ∈ θ , and according to the noise ∈ θ , the merged layout is denoised to obtain the layout X T-1 ; resampling is performed, and through the forward process of the diffusion model, the layout X T-1 is returned to the layout X T , and the process of merging the layout X T with user-built constraints is executed again to obtain the layout X T-1 , and the process is repeated n times; the complete generation process is iterated T times, and finally the layout X0 is generated as the output of the model; (7) the system front end displays the graph visualization layout result and supports user exploration of different model outputs; (8) the user iteratively creates layout constraints to build a data flow visual exploration process.

2. The method of claim 1, wherein, The specific method of step (1) is: Graph data augmentation involves node feature augmentation and graph structure augmentation; a Laplacian position encoding method is used as the initial node feature; Laplace position encoding L is defined by factorizing the graph Laplacian matrix: Here, I is an N*N identity matrix, D is a node degree matrix, B represents an adjacency matrix, and Lambda and U represent eigenvalues and eigenvectors, respectively; the k smallest non-trivial eigenvectors are used as the initial node features, and k is a hyperparameter; The original graph topology is augmented to improve the message passing efficiency between nodes; graph structure augmentation involves establishing connections between nodes, which is divided into three types: original edges, local edges and global edges; (a) The original edge represents an edge from the original graph, which preserves the inherent connection in the given graph and directly reflects its topology; (b) The local edge connects each node to its n-order neighbor as an extension of the original graph topology, which expands the node's perception range and helps the model effectively generate the local structure around each node, where n is determined by the hyperparameter adjustment of the grid search; (c) The global edge is used to dynamically connect nodes within a distance threshold tau, and the radius cutoff value is determined according to the specific scale of the layout. 3.The method of claim 1, wherein, The specific method of step (2) is: The graph layout model structure design includes two processes: the forward process and the reverse process; The graph layout is defined as a system composed of multiple nodes and edges in two-dimensional space; The forward process involves transforming the system from a balanced state to a chaotic state, while the reverse process optimizes the system from a chaotic state to restore it to a balanced state; The equilibrium state represents an orderly layout, while the chaotic state represents a disordered and noisy layout; define X0as the graph layout in the equilibrium state; X t is the graph layout in the chaotic state (t = 1,..., T), where t represents the index of the diffusion step; the overall goal of the model is to learn how to recover X0from X T ; for a given graph G, the forward process gradually introduces a slight Gaussian noise into the layout X0; through several steps of perturbation, it is transformed into a completely noisy layout X T , while ensuring that X T follows the prior distribution N(0, 1); in each step, the signal-to-noise ratio is determined by a fixed variance pattern β1,..., β T ; represent the forward process as a Markov chain: i.e. for a given graph G, a state X t In the previous state X t-1 is subject to a Gaussian distribution where denotes the mean of the Gaussian distribution, β t I denotes the covariance matrix, where I is the N x N identity matrix; For t independent and identically distributed Gaussian noise N(μ, σ 2 ), which is still a Gaussian noise after superposition; therefore, define α t = 1-β t and then X t can be sampled given X0in any time step t. That is, for a given graph G, the state X t Under the condition of the initial state X0, it is subject to a Gaussian distribution This means that for any state X t of the graph G, it is possible to add a corresponding Gaussian noise on the initial state X0 The inverse process is an iterative optimization process that samples a completely noisy layout X from the prior distribution N(0, 1) T This process involves T denoising steps that gradually optimize the layout system from a chaotic state to an equilibrium state; the inverse process is also a Markov chain: That is, for a given graph G of X t-1 , the state is subject to a Gaussian distribution t under the condition that X where μ θ is a function for estimating the mean of the distribution, denotes an arbitrary variance; μ θ is calculated as follows: ∈ θ is a trainable function used to predict noise on layout X t , which is also the training target: The function is a loss function that measures the difference between the predicted noise ∈ θ at time t and the true noise ∈ θ By minimizing this loss function, a model that accurately predicts future noise ∈ t can be trained. An equivariant graph neural network is employed to parameterize θ The equivariant graph neural network is composed of multiple layers of equivariant convolution layers: The first equation represents the message passing mechanism in equivariant graph neural networks: each node i sends a message m to its neighbor node j ij ; the content of the message includes the hidden state of the current layer of nodes i and j and the squared distance between them and the edge feature e ij ; is a fully connected neural network that aggregates the edge endpoint embeddings and the node distance; The second equation represents the mechanism by which the isometric mapping neural network updates the hidden state of a node: the next-layer digital state of node i is obtained by summing the messages from all neighbors and passing them through a fully connected neural network ; here N(i) denotes the set of neighbors of node i. The third formula represents the mechanism of updating the coordinates of the nodes of the isometric graph neural network: the next layer digital coordinates of node i are obtained by the current coordinates plus a correction term; the correction term takes into account the information of the relative positions between nodes and the updated hidden state; is a fully connected neural network responsible for projecting the updated end-point embedding and edge embedding into a two-dimensional space; where: l is the index of the layer; denotes the node embedding vector, the initial value is obtained from the Laplacian eigen-vector position encoding; corresponding to the node coordinates, quantifies the Euclidean distance between node i and node j; e ij denotes the type of edge, including original, local and global edge; N(i) denotes the neighborhood of node i; is a fully connected neural network used to aggregate the edge endpoint embedding and node distance; by using the information of the neighbor nodes to update the node embedding, and using the graph isomorphism neural network for parameterization; the node coordinates are updated by the weighted difference of the invariant features ; ∈ θ ensure the rotation and translation invariance of the graph layout; Transform the centroid of the X0 layout to the origin to obtain the graph layout. The centroid represents the x-coordinate of all nodes in the layout. i The weighted average value is transformed as follows:

4. The method of claim 1, wherein, The specific method of step (3) is: For the pre-training of the graph layout model, layout results of several graphs are collected, including layout results of different types of graphs and layout results of the same graph obtained using different layout algorithms. During the training process, a random time step t is selected, and noise ∈ sampled from the prior distribution N(0,1) is added to the graph layout X0 in the equilibrium state. The signal-to-noise ratio is determined by t and the predefined variance pattern β1, ..., β T The decision is made that the perturbation results in a chaotic layout X. t Next, the perturbed layout X t Input to denoising network ∈ θ In the middle, ∈ is used as a supervision signal to train the network's ability to predict layout noise; when ∈ θ Able to accurately predict X t When noise is present, the model restores the disordered graph layout to a balanced, high-quality graph layout through a defined reverse process.

5. The method of claim 1, wherein, The specific method of step (4) is: The user loads and uploads graph data in JSON format to the system back end through the system front end, and the system back end automatically parses the data after receiving it, and augments the original graph data, enriches the node attributes in the graph, and enhances the topology of the graph; the system will use the pre-trained model to generate an initial layout without any constraints for the uploaded graph data; The output results of the model are automatically visualized in the system.

6. The method of claim 1, wherein, The specific method of step (5) is: Support users to quickly build layout constraints according to their expectations of graph layout, users select specific nodes to move to their desired positions in the system front-end visualization results, or collectively move multiple nodes by marquee and drag; the system control panel is used for users to add desired layout constraint information for this graph; in order to help users to efficiently build graph layout constraints, the system front-end provides four common interactive operations: horizontal flip, vertical flip, zoom, and rotate, and four commonly used layout modes to arrange selected nodes into corresponding shapes: straight line, spiral, circle, and regular polygon; by building layout constraints, users control the graph layout generation process, guide the model to generate layout results that match the layout constraints, and generate graph layout visual forms that meet user expectations.

7. The method of claim 3, wherein, The specific method of step (6) is: The system backend resolves the layout constraints built by the user in step (5) into model-acceptable inputs and fuses them into the layout generation process at every step; at every time step t in the inverse process in step (2), the user-provided constraints are merged with the generation results of the model; at every time step t, resampling is performed by X is returned to t-1 X is returned to t This process is repeated n times to ensure that the model can align the layout constraints and the generation results. 8.The method of claim 1, wherein, The specific method of step (7) is: The system front-end visualization module visualizes the graph layout results generated by the system back-end model; the visualization part of the system is completed based on the visualization toolkit D3; the graph visualization form drawing result is automatically adjusted according to the window size; After displaying the graph visualization form on the computer screen, users interact with the graph visualization form, rotate, zoom, or interactively select a part of the data to view the details, and explore the graph layout results; at the same time, users select different data sets to obtain pre-trained models and adjust layout hyperparameters to explore different graph layout results of the models. 9.The method of claim 1, wherein, The specific method of step (8) is: The system supports users to iteratively create layout constraints to build a data flow visual exploration process; the system draws on the classic data flow model, users define layout constraints in each module of the data flow according to step (5), the system generates graph layout results based on the provided constraints, and the layout results are used as the input of the next module; when the data in a module is updated, all subsequent module inputs and outputs connected to it will be automatically updated; at the same time, in order to support the reuse of graph layout constraints, the system also provides a module duplication function, users can duplicate part of the modules in the data flow, and the input of the duplicated sub-data flow is determined by the new connection relationship; users can iteratively create layout constraints to adjust the graph layout results until they get the graph layout visualization form that meets their needs.

Images