Visualization system and method for relational analysis of relational data

Abstract

The present disclosure proposes a visualization system for relationship analysis of relationship data, comprising: a receiving module configured to receive a relationship data matrix with dimension MxM; a processing module configured to reorder the relationship data matrix by applying a trained reordering model to obtain a reordered matrix, such that at least one of the following structures in the relationship data is highlighted in the reordered matrix: a cluster, a bipartite graph, a central node, and a path structure, wherein the trained reordering model is trained using multiple instances of relationship data matrices with dimension NxN and their corresponding optimal reordered matrices as training samples, and wherein N is greater than or equal to M; and a display module configured to display the relationship data matrix and the reordered matrix on a display plane. The visualization system can effectively highlight different structures existing in the relationship data.

Description

Technical Field

[0001] This application relates to the field of data visualization technology, and in particular to visualization systems and methods for performing relational analysis on relational data. Background Technology

[0002] The analysis of relational data is important because it allows users to identify detailed relationships within the data, enabling them to make further decisions based on these relationships. Visualizing relational data allows users to more intuitively "see" the relationships within the data and is a common method for analysis. How to analyze relational data and visualize the relationships within it is an important research topic in the field of visualization. Relational data analysis reveals the relational information contained within the data, and visualizing these relationships reveals the structure and trends within the data, making it easier to make relevant decisions. Currently, a common method for analyzing relational data is to use relational data matrices and then visualize them. However, existing relational data matrices often fail to clearly reflect the structure of the relational data, and visualization methods often result in disorganized relational data matrices, making it difficult for users to discern the underlying structure. Summary of the Invention

[0003] The following description includes exemplary methods, systems, techniques, and sequences of instructions embodying the techniques of this invention. However, it should be understood that the described invention can be practiced in one or more aspects without these specific details. In other instances, well-known protocols, structures, and techniques have not been shown in detail so as not to obscure the invention. Those skilled in the art will understand that the described techniques and mechanisms can be applied to various architectures for visualizing samples.

[0004] According to one aspect of the present invention, a visualization system for performing relational analysis on relational data is proposed. The system includes: a receiving module configured to receive relational data, the relational data being represented using a relational data matrix of dimension M×M, wherein rows and columns of the relational data matrix represent objects, and the value of an element of the relational data matrix represents the degree of relation between the object corresponding to the row of the element and the object corresponding to the column of the element; a processing module configured to apply a trained reordering model to the relational data matrix to reorder it to obtain a reordered matrix, such that the reordered matrix highlights at least one of the following structures in the relational data: clustering, bipartite graph, centroid, and path structure, wherein the trained reordering model is trained using multiple instances of relational data matrices of dimension N×N with different relations and their corresponding optimal reordered matrices as training samples, and wherein N is greater than or equal to M; and a display module configured to display the relational data matrix and the reordered matrix on a display plane. This visualization system effectively highlights the different structures existing in relational data, and uses a neural network model to effectively reorder different relational data, thereby improving the effectiveness and applicability of the visualization results. Furthermore, the visualization of the reordering matrix makes it easier for users to see the structure existing in the relational data, thus providing users with a more accurate and universal relational data analysis tool.

[0005] According to another aspect of the present invention, a visualization method for performing relational analysis on relational data is proposed. The visualization method includes: receiving relational data, the relational data being represented using a relational data matrix of dimension M×M, wherein rows and columns of the relational data matrix represent objects, and the value of an element of the relational data matrix represents the degree of relationship between the object corresponding to the row of that element and the object corresponding to the column of that element; reordering the relational data matrix by applying a trained reordering model to obtain a reordered matrix, such that the reordered matrix highlights at least one of the following structures in the relational data: clustering, bipartite graph, centroid, and path structure, wherein the trained reordering model is trained using multiple instances of relational data matrices of dimension N×N with different relations and their corresponding optimal reordered matrices as training samples, and wherein N is greater than or equal to M; and displaying the relational data matrix and the reordered matrix on a display plane. This visualization method effectively highlights the different structures existing in relational data, and by using a neural network model, it can effectively reorder different relational data, thereby improving the effectiveness and applicability of the visualization results. Furthermore, the visualization of the reordered matrix makes it easier for users to see the structure existing in the relational data, thus providing users with more accurate and universal relational data analysis.

[0006] According to another aspect of the present invention, a computer program product is provided, the computer program product comprising program instructions executable by a computing device to cause the computing device to perform the method described above. Attached Figure Description

[0007] A better understanding of the invention itself, its preferred modes of use, objectives, features, and advantages can be achieved by referring to the following detailed description of illustrative embodiments, in which: Figure 1 An example of a relational data matrix is ​​shown; Figure 2 The diagram illustrates how clustering, bipartite graphs, central nodes, and path structures are represented in a relational data matrix. Figure 3 A structural block diagram of a visualization system for performing relational analysis on relational data according to an embodiment of the present invention is shown; Figure 4 This illustrates a reordering model structure; Figure 5 The example shows an 8×8 optimal reordering matrix containing a clustering structure, and three corresponding relational data matrix examples obtained by swapping the corresponding rows and columns three times. Figure 6 Four examples of optimal reordering matrices containing clustering structures are shown, formed by replacing all elements of the squares of random size, number, and position on the main diagonal of an 8×8 initial matrix with 1. Figure 7 The optimal reordering matrix with two dimensions of 8×8 containing a cluster structure is shown; Figure 8 A flowchart of a method for obtaining a reordering matrix of dimension M×M according to an embodiment of the present invention is shown; Figure 9 This demonstrates the use of a relational data matrix. Figure 8 An example of an intermediate result of the process reordering shown; Figure 10 An example is shown where each element of a relational data matrix and a reordering matrix is ​​displayed on a display plane using squares; Figure 11 A flowchart illustrating a visualization method for performing relational analysis on relational data according to an embodiment of the present invention is shown. Detailed Implementation

[0008] Embodiments of the present invention will now be described with reference to the accompanying drawings. Numerous specific details are set forth in the following description to provide a more complete understanding of the invention. However, it will be apparent to those skilled in the art that implementations of the invention may not include some of these specific details. Furthermore, it should be understood that the invention is not limited to the specific embodiments described. Rather, the invention can be practiced using any combination of the following features and elements, regardless of whether they relate to different embodiments. Moreover, the specific steps of the methods in different embodiments are not strictly ordered; that is, in a method comprising a first step and a second step, the first step may be performed before the second step, or vice versa. Therefore, the following aspects, features, embodiments, and advantages are for illustrative purposes only and should not be construed as elements or limitations of the appended claims unless expressly stated in the claims.

[0009] As mentioned earlier, existing relational data matrices often fail to clearly reflect the structure within relational data, and visualization methods often display them in a disorganized manner, making it difficult for users to discern the underlying structure. This is because the default arrangement of relational data matrices typically fails to accurately reflect the structure existing within the data. Rearranging the rows and columns of a relational data matrix can group related or similar objects together, thus more clearly revealing the relational structure between the data. Therefore, reordering relational data matrices is essential.

[0010] Figure 1 An example of a relational data matrix is ​​shown. This matrix is ​​a 5x5 matrix representing 5 objects in the relational data. The value of the element in the i-th row and j-th column indicates the degree of relationship between the i-th and j-th objects; the larger the value, the stronger the relationship. For example... Figure 1 The relationship between the first and second objects is 1, the relationship between the third and fourth objects is 0.5, and the relationship between the third and fifth objects is 0. It is evident that each relational data matrix is ​​a square matrix, and a symmetric diagonal matrix at that. Objects in the relational data can be, for example, large amounts of images, text, audio, and / or video for relational analysis, a large number of people in a social network, species in species relational analysis, etc.

[0011] Figure 2 This illustrates how clustering, bipartite graphs, central nodes, and path structures are represented in a relational data matrix. Figure 2 The example uses a binary relational data matrix. Those skilled in the art will understand that, since the values ​​of the elements in a relational data matrix can be real numbers, the above structure is not limited to a binary relational data matrix; similar structures expressed by other real number element values ​​are also possible. Figure 2In the diagram, clustering structure 201 is represented as a square on the main diagonal of the relational data matrix. This square structure represents densely connected blocks in the relational data that are related to each other. For example, in social network analysis, the elements of the relational data matrix can represent friend relationships, and the objects corresponding to the square elements can represent a group, with the square representing a shared group of friends. Bipartite graph structure 202 is represented as a rectangle in the relational data matrix that does not touch the main diagonal. This rectangular structure represents two sets of objects in the relational data that are related to each other. For example, in ecology, the elements of the relational data matrix can represent predator-prey relationships, and the objects corresponding to the rectangle elements can represent two groups of species, one representing predator species and the other representing prey species. Central node structure 203... The horizontal and vertical lines intersecting the main diagonal of the relational data matrix are represented as unnecessary horizontal and vertical lines that span the entire matrix. These lines represent relationships between an object and a large number of other objects. For example, in social network analysis, elements of the relational data matrix can represent mutual acquaintances, and the objects corresponding to the horizontal and vertical lines can represent a group that all know the same high-influence individual. The path structure 204 is represented as a straight line parallel to the main diagonal of the relational data matrix. This parallel line structure represents one or more connection paths in the relational data. For example, in social network analysis, elements of the relational data matrix can represent information transmission, and the objects corresponding to the straight lines can represent individual objects in the information propagation path.

[0012] Existing relational data analysis systems often employ statistical, spectral, graph-theoretic, and heuristic methods when performing matrix reordering for relational data analysis. These methods are typically developed and evaluated on small-scale examples of relational data matrices and can only highlight the clustering and bipartite graph structures within the matrix. This approach sometimes results in reordered relational data matrices that fail to highlight the different categories, quantities, and sizes present in the relational data, particularly the central nodes and path structures.

[0013] Another type of existing relational data analysis system uses a deep neural network-based reordering method when reordering relational data matrices. This method, for an input relational data matrix, first obtains multiple reordering results using various reordering methods mentioned in the previous type of relational data analysis system. The input relational data matrix and the multiple reordering results are then used as multiple sets of training data. A deep neural network is then used to learn the features and structure of the relational data matrix, resulting in a reordering model. The drawback of this reordering model is that it only learns the features and structure of the input relational data matrix, and can only reorder that specific input relational data matrix, not other relational data matrices. For each other relational data matrix, a new reordering model needs to be trained, which greatly limits its application.

[0014] To address the problems of existing technologies, this invention discloses a visualization system for relational data analysis. This system reorders a received M×M relational data matrix using a trained reordering model to obtain a reordered matrix. The reordered matrix highlights at least one of the following in the relational data: clustering, bipartite graphs, central nodes, and path structures. The trained reordering model is trained using multiple N×N relational data matrix instances with different relations and their corresponding optimal reordered matrices, where N is greater than or equal to M. The system can also display the relational data matrix and the reordered matrix on a display plane. Each element of the matrix is ​​represented by a shape, with different shades or colors indicating the degree of relational relationship. This system effectively highlights different structures in relational data of various scales, especially large-scale data. Using a neural network model, it can effectively reorder different relational data, thereby improving the effectiveness and applicability of the visualization results. Furthermore, the visualization of the reordered matrix makes it easier for users to see the structures within the relational data, providing a more accurate and universal relational data analysis tool.

[0015] Figure 3 A structural block diagram of a visualization system 300 for performing relational analysis on relational data according to an embodiment of the present invention is shown. Figure 3 System 300 includes a receiving module 310, a processing module 320, and a display module 330. The receiving module 310 is configured to receive a relational data matrix 301. This relational data matrix 301 represents relational data using an M×M matrix format. Rows and columns of this relational data matrix represent objects, and the value of each element in the relational data matrix 301 indicates the degree of relationship between the object corresponding to that element's row and the object corresponding to that element's column. For example... Figure 1 The relational data matrix shown can then be received by the receiving module 310.

[0016] The processing module 320 is configured to apply a trained reordering model 304 to the relational data matrix 301 to reorder it to obtain a reordered matrix 305, such that the reordered matrix 305 highlights at least one of the following structures in the relational data: clustering, bipartite graph, central node, and path structure, wherein the trained reordering model 304 is trained using multiple N×N relational data matrix instances with different relations and their corresponding best reordered matrices as training samples, and wherein N is greater than or equal to M.

[0017] The display module 330 is configured to display the relational data matrix 301 and the reordering matrix 305 on the display plane.

[0018] The visualization system 300 for performing relational analysis on relational data can be implemented as an application on a general computer system, or as an application on a server system, or as a network application, or as an application on a cloud platform.

[0019] In one implementation, the reordering model 304 in the processing module can be a machine learning model, more specifically, a neural network model. This neural network model can be trained using a large number of relational data matrix instances and their corresponding optimal reordering matrices as training samples. During training, the optimal reordering matrix used for training can be compared with the reordering matrix output by the reordering model 304. The network parameters of the reordering model 304 are adjusted in reverse by calculating a loss function, thereby obtaining a network model with acceptable loss, which is then used as the reordering model 304. The loss function can be a cross-entropy loss function, a mean squared loss function, etc. Other types of loss functions known to those skilled in the art can also be used.

[0020] In one implementation, the neural network model can employ... Figure 4 The reordering model structure 400 is shown. (Example) Figure 4As shown, the reordering model structure 400 includes a fused convolutional layer 401, four identical non-downsampled residual blocks 402, an outer product layer 403, a linear layer 404, a permutation layer 405, and a permutation layer 406. This reordering model 400 adds outer product layers, permutation layers, and permutation layers to the ResNet-18 model presented by Kaiming He et al. in their paper "Deep Residual Learning for Image Recognition" (article link: https: / / arxiv.org / abs / 1512.03385) at The IEEE / CVF Computer Vision and Pattern Recognition Conference (CVPR) 2016. The fusion convolutional layer 401 of the reordering model 400 is configured to calculate the Euclidean distance between any two rows of the input N×N relational data matrix 410, obtain the distance matrix, and then fuse the relational data matrix and the distance matrix through a 1×1 convolution before outputting the result. Any unsampling residual block 402 contains two convolutional layers, two normalization layers, two non-linear activation layers, and one residual connection. Unlike ResNet-18, it does not perform feature downsampling during convolution. The outer layer 403 calculates the inner product between any two rows of each feature channel in the output of the unsampling residual block 402-2 as the similarity. The similarity matrix is ​​obtained as the feature output; the linear layer 404 receives the features output by the unsampling residual block 402-4, uses a fully connected layer to reduce the number of channels of the features to 1, and outputs them to the permutation layer 405; the permutation layer 405 receives the features output by the linear layer 404, calculates them using the Sinkhorn operator, and outputs a matrix permutation 420; the permutation layer 406 receives the relational data matrix 410 of dimension N×N input to the reordering model and the matrix permutation 420 output by the permutation layer 405, and reorders the rows and columns of the relational data matrix 410 according to the matrix permutation 420 to obtain a reordering matrix 430 of dimension N×N.

[0021] Those skilled in the art will know that Figure 4 The neural network structure shown is only one implementation of the reordering model 304; other implementations can also be used. Figure 4 Improved neural network structures can be created by replacing the outer product operation of the outer product layer with other similarity operations, or by changing the number of layers and feature dimensions of the network. Other types of neural network structures can also be used, such as neural network structures based on attention mechanisms.

[0022] After determining the neural network model structure of the reordering model 304 to be used, the model needs to be trained. Unlike existing methods, this invention aims to use a large number of N×N relational data matrix instances 302 and their corresponding optimal reordering matrices 303 as training samples. The objects in these N×N relational data matrix instances 302 have different relationships; that is, these matrices are completely different matrices, and the corresponding optimal reordering matrices 303 are not exactly the same. Once the reordering model 304 is trained, for a received M×M relational data matrix, if M is less than or equal to N, its optimal reordering matrix can be directly obtained after inputting it into the reordering model 304. In other words, the input M×M relational data matrix is ​​unrelated to the multiple relational data matrix instances 302 used for training; therefore, this reordering model 304 can be applied to the reordering of any relational data matrix less than or equal to N.

[0023] To obtain multiple relational data matrix instances 302 with different relationships and their corresponding optimal reordering matrices 303 as training samples, embodiments of the present invention first generate multiple optimal reordering matrices. These optimal reordering matrices contain various structures such as clustering, bipartite graphs, central nodes, and path structures. Then, for each specific optimal reordering matrix among the generated multiple optimal reordering matrices, the corresponding rows and columns are swapped multiple times to obtain multiple corresponding relational data matrix instances. In this way, one corresponding relational data matrix instance and that specific optimal reordering matrix form a set of training samples. Assuming that one generated optimal reordering matrix 303 corresponds to K corresponding relational data matrix instances, K sets of training samples are formed. Assuming that W optimal reordering matrices 303 are generated, then K×W sets of training samples are generated. Such a large number of training samples satisfy the above training requirements. Therefore, in one implementation, the following approach can be used: First, multiple N×N optimal reordering matrix instances 303 are generated, wherein each optimal reordering matrix instance 303 satisfies at least one of the following four structures in the salient relation data: clustering, bipartite graph, centroid, and path structure, and the multiple optimal reordering matrix instances contain all four structures. The generation of the optimal reordering matrix instances 303 with the above four structures can be done by equal distribution, for example, generating W / 4 optimal reordering matrices containing each structure from W optimal reordering matrices 303; it can also be done by probability distribution, generating optimal reordering matrices 303 containing each structure with a certain probability, and so on. Furthermore, a single optimal reordering matrix instance 303 can even contain one, two, three, or all four of the above four structures. Then, for each N×N optimal reordering matrix instance 303, perform one or more row and column swap operations multiple times. In each row and column swap operation, swap two rows and their corresponding columns to obtain multiple corresponding relational data matrix instances 302. These multiple (e.g., K) corresponding relational data matrix instances 302 and the N×N optimal reordering matrix 303 are multiple (e.g., K) training samples for the reordering model 304. Figure 5 The example 501 is an 8×8 optimal reordering matrix containing a clustering structure, and the three corresponding relational data matrix examples 502, 503 and 504 obtained by swapping the corresponding rows and columns three times. Here, (502, 501), (503, 501) and (504, 501) form the three sets of training samples for the reordering model.

[0024] In one implementation, generating multiple instances 303 of the optimal reordering matrix with dimension N×N may include: first, establishing an initial matrix of dimension N×N, where each element of the initial matrix can be 0 or set to any other irrelevant value, such as a negative number, which indicates that there is no relationship between the objects. Then, in order to obtain multiple relational data matrices containing clustering structures, the elements contained in squares of random size, number, and position on the main diagonal of the initial matrix may be replaced multiple times with non-zero real numbers.

[0025] Figure 6 Examples of optimal reordering matrices containing clustering structures are shown, formed by replacing the elements of squares of random size, number, and position on the main diagonal of an initial 8×8 matrix with 1. In optimal reordering matrix 601, the random size of the squares (the number of elements in the row corresponding to the square's side) is 4, the number of squares is 1, and the position of the squares is 1 (the first element on the diagonal is the top-left corner of the square). Compared to optimal reordering matrix 601, the cluster size changes in optimal reordering matrix 602, and the cluster size, number, and position change in optimal reordering matrices 603 and 604. In one implementation, the shapes of the random sizes, numbers, and positions in the optimal reordering matrices do not overlap within the same relational data matrix, for example... Figure 6 In 603, the three squares do not overlap. In another implementation, the shapes of random sizes, quantities, and positions in the optimal reordering matrix can overlap within the same optimal reordering matrix, for example... Figure 6 In 604, two squares can also overlap each other.

[0026] Similarly, to form multiple optimal reordering matrices containing bipartite graph structures, the elements contained in rectangles of random size, number, and position that do not touch the main diagonal of the initial matrix can be repeatedly replaced with non-zero real numbers. To form multiple optimal reordering matrices containing center node structures, the elements contained in horizontal and vertical lines of random size, number, and position that intersect the main diagonal of the initial matrix and do not need to cross the entire matrix can be repeatedly replaced with non-zero real numbers. To form multiple optimal reordering matrices containing path structures, the elements contained in lines of random size, number, and position that are parallel to the main diagonal of the initial matrix can be repeatedly replaced with non-zero real numbers. For simplicity, specific examples are not given here. To ensure that multiple optimal reordering matrix instances contain the above four structures, all relational data matrix generation methods with the above four structures must be used.

[0027] In one implementation, the optimal reordering matrix generation methods with the above four structures can be used in combination. For example, an optimal reordering matrix can contain one, two, three, or even all four of the above four structures.

[0028] In one implementation, the values ​​of the replaced non-zero real numbers in the generated N×N optimal reordering matrix instance 303 can all be 1, representing that the optimal reordering matrix is ​​a binary matrix. In another implementation, the values ​​of the replaced non-zero real numbers in the generated N×N optimal reordering matrix instance 303 can be different real numbers between [0,1], representing that the optimal reordering matrix is ​​a continuous value matrix. In yet another implementation, in the generated N×N optimal reordering matrix instance 303, the values ​​of the replaced non-zero real numbers in some parts are all 1, while the values ​​of the replaced non-zero real numbers in other parts can be different real numbers between [0,1], representing that part of the optimal reordering matrix is ​​a binary matrix and the other part is a continuous value matrix.

[0029] In one implementation, generating multiple N×N optimal reordering matrix instances 303 may further include: in the generated multiple optimal reordering matrices containing clustering structures and centroid structures, for each clustering structure and centroid structure, the non-zero real values ​​of the elements of the matrix closer to the main diagonal of the optimal reordering matrix can be increased. That is, the values ​​of the elements of the matrix closer to the axis are defined as larger, and the values ​​of the elements of the matrix farther from the axis are defined as smaller. This makes similar rows and columns closer to each other, which conforms to human perception, is clearer in display, and allows users to better identify the structure. Furthermore, a common tool in relational data analysis is filtering elements with values ​​less than a certain threshold to reduce noise and highlight potential structures in the relationships. Using the above scheme can effectively maintain the overall structure of the relational data.

[0030] Figure 7 Two optimal reordering matrices, 701 and 702, with a dimension of 8×8 and each containing a clustering structure, are shown. In optimal reordering matrix 701, the elements closer to the main diagonal of the relational data matrix have larger values; for example, the element with the largest value (1) is located on the main diagonal. The values ​​of the next elements adjacent to the main diagonal are 0.75, 0.8, and 0.9, all greater than the element farther from the main diagonal (0.5). The element furthest from the main diagonal has the smallest value (0.25). Optimal reordering matrix 702 violates the principle that elements closer to the main diagonal of the relational data matrix have larger values ​​in the clustering structure.

[0031] In one implementation, generating multiple instances 303 of optimal reordering matrices with dimensions N×N may further include: in the generated multiple relational data matrices containing bipartite graph structures, for each bipartite graph structure, the non-zero real values ​​of elements in the matrix closer to any diagonal or antidiagonal of the relational data matrix can be increased. This brings similar rows and columns closer together, which aligns with human perception and makes the display clearer, allowing users to better identify the structure. Furthermore, a common tool in relational data analysis is filtering elements with values ​​less than a certain threshold to reduce noise and highlight potential structures within the relationships. Using the above scheme can effectively preserve the overall structure of the relational data.

[0032] In one implementation, generating multiple N×N optimal reordering matrix instances 303 may further include adding noise to the generated multiple optimal reordering matrices 701. For a generated optimal reordering matrix 701, various types and levels of noise can be added using different methods to form multiple different optimal reordering matrices with added noise.

[0033] In one implementation, for a binary matrix, the type of noise added can be salt-and-pepper noise, which means that for the elements of the optimal reordering matrix designated as noise, the element values ​​are reversed to indicate whether they are 0. In another implementation, for a continuous matrix, the type of noise added can be uniform noise, which means that for the elements of the optimal reordering matrix designated as noise, the element values ​​are replaced with a uniformly distributed random real number in the interval [0,1].

[0034] In one implementation, adding noise can be done by adding random noise that does not contain clustering, bipartite graphs, center nodes, path structures, or other possible structures. Specifically, multiple different noise levels p% are selected, and p / 2% of the elements in the upper triangular matrix of the optimal reordering matrix are randomly selected as noise. Noise is added according to the type of noise, and then the noise is symmetrically distributed to the elements of the lower triangular matrix according to the symmetry of the optimal reordering matrix to obtain the optimal reordering matrix after adding noise. In another implementation, adding noise can be done by adding structural noise that is similar across different rows and columns, containing small clustering and bipartite graph structures. Specifically, a set of row noise of size L is generated, where L is less than M. Each row noise in the set is a 1-row, M-column matrix, and m positions are randomly selected from the M positions corresponding to the M columns as noise positions, where m is less than M and is a positive integer that varies with the noise level. For each row of the optimal reordering matrix, a row noise is randomly selected from the row noise set, and its corresponding m noise positions are designated as noise positions in the relational data matrix. Then, based on the symmetry of the optimal reordering matrix, the noise positions in the upper triangular matrix and the noise positions in the lower triangular matrix are superimposed. Finally, noise is added according to the type of noise to obtain the optimal reordering matrix after adding noise.

[0035] After training the reordering model 304 using the multiple sets of training samples generated above, the resulting model can reorder relational data matrices of dimension M×M. However, since M is less than or equal to N, it is necessary to expand the relational data matrix of dimension M×M into an N×N dimension matrix input, and restore the output N×N dimension reordering matrix to a reordering matrix of dimension M×M. Therefore, in one embodiment, the processing module 320 applies the trained reordering model 304 to the relational data matrix 301 to reorder it to obtain the reordered matrix 305, including: in response to the fact that the dimension M of the received relational data 301 is less than N, the received relational data matrix 301 can be expanded into a matrix of dimension N×N, wherein the elements in rows M+1 to N and columns M+1 to N are filled with 0; the expanded matrix of dimension N×N is input into the input of the reordering model 304 to obtain the corresponding reordered matrix of dimension N×N; the NM rows and NM columns corresponding to the expanded elements filled with 0 in the corresponding reordered matrix of dimension N×N are deleted to obtain the corresponding reordered matrix of dimension M×M 305. Figure 8 A flowchart of a method 800 for obtaining a reordering matrix of dimension M×M according to an embodiment of the present invention is shown. Figure 8 In step 810, it is determined whether the dimension M of the received relational data matrix is ​​less than N. If so, proceed to step 820. Otherwise, proceed to step 830.

[0036] In step 820, the received relational data matrix 301 is expanded into a matrix of dimension N×N, wherein the elements in rows M+1 to N and columns M+1 to N are filled with 0.

[0037] In step 830, the matrix is ​​expanded to an N×N dimension and used as input to the reordering model 304 to obtain the corresponding N×N dimension reordering matrix.

[0038] In step 840, after deleting the expanded rows and columns from the corresponding N×N reordered matrix, a corresponding M×M reordered matrix is ​​obtained. This M×M reordered matrix is ​​then returned, ending the process. Figure 9 This demonstrates the use of a relational data matrix. Figure 8 The example shown is an intermediate result of the process reordering, where M is 6 and N is 8. Figure 9 Zhong901 is a 6×6 relational data matrix that receives data. Figure 9 Use Figure 8 The process involves filling the elements in rows 7 and columns 7 and 8 of the relational data matrix 901 with 0 to expand it into an 8×8 matrix 902. Then, the 8×8 matrix 902 is used as input to the reordering model to obtain the corresponding 8×8 reordering matrix 903. Finally, the expanded rows and columns of the 8×8 reordering matrix 903 are deleted to obtain the corresponding 6×6 reordering matrix 904.

[0039] Those skilled in the art will know that M and N are both positive integers. If a large-scale or even larger-dimensional relational data matrix is ​​to be processed, N can be set to a larger positive integer during the training of the reordering model to generate higher-dimensional training samples. This will enable the reordering of the larger-dimensional relational data matrix. Furthermore, since the trained reordering model is used, the input of a large-scale relational data matrix will not generate a large amount of computation, and the reordering result of the input relational data matrix can be obtained quickly.

[0040] In one implementation, a display module can be used to display the relational data matrix and the reordering matrix on a display plane. In another implementation, each element of the relational data matrix and the reordering matrix can be represented by a geometric shape of the same color, where the shade of the color indicates the degree of the relation. Figure 10 An example is shown where each element of the relational data matrix 1001 and the reordering matrix 1002 is displayed on a display plane using squares, where grayscale represents the degree of the relation, with darker grayscale indicating a stronger relation. Those skilled in the art can more easily discern the different structures present in the relational data from the displayed reordering matrix 1002, thereby enabling more accurate further analysis of the data.

[0041] In another implementation, each element of the relational data matrix and the reordering matrix can be represented using a geometric shape of a different color, where the different colors of the geometric shape indicate the degree of the relation.

[0042] Based on the same inventive concept Figure 11 A flowchart of a visualization method 1100 for performing relational analysis on relational data according to an embodiment of the present invention is shown. Figure 11 In step 1110, relational data is received, represented using an M×M relational data matrix. Rows and columns of this matrix represent objects, and the values ​​of its elements indicate the degree of relationship between the objects corresponding to the row and column of that element. In step 1120, a trained reordering model is applied to the relational data matrix to reorder it, resulting in a reordered matrix that highlights at least one of the following structures in the relational data: clustering, bipartite graph, centroid, and path structure. The trained reordering model is trained using multiple N×N relational data matrix instances with different relationships and their corresponding optimal reordered matrices as training samples, where N is greater than or equal to M. In step 1130, the relational data matrix and the reordered matrix are displayed on a display plane.

[0043] In one implementation, the multiple relational data matrix instances with different relationships and their corresponding optimal reordering matrices used in the reordering model are generated as follows: First, multiple optimal reordering matrix instances of dimension N×N are generated, wherein each optimal reordering matrix instance satisfies the following four structures: clustering, bipartite graph, central node, and path structure, and the multiple optimal reordering matrix instances contain the four structures; then, for each of the multiple optimal reordering matrix instances of dimension N×N, the following steps are performed: the optimal reordering matrix is ​​subjected to one or more row and column swapping operations multiple times, in which two rows and their corresponding columns are swapped in each row and column swapping operation to obtain multiple corresponding relational data matrix instances, wherein the multiple corresponding relational data matrix instances and the optimal reordering matrix constitute multiple sets of training samples for the reordering model.

[0044] In one implementation, generating multiple instances of optimal reordering matrices of dimension N×N includes: establishing an initial matrix of dimension N×N, where each element of the initial matrix is ​​0; repeatedly replacing the elements contained in squares of random size, number, and position on the main diagonal of the initial matrix with non-zero real numbers to form multiple optimal reordering matrices containing clustering structures; repeatedly replacing the elements contained in rectangles of random size, number, and position on the initial matrix that do not touch the main diagonal with non-zero real numbers to form multiple optimal reordering matrices containing bipartite graph structures; repeatedly replacing the elements contained in horizontal and vertical lines of random size, number, and position on the initial matrix that intersect the main diagonal but do not need to cross the entire matrix with non-zero real numbers to form multiple optimal reordering matrices containing center node structures; and repeatedly replacing the elements contained in lines of random size, number, and position on the initial matrix that are parallel to the main diagonal with non-zero real numbers to form multiple optimal reordering matrices containing path structures.

[0045] In one implementation, generating multiple instances of optimal reordering matrices with dimension N×N further includes: in the generated multiple optimal reordering matrices containing clustering structures and / or centroid structures, for each clustering structure and / or centroid structure, using the main diagonal of the optimal reordering matrix as an axis, making the non-zero real values ​​of the elements of the matrix closer to the axis larger; and in the generated multiple optimal reordering matrices containing bipartite graph structures, for each bipartite graph structure, using any diagonal or antidiagonal of the optimal reordering matrix as an axis, making the non-zero real values ​​of the elements of the matrix closer to that axis larger.

[0046] In one implementation, generating multiple instances of the best reordering matrix with dimensions N×N further includes adding noise to the generated multiple best reordering matrices.

[0047] In one implementation, step 1120 further includes: in response to the received relational data matrix having a dimension M less than N, expanding the relational data matrix into a matrix of dimension N×N, wherein the elements in rows M+1 to N and columns M+1 to N are filled with 0; inputting the expanded matrix of dimension N×N into the reordering model to obtain a corresponding reordering matrix of dimension N×N; deleting the NM rows and NM columns in the corresponding reordering matrix of dimension N×N that correspond to the expanded elements filled with 0, thereby obtaining a corresponding reordering matrix of dimension M×M.

[0048] In one implementation, the reordering model uses a neural network model, which compares the optimal reordering matrix used during training with the reordering matrix output by the neural network model, and adjusts the network parameters of the neural network model in reverse by calculating a loss function.

[0049] In one implementation, step 1130 further includes one of the following: representing each element of the relational data matrix and the reordering matrix using a geometric shape of the same color, the shade of which indicates the degree of the relation; and representing each element of the relational data matrix and the reordering matrix using a geometric shape of a different color, the different colors of which indicate the degree of the relation.

[0050] In one embodiment, embodiments of the present invention also disclose a computer system comprising: a memory; and at least one processor operatively coupled to the memory and configured to perform the methods described above.

[0051] This invention can be a system, method, and / or computer program product. The computer program product includes a computer-readable storage medium. The computer-readable storage medium carries computer-readable program instructions for causing a processor to implement various aspects of the invention. The methods of this invention can be executed on a standalone computer system, on a distributed computing system, or even on a cloud platform.

[0052] Various aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer-readable storage media according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0053] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer-readable storage media according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved.

[0054] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A visualization system for performing relational analysis on relational data, the system comprising: The receiving module is configured to receive relational data, which is represented by a relational data matrix of dimension M×M. The rows and columns of the relational data matrix represent objects, and the value of each element of the relational data matrix represents the degree of relationship between the object corresponding to the row of the element and the object corresponding to the column of the element. The objects in the relational data are images, text, voice and / or video for relational analysis, people in social networks, and species in species relational analysis. A processing module is configured to apply a trained reordering model to the relational data matrix to reorder it, thereby obtaining a reordered matrix that highlights at least one of the following structures in the relational data: clustering, bipartite graphs, centroids, and path structures. The trained reordering model is a neural network model trained using multiple N×N relational data matrix instances with different relations and their corresponding optimal reordered matrices as training samples. The multiple N×N optimal reordered matrix instances include all four structures: clustering, bipartite graphs, centroids, and path structures. Once trained, the neural network model can apply the trained reordering model to any received M×M relational data matrix without retraining the reordering model for that received relational data matrix, where N is greater than or equal to M. The display module is configured to simultaneously display the relational data matrix and the reordering matrix on a display plane, wherein each element of the relational data matrix and the reordering matrix is ​​represented by a geometric shape of a different color, and the different colors of the geometric shapes represent the degree of the relationship.

2. The visualization system according to claim 1, wherein the multiple relation data matrix instances with different relations and their corresponding optimal reordering matrix used in the reordering model of the processing module are generated in the following manner: Generate multiple instances of optimal reordering matrices with dimension N×N, wherein each instance contains at least one of the following four structures: clustering, bipartite graph, centroid, and path structure, and the multiple instances of optimal reordering matrices contain all four structures; and For each of the multiple N×N optimal reordering matrix instances, perform the following steps: The optimal reordering matrix is ​​subjected to one or more row and column swap operations multiple times. In each row and column swap operation, two rows and their corresponding columns are swapped to obtain multiple corresponding relational data matrix instances. The multiple corresponding relational data matrix instances and the optimal reordering matrix constitute multiple sets of training samples for the reordering model.

3. The visualization system according to claim 2, wherein generating multiple instances of the optimal reordering matrix with dimensions N×N includes: Create an initial matrix of dimension N×N, wherein each element of the initial matrix is ​​0; The elements contained in the squares of random size, number, and position on the main diagonal of the initial matrix are replaced with non-zero real numbers multiple times to form multiple optimal reordering matrices containing clustering structures. The elements contained in rectangles of random size, number, and position that do not touch the main diagonal of the initial matrix are repeatedly replaced with non-zero real numbers to form multiple optimal reorder matrices containing bipartite graph structures. The elements contained in the horizontal and vertical lines of random size, number, and position that intersect the main diagonal of the initial matrix are repeatedly replaced with non-zero real numbers to form multiple optimal reorder matrices containing a central node structure. as well as The elements contained in the lines of random size, number, and position parallel to the main diagonal of the initial matrix are repeatedly replaced with non-zero real numbers to form multiple optimal reorder matrices containing path structures.

4. The visualization system according to claim 3, wherein generating multiple instances of the optimal reordering matrix with dimensions N×N further includes: In the generated optimal reordering matrices containing clustering structures and / or centroid structures, for each clustering structure and / or centroid structure, with the main diagonal of the optimal reordering matrix as the axis, the non-zero real values ​​of the elements of the matrix closer to the axis are increased; and In the generated multiple optimal reordering matrices containing bipartite graph structures, for each bipartite graph structure, with any diagonal or antidiagonal of the optimal reordering matrix as an axis, the non-zero real values ​​of the elements of the matrix closer to that axis are larger.

5. The visualization system according to any one of claims 1-4, wherein the processing module applies a trained reordering model to the relational data matrix to reorder it to obtain a reordered matrix, comprising: In response to the fact that the dimension M of the received relational data matrix is ​​less than N, the relational data matrix is ​​expanded into a matrix of dimension N×N, wherein the elements in rows M+1 to N and columns M+1 to N are filled with 0; Input the expanded N×N matrix into the reordering model to obtain the corresponding N×N reordering matrix. as well as The corresponding N×N dimension reordering matrix is ​​obtained by deleting the NM rows and NM columns that are filled with 0 corresponding to the expanded elements.

6. A visualization method for performing relational analysis on relational data, the visualization method comprising: Receive relational data, which is represented by a relational data matrix of dimension M×M. The rows and columns of the relational data matrix represent objects. The value of each element of the relational data matrix represents the degree of relationship between the object corresponding to the row of the element and the object corresponding to the column of the element. The objects in the relational data are images, text, voice and / or video for relational analysis, people in social networks, and species in species relational analysis. The relational data matrix is ​​reordered using a trained reordering model to obtain a reordered matrix that highlights at least one of the following structures in the relational data: clustering, bipartite graph, centroid, and path structure. The trained reordering model is a neural network model trained using multiple N×N relational data matrix instances with different relations and their corresponding optimal reordered matrices as training samples. These multiple N×N optimal reordered matrix instances include all four structures: clustering, bipartite graph, centroid, and path structure. Once trained, the neural network model can reorder any received M×M relational data matrix without retraining the model for that matrix, where N is greater than or equal to M. The relational data matrix and the reordering matrix are displayed simultaneously on the display plane, wherein each element of the relational data matrix and the reordering matrix is ​​represented by a geometric shape of a different color, and the different colors of the geometric shapes represent the degree of the relation.

7. The visualization method according to claim 6, wherein the multiple relation data matrix instances with different relations used in the reordering model and their corresponding optimal reordering matrix are generated in the following manner: Generate multiple instances of optimal reordering matrices with dimension N×N, wherein each instance contains at least one of the following four structures: clustering, bipartite graph, centroid, and path structure, and the multiple instances of optimal reordering matrices contain all four structures; and For each of the multiple N×N optimal reordering matrix instances, perform the following steps: The optimal reordering matrix is ​​subjected to one or more row and column swap operations multiple times. In each row and column swap operation, two rows and their corresponding columns are swapped to obtain multiple corresponding relational data matrix instances. The multiple corresponding relational data matrix instances and the optimal reordering matrix constitute multiple sets of training samples for the reordering model.

8. The visualization method according to claim 7, wherein generating multiple instances of the optimal reordering matrix with dimensions N×N comprises: Create an initial matrix of dimension N×N, wherein each element of the initial matrix is ​​0; The elements contained in the squares of random size, number, and position on the main diagonal of the initial matrix are replaced with non-zero real numbers multiple times to form multiple optimal reordering matrices containing clustering structures. The elements contained in rectangles of random size, number, and position that do not touch the main diagonal of the initial matrix are repeatedly replaced with non-zero real numbers to form multiple optimal reorder matrices containing bipartite graph structures. The elements contained in the horizontal and vertical lines of random size, number, and position that intersect the main diagonal of the initial matrix are repeatedly replaced with non-zero real numbers to form multiple optimal reorder matrices containing a central node structure. as well as The elements contained in the lines of random size, number, and position parallel to the main diagonal of the initial matrix are repeatedly replaced with non-zero real numbers to form multiple optimal reorder matrices containing path structures.

9. The visualization method according to any one of claims 6-8, wherein applying a trained reordering model to the relational data matrix to reorder it to obtain a reordered matrix comprises: In response to the fact that the dimension M of the received relational data matrix is ​​less than N, the relational data matrix is ​​expanded into a matrix of dimension N×N, wherein the elements in rows M+1 to N and columns M+1 to N are filled with 0; Input the expanded N×N matrix into the reordering model to obtain the corresponding N×N reordering matrix. as well as The corresponding N×N dimension reordering matrix is ​​obtained by deleting the NM rows and NM columns that are filled with 0 corresponding to the expanded elements.

10. A computer program product comprising a computer-readable storage medium having program instructions stored therein, the program instructions being executable by a computing device to cause the computing device to perform the method as described in any one of claims 6-9.

Images