Data visualization system, data visualization method, and data visualization program
The data visualization system addresses the challenge of visualizing data with varying characteristics by grouping, clustering, and merging datasets based on string similarities, facilitating clear data relationship visualization with reduced computational load.
Patent Information
- Application Number
- PCT/JP2026/015656
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2025-05-15
- Filing Date
- 2026-04-17
- Publication Date
- 2026-11-19
AI Technical Summary
Existing data visualization techniques struggle to provide an overall picture of data regardless of the characteristics of the values contained in the data, particularly when dealing with variations in spelling, missing parts of string values, proper nouns, and technical terms, which can hinder accurate processing by machine learning models and increase computational load.
A data visualization system that groups datasets based on a first string value, performs clustering on these datasets based on a second string value, calculates the similarity of common strings between clusters, merges clusters with high similarity, and represents the relationships using a graph with nodes and edges, thereby visualizing the data structure independently of the data values.
Enables accurate visualization of data relationships across datasets, reducing computational load and maintaining stability even with unknown words or variations, while providing a clear understanding of the overall data picture.
Smart Images

Figure JP2026015656_19112026_PF_FP_ABST
Abstract
Description
Data visualization system, data visualization method, and data visualization program
[0001] One aspect of the present disclosure relates to a data visualization system, a data visualization method, and a data visualization program.
[0002] Techniques related to data visualization have been conventionally known. For example, Patent Document 1 describes a computer system comprising an artificial intelligence (AI) platform composed of one or more tools for constructing a taxonomy. The one or more tools include a natural language processing (NLP) engine that extracts a set of hyponym-hypernym term pairs from a corpus, a graph manager that constructs a cross-domain directed graph based on the extracted set of hyponym-hypernym term pairs, and a machine learning manager that trains a machine learning (ML) model to construct a taxonomy structure for a target domain as a directed acyclic graph (DAG).
[0003] U.S. Patent No. 11423307 Specification
[0004] It is desired to visualize the overall picture of data regardless of the characteristics of the values contained in the data.
[0005] A data visualization system relating to one aspect of this disclosure comprises at least one processor. The at least one processor acquires a plurality of datasets, each containing a plurality of data records representing a first data item and a second data item, in a format grouped based on a first string value of the first data item; performs clustering on each of the plurality of datasets based on a second string value of the second data item to divide the dataset into a plurality of clusters; for each of the plurality of clusters, obtains a common string from the set of second string values included in the cluster and associates the common string with the cluster; repeatedly performs a process of calculating the similarity of the common string between each cluster of the first dataset and each cluster of the second dataset while changing the combination of the first dataset and the second dataset selected from the plurality of datasets; merges clusters whose similarity is above a predetermined threshold into a single cluster; represents each of the plurality of datasets as a first node, and represents each of the plurality of clusters derived from one or more datasets by division or merger as a second node; generates a graph in which the relationship between the first node and the second node is represented by edges; and outputs the graph to be displayed on a display device.
[0006] In this respect, a dataset grouped based on the first string value of the first data item is divided into multiple clusters based on the second string value of the second data item. Each cluster in each dataset is associated with a common string based on the set of second string values of that cluster. Subsequently, the similarity of the common strings between individual clusters is calculated across different datasets, and clusters with relatively high similarity are merged across datasets. The relationship between each dataset and each cluster, i.e., the relationship between the first data item and the second data item, is then visualized using a graph represented by nodes and edges. Since this series of processes is performed based on strings as symbols, that is, based on the structure of the strings themselves, the overall picture of the data can be visualized regardless of the characteristics of the values contained in the data.
[0007] According to one aspect of this disclosure, the overall picture of the data can be visualized regardless of the characteristics of the values contained in the data.
[0008] This figure shows an example of the functional configuration of a data visualization system. This figure shows an example of the hardware configuration of the computers that make up the data visualization system. This flowchart shows an example of the operation of the data visualization system. This figure shows an example of processing on target data. This flowchart shows an example of the process of calculating the similarity of common strings. This figure shows an example of cluster merging. This figure shows an example of updating clusters and graphs.
[0009] The following describes various examples in this disclosure in detail with reference to the attached drawings. In the description of the drawings, identical or equivalent elements are denoted by the same reference numeral, and redundant descriptions are omitted.
[0010] [System Overview] The data visualization system described in this disclosure is a computer system that visualizes the relationships between data items that constitute data using graphs. A data item refers to the smallest unit of information that constitutes data. Data consists of multiple data records, each representing multiple data items. Data can be represented in tabular form, in which case data items are represented as columns. In this disclosure, a graph refers to a method of representing the relationships between elements using nodes and edges connecting the nodes.
[0011] In this disclosure, the data processed by the data visualization system is also referred to as "target data." The data visualization system groups the target data based on the first string value of the first data item to obtain multiple datasets. Alternatively, the data visualization system obtains multiple datasets that have been prepared by performing the grouping on the target data in advance. In this disclosure, such datasets are also referred to as "groups." For each of the multiple datasets (multiple groups), the data visualization system performs clustering based on the second string value of the second data item to divide the dataset into multiple clusters. The data visualization system calculates the similarity between clusters between different datasets and merges clusters with relatively high similarity into a single cluster. The data visualization system visualizes the target data using a graph that connects the relationships between a first node representing a dataset (group) and a second node representing the clusters obtained by the division or merge with edges.
[0012] In this way, the data visualization system visualizes the target data based on the string values of the data items. A string value is a value represented by a sequence of one or more characters. The data visualization system performs the above series of processes based on the string as a symbol, that is, based on the structure of the string itself. Therefore, it can visualize the overall picture of the target data regardless of the characteristics of the values contained in the target data.
[0013] It should be noted that the ordinal numbers "1st" for the first data item and "2nd" for the second data item do not depend on the position and order of the data items in the target data, but are used to identify the two data items used in processing by the data visualization system. The first data item may correspond to the first column in the target data and each dataset, or it may correspond to the second or subsequent columns. The second data item may correspond to the second column in the target data and each dataset, or it may correspond to the first column, or it may correspond to the third or subsequent columns.
[0014] In this disclosure, the first data item is a data item that functions as a key for dividing (grouping) the target data into multiple datasets, and the second data item is a data item used for comparison to evaluate the similarity between data records within each dataset.
[0015] The target data may also be purchase data for goods. In this case, the first data item and the second data item may be the supplier and the purchased goods, respectively. The target data may also have other data items such as the quantity purchased and the purchase price.
[0016] [System Configuration] The configuration of a data visualization system 1, as an example, will be described with reference to Figure 1. Figure 1 is a diagram showing an example of the functional configuration of the data visualization system 1.
[0017] The data visualization system 1 comprises the following functional modules: a data acquisition unit 11, a group generation unit 12, a cluster generation unit 13, a similarity calculation unit 14, a cluster integration unit 15, and a graph generation unit 16. The data acquisition unit 11 is a functional module that acquires target data. The group generation unit 12 is a functional module that groups the target data based on the first string value of the first data item and acquires multiple datasets. The cluster generation unit 13 is a functional module that divides each dataset into multiple clusters based on the second string value of the second data item. The similarity calculation unit 14 is a functional module that calculates the similarity between clusters between different datasets. The cluster integration unit 15 is a functional module that integrates clusters whose similarity is above a predetermined threshold into a single cluster. The graph generation unit 16 is a functional module that generates a graph showing the relationship between datasets and clusters.
[0018] Figure 2 shows an example of the hardware configuration of a computer 100 that constitutes the data visualization system 1. For example, the computer 100 includes a processor 101, a main memory unit 102, an auxiliary memory unit 103, a communication control unit 104, an input device 105, and an output device 106. The processor 101 executes the operating system and application programs. The main memory unit 102 is composed of, for example, ROM and RAM. The auxiliary memory unit 103 is composed of, for example, a hard disk or flash memory, and generally stores a larger amount of data than the main memory unit 102. The communication control unit 104 is composed of, for example, a network card or a wireless communication module. The input device 105 is composed of, for example, a keyboard, mouse, touch panel, etc. The output device 106 is composed of, for example, a monitor and speakers.
[0019] Each functional module of the data visualization system 1 is realized by a data visualization program 110 pre-stored in the auxiliary storage unit 103. Each functional module is realized by loading the data visualization program 110 onto the processor 101 or the main memory unit 102 and having the processor 101 execute the data visualization program 110. The processor 101 operates the communication control unit 104, the input device 105, or the output device 106 according to the data visualization program 110 to read and write data to the main memory unit 102 or the auxiliary storage unit 103.
[0020] The data visualization program 110 may be provided on a non-temporary recording medium such as a CD-ROM, DVD-ROM, or semiconductor memory. Alternatively, the data visualization program 110 may be provided via a communication network as a data signal superimposed on a carrier wave.
[0021] The data visualization system 1 may consist of one computer 100 or multiple computers 100. When multiple computers 100 are used, these computers 100 are connected via a communication network such as the Internet or an intranet to logically construct a single data visualization system 1. The data visualization system 1 may also be constructed by combining multiple types of computers.
[0022] [System Operation] Referring to Figure 3, the operation of the data visualization system 1 will be described as an example of the data visualization method related to this disclosure. Figure 3 is a flowchart showing an example of its operation as processing flow S1.
[0023] In step S11, the data acquisition unit 11 acquires the target data. For example, the data acquisition unit 11 accesses a predetermined storage device in response to an instruction entered by a user of the data visualization system 1 and acquires the target data from the storage device. Alternatively, the data acquisition unit 11 may receive the target data from another computer. The data acquisition unit 11 further acquires designation information indicating which of the multiple data items of the target data are the first data item and the second data item. For example, the data acquisition unit 11 may accept designation information entered by a user, or it may receive designation information from another computer.
[0024] In step S12, the group generation unit 12 groups the target data based on the first string value of the first data item to obtain multiple datasets. That is, the group generation unit 12 divides the target data into multiple groups based on the first string value. As a result, the group generation unit 12 obtains multiple datasets, each containing multiple data records representing the first data item and the second data item, in a format grouped based on the first string value of the first data item. For example, the group generation unit 12 performs sorting based on the first data item and generates multiple datasets such that the first string value is the same among one or more data records in each dataset.
[0025] In step S13, the cluster generation unit 13 performs clustering on each of the multiple datasets (multiple groups) based on the second string value of the second data item, thereby dividing the dataset (same group) into multiple clusters. The cluster generation unit 13 may use hierarchical clustering as the clustering method, without specifying the number of clusters to be generated. In one example, in clustering for each dataset, the cluster generation unit 13 calculates the Levenshtein distance for all combinations of two second string values contained in the dataset, and divides the dataset into multiple clusters based on the individual Levenshtein distances. The Levenshtein distance is represented by the minimum number of editing operations required to make the two strings the same.
[0026] In step S14, the cluster generation unit 13 obtains a common string from the set of second string values included in each cluster of each dataset and associates the common string with the cluster. That is, the cluster generation unit 13 sets a common string for each cluster for each group. The common string is the longest continuous string that is common to all second string values included in a given cluster.
[0027] Figure 4 shows an example of processing on target data. In this example, the data acquisition unit 11 acquires purchase data 20, which includes supplier, product name, and unit price as data items, as target data. In the example shown in Figure 4, each row of the purchase data 20 corresponds to one data record. Below, assuming that the data acquisition unit 11 has acquired specification information indicating that the first data item is the supplier and the second data item is the product name, a series of processes from grouping to setting a common string will be explained.
[0028] The group generation unit 12 divides the purchase data 20 into multiple datasets 30 based on the first string value of the first data item 21. In the example shown in Figure 4, the group generation unit 12 divides the purchase data 20 such that groups of data records with the same first string value of the first data item 21 constitute one dataset 30. This grouping generates multiple datasets 30 from the purchase data 20, including a dataset 301 showing purchase data where the supplier is company A, and a dataset 302 showing purchase data where the supplier is company B.
[0029] The cluster generation unit 13 performs clustering for each of the multiple datasets 30 based on the second string value of the second data item 22. For example, the cluster generation unit 13 calculates the Levenshtein distance for all combinations of two second string values included in the dataset 30 and divides the dataset 30 into multiple clusters 31 based on the individual Levenshtein distances. Figure 4 shows that the cluster generation unit 13 has divided the dataset 301 into at least three clusters 311 to 313 and the dataset 302 into at least two clusters 314 and 315.
[0030] The cluster generation unit 13 associates a common string 32 with each of the clusters 31. For example, the second string values of the two data records included in cluster 311 are "polybukurotoumeikuro" and "polybukurotoumeishiro". The cluster generation unit 13 associates the common string 321, "polybukurotoumei", which is the common part of these second string values, with cluster 311. The cluster generation unit 13 performs the same process for all the remaining clusters, including clusters 312 to 315.
[0031] Returning to Figure 3, in step S15, the similarity calculation unit 14 calculates the similarity between clusters between different datasets. The similarity between clusters is represented by the similarity between the common strings associated with one cluster and the common strings associated with the other cluster. The calculation of the similarity of common strings will be explained in detail with reference to Figure 5. Figure 5 is a flowchart of an example of this process. In this disclosure, the similarity of common strings is also simply referred to as "similarity".
[0032] In step S151, the similarity calculation unit 14 selects one of the multiple datasets as the first dataset. In step S152, the similarity calculation unit 14 selects a dataset different from the first dataset as the second dataset.
[0033] In step S153, the similarity calculation unit 14 selects one of the multiple clusters included in the first dataset as the reference cluster. In step S154, the similarity calculation unit 14 calculates the similarity of common strings between the reference cluster and each of the multiple clusters included in the second dataset. For example, the similarity calculation unit 14 may calculate the similarity of common strings between two clusters based on the Levenshtein distance between the common strings of one cluster and the common strings of the other cluster. As another example, the similarity calculation unit 14 may calculate embedding vectors for the common strings of each cluster and calculate the similarity of common strings based on the cosine similarity between the embedding vectors between the two clusters. Methods other than the Levenshtein distance and cosine similarity may be used to calculate similarity. For example, various similarity indices such as Damerau-Levenshtein distance, edit distance, N-gram-based similarity (e.g., Jacquard coefficient), and Hamming distance may be used. The method for calculating similarity may be appropriately selected depending on the string length of the common string, the degree of variation in notation, or the application.
[0034] As shown in step S155, the similarity calculation unit 14 processes all clusters included in the first dataset as reference clusters. If there are clusters in the first dataset that have not been processed as reference clusters (NO in step S155), the process returns to step S153. In the repeated step S153, the similarity calculation unit 14 selects the unprocessed clusters in the first dataset as new reference clusters. In the repeated step S154, the similarity calculation unit 14 calculates the similarity between the new reference clusters and each of the multiple clusters included in the second dataset. If all clusters included in the first dataset have been processed as reference clusters (YES in step S155), the process proceeds to step S156.
[0035] As shown in step S156, the similarity calculation unit 14 calculates the similarity between clusters between the selected first dataset and all other datasets. If there are datasets for which similarity has not been calculated with the first dataset (NO in step S156), the process returns to step S152. In the repeated step S152, the similarity calculation unit 14 selects a dataset that has not been processed as a second dataset as a new second dataset. Then, the similarity calculation unit 14 performs the processes of steps S153 to S155 for the combination of the selected first dataset and the new second dataset. If the similarity of common strings has been calculated between the selected first dataset and all other datasets (YES in step S156), the process proceeds to step S157.
[0036] As shown in step S157, the similarity calculation unit 14 processes all datasets as the first dataset. If there are datasets that have not been processed as the first dataset (NO in step S157), the process returns to step S151. In the repeated step S151, the similarity calculation unit 14 selects the datasets that have not been processed as the first dataset as new first datasets. The similarity calculation unit 14 then performs the processes in steps S152 to S156 for that first dataset. If all datasets have been processed as the first dataset (YES in step S157), step S15 ends.
[0037] As described above for step S15, the similarity calculation unit 14 repeatedly performs the process of calculating the similarity of common strings between each cluster of the first dataset and each cluster of the second dataset, while changing the combination of the first dataset and the second dataset selected from the multiple datasets. If a combination of two clusters for which similarity has already been calculated appears again during the repetition of steps S151 to S157, the similarity calculation unit 14 may omit calculating the similarity for that combination.
[0038] Returning to Figure 3, in step S16, the cluster integration unit 15 integrates clusters whose common string similarity is above a predetermined threshold into a single cluster.
[0039] In the example shown in Figure 4, cluster 311 derived from dataset 301 and cluster 315 derived from dataset 302 show relatively high similarity because their common strings 321 and 325 contain the common part "polybuklo". Therefore, the cluster integration unit 15 integrates cluster 311 and cluster 315 into a single cluster across datasets. Cluster 312 derived from dataset 301 and cluster 314 derived from dataset 302 show the highest similarity because their common strings 322 and 324 are the same. Therefore, the cluster integration unit 15 integrates cluster 312 and cluster 314 into a single cluster across datasets.
[0040] The cluster integration unit 15 can integrate three or more clusters into a single cluster. As an example, let's assume cluster Cx derived from dataset Dx, cluster Cy derived from dataset Dy, and cluster Cz derived from dataset Dz. If the similarity between clusters Cx and Cy, between clusters Cy and Cz, and between clusters Cx and Cz is above a threshold, the cluster integration unit 15 integrates the three clusters Cx, Cy, and Cz into a single cluster. If the similarity between clusters Cx and Cy and between clusters Cy and Cz is above a threshold, but the similarity between clusters Cx and Cz is below a threshold, the cluster integration unit 15 integrates the two clusters Cx and Cy into a single cluster, and integrates the two clusters Cy and Cz into another single cluster.
[0041] In step S17, the graph generation unit 16 generates a graph showing the relationship between datasets (groups) and clusters. The graph generation unit 16 represents each of the multiple datasets as a first node and each of the multiple clusters as a second node, and generates a graph in which the relationship between the first node and the second node is represented by edges. The second node represents a cluster created when the dataset is divided in step S13, or a cluster created when multiple clusters are merged in step S16. That is, the second node represents a cluster derived from one or more datasets by division or merger. The graph generation unit 16 may generate a directed graph in which each edge points from the first node to the second node. Alternatively, the graph generation unit 16 may generate an undirected graph.
[0042] In step S18, the graph generation unit 16 outputs the generated graph to display it on a display device. For example, the graph generation unit 16 may display the graph on the output device 106, or it may transmit the graph to an external terminal via the communication control unit 104 and display the graph on the terminal. As yet another example, the graph generation unit 16 may store the graph in a storage device such as the auxiliary storage unit 103 so that the graph can be output at any time.
[0043] FIG. 6 is a diagram illustrating an example of integrating clusters across datasets. Directed graphs 81 and 82 shown in FIG. 6 are based on the purchase data 20 shown in FIG. 4.
[0044] The directed graph 81 is an example of a graph generated when cluster integration is not performed because steps S14 to S16 are not executed. Each of the plurality of first nodes 40 represents the dataset 30 corresponding to the first node 40. The first nodes 401 and 402 correspond to the datasets 301 and 302, respectively. The first node 403 corresponds to the dataset of Company C. Each of the plurality of first nodes 40 is connected by a directed edge to a second node 50 representing a cluster derived from the dataset 30 corresponding to the first node 40. For example, the first node 401 is connected to each of the second nodes 501, 502, and 503 respectively corresponding to the clusters 311, 312, and 313 derived from the dataset 301.
[0045] The directed graph 82 is an example of a graph generated when cluster integration is performed as steps S14 to S16 are executed. In this example, the cluster integration unit 15 integrates the cluster 311 and the cluster 315 into one cluster, and the graph generation unit 16 sets a second node 511 representing the integrated cluster. In addition, the cluster integration unit 15 integrates the cluster 312 and the cluster 314 into one cluster, and the graph generation unit 16 sets a second node 512 representing the integrated cluster. The second node 513 also represents an integrated cluster. In this way, by integrating a plurality of clusters into one cluster across datasets (groups), the number of nodes in the graph can be reduced, so that a more readable graph can be provided to the user. In addition, the graph reflecting the integration enables more direct visualization of the relationships between the datasets (groups).
[0046] The graph generation unit 16 may set the character string displayed on each first node 40 and each second node 50 according to an arbitrary rule. For example, the graph generation unit 16 may set a character string value in the target data as the character string of the first node, or may set a character string obtained by editing the character string value according to a predetermined rule. As the character string of the second node, the graph generation unit 16 may set a common character string corresponding to the second node, or may set a common part of two or more common character strings corresponding to the integrated cluster. In the example shown in FIG. 6, the character string "Polybukuro" of the second node 511 is the common part of the common character string "Polybukuro Toumei" associated with the cluster 311 and the common character string "L Polybukuro Kuro" associated with the cluster 315.
[0047] The graph generation unit 16 may set the size of each of the plurality of second nodes according to the numerical value corresponding to the second node. For example, the graph generation unit 16 may generate the graph such that the larger the numerical value corresponding to the second node, the larger the second node. The "numerical value corresponding to the second node" refers to a numerical value related to one or more certain data items included in the data records constituting the cluster corresponding to the second node. For example, the numerical value corresponding to the second node may be a representative value or a sum total for one specific data item. Examples of the representative value include statistical values such as a maximum value, a minimum value, a median value, and an average value. Alternatively, the numerical value corresponding to the second node may be a sum of products for two specific data items.
[0048] The graph generation unit 16 may set the visual representation of the second node representing the cluster obtained by integration according to the difference between two or more numerical values corresponding to two or more clusters used to integrate the cluster. The graph generation unit 16 may use various visual representations such as color and hatching. For example, the graph generation unit 16 sets a representative value for each of the two or more clusters before integration. As described above, the representative value may be a statistical value. The graph generation unit 16 may generate the graph such that the larger the difference between the representative values among the two or more clusters, the darker the color of the second node representing the integrated cluster.
[0049] [Variations] The various examples described above have been explained in detail. However, this disclosure is not limited to the examples above. Various modifications are possible with respect to this disclosure, as long as they do not deviate from its essence.
[0050] The data visualization system 1 may update clusters in response to user instructions and generate and output graphs that reflect these updates. These updates are not merely changes in display, but are achieved by performing clustering or integration processing again on the target data or clusters. This process reconfigures the clusters.
[0051] For example, the cluster generation unit 13 and the similarity calculation unit 14 may work together to perform granularly modified clustering on the target cluster, which is the cluster corresponding to the selected second node, in response to a second node being selected by the user, thereby dividing the target cluster into two or more clusters. "Granularly modified clustering" may be a process that returns a single cluster created by integration by the cluster integration unit 15 back to the multiple clusters that existed before the integration. In response to such divisions, the graph generation unit 16 updates the graph to replace the selected second node with two or more second nodes corresponding to the two or more divided clusters.
[0052] As another example, the cluster integration unit 15 may, in response to two or more second nodes being selected by the user, integrate two or more clusters corresponding to the selected two or more second nodes into a single cluster. In this case, the graph generation unit 16 updates the graph to replace the two or more selected second nodes with a single second node corresponding to the integrated cluster.
[0053] Figure 7 shows an example of updating clusters and graphs like these. In the directed graph 91 shown in this figure, each of the first nodes 401, 402, and 403 is connected to the second node 520 by a directed edge. When the second node 520 of this directed graph 91 is selected by the user, the cluster generation unit 13 and the similarity calculation unit 14 work together to perform clustering with changed granularity and divide the cluster "Denchi" corresponding to the second node 520 into four clusters related to four types of batteries. The graph generation unit 16 replaces the second node 520 with four second nodes 531 to 534 corresponding to its four clusters. The graph generation unit 16 then replaces the directed edges to show the relationship between the first nodes 401 to 403 and the second nodes 531 to 534. Through these processes, the graph generation unit 16 updates the directed graph 91 to the directed graph 92. On the other hand, if the second nodes 531 to 534 of the directed graph 92 are selected by the user, the cluster integration unit 15 integrates the four clusters corresponding to the second nodes 531 to 534 into a single cluster. The graph generation unit 16 replaces the second nodes 531 to 534 with the second node 520 corresponding to the integrated cluster. The graph generation unit 16 then replaces the directed edges to show the relationship between the first nodes 401 to 403 and the second node 520. Through these processes, the graph generation unit 16 updates the directed graph 92 to the directed graph 91.
[0054] In the example above, the group generation unit 12 divides the target data into multiple datasets in order to obtain multiple datasets in a format grouped based on the first string value of the first data item. However, the data visualization system may obtain multiple datasets that have already been grouped based on the first string value of the first data item. Therefore, the data visualization system does not need to have a functional module equivalent to the group generation unit 12.
[0055] In this disclosure, the expression "at least one processor executes a first process, a second process, ... and the nth process," or a corresponding expression, is a concept that includes cases where the entity executing the n processes from the first process to the nth process changes midway through. In other words, this expression is a concept that includes both cases where all n processes are executed by the same processor and cases where the processor changes at an arbitrary rate for the n processes.
[0056] The processing steps for a method executed by at least one processor are not limited to the examples above. For example, some of the steps described above may be omitted, or each step may be executed in a different order. Also, any two or more of the steps described above may be combined, or some of the steps may be modified or deleted. Alternatively, other steps may be executed in addition to each of the steps described above.
[0057] In comparing the relative magnitudes of two numerical values in this disclosure, either of the two criteria, "greater than or equal to" and "greater than," may be used, or either of the two criteria, "less than or equal to" and "less than," may be used.
[0058] [Note] As can be seen from the various examples above, this disclosure includes the following aspects: <Item 1> A system comprising at least one processor, the at least one processor acquires a plurality of datasets, each containing a plurality of data records representing a first data item and a second data item, in a format grouped based on a first string value of the first data item, for each of the plurality of datasets, clustering is performed based on a second string value of the second data item to divide the dataset into a plurality of clusters, for each of the plurality of clusters, a common string is obtained from the set of second string values included in the cluster, and the common string is associated with the cluster, the process of calculating the similarity of the common string between each cluster of the first dataset and each cluster of the second dataset is repeatedly performed while changing the combination of the first dataset and the second dataset selected from the plurality of datasets, the clusters whose similarity is above a predetermined threshold are merged into a single cluster, each of the plurality of datasets is represented as a first node, and each of the plurality of clusters derived from one or more datasets by the division or merger is represented as a second node, and a graph is generated in which the relationship between the first node and the second node is represented by edges, and the graph is output for display on a display device. Data visualization system. <Item 2> The data visualization system according to Item 1, wherein at least one processor generates a directed graph as the graph, in which each edge points from the first node to the second node. <Item 3> The data visualization system according to Item 1 or 2, wherein at least one processor calculates the similarity based on the Levenshtein distance between the common strings. <Item 4> The data visualization system according to Item 1 or 2, wherein at least one processor calculates an embedding vector for each of the plurality of common strings, and calculates the similarity based on the cosine similarity between the embedding vectors.<Item 5> The data visualization system according to any one of Items 1 to 4, wherein at least one processor, in response to the selection of the second node, performs the granularly modified clustering on the target cluster which is the cluster corresponding to the selected second node, dividing the target cluster into two or more clusters, and updates the graph so that the selected second node is replaced with two or more second nodes corresponding to the two or more clusters. <Item 6> The data visualization system according to any one of Items 1 to 5, wherein at least one processor, in response to the selection of two or more second nodes, integrates two or more clusters corresponding to the selected two or more second nodes into a single cluster, and updates the graph so that the selected two or more second nodes are replaced with a single second node corresponding to the integrated cluster. <Item 7> The data visualization system according to any one of Items 1 to 6, wherein at least one processor sets the size of each of the multiple second nodes according to the numerical value corresponding to the second node. <Item 8> The data visualization system according to any one of Items 1 to 7, wherein at least one processor sets the visual representation of the second node representing the cluster obtained by the integration according to the difference between two or more numerical values corresponding to two or more clusters used to integrate the cluster.<Item 9> A data visualization method performed by a data visualization system comprising at least one processor, comprising the steps of: acquiring a plurality of datasets, each containing a plurality of data records representing a first data item and a second data item, in a form grouped based on a first string value of the first data item; performing clustering on each of the plurality of datasets based on a second string value of the second data item to divide the dataset into a plurality of clusters; for each of the plurality of clusters, obtaining a common string from the set of second string values included in the cluster and associating the common string with the cluster; repeatedly performing a process to calculate the similarity of the common string between each cluster of the first dataset and each cluster of the second dataset while changing the combination of the first dataset and the second dataset selected from the plurality of datasets; integrating clusters whose similarity is above a predetermined threshold into a single cluster; and generating a graph in which each of the plurality of datasets is represented as a first node, and each of the plurality of clusters derived from one or more datasets by the division or integration is represented as a second node, and the relationship between the first node and the second node is represented by edges. A data visualization method comprising the step of outputting the graph in order to display the graph on a display device.<Item 10> Steps to obtain multiple datasets, each containing multiple data records representing a first data item and a second data item, in a format grouped based on the first string value of the first data item; Steps to divide each of the multiple datasets into multiple clusters by performing clustering based on the second string value of the second data item; Steps to obtain a common string from the set of second string values included in each of the multiple clusters and associate the common string with the cluster; Steps to repeatedly perform a process to calculate the similarity of the common string between each cluster of the first dataset and each cluster of the second dataset while changing the combination of the first dataset and the second dataset selected from the multiple datasets; Steps to merge clusters whose similarity is above a predetermined threshold into a single cluster; Steps to generate a graph in which each of the multiple datasets is represented as a first node, and each of the multiple clusters derived from one or more datasets by the division or merge is represented as a second node, and the relationship between the first node and the second node is represented by edges; Steps to output the graph for display on a display device. A data visualization program that causes a computer to perform the following actions.
[0059] According to items 1, 9, and 10, a dataset grouped based on the first string value of the first data item is divided into multiple clusters based on the second string value of the second data item. Each cluster in each dataset is associated with a common string based on the set of second string values of that cluster. Subsequently, the similarity of the common strings between individual clusters is calculated across different datasets, and clusters with relatively high similarity are merged across datasets. The relationship between each dataset and each cluster, i.e., the relationship between the first data item and the second data item, is then visualized using a graph represented by nodes and edges. Since this series of processes is performed based on strings as symbols, that is, based on the structure of the strings themselves, the overall picture of the data can be visualized regardless of the characteristics of the values contained in the data.
[0060] Generally, data contains various representations such as variations in spelling, missing parts of string values, proper nouns, and technical terms. When a single meaning in data is represented by multiple expressions, or when unfamiliar representations appear in the data, data visualization can become difficult. For example, such various representations can hinder accurate processing by machine learning models, such as large-scale language models, and therefore visualization using trained models is also difficult. Improving the accuracy of trained models can address these challenges, but this requires the use of larger amounts of training data, necessitating the introduction of more powerful computers to absorb the increased computational load. As described above, the data visualization system disclosed here performs a series of processes based on the structure of the string itself as a symbol, rather than processing based on semantic understanding or trained models, thus enabling data visualization while suppressing the processing load on the computer. For example, it can visualize data while avoiding the increased processing load associated with training or inference of machine learning models. Furthermore, because the processing does not depend on dictionaries or training data, stable visualization can be achieved even when dealing with data containing unknown words, proper nouns, technical terms, or variations in spelling, making it possible to grasp the relationships between multiple datasets across different datasets.
[0061] According to item 2, directed graphs make the relationship between datasets and clusters (the relationship between the first data item and the second data item) more visually clear, thus enabling data visualization that makes it easier to understand the overall picture of the data.
[0062] According to item 3, the similarity is calculated using the Levenshtein distance, a method that compares strings character by character. Therefore, it can accurately calculate similarity by absorbing some differences in notation, such as variations in spelling or missing parts of string values. As a result, the overall picture of the data can be visualized with high accuracy.
[0063] According to item 4, the similarity is calculated based on the cosine similarity between the embedding vectors calculated for each of the two common strings, allowing the similarity to be calculated in a way that further considers the meaning of the common strings. As a result, the overall picture of the data can be visualized with high accuracy.
[0064] According to item 5, in response to an operation to select a second node representing the target cluster, the target cluster is divided into two or more clusters, and the selected second node is updated to two or more second nodes to reflect this division. By updating the graph according to the user's request, the convenience of data visualization for the user can be improved.
[0065] According to item 6, in response to an operation to select two or more second nodes, the two or more clusters corresponding to these second nodes are merged into a single cluster, and the selected second nodes are updated to a single second node in accordance with this merger. Updating the graph according to the user's request can improve the convenience of data visualization for the user.
[0066] According to item 7, the numerical value corresponding to the second node is visualized by the size of that second node. By reflecting further information in the second node in this way, the amount of information that can be read from the graph can be increased.
[0067] According to item 8, the differences between two or more integrated clusters are visualized by the visual representation of the second node. This allows the relationship between two or more integrated clusters to be easily shown using the second node.
[0068] 1...Data visualization system, 11...Data acquisition unit, 12...Group generation unit, 13...Cluster generation unit, 14...Similarity calculation unit, 15...Cluster integration unit, 16...Graph generation unit.
Claims
1. A system comprising at least one processor, the system obtaining a plurality of datasets, each containing a plurality of data records representing a first data item and a second data item, in a grouped format based on a first string value of the first data item, for each of the plurality of datasets, clustering based on a second string value of the second data item to divide the dataset into a plurality of clusters, for each of the plurality of clusters, obtaining a common string from the set of second string values included in the cluster and associating the common string with the cluster, repeatedly performing a process to calculate the similarity of the common string between each cluster of the first dataset and each cluster of the second dataset while changing the combination of the first dataset and the second dataset selected from the plurality of datasets, integrating clusters whose similarity is above a predetermined threshold into a single cluster, representing each of the plurality of datasets as a first node, and representing each of the plurality of clusters derived from one or more datasets by the division or integration as a second node, generating a graph in which the relationship between the first node and the second node is represented by edges, and outputting the graph to display the graph on a display device. Data visualization system.
2. The data visualization system according to claim 1, wherein at least one processor generates a directed graph as the graph, in which each edge points from the first node to the second node.
3. The data visualization system according to claim 1, wherein at least one processor calculates the similarity based on the Levenshtein distance between the common strings.
4. The data visualization system according to claim 1, wherein at least one processor calculates an embedding vector for each of the plurality of common strings and calculates the similarity based on the cosine similarity between the embedding vectors.
5. The data visualization system according to any one of claims 1 to 4, wherein at least one processor, in response to the selection of the second node, performs the granularly modified clustering on the target cluster which is the cluster corresponding to the selected second node, thereby dividing the target cluster into two or more clusters, and updates the graph so as to replace the selected second node with two or more second nodes corresponding to the two or more clusters.
6. The data visualization system according to any one of claims 1 to 4, wherein at least one processor updates the graph in such a way that, in response to the selection of two or more second nodes, two or more clusters corresponding to the selected two or more second nodes are integrated into one cluster, and the selected two or more second nodes are replaced by one second node corresponding to the integrated cluster.
7. The data visualization system according to any one of claims 1 to 4, wherein at least one processor sets the size of each of the plurality of second nodes according to a numerical value corresponding to the second node.
8. The data visualization system according to any one of claims 1 to 4, wherein at least one processor sets the visual representation of the second node representing the cluster obtained by the integration according to the difference between two or more numerical values corresponding to two or more clusters used to integrate the cluster.
9. A data visualization method performed by a data visualization system comprising at least one processor, comprising the steps of: acquiring a plurality of datasets, each containing a plurality of data records representing a first data item and a second data item, in a form grouped based on a first string value of the first data item; performing clustering on each of the plurality of datasets based on a second string value of the second data item to divide the dataset into a plurality of clusters; for each of the plurality of clusters, obtaining a common string from the set of second string values included in the cluster and associating the common string with the cluster; repeatedly performing a process to calculate the similarity of the common string between each cluster of the first dataset and each cluster of the second dataset while changing the combination of the first dataset and the second dataset selected from the plurality of datasets; integrating clusters whose similarity is above a predetermined threshold into a single cluster; and generating a graph in which each of the plurality of datasets is represented as a first node, and each of the plurality of clusters derived from one or more datasets by the division or integration is represented as a second node, and the relationship between the first node and the second node is represented by edges. A data visualization method comprising the step of outputting the graph in order to display the graph on a display device.
10. Steps to obtain multiple datasets, each containing multiple data records representing a first data item and a second data item, in a grouped format based on the first string value of the first data item; Steps to divide each of the multiple datasets into multiple clusters by performing clustering based on the second string value of the second data item; Steps to obtain a common string from the set of second string values included in each of the multiple clusters and associate the common string with the cluster; Steps to repeatedly perform a process to calculate the similarity of the common string between each cluster of the first dataset and each cluster of the second dataset while changing the combination of the first dataset and the second dataset selected from the multiple datasets; Steps to merge clusters whose similarity is above a predetermined threshold into a single cluster; Steps to generate a graph in which each of the multiple datasets is represented as a first node, and each of the multiple clusters derived from one or more datasets by the division or merge is represented as a second node, and the relationship between the first node and the second node is represented by edges; Steps to output the graph for display on a display device. A data visualization program that causes a computer to perform the following actions.
