Cluster analysis method, cluster analysis system, and cluster analysis program
The clustering analysis method addresses the challenge of classifying and understanding relationships in large document sets by using inter-document and inter-cluster similarity calculations, facilitating accurate temporal relationship analysis.
Patent Information
- Application Number
- JP2024005074
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2024-01-17
- Publication Date
- 2025-07-29
- Estimated Expiration
- 2039-05-17
AI Technical Summary
Existing clustering methods fail to effectively classify large numbers of documents into clusters and understand the relationships between clusters, particularly temporal relationships, leading to variations in accuracy and difficulty in grasping complex document sets.
A clustering analysis method that classifies documents into clusters using inter-document and inter-cluster similarity calculations, generating association information across sets to understand relationships like temporal transitions.
Enables the classification of large document sets into clusters with understood relationships, particularly temporal changes, reducing processing load and enhancing accuracy.
Smart Images

Figure 0007714256000001 
Figure 0007714256000002 
Figure 0007714256000003
Abstract
Description
Technical Field
[0001] The present invention relates to a clustering analysis method, a clustering analysis system, and a clustering analysis program that classify a plurality of documents into clusters according to their contents and generate display data indicating the relationship between the clusters according to the time series.
Background Art
[0002] Conventionally, when analyzing a large number of documents such as academic papers and literature, people have read the documents and classified them according to their contents, or created summaries. In the analysis by humans, it takes time, and when multiple people perform the analysis, there is a tendency for variations in the accuracy of classification and summary creation depending on the experience and knowledge of the workers.
[0003] In addition, complex and highly specialized documents such as academic papers require advanced specialized knowledge to understand their contents. However, there is a demand for those who do not have such specialized knowledge to easily obtain, understand, and utilize the latest information.
[0004] For example, a clustering analysis method has been proposed in which morphological analysis is performed on technical documents retrieved by concept search, weights are assigned to each word obtained therefrom, each technical document is vectorized, and technical documents with similar vector directions are grouped into one cluster (for example, "Patent Document 1").
[0005] Although it is possible to classify information into clusters by such a technique, it has not reached the point of generating clusters based on different time axes and understanding the relationships between different clusters.
Prior Art Documents
Patent Documents
[0006]
Patent Document 1
Summary of the Invention
Problems to be Solved by the Invention
[0007] The present invention aims to provide a clustering analysis method, a clustering analysis system, and a clustering analysis program that can understand the relationship between clusters across sets by classifying a large number of documents, particularly a huge number of documents, into clusters composed of similar documents and grasping the relationships between clusters in other sets, such as the temporal relationship of clusters.
Means for Solving the Problems
[0008] That is, the present invention is a clustering analysis method in which a computer classifies a plurality of documents into clusters according to their contents, comprising: a first set extraction step of extracting a first set from the plurality of documents according to a first condition; a first inter-document similarity calculation step of calculating the inter-document similarity between the content of one document included in the first set and the content of another document included in the first set; a first clustering step of classifying each document into a plurality of clusters based on the inter-document similarity calculated in the first similarity calculation step within the first set; a second set extraction step of extracting a second set from the plurality of documents according to a second condition different from the first condition; a second inter-document similarity calculation step of calculating the inter-document similarity between the content of one document included in the second set and the content of another document included in the second set; a second clustering step of classifying each document into a plurality of clusters based on the inter-document similarity calculated in the second similarity calculation step within the second set; an inter-cluster similarity calculation step of calculating the inter-cluster similarity between the clusters classified in the first clustering step and the clusters classified in the second clustering step; and a cluster association step of generating association information associating related clusters across the first set and the second set based on the inter-cluster similarity calculated in the inter-cluster similarity calculation step.
Effects of the Invention
[0009] According to the present invention, by classifying a large number of documents, particularly an extremely large number of documents, into document groups (clusters) composed of similar documents, and being able to grasp the relationships between clusters, such as the temporal relationships of clusters, in other sets, it becomes possible to understand the relationships between clusters.
Brief Description of the Drawings
[0010]
Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Modes for Carrying Out the Invention
[0011] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
[0012] FIG. 1 is an overall configuration diagram showing a cluster analysis system according to an embodiment of the present invention, and the configuration of this embodiment will be described based on this figure.
[0013] As shown in FIG. 1, the cluster analysis system 1 according to the present embodiment has a document database 2 (hereinafter, the database is referred to as "DB"), an information terminal 3, and a server 4 connected via a communication network N. The communication network N is, for example, the Internet, an intranet, a VPN (Virtual Private Network), etc., and is a communication network capable of two-way transmission of information using wired or wireless communication means. Also in FIG. 1, for simplicity of explanation, one document DB 2 and one information terminal 3 are connected to one server 4, but the server 4 can be connected to a plurality of document DBs and a plurality of information terminals 3.
[0014] The document DB 2 is a database that stores information on documents such as academic papers, patent documents, magazines, books, and newspaper articles, and makes the stored documents publicly available to limited or unlimited persons. In the present embodiment, the document DB 2 will be described as an example of a document DB that stores information on medical literature. However, there are no restrictions on the content, field, and type of documents that can be stored in the document DB of the present invention. In the present embodiment, the information on medical literature includes bibliographic items such as author names, publication dates (time information), the author's affiliated institutions, etc., content items of papers such as the title, abstract, and text of the paper, citation / cited information such as the number of citations / cited and the document name, and publication information such as the name of the academic society, magazine, or publishing house in which the document is published.
[0015] The information terminal 3 is, for example, a personal computer (hereinafter referred to as "PC") or a mobile terminal such as a smartphone, a tablet PC, and a mobile phone, and has an output unit 10 and an input unit 11.
[0016] The output unit 10 is a device such as a display or a printer, and can visually display the display data generated by the server 4.
[0017] The input unit 11 is a device such as a keyboard or a mouse, and can input and operate information. The output unit 10 and the input unit 11 may be integrated to form, for example, a touch panel.
[0018] The user who uses the information terminal 3 can view the display data generated by the server 4 at the output unit 10 and can also issue various instructions to the server 4 via the input unit 11.
[0019] The server 4 is composed of one or more servers (computers) that classify a plurality of documents into clusters according to their contents and generate display data indicating the relationships between the documents. The server 4 has various arithmetic units and storage units, for example, a document storage unit 20, a set extraction unit 21, a document similarity calculation unit 22, a cluster classification unit 23, an index calculation unit 24, a network storage unit 25, a cluster similarity calculation unit 26, a cluster association unit 27, and a display data generation unit 28.
[0020] Specifically, the document storage unit 20 is a storage unit that is connected to the document DB2 via the communication network N and acquires and stores the information of necessary documents from the document DB2. For example, in this embodiment, medical documents are acquired from the document DB2 and stored. The document storage unit 20 also has a function of automatically updating the documents in the document storage unit 20 synchronously when updates such as addition or deletion of documents are performed in the document DB2.
[0021] The set extraction unit 21 has a function of extracting a set according to conditions using time information from the document storage unit 20. For example, the set extraction unit 21 can extract a set limited to medical documents published in a predetermined period (for example, a predetermined year) using the publication date of the documents. The conditions for extracting the set may use not only time information but also other conditions or additional other conditions. For example, conditions such as medical documents related to a specific disease, medical documents published in a specific academic society, etc. may be used, added, or a plurality of these conditions may be used. Furthermore, it is also possible to narrow down the number of documents included in one set to a predetermined number. Also, when the documents are updated in the document storage unit 20, the set extraction unit 21 extracts the documents that meet the conditions again based on the updated information.
[0022] The document similarity calculation unit 22 has a function of calculating the similarity between the content of one document and the content of other documents for the documents within the set extracted by the set extraction unit 21. For calculating the similarity, for example, TF-IDF or cosine similarity can be used. That is, the document similarity calculation unit 22 extracts the words used in the content of each document, weights the words based on the product of the frequency of occurrence (TF: Term Frequency) of each word in the document and the rarity (IDF: Inverse Document Frequency) of the words used in other documents, and vectorizes the document. Then, the document similarity calculation unit 22 calculates the value of the cosine (cos) between the vectorized documents as the value of the similarity between the documents. For example, the similarity between the first document and the second document is 0.856, and the similarity between the first document and the third document is 0.732. The similarity is represented by a value between 0 and 1, and the closer it is to 1, the more similar the documents are indicated.
[0023] The cluster classification unit 23 generates a network connected by lines (hereinafter referred to as "edges") including each document based on the similarity calculated by the document similarity calculation unit 22, and classifies similar documents into clusters (document groups). The clustering algorithm is not particularly limited, but for example, an algorithm (so-called Girvan-Newman algorithm) that repeatedly identifies clusters that maintain the connectivity between nodes as much as possible even when edges are separated can be used.
[0024] The index calculation unit 24 has a function of calculating a centrality index indicating the centrality of each document in the network generated by the cluster classification unit 23. The algorithm for calculating the centrality index is not particularly limited, but for example, eigenvector centrality, PageRank, betweenness centrality, and degree centrality can be used. In this embodiment, eigenvector centrality is used. Eigenvector centrality is indicated by the probability of passing through a node in the network when starting from an arbitrary node in the network and repeatedly following the edges.
[0025] The network storage unit 25 is a storage unit that stores network information after clustering for each set of documents extracted by the set extraction unit 21. For example, when the set extraction unit 21 generates sets for each year based on the publication year of the documents, the network information for each year will be stored in the network storage unit 25. Each network information stored here is converted into network display data by the display data generation unit 28 and can be displayed by the output unit 10 of the information terminal 3.
[0026] FIG. 2 is an example of the display of one network as the cluster analysis result displayed on the output unit of the information terminal, and FIG. 3 is an explanatory diagram of the network. Based on these figures, the display of the network in one set will be described.
[0027] As shown in FIGS. 2 and 3, the network in one set is shown by expressions according to the centrality index, expressions according to the type of cluster, and expressions according to the degree of similarity between each document for each document in the set.
[0028] Specifically, as shown in FIG. 3, one document (node) on the network is shown by one circle, the centrality index is represented by the size of the circle, the type of cluster is represented by color, and the degree of similarity is represented by the thickness of the edge.
[0029] FIG. 3 shows 10 nodes 30a to 30j (hereinafter also collectively referred to as "nodes 30"). The four nodes 30a to 30d in the upper left belong to the first cluster, and the six nodes 30e to 30j in the lower right belong to the second cluster. Note that the first cluster and the second cluster can be shown in different colors. In FIG. 3, the difference in color is shown by the difference in hatching.
[0030] The size of node 30 indicates the size of centrality. In FIG. 3, it can be seen that nodes 30a and 30e are documents with high centrality. Also, the thickness of edge 32 connecting nodes 30 indicates the magnitude of the similarity between the documents connected by the edge 32. Therefore, in FIG. 3, since the edges 32 between node 30a and node 30c and between node 30e and node 30h are thick, it can be seen that the similarity between the documents between these nodes is high.
[0031] In the network storage unit 25, network information serving as a basis for such network display is stored for each set.
[0032] The inter-cluster similarity calculation unit 26 has a function of calculating the inter-cluster similarity between clusters of a plurality of sets stored in the network storage unit 25. For the calculation of the inter-cluster similarity, TF-IDF or cosine similarity can be used in the same manner as the document similarity calculation unit 22. That is, the inter-cluster similarity calculation unit 26 extracts the words used for the content of the documents within each cluster in each set, weights the words based on the product of the frequency of occurrence within the cluster (TF: Term Frequency) for each word and the rarity (IDF: Inverse Document Frequency) for the words used in other clusters, and vectorizes each cluster. Then, the inter-cluster similarity calculation unit 26 calculates the value of the cosine (cos) between the cluster vectorized in the first set and the cluster vectorized in the second set as the value of the inter-cluster similarity.
[0033] The cluster association unit 27 has a function of generating cluster association information by regarding clusters with an inter-cluster similarity equal to or higher than a predetermined threshold as related clusters. That is, the cluster association unit 27 links related clusters across sets.
[0034] The representation data generation unit 28 is capable of generating network display data based on the network information stored in the above-described network storage unit 25, and has a function of generating time-series display data indicating the relationship between clusters across the sets associated in the cluster association unit 27.
[0035] FIG. 4 shows the relationship between clusters across sets, and FIG. 5 shows an example of the display of time-series display data.
[0036] FIG. 4 shows, as an example of a network showing the network in the set shown in FIG. 3 above as a set of medical documents published in 2018. Further, FIG. 4 arranges in time series the networks showing the sets of medical documents published in 2017 and 2016.
[0037] The inter-cluster similarity calculation unit 26 calculates the similarity between clusters across sets from the similarity between the documents in the clusters of the 2018 set and the documents in the clusters of the 2017 set, as shown by the solid and dotted lines extending between the sets in FIG. 4. Also, the inter-cluster similarity calculation unit 26 can calculate the similarity between clusters over time by performing the same processing for the 2017 set and the 2016 set.
[0038] The time-series display in FIG. 5 arranges the main clusters belonging to the sets of medical documents published each year from 2014 to 2018 in chronological order. The clusters are shown as circles, the size of the circle represents the number of documents belonging to the cluster, and the number written inside the circle indicates the number of documents.
[0039] FIG. 5 performs cluster association based on the latest 2018. Four clusters 40a to 40d with a large number of documents in 2018 are displayed, and the relationship with past clusters is shown by lines (edges 50, 51) based on these clusters. Note that, as in FIG. 3, each cluster is shown in a different color, but in FIG. 5, the difference in color is shown by the difference in hatching.
[0040] The thicknesses of edges 50 and 51 indicate the degree of similarity between clusters, and the display data generation unit 28 generates display data so as to display only similarities equal to or higher than a predetermined threshold value. Further, there are two types of edges: main edge 50 that connects the cluster with the highest similarity to the reference cluster, and sub-edge 51 that connects clusters with the second and subsequent similarities. The clusters connected by the main edge 50 are shown in the same color (hatching) assuming that they are clusters of the same attribute. On the other hand, the sub-edge 51 connects clusters of different attributes. Note that the attribute of a cluster corresponds to, for example, a research theme in the case of medical literature.
[0041] When the diagram 4 in which the clusters of each year are connected by the edges 50 and 51 is time-series display data in medical literature as described above, the following can be inferred.
[0042] For example, in 2018, the attribute of the cluster 40a with the largest number of documents is also ranked first (clusters 41a, 42a) in 2017 and 2016, but is ranked second (clusters 43a, 44a) in 2015 and 2014, and the number of documents has increased rapidly from 2015 to 2016. Therefore, it can be inferred that the research theme of the cluster 40a has been conventionally attracting attention, but in particular, more attention-grabbing events have occurred from 2015 to 2016.
[0043] On the other hand, since the number of documents of the cluster 40b ranked second in 2018 has decreased from 2015 to 2016, it can be inferred that, for example, a treatment method in the research theme of the cluster 40b has been established during this period. Further, since this research theme is connected by the sub-edge 51 to the clusters 43c and 40c ranked third in the number of documents from 2014 to 2015 and from 2017 to 2018, it is inferred that the research theme has branched out.
[0044] In 2018, cluster 40c, which ranked third in terms of the number of documents, has always ranked third in terms of the number of documents since 2014. However, the number of documents is on an increasing trend, and it can be inferred that it is a research theme that may continue to develop in the future.
[0045] In 2018, for cluster 40d, which ranked fourth in terms of the number of documents, it is an attribute that emerged since 2017, indicating that it is a relatively new research theme. Furthermore, it can be inferred that clusters 42d, 43d, and 44d, which ranked fourth in terms of the number of documents from 2016 to 2014, were integrated into cluster 41b, which ranked second in terms of the number of documents in 2017.
[0046] By showing the relevance between clusters across sets in this way, the changes in clusters can be observed.
[0047] The display data generation unit 28 transmits the generated network display data and time-series display data to the information terminal 3 connected via the communication network N to the server 4.
[0048] In the cluster analysis system 1 configured in this way, for example, when a user inputs information regarding medical literature such as a specific disease name to the server 4 via the input unit 11 of the information terminal 3, the server 4 outputs network display data as shown in FIGS. 2 and 3 and time-series display data as shown in FIG. 5 corresponding to the input information to the output unit 10 of the information terminal 3.
[0049] FIG. 6 shows a flowchart of a cluster analysis routine for generating time-series display data executed by the server 4 of the cluster analysis system 1. The cluster analysis method of the present embodiment will be described in detail according to the following flowchart.
[0050] When the server 4 receives input information such as a specific disease name, a time-series period, and a period delimiter method from the information terminal 3, as step S1, the set extraction unit 21 extracts a set of documents that meet the conditions from the document storage unit 20. For example, when the time-series display as shown in FIG. 5 above is requested, first, a set of medical documents published in 2018 (the first set) is extracted.
[0051] In the subsequent step S2, the document - to - document similarity calculation unit 22 calculates the document - to - document similarity between the documents that make up the set extracted in step S1.
[0052] In step S3, the cluster classification unit 23 generates a network between documents based on the similarity calculated in step S2, and classifies the set of similar documents so that they form clusters.
[0053] In step S4, the index calculation unit 24 calculates a centrality index indicating the centrality of the documents in the network generated in step S3. As a result, network information related to the set extracted in step S1 is generated and stored in the network storage unit 25.
[0054] In step S5, the inter - cluster similarity calculation unit 26 determines whether a network of a set that meets the conditions is stored in the network storage unit 25. If the determination result is false (No), the process returns to step S1. For example, in the case of the time - series display in FIG. 5 above, if the network has not been generated for each year's set from 2014 to 2018, the process returns to step S1, extracts the set of the year for which the network has not been generated, and executes steps S2 to S4 above to generate a network.
[0055] If the determination result in step S5 is true (Yes), that is, if the network of the set that meets the conditions has been generated, the process proceeds to step S6.
[0056] In step S6, the inter - cluster similarity calculation unit 26 calculates the inter - cluster similarity between the clusters of the multiple sets stored in the network storage unit 25. For example, in the case of the time - series display in FIG. 5, the inter - cluster similarity between the clusters of the sets in 2018 and 2017 is calculated, and then the inter - cluster similarities between the clusters of the sets in 2017 and 2016, 2016 and 2015, 2015 and 2014 are calculated in sequence.
[0057] In step S7, the cluster association unit 27 generates cluster association information by regarding clusters with a cluster - to - cluster similarity equal to or greater than a predetermined threshold as related clusters. For example, in the case of the time - series display in FIG. 5, clusters with a similarity equal to or greater than a predetermined threshold among the clusters for each year are connected by edges 50 and 51.
[0058] In step S8, the display data generation unit 28 generates time - series data as shown in FIG. 5, transmits it to the information terminal 3, and ends the routine.
[0059] As described above, in the cluster analysis system 1 according to the present embodiment, a plurality of sets with different time conditions are extracted, a network is formed based on the document - to - document similarity within this set, clusters of similar documents are formed, the cluster - to - cluster similarity is calculated, and the association of clusters across sets is performed. As a result, it becomes possible to show the temporal transition of clusters.
[0060] Also, the cluster association targets clusters with a cluster - to - cluster similarity equal to or greater than a predetermined threshold, thereby reducing unnecessary information, reducing the processing load on the server 4, and reducing the amount of information sent to the information terminal 3.
[0061] Furthermore, by generating time - series display data showing the relationship between clusters across associated sets as shown in FIG. 5, it becomes possible to overview the transition of clusters.
[0062] Thus, according to the present embodiment, by classifying a large number of documents, especially a huge number of documents, into clusters composed of similar documents and being able to grasp the temporal relationship of each cluster, it is possible to understand the process between clusters.
[0063] As described above, a specific embodiment of the present invention has been explained. However, the present invention is not limited to the embodiment, and it is understood that various changes and modifications can be made by those skilled in the art without departing from the scope or spirit of the present invention defined in the appended claims.
[0064] In the above embodiment, as shown in FIG. 5, the display data generation unit 28 represents the time-series display by expressing the cluster as a circle, the number of documents as the size of the circle, and the similarity between clusters as the thickness of the edge. However, the expression of the time-series display is not limited to this, and other expressions may be used.
[0065] In the cluster analysis system 1 in the above embodiment, by using time information as a condition for extracting a set, it is possible to grasp the time-series relationship between clusters across sets. However, the condition for extracting a set is not limited to time information. For example, in the case of medical literature, by extracting a set based on the target disease or the type of pharmaceutical, the relevance between clusters related to the disease or pharmaceutical can be visualized. Or, in the case of technical literature, by extracting a set based on the technical field, the relevance between clusters related to a specific technology can be visualized. By thus being able to grasp the relationship between clusters in various sets according to the condition for extracting a set, it is possible to understand the relationship between corresponding clusters in different sets.
Explanation of Reference Numerals
[0066] 1 Cluster analysis system 2 Document DB 3 Information terminal 4 Server 10 Output unit 11 Input unit 20 Document storage unit 21 Set extraction unit 22 Document similarity calculation unit 23 Cluster classification unit 24 Index calculation unit 25 Network storage unit 26 Cluster similarity calculation unit 27 Cluster Association Unit 28 Display Data Generation Unit
Claims
1. A clustering analysis method in which a computer classifies a plurality of documents each containing at least time information into clusters according to the information of the documents, comprising: a first set extraction step of extracting a first set from the plurality of documents according to a first condition using the time information; a first document similarity calculation step of calculating the similarity between documents between the content of one document included in the first set and the content of another document included in the first set; a first clustering step of classifying the documents included in the first set into a plurality of clusters based on the similarity between documents calculated in the first similarity calculation step; a first index calculation step of calculating a centrality index indicating the centrality of each document in the cluster classified in the first clustering step and generating network information related to the first set; a second set extraction step of extracting a second set from the plurality of documents using the time information according to a second condition different from the first condition and the time information; a second document similarity calculation step of calculating the similarity between documents between the content of one document included in the second set and the content of another document included in the second set; a second clustering step of classifying the documents included in the second set into a plurality of clusters based on the similarity between documents calculated in the second similarity calculation step; a second index calculation step of calculating a centrality index indicating the centrality of each document in the cluster classified in the second clustering step and generating network information related to the second set; a cluster similarity calculation step of calculating the similarity between clusters between the cluster of the network information related to the first set generated in the first index calculation step and the cluster of the network information related to the second set generated in the second index calculation step; a cluster association step of generating association information that associates related clusters across the first set and the second set based on the similarity between clusters calculated in the cluster similarity calculation step; A display data generation step of generating first network display data based on network information related to a first set generated in the first index calculation step, second network display data based on network information related to a second set generated in the second index calculation step, and time-series display data indicating the relationship between clusters across the sets associated in the cluster association step; comprising; The display data generation step is a cluster analysis method in which one document is used as a node for the first network display data and the second network display data, and a centrality index is represented by the size of the node, and the time-series display data is represented by arranging the first set and the second set in time series.
2. The cluster analysis method according to claim 1, wherein in the cluster association step, clusters having a cluster-to-cluster similarity calculated in the cluster-to-cluster similarity calculation step equal to or greater than a predetermined threshold are linked to each other.
3. The cluster analysis method according to claim 1 or 2, wherein in the display data generation step, clusters of the first set and clusters of the second set are arranged in time series order, and the time-series display data is generated by connecting clusters having relevance across the first set and the second set with lines.
4. The cluster analysis method according to any one of claims 1 to 3, wherein in the display data generation step, the clusters are represented by circles, the number of documents belonging to the clusters is represented by the size of the circles, and the time-series display data is generated by representing the cluster-to-cluster similarity by the thickness of the lines.
5. A cluster analysis system for classifying a plurality of documents each including at least time information into clusters according to the information of the documents, a set extraction unit that extracts a first set from the plurality of documents according to a first condition using the time information, and extracts a second set according to a second condition different from the first condition and using the time information; A document similarity calculation unit that calculates the similarity between the content of one document included in the first set and the content of other documents included in the first set, and calculates the similarity between the content of one document included in the second set and the content of other documents included in the second set; A cluster classification unit that classifies the documents included in the first set into a plurality of clusters based on the document similarity calculated by the document similarity calculation unit, and classifies the documents included in the second set into a plurality of clusters based on the similarity calculated by the document similarity calculation unit. An index calculation unit that calculates a centrality index indicating the centrality of each document in the cluster classified in the first set and generates network information related to the first set, and calculates a centrality index indicating the centrality of each document in the cluster classified in the second set and generates network information related to the second set. A cluster similarity calculation unit that calculates the similarity between the cluster of network information related to the first set and the cluster of network information related to the second set. A cluster association unit that generates association information that links related clusters across the first set and the second set based on the cluster similarity calculated by the cluster similarity calculation unit. A display data generation unit that generates first network display data based on the network information related to the first set generated by the index calculation unit, second network display data based on the network information related to the second set generated by the index calculation unit, and time series display data indicating the relationship between clusters across the sets associated by the cluster association unit. Comprising The display data generation unit is a cluster analysis system that uses one document as a node, expresses the centrality index as the size of the node as the first network display data and the second network display data, and expresses the first set and the second set in time series as the time series display data.
6. A cluster analysis program for causing a computer to classify a plurality of documents each including at least time information into clusters according to the information of the documents, A first set extraction step of extracting a first set from the plurality of documents according to a first condition using the time information. A first document similarity calculation step of calculating the similarity between the content of one document included in the first set and the content of other documents included in the first set; A first clustering step of classifying the documents included in the first set into a plurality of clusters based on the document similarity calculated in the first similarity calculation step; A first index calculation step of calculating a centrality index indicating the centrality of each document in the cluster classified in the first clustering step and generating network information related to the first set; A second set extraction step of extracting a second set from the plurality of documents using the time information and according to a second condition different from the first condition and the time information; A second document similarity calculation step of calculating the similarity between the content of one document included in the second set and the content of other documents included in the second set; A second clustering step of classifying the documents included in the second set into a plurality of clusters based on the document similarity calculated in the second similarity calculation step; A second index calculation step of calculating a centrality index indicating the centrality of each document in the cluster classified in the second clustering step and generating network information related to the second set; A cluster similarity calculation step of calculating the similarity between the clusters in the first network information generated in the first index calculation step and the clusters in the second network information generated in the second index calculation step; A cluster association step of generating association information that associates related clusters across the first set and the second set based on the cluster similarity calculated in the cluster similarity calculation step; A display data generation step of generating first network display data based on the network information related to the first set generated in the first index calculation step, second network display data based on the network information related to the second set generated in the second index calculation step, and time-series display data indicating the relationship between the clusters across the sets associated in the cluster association step; to execute; The said display data generation step is an analysis program that uses one document as a node for the said first network display data and the said second network display data, expresses the centrality index by the size of the node, and expresses the said time-series display data by arranging the first set and the second set in time series.
7. A cluster analysis method in which a computer classifies information of a plurality of documents each containing at least time information into clusters according to the information of the documents, An index calculation step of calculating a centrality index indicating the centrality of each document in the cluster classified in the first set extracted according to the condition using the time information from the said plurality of documents, generating network information related to the first set, calculating a centrality index indicating the centrality of each document in the cluster classified in the second set whose time information is different from the first set extracted from the said plurality of documents, and generating network information related to the second set; An inter-cluster similarity calculation step of calculating the inter-cluster similarity between the cluster of the network information related to the first set generated in the said index calculation step and the cluster of the network information related to the second set; A cluster association step of generating association information that associates clusters related across the first set and the second set based on the inter-cluster similarity calculated in the said inter-cluster similarity calculation step; A display data generation step of generating first network display data based on the network information related to the first set generated in the said index calculation step, second network display data based on the network information related to the second set, and time-series display data indicating the relationship between clusters across the sets associated in the said cluster association step; characterized by comprising: The said display data generation step is a cluster analysis method that uses one document as a node for the said first network display data and the said second network display data, expresses the centrality index by the size of the node, and expresses the said time-series display data by arranging the first set and the second set in time series.
Citation Information
Patent Citations
Cluster analyzing device and cluster analyzing method
JP2005092443A
Document aggregate analyzer, document aggregate analytical method, program mounted with method, and recording medium for storing program
JP2008059442A
Data classification device, data classification method, and data classification program
JP2011003156A
Apparatus and method for extracting change of topic
JP2011086032A
Document clustering device, method, and program
JP2012243214A