Clustering device, clustering method, and program

Abstract

Provided is a clustering device that can securely perform hierarchical clustering while keeping the calculation process and intermediate values completely confidential. The clustering device includes: a cluster ID updating unit that joins the two closest clusters and updates cluster IDs on a cluster ID table that has data IDs and cluster IDs corresponding one-to-one; and an inter-cluster distance updating unit that deletes information corresponding to a cluster to be joined, from an inter-cluster distance table which is a table of distances between all clusters, executes addition processing, which adds the distance between newly joined clusters and other clusters to the inter-cluster distance table, and updates the inter-cluster distance table. The information in the cluster ID table and the inter-cluster distance table is encrypted and processing performed by the cluster ID updating unit and the addition processing performed by the inter-cluster distance updating unit is performed using the encrypted information without decryption.

Description

[Technical field]

[0001] The present invention relates to hierarchical clustering techniques. [Background technology]

[0002] Non-Patent Document 1 is known as a conventional technique for hierarchical clustering. In addition, the hclust function of R and the linkage function of scipy perform hierarchical clustering in plain text. R is a language and environment for statistical calculations and graphics, and scipy is a library for performing advanced scientific calculations. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Stephen C Johnson, "Hierarchical clustering schemes", Psychometrika, Vol. 32, No. 3, pp. 241-254, 1967. [Non-Patent Document 2] Godfrey N Lance and William Thomas Williams. A general theory of classificatory sorting strategies:1. hierarchical systems. The computer journal, Vol. 9, No. 4, pp. 373-380, 1967. Summary of the Invention [Problem to be solved by the invention]

[0004] However, in conventional techniques, it is not possible to perform secure hierarchical clustering calculations while keeping the training data and intermediate values ​​in the calculations encrypted.

[0005] The present invention aims to provide a clustering device, a clustering method, and a program that can perform hierarchical clustering safely while keeping all calculation processes and intermediate values ​​secret by using a table to manage data points included in a cluster, distances between data or between clusters, etc. [Means for solving the problem]

[0006] In order to solve the above problems, according to one aspect of the present invention, a clustering device includes a cluster ID updating unit that combines two closest clusters and updates the cluster IDs in a cluster ID table in which data IDs and cluster IDs correspond one-to-one, and an inter-cluster distance updating unit that performs a deletion process that deletes information corresponding to the cluster to be combined from a cluster distance table, which is a table of distances between all clusters, and an addition process that adds distances between the newly combined cluster and other clusters to the cluster distance table, thereby updating the cluster distance table, and the information in the cluster ID table and the cluster distance table is encrypted, and the processing in the cluster ID updating unit and the addition processing in the cluster distance updating unit are performed using the encrypted information without decryption. Effect of the Invention

[0007] According to the present invention, it is possible to perform hierarchical clustering safely while keeping all calculation processes and intermediate values ​​secret. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram for explaining hierarchical clustering. [Diagram 2] A diagram showing an example of a dendrogram that can be obtained by hierarchical clustering. [Diagram 3] FIG. 1 is a diagram for explaining the group average method for calculating the distance between two clusters. [Figure 4] FIG. 13 is a diagram for explaining a procedure for updating a cluster ID table. [Diagram 5]5A to 5C are diagrams showing examples of distance tables. [Figure 6] A diagram to explain the main algorithm [Method 1] of secure hierarchical clustering. [Figure 7] FIG. 13 is a diagram for explaining an update algorithm of a cluster ID table. [Figure 8] FIG. 13 is a diagram for explaining an algorithm for deleting unnecessary information. [Figure 9] FIG. 11 is a diagram for explaining an image of processing after combining clusters in the first embodiment. [Figure 10] A diagram to explain the calculation of groupBy Sum and groupBy Count and the relationship between key information → k. [Figure 11] FIG. 1 is a diagram for explaining the process flow for creating key information → k. [Figure 12] FIG. 2 is a functional block diagram of a clustering device according to the first embodiment. [Figure 13] FIG. 4 is a diagram showing an example of a processing flow of the clustering device according to the first embodiment. [Figure 14] A diagram to explain the main algorithm [Method 2] of secure hierarchical clustering. [Figure 15] FIG. 11 is a diagram for explaining an image of processing after combining clusters in the second embodiment. [Figure 16] 13A to 13C are diagrams showing the steps of a cluster distance calculation process according to the second embodiment. [Figure 17] FIG. 11 is a functional block diagram of a clustering device according to a second embodiment. [Figure 18] FIG. 11 is a diagram showing an example of a processing flow of a clustering device according to the second embodiment. [Figure 19] FIG. 13 is a diagram showing an example of the configuration of a computer to which the present technique is applied. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment of the present invention will be described. In the drawings used in the following description, components having the same functions and steps performing the same processes are denoted by the same reference numerals, and duplicated explanations will be omitted. In the following description, the symbols " → " etc. should be written directly above the character that immediately follows, but due to limitations in text notation, they are written immediately before the character in question. Within formulas, these symbols are written in the correct position. Furthermore, unless otherwise specified, processing performed on each element of a vector or matrix applies to all elements of that vector or matrix.

[0010] First, before describing the embodiments, a technique used in the embodiments will be described.

[0011] <Hierarchical clustering> Clustering is one of the machine learning methods classified as unsupervised learning. In supervised learning such as regression analysis and classification, the desired output (teacher data) is prepared and the aim is to build a model that reproduces that output with high accuracy, whereas in unsupervised learning such as clustering, the desired output is not determined in advance.

[0012] Clustering aims to calculate the distance between multiple given data and find data that is close in distance, i.e. similar data. Clustering methods are broadly classified into two types: non-hierarchical, such as the k-means method, in which the number of clusters to be created is determined in advance, and hierarchical, in which the number of clusters is not defined in advance and data is clustered in order starting with the most similar data. The embodiment deals with the latter type, "hierarchical clustering."

[0013] Hierarchical clustering is an analytical method that organizes data into clusters in order of similarity, as shown in Figure 1. The general calculation steps for hierarchical clustering are shown below.

[0014] 1. Calculate the distance between all data points.

[0015] 2. The two data points that are closest to each other are grouped into a new cluster.

[0016] 3. Calculate the distance between the newly created cluster and other data points or other clusters.

[0017] 4. The closest data points, or the data point and the cluster, or the clusters are taken as a new cluster.

[0018] 5. Repeat steps 3 and 4 until all data points are in a single cluster.

[0019] The purpose of hierarchical clustering is to calculate the distances between all data using this calculation procedure, and ultimately obtain a dendrogram like the one in Figure 2. The dendrogram in Figure 2 corresponds to the clustering results in Figure 1, and shows that data A and B are closest, followed by C and D, then the clusters consisting of A and B and C and D, and E is the furthest. Hierarchical clustering is used when you want to look in detail at the closeness of each piece of data like this.

[0020] When calculating hierarchical clustering, the important points are “how to calculate the distance between data” and “how to calculate the distance between clusters.” There are a number of methods for doing this, and the method used in the embodiment will be described below.

[0021] <Euclidean distance> Euclidean distance is one of the most well-known methods for calculating the distance between data. → x = (x1, x2, …, x n )and → y=(y1,y2,…,y n ) distance d( → x, → y) is expressed by the following formula (1).

number

[0022] When calculating the distance between a cluster including data point ABC and two clusters including data point DE using the group average method, as shown in Figure 3, the average of all the distances between data point ABC and data point DE is taken as the distance between the clusters.

[0023] As explained above in <Hierarchical Clustering>, the distances between all data points are calculated at the beginning, so when calculating the distance between clusters, it is only necessary to calculate the average value.

[0024] <Lance-Williams updating formula> In hierarchical clustering, the Lance-Williams updating formula (Non-Patent Document 2) is known as an efficient method for calculating the distance between a newly created cluster and other data points / clusters.

[0025] Newly created cluster C1=C 1a ∪C 1b The update formula for determining the distance between this new cluster C1 and another cluster C2 is shown in formula (2).

[0026] d(C1, C2) = α a d(C 1a ,C2)+α b d(C 1b ,C2)+βd(C 1a ,C 1b )+γ|d(C 1a ,C2)-d(C 1b ,C2)| (2) n i is cluster Ci The number of elements in, d(C i ,C j ) represents the distance between clusters. Also, α a ,α b ,β,γ are coefficients according to the method of calculating the distance between clusters. For example, in the case of the group average method, α a =n 1a / n1,α b =n 1b / n1, β = 0, γ = 0, and substituting this into equation (2) gives equation (3).

[0027] d(C1, C2) = (n 1a / n1)d(C 1a ,C2)+(n 1b / n1)d(C 1b ,C2) (3) The necessary information is Cluster C 1a The number of data in and the distance from other clusters C2, C 1b The only thing that is required is the number of data points in the cluster and the distance between the cluster and the other cluster C2. In this way, the distance between new clusters can be calculated very easily by using the Lance-Williams updating formula.

[0028] <Secret calculation> Notation: In order to distinguish between plaintext and ciphertext, the ciphertext of a is written as [[a]], and anything without parentheses is plaintext. The main functions used in the embodiment are shown below.

[0029] Addition, subtraction, and multiplication: Addition, subtraction, and multiplication of two ciphertexts [[a]] and [[b]] ​​are the processes that calculate the ciphertexts [[a+b]], [[ab]], and [[a×b]], respectively. These operations are written as [[a]]+[[b]], [[a]]-[[b]], and [[a]]×[[b]], respectively, and the input is the vector [[ → a]],[[ → The same notation applies to the case of b]].

[0030] · Multiplication and accumulation: The process of calculating the multiplication and accumulation of each row of matrices [[A]] and [[B]] is written as hpsum([[A]],[[B]]), and hpsum([[A]],[[B]]) is a vector [[ → c]].

[0031] Group-By operation: Group-By Common is a process that generates intermediate data that can be used in common in various Group-By operations such as Group-By Sum and Group-By Count. The intermediate data is stored in a substitution table [[ → π]] and a flag indicating whether the key value is on the boundary [[ → By reusing these, various Group-By operations using the same key can be performed efficiently.

[0032] A vector of keys [[ → k]] and perform Group-By Common using the sorted value attribute vector [[ → a']] and flag [[ → Performing Group-By Sum using [e]] is written as equation (5), and performing Group-By Count is written as equation (6).

[0033] [[ → π]],[[ → e]],[[ → a']]←groupByCommon([[ → a]],[[ → k]]) (4) [[ → c]]←groupBySum([[ → a']],[[ → e]]) (5) [[ → d]]←groupByCount([[ → a']],[[ → e]]) (6) It is possible to calculate secure hierarchical clustering without processing such as Group-By Common, as long as Group-By Sum and Group-By Count can be executed. However, using Group-By Common is more efficient.

[0034] Other: equality([[a]],[[b]]) for equality check, shuffle([[ → a]]), sort([[ → These include logical operations such as and([[a]],[[b]]) (logical multiplication), or([[a]],[[b]]) (logical sum), and not([[a]]) (logical negation). When the input for private shuffle or sort is a matrix, the shuffle is performed by swapping each row while maintaining the correspondence between each row. The process of decrypting ciphertext is written as reveal([[a]]).

[0035] The above-mentioned secure computation can be realized using various conventional techniques, and therefore a detailed description thereof will be omitted.

[0036] Next, the secure computation hierarchical clustering performed in the embodiment will be described.

[0037] <Secure Computation Hierarchical Clustering> In the secure computation hierarchical clustering of the embodiment, information other than the number of pieces of data and values ​​obvious from the number of pieces of data (such as the number of times the clustering procedure is repeated) is kept secret. Specifically, the following information is calculated while being kept secret:

[0038] Data features Distance between data, or between data and clusters, or between clusters Which data points or clusters were combined What data is contained within each cluster? The number of data points in each cluster <Key points of secure computing hierarchical clustering> This paper shows the key points for performing hierarchical clustering while concealing information such as how many data points are in each cluster and what data they contain.

[0039] (Manage cluster information) In the hierarchical clustering of the embodiment, it is necessary to store information on "what data is included in each cluster" and to update the information every time a data point or cluster is combined. In the embodiment, a table (cluster ID table) C, which has one-to-one correspondence between data IDs and cluster IDs, as shown in FIG. id Figure 4 shows the cluster ID table C corresponding to Figures 1 and 2. id The diagram below shows how the cluster ID is updated, from left to right.

[0040] Cluster ID Table C id The following shows the general update procedure.

[0041] 1.As the initial value of the cluster ID, assign numbers starting from 0 to each data. id (t1) is the cluster ID table after allocation.

[0042] 2. Change the cluster IDs of the data points in the cluster to be merged to new ones (the new cluster IDs are assigned sequentially). id (t2)~C id (t4) shows the cluster ID table after rewriting.

[0043] 3. Repeat (2) until all data is grouped into one cluster. id (t5) shows the cluster ID table after the repetition.

[0044] In the first embodiment, of the two methods for managing distance information, [Method 1] and [Method 2], [Method 1] is used. Method 2 is used in the second embodiment. (Management of distance information [Method 1]) In the hierarchical clustering [Method 1] of the first embodiment, it is necessary to store the distance between data points and store and update the distance between clusters. In the secure computation hierarchical clustering of this embodiment, distance information is managed in the following three tables.

[0045] · Table of distances between all data (data distance table) D data · Table of distances between all clusters (inter-cluster distance table) D clust Table of distances between joined clusters (output distance table) D out FIG. 5 shows an example of each distance table.

[0046] Data Distance Table D data is a table that stores information on the distance between data points ID1 and ID2. For example, if ID1=A and ID2=B, the distance between data points A and B is stored in the third column.

[0047] Cluster Distance Table D clust is a table that stores the distance information between clusters ID1 and ID2. At the time of initialization, each data point is considered as one cluster, so D data and D clust is the same table.

[0048] Output Distance Table D out corresponds to the output of hierarchical clustering, and finally, a dendrogram is created based on this table. Output distance table D out is an empty table at the time of initialization, but each time a cluster is joined, the IDs and distance information of the two joined clusters are added.

[0049] Data Distance Table D data is created once and reused, but the cluster distance table D clust and output distance table D out needs to be updated every time a cluster is joined.

[0050] In order to realize the secure computation hierarchical clustering of the first embodiment, the following two processes are particularly important.

[0051] Cluster ID table C id Update · Cluster Distance Table D clust Update In the first embodiment, of two methods, Method 1 and Method 2, Method 1 is used as the main algorithm of the secure computation hierarchical clustering. Method 2 is used in the second embodiment.

[0052] <Main algorithm of secure hierarchical clustering [Method 1]> The main algorithm of the secure hierarchical clustering method [Method 1] is shown in Algorithm 1 in Figure 6, and then the detailed processing of each function will be shown.

[0053] The function calcDataDist(X) on the fourth line is a process for calculating the distance between all data. Any method for calculating the distance may be used, but in the first embodiment, the Euclidean distance shown in formula (1) is used. The Euclidean distance can be secretly calculated using only subtraction and product-sum operations. Formula (1) includes a square root calculation, but in clustering, attention is focused only on the magnitude relationship, and the result does not change even if the square root calculation is omitted, so the square root calculation is also omitted in this embodiment.

[0054] The function calcClustDist(X) in the fifth line calculates the distance between all clusters. In the initial state, each data point is considered as a cluster, so the data distance table D data and cluster distance table D clust is the same table.

[0055] The function getClosestClust(D clust ) is a process to obtain the IDs (cid1,cid2) and distance d of the two closest clusters, and uses the distance as a key to create a cluster distance table D clust After sorting using secret computation, the first element (the element with the smallest distance) is taken.

[0056] The function updateCid on line 8 updates the cluster ID table C id . The calculation procedure of updateCid is shown in Algorithm 2 of Figure 7. The cluster ID of the newly combined cluster in the cluster ID table C id is updated.

[0057] The function updateDout on line 9 is simply a process of adding cid1, cid2, and d to the end of the output distance table Dout.

[0058] <updateClustDist[Method 1]> In the first embodiment, as the processing method of the function updateClustDist on line 10, [Method 1] is used among the two methods [Method 1, [Method 2]]. Method 2 is used in the second embodiment. The function updateClustDist on line 10 updates the inter-cluster distance table D clust . The calculation procedure of the function updateClustDist mainly consists of the following two processes.

[0059] 1. Delete the information that becomes unnecessary due to the combination of clusters from [[D clust (see Figure 8).

[0060] 2. Calculate the distances between the newly combined cluster and other clusters and add them to [[D clust .

[0061] The calculation procedure for deleting the information that becomes unnecessary due to the combination of clusters is shown in Algorithm 3 of Figure 8.

[0062] In Algorithm 3 of Figure 8, since the information regarding [[cid1]] and [[cid2]] that are combined into one cluster becomes unnecessary, the process of leaving the other information is performed. At line 11 of Figure 8, → c not]] is decrypted, but because the first line is shuffled in advance, the only information that can be determined from the decryption result is the number of lines to be deleted. The number of lines to be deleted at each step is obvious from the number of records in the training data, so there is no problem.

[0063] By adding the distances between the new cluster created by the cluster merging and other clusters, calculated in the previous step, to the table from which unnecessary rows have been deleted in Algorithm 3, the process corresponding to updateClustDist (updating the distance table between all clusters) in Algorithm 1 can be achieved.

[0064] (How to calculate the distance between clusters) Next, the procedure for calculating the distance between a new cluster formed by combining clusters and other clusters will be described.

[0065] As a concrete example, Fig. 9 shows an image of the processing after combining clusters with ID=1 and ID=2 when there are a total of four data points.

[0066] After merging the clusters with ID=1 and ID=2 into a new cluster, we calculate the distance between the new cluster and the other clusters (ID=0, ID=3). To do this, we use the data distance table D data The data that needs to be brought in from satisfies the following two conditions simultaneously:

[0067] 1. Data ID1 is included in the two clusters to be joined.

[0068] 2. Data ID2 is not included in the two clusters to be joined.

[0069] The data contained in the two clusters to be joined are ID=1 and ID=2, so ID1=1 or ID1=2. The data not contained in the two clusters to be joined are ID=0 and ID=3, so ID2=0 or ID2=3. It is sufficient to extract the four combinations of ID1 and ID2: (1,0), (1,3), (2,0), and (2,3). When calculating the distance between clusters using the group average method, the average value of the distance between (1,0) and (2,0) is the distance d between the new cluster (cluster ID=4) and cluster ID=0. 04 The average distance between (1,3) and (2,3) is the distance d between the new cluster (cluster ID=4) and the cluster with ID=3. 34 become.

[0070] Data Distance Table D data Therefore, to find the distance between the new cluster and other clusters, it is necessary to calculate the sum of the distance and the number of data included in the cluster while keeping the data encrypted, and then divide the sum by the number of data. Therefore, the sum of the distance is calculated using groupBy Sum, and the number of data included in the cluster is calculated using groupBy Count. To calculate groupBy Sum and groupBy Count, the key information for Group-By operation is required. → It is necessary to create a good k. Specifically, the key information shown in Figure 10 → If we can create k, we can obtain the desired Group-By operation result.

[0071] Actual key information → The process flow for creating k is shown below, and a specific example corresponding to the process flow is shown in Figure 11. The key information obtained in Figure 11 → The key information k needs in Figure 10 → This key information matches k. → Using k, we can execute a Group-By operation with the third column (distance) of the distance table for all data as the value attribute. Unnecessary information will be at the top with ID=-1, so if we extract elements other than the top element from the Group-By calculation result, only the desired information will remain.

[0072] 1. Create a flag vector d1 in which ID1 in the distance table between all data is 1 for data contained in the two clusters to be joined and 0 for other data.

[0073] 2. Create a flag vector d2 in which ID2 in the distance table between all data is 0 for data contained in the two clusters to be joined and 1 for all other data.

[0074] 3. Take the logical product d3 of flag vector d1 and flag vector d2.

[0075] 4. Create a vector d4 by subtracting 1 from all elements of the logical product d3.

[0076] 5. Multiply the logical product d3 by the cluster ID corresponding to data ID2 to create vector d5.

[0077] 6. Add vector d4 and vector d5, and get the key information → Get k.

[0078] It can be seen that the above steps 1 to 6 can be achieved by combining only lightweight operations such as equality, logical operations, and addition, subtraction, and multiplication. This makes it possible to calculate distance information for new clusters while keeping all information encrypted, such as how many pieces of data are contained in each cluster and which clusters have been combined.

[0079] Algorithm 1’s updateClustDist (updating the distance table between all clusters) is realized by deleting old distance information using Algorithm 3 and calculating distance information for new clusters using the method described above.

[0080] By implementing hierarchical clustering using secret computation, secure hierarchical clustering can be performed while keeping all of the following information secret:

[0081] Data features · Distance between each data point · Distance between clusters Which data points or clusters were combined What data is contained within each cluster? The number of data points in each cluster <Key Points of the First Embodiment> The following three points make secure hierarchical clustering possible:

[0082] Cluster ID table C id Managing the data points included in each cluster by Distance Table D data , D clust Managing distances between data and clusters · Calculating distance between clusters using Group-By operations A clustering device that realizes the above-mentioned secure computation hierarchical clustering will now be described.

[0083] First Embodiment FIG. 12 is a functional block diagram of the clustering device according to the first embodiment, and FIG. 13 shows the processing flow thereof.

[0084] The clustering device includes an initialization unit 110, a data distance calculation unit 120, a data distance memory unit 122, a joining cluster identification unit 140, a cluster ID update unit 150, a cluster ID memory unit 152, an output distance table update unit 160, an output distance memory unit 162, a cluster distance update unit 170, and a cluster distance memory unit 172.

[0085] The clustering device takes encrypted data [[X]] as input, clusters the data while keeping it secret, and outputs a distance table [[D out ]] is output.

[0086] For example, the data [[X]] is an encrypted m × n matrix. As mentioned above, when the input for a private shuffle or private sort is a matrix, the shuffle is performed by swapping each row while preserving the correspondence between each row.

[0087] The clustering device is a special device configured by loading a special program into a publicly known or dedicated computer having, for example, a central processing unit (CPU), a main storage device (RAM), and the like. The clustering device executes each process under the control of, for example, the central processing unit. Data input to the clustering device and data obtained in each process are stored in, for example, the main storage device, and the data stored in the main storage device is read out to the central processing unit as necessary and used for other processes. At least a part of each processing unit of the clustering device may be configured by hardware such as an integrated circuit. Each storage unit of the clustering device may be configured by, for example, a main storage device such as a random access memory (RAM), or middleware such as a relational database or a key-value store. However, each storage unit does not necessarily need to be provided inside the clustering device, and may be configured by an auxiliary storage device configured by a semiconductor memory element such as a hard disk, optical disk, or flash memory, and may be configured to be provided outside the clustering device.

[0088] Each part will be explained below.

[0089] <Initialization section 110> The initialization unit 110 receives data [[X]] as input, assigns numbers (cluster IDs) to each row of data in sequence starting from 0, and creates a cluster ID table [[C id ]] is initialized and stored in the cluster ID storage unit 152. In addition, a variable [[cid new ]] to [[cid new Let ]]=[[m]], initialize, and output.

[0090] The initialization unit 110 generates an output distance table [[Dout ]] is initialized as an empty table and stored in output distance storage unit 162.

[0091] <Data Distance Calculation Unit 120 and Data Distance Storage Unit 122> The data distance calculation unit 120 receives data [[X]] as input, calculates the distances between all data, and generates a data distance table [[D data ]] is obtained (S120) and stored in inter-data distance storage unit 122. The method of calculating the distance is as described above, and the calculation can be performed while keeping it secret.

[0092] In addition, since each data point is initially considered to be a cluster, the inter-category distance table [[D clust ]] and data distance table [[D data ]] is the same table. Therefore, the data distance calculation unit 120 calculates the calculated [[D clust ]](=[[D data ]]) is stored in the inter-cluster distance storage unit 172.

[0093] <Bonded cluster identification unit 140> The combined cluster identifying unit 140 retrieves the cluster distance table [[D clust ]], obtains two cluster IDs ([[cid1]], [[cid2]]) indicating the closest clusters and their distances [[d]] (S140), and outputs them. For example, the combined cluster identifying unit 140 creates an inter-cluster distance table [[D clust After sorting ]], take the top element (the element with the smallest distance).

[0094] <Cluster ID update unit 150> The cluster ID update unit 150 updates two cluster IDs ([[cid1]], [[cid2]]) that indicate the closest clusters with the variable [[cid new ]] is input, and the cluster ID table before the update [[C id ]] and the cluster ID table [[Cid The two cluster IDs, [[cid1]] and [[cid2]], of new ]] (S150), and the updated cluster ID table [[C id ]] is stored in the cluster ID storage unit 152. For example, the cluster ID table [[C id ]]. Note that the variable [[cid new ]] is incremented every time an update process is performed.

[0095] <Output Distance Table Update Unit 160 and Output Distance Storage Unit 162> The output distance table update unit 160 receives two cluster IDs ([[cid1]], [[cid2]]) (before update) indicating the closest clusters and their distance [[d]], and stores the output distance table [[D out ]] and output distance table [[D out Add [[cid1]], [[cid2]], [[d]] to the end of the output distance table [[D out ]] and update the output distance table [[D out ]] is stored in output distance storage unit 162.

[0096] <Inter-cluster distance update unit 170 and inter-cluster distance storage unit 172> The inter-cluster distance update unit 170 receives two cluster IDs ([[cid1]], [[cid2]]) (before update) indicating the closest clusters, and updates the inter-cluster distance table [[D clust ]] and store the unnecessary information in the cluster distance table [[D clust For example, the deletion is performed by the method described in Algorithm 3 above.

[0097] Furthermore, the inter-cluster distance update unit 170 reads the inter-data distance table [[D data]], calculate the distance between the newly merged cluster and other clusters, and create a cluster distance table [[D clust ]] to create the cluster distance table [[D clust ]] is updated (S170-2). For example, the distance between the newly combined cluster and the other clusters is calculated using the method described above in (Method of calculating distance between clusters).

[0098] The inter-cluster distance update unit 170 updates the inter-cluster distance table [[D clust ]] is stored in inter-cluster distance storage unit 172.

[0099] The cluster device repeats S140, S150, S160, S170-1, and S170-2 by cluster ID update processing S150 until the number of clusters becomes one (YES in S151), and outputs the output distance table [[D out ]] is output (NO in S151).

[0100] <Effects> With the above configuration, hierarchical clustering can be performed safely while keeping all calculation processes and intermediate values ​​secret.

[0101] (Distance information management [Method 2]) In the hierarchical clustering [Method 2] of the second embodiment, it is necessary to store and update the distance between clusters. In the secure hierarchical clustering of the second embodiment, the two tables (D clust ,D out ) to manage distance information, and the data distance table D shown in the figure data Do not use.

[0102] · Cluster Distance Table D clust Output Distance Table D out <Main algorithm of secure hierarchical clustering [Method 2]> The main algorithm of secret calculation hierarchical clustering [Method 2] is shown in Fig. 14. The difference from [Method 1] is that the calculation of the table [[D data of the distances between all data [[D data = calcDataDist([[X]]) performed in [Method 1] is not performed in [Method 2], and the details of the processing method of updateClustDist are different.

[0103] <updateClustDist[Method 2]> In the second embodiment, [Method 2] is used as the processing method of the function updateClustDist in the 9th line. updateClustDist is a process for updating the distance table between all clusters, and mainly consists of the following three processes.

[0104] 1. Calculate the distance between the newly combined cluster and other clusters.

[0105] 2. Delete the information that becomes unnecessary due to the combination of clusters from [[D clust .

[0106] 3. Add the newly calculated distance to [[D clust .

[0107] (Calculation method of distance between clusters [Lance-Williams updating formula]) In the second embodiment, the Lance-Williams updating formula is used as the calculation method of the distance between clusters.

[0108] First, the calculation procedure of the distance between the newly formed cluster and other clusters due to the combination of clusters will be described. The information required when calculating the distance between clusters by the group average method using the Lance-Williams updating formula is only the number of data included in the clusters before combination and the distance from other clusters. In order to facilitate the calculation in secret calculation in advance, the formula (3) is transformed as follows.

number

[0109] Number of data included in cluster ID=1 Number of data included in cluster ID=2 ·Distance between clusters ID=1 and ID=0 and distance between clusters ID=2 and ID=0 ·Distance between clusters ID=1 and ID=3 and distance between clusters ID=2 and ID=3 A specific image of the processing is shown in Figure 15.

[0110] The number of data contained in the cluster with ID=1 and the number of data contained in the cluster with ID=2 can be easily calculated, for example, by the following procedure. This will give the denominator of equation (7).

[0111] 1. Determine for each row whether the value of the cluster ID for which you want to know the number of data matches the value in the cluster ID column of the cluster ID table (rows that match are 1, rows that do not match are 0).

[0112] 2.The sum of all the results above is the number of data in the cluster.

[0113] Next, calculate the numerator of formula (7). The general flow of calculating the numerator is as follows. Figure 16 shows the diagrams corresponding to each step.

[0114] 1. From the table of distances between all clusters, extract the rows needed to calculate new distances.

[0115] 2. Multiply the distance for the cluster with ID=1 by the number of data in the cluster with ID=1.

[0116] 3. Multiply the distance for the cluster with ID=2 by the number of data in the cluster with ID=2.

[0117] 4. Add the results of steps 2 and 3.

[0118] First, we explain how to achieve step 1. The lines required to calculate the new distance must simultaneously satisfy the following two conditions: Either cluster ID1 or ID2 is the ID of the cluster to be joined.

[0119] The other cluster ID is the ID of the cluster that is not included in the cluster to be joined.

[0120] For example, in this example, we need to extract only the rows where either cluster ID1 or ID2 has ID=1 or ID=2, and the other has ID=0 or ID=3.

[0121] Next, we will explain how to achieve steps 2 and 3. We will omit the number of data in each cluster because it is known when calculating the denominator earlier. As a method for multiplying the number of data only by the distance related to the cluster with ID=1, for example, the following procedure can be considered.

[0122] 1.For the cluster ID 1 column of the table created in step 1, determine whether the ID is 1 (if it is 1, then it is 1; if not, then it is 0).

[0123] 2. Similarly, for the ID2 column, determine whether the ID is 1.

[0124] 3. Take the logical sum of the above two judgment results.

[0125] 4. Multiply the result of the logical OR by the number of data items.

[0126] 5. Multiply the distance column by the result above.

[0127] By performing this operation, the number of data items is multiplied only by the distance related to the cluster with ID=1, and the other distances are all 0. The same operation is then performed on the cluster with ID=2, and the two results are added together to obtain the result of step 3.

[0128] Finally, processing like step 4 can be achieved by using groupBySum.

[0129] Since the numerator of equation (7) has been calculated in steps 1 to 4, by dividing it by the previously calculated denominator (the number of data contained in the cluster to be merged), the distance between the newly formed cluster and other clusters can be calculated.

[0130] A clustering device that realizes the above-mentioned secure computation hierarchical clustering will now be described.

[0131] Second Embodiment FIG. 17 is a functional block diagram of a clustering device according to the second embodiment, and FIG. 18 shows the processing flow thereof.

[0132] The clustering device of the second embodiment includes an initialization unit 110, a joining cluster identification unit 140, a cluster ID update unit 150, a cluster ID storage unit 152, an output distance table update unit 160, an output distance storage unit 162, an inter-cluster distance update unit 270, and an inter-cluster distance storage unit 172.

[0133] As described above, the second embodiment does not use inter-data distance, and therefore the inter-data distance calculation unit 120 and the inter-data distance storage unit 122 present in the first embodiment are omitted. Also, the second embodiment calculates inter-cluster distance using the Lance-Williams updating formula, and therefore the inter-cluster distance update unit 170 in the first embodiment is changed to an inter-cluster distance update unit 270 in this embodiment. Step S270-2, which is different from the first embodiment, will be described below.

[0134] The inter-cluster distance update unit 270 updates the distance between the two clusters (C 1a ,C 1b ) the number of data included in each 1a ,n 1b and one of the clusters C 1a and the distance d(C 1a , C2), and the other cluster C 1b and the distance d(C 1b ,C2), the Lance-Williams updating formula (Equation 7) Based on this, calculate the distance between the newly merged cluster and other clusters and create a cluster distance table [[D clust ]] to create the cluster distance table [[D clust The details of the process are described in (Calculation method for distance between clusters [Lance-Williams updating formula]).

[0135] <Modification> In the above embodiment, the Euclidean distance is used as a method for calculating the distance between data, and the group average method is used as a method for calculating the distance between clusters, but other methods may also be used for each.

[0136] <Other Modifications> The present invention is not limited to the above-mentioned embodiment and modified examples. For example, the above-mentioned various processes may be executed not only in chronological order as described, but also in parallel or individually depending on the processing capacity of the device executing the processes or as necessary. In addition, appropriate modifications are possible within the scope of the present invention.

[0137] <Programs and recording media> The various processes described above can be implemented by loading a program for executing each step of the above method into the memory unit 2020 of a computer shown in FIG. 19 and operating the control unit 2010, input unit 2030, output unit 2040, etc.

[0138] The program describing the processing contents can be recorded on a computer-readable recording medium, which may be, for example, a magnetic recording device, an optical disk, a magneto-optical recording medium, a semiconductor memory, or any other type of recording medium.

[0139] The program may be distributed, for example, by selling, transferring, lending, etc. portable recording media such as DVDs and CD-ROMs on which the program is recorded. Furthermore, the program may be distributed by storing the program in a storage device of a server computer and transferring the program from the server computer to other computers via a network.

[0140] A computer that executes such a program, for example, first stores the program recorded on a portable recording medium or the program transferred from a server computer in its own storage device. Then, when executing a process, the computer reads the program stored in its own recording medium and executes a process according to the read program. As another execution form of the program, the computer may directly read the program from the portable recording medium and execute a process according to the program, or may execute a process according to the received program each time a program is transferred from the server computer to the computer. The above-mentioned process may be executed by a so-called ASP (Application Service Provider) type service that realizes a processing function only by issuing an execution instruction and obtaining a result, without transferring a program from the server computer to the computer. Note that the program in this embodiment includes information used for processing by an electronic computer and equivalent to a program (data that is not a direct command to the computer but has a nature that specifies the processing of the computer, etc.).

[0141] In addition, in this embodiment, the present device is configured by executing a specific program on a computer, but at least a part of the processing contents may be realized by hardware.

Claims

1. a cluster ID update unit that combines two closest clusters and updates the cluster IDs in a cluster ID table that has a one-to-one correspondence between data IDs and cluster IDs; a cluster distance update unit that executes a deletion process for deleting information corresponding to a cluster to be joined from a cluster distance table, which is a table of distances between all clusters, and an addition process for adding distances between the newly joined cluster and other clusters to the cluster distance table, thereby updating the cluster distance table; The information in the cluster ID table and the inter-cluster distance table is encrypted, performing the process in the cluster ID update unit and the additional process in the inter-cluster distance update unit using the encrypted information without decrypting it; Clustering device.

2. 2. The clustering device of claim 1, an output distance table update unit that adds two cluster IDs of the clusters to be joined and their distances to an output distance table that is a table of distances between the joined clusters, and updates the output distance table; The information in the output distance table is encrypted, performing processing in the output distance table update unit using the encrypted information without decrypting it; Clustering device.

3. The clustering device according to claim 1 or 2, the cluster distance update unit uses a data distance table, which is a table of distances between all data, and two cluster IDs of the clusters to be joined, to generate key information to be used for extracting necessary information from the data distance table when performing the addition process in the cluster distance update unit, and calculates the distance between the newly joined cluster in the cluster distance table and other clusters using the key information and the data distance table; Clustering device.

4. A clustering method performed by a clustering device, comprising: a cluster ID update step of combining the two closest clusters and updating the cluster IDs in a cluster ID table in which data IDs and cluster IDs are associated one-to-one; a cluster distance update step of executing a deletion process of deleting information corresponding to a cluster to be joined from a cluster distance table which is a table of distances between all clusters, and an addition process of adding distances between the newly joined cluster and other clusters to the cluster distance table, thereby updating the cluster distance table; The information in the cluster ID table and the inter-cluster distance table is encrypted, performing the process in the cluster ID update step and the additional process in the inter-cluster distance update step using the encrypted information without decrypting it; Clustering methods.

5. 5. The clustering method of claim 4, further comprising: an output distance table update step of adding two cluster IDs of the clusters to be joined and their distances to an output distance table, which is a table of distances between joined clusters, and updating the output distance table; The information in the output distance table is encrypted, performing processing in the output distance table update step using the encrypted information without decrypting it; Clustering methods.

6. The clustering method according to claim 4 or 5, further comprising the steps of: the cluster distance update step uses a data distance table, which is a table of distances between all data, and two cluster IDs of the clusters to be joined, to generate key information to be used for retrieving necessary information from the data distance table when performing the addition process in the cluster distance update step, and calculates the distance between the newly joined cluster in the cluster distance table and other clusters using the key information and the data distance table; Clustering methods.

7. A program for causing a computer to function as the clustering device according to any one of claims 1 to 3.

Images