Clustering device, clustering method and program

Abstract

To provide an apparatus, method and program for efficiently clustering sparse data.SOLUTION: A clustering apparatus is configured to cluster object feature vectors composed of one or more features representing an object and feature quantities thereof, into a plurality of clusters, the clustering apparatus being configured to: store data, for each feature identifier identifying the feature, the data being composed of sequence elements that store pairs formed by associating a cluster identifier identifying a cluster with a first non-zero feature quantity related to a feature identifier of a mean feature vector that represents an average of the object feature vectors belonging to the cluster; calculating, based on the data, similarity between mean feature vectors of the clusters and object feature vectors of sparse representation that represents object feature vectors with a pair formed by associating a second non-zero feature quantity of the object feature vectors with a feature identifier of the second non-zero feature quantity; and allocating, based on the similarity, the object feature vectors to one of the multiple clusters.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] The present disclosure relates to a clustering device, a clustering method, and a program. [Background technology]

[0002] Clustering methods, such as the k-means method, are known as a method for efficiently processing large amounts of information. Clustering methods can group multiple pieces of information into one or divide a given set of information into multiple groups using a measure that represents the relationship between each piece of information (similarity, dissimilarity, distance, etc.). Hereinafter, the "information" that is the target of clustering will be referred to as an "object."

[0003] In the k-means method, for example, vector data representing the features of an object (hereinafter referred to as an object feature vector) is expressed as x i Let the data set X={x0,x1,...,x N-1} and the number of clusters K, data set X is divided into K clusters. In the k-means method, for example, cluster C j Each object feature vector x i Vector μ representing the mean of j (hereinafter referred to as the mean feature vector) is used as the objective function, and each object feature vector x is calculated to maximize the objective function. i Cluster C to which j However, since calculating the exact solution for the k-means method requires a large amount of calculation, a heuristic method called the Lloyd method is often used.

[0004] In recent years, object feature vectors are often represented by high-dimensional, sparse data. For example, when objects include documents, purchase histories that represent the relationship between purchasers (or users) and purchased goods (or items), and images represented by a bag-of-visual words, these objects are often represented by high-dimensional, sparse object feature vectors. Hereinafter, a collection of large amounts of high-dimensional, sparse data is referred to as a large-scale, high-dimensional, sparse data set.

[0005] The Lloyd method consists of an initial state setting step, an assignment step, an update step, and a termination decision step. In the assignment step, each object feature vector x i and each mean feature vector μ j Since it is necessary to calculate the inner product of , the time complexity of the allocation step is dominant. Therefore, there is a need to reduce the time complexity of the allocation step.

[0006] The assignment step of the Lloyd method is equivalent to a most similar search problem if the set of object feature vectors is considered as a query object set and the set of mean feature vectors is considered as a searched data object set. One of the representative methods that can quickly solve the most similar search problem for large-scale, high-dimensional sparse data sets is a method that uses an inverted file (also called an inverted index) as a data structure. For example, Non-Patent Documents 1 and 2 propose high-speed methods that efficiently use inverted files. Furthermore, Non-Patent Document 3 proposes a high-speed method that expresses a mean feature vector set as an inverted file. Although not a method that uses an inverted file, Non-Patent Document 4 proposes a method (called the Ding method) that accelerates the Lloyd method by using an upper limit value of similarity (or a lower limit value of distance). [Prior art documents] [Non-patent literature]

[0007] [Non-Patent Document 1] M. Fontoura, V. Josifovski, J. Liu, S. Venkatesan, X. Zhu, and J. Zien, "Evaluation strategies for top-k queries over memory-resident inverted indexes," Proc. VLDB Endowment, vol. 5, no. 12, 2011. [Non-patent document 2] Y. Li, J. Wang, B. Pullman, N. Bandeira, and Y. Papakonstantinou, "Index-based, high-dimensional, cosine threshold querying with optimality guarantees," Proc. Int. Conf. Database Theory (ICDT), 2019. [Non-patent document 3] Kazuo Aoyama and Kazumi Saito, "Structured inverted-file k-means clustering for high-dimensional sparse data," arXiv: 2103.16141, https: / / arxiv.org / abs / 2103.16141. [Non-patent document 4] Y. Ding, Y. Zhao, X. Shen, M. Musuvathi, and T. Mykowicz, "Yinyang K-means: A drop-in replacement of the classic K-means with consistent speedup," Proc. Int. Conf. Machine Learning (ICML), 2015. Summary of the Invention [Problem to be solved by the invention]

[0008] However, further speedup of clustering methods for large-scale, high-dimensional sparse data sets is desired.

[0009] The present disclosure has been made in consideration of the above points, and provides a technique for efficiently clustering sparse data. [Means for solving the problem]

[0010] A clustering device according to one aspect of the present disclosure is a clustering device that clusters object feature vectors, each of which is composed of one or more features that represent an object and one or more feature values ​​that are the respective values ​​of the one or more features, into a plurality of clusters, and includes: a memory control unit configured to store, in a memory unit, data composed of array elements that store, for each feature identifier that identifies the feature, a pair of a cluster identifier that identifies the cluster and a first non-zero feature value related to the feature identifier of a mean feature vector that represents the mean of object feature vectors belonging to the cluster; and a memory control unit configured to store, based on the data, data composed of array elements that store, for each feature identifier that identifies the feature, a pair of a second non-zero feature value of the object feature vector and a feature identifier of the second non-zero feature value. The data includes a similarity calculation unit configured to calculate a similarity between a sparsely represented object feature vector representing the object feature vector and the mean feature vector of each cluster, and an allocation unit configured to allocate the object feature vector to one of the plurality of clusters based on the similarity, wherein the data is composed of first array elements representing the array elements whose feature identifier is less than a first threshold, second array elements representing the array elements whose feature identifier is equal to or greater than the first threshold and whose first non-zero feature value is equal to or greater than a second threshold, and third array elements representing the array elements whose feature identifier is equal to or greater than the first threshold and whose first non-zero feature value is less than the second threshold. [Effects of the Invention]

[0011] Techniques are provided for efficiently clustering sparse data. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 10 is a diagram schematically illustrating an example of the relationship between df and the number of words that have that df. [Figure 2] This is a diagram (part 1) showing an example of the relationship between df and the number of words that have that df. [Figure 3] This is a diagram (part 2) showing an example of the relationship between df and the number of words that have that df. [Figure 4] This is a diagram (part 3) showing an example of the relationship between df and the number of words that have that df. [Figure 5] FIG. 10 is a diagram illustrating an example of the relationship between df for a certain word and the number of mean feature vectors including features related to that word. [Figure 6] FIG. 10 is a diagram schematically illustrating an example of the number of multiplications in one iteration of similarity calculation in the k-means method. [Figure 7] FIG. 10 is a diagram schematically illustrating an example of a mean feature vector set in an inverted file representation in the first embodiment. [Figure 8] FIG. 2 illustrates an example of a hardware configuration of a clustering device according to a first embodiment. [Figure 9] FIG. 2 illustrates an example of a functional configuration of a clustering device according to a first embodiment. [Figure 10] 10 is a flowchart illustrating an example of a clustering process according to the first embodiment. [Figure 11] 10 is an algorithm showing an example of an allocation step in the first embodiment. [Figure 12] FIG. 10 is a diagram illustrating an example of evaluation of the number of approximate multiplications. [Figure 13] FIG. 1 is a diagram (part 1) showing an example of the experimental results of Example 1. [Figure 14] FIG. 2 is a diagram (part 2) showing an example of the experimental results of Example 1. [Figure 15] FIG. 10 is a diagram (part 3) showing an example of the experimental results of Example 1. [Figure 16] FIG. 11 is a diagram schematically illustrating an example of a mean feature vector set in an inverted file representation in the second embodiment. [Figure 17]10 is an algorithm showing an example of an allocation step in the second embodiment. [Figure 18] FIG. 1 is a diagram (part 1) showing an example of the experimental results of Example 2. [Figure 19] FIG. 2 is a diagram (part 2) showing an example of the experimental results of Example 2. DETAILED DESCRIPTION OF THE INVENTION

[0013] An embodiment of the present invention will be described below, which is a clustering device 10 capable of efficiently clustering sparse data.

[0014] [Preparation] First, as a preparation for this embodiment, data to be clustered, the k-means method, and the Lloyd method will be described.

[0015] Data to be clustered In the following, we consider a large-scale, high-dimensional, sparse data set as the target of clustering. An example of such data is object feature vectors that represent the characteristics of documents when clustering a large number of documents as objects.

[0016] When the object is a document, the bag-of-words model (hereinafter referred to as "BoW") is often used to treat the object as a point in Euclidean space, which is a feature space. Note that documents are also called texts or documents, but these are collectively referred to as "documents."

[0017] In BoW, a document is divided into blocks of character strings represented by the words that appear in it, and each word is considered one of the document's features. A quantity corresponding to the feature is then used as a feature, and the document is represented as a vector whose elements are these feature quantities. Note that words are also called vocabulary or terms, but from here on, we will refer to words, vocabulary, and terms collectively as "words."

[0018] A feature is expressed as a function value whose input variables are the frequency with which a certain word appears in a document (term frequency) and the frequency with which the word appears in each document of an object set (document set). For example, the function is often tf-idf (term-frequency-inverse document frequency), and a typical function is as follows:

[0019] tf(t h ,x i ;X): Given X, x i Appears in h Frequency of df(t h ,X): Given X, t h The number of documents (objects) containing h idf(t h ;X)=log(N / n h ) tf-idf(t h ,x i ;X)=tf(t h ,x i ;X)×idf(t h ;X) Here, X is the object feature vector set corresponding to the object set (document set). X={x0, x1, , x N-1} and x i (i=0,1,···,N-1) is the object feature vector. x i The number of dimensions (i.e., the number of features) is D. The number of objects |X| is |X|=N, and n h is the word t h The number of documents containing IDF is the number of documents that contain IDF. The base of the logarithm of the function representing IDF is not limited. The maximum value of h is the number of distinct words in the object set. Although subscripts (indexes) such as i may start from 1, in this embodiment, the indexes start from 0 to match the array indexes described later.

[0020] In this case, if one word that appears in an object is considered one dimension, the number of dimensions of the feature space is the number of different words in all objects included in the object set, which becomes very large. In other words, the feature space becomes a high-dimensional space.

[0021] On the other hand, since the number of words appearing in a certain object (document) is limited, the object feature vector x i is a sparse vector. That is, the object feature vector x i is a vector with a very small number of non-zero elements.

[0022] In this way, when a large amount of documents are clustered, the set of data (object feature vectors) to be clustered becomes a large set, and each object feature vector becomes a high-dimensional, sparse vector.

[0023] In addition to documents, objects such as purchase histories that represent the relationship between purchasers (or users) and purchased goods (or items), and images whose extracted features are expressed as bag-of-visual words, can also be treated as documents. In other words, feature vectors of a large amount of purchase histories and feature vectors of a large amount of images expressed as bag-of-visual words are also large-scale, high-dimensional sparse data sets. In the following, we will explain the case where the object is a document as an example.

[0024] For simplicity, the relationship between object feature vectors is expressed as cosine similarity using normalized object feature vectors. In this case, the similarity between two feature vectors (object feature vector, mean feature vector) is their inner product.

[0025] ·k-means method The k-means method is a set of object feature vectors X = {x0, x1, , x N-1This is a method for dividing an object feature vector set X into K clusters when {x,y} and the number of clusters K are given. The k-means method, which uses the inner product as the similarity measure, is sometimes called the spherical k-means method (hyperspherical k-means). Hereinafter, the k-means method and the spherical k-means method will be considered the same.

[0026] In the k-means method, when dividing the object feature vector set X into K clusters, each object feature vector x i and each cluster C j The mean feature vector μ j The objective function is the inner product of x and x, and each object feature vector x is calculated to maximize the objective function. i Cluster C to which j That is, each object feature vector x is determined so as to maximize the objective function J(C;X,K) shown in the following equation (1). i Cluster C to which j where C={C0,C1,...,C K-1}.

[0027]

number

[0028] ·Lloyd method Since solving the above equation (1) exactly requires a large amount of calculation, an iterative heuristic method called the Lloyd method is often used. Below, we will explain the outline of the Lloyd method used in the k-means method. The Lloyd method consists of the following steps: initial state setting step, allocation step, update step, and termination judgment step.

[0029] S1: (Initial state setting step) K mean feature vectors μ j Initialize (j=0,1,...,K-1) to an appropriate value.

[0030] S2: (Assignment step) Object feature vector x i For each of the mean feature vectors μ j and the object feature vector x i Calculate the inner product (Equation (1)) of the mean feature vector μ j Cluster C corresponding to j The object feature vector x i Assign.

[0031] S3: (Update step) All object feature vectors x i After the allocation of j object feature vector x belonging to i By using the mean feature vector μ j Calculate and update

[0032] S4: (Termination determination step) It is determined whether or not a predetermined termination condition is met. If it is determined that the predetermined termination condition is met, the process is terminated, whereas if it is determined that the predetermined termination condition is not met, the process returns to the allocation step of S2 described above. As a result, S2 and S3 are repeatedly executed until the predetermined termination condition is met. This repetition is also called an iteration, and the currently executed iteration is referred to as the "current iteration", and the iteration executed immediately before the current iteration is referred to as the "previous iteration", etc. The predetermined termination condition may be, for example, i belongs to cluster C j becomes immutable.

[0033] Here, the Lloyd method requires a large amount of time for the assignment step of S2. Therefore, in order to speed up the Lloyd method, it is necessary to shorten the time required for the assignment step, that is, the time required for all object feature vectors x i and all mean feature vectors μ j Therefore, in the following, a method for reducing the time required for calculating the inner product is proposed, and this proposed method is used to cluster a large-scale, high-dimensional sparse data set by the clustering device 10.

[0034] [Example 1] The first embodiment will be described below.

[0035] <Proposed method> When dealing with high-dimensional, sparse object feature vectors, a technique of sparsely representing the object feature vector is often used (Non-Patent Document 3, Reference Document 1). A sparsely represented object feature vector is represented as a pair (tuple) of a feature ID (e.g., a word ID in the case of a document) indicating the position of a non-zero element in the object feature vector and a feature value (e.g., tf-idf in the case of a document) indicating the value of the non-zero element. Specifically, if the object is a document, the object feature vector is represented as a pair (word ID, non-zero value) where the word ID is the word ID and the feature value related to the word of the word ID is the non-zero value. By adopting such a sparse representation, the memory size used by the object feature vector set can be reduced. In particular, when the object set is large, sparse representation is essential from the perspective of efficient use of memory, which corresponds to the amount of space calculation. The feature ID may also be called, for example, a feature identifier.

[0036] In the following, as in Non-Patent Document 3 and Reference 1, the object feature vector x i is assumed to be sparsely represented. Also, the sparsely represented object feature vector x i is the object feature vector x iLet the number of non - zero elements contained in it be num_terms[i], which is realized by the two - dimensional array Obj[i][t] (t = 0, 1, ···, num_terms[i] - 1), and the t - th feature ID (termID) of x is accessed by Obj[i][t].termID, and the t - th feature quantity of x is accessed by Obj[i][t].Feature i respectively. Also, assume that the feature IDs are sorted such that 0 ≤ Obj[i][0].termID < Obj[i][1].termID < ··· < Obj[i][num_terms[i] - 1].termID ≤ D - 1. Note that the two - dimensional array Obj[i][t] is assumed to be a structure array with termID and Feature, but this is just an example and not limited to this. For example, the object feature vector x i may be realized by a two - dimensional array of termID and a two - dimensional array of Feature i On the other hand, since the mean feature vector μ

[0037] is also used in the inner - product calculation, the mean feature vector μ j is also used in the inner - product calculation, so the mean feature vector μ jThe representation method or data structure of the mean feature vector set is also important for efficiently executing the Lloyd method. Non-Patent Documents 1 and 2 describe a naive method (called naive IVF) in which a mean feature vector set is represented as a transposed file. Non-Patent Document 3 and Reference 1 describe a method (called SIVF) that introduces an invariant centroid-pair skipping filter (ICP) into naive IVF. Here, the mean feature vector set represented as a transposed file is represented by a tuple consisting of the ID of the mean feature vector (hereinafter referred to as meanID, where meanID is the cluster ID) that includes the feature of that feature ID, for each feature ID (termID). For example, the set of mean feature vectors in the transposed file representation can be realized as a two-dimensional array INV'[t][h] (t=0,1,···,D-1,h=0,1,···,MF[t]-1), where D is the total number of feature IDs (termIDs) and MF[t] is the number of mean feature vectors containing the feature of the t-th feature ID. In this case, INV'[t][h].meanID is the mean feature vector μ containing the feature of the t-th feature ID. j Among them, the h-th mean feature vector μ j In addition, INV'[t][h].Feature can access the mean feature vector μ j It is assumed that the feature quantities of the two-dimensional array INV'[t][h] can be accessed. Note that the two-dimensional array INV'[t][h] is a structure array having meanID and Feature, but this is just an example and is not limiting. For example, a mean feature vector set may be realized by a two-dimensional array of termID and a two-dimensional array of Feature.

[0038] In the following, each column of the mean feature vector set INV'[t][h] expressed in the inverted file (i.e., INV'[t][0], , INV'[t][MF[t]-1]) is called an inverted-file array. For simplicity, the t-th feature ID is assumed to be t.

[0039] In this proposed method, the mean feature vector set is represented as an inverted file, and the inverted file is divided into three regions according to the time required to calculate the dot product with the object feature vector. Furthermore, for the region with the highest time calculation, the part that performs the exact calculation of the dot product (similarity) using the similarity upper limit is extracted, thereby reducing the time calculation. The data structure of the inverted file in this proposed method is explained below.

[0040] The document frequency (df) of a word contained in a document and the number of words with that df are expressed as f(x)=ax, where a and b are coefficients, as shown in Figure 1. -b Here, x is df and f(x) is the number of words. In other words, a power law relationship between df and the number of words approximately holds, which is a universal property of large-scale, high-dimensional sparse datasets. In the graph shown in Figure 1, the vertical axis represents the logarithmic scale of the number of words, and the horizontal axis represents the logarithmic scale of df.

[0041] Specific examples are shown in Figures 2 to 4. The example shown in Figure 2 shows the relationship between df for words contained in a PubMed database of a certain size and the number of words with that df. The example shown in Figure 3 shows the relationship between df for words contained in a PubMed database of a larger size than the example shown in Figure 2 and the number of words with that df. The example shown in Figure 4 shows the relationship between df for words contained in a newspaper article of a certain size and the number of words with that df. It can be seen that the power law holds true in all of the examples shown in Figures 2 to 4. Note that in all of Figures 2 to 4, the vertical axis represents the logarithmic scale of the number of words, and the horizontal axis represents the logarithmic scale of df.

[0042] Next, when clustering using the k-means method, there is a positive correlation between df for a word and the number of mean feature vectors (mf: mean frequency) containing features related to that word, with a slope of approximately 1. However, because the maximum value of mf is the maximum number of clusters K, in areas where df is large the slope becomes gentler and mf asymptotically approaches K, and may even become K. A specific example is shown in Figure 5. The example shown in Figure 5 shows the relationship between df for a word contained in the same PubMed database used in Figure 2 and the number of mean feature vectors (mf) containing features related to that word. It can be seen that df and mf are positively correlated, with a slope of approximately 1. Note that in the example shown in Figure 5, the vertical axis represents the logarithmic scale of mf, and the horizontal axis represents the logarithmic scale of df.

[0043] Considering the above two relationships, the number of multiplications in the similarity calculation for one iteration of clustering using the k-means method corresponds to the volume of the area enclosed by the curves in the rectangular parallelepiped shown in Figure 6. In other words, considering the mean feature vector set INV'[t][h] in the inverted file representation, as t becomes larger to a certain extent, df also increases, so the object feature vector x i The number of multiplications increases.

[0044] Therefore, in this embodiment, a new set of mean feature vectors INV[t][h] expressed in a transposed file is constructed from the set of mean feature vectors INV'[t][h] expressed in a transposed file by the following steps (a) to (c).

[0045] (a) Each feature ID and the inverted-file array corresponding to that feature ID in INV'[t][h] are sorted in ascending order of df for the words related to that feature ID. After sorting, the feature IDs are reassigned in order from the beginning. Here, reassigning the feature IDs in order from the beginning means, for example, that the result of sorting the feature ID sequence {0, ,D-1} in ascending order of df is {m0, ,m D-1}, then m0, ,m D-1 The idea is to reassign the feature IDs to 0, ,D-1 respectively.

[0046] (b) Set a threshold ThTerm for the index t representing the t-th feature ID, and let the region satisfying INV'[t][h] (t < ThTerm) be region R1.

[0047] (c) For each t satisfying t ≥ ThTerm, after sorting in descending order of INV'[t][h].Feature, set a threshold ThVal for INV'[t][h].Feature. Let the region satisfying t ≥ ThTerm and INV'[t][h].Feature ≥ ThVal be region R2, and the region satisfying t ≥ ThTerm and INV'[t][h].Feature < ThVal be region R3.

[0048] However, in the above (c), for example, for each t satisfying t ≥ ThTerm, compare INV'[t][h].Feature with the threshold ThVal set for INV'[t][h].Feature. For the pair (INV'[t][h].meanID, INV'[t][h].Feature) satisfying INV'[t][h].Feature ≥ ThVal, assign numbers in order from h = 0 as local numbers. Let the transposed file part with the assigned numbers be region R2, and the remaining pairs be region R3. In this case, it is not necessary to sort INV'[t][h].Feature, and region R3 may not be represented as a transposed file (that is, region R may be represented as a standard file).

[0049] INV[t][h] constructed using the above (a) to (c) is shown in Figure 7. In the example shown in Figure 7, INV[t][h] is expressed in the term-mf plane. In this way, this embodiment uses a mean feature vector set expressed in a three-partition inverted file. As a result, in region R1, similarity calculations are performed using a method similar to naive IVF, while in region R3, which has a particularly large number of multiplications and requires a large amount of time, the time required for calculation can be reduced by extracting the part that strictly calculates the inner product (similarity) using the similarity upper limit. Note that in region R2, similarity calculations are performed, and calculations necessary for calculating the similarity upper limit are also performed. Hereinafter, INV[t][h] constructed using the above (a) to (c) will be referred to as the "mean feature vector set of the inverted file representation" or the "mean feature vector set of the three-partition inverted file representation."

[0050] <Example of Hardware Configuration of Clustering Device 10> An example of a hardware configuration of the clustering device 10 in the first embodiment is shown in Fig. 8. As shown in Fig. 8, the clustering device 10 in the first embodiment includes an input device 101, a display device 102, an external I / F 103, a communication I / F 104, a RAM (Random Access Memory) 105, a ROM (Read Only Memory) 106, an auxiliary storage device 107, and a processor 108. Each of these pieces of hardware is connected to each other via a bus 109 so as to be able to communicate with each other.

[0051] The input device 101 is, for example, a keyboard, a mouse, a touch panel, a physical button, etc. The display device 102 is, for example, a display, a display panel, etc. Note that the clustering device 10 does not necessarily have to have at least one of the input device 101 and the display device 102, for example.

[0052] The external I / F 103 is an interface with an external device such as a recording medium 103a. The clustering device 10 can read from and write to the recording medium 103a via the external I / F 103. Examples of the recording medium 103a include a flexible disk, a CD (Compact Disc), a DVD (Digital Versatile Disk), an SD memory card (Secure Digital memory card), and a USB (Universal Serial Bus) memory card.

[0053] The communication I / F 104 is an interface for connecting the clustering device 10 to a communication network. The RAM 105 is a volatile semiconductor memory (storage device) that temporarily stores programs and data. The ROM 106 is a non-volatile semiconductor memory (storage device) that can store programs and data even when the power is turned off. The auxiliary storage device 107 is a storage device (storage device) such as an HDD (Hard Disk Drive), an SSD (Solid State Drive), or a flash memory. The processor 108 is an arithmetic device such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit).

[0054] The clustering device 10 in the first embodiment has the hardware configuration shown in Fig. 8, and is thereby able to realize the clustering process described below. Note that the hardware configuration shown in Fig. 8 is merely an example, and the hardware configuration of the clustering device 10 is not limited to this. For example, the clustering device 10 may have multiple auxiliary storage devices 107 or multiple processors 108, may not have some of the hardware shown in the figure, or may have various hardware other than the hardware shown in the figure.

[0055] <Example of functional configuration of clustering device 10> FIG. 9 shows an example of a functional configuration of the clustering device 10 in the first embodiment. As shown in FIG. 9, the clustering device 10 in the first embodiment includes a preprocessing unit 110 and a clustering processing unit 120. These units are realized, for example, by a process in which one or more programs installed in the clustering device 10 are executed by the processor 108. The clustering device 10 in the first embodiment also includes a storage unit 130. The storage unit 130 is realized, for example, by the RAM 105 or the auxiliary storage device 107. Note that, for example, the preprocessing unit 110 and the clustering processing unit 120 may be realized by a process in which different programs are executed by the processor 108.

[0056] The preprocessing unit 110 processes a given object feature vector set X={x0, x1, . . . , x N-1 Each object feature vector x i Each of these is used as an object feature vector in sparse representation. The object feature vector in sparse representation may be created by the method described in Non-Patent Document 3 or Reference Document 1. In the following, the object feature vector in sparse representation is also referred to as x i (i=0,1,...,N-1), and the set of them is denoted by X. The sparse representation of the object feature vector x i The set X of the object feature vectors x is stored in the storage unit 130. i If x is already a sparse representation, the preprocessing unit 110 converts this object feature vector x i There is no need to have a sparse representation of

[0057] Furthermore, the preprocessing unit 110 calculates a threshold value ThTerm as preprocessing before the clustering process is executed. The threshold value ThTerm is stored in the storage unit 130, for example.

[0058] The clustering unit 120 calculates the sparsely represented object feature vector x iThe set X of clusters, the number of clusters K, and the thresholds ThTerm and ThVal are used as inputs to generate a sparse representation of the object feature vector x i into K clusters. Here, the clustering processing unit 120 includes an input unit 121, an initial setting unit 122, an allocation unit 123, an update unit 124, an end determination unit 125, and an output unit 126.

[0059] The input unit 121 receives a sparsely represented object feature vector x i The input unit 121 inputs the set X and the number of clusters K from, for example, the storage unit 130. However, the input unit 121 may input the set X and the number of clusters K that are output from, for example, another device or another program.

[0060] The initial setting unit 122 performs processing equivalent to the above-mentioned initial state setting step. At this time, the initial setting unit 122 creates a mean feature vector set INV[t][h] of the 3-division structure inverted file representation.

[0061] The allocation unit 123 performs processing equivalent to the allocation step described above. At this time, the allocation unit 123 calculates similarity in region R1 of the mean feature vector set INV[t][h] of the 3-partition structure inverted file representation by a method similar to naive IVF, while in region R2, it calculates similarity by a method similar to naive IVF while also performing calculations necessary for calculating the upper limit of similarity, and in region R3, it extracts a portion for which the inner product (similarity) is strictly calculated using the upper limit of similarity, and then calculates similarity only for the extracted portion.

[0062] The update unit 124 performs processing equivalent to the above-described update step. The update unit 124 also optimizes the threshold ThVal in an iteration with an iteration count of 2 or more (preferably, an iteration with an iteration count of 2). Before the threshold ThVal is optimized by the update unit 124, its initial value is used as the threshold ThVal. The threshold ThVal is stored in the storage unit 130.

[0063] The end determination unit 125 performs a process corresponding to the end determination step described above. The output unit 126 outputs, for example, the sparsely represented object feature vector x i Cluster C j The output unit 126 outputs the allocation result to each cluster C (which can be regarded as the clustering result of the object). j The mean feature vector μ j The output destination of the output unit 126 is not limited. Specific examples of the output destination include the display device 102 such as a display, the auxiliary storage device 107, the recording medium 103a, and other devices connected via a communication network.

[0064] 9 is an example, and the functional configuration of the clustering device 10 is not limited to this. For example, the clustering device 10 may not have the pre-processing unit 110, and another device may have the pre-processing unit 110. In this case, the clustering unit 120 of the clustering device 10 may perform the clustering on the sparsely represented object vector x created by the pre-processing unit 110 of the other device. i Just input the set X and the threshold ThTerm.

[0065] <Clustering process> The clustering process in the first embodiment will be described below with reference to Fig. 10. It is assumed below that the threshold value ThTerm calculated by the preprocessing unit 110 is stored in the storage unit 130. It is also assumed below that the threshold value ThTerm having a predetermined initial value is stored in the storage unit 130 in the first iteration, and that the threshold value ThVal having a value optimized by the update unit 124 is stored in the storage unit 130 in the second iteration and thereafter. A method for calculating the threshold value ThTerm and a method for optimizing the threshold value ThVal will be described later.

[0066] Step S101: First, the input unit 121 of the clustering processing unit 120 receives the sparsely represented object feature vector x iInput the set \(X\), the number of clusters \(K\), and the threshold values \(ThTerm\) and \(ThVal\). Assume that \(|X| = N\), \(K\) is an integer and \(1 < K < N\). Hereinafter, assume that the set of object feature vectors \(X\) in sparse representation is realized by a two-dimensional array \(Obj[i][t]\) (\(i = 0, 1, \cdots, N - 1\), \(t = 0, 1, \cdots, num\_terms[i] - 1\)).

[0067] Step S102: Next, the initialization unit 122 of the clustering processing unit 120 initializes each mean feature vector \(\mu\) j (\(j = 0, 1, \cdots, K - 1\)) to appropriate values, and then represents the set of these mean feature vectors \(\mu\) j in a three-part structure transposed file. That is, the initialization unit 122 uses the mean feature vectors \(\mu\) j (\(j = 0, 1, \cdots, K - 1\)) after initialization, the threshold values \(ThTerm\) and \(ThVal\) to create a set of mean feature vectors in the three-part structure transposed file representation. However, the three-part structure transposed file does not have to be the same transposed file. The set of mean feature vectors in the three-part structure transposed file representation is stored in, for example, the storage unit 130. Hereinafter, assume that the set of mean feature vectors in the three-part structure transposed file representation is realized by a two-dimensional array \(INV[t][h]\) (\(t = 0, 1, \cdots, D - 1\), \(h = 0, \cdots, MF[t] - 1\)).

[0068] Note that any method can be used for the initialization method of the mean feature vector \(\mu\) j . For example, each mean feature vector \(\mu\) j can be initialized to random values, or any \(K\) different object feature vectors \(x\) i can be used as the mean feature vectors \(\mu\) j respectively, or each object feature vector \(x\) i in the sparse representation is randomly assigned to the cluster \(C\) j , and then the mean feature vector \(\mu\) j is calculated from the \(x\) i belonging to each cluster \(C\) j for initialization.

[0069] Step S103: Next, the allocation unit 123 of the clustering processing unit 120 uses the mean feature vector set of the 3-partition structure inverted file representation to sparsely represent the object feature vector x i For each of the mean feature vectors μ j The similarity between the cluster C and the cluster C is calculated based on the similarity. j The details of this assignment step will be described later. Note that the sparsely represented object feature vector x i Cluster C j The allocation result to is stored in the storage unit 130, for example.

[0070] Step S104: The update unit 124 of the clustering processing unit 120 updates each cluster C j The sparse representation of the object feature vector x i For example, the update unit 124 updates the mean feature vector set of the three-partition structure inverted file representation by j The sparse representation of the object feature vector x i Therefore, the cluster C j The mean feature vector μ j After calculating these mean feature vectors μ j The three-part structure transposed file may be updated by j You can recreate the three-part structure transposition file from

[0071] Furthermore, for example, when the current iteration is the second iteration, the update unit 124 of the clustering processing unit 120 optimizes the threshold ThVal. As a result, the threshold ThVal stored in the storage unit 130 is updated with the optimized threshold ThVal. However, optimization of the threshold ThVal is not limited to the second iteration. In order to reduce the amount of calculation required to optimize the threshold ThVal, it is desirable to perform optimization in the earliest iteration possible after the clustering result has stabilized to a certain extent. The method for optimizing the threshold ThVal will be described in detail later.

[0072] Step S105: The termination determination unit 125 of the clustering processing unit 120 determines whether a predetermined termination condition is satisfied. If it is determined that the predetermined termination condition is satisfied, the clustering processing unit 120 proceeds to step S106. On the other hand, if it is determined that the predetermined termination condition is not satisfied, the clustering processing unit 120 adds 1 to the iteration and returns to step S103.

[0073] Step S106: Finally, the output unit 126 of the clustering processing unit 120 outputs, for example, the sparsely represented object feature vector x i Cluster C j Outputs the allocation results to

[0074] Algorithm of allocation step in Example 1 The algorithm of the allocation step (step S103 in FIG. 10) in the first embodiment will be described with reference to FIG.

[0075] The allocation unit 123 repeatedly executes lines 2 to 19 for i=0, . . . , N-1 (line 1). The repetition of lines 2 to 19 for a certain i will be explained below.

[0076] The allocation unit 123 repeatedly executes lines 3 to 5 for t=0, , Local_ThTerm[i]-1 (line 2). Here, Local_ThTerm[i] is the smallest t that satisfies Obj[i][t].termID≧ThTerm. For example, if num_terms[i]=3 and Obj[i][0].termID to Obj[i][1].termID are smaller than ThTerm, then Local_ThTerm[i]=2. Below, we will explain the repetition of lines 3 to 5 for a certain t.

[0077] The allocation unit 123 sets Obj[i][t].termID for tid (line 3). Note that Obj[i][t].termID can take {0, , D-1}. Next, the allocation unit 123 repeatedly executes line 5 for h=0, , MF[tid]-1. Note that MF[tid] represents the number of mean feature vectors containing the feature value of feature ID "tid" (in other words, the number of tuples included in the inverted-file array corresponding to feature ID "tid"). The repetition of line 5 for a certain h will be explained below.

[0078] The assigning unit 123 adds the product of Obj[i][t].Feature and INV[t][h].Feature to Sim[k] (line 5), where k=INV[t][h].meanID. Here, Sim[k] is the object feature vector x i and the mean feature vector μ k This is a variable that stores the similarity (inner product) between k and k. Note that k = 0, , K-1.

[0079] The second to fifth lines above calculate the inner product of the mean feature vector set of the three-partition structure transposed file representation and the feature amounts belonging to region R1.

[0080] Next, the allocation unit 123 repeatedly executes the lines 7 to 10 for t = Local_ThTerm[i], ···, num_terms[i] - 1 (line 6). Hereinafter, the repetition of lines 7 to 10 for a certain t will be described.

[0081] The allocation unit 123 sets Obj[i][t].termID for tid (line 7). Next, the allocation unit 123 repeatedly executes the lines 9 to 10 for h = 0, ···, MF_ThVal[tid] - 1 (line 8). Here, MF_ThVal[tid] is the smallest h that satisfies INV[t][h].Feature < ThVal. For example, when MF[tid] = 5 and INV[t][0].Feature to INV[t][3].Feature are all greater than or equal to ThVal, MF_ThVal[tid] = 4. Hereinafter, the repetition of lines 9 to 10 for a certain h will be described.

[0082] The allocation unit 123 adds the product of Obj[i][t].Feature and INV[t][h].Feature to Sim[k] with k = INV[t][h].meanID (line 9). Next, the allocation unit 123 subtracts Obj[i][t].Feature from Obj_Partial_L1norm[i][k] with k = INV[t][h].meanID (line 10). Here, Obj_Partial_L1norm[i][k] (k = 0, ···, K - 1) is called the partial L1 norm. For example, at the start of the allocation step (or at any time before that), it is pre-calculated as the sum of Obj[i][t].Feature from t = Local_ThTerm[i] to t = num_terms[i] for k = 0, ···, K - 1 and i = 0, ···, N - 1. This partial L1 norm Obj_Partial_L1norm[i][k] is used when calculating the similarity upper limit (line 12 described later).

[0083] By the above lines 6 to 10, the inner product with the feature amount belonging to the region R2 of the mean feature vector set of the 3 - partition structure transposed file representation has been calculated.

[0084] Next, the allocation unit 123 repeatedly executes the 12th to 14th lines for k = 0, ···, K (line 11). Hereinafter, the repetition of the 12th to 14th lines for a certain k will be described.

[0085] The allocation unit 123 sets Sim[k] + ThVal × Obj_Partial_L1norm[i][k] to UpperBound[k] (line 12). Here, UpperBound[k] represents the similarity upper limit regarding the cluster C k represents the similarity upper limit regarding the cluster C.

[0086] Next, the allocation unit 123 determines whether UpperBound[k] < Prev_Sim[i] is satisfied (line 13). If UpperBound[k] < Prev_Sim[i] is satisfied, the 14th line is skipped. Here, Prev_Sim[i] is the similarity between the object feature vector x i and the mean feature vector of the cluster to which it belongs in the current iteration. This 13th line means that if the similarity upper limit of cluster k is smaller than the similarity between the object feature vector x i and the mean feature vector of the cluster to which it currently belongs, the exact calculation of the similarity regarding that cluster k is skipped.

[0087] If it is determined that UpperBound[k] < Prev_Sim[i] is not satisfied, the allocation unit 123 sets k to Candidate[candMeans] and Sim[k] to CandidateSim[candMeans], and then adds 1 to candMeans (line 14). It is assumed that the initial value of candMeans is 0. As a result, the index k of the cluster to be exactly calculated for similarity and its Sim[k] are set to Candidate[candMeans] and CandidateSim[candMeans], respectively.

[0088] Next, the allocation unit 123 repeatedly executes lines 16 to 18 for t=Local_ThTerm[i], . . . , num_terms[i]-1 (line 15). The repetition of lines 16 to 18 for a certain t will be described below.

[0089] The allocation unit 123 repeatedly executes lines 17 and 18 for s=0, . . . , candMeans-1 (line 16). The repetition of lines 17 and 18 for a certain s will be explained below.

[0090] The allocation unit 123 sets Candidate[s] to k (line 17). Next, the allocation unit 123 adds Obj[i][t].Feature×INV[t][h].Feature to CandidateSim[s] using h that satisfies INV[t][h].meanID=k (line 18).

[0091] Then, the allocation unit 123 assigns the object feature vector x to the cluster corresponding to Candidate[s] for which CandidateSim[s] (s=0, , candMeans-1) is maximized. i (Line 19) where the object feature vector x i The allocation of a can be equated with the allocation of an object.

[0092] As a result, objects are assigned to clusters in the assignment step. At this time, in the first embodiment, for clusters whose similarity to the mean feature vector of the cluster to which they currently belong is greater than the upper similarity limit (lines 12 to 18), the strict calculation of similarity is skipped. This makes it possible to reduce the number of multiplications in region R3, which has a particularly large time calculation amount, and thus the time calculation amount can be reduced.

[0093] Note that "ThVal × Obj_Partial_L1norm[i][k]" on line 12 can be replaced with the Cauchy-Schwarz inequality. For example, Mean_Partial_L2norm[k] is the L2 norm of INV[t][h].Feature for t, where t≧ThTerm, calculated using h such that INV[t][h].meanID=k. Alternatively, for k=0,···,K-1 and i=0,···,N-1, the sum of squares of Obj[i][t].Feature from t=Local_ThTerm[i] to t=num_terms[i] is calculated in advance, and the square of each Obj[i][t].Feature on line 10 is subtracted from this sum of squares, and finally the square root is taken, which is defined as Obj_Partial_L2norm[i][k]. In this case, according to the Cauchy-Schwarz inequality, the upper limit of the inner product is Mean_Partial_L2norm[k]+Obj_Partial_L2norm[i][k], so the sum of this upper limit of the inner product and Sim[k] may be set as UpperBound[k].

[0094] <Calculation method of the threshold value ThTerm and optimization method of the threshold value ThVal> A method for calculating the threshold value ThTerm and a method for optimizing the threshold value ThVal will be described below.

[0095] ·How to calculate the threshold ThTerm The preprocessing unit 110 calculates the threshold value ThTerm through the following steps 11 and 12 in preprocessing before the clustering process is executed.

[0096] Step 11: As shown in Figure 1, a power law relationship exists between the df of a word contained in a document and the number of words with that df. In other words, if x is df and f(x) is the number of words, then f(x) = ax with certain coefficients a and b. -b So, this f(x)=ax -b By extrapolating this relationship, we find the df where the number of words is 1 (i.e., the x where f(x) = 1).

[0097] Step 12: The feature ID (word ID) corresponding to x obtained in step 11 above is set as ThTerm.

[0098] Optimization method for the threshold ThVal The update unit 124 optimizes the threshold value ThVal through the following steps 21 and 22 in the update step in the iteration where the number of iterations is 2 or more (preferably the iteration where the number of iterations is 2).

[0099] Step 21: The number of approximate multiplications is calculated using ThVal as a variable. The method for calculating the number of approximate multiplications will be described later.

[0100] Step 22: The threshold value that minimizes the number of approximate multiplications calculated in step 21 above is set as ThVal. This ThVal is the optimized ThVal.

[0101] Approximate multiplication calculation method A method for calculating the approximate number of multiplications will be described below.

[0102] An object feature vector x i Let p(x) be the similarity distribution between the object feature vector and the mean feature vector. Here, x is a continuous value relaxation. i and each mean feature vector μ j The average similarity between the object and the feature vector x i Let s(i) be the similarity between an object feature vector and the mean feature vector of the cluster to which it currently belongs. Note that the symbol "μ" is used for both the mean feature vector and the average similarity between an object feature vector and each mean feature vector. However, in the following, please note that "μ" with a subscript represents the mean feature vector, while "μ" without a subscript represents the average similarity. The same applies to "μ'" described below, which also represents the average similarity.

[0103] Furthermore, let p'(x) be the upper limit value distribution of similarity. Furthermore, let u be ThTerm and v be ThVal, and let x be the object feature vector. i The average upper limit of similarity between each mean feature vector and μ'(n)=μ(i;u,v).

[0104] In this case, the following three assumptions are made:

[0105] Assumption 1: The similarity distribution function is expressed as follows:

[0106] p(x|x≧μ(i))=a·e -b(x-μ(i)) where:

[0107]

number

[0108] Assumption 2: The median and mean of the distribution function are approximately equal.

[0109]

number

[0110] Assumption 3: The upper similarity limit distribution p'(x) is a parallel translation of the similarity distribution p(x). That is, it can be expressed as p'(x-Δμ)=p(x) and Δμ=μ(i;u,v)-μ(i)=μ'-μ.

[0111] Under the above assumptions, the approximate number of multiplications c(u, v) is calculated by steps 31 to 33 below.

[0112] Step 31: Calculate the distribution function using the above assumptions 1 to 3. That is,

[0113]

number

[0114]

number

[0115] Step 32: The probability that the upper limit value of the similarity is equal to or greater than s(i) (i.e., the probability that the exact calculation of the similarity cannot be skipped) is calculated. This can be calculated as follows:

[0116]

number

[0117] Step 33: The sum of the following equations (2) and (3) is set as the approximate number of multiplications c(u, v).

[0118]

number

[0119]

number

[0120]

number

[0121] That is, the approximate number of multiplications c(u,v) is as follows:

[0122]

number

[0123]

number

[0124] The approximate number of multiplications c(u,v) can be calculated in this manner. As an example, FIG. 12 shows the results of evaluating the approximate number of multiplications c(u,v) using the same PubMed as the example shown in FIG. 3. In FIG. 12, Actual represents the actual number of multiplications, and Approximate represents the approximate number of multiplications c(u,v). As shown in FIG. 12, it can be seen that the approximate number of multiplications c(u,v) can accurately approximate the actual number of multiplications.

[0125] <Experimental Results of Example 1> Below, we will explain the results of an experiment conducted to evaluate the clustering device 10 in Example 1. In this experiment, the clustering device 10 was implemented on a general PC, and the same PubMed documents as the example shown in Figure 3 were used for comparison with existing methods.

[0126] The number of multiplications in similarity calculation per iteration is shown in Fig. 13. In Fig. 13, Proposed is the clustering device 10 in Example 1, and naive IVF and SIVF are existing methods. As shown in Fig. 13, it can be seen that the clustering device 10 in Example 1 can reduce the number of multiplications compared to the existing methods. In particular, it can be seen that the number of multiplications can be reduced to approximately 1 / 34 compared to naive IVF.

[0127] Next, the average elapsed time per iteration is shown in Fig. 14. As shown in Fig. 14, it can be seen that the clustering device 10 in Example 1 can reduce the average elapsed time compared to existing methods. In particular, it can be seen that the average elapsed time can be reduced to approximately 1 / 28 compared to the Ding method.

[0128] Next, memory usage is shown in Fig. 15. As shown in Fig. 15, it can be seen that the clustering device 10 in the first embodiment can reduce memory usage to approximately 1 / 7 compared to the Ding method.

[0129] As described above, the clustering device 10 according to the first embodiment can reduce the number of multiplications required for similarity calculation in the k-means algorithm, thereby reducing the average elapsed time per iteration. In addition, the amount of memory used can be reduced compared to the Ding algorithm.

[0130] [Example 2] Hereinafter, a description will be given of Example 2. Note that in Example 2, the description of the same components as those in Example 1 will be omitted.

[0131] <Proposed method> In the second embodiment, we introduce an invariant centroid-pair skipping filter (ICP) to the mean feature vector set of the 3-partition structure inverted file representation. ICP is a filter that calculates the mean feature vector set of an object x i Regarding C j and C j´ is an invariant cluster, and (x i ∈C j )∧(C j ≠C j´ ), then the object feature vector x i and the mean feature vector μ j´ This means that there is no need to calculate the similarity between an object feature vector x i belongs to cluster C j is an invariant cluster, then in the current iteration, the object feature vector x i is another invariant cluster C j´ This is because the object feature vector x i ) remained unchanged in cluster C. j This is what I mean.

[0132] Therefore, in the three-part structure inverted file in Example 2, the inverted-file array is divided into a first half and a second half in each region, with the first half storing mean feature vector tuples that are not invariant clusters and the second half storing mean feature vector tuples that are invariant clusters. The mean feature vectors in the three-part structure inverted file configured in this way are shown in FIG. 16. As shown in FIG. 16, in region R1, the first half of the inverted-file array stores mean feature vector tuples that are not invariant clusters, and the second half stores mean feature vector tuples that are invariant clusters. Meanwhile, in regions R2 and R3, the portions of the inverted-file array belonging to region R2 and region R3 are divided into first and second half portions, respectively. That is, in the portion of the inverted-file array belonging to region R2, mean feature vector tuples that are not invariant clusters and have feature values ​​equal to or greater than ThVal are stored in the first half, and mean feature vector tuples that are invariant clusters and have feature values ​​equal to or greater than ThVal are stored in the second half. Similarly, in the part of the inverted-file array belonging to region R3, tuples of mean feature vectors that are not invariant clusters and have feature values ​​less than ThVal are stored in the first half, and tuples of mean feature vectors that are invariant clusters and have feature values ​​less than ThVal are stored in the second half. However, the part belonging to region R3 may have the same configuration as in Example 1.

[0133] Cluster C j Whether or not is an invariant cluster is managed by the flag f[j], as in Reference 1. In the following, when f[j]=0, the cluster C j is an invariant cluster, and when f[j]=1, cluster C j is not an immutable cluster.

[0134] Algorithm of allocation step in Example 2 The algorithm of the allocation step (step S103 in FIG. 10) in the second embodiment will be described with reference to FIG.

[0135] The allocation unit 123 repeatedly executes lines 2 to 20 for i=0, . . . , N-1 (line 1). The repetition of lines 2 to 20 for a certain i will be explained below.

[0136] The assignment unit 123 assigns the object feature vector x i belongs to cluster C j is an invariant cluster (i.e., determines whether f[j]=0 or 1) (line 2). If f[j]=0, the allocation unit 123 executes lines 3 to 18, and if f[j]=1, the allocation unit 123 executes line 20.

[0137] When f[j]=0 (line 3), the allocation unit 123 repeatedly executes lines 5 to 7 for t=0, , Local_ThTerm[i]-1 (line 4). The repetition of lines 5 to 7 for a certain t will be explained below.

[0138] The allocation unit 123 sets Obj[i][t].termID for tid (line 5). Next, the allocation unit 123 repeatedly executes line 7 for h = 0,...,end0[tid]-1 (line 6). Here, end0[tid] represents the smallest h among the tuples INV[tid][h] of the mean feature vectors of the invariant cluster in the inverted-file array corresponding to the feature ID "tid". For example, if INV[tid][0] to INV[tid][h'-1] are tuples of the mean feature vectors that are not invariant clusters, and INV[tid][h'] to INV[tid][MF[tid]-1] are tuples of the mean feature vectors of the invariant cluster, then end0[tid] = h'.

[0139] The allocation unit 123 performs the same process as line 5 in Fig. 11 (line 7). The allocation unit 123 repeatedly executes lines 9 to 11 for t = Local_ThTerm[i], ,num_terms[i]-1 (line 8). The repetition of lines 9 to 11 for a certain t will be described below.

[0140] The allocation unit 123 sets Obj[i][t].termID for tid (line 9). Next, the allocation unit 123 repeatedly executes line 11 for h = 0,...,end0[tid]-1 (line 10). Here, end0[tid] represents the smallest h among the tuples INV[tid][h] of the mean feature vectors of the invariant cluster in the part of region R2 of the inverted-file array corresponding to feature ID "tid". For example, if INV[tid][0] to INV[tid][h'-1] are tuples of the mean feature vectors that are not invariant clusters, and INV[tid][h'] to INV[tid][MF_ThVal[tid]-1] are tuples of the mean feature vectors of the invariant cluster, then end1[tid] = h'.

[0141] The allocation unit 123 performs the same processing as lines 9 to 10 in Fig. 11 (line 11). Next, the allocation unit 123 performs the same processing as lines 11 to 14 in Fig. 11 (line 12), and then repeatedly executes lines 14 to 17 for t = Local_ThTerm[i], ,num_terms[i]-1 (line 13). The repetition of lines 14 to 17 for a certain t will be described below.

[0142] The allocation unit 123 sets Obj[i][t].termID for tid (line 14). Next, the allocation unit 123 repeatedly executes lines 16 and 17 for s=0, ,candMeans-1 (line 15). The repetition of lines 16 and 17 for a certain s will be explained below.

[0143] The allocation unit 123 sets Candidate[s] to k (line 16). Next, the allocation unit 123 adds Obj[i][t].Feature×PartialMeanIVFMV[tid-ThTerm][k] to CandidateSim[s] using PartialMeanIVFMV[tid-ThTerm][k] (line 17). Here, PartialMeanIVFMV is an array that stores the feature quantities of the mean feature vector of clusters that are not invariant clusters, among the feature quantities belonging to region R3. In this array, the feature quantity of feature ID "tid" included in the mean feature vector of cluster k is stored in the array element PartialMeanIVFMV[tid-ThTerm][k]. Then, the allocation unit 123 performs the same processing as that on line 19 of FIG. 11 (line 18).

[0144] If f[j]=1 (line 20), the allocation unit 123 performs the same processing as lines 2 to 19 in FIG. 11 (line 20).

[0145] In this way, the objects are assigned to the clusters in the assignment step. i belongs to the invariant cluster, the similarity calculation with the mean feature vector of the invariant cluster can be skipped. This makes it possible to further reduce the number of multiplications compared to the first embodiment, and further reduce the time and amount of calculation.

[0146] <Experimental Results of Example 2> The following describes the results of an experiment conducted to evaluate the clustering device 10 in Example 2. The settings for this experiment were the same as those for Example 1.

[0147] The number of multiplications in similarity calculation per iteration is shown in Fig. 18. In Fig. 18, Proposed is the clustering device 10 in Example 1, and Advanced is the clustering device 10 in Example 2. As shown in Fig. 18, it can be seen that the clustering device 10 in Example 2 is able to further reduce the number of multiplications compared to Example 1.

[0148] Next, the average elapsed time per iteration is shown in Fig. 19. As shown in Fig. 19, it can be seen that the clustering device 10 in the second embodiment can further reduce the average elapsed time compared to the first embodiment.

[0149] As described above, according to the clustering device 10 of the second embodiment, it is possible to further reduce the number of multiplications in the similarity calculation in the k-means method, and as a result, it is possible to further reduce the average elapsed time per iteration.

[0150] The present invention is not limited to the above-described specifically disclosed embodiments, and various modifications, changes, and combinations with known technologies are possible without departing from the scope of the claims.

[0151] [References] Reference 1: JP 2019-204436 A [Explanation of symbols]

[0152] 10 Clustering Device 101 Input Device 102 Display device 103 External I / F 103a Recording media 104 Communication I / F 105 RAM 106 ROM 107 Auxiliary storage 108 processors 109 Bus 110 Pretreatment section 120 Clustering Processing Unit 121 Input section 122 Initial setting section 123 Allocation Section 124 Update Department 125 End Determination Section 126 Output section 130 Storage section

Claims

1. 1. A clustering device that clusters an object feature vector, which is composed of one or more features representing an object and one or more feature amounts that are values ​​of each of the one or more features, into a plurality of clusters, a storage control unit configured to store, in the storage unit, data composed of array elements storing, for each feature identifier identifying the feature, a pair of a cluster identifier identifying the cluster and a first non-zero feature value related to the feature identifier of a mean feature vector representing an average of object feature vectors belonging to the cluster; a similarity calculation unit configured to calculate a similarity between an object feature vector of a sparse representation, in which the object feature vector is expressed as a set of second non-zero feature amounts of the object feature vector and the feature identifiers of the second non-zero feature amounts, based on the data, and the mean feature vector of each cluster; an assigning unit configured to assign the object feature vector to one of the plurality of clusters based on the similarity; and The data is a first array element representing the array element whose feature identifier is less than a first threshold, a second array element representing the array element whose feature identifier is equal to or greater than the first threshold and whose first non-zero feature value is equal to or greater than a second threshold, and a third array element representing the array element whose feature identifier is equal to or greater than the first threshold and whose first non-zero feature value is less than the second threshold, each of which is configured as an inverted file; and

2. The similarity calculation unit if the feature identifier of a second non-zero feature included in the object feature vector of the sparse representation is less than the first threshold, calculating the similarity by calculating an inner product of the second non-zero feature and a first non-zero feature stored in an array element that corresponds to the feature identifier among array elements that make up the data; when the feature identifier of a second non-zero feature included in the sparsely represented object feature vector is equal to or greater than the first threshold and a first non-zero feature that is a target for calculating a similarity with the second non-zero feature is equal to or greater than the second threshold, calculating an inner product of the second non-zero feature and the first non-zero feature to calculate the similarity and also calculate a partial norm of the sparsely represented object feature vector; 2. The clustering device according to claim 1, wherein, when the feature identifier of a second non-zero feature included in the object feature vector of the sparse representation is equal to or greater than the first threshold and a first non-zero feature that is a target of calculation of similarity with the second non-zero feature is less than the second threshold, and when the similarity between the object feature vector of the sparse representation and a mean feature vector of a cluster to which the object feature vector currently belongs is equal to or less than an upper similarity upper limit value calculated from the norm, the clustering device is configured to calculate the similarity by calculating an inner product of the second non-zero feature and the first non-zero feature.

3. The data is In each of the first array elements and the second array elements, there are distinguishably present an array element for storing a set including a first non-zero element of a mean feature vector corresponding to a changed cluster indicating a cluster in which an object feature vector belonging to the cluster has changed before and after the allocation by the allocation unit, and an array element for storing a set including a first non-zero element of a mean feature vector corresponding to an unchanged cluster indicating a cluster in which an object feature vector belonging to the cluster has remained unchanged before and after the allocation by the allocation unit, The similarity calculation unit 3. The clustering device according to claim 1, wherein, when a cluster to which the sparsely represented object feature vector belongs is an unchanging cluster, the clustering device is configured to calculate the similarity by calculating an inner product of a second non-zero feature included in the sparsely represented object feature vector and a first non-zero element of a mean feature vector corresponding to the changed cluster.

4. 2. The clustering apparatus according to claim 1, further comprising an optimization unit configured to optimize the second threshold based on a result of the similarity calculation by the similarity calculation unit and a result of the allocation by the allocation unit in a previous iteration.

5. A clustering device that clusters an object feature vector, which is composed of one or more features representing an object and one or more feature amounts that are values ​​of each of the one or more features, into a plurality of clusters, a storage control procedure for storing, in a storage unit, data configured with array elements storing, for each feature identifier identifying the feature, a pair of a cluster identifier identifying the cluster and a first non-zero feature value related to the feature identifier of a mean feature vector representing an average of object feature vectors belonging to the cluster; a similarity calculation step of calculating a similarity between a sparsely represented object feature vector, in which the object feature vector is expressed as a set of second non-zero feature amounts of the object feature vector and the feature identifiers of the second non-zero feature amounts, based on the data, and the mean feature vector of each cluster; an assignment procedure for assigning the object feature vector to one of the plurality of clusters based on the similarity; Run The data is a first array element representing the array element whose feature identifier is less than a first threshold, a second array element representing the array element whose feature identifier is equal to or greater than the first threshold and whose first non-zero feature value is equal to or greater than a second threshold, and a third array element representing the array element whose feature identifier is equal to or greater than the first threshold and whose first non-zero feature value is less than the second threshold, each of which is configured as an inverted file;

6. A program that causes a computer to function as the clustering device according to claim 1.

Images