Control method for information processor, information processor, and program

Abstract

To realize non-teacher-type data classification in a more preferable aspect.SOLUTION: A calculation part 12 generates a similarity matrix whose elements are similarities between each of feature quantities extracted from each of a series of data to be classified. A correction part 13 corrects the similarity matrix by replacing at least some of the elements of the similarity matrix with the similarity to which conversion has been applied according to a degree of similarity corresponding to the elements. A classification part 14 classifies the series of data into one or more groups based on the similarity matrix after correction by the correction part 13.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a method for classifying data. [Background technology]

[0002] With the development of AI-based recognition technology, systems with performance exceeding that of humans in various fields, such as voice recognition, object identification, face recognition, and similar image search, are being put to practical use. One of the technologies that supports these is machine learning technology known as deep learning. In order to improve the performance of deep learning (in other words, the performance of the constructed AI), a large amount of training data with correct labels is generally required. For example, in face recognition technology, thousands to tens of millions of image data are used to train the AI. When training an AI used for face recognition by deep learning, a large number of image pairs (image and label pairs) of the same person and image pairs of other people are given, and training is performed to distinguish whether the target person is the same person or a different person. In this learning method, it is important that the same label is correctly assigned to the same person as the teacher value (internal connection), and different labels are assigned to different people (external separation). In this disclosure, internal connection and external separation are also collectively referred to as the "quality" of the training data. Since large-scale image data is often not assigned teacher values, and it is costly to manually assign labels under such circumstances, a technology is applied that automatically assigns labels to the target image data. Non-Patent Document 1 discloses a technology that automatically assigns labels to the target data. In these methods, the image is converted into features, a similarity matrix consisting of the similarity between pairs of features is created, and similarity-based clustering is performed to determine the label. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Zheng Zhu et al., WebFace260M:A Benchmark Unveiling the Power of Million-Scale Deep Face Recognition, arXiv:2103.04098v1, 2021 [Non-Patent Document 2] Hubert, Min and max hierarchical clustering using asymmetric similarity measures, Psychometrika, 1973 Summary of the Invention [Problem to be solved by the invention]

[0004] The learning datasets used for training AI for facial recognition tend to have a relatively large number of IDs, with over tens of millions, but the number of data per ID is relatively small, at just a few. In addition, the set of data may include data of similar people, such as siblings, or data of poor image quality that is difficult to identify the person. When conventional unsupervised clustering methods are applied to such datasets, poor quality results (e.g., low recognition accuracy) tend to result. Even if a method that can obtain good quality results is applied, there is often a trade-off between quality and quantity, such as a small number of clusters.

[0005] In view of the above problems, an object of the present invention is to realize unsupervised data classification in a more suitable manner. [Means for solving the problem]

[0006] The control method for an information processing device according to the present invention is characterized in that it includes a generation step of generating a similarity matrix whose elements are the similarities between each of the features extracted from each of a series of data to be classified; a correction step of correcting the similarity matrix by replacing at least a portion of the elements of the similarity matrix with a similarity that has been transformed according to the level of the similarity corresponding to the element; and a classification step of classifying the series of data into one or more groups based on the similarity matrix corrected in the correction step. Effect of the Invention

[0007] According to the present invention, it is possible to realize unsupervised data classification in a more suitable manner. [Brief description of the drawings]

[0008] [Figure 1] FIG. 2 is a diagram illustrating an example of a functional configuration of an information processing device. [Diagram 2] FIG. 13 is a diagram illustrating an example of a similarity matrix. [Diagram 3] FIG. 13 is a conceptual diagram showing an overview of correction of a similarity matrix. [Figure 4] 11A and 11B are diagrams illustrating an example of a process related to correction of a similarity matrix. [Diagram 5] FIG. 2 is a diagram illustrating an example of processing by an information processing device. [Figure 6] FIG. 1 is a diagram for explaining an overview of clustering. [Figure 7] FIG. 2 is a diagram illustrating an example of processing by an information processing device. [Figure 8] FIG. 13 is a conceptual diagram for explaining correction of a similarity matrix. [Figure 9] FIG. 2 is a diagram illustrating an example of processing by an information processing device. [Figure 10] FIG. 2 is a diagram illustrating an example of a functional configuration of an information processing device. [Figure 11] FIG. 11 is a diagram showing an example of a clustering result. [Figure 12] FIG. 2 is a diagram illustrating an example of a functional configuration of an information processing device. [Figure 13] FIG. 11 is a diagram showing an example of a clustering result. [Figure 14] FIG. 2 is a diagram illustrating an example of a functional configuration of an information processing device. [Figure 15] FIG. 2 is a diagram illustrating an example of processing by an information processing device. [Figure 16] FIG. 13 is a diagram showing an example of a method for calculating a fluctuation similarity. [Figure 17] FIG. 13 is a diagram showing an example of a method for calculating a fluctuation similarity. [Figure 18] FIG. 11 is a diagram for explaining an example of a method for calculating a fluctuation similarity. [Figure 19] FIG. 2 is a diagram illustrating an example of a hardware configuration of an information processing device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In this specification and the drawings, components having substantially the same functional configurations are denoted by the same reference numerals, and redundant description will be omitted. Although the present invention will be described mainly with reference to the creation of a learning dataset for a face recognition system as an example, the present invention is not limited to this and may be used for a handwritten character recognition system or any other system. The purpose is also not limited to the creation of a learning dataset and may be, for example, to classify images by person for automatic album editing.

[0010] <Hardware configuration> An example of a hardware configuration of an information processing device according to this embodiment will be described with reference to Fig. 19. An information processing device 900 shown in Fig. 19 can be applied as an information processing device described later with reference to Fig. 1, for example.

[0011] The information processing device 900 has a CPU (Central Processing Unit) 904, a RAM (Random Access Memory) 905, and a ROM (Read Only Memory) 906. The information processing device 900 also has a HDD (Hard Disk Drive) 907, an input unit 901, a display unit 902, and a network I / F unit 903. The input unit 901, the display unit 902, the network I / F unit 903, the CPU 904, the RAM 905, the ROM 906, and the HDD 907 are each connected via a data bus 908 so as to be able to transmit and receive information to and from each other.

[0012] The CPU 904 performs various controls by reading out a control computer program stored in the ROM 906, expanding the control computer program in the RAM 905, and executing the control computer program. The RAM 905 is used as a work area and a temporary storage area for expanding the programs executed by the CPU 904. The HDD 907 is a storage area for storing image data, setting parameters, various programs, etc. Instead of the HDD 907 or together with the HDD 907, other auxiliary storage devices such as non-volatile memories typified by SSDs (Solid State Drives) may be used.

[0013] The network I / F unit 903 is a network I / F for connecting the information processing device 900 to a network such as the Internet, and transmits and receives various information to and from other devices via the network. Image data and the like received from an external device via the network I / F unit 903 is transmitted and received between the CPU 904, the RAM 905, and the HDD 907 via a data bus 908.

[0014] Image processing of image data is realized by the CPU 904 executing an image processing program stored in the ROM 906 or HDD 907. In addition, the HDD 907 is capable of inputting data from an external device via the network I / F unit 903.

[0015] The input unit 901 is realized by, for example, input devices such as a keyboard, a mouse, a touch panel, etc. The display unit 902 is realized by, for example, a display device such as a liquid crystal display.

[0016] <First embodiment> As a first embodiment of the present disclosure, an example of a mechanism for classifying a series of data into one or more groups, called non-supervised clustering, will be described. In this embodiment, for convenience, various explanations will be given focusing on the case where learning data used for face recognition is classified. Images of a person's face used for face recognition (hereinafter referred to as face images) do not necessarily have a higher similarity between images of the person than between images of other people due to various conditions such as the angle of the face direction, similar others such as siblings, changes due to aging, and blurring of the image. Therefore, in this embodiment, an example of a mechanism for further improving the quality of the classification result while ensuring a certain number of clusters (the number of sets after classification, the number of groupings) even in a situation where the similarity between such related data is incomplete.

[0017] An example of a functional configuration of an information processing device according to this embodiment will be described with reference to Fig. 1. The information processing device according to this embodiment includes an input unit 11, a calculation unit 12, a correction unit 13, a classification unit 14, and an output unit 15. The input unit 11 receives an input of a series of data to be classified, and acquires feature quantities 111 (that is, information quantitatively expressing the features of the data) of each of the series of data. The calculation unit 12 calculates the similarity of each of a series of pairs for feature quantities 111 extracted from each of a series of data to be classified acquired by the input unit 11. Then, the calculation unit 12 generates a similarity matrix 121 including the calculated series of similarities as elements. The similarity matrix 121 will be described in detail later with reference to FIG. 2. The correction unit 13 generates a corrected similarity matrix 131 by correcting the similarity matrix 121 generated by the calculation unit 12. When generating the corrected similarity matrix 131, the correction unit 13 outputs a table of elements that have been corrected (hereinafter, also referred to as a correction element table 132). The classification unit 14 performs clustering (that is, classification into one or more groups) on a series of data to be classified based on the corrected similarity matrix 131 and the correction element table 132, and outputs the clustering result 141. The output unit 15 outputs the clustering result 141 to a predetermined output destination (for example, for subsequent processing).

[0018] Hereinafter, each of the components of the information processing device according to this embodiment shown in FIG. 1 will be described in more detail. The input unit 11 acquires (extracts) features from the input data (i.e., data to be classified). As a specific example, the extraction of features from each data when creating a data set used in a face recognition system will be described below. By applying feature extraction to each of the many face images included in the data set, feature amounts corresponding to each face image are obtained. The feature amounts are generally represented by vectors of about 100 to 1000 dimensions. The feature extractor is designed to extract similar features, that is, features showing a higher similarity value, from face images of the same person, for example. The similarity will be described together with details of the calculation unit 12. Well-known such feature extractors include FaceNet and ArcFace, but other ones may also be used. For example, if 1 million face images, each with an average of 10 images per person, are given, 10 million feature amounts will be obtained. 1 ~F N It indicates a set of N features (N is an integer equal to or greater than 1) of the above. The input unit 11 acquires the N feature quantities as feature quantities 111 .

[0019] The calculation unit 12 calculates the similarity of each pair of a series of feature amounts (i.e., feature amounts F 1 ~F N ) included in the feature amount 111, and generates a similarity matrix 121 having the similarities as elements. Here, with reference to FIG. 2, an example of the similarity matrix 121 will be described in more detail. The similarity matrix 121 is an N×N matrix, and the element S ij in the i-th row and j-th column (each of i and j is an integer of 1 or more) is set as the similarity between the feature amount F i and the feature amount F j . Generally, similarity or distance is used as an index for whether two feature amounts are similar. The similarity indicates a higher value as the comparison targets are more similar. Also, the distance indicates a lower value as the comparison targets are more similar. Regarding what is optimally used as an index for whether feature amounts are similar to each other, it can be determined, for example, by the applied feature extractor. As a specific example, in the case of ArcFace, an index called cosine similarity is used. Note that hereinafter, for convenience, the description will be made assuming that the similarity takes a value in the range of 0 to 1, but this does not necessarily limit the values that can be taken as the similarity. As a specific example, in the case of cosine similarity, it can take a value in the range of -1 to 1, and if negative values are set to 0, the result will be a value in the range of 0 to 1. Also, regarding the conversion from the distance d to the similarity s, when m is a constant of 0 < m < 1 and n is a constant of n > 0, for example, it can be realized by the relational expression shown as (Equation 1) below.

[0020]

Equation

[0021] Note that in this embodiment, various descriptions will be made assuming that the similarity matrix satisfies the relational expressions shown as (Equation 2) and (Equation 3) below, but this does not necessarily limit the characteristics of the similarity matrix according to this embodiment.

[0022]

Equation

[0023] For example, in the AP method, which is an example of a clustering method, the values ​​of the diagonal elements have a special meaning, so that the condition shown in (Equation 2) is not satisfied. Furthermore, Non-Patent Document 2 discloses a clustering method using an asymmetric similarity matrix that does not satisfy the condition shown in (Equation 3). The technology according to the present disclosure can be applied to any of the methods exemplified above.

[0024] The correction unit 13 generates a corrected similarity matrix 131 by correcting the similarity matrix 121. In a normal clustering method, clustering is performed using the similarity matrix 121 as it is, whereas in this embodiment, a correction is made to the similarity matrix 121 as described below with reference to Fig. 3. Fig. 3 is a conceptual diagram showing an overview of the correction of the similarity matrix. In many cases, the feature amount is expressed as a high-dimensional vector, but in this case, for simplicity, each feature amount (feature amounts 31 to 39, 310, and 311) is plotted in two dimensions. Note that in Fig. 3(b) to Fig. 3(d), numbers are not assigned to the markers indicating the feature amounts to avoid complicating the drawings, but the symbols assigned to feature amounts at similar positions in the example shown in Fig. 3(a) are assumed to correspond to similar feature amounts.

[0025] FIG. 3(a) shows a schematic diagram of a state in which 11 feature amounts (feature amounts 31 to 39, 310, and 311) are given. In the examples shown in FIG. 3(a) to FIG. 3(d), the closer the positions of two feature amounts are, the more similar the two feature amounts are, i.e., the higher the similarity is, and the farther the positions of the two feature amounts are, the lower the similarity is. In the following explanation, for convenience, a close distance between feature amounts is also referred to as a high similarity, and a far distance between feature amounts is also referred to as a low similarity. In the example shown in Fig. 3, clustering is performed on the above-mentioned 11 feature amounts. Here, an example of a case where each feature amount is simply compared with surrounding feature amounts, and feature amounts with high similarity are grouped together into a cluster will be described.

[0026] Observing Fig. 3(a), it seems intuitively better to perform clustering as shown in Fig. 3(d). Specifically, in the example shown in Fig. 3(d), one cluster 301 is formed by features 31 to 35, one cluster 302 is formed by features 36 to 39 and 310, and feature 311 is an isolated feature that does not belong to any cluster. On the other hand, if clustering is performed using a method that determines whether to merge multiple features based on a threshold by comparing the similarity between features, it is possible to imagine situations in which results such as those shown in Figures 3(b) and 3(c) will result. In particular, when it is determined that some features (e.g., feature 34) belong to multiple clusters as in Figure 3(c), a series of features (i.e., 11 features) that belong to the multiple clusters may end up being merged to belong to a single cluster.

[0027] In the example shown in FIG. 3(b), features 36 to 39 and 310 form one cluster 302, while features 31 to 35 do not form any clusters, and the number (quantity) of clusters is smaller than in the example shown in FIG. 3(d). In the example shown in FIG. 3(c), one cluster 301 is formed by the features 31 to 35, while one cluster 303 is formed to include the features 34 and 311 in addition to the features 36 to 39 and 310. Focusing on the cluster 303, the features 34 and 311 included in the cluster 303 are separated from the other features 36 to 39 and 310, and are likely to correspond to a person different from the person corresponding to the other features 36 to 39 and 310. That is, in the example shown in FIG. 3(c), it cannot be said that high-quality clustering has been performed. Furthermore, when the clusters 301 and 303 are merged in FIG. 3(c) to form one cluster, the quality and quantity are significantly reduced.

[0028] In view of the above, in this embodiment, an example of a clustering method that can obtain the result as shown in Fig. 3(d) will be described. That is, by applying the clustering method according to this embodiment, it is possible to obtain a larger number (amount) of clusters than in the example shown in Fig. 3(b), and it is possible to obtain clusters of higher quality (higher purity of similarity) than in the example shown in Fig. 3(c). Specifically, the correction unit 13 corrects the similarity matrix 121 so as to obtain a clustering result such as that shown in FIG. 3(d).

[0029] Here, the content of the process related to the correction of the similarity matrix 121 by the correction unit 13 will be described with reference to FIG. 4. FIG. 4(a) and FIG. 4(b) show an example of the case where the correction unit 13 applies correction to the features 41 to 44. Specifically, FIG. 4(a) shows a state before correction, and FIG. 4(b) shows a state after correction. In FIG. 4(a) and FIG. 4(b), the arrows show the content of the correction, that is, the feature that is the target and the correction that is applied. Note that the correction by the correction unit 13 is performed on the similarity, and the feature does not actually move. That is, the arrows shown in FIG. 4(a) and FIG. 4(b) show that the correction by the correction unit 13 converts the similarity of the target feature to the similarity of the feature that has moved to the tip of the arrow. Note that, hereinafter, in order to make the explanation easier to understand, the correction of the similarity may be described as "the feature moves" for convenience.

[0030] In the example shown in Figures 4(a) and 4(b), the feature amount 41 is taken as the reference, and the similarities between the feature amount 41 and each of the other features 42 to 44 are as shown below. Feature 43 has a higher similarity to feature 41 than features 42 and 44. Feature 44 has a lower similarity to feature 41 than features 42 and 43. Feature 42 has an intermediate similarity between features 41 and 43 and features 41 and 44. In response to this, the correction unit 13 corrects the similarity as follows: Note that the value in parentheses indicates the feature amount equivalent to the similarity after correction. Increase the similarity between feature 41 and feature 43 (feature 431) Decrease the similarity between feature 41 and feature 44 (feature 441) · The similarity between feature 41 and feature 42 is not changed (feature 42) That is, when the similarity is high, the correction unit 13 corrects the similarity to be higher, and when the similarity is low, the correction unit 13 corrects the similarity to be lower.

[0031] The processing of the correction unit 13 will be described in more detail with reference to FIGS. In S51, the correction unit 13 obtains the similarity matrix 121 described with reference to Fig. 1 and Fig. 2. Then, the correction unit 13 repeatedly executes the processes of S52 to S56 for each row of the similarity matrix 121. In the repetitive processing, the correction unit 13 sets the target row as a similarity row vector 511, and executes the processing described below with respect to this row vector.

[0032] As can be seen from FIG. 2, in the i-th repetition, the similarity row vector 511 is the i-th row of the similarity matrix 121. This is because the feature F i The value of the jth element corresponds to the similarity between feature F i and feature F j The similarity is In clustering, elements (feature amounts) with a relatively high degree of similarity tend to be placed in the same cluster, and elements with clearly low similarity are unlikely to be placed in the same cluster. Therefore, the correction unit 13 excludes elements with obviously low similarity from the correction target. In the following, a threshold value (correction element selection threshold value) applied to the judgment of whether or not to exclude an element from the correction target is set to th 1 Let us assume that.

[0033] When a histogram of the distribution of all elements of similarity row vector 511 is created, most of the elements are similar to the facial images of other people, so they are concentrated near 0, forming a very high peak. In consideration of such characteristics, correction unit 13 targets elements other than the elements concentrated near 0 as the targets of correction. This means that most elements are excluded from the targets of correction, and as a result, the number of elements to be corrected is significantly reduced, which is expected to have the effect of significantly reducing the amount of calculations required for correction.

[0034] In S52, the correction unit 13 selects a correction element selection threshold (th 1 ) to determine Correction element selection threshold (th 1 Various methods can be applied to determine the correction factor selection threshold (th1 Here is an example of how to determine the above. (Method 1) Set a specific value as the correction element selection threshold (th 1 ) (Method 2) A histogram of the element values ​​of the similarity row vector 511 is created, and assuming a bimodal distribution, the value at the bottom of the bimodal distribution is set as the correction element selection threshold (th 1 ) In particular, when it is known in advance to what extent a set of elements near 0 is distributed, the effect of simplifying the processing can be expected by applying, for example, method 1. Method 1 can be applied in many cases, including the creation of large-scale data sets for face recognition systems. On the other hand, when the distribution cannot be predicted in advance, such as when creating a small data set, as shown in method 2, a histogram can be actually created with the element values ​​of the similarity row vector 511, and a value that is the valley of the bimodal distribution can be taken assuming a bimodal distribution. This is because, as already explained, most elements (similarities) are concentrated near 0, forming a very large peak. In method 2, this large peak is separated assuming a bimodal distribution. A specific example of method 2 is discriminant analysis (Otsu's binarization). This discriminant analysis is well known, so a detailed explanation will be omitted. In addition, the correction factor selection threshold (th 1 The determination of the correction factor selection threshold (th 1 Various techniques can be applied in determining .

[0035] In S53, the correction unit 13 checks whether each element of the similarity row vector 511 is greater than or equal to the correction element selection threshold (th 1 ), the correction unit 13 selects the element as a correction element. Then, the correction unit 13 outputs a correction element vector 512 obtained by vectorizing the value of the correction element and a correction element list 513 including information for identifying the correction element. The correction element list 513 can be realized, for example, as a vector of indices indicating which element of the similarity row vector 511 the selected correction element is. In this case, the length of the correction element vector 512 and the length of the correction element list 513 (vector) are equal. In addition, the i-th element of the similarity row vector 511 is denoted by s(i), and the j-th element of the correction element vector 512 is denoted by s(i). mod (j), the k-th element of the correction element list 513 is s idx If (k), then the relational expression shown below as (Equation 4) is satisfied.

[0036]

number

[0037] In the following, the correction element list 513 will be described as being realized as an index vector as described above, but as long as it is possible to identify the correction element, the type of information applied as the correction element list 513 is not particularly limited.

[0038] In S54, the correction unit 13 calculates the correction reference value (th 2 For example, in the case of the example shown in FIG. 4(a), the correction reference value (th 2 ) corresponds to the similarity between the feature amount 41 and the feature amount 42.

[0039] Here, referring to FIG. 6, the basic concept of clustering in this embodiment and the correction reference value (th 2 The following explains how this affects First, the basic concept of clustering will be described with reference to FIG. 6(a). In FIG. 6, small round markers indicate individual feature amounts. In the example shown in FIG. 6, each feature amount is plotted two-dimensionally for ease of explanation, as in the example shown in FIG. 3. The feature amount 61 corresponding to the marker located at the center corresponds to the feature amount of interest, and when the similarity row vector 511 is the row vector of the i-th row of the similarity matrix 121, the feature amount F iIn the following, for the sake of simplicity, the feature of interest is represented as feature F i Then, the feature value F i and other features F j The similarity to the feature F j This is also called the "similarity of

[0040] The circular boundary 601 is a boundary where the similarity to the feature amount 61 is a correction element selection threshold (th 1 ) is the iso-similarity line. Outside this boundary 601, the similarity is the correction factor selection threshold (th 1 ) is an area smaller than the boundary 601. Since the feature amount 62 is located in an area outside the boundary 601, it is excluded from the selection target as a correction element by the process of S53 in Fig. 5. In other words, the feature amounts located in an area inside the boundary 601 are the targets for similarity correction.

[0041] The circular boundary 602 is an isosimilarity line whose similarity to the feature 61 is the cluster boundary similarity, and the features inside this boundary 602 are judged to be (candidates for) the same cluster as the feature 61 of interest. For example, feature amount 63 is located in an area inside boundary 602, and is therefore determined to belong to the same cluster as feature amount 61. Meanwhile, feature amount 64 is located in an area outside boundary 602, and is therefore not determined to belong to the same cluster as feature amount 61.

[0042] FIG. 6(b) is a diagram showing an example of how correction is performed when there are many feature quantities that have a relatively high similarity to the feature quantity 61. In FIG. 6(b), a circular boundary 603 shown by a dashed line is added. This boundary 603 is a boundary between the feature quantity 61 and the correction reference value (th 2 ) is the iso-similarity line. 6(b), the feature amounts located outside the boundary 603 are corrected so as to move in a more outward direction, that is, in a direction in which the similarity decreases, while the feature amounts located inside the boundary 603 are corrected so as to move in a more inward direction, that is, in a direction in which the similarity increases. Here, attention is paid to feature quantities 631 and 632. The similarity between feature quantities 631 and 632 is slightly greater than the cluster boundary similarity. Also, feature quantities 631 and 632 were located inside boundary 602 before correction, but have moved outside boundary 602 after correction. In this case, feature quantities 631 and 632 were determined to belong to the same cluster as feature quantity 61 before correction, but are no longer determined to belong to the same cluster as feature quantity 61 after correction.

[0043] On the other hand, Fig. 6(c) is a diagram showing an example of how correction is performed when there are few features that have a relatively high similarity to the feature 61. In Fig. 6(c) as well, a circular boundary 603 indicated by a dashed line is added, as in Fig. 6(b). Here, attention is paid to the feature quantities 641 to 643. The similarity of each of the feature quantities 641 to 643 is slightly smaller than the cluster boundary similarity. Also, each of the feature quantities 641 to 643 was located outside the boundary 602 before the correction, but moved inside the boundary 602 after the correction. In this case, each of the feature quantities 641 to 643 was not determined to belong to the same cluster as the feature quantity 61 before the correction, but is determined to belong to the same cluster as the feature quantity 61 after the correction.

[0044] The above effects can be expressed as follows. (Effect 1) When there are many features with relatively high similarity to the feature of interest (feature 61) (i.e., when features with relatively high similarity are distributed at high density as shown in FIG. 6(b)), a feature with a similarity slightly larger than the cluster boundary similarity (boundary 602) is not determined to belong to the same cluster as feature 61. As a result, there is an effect of improving the quality of the cluster. (Effect 2) When there are few features with relatively high similarity to the feature of interest (feature 61) (i.e., when features with relatively high similarity are distributed at low density as shown in FIG. 6(c)), it is determined that features with similarity slightly smaller than the cluster boundary similarity (boundary 602) belong to the same cluster as feature 61. As a result, there is an effect of increasing the number of clusters.

[0045] In order to achieve this effect, the similarity corresponding to the boundary 603, that is, the correction reference value (th 2 ) can be determined based on the number (density) of features with relatively high similarity.

[0046] Correction standard value (th 2 ) can be determined, for example, in the following manner. In any of the methods, the similarity of the target feature is determined by the correction factor selection threshold (th 1 ) is higher. (Method 3) Average similarity score (Method 4) Weighted average of the mean similarity and 0 (Method 5) Weighted average of the average similarity and 1 (Method 6) R percentile of similarity (0 <R<100) These methods determine a small value as the correction reference value (th2) when the density is high, and a large value when the density is low. The above method is merely an example, and the correction reference value (th 2 In other words, as described above, if the method is such that the correction reference value (th 2) can be applied as a method for determining

[0047] Here, reference is made again to FIG. In S55, the correction unit 13 calculates the correction reference value (th 2 ) is used to perform correction on the correction element vector 512 to calculate a corrected element vector 514. Fig. 4(c) shows an example of the correction function applied to the correction of the similarity in the process of S55. In the graph shown in Fig. 4(c), the horizontal axis indicates the similarity before correction, and the vertical axis indicates the similarity after correction. As described above, in the example shown in Fig. 4(c), the range of the similarity before and after correction is limited to 0 or more and 1 or less. th 2 indicates the correction reference value determined in the process of S54. As can be seen from FIG. 4(c), the similarity before correction is s in , the correction function is f mod() Then, the corrected similarity s out =f mod (s in ) and has the properties shown in (Equation 5) to (Equation 7) below.

[0048]

number

[0049] The relational expressions (5) to (7) indicate that the correction function is a monotonically increasing function (monotonically non-decreasing function) and is downward convex in the region where the pre-correction similarity is smaller than the correction reference value, and is upward convex in the region where it is larger than the correction reference value. Examples of such correction functions include functions shown below as (Equation 8) to (Equation 10) with p and q as constants. Note that the functions shown below as (Equation 8) to (Equation 10) are merely examples, and are not particularly limited as long as they satisfy the relational expressions shown as (Equation 5) to (Equation 7).

[0050]

number

[0051] When the correction function satisfying the relational expressions shown as (Equation 5) to (Equation 7) is used, the similarity of the feature amount 43 shown in FIG. 4(a) is calculated by the correction reference value th 2 Greater than (correction reference value th 2 4(b) ), correction is made to move to feature amount 431 (FIG. 4(b)) which has an even greater similarity. In addition, the similarity of feature 44 is the correction standard value th 2 Smaller than (correction reference value th 2 Therefore, a correction is made to move to feature amount 441, which has an even smaller similarity. On the other hand, for the similarity of feature 42, the correction standard value th 2 Therefore, the similarity remains unchanged even after the correction.

[0052] In S56, the correction unit 13 reflects the corrected element vector 514 in the similarity row vector 511 to generate a corrected similarity row vector 515. Corrected element vector 514 has a smaller number of elements than similarity row vector 511. Therefore, in order to output an N×N corrected similarity matrix, correction unit 13 generates corrected similarity row vector 515 having N elements. Corrected similarity row vector 515 can be generated by replacing elements of similarity row vector 511 identified in correction element list 513 with corresponding elements of corrected element vector 514.

[0053] Here, the process of S56 shown in FIG. 5 will be described in detail with reference to FIG. In the process of S56, correction unit 13 first creates a copy of similarity row vector 511 in S561. That is, correction unit 13 copies similarity row vector 511 shown in Fig. 5 to create corrected similarity row vector 515. At this point, corrected similarity row vector 515 is simply a copy of similarity row vector 511, but by replacing some elements in the subsequent process, it becomes the final corrected similarity row vector 515.

[0054] In S562, the correction unit 13 reads the corrected element vector 514 and the correction element list 513. As described above, the correction element list 513 indicates the index of the similarity row vector 511 (and the corrected similarity row vector 515). As a specific example, if the third element is 10, the correction element list 513 indicates that "the 10th element of the corrected similarity row vector 515 is the third element of the correction element vector 512." Therefore, the correction unit 13 repeats the process of S563 the number of times equal to the length of the correction element list 513, and updates the corrected similarity row vector 515. Specifically, the correction unit 13 extracts an element of the corrected element vector 514 from the top, and sets this element as the correction element 701. Similarly, the correction unit 13 extracts an element of the correction element list 513 from the top, and sets this element as the correction element index 702. Then, in S563, the correction unit 13 executes a process of replacing the element of the corrected similarity row vector 515 with the correction element 701 as a similarity replacement process. When the correction unit 13 completes the repeated process of S563, the corrected similarity row vector 515 is in an updated state, and therefore the corrected similarity row vector 515 is output in S564.

[0055] Fig. 8 shows an example of the relationship between similarity row vector 511, correction element list 513, corrected element vector 514, and corrected similarity row vector 515. In the example shown in Fig. 8, for convenience, the length of similarity row vector 511 is set to 25, and the lengths of corrected element vector 514 and correction element list 513 are set to 5. Corrected similarity row vector 515 is based on a copy of similarity row vector 511, so its length is 25. Because the first element of correction element list 513 is 3, the first element of corrected element vector 514 becomes the third value of corrected similarity row vector 515. Similarly, the 2nd, 3rd, 4th, and 5th values ​​of corrected element vector 514 become the 5th, 10th, 17th, and 21st values ​​of corrected similarity row vector 515. Note that the values ​​of the elements of corrected similarity row vector 515 other than these become the values ​​of similarity row vector 511 at the corresponding positions. In this way, a corrected similarity row vector 515 is obtained.

[0056] Here, reference is made again to FIG. The correction unit 13 generates a corrected similarity matrix 131 by combining the corrected similarity row vectors 515 for the number of iterations. Note that the first row of the corrected similarity matrix 131 is the corrected similarity row vector 515 generated in the first iteration, and the second row is the corrected similarity row vector 515 generated in the second iteration. Similarly, the final row of the corrected similarity matrix 131 is the corrected similarity row vector 515 generated in the Nth iteration. Moreover, the correction unit 13 creates a correction element table 132 by compiling the correction element list 513 in order to record which elements of the corrected similarity matrix are to be corrected.

[0057] Here, reference is made again to FIG. The classification unit 14 performs clustering (that is, classification into one or more groups) on a series of data to be classified based on the corrected similarity matrix 131 and the correction element table 132, and outputs the result as a clustering result 141. The method applied to clustering of a series of data to be classified by the classification unit 14 is not particularly limited as long as the corrected similarity matrix 131 is used.

[0058] Now, with reference to FIG. 9, an example of the processing of the classification unit 14 will be described, focusing on the processing related to clustering of a series of data to be classified. In S141, the classification unit 14 acquires the corrected similarity matrix 131 and the correction element table 132 which are outputs of the correction unit 13 (see FIG. 1). Then, the classification unit 14 repeatedly executes the processes of S142 to S145 for each row of the corrected similarity matrix 131. Corrected similarity row vector 921 is a vector corresponding to a row of corrected similarity matrix 131 (in other words, a group of elements corresponding to a row). Specifically, in the first iterative process, the first row of corrected similarity matrix 131 is treated as corrected similarity row vector 921 in the iterative process. In the second iterative process, the second row of corrected similarity matrix 131 is treated as corrected similarity row vector 921 in the iterative process. Similarly, in the final iterative process (Nth iterative process), the Nth row of corrected similarity matrix 131 is treated as corrected similarity row vector 921 in the iterative process. The correction element list 922 is a list of correction elements of the row corresponding to the corrected similarity row vector 921 . Among the corrected similarity row vector 921, the elements represented by the correction element list 922 are the elements that are the subject of correction by the correction unit 13. In S142, the classification unit 14 generates a correction element vector 923 that groups together these elements. The correction element list 922 and the correction element vector 923 are similar to the corrected element vector 514 and the correction element list 513 shown in FIG.

[0059] In S143, the classification unit 14 extracts elements included in the cluster candidates. Specifically, the classification unit 14 outputs the indexes of elements of the correction element vector 923 having a value equal to or greater than a given cluster similarity threshold 911 (th3) as the cluster candidate element list 924. For example, if the third element of the correction element vector 923 has a value equal to or greater than the cluster similarity threshold 911 (th3), the third value of the correction element list 922 is included in the cluster candidate element list 924. In addition, the number of times the process of S142 to S145 is repeated is also added as an index to the cluster candidate element list 924. For example, in the fifth repeated process, the fifth feature amount (F 5) is stored in the corrected similarity row vector 921. 5 ) is the cluster similarity threshold 911 (th 3 ) or more will be included in the cluster candidate element list 924. The cluster candidate element list 924 also includes the index "5" which is the number of repetitions, but this is because the fifth feature (F 5 ) and myself (F 5 ) will naturally be in the same cluster.

[0060] It is considered that the elements of the indexes included in the cluster candidate element list 924 are likely to be included in the same cluster, but there is also a possibility that the elements are outliers. In order to avoid generating a cluster of such outliers, in S144, the classification unit 14 checks whether the number of elements included in the cluster candidate list 924 is greater than or equal to a given cluster element threshold 912 (th 4 ) or more.

[0061] In S145, if there are not enough elements (i.e., if the number of elements is less than the cluster element threshold 912), the classification unit 14 discards the cluster candidate element list 924 and executes the next iterative process. On the other hand, if there are enough elements (i.e., if the number of elements is equal to or greater than the cluster element threshold 912), the classification unit 14 outputs the cluster candidate element list 924, assuming that the elements included in the cluster candidate list 924 constitute the same cluster.

[0062] In S146, the classification unit 14 consolidates the cluster candidate element lists 924 from S145 to generate a clustering result 141. This cluster candidate element list 924 indicates a subset of elements classified into the same cluster. Therefore, by consolidating a series of cluster candidate element lists 924, it is possible to obtain elements of each of a series of clusters as a clustering result 141. The output unit 15 outputs this clustering result 141 to a predetermined output destination (for example, a subsequent process not shown in FIG. 1).

[0063] As described above, as the first embodiment of the present disclosure, an example of a mechanism for classifying a series of data into one or more groups, which is called non-supervised clustering, has been described. In this embodiment, an example has been described in which the correction unit 13 and the classification unit 14 process the similarity matrix 121 and the corrected similarity matrix 131 on a row-by-row basis, but the processing may also be performed on other processing units, such as on a column-by-column basis. In addition, in the present embodiment, an example in which the correction unit 13 outputs the corrected similarity matrix 131 and the correction element table 132 has been described, but the elements of the corrected similarity matrix 131 may be set to 0 or a value outside the range of similarity (a negative value or a value exceeding 1). By applying such control, it becomes unnecessary to output the correction element table 132. In this case, the classification unit 14 can determine whether or not a value is a correction element by checking the value of the element. Also, an example has been described in which the correction unit 13 outputs the corrected similarity matrix 131 and the correction element table 132. However, instead of the corrected similarity matrix 131, a list of corrected similarity row vectors 515 shown in FIG. With the above-described mechanism, it is possible to improve the quality of data while minimizing the reduction in data volume when creating a training dataset.

[0064] <Second embodiment> A second embodiment of the present disclosure will be described below. In the first embodiment, as shown in (Equation 3), the similarity matrix 121 is a symmetric matrix, but the corrected similarity matrix 131 is not a symmetric matrix. That is, the element in the i-th row and j-th column of the corrected similarity matrix is ​​S mod If (i, j), then the relational expression shown below as (Equation 11) is not satisfied.

[0065]

number

[0066] On the other hand, in the first embodiment, the clustering method applied by the classification unit 14 uses the asymmetric corrected similarity matrix 131, but it is also possible to apply a clustering method premised on a symmetric matrix. Therefore, in the present embodiment, an example of the case where a clustering method premised on a symmetric matrix is ​​applied will be described.

[0067] An example of the functional configuration of the information processing device according to this embodiment will be described with reference to Fig. 10. In this embodiment, the configuration of the portion corresponding to the correction unit 13 shown in Fig. 1 is replaced with the configuration shown in Fig. 10. Specifically, in this embodiment, an objectification unit 1301 is provided after the correction unit 13, and the correction unit 13 and the objectification unit 1301 form a correction unit 1300. That is, in this embodiment, in the functional configuration illustrated in Fig. 1, the correction unit 13 is replaced with the correction unit 1300 illustrated in Fig. 10. The operation of the correction unit 13 is similar to that of the first embodiment. Note that, in this embodiment, for convenience, the output of the correction unit 13 is also referred to as a correction element table 1320 and a pre-symmetrization corrected similarity matrix 1310.

[0068] The objectification unit 1301 receives as input a pre-symmetrization corrected similarity matrix 1310 which is an asymmetric matrix, performs symmetrization on the pre-symmetrization corrected similarity matrix 1310 , and outputs the result as a post-symmetrization corrected similarity matrix 1311 . Various methods can be applied to make an asymmetric matrix into a symmetric matrix. The values ​​of the i-th row and j-th column of the pre-symmetrization corrected similarity matrix 1310 and the post-symmetrization corrected similarity matrix 1311 are respectively expressed as S before (i,j) and S after If (i, j), then it is possible to apply, for example, the method illustrated below.

[0069]

number

[0070] Method 7 is a method in which the upper triangular matrix of the pre-symmetrization corrected similarity matrix 1310 is replaced with the upper triangular matrix of the transposed matrix of the pre-symmetrization corrected similarity matrix 1310 to obtain the post-symmetrization corrected similarity matrix 1311. Method 8 is a method in which the lower triangular matrix of the pre-symmetrization corrected similarity matrix 1310 is replaced with a lower triangular matrix of the transposed matrix of the pre-symmetrization corrected similarity matrix 1310 to obtain the post-symmetrization corrected similarity matrix 1311. Method 9 is a method in which the average of each element of the pre-symmetrization corrected similarity matrix 1310 and its transpose is taken as the post-symmetrization corrected similarity matrix 1311. Method 10 is a method in which the maximum value for each element of the pre-symmetrization corrected similarity matrix 1310 and its transpose matrix is ​​set as the post-symmetrization corrected similarity matrix 1311. Method 11 is a method in which the minimum value for each element of the pre-symmetrization corrected similarity matrix 1310 and its transpose is set as the post-symmetrization corrected similarity matrix 1311.

[0071] The correction section 1300 outputs the symmetrized corrected similarity matrix 1311 as the corrected similarity matrix 131 and the correction element table 1320 as the correction element table 132 to the subsequent classifier 14 (see FIG. 1).

[0072] As described above, by replacing the correction unit 13 shown in FIG. 1 with the correction unit 1300 including the objectification unit 1301 shown in FIG. 10, it becomes possible to apply a clustering method based on a symmetric matrix in the classification unit 14. In addition, for the corrected similarity matrix 131, the value of row i, column j is S mod If (i,j), then S mod (i,j)=S mod (j, i), it is possible to reduce the memory usage by about half. Therefore, by applying the technology according to this embodiment, it is expected that the actual memory usage can be significantly reduced.

[0073] <Third embodiment> A third embodiment of the present disclosure will be described below. In various classification methods (clustering methods), the classification results of data may include poor-quality clusters in which internal connections are not achieved. As a specific example, in a situation where a person's face images are classified, the classification results of a series of face images may include poor-quality clusters containing other people's face images. Such poor-quality clusters often have a larger variance in the similarity between features compared to other clusters. An example of such a situation is shown in FIG. 11. In FIG. 11, as in the example shown in FIG. 3, each feature is plotted in two dimensions for ease of explanation. In the example shown in FIG. 11, the features included in clusters 204, 2041, and 2042 are hatched to distinguish them from other features.

[0074] In Fig. 11(a), four clusters, clusters 201 to 204, are formed, and cluster 204 is distributed over a wider range than the other three clusters. Clusters distributed over such a wide range tend to be poor quality clusters that do not achieve internal connections. Therefore, the information processing device according to this embodiment focuses on cluster 204 and performs clustering again on the eight feature amounts included in cluster 204 by the methods described as the first and second embodiments. 11(b) shows an example of the result of the second clustering. In the example shown in FIG. 11(b), the cluster 204 is divided into two clusters, cluster 2041 and cluster 2042.

[0075] Here, an example of the functional configuration of the information processing device according to this embodiment will be described with reference to Fig. 12. The information processing device according to this embodiment includes a preliminary input unit 101, a preliminary classification unit 102, a first selection unit 103, a first reclassification unit 104, a first update unit 105, and a preliminary output unit 106. Note that the clustering result 1021 is accessed from a plurality of components, and therefore the direction of data flow is indicated by a dashed line.

[0076] The auxiliary input unit 101 acquires auxiliary features 1011 to be classified. The auxiliary input unit 101 corresponds to the input unit 11 in the example shown in FIG. The preliminary classification unit 102 performs clustering on the preliminary features 1011 acquired by the preliminary input unit 101, and outputs the result as a clustering result 1021. Note that there is no particular limitation on the method that the preliminary classification unit 102 applies to the clustering of the preliminary features 1011. As a specific example, the method described as the first embodiment or the second embodiment may be applied, or an existing clustering method may be applied. Also, the clustering result 1021 corresponds to an example of a classification result of a plurality of data items performed in advance.

[0077] The first selection unit 103 selects clusters from the clustering result 1021, in which the variance of the similarity between features is equal to or greater than a predetermined value. The similarity between features is used in many clustering methods, and if the preliminary classification unit 102 has it as an additional output, it can be used as is. As for the method of calculating the predetermined value used for evaluating the variance of the similarity related to the selection of the clusters, for example, a method may be applied in which the variance of the similarity of all clusters is calculated, a histogram is created, and the value is set to be the valley of the bimodal distribution on the assumption that the distribution is bimodal. This value calculated for a known data set may also be used as is. When the preliminary classification unit 102 does not use the similarity or when the similarity is not an additional output, the first selection unit 103 may calculate the similarity between the feature amounts. As a method for calculating the similarity between the feature amounts, for example, the method described in the first embodiment may be applied.

[0078] The first selection unit 103 regards the selected cluster group as a set of features constituting each cluster, compiles them as a list, and outputs the list to a first partial feature list 1031. As a specific example, if 10 clusters are selected, a set of 10 features will be listed. Furthermore, the first selection unit 103 removes each feature included in the first partial feature list 1031 from the clustering result 1021. Note that, as a result of removing the features, the first selection unit 103 also deletes the cluster itself if the number of features becomes zero.

[0079] Next, the first reclassification unit 104 and the first update unit 105 execute processing on each feature set in the first partial feature list 1031 . The first re-sorting unit 104 clusters each feature set in the first partial feature list 1031. For example, the techniques described as the first and second embodiments may be applied to the clustering of each feature set in the first partial feature list 1031 by the first re-sorting unit 104. Furthermore, the techniques are not limited to the techniques described as the first and second embodiments, and the techniques according to the present embodiment may be applied recursively, and various forms of classification methods according to the present disclosure may be applied. The first reclassification unit 104 outputs the result of clustering each feature set in the first partial feature list 1031 as a first partial clustering result 1041. The first updating unit 105 reflects the first partial clustering result 1041 in the clustering result 1021. As a specific example, the first updating unit 105 adds the first partial clustering result 1041 to the clustering result 1021. In this way, the clustering result 1021 is updated. The above-described processing by the first reclassification unit 104 and the first update unit 105 is repeatedly applied to a series of feature sets included in the first partial feature list 1031. By executing the above-described processing, the clustering result 1021 contains the final clustering result.

[0080] The preliminary output unit 106 outputs the clustering result 1021 reflecting the above-mentioned series of processes to a predetermined output destination (for example, a subsequent process not shown).

[0081] With the above-described mechanism, it becomes possible to reclassify poor-quality clusters based on any clustering result. Narrowing the targets of clustering in this way makes it possible to reduce low-quality clusters and expand high-quality clusters (improving quality and ensuring quantity) without affecting clusters whose quantity and quality have been ensured in the clustering results that have already been implemented.

[0082] <Fourth embodiment> A fourth embodiment of the present disclosure will be described below. In the third embodiment, an example of a method for reclassifying poor-quality clusters included in any clustering result will be described. In this embodiment, an example of a method for reclassifying isolated elements or clusters that do not contain a sufficient number of elements (for example, clusters with the number of elements less than a threshold) included in any clustering result will be described. An isolated element refers to an element (feature) that is determined as not belonging to any cluster as a result of clustering. Depending on the clustering method, all elements may be included in one of the clusters, in which case a cluster with one or a small number of elements (a cluster with a number less than a threshold) may be included.

[0083] In various classification methods (clustering methods), the classification result of data may include isolated elements or clusters that do not contain a sufficient number of elements. Clusters that do not contain a sufficient number of elements are often false clusters formed by outliers by chance, and it may be desirable to exclude them, together with isolated elements, from the final clustering result as unreliable elements. In view of such a situation, the information processing device according to the present embodiment performs clustering again, focusing on isolated elements and clusters that do not contain a sufficient number of elements.

[0084] An example of the relationship between features and clusters will be described with reference to Fig. 13 in order to explain the effect that can be achieved by applying clustering by the information processing device according to this embodiment. In Fig. 13, similar to the example shown in Fig. 3, each feature is plotted in two dimensions for ease of explanation. In addition, in the example shown in Fig. 13, some feature are hatched to distinguish them from other feature.

[0085] 13(a), six clusters shown as clusters 211 to 216 are created, of which clusters 214 to 216 contain fewer than three elements, meaning that they do not contain a sufficient number of elements. Note that although the threshold value for determining whether the number of elements is sufficient is set to "3" here, this is merely an example, and in actual applications, it is desirable to set a suitable value according to the anticipated use case. In the example shown in FIG. 13(a), the four hatched feature amounts included in the clusters 214 to 216 do not contribute to the final number of clusters (number of IDs).

[0086] The example shown in Fig. 13(b) shows an example of the result of reclustering for the example shown in Fig. 13(a). In the example shown in Fig. 13(b), two clusters, clusters 214 and 2151, are generated by focusing on the four hatched feature amounts. Cluster 214 has less than three elements and therefore does not contribute to the final number of clusters (number of IDs). On the other hand, cluster 2151 has three or more elements and therefore is included in the final clustering result.

[0087] Moreover, the example shown in FIG. 13(c) shows an example of a result of reclustering different from the example shown in FIG. 13(b). In the example shown in FIG. 13(c), two clusters, clusters 214 and 2152, are generated by focusing on the four hatched feature amounts. Since the number of elements of cluster 214 is less than 3, it does not contribute to the final number of clusters (number of IDs). On the other hand, the number of elements of cluster 2152 is 3 or more. However, cluster 2152 includes element 2131 included in cluster 213 defined as a result of clustering that has already been performed. In such a case, cluster 2152 is determined to be a poor quality cluster and is not included in the final clustering result.

[0088] If cluster 2152 is determined to be a valid cluster, cluster 2152 and cluster 213 are merged to define cluster 2130 including six elements. On the other hand, among the series of elements included in cluster 2130, three elements not included in cluster 213 are likely to be different people (for example, similar strangers) from the person represented by cluster 213. In other words, cluster 2130 is likely to be a poor quality cluster that does not achieve internal bonding. Therefore, in such a case, cluster 2152 is determined to be an invalid cluster and will not be included in the final clustering result. That is, in the case of the example shown in FIG. 13(c), as a result, there is no change in the clustering result compared to the example shown in FIG. 13(a).

[0089] Here, an example of the functional configuration of the information processing device according to this embodiment will be described with reference to Fig. 14. Note that the information processing device according to this embodiment differs from the example shown in Fig. 12 in that it includes a second selection unit 107, a second re-sorting unit 108, and a second update unit 109 instead of the first selection unit 103, the first re-sorting unit 104, and the first update unit 105. Therefore, hereinafter, the description will be made mainly focusing on the parts different from the example shown in Fig. 12.

[0090] The second selection unit 107 extracts elements that constitute clusters that do not include isolated elements or a sufficient number of elements from the clustering result 1021, and outputs a second partial feature set 1071. The second selection unit 107 also removes each feature included in the second partial feature set 1071 from the clustering result 1021. Note that, for clusters in which the number of features becomes zero as a result of the feature removal, the second selection unit 107 also deletes the cluster itself.

[0091] The second re-classification unit 108 focuses on the features included in the second partial feature set 1071 and performs clustering on a series of features (the clustering result 1021 and the features included in the second partial feature set 1071). Specifically, the second re-classification unit 108 performs the repeated processing of S142 to S145 shown in FIG. 9 only on rows corresponding to the features included in the second partial feature set 1071, rather than on all rows of the corrected similarity matrix 131. For example, if there are N features F 1 ~F N The corrected similarity matrix 131 is created from the second partial feature set 1071, and the three features F i , F j , and F k In this case, the i-th, j-th, and k-th rows of the corrected similarity matrix 131 are successively set as the corrected similarity row vector 921, and the processes of S142 to S145 shown in FIG. Then, the second reclassification section 108 outputs a second partial clustering result 1081.

[0092] The second update unit 109 updates the clustering result 1021 with the second partial clustering result 1081 . Here, an example of the process of the second updating section 109 will be described with reference to FIG. In S1091, the second updating unit 109 obtains the second partial clustering result 1081. Then, the second updating section 109 executes the processes of S10911 and S10912 for each of the series of clusters (cluster candidates 10811) included in the acquired second partial clustering result 1081. Specifically, in S10911, the second updating section 109 checks whether or not any of the elements constituting the cluster candidate 10811 is included in the clustering result 1021. If any of the elements constituting the cluster candidate 10811 is included in the clustering result 1021, the second update unit 109 interrupts the processing for that cluster candidate 10811 and starts processing for the next cluster candidate 10811. On the other hand, if any of the elements constituting the cluster candidate 10811 is not included in the clustering result 1021 , the second updating unit 109 adds the cluster candidate 10811 to the clustering result 1021 . In this manner, second updating section 109 completes the process for each of the series of cluster candidates 10811. As a result of this series of processes, clustering result 1021 contains the final clustering result.

[0093] Then, the preliminary output unit 106 outputs the clustering result 1021 to a predetermined output destination (for example, a subsequent processing stage not shown). The preliminary output unit 106 is substantially similar to the preliminary output unit 106 shown in FIG.

[0094] By applying the above-described control, it becomes possible to reclassify isolated features or clusters that do not contain a sufficient number of features based on any clustering result. Narrowing the targets of clustering in this way makes it possible to reduce low-quality clusters and expand high-quality clusters (improving quality and ensuring quantity) without affecting clusters whose quantity and quality have been ensured in the clustering results that have already been implemented.

[0095] <Fifth embodiment> A fifth embodiment of the present disclosure will be described below. In the first embodiment, the correction function f mod () is an example of the case where a correction function with one input is applied. This is the feature F i and feature F j Similarity S ij However, when determining whether the features to be compared for similarity belong to the same cluster (for example, whether they belong to the same person), a single similarity S ij The basis for similarity calculation is feature values, but feature values ​​calculated from images of other people with various appearances do not always yield low similarity values, and there may be image pairs of other people that yield relatively high similarity values.

[0096] In view of such a situation, the information processing device according to the present embodiment determines whether or not features showing high feature values ​​are the same person by using a similarity S ij In other words, the information processing device according to the present embodiment uses a similarity S between features that show high feature values. ij For the similarity S ij Specifically, the information processing device performs the conversion using a similarity different from the feature quantity F j Face image D from which j A variation image D' obtained by applying variation to j and generate a variation image D' j The feature obtained by the feature extractor for j The similarity is corrected using the variation image D'. j is face image D j Since the facial image is the same as that of the person, the feature F j and feature F i If they are in the same cluster (person), the fluctuation feature F' j The feature value F i are expected to be in the same cluster.

[0097] Now, referring to FIG. 16, the fluctuation feature F' jAn example of a method for calculating the fluctuation similarity using the following will be described. In the example shown in FIG. j ), the feature extractor 191 extracts the feature 1101 (F j Similarly, a feature amount 1100 (F i ) is calculated. Then, the similarity calculator 192 calculates the feature amount 1100 (F i ) and feature 1101(F j ) similarity 1201(S ij The similarity calculator 192 corresponds to the part of the calculator 12 in the example shown in FIG. 1 that is particularly related to similarity calculation. The above is the similarity used in the above-described embodiment (in this embodiment, similarity 1201(S ij ) is an example of a method for calculating

[0098] Next, the variation similarity 1202 (S' ij In this calculation method, first, a face image 1001 (D j ) to the changed face image 1002 (D' j ) is generated. Then, the similarity 1201 (S ij ), the fluctuation feature amount 1102 (F' j ) is calculated, and the feature quantity 1100 (F j ) and the fluctuation feature 1102 (F' j ) and similarity 1202 (S' ij ) is calculated.

[0099] In addition, face image 1001 (D j ) to the changed face image 1002 (D' j Regarding the method of generating the face image 1001 (D), a general method called Data Augmentation (DA) can be applied, so a detailed description will be omitted. The following method is an example of a commonly used DA method applied to an image. Note that the following method is merely an example, and the face image 1001 (D j ) to the changed face image 1002 (D' j ) is not limited to a method for generating the same. Change the color tone (for example, add a reddish tint) Blur or make it more detailed Add rotation Add noise Fill part of an image with a specific color (For face images) Wear glasses or sunglasses (For face images) Apply a mask

[0100] In this embodiment, the correction function f mod (S in ,S' in ) is the similarity S in and the variation similarity S' in The two parameters are input. The correction function f according to this embodiment is expressed as (Equation 12) and (Equation 13) below. mod (S in ,S' in ) is shown below. Note that the correction function f mod (S in ), the correction functions exemplified as (Formula 8) to (Formula 10) in the first embodiment can be applied. In addition, in the following, averaging() represents a function that takes the average of a series of values ​​passed as an argument.

[0101]

number

[0102] In the above example, as shown in Equation 12, S in <th 2 If S' is the fluctuating similarity, in is not used, but the similarity S in and the variation similarity S' in Both may be used. In the example shown above, S in ≧th 2 If S, the similarity in Similarity other than the above (i.e., the variation similarity S' in ) was also used, but for a given value th 4 Using S in≧th 4 It may be possible to do so at the time of S. in When is large, S in It is important to use similarity measures other than In addition, in Equation 13, the similarity S in and the variation similarity S' in However, instead of the average, the maximum value or the minimum value may be applied, or another conditional expression may be applied. In the above example, the similarity S in and the variation similarity S' in Although an example in which the above two values ​​are used has been shown, a variation similarity calculated from more variation face images may be used.

[0103] In addition, multiple feature extractors may be used to extract the feature amount and the fluctuation feature amount. For example, Fig. 17 is a diagram showing another example of a method for calculating the fluctuation similarity. In the example shown in Fig. 17, in addition to the feature extractor 191, another feature extractor 1911 having characteristics related to the extraction of the feature amount different from that of the feature extractor 191 is provided.

[0104] Specifically, face image 1001 (D j ), the feature extractor 191 extracts the feature 1101 (F j ) is calculated. Similarly, the reference face image 1000 (D i ), the feature extractor 191 extracts the feature 1100 (F i ) is calculated. Then, the similarity calculator 192 calculates the feature amount 1100 (F i ) and feature 1101(F j ) similarity 1201(S ij ) is calculated. In the example shown in FIG. ij ) is calculated from the feature quantity 1100 (F i ) and feature 1101(F j A feature extractor 1911 different from the feature extractor 191 for extracting feature quantity 1101 (F j ) and feature quantity 1100 (Fi ) is extracted from face image 1001 (D j ) and 1000 face images (D i ) is input. Specifically, a face image 1001 (D j ), other features 1111 (F оj ) is calculated. Similarly, the face image 1000 (D i ) from the feature extractor 1911, other features 1110 (F оi ) is calculated. Then, another similarity calculator 1921 calculates another feature quantity 1110 (F оi ) and other features 1111(F оj ) is the similarity of the other similarity 1203 (S оij ) is calculated. At this time, face image 1001 (D j ) and 1000 face images (D i ) are face images of the same person, the similarity will most likely show a higher value regardless of whether feature extractor 191 or feature extractor 1911 is applied. Therefore, even in such a case, it is possible to handle it in the same way as the variable similarity described with reference to FIG.

[0105] As another example, a face image 1001 (D j ) and 1000 face images (D i ) is calculated, and the correction function f is applied according to the image characteristics (e.g., brightness and darkness of the image, amount of noise, etc.) mod The () itself may be controlled (e.g., changed). For example, Fig. 18 is a diagram showing an example of the relationship of similarity before and after correction accompanying control of the correction function. In the example shown in Fig. 18, the correction function is controlled so that the relationship of similarity before and after correction changes in the direction indicated by the arrow as the image becomes darker and the noise increases.

[0106] As described above, by using one or more variation similarities in addition to similarities, it is expected that the robustness of determining whether a series of data to be classified belongs to the same cluster or not can be improved when clustering the series of data to be classified. Therefore, it is expected that the formation of poor quality clusters that do not achieve internal connection, such as a case where a different person is judged to be the same person, can be prevented.

[0107] <Other embodiments> The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions. In addition, various modifications may be made within the scope of the basic technical idea of ​​the technology according to the embodiment of the present disclosure described above. As a specific example, in each of the above-mentioned embodiments, an example of a case where clustering is performed on face images that are mainly applied to face recognition has been described, but the type of data to be clustered is not particularly limited. In other words, as long as the data is subject to application of unsupervised data classification, it can be similarly applied to clustering based on the technology according to the present disclosure.

[0108] The disclosure of the present embodiment also includes the following methods, configurations, and programs. (Method 1) A control method for an information processing device, comprising: a generation step of generating a similarity matrix whose elements are the similarities between each of the features extracted from each of a series of data to be classified; a correction step of correcting the similarity matrix by replacing at least a portion of the elements of the similarity matrix with a similarity that has been transformed according to the level of the similarity corresponding to the element; and a classification step of classifying the series of data into one or more groups based on the similarity matrix corrected in the correction step. (Method 2) A method for controlling an information processing device described in Method 1, characterized in that the correction step corrects the similarity matrix by applying a transformation according to the similarity that exceeds a first threshold among the similarities corresponding to each of a series of the elements of the similarity matrix. (Method 3) The method for controlling an information processing device according to Method 2, wherein the first threshold value is a constant value designated in advance. (Method 4) The method for controlling an information processing device according to Method 2, wherein the first threshold value is a value that is a valley between two peaks when a histogram created for each row or column of the similarity matrix is ​​assumed to have a bimodal distribution. (Method 5) The method for controlling an information processing device according to any one of Methods 1 to 4, wherein the correction step applies different conversions to similarities exceeding a reference value and similarities below the reference value. (Method 6) A control method for an information processing device described in Method 5, characterized in that the correction step converts similarities that exceed the reference value into similarities with values ​​greater than before the conversion using a monotonically increasing function, and converts similarities that are less than the reference value into similarities with values ​​smaller than before the conversion. (Method 7) The control method for the information processing device described in Method 6, wherein the correction step comprises converting a first similarity that exceeds the reference value based on a second similarity of the two feature amounts when a variation is applied to one of the two feature amounts from which the first similarity is calculated. (Method 8) The control method of the information processing device described in Method 7, wherein the correction step applies any one of the average, maximum value, and minimum value of the first similarity and the second similarity as the converted similarity. (Method 9) A control method for an information processing device described in any one of Methods 6 to 8, characterized in that the correction step applies a conversion to a first similarity that exceeds the reference value based on a third similarity between two other feature amounts extracted by a second extraction means different from a first extraction means that extracted the two feature amounts from which the first similarity is calculated, using two data sets as inputs from which the two feature amounts are extracted. (Method 10) It includes a selection step of selecting at least some of the elements of the similarity matrix, and the reference value is the average value of the elements selected as the target in the selection step. The control method of the information processing apparatus according to any one of Methods 5 to 9, characterized in that. (Method 11) It includes a selection step of selecting at least some of the elements of the similarity matrix, and the reference value is the weighted average of the average value of the elements selected as the target in the selection step and 0 or 1. The control method of the information processing apparatus according to any one of Methods 5 to 9, characterized in that. (Method 12) The similarity matrix is an N×N matrix (N is an integer of 1 or more), with respect to the feature amounts F1 to FN extracted from each of N pieces of data, and the similarity between the i-th (i is an integer of 1 or more) feature amount Fi and the j-th (j is an integer of 1 or more) feature amount Fj is the element in the i-th row and j-th column. The control method of the information processing apparatus according to any one of Methods 1 to 11, characterized in that. (Method 13) The correction step performs targeting on the corrected similarity matrix so that the value in the i-th row and j-th column is equal to the value in the j-th row and i-th column. The control method of the information processing apparatus according to Method 12, characterized in that. (Method 14) When i < j, the correction step performs the targeting by replacing the value in the i-th row and j-th column with the value in the j-th row and i-th column, or replacing the value in the j-th row and i-th column with the value in the i-th row and j-th column, on the corrected similarity matrix. The control method of the information processing apparatus according to Method 13, characterized in that. (Method 15) The correction step performs the targeting by replacing the value in the i-th row and j-th column and the value in the j-th row and i-th column with any one of the average value, maximum value, and minimum value of the value in the i-th row and j-th column and the value in the j-th row and i-th column, on the corrected similarity matrix. The control method of the information processing apparatus according to Method 13, characterized in that. (Method 16) The control method for an information processing device described in any one of Methods 1 to 15, characterized in that the classification step classifies the series of data so that, for each group of elements in each row or column of the similarity matrix, if the number of elements whose values ​​exceed the second threshold is equal to or greater than a third threshold, the two feature quantities used to calculate the corresponding similarity belong to the same group. (Method 17) The method for controlling an information processing device according to Method 1, characterized in that the generating step includes an updating step of generating the similarity matrix for each group in which the variance between features extracted from each piece of data included in a common group is equal to or greater than a threshold value among the classification results of a plurality of data items performed in advance, using a set of features included in the group as the series of data to be classified, and updating the classification results of the plurality of data items performed in advance based on the classification results of the series of data based on the corrected similarity matrix in the classification step. (Method 18) A control method for an information processing device according to Method 1, further comprising a selection step of selecting a set of features included in a group that is an outlier or has a number of elements below a threshold value from among the results of classification of a plurality of data items performed in advance, the classification step determining which features should be classified into the same group as the selected features, updating the results of the classification of a plurality of data items performed in advance based on the results of the determination, and discarding the group if an element included in the group after the updated classification result is included in the results of the classification of a plurality of data items performed in advance. (Configuration 1) An information processing device comprising: a generating means for generating a similarity matrix whose elements are the similarities between respective features extracted from a series of data to be classified; a correcting means for correcting the similarity matrix by replacing at least a portion of the elements of the similarity matrix with a similarity that has been transformed according to the level of the similarity corresponding to the element; and a classifying means for classifying the series of data into one or more groups based on the similarity matrix corrected by the correcting means. (Program 1) A program for causing a computer to function as an information processing device, comprising: a generation means for generating a similarity matrix whose elements are the similarities between each of the features extracted from each of a series of data to be classified; a correction means for correcting the similarity matrix by replacing at least a portion of the elements of the similarity matrix with a similarity that has been converted according to the level of the similarity corresponding to the element; and a classification means for classifying the series of data into one or more groups based on the similarity matrix corrected by the correction means. [Explanation of symbols]

[0109] 12 Calculation section 13 Correction section 14 Classification section

Claims

1. A method for controlling an information processing device, comprising: A generation step of generating a similarity matrix having elements each representing a similarity between each of the feature quantities extracted from each of the series of data to be classified; a correction step of correcting the similarity matrix by replacing at least some of the elements of the similarity matrix with similarities that have been converted according to the magnitude of the similarity corresponding to the elements; a classification step of classifying the series of data into one or more groups based on the similarity matrix corrected in the correction step; 2. A method for controlling an information processing device, comprising:

2. The control method for an information processing device as described in claim 1, characterized in that the correction step corrects the similarity matrix by applying a transformation according to the similarity that exceeds a first threshold among the similarities corresponding to each of a series of the elements of the similarity matrix.

3. 3. The method according to claim 2, wherein the first threshold value is a constant value specified in advance.

4. 3. The control method for an information processing device according to claim 2, wherein the first threshold value is a value that is a valley of a bimodal distribution when a histogram created for each row or column of the similarity matrix is ​​assumed to have a bimodal distribution.

5. 2. The method according to claim 1, wherein the correction step performs different conversions on similarities exceeding a reference value and similarities below the reference value.

6. The correction step includes: By a monotonically increasing function, converting the similarity exceeding the reference value into a similarity having a larger value than before the conversion; The similarity less than the reference value is converted to a similarity having a smaller value than before the conversion.

6. The method for controlling an information processing apparatus according to claim 5.

7. 7. The control method for an information processing device according to claim 6, wherein the correction step comprises converting a first similarity that exceeds the reference value based on a second similarity of the two feature quantities when a variation is applied to one of the two feature quantities from which the first similarity is calculated.

8. 8. The method according to claim 7, wherein the correction step applies any one of an average, a maximum value, and a minimum value of the first similarity and the second similarity as the converted similarity.

9. 7. The control method for an information processing device according to claim 6, wherein the correction step converts a first similarity exceeding the reference value based on a third similarity between two other feature amounts extracted by a second extraction means different from a first extraction means that extracted the two feature amounts from which the first similarity is calculated, using as input two data sets from which the two feature amounts are extracted.

10. A selection step of selecting at least some of the elements of the similarity matrix as targets; The reference value is the average value of the elements selected as targets in the selection step.

6. The method for controlling an information processing apparatus according to claim 5.

11. A selection step of selecting at least some of the elements of the similarity matrix as targets; The reference value is a weighted average of the average value of the elements selected as targets in the selection step and 0 or 1.

6. The method for controlling an information processing apparatus according to claim 5.

12. 2. The control method for an information processing device according to claim 1, wherein the similarity matrix is ​​an N-row, N-column matrix in which the i-th (i is an integer of 1 or more) feature Fi and the j-th (j is an integer of 1 or more) feature Fj are elements of the i-th row and j-th column, for features F1 to FN extracted from each of N pieces of data (N is an integer of 1 or more).

13. 13. The method of controlling an information processing device according to claim 12, wherein said correcting step performs objectification on said corrected similarity matrix so that a value in row i, column j and a value in row j, column i become equal.

14. The correction step includes: When i<j, the similarity matrix after the correction is Replace the value in row i, column j with the value in row j, column i, or By replacing the value in row j, column i with the value in row i, column j, Applying the objectification 14. The method for controlling an information processing apparatus according to claim 13.

15. 14. The control method for an information processing device according to claim 13, wherein the correction step performs the objectification by replacing the value in row i, column j and the value in row j, column i of the corrected similarity matrix with one of an average value, a maximum value, and a minimum value of the value in row i, column j and the value in row j, column i of the similarity matrix after the correction.

16. The classification step includes: For each group of elements in each row or column of the similarity matrix, If the number of elements whose values ​​exceed the second threshold is equal to or greater than a third threshold, The series of data is classified so that, for each of the elements, the two feature quantities that are the basis for calculating the corresponding similarity belong to the same group.

2. The method for controlling an information processing apparatus according to claim 1.

17. The generating step generates the similarity matrix for each group in which the variance between features extracted from each piece of data included in a common group is equal to or greater than a threshold value among the results of classifying a plurality of pieces of data previously, using a set of features included in the group as the series of data to be classified; and updating a result of classifying the plurality of data items previously performed based on a result of classifying the series of data items based on the similarity matrix after the correction in the classifying step.

2. The method for controlling an information processing apparatus according to claim 1.

18. The method further includes a selection step of selecting a set of features included in an outlier or a group with a number of elements equal to or less than a threshold value from among a plurality of data classification results performed in advance, The classification step determines a feature to be classified into the same group as the selected feature, updates a classification result of the plurality of data items carried out in advance based on the result of the determination, and discards the group when an element included in the group after the classification result is updated is included in the classification result of the plurality of data items carried out in advance.

2. The method for controlling an information processing apparatus according to claim 1.

19. A generating means for generating a similarity matrix having elements each representing a similarity between each of the feature quantities extracted from each of a series of data to be classified; a correction means for correcting the similarity matrix by replacing at least some of the elements of the similarity matrix with similarities that have been converted in accordance with the magnitude of the similarity corresponding to the elements; a classification means for classifying the series of data into one or more groups based on the similarity matrix corrected by the correction means; An information processing device comprising:

20. Computer, A generating means for generating a similarity matrix having elements each representing a similarity between each of the feature quantities extracted from each of a series of data to be classified; a correction means for correcting the similarity matrix by replacing at least some of the elements of the similarity matrix with similarities that have been converted in accordance with the magnitude of the similarity corresponding to the elements; a classification means for classifying the series of data into one or more groups based on the similarity matrix corrected by the correction means; A program for causing an information processing device to function, comprising:

Images