Information processing device, information processing method, and program

Abstract

A learning processing unit (1) acquires a plurality of initial learning results by a plurality of unsupervised learning methods. A separation degree calculation unit (2) calculates the degree of separation between clusters on the basis of a side of interest. A deletion processing unit (3) deletes a side with a threshold or more from the initial learning results. A distance matrix calculation unit (4) calculates distance matrices. An evaluation index calculation unit (5) calculates an evaluation index of each distance matrix. A score calculation unit (6) calculates the score of each intermediate learning result for each cluster. A uniformity level calculation unit (7) acquires the minimum values of the calculated scores, and calculates the maximum value of the minimum values as a uniformity level. An end determination unit (8) updates the value of M if the uniformity level is larger than a threshold, or otherwise ends side deletion processing. A learning result selection unit (9) selects final learning results. A clustering unit (10) performs clustering of the final learning results. The information processing device repeats specific processing out of the above each time the value of M is updated.

Description

[Technical field]

[0001] The present disclosure relates to an information processing device, an information processing method, and a program, and, for example, to an information processing device, an information processing method, and a program that sequentially input input vectors belonging to an arbitrary class and learn an input distribution structure of the input vectors. [Background technology]

[0002] A method called Self-Organizing Incremental Neural Network (SOINN) has been proposed as a learning method for multiplying neurons as necessary during learning (Patent Document 1). SOINN has many advantages, such as being able to learn non-stationary inputs by autonomously managing the number of nodes, and being able to extract an appropriate number of classes and a topological structure even for classes with complex distribution shapes. As an example of an application of SOINN, in pattern recognition, after learning classes of hiragana characters, it is possible to additionally learn classes of katakana characters, etc.

[0003] As an example of such SOINN, a method called E-SOINN (Enhanced SOINN) has been proposed. E-SOINN allows online additional learning, which adds learning as needed, and has the advantage of being more efficient than batch learning. Therefore, E-SOINN allows additional learning even when the learning environment changes to a new environment. E-SOINN also has the advantage of being highly resistant to noise in the input data.

[0004] However, SOINN including E-SOINN has problems such as the learning result differing depending on the input order of input data, and the learning accuracy decreasing when high-dimensional data of 1000 dimensions or more is used as input data. In order to solve these problems, a method called LB-SOINN (Load Balance Self-Organizing Incremental Neural Network) has been proposed (Patent Document 2). LB-SOINN treats the load of a node in a network as a node learning time, detects a node with a large node learning time, and generates a new node on the edge connecting the detected node and its adjacent node, the new node having a weight vector determined based on the weight vector of the detected node. This alleviates the increase in the learning time of the detected node, and by generating a new node in its vicinity, the structure of the input data can be learned more accurately. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2008-217246 A [Patent Document 2] JP 2014-164396 A Summary of the Invention [Problem to be solved by the invention]

[0006] However, when learning input vectors using SOINN as described above, it was found that the influence of noise can cause clusters that should be separated to be connected by edges, resulting in the problem that the distribution of input vectors cannot be learned accurately.

[0007] Figure 13 shows an example of input data with little noise. The input data shown in Figure 13 is a combination of two parts where two-dimensional input vectors are distributed in an arc shape. Figure 14 shows the network structure obtained by learning the input data in Figure 13 using the above-mentioned SOINN. In this case, since there is little noise, the distribution of the input vectors is learned almost accurately.

[0008] However, in noisy input data, the influence of the input vector caused by the noise becomes large between the regions (clusters) in which the input data is distributed. Figure 15 shows an example of input data in which noise has been added to the input data in Figure 13. The input data shown in Figure 15 has two parts in which the two-dimensional input vector is distributed in an arc shape, but the boundary is unclear due to the influence of noise. Figure 16 shows the network structure obtained by learning the input data in Figure 15 using the above-mentioned SOINN. In this case, due to the influence of noise, two clusters that should be separated are connected by an edge caused by the noise and become one cluster. Therefore, it can be seen that the distribution of the input vector cannot be learned accurately.

[0009] In this way, even if clusters that should be separated are not separated, it is possible to appropriately separate the clusters by deleting unnecessary edges. For example, a length threshold that specifies the length of the edges to be deleted may be given, and edges longer than the length threshold may be deleted. However, in order to determine an appropriate value for the length threshold, it is necessary to repeat the edge deletion process while changing the length threshold, observe the learning results after edge deletion, and carry out trial and error to find an appropriate length threshold.

[0010] For example, even if edges are deleted using the average edge length as the length threshold, there may be cases where the deletion of edges is insufficient. FIG. 17 shows an example of edge deletion. As shown in FIG. 17, in the upper network before edge deletion, edges are deleted using the average edge length as the length threshold, but unnecessary edges may remain as in the central network after edge deletion. In such a case, unnecessary edges can be deleted as in the lower network of FIG. 17 by manually adjusting the length threshold to a shorter value and performing edge deletion again. However, since manual work is required, this is time-consuming and inefficient.

[0011] In this way, it is often the case that clusters cannot be separated using a single length threshold, but it is also possible to use a multi-stage edge removal process in which an edge removal process is performed using a certain length threshold to separate clusters, and then an edge removal process is performed again using a different length threshold. Even in this case, however, the optimal value for the length threshold and the optimal number of times to repeat the edge removal process must be found by trial and error.

[0012] The present invention has been made in consideration of the above circumstances, and aims to provide an information processing device, an information processing method, and a program that can accurately learn the structure of input data by automatically performing edge deletion processing. [Means for solving the problem]

[0013] An information processing device according to one embodiment receives input vectors in sequence and learns an input distribution structure of the input vector as a network structure in which a plurality of nodes described by multidimensional vectors and a plurality of edges connecting two of the nodes are arranged, the information processing device comprising: a learning processing unit that acquires a plurality of initial learning results obtained by learning the same input data by a plurality of unsupervised learning methods; a separation degree calculation unit that sequentially selects one initial learning result from the plurality of initial learning results, sequentially selects each of a plurality of edges included in the selected initial learning result as an edge of interest, and calculates a degree of cluster separation in a case where an edge longer than the edge of interest is deleted from the selected initial learning result and in a case where an edge having a length equal to or longer than the edge of interest is deleted from the selected initial learning result; an edge deletion processing unit that deletes from the selected initial learning result an edge that is equal to or longer than a length threshold that is the length of the edge of interest corresponding to the Mth largest degree of cluster separation among the degrees of cluster separation calculated for the plurality of edges; and two intermediate learning results that can be selected from a plurality of intermediate learning results that are the plurality of initial learning results after the edge deletion by the edge deletion processing unit. a distance matrix calculation unit that selects a learning result pair, selects a cluster pair consisting of two clusters selected one from each of the two intermediate learning results of the selected learning result pair, and calculates a distance matrix indicating a distance between a node belonging to one cluster and a node belonging to the other cluster for all selectable cluster pairs; an evaluation index calculation unit that obtains a minimum value for each column and each row of each distance matrix, and calculates an evaluation index for each distance matrix based on the number of the obtained minimum values ​​that satisfy a predetermined condition; a score calculation unit that calculates a score indicating a degree of similarity between one cluster included in one intermediate learning result and a cluster of another intermediate learning result, for each cluster of each intermediate learning result, based on the evaluation index of the distance matrix calculated for all the learning result pairs; a stroke calculation unit that obtains a minimum value of the scores calculated for each cluster of each intermediate learning result, and calculates a maximum value of the obtained multiple minimum values ​​as a stroke length; and a termination determination unit that adds 1 to M to update the value of M when the stroke length is greater than a stroke length threshold, and terminates the process of deleting edges when the stroke length is less than the stroke length threshold.The system further comprises a learning result selection unit which selects a final learning result from a plurality of intermediate learning results based on the score of each cluster of each intermediate learning result calculated by the score calculation unit when the end determination unit determines that the process of deleting edges is to be ended, and a clustering unit which clusters the final learning results, and the edge deletion processing unit, the distance matrix calculation unit, the score calculation unit, the pixel number calculation unit and the end determination unit repeat the process every time the value of M is updated.

[0014] In one embodiment of an information processing device, in the above information processing device, the degree of separation of the clusters is desirably 0 when the cluster to which the target edge belongs is not separated into two clusters after the edge is deleted, and has a value other than 0 when the cluster is separated.

[0015] In an information processing device according to an embodiment, in the above information processing device, it is preferable that the degree of separation of the clusters has a positive or negative value when the cluster to which the edge of interest belongs is separated into two clusters.

[0016] In an information processing device according to one embodiment, in the above information processing device, when the cluster to which the target edge belongs is separated into two clusters, it is desirable that the degree of separation of the clusters be a larger value in the case of a positive value and a smaller value in the case of a negative value, the closer the numbers of nodes of the two separated clusters are.

[0017] In one embodiment of the information processing device, in the above information processing device, when the cluster to which the target edge belongs is separated into two clusters, it is desirable that the degree of separation of the clusters becomes larger in value if the value is positive, and becomes smaller in value if the value is negative, as the number of nodes contained in the two separated clusters increases.

[0018] An information processing device according to an embodiment of the present invention is the above information processing device, n k1 The number of nodes in cluster k1 that exists when edges longer than the edge in question are deleted is n k2is the number of nodes in cluster k2 that exists when an edge having a length equal to or greater than the edge of interest is deleted, the degree of separation D of the clusters is desirably defined by the following formula:

number

[0019] In an information processing device according to one embodiment, in the above information processing device, it is preferable that the score calculation unit generates an evaluation index matrix composed of evaluation indexes for all pairs between two clusters selected one by one from each of the two intermediate learning results based on the evaluation indexes calculated for all distance matrices, calculates a score indicating whether a cluster included in one learning result is similar to a cluster of another learning result based on the evaluation index included in the evaluation index matrix, and generates a score matrix including scores for the clusters belonging to all learning results, and the learning result selection unit selects a final learning result from the multiple learning results based on the score matrix.

[0020] In one embodiment of the information processing device, in the above information processing device, it is preferable that the evaluation index calculation unit calculates an average distance between nodes belonging to each of the two clusters, calculates the average distance of the two average distances thus calculated, and calculates an evaluation index for each distance matrix based on the number of minimum values ​​in each column and each row of each distance matrix that are equal to or greater than the average value.

[0021] In an information processing device according to one embodiment, in the above information processing device, it is desirable that the evaluation index is a value obtained by dividing the number of the minimum values ​​in each column and each row of each distance matrix that are equal to or greater than the average value by the total number of nodes belonging to the two clusters.

[0022] In one embodiment of the information processing device, in the above-mentioned information processing device, it is desirable that the score calculation unit compares each evaluation index included in the evaluation index matrix with a threshold value, assigns a score based on the comparison result, and calculates a total score for each cluster.

[0023] In one embodiment of the information processing device, in the above-mentioned information processing device, it is desirable that the learning result selection unit obtains the minimum value among the total scores of the clusters belonging to each intermediate learning result, and selects the intermediate learning result corresponding to the maximum value among the obtained minimum values.

[0024] An information processing method according to one embodiment is an information processing method for sequentially inputting an input vector and learning an input distribution structure of the input vector as a network structure in which a plurality of nodes described by a multidimensional vector and a plurality of edges connecting two of the nodes are arranged, the information processing method includes: acquiring a plurality of initial learning results obtained by learning the same input data using a plurality of unsupervised learning methods; sequentially selecting one initial learning result from the plurality of initial learning results; sequentially selecting one cluster from the one selected initial learning result; sequentially selecting each of a plurality of edges included in the selected cluster as an edge of interest; calculating a degree of separation of the cluster in a case where an edge longer than the edge of interest is deleted from the network structure and a case where an edge having a length equal to or longer than the edge of interest is deleted from the network structure; deleting an edge having a length equal to or longer than the edge of interest from the selected cluster based on the degree of cluster separation calculated for each edge of interest; a cluster pair consisting of two clusters selected one by one from each of the intermediate learning results, for all of the selectable cluster pairs, calculate a distance matrix indicating the distance between a node belonging to one cluster and a node belonging to the other cluster, obtain a minimum value for each column and row of each distance matrix, calculate an evaluation index for each distance matrix based on the number of the obtained minimum values ​​that satisfy a predetermined condition, calculate a score indicating the degree of similarity between one cluster included in one intermediate learning result and a cluster of another intermediate learning result based on the evaluation index of the distance matrix calculated for all of the learning result pairs, obtain a minimum value of the scores calculated for each cluster of each intermediate learning result, calculate a maximum value of the obtained multiple minimum values ​​as a stroke length, add 1 to M to update the value of M if the stroke length is greater than a stroke length threshold, terminate the process of deleting edges if the stroke length is less than the stroke length threshold, and when it is determined that the process of deleting edges is to be terminated, select a final learning result from the multiple intermediate learning results based on the calculated scores of each cluster of each intermediate learning result;The final learning result is clustered, and each time the value of M is updated, the deletion of the edges, the calculation of the distance matrix, the calculation of the score, the calculation of the pixel count, and the determination of the end are repeated.

[0025] A program according to one embodiment causes a computer to sequentially input an input vector and learn an input distribution structure of the input vector as a network structure in which a plurality of nodes described by a multidimensional vector and a plurality of edges connecting two of the nodes are arranged, the program including: a process of acquiring a plurality of initial learning results obtained by learning the same input data using a plurality of unsupervised learning methods; a process of sequentially selecting one initial learning result from the plurality of initial learning results and sequentially selecting one cluster from the one selected initial learning result; a process of sequentially selecting each of a plurality of edges included in the selected cluster as an edge of interest, and calculating a degree of separation of the cluster in a case where an edge longer than the edge of interest is deleted from the network structure and a case where an edge having a length equal to or longer than the edge of interest is deleted from the network structure; a process of deleting edges from the selected cluster that are equal to or longer than a length threshold, which is the length of the edge of interest corresponding to the Mth largest degree of cluster separation, based on the degree of cluster separation calculated for each edge of interest; a process of selecting a learning result pair consisting of intermediate learning results, selecting a cluster pair consisting of two clusters selected one from each of the two selected intermediate learning results, and calculating a distance matrix indicating the distance between a node belonging to one cluster and a node belonging to the other cluster for all selectable cluster pairs; a process of obtaining a minimum value for each column and row of each distance matrix, and calculating an evaluation index for each distance matrix based on the number of the obtained minimum values ​​that satisfy a predetermined condition; a process of calculating a score indicating the degree of similarity between a cluster included in one intermediate learning result and a cluster of another intermediate learning result, for each cluster of each intermediate learning result, based on the evaluation index of the distance matrix calculated for all the learning result pairs; a process of obtaining a minimum value of the scores calculated for each cluster of each intermediate learning result, and calculating a maximum value of the obtained multiple minimum values ​​as a stroke length; a process of updating the value of M by adding 1 if the stroke length is greater than a stroke length threshold, and terminating the process of deleting edges if the stroke length is less than the stroke length threshold;A process of selecting a final learning result from a plurality of intermediate learning results based on the score of each cluster of each calculated intermediate learning result, and a process of clustering the final learning results are executed by a computer, and each time the value of M is updated, the process of deleting the edges, the process of calculating the distance matrix, the process of calculating the score, the process of calculating the pixel count, and the process of determining the end are repeated. Effect of the Invention

[0026] According to the present invention, it is possible to provide an information processing device, an information processing method, and a program capable of accurately learning the structure of input data by automatically performing edge deletion processing. [Brief description of the drawings]

[0027] [Figure 1] FIG. 2 is a diagram illustrating an example of a system configuration for implementing an information processing device according to a first embodiment. [Diagram 2] FIG. 1 is a diagram illustrating a schematic configuration of an information processing device according to a first embodiment. [Diagram 3] 4 is a flowchart of a process in the information processing device according to the first embodiment. [Figure 4] 4 is a flowchart of a process in the information processing device according to the first embodiment. [Diagram 5] This figure shows a schematic diagram of a first example of the change in cluster between the case where a side longer than a given side is deleted (case 1) and the case where a side longer than that side is deleted (case 2). [Figure 6] This figure shows a schematic diagram of a second example of the change in cluster between the case where an edge longer than the edge is deleted (case 1) and the case where an edge longer than the edge is deleted (case 2). [Figure 7] FIG. 13 is a diagram illustrating an example where a cluster diversity change amount dS is 0. [Figure 8] FIG. 13 is a diagram illustrating an example of a case where a cluster diversity change amount dS is not 0. [Figure 9]FIG. 2 is a diagram showing an overview of edge deletion in the first embodiment. [Figure 10] FIG. 11 is a diagram showing an example of input data used for edge deletion in the first embodiment. [Figure 11] FIG. 11 is a diagram showing a network structure when edge deletion processing is performed on the input data in FIG. [Figure 12] FIG. 13 is a diagram showing an example of a network and pixel count obtained when a two-stage edge deletion process is performed. [Figure 13] FIG. 13 is a diagram illustrating an example of input data with little noise. [Figure 14] FIG. 14 is a diagram showing a network structure obtained by learning the input data of FIG. 13 using the above-mentioned SOINN. [Figure 15] FIG. 14 is a diagram showing an example of input data obtained by adding noise to the input data of FIG. 13. [Figure 16] FIG. 16 is a diagram showing a network structure obtained by learning the input data of FIG. 15 using the above-mentioned SOINN. [Figure 17] FIG. 13 is a diagram illustrating an example of deleting an edge. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0028] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In the drawings, the same elements are given the same reference numerals, and duplicated explanations will be omitted as necessary.

[0029] First embodiment FIG. 1 is a diagram showing an example of a system configuration for realizing an information processing device according to the first embodiment. The information processing device 100 can be realized by a computer 1000 such as a dedicated computer or a personal computer (PC). However, the computer does not need to be a single computer physically, and may be multiple computers when performing distributed processing. As shown in FIG. 1, the computer 1000 has a CPU (Central Processing Unit) 1001, a ROM (Read Only Memory) 1002, and a RAM (Random Access Memory) 1003, which are connected to each other via a bus 1004. Note that an OS software for operating the computer will not be described, but it is assumed that the computer that constructs this information processing device also has such software.

[0030] An input / output interface 1005 is also connected to the bus 1004. To the input / output interface 1005, for example, an input unit 1006 including a keyboard, a mouse, a sensor, etc., a display including a CRT, an LCD, etc., an output unit 1007 including a headphone, a speaker, etc., a storage unit 1008 including a hard disk, etc., and a communication unit 1009 including a modem, a terminal adapter, etc. are connected.

[0031] The CPU 1001 executes various processes according to various programs stored in the ROM 1002 or various programs loaded from the storage unit 1008 to the RAM 1003, for example, processes of each unit of the information processing device 100 described later in this embodiment. A GPU (Graphics Processing Unit) may be provided to execute various processes according to various programs stored in the ROM 1002 or various programs loaded from the storage unit 1008 to the RAM 1003, similarly to the CPU 1001, for example, processes of each unit of the information processing device 100 described later in this embodiment. The GPU is suitable for performing typical processes in parallel, and can improve the processing speed compared to the CPU 1001 by applying it to processing in a neural network described later. The RAM 1003 also appropriately stores data necessary for the CPU 1001 and the GPU to execute various processes.

[0032] The communication unit 1009 performs communication processing, for example, via the Internet (not shown), transmits data provided by the CPU 1001, and outputs data received from a communication partner to the CPU 1001, RAM 1003, and storage unit 1008. The storage unit 1008 exchanges information with the CPU 1001 and stores and erases information. The communication unit 1009 also performs communication processing of analog or digital signals with other devices.

[0033] The input / output interface 1005 is also connected to a drive 1010 as necessary, and, for example, a magnetic disk 1011, an optical disk 1012, a flexible disk 1013, or a semiconductor memory 1014 is appropriately attached, and computer programs read from them are installed in the memory unit 1008 as necessary.

[0034] Next, each process in the information processing device 100 according to the present embodiment will be described. The information processing device 100 has a neural network with a non-hierarchical structure in which nodes described by multidimensional vectors are arranged, and the neural network is stored in a storage unit such as the RAM 1003.

[0035] The neural network in this embodiment is a self-replicating neural network that inputs an input vector into the neural network and automatically increases the number of nodes placed in the neural network based on the input vector. By using a self-replicating neural network, the number of nodes can be automatically increased, so that the neural network is not limited to stationary environments in which an input vector is given randomly from the input vector space, but can also be used in non-stationary environments in which, for example, the class to which the input vector belongs is switched at regular intervals and an input vector is given randomly from the class after the switch.

[0036] The neural network in this embodiment has a non-hierarchical structure. By adopting a non-hierarchical structure, additional learning can be performed without specifying the timing to start learning in other layers. In other words, additional learning can be performed online.

[0037] The input data is input as a multidimensional input vector belonging to an arbitrary class. For example, the input vector is stored in a temporary storage unit (for example, the RAM 1003) and is sequentially input to the neural network stored in the temporary storage unit.

[0038] In the following, for the sake of simplicity, the neural network in this embodiment will also be simply referred to as a network.

[0039] The information processing device 100 according to the first embodiment will be specifically described below. The information processing device 100 learns (clusters) input data including a plurality of multidimensional vectors as data elements by an unsupervised learning method, for example, the various SOINN methods described above, and acquires a learning result represented as a network composed of a plurality of nodes and edges connecting two nodes. The information processing device 100 then automatically determines a length threshold that is a criterion for edge deletion, and can acquire a learning result having a network with a more suitable structure by deleting edges that are equal to or greater than this length threshold.

[0040] Fig. 2 shows a schematic configuration of an information processing device 100 according to the first embodiment. The information processing device 100 includes a learning processing unit 1, a separation degree calculation unit 2, an edge deletion processing unit 3, a distance matrix calculation unit 4, an evaluation index calculation unit 5, a score calculation unit 6, an image area calculation unit 7, a termination determination unit 8, a learning result selection unit 9, and a clustering unit 10. Note that, on the hardware side, the functional configuration shown in Fig. 2 is realized by cooperation between software and hardware resources such as the above-mentioned CPU 1001.

[0041] The learning processing unit 1 learns a plurality of input vectors, for example, by the above-mentioned E-SOINN or LB-SOINN, to acquire a network structure composed of nodes and edges. In this embodiment, the learning processing unit 1 is configured to learn (cluster) input data including a plurality of multidimensional vectors under a plurality of different unsupervised learning conditions, to acquire a plurality of learning results with different network structures. Hereinafter, the plurality of learning results with different network structures acquired by the learning processing unit 1 are referred to as a plurality of initial learning results.

[0042] The separation degree calculation unit 2 calculates the degree to which the clusters are separated by deleting an edge of the selected cluster when the edge is deleted.

[0043] The edge deletion processing unit 3 determines a length threshold to be used for edge deletion based on the calculated degree of cluster separation, and performs a process of deleting edges in the selected cluster based on the determined length threshold. Note that edge deletion can be performed collectively after the learning result is acquired, so it is a relatively simple process and can be easily introduced. Also, it can be applied to any network consisting of nodes and edges, regardless of the learning method. Note that, hereinafter, the network structure obtained after deleting edges from the initial learning result is referred to as the intermediate learning result.

[0044] After completing the edge deletion process for all clusters included in all initial learning results and acquiring multiple intermediate learning results, the distance matrix calculation unit 4 selects two clusters from each of two different intermediate learning results and calculates a distance matrix whose components are the distances between the nodes included therein.

[0045] The evaluation index calculation unit 5 calculates an evaluation index indicating the similarity between the two selected clusters based on the distance matrix. After calculating the evaluation indexes for all pairs of two clusters selectable from the two selected different intermediate learning results, the evaluation index calculation unit 5 may output an evaluation index matrix whose components are the cluster similarities between the two different intermediate learning results.

[0046] The score calculation unit 6 calculates a score indicating the similarity between the two selected different intermediate learning results based on the evaluation matrix index. After calculating the scores for all pairs of two selectable different intermediate learning results, the score calculation unit 6 may output a score matrix whose components are the scores of the two different intermediate learning results.

[0047] The stroke ratio calculation unit 7 calculates a stroke ratio indicating the similarity between a plurality of intermediate learning results.

[0048] The end determination unit 8 determines whether to repeat the edge deletion process based on the calculated pixel count.

[0049] When it is determined that the edge deletion process is completed, the learning result selection unit 9 selects one intermediate learning result from the multiple intermediate learning results after the edge deletion process in accordance with a predetermined condition.

[0050] The clustering unit 10 clusters the selected initial learning result and outputs the final learning result.

[0051] A specific description will be given below of the processing performed by the information processing device 100. Fig. 3 and Fig. 4 show a flowchart of the processing performed by the information processing device 100 according to the first embodiment.

[0052] Step SA1 The learning processing unit 1 performs learning of the input data through the following steps SA11 and SA12.

[0053] Step SA11 The learning processing unit 1 reads input data. The input data is stored in, for example, the above-mentioned RAM 1003 or the storage unit 1008, and the learning processing unit 1 can read the input data as needed.

[0054] Step SA12 The learning processing unit 1 sequentially inputs and learns multidimensional input vectors, which are elements of input data, to obtain a learning result having a network structure composed of nodes and edges connecting the nodes. In this embodiment, the learning processing unit 1 obtains a plurality of different initial learning results by learning the same input data under a plurality of different unsupervised learning conditions. The plurality of different unsupervised learning conditions may be the same unsupervised learning method with changed parameters used for learning, or may be different unsupervised learning methods, or may be a combination of both. As described above, the learning result obtained by the learning processing unit 1 is referred to as the initial learning result. Needless to say, the network structures of the plurality of initial learning results obtained by learning under a plurality of different unsupervised learning conditions are different from each other.

[0055] Step SA2 The separation degree calculation unit 2 calculates the amount of change in cluster diversity, which is the degree of cluster separation, through the following steps SA21 to SA27.

[0056] Step SA21 The separation degree calculation unit 2 determines whether or not there is any initial learning result that has not yet been selected as the selected learning result SR among the multiple initial learning results.

[0057] Step SA22 When there are initial learning results that have not been selected as the selected learning result SR, the separation degree calculation unit 2 selects one of the initial learning results as the selected learning result SR.

[0058] Step SA23 The separation degree calculation unit 2 determines whether there is any side that has not been selected as the side of interest among the sides included in the selection learning result SR. If there is no side that has not been selected as the side of interest, the process returns to step SA21.

[0059] Step SA24 If there is an edge that has not been selected as the edge of interest among the edges included in the selected cluster, the separation degree calculation unit 2 selects one edge from among the edges as the edge of interest.

[0060] Step SA25 The separation degree calculation unit 2 calculates a cluster diversity amount S1 when an edge longer than the edge of interest is removed, and a cluster diversity amount S2 when an edge longer than the edge of interest is removed.

[0061] Next, the degree of cluster separation used in determining the length threshold in this embodiment will be described. When determining the length threshold, it is desirable to select the length of the shortest edge among the edges connecting the clusters as the length threshold. In other words, it is desirable to select the length of the edge deleted when the clusters are first separated when the edges between the clusters are deleted in order of length as the length threshold. If it can be set in this way, the clusters can be separated without any edges remaining between the clusters and without needlessly deleting edges within the clusters. However, even if edges of noise nodes or noise clusters (clusters formed by gathering noise nodes) are deleted, it is recognized as a kind of cluster separation, and the length threshold that causes this kind of cluster separation cannot be said to be appropriate. Therefore, when determining the length threshold, it is necessary to take into consideration the state of the clusters formed by separation (especially the number of nodes belonging to the cluster and the ratio of divided nodes).

[0062] Therefore, in this embodiment, the degree of cluster separation (in this embodiment, the amount of change in cluster diversity described later) is introduced to determine whether clusters are separated and whether the separated cluster is a noise cluster only when an edge with a length equal to or greater than the edge length is deleted. This makes it possible to delete edges using an appropriate length threshold determined based on the degree of cluster separation.

[0063] In this embodiment, the degree of cluster separation is a value that each side has, and when comparing the case in which an edge longer than that side is deleted (referred to as case 1) with the case in which an edge longer than that side is deleted (referred to as case 2), if the number of clusters is increased in case 2 compared to case 1, it is an index that has the property of taking a finite value in the first direction, and taking 0 if there is no change.

[0064] The change in clusters when comparing Case 1 and Case 2 will be described below with reference to the drawings. FIG. 5 shows a schematic diagram of a first example of the change in clusters when an edge longer than a certain edge is deleted (Case 1) and when an edge longer than that edge is deleted (Case 2). In this example, edge E2 is the edge of interest and is deleted, and in both Cases 1 and 2, the cluster before the edge deletion remains as one cluster without being separated even though the number of edges is reduced. In this embodiment, the degree of cluster separation at this time is 0 as described above.

[0065] 6 shows a second example of the change in clusters between the case where an edge longer than a certain edge is deleted (Case 1) and the case where an edge longer than that edge is deleted (Case 2). In this example, edge E1 is the edge of interest and is deleted, and in Case 1, the cluster before the edge deletion remains as one cluster without separation even though the number of edges is reduced, whereas in Case 2, it is separated into two clusters. In this embodiment, as described above, the degree of cluster separation at this time is a finite value in the first direction.

[0066] In both of FIG. 5 and FIG. 6, the difference between the edge connection structure in case 1 and the edge connection structure in case 2 is whether the edge of interest remains without being deleted. It can be understood that there are two cluster change modes: a non-separation mode (FIG. 5) in which the original cluster does not separate even if the edge of interest is deleted, and a separation mode (FIG. 6) in which the original cluster is separated into two new clusters by deleting the edge of interest. As described above, in the non-separation mode, the cluster separation degree of the edge of interest (edge ​​E2 in FIG. 5) is 0, and in the separation mode (edge ​​E1 in FIG. 6), the cluster separation degree is a finite value in the first direction. Therefore, by acquiring the cluster separation degree for a certain edge of interest, it is possible to determine whether the original cluster will not separate (non-separation mode) or will separate into two clusters by deleting the edge of interest.

[0067] In this embodiment, the cluster separation degree D is defined as follows:

number

number

[0068] As described above, the degree of cluster separation shown in equations [2] and [3] is 0 if the number of clusters does not increase, and takes a finite value (a positive value in the configuration of this embodiment) in the first direction if the number of clusters increases.

[0069] This makes it possible to detect which edge length should be used as the threshold to separate the clusters. As described later, the degree of cluster separation has the property that, when the corresponding pre-separation cluster separates, the closer the node ratios of new cluster 1 and new cluster 2 are, the larger the degree of cluster separation becomes in the first direction (positive direction in the configuration of this embodiment), and further, the larger the number of nodes in the old cluster becomes, the larger the degree of cluster separation becomes in the first direction (positive direction in the configuration of this embodiment). Therefore, based on the magnitude of the degree of cluster separation, it is possible to detect the length threshold at which the original cluster separates into a suitable cluster.

[0070] The formulas [2] and [3] are merely examples of the degree of cluster separation, and a value obtained by multiplying the value defined by the formulas [2] and [3] by a predetermined value may be used as the degree of cluster separation. Note that the degree of cluster separation may be defined using other functions that behave in a similar manner, other than functions such as the formulas [2] and [3].

[0071] When clusters are separated, it is desirable that the value of the cluster separation degree takes a finite value in the first direction (a positive value in the configuration of this embodiment) regardless of how the cluster is divided, but when the cluster is divided into a small cluster and a large cluster, it does not have to take a finite value in the first direction (it may take 0 or a negative value in the configuration of this embodiment). For example, the cluster separation degree may be defined as follows using natural logarithms:

number

[0072] In formula [4], if a cluster with 100 nodes is assumed to be divided into a cluster with 99 nodes and a cluster with 1 node, the degree of cluster separation will be -0.010..., a negative value. If the cluster is divided into two clusters at a ratio other than this ratio, the degree of cluster separation will be a positive value. A cluster with 100 nodes may be divided into a cluster with 99 nodes and a cluster with 1 node when nodes generated by noise or the like are divided into different clusters, and such division is considered to be insignificant. Therefore, the degree of cluster separation defined in formula [4] may be used to appropriately express cluster division.

[0073] In the above description, the first direction is assumed to be the positive direction, but the first direction may be the negative direction. For example, the value obtained by multiplying the right side of equation [4] by a negative number may be used as the degree of cluster separation.

[0074] In this embodiment, the degree of cluster separation may be calculated as a cluster diversity change amount dS, which is a difference in cluster diversity amount S defined by the following formula [5].

number

[0075] Using the cluster diversity S defined as equation [5], for each edge, the cluster diversity when an edge longer than that edge is deleted (case 1) and the cluster diversity when an edge longer than that edge is deleted (case 2) are calculated, and the difference dS is calculated, which is the same as equation [2].

[0076] In the following, in this embodiment, for ease of understanding, an example will be described in which the cluster diversity is calculated in both cases 1 and 2 as the degree of cluster separation, and the cluster diversity change amount dS, which is the difference between the cluster diversity calculations, is calculated. Needless to say, the following example is merely one example, and the degree of cluster separation may be calculated directly without explicitly defining the cluster diversity amount as in formula [5].

[0077] Step SA26 The separation degree calculation unit 2 calculates the above-mentioned cluster diversity change amount dS=S2-S1 as the cluster separation degree, and returns the process to step SA23.

[0078] Step SA27 If it is determined in step SA21 that there is no initial learning result that has not been selected as the selected learning result SR among the multiple initial learning results, the separation degree calculation unit 2 resets the learning results and the edge selection history, and then proceeds to step SA3.

[0079] Here, the cluster diversity change amount dS will be briefly explained. Fig. 7 shows a schematic example of a case where the cluster diversity change amount dS is 0. Fig. 7 shows an example of a cluster having a total of 12 nodes, 3 in the vertical direction and 4 in the horizontal direction, and these 12 nodes are connected in a lattice shape by edges. In addition, the three edges extending horizontally in the center are designated E1 to E3 in order from the bottom, with the length of the edges increasing in the order of edge E1, edge E2, and edge E3.

[0080] In FIG. 7, when the edge of interest is edge E2, the cluster diversity amount S1 is calculated with edge E3 removed, and S1=12·log(12)-12·log(12)=0. Also, when the cluster diversity amount S2 is calculated with edges E2 and E3 removed, S2=12·log(12)-12·log(12)=0. In this case, the cluster diversity change amount dS is S2-S1=0, because the clusters were not separated when the edge of interest is E2. As shown in FIG. 7, when the edge of interest is E2, it can be seen that the clusters are not separated from case 1 to case 2. Therefore, the cluster diversity amount does not change. As described above, the cluster diversity change amount, which is the degree of cluster separation, is 0 when the number of clusters does not change.

[0081] Next, FIG. 8 shows a schematic example of a case where the cluster diversity change amount dS is not 0. The clusters in FIG. 8 are the same as those in FIG. 7. In FIG. 8, when the target side is side E1, if the cluster diversity amount S1 is calculated in a state where sides E2 and E3 are removed, S1=12·log(12)-12·log(12)=0. In contrast, if the cluster diversity amount S2 is calculated in a state where sides E1 to E3 are removed, S2=12·log(12)-6·log(6)-6·log(6)=8.137. In this case, the cluster diversity change amount dS is S2-S1=8.317, which is because the clusters are separated when the target side is E1. As shown in FIG. 8, it can be seen that the clusters are separated from case 1 to case 2 when the target side is E1. Therefore, the cluster diversity amount also changes. As described above, the cluster diversity change amount, which is the degree of cluster separation, becomes a finite value in the first direction (positive direction in the present embodiment) when the number of clusters changes.

[0082] In this way, by calculating the cluster diversity change amount dS for the edge of interest and checking whether the cluster diversity change amount dS is a value other than 0, it is possible to determine whether or not the cluster is to be divided.

[0083] The cluster diversity change amount dS has the property that it is a large value when a cluster is divided evenly and a small value when the cluster is divided unequally. For example, when a cluster with 100 nodes is divided into a cluster with 10 nodes and a cluster with 90 nodes, the cluster diversity change amount dS is 32.5. In contrast, when a cluster with 100 nodes is divided into two clusters with 50 nodes, the cluster diversity change amount dS is 69.3. In this way, it can be understood that even when clusters with the same number of nodes are divided, the cluster diversity change amount dS is larger when the number of divided nodes is equal.

[0084] In addition, the cluster diversity change amount dS has the property that the larger the number of nodes belonging to each cluster, the larger the value, even if the number of clusters after division is the same. If we assume that a cluster with two nodes is divided into two clusters with one node, the number of nodes in the two clusters after division will be the same, but the cluster diversity change amount dS will be a small value of 1.39. In this way, when the number of nodes in the original cluster is small, the cluster diversity change amount dS will be a small value. In this case, it can be interpreted that a small cluster generated by noise or the like has been divided.

[0085] Based on the above properties of the cluster diversity change amount dS, by specifying the length of the edge to be deleted based on the magnitude of the value of the cluster diversity change amount dS, it is possible to preferably detect cases in which the number of nodes is large and the clusters are divided into approximately equal clusters, and to eliminate cases in which the clusters are divided into clusters with an unbalanced number of nodes or cases in which the clusters before division have a small number of nodes.

[0086] Step SA3 The edge deletion processor 3 deletes a predetermined edge from each learning result through the following steps SA31 to SA37.

[0087] Step SA31 The edge deletion processor 3 sets the initial value of the edge deletion level M, which indicates the upper limit of the number of repetitions of the edge deletion process, to "0" (that is, M=0).

[0088] Step SA32 The edge deletion processing unit 3 determines whether there is any initial learning result that has not yet been selected as the selected learning result SR among the multiple initial learning results. If there is no initial learning result that has not yet been selected as the selected learning result SR among the multiple initial learning results, the process proceeds to step SA37.

[0089] Step SA33 If there is an initial learning result that has not been selected as the selected learning result SR, the edge deletion processing unit 3 One of the initial learning results is selected as a selected learning result SR.

[0090] Step SA34 The edge deletion processing unit 3 determines whether there is an Mth largest value among the cluster diversity change amounts dS calculated for each edge of the selection learning result SR. For example, the edge deletion processing unit 3 may sort the calculated cluster diversity change amounts dS in descending order and determine whether there is an Mth cluster diversity change amount dS. If there is no Mth cluster diversity change amount dS, the edge deletion processing unit 3 returns the process to step SA32.

[0091] Step SA35 The edge deletion processing unit 3 selects the edge with the Mth largest value among the cluster diversity change amounts dS calculated for each edge of the selection learning result SR, and reduces the length of the target edge corresponding to the selected cluster diversity change amount dS by a length threshold L TH Note that M=0 is executed so that no edge is deleted. For example, the edge length longer than the length of all edges is determined as a threshold L TH It is determined as follows.

[0092] Step SA36 The edge deletion processing unit 3 calculates the length of each edge included in the selected learning result SR by a length threshold L TH Compared with, the edge length is the length threshold L TH The above edges are deleted from the selected learning result SR, and then the process returns to step SA32.

[0093] As described above, the network structure obtained after the above-mentioned edge deletion is performed on the initial learning result is called an intermediate learning result.

[0094] The effect of the edge deletion process according to the present embodiment will be considered below. FIG. 9 shows an overview of edge deletion in the first embodiment. First, the length threshold L TH1 Based on this, the threshold L TH1 Edges with a length equal to or greater than this are deleted (first edge deletion). This separates the original cluster into two clusters, left and right.

[0095] Next, the length threshold L corresponding to the cluster diversity change amount dS2 next to the maximum cluster diversity change amount dS1 is TH2 is set, and the threshold L TH2 Edges with a length equal to or greater than this are deleted (second edge deletion). This separates the left cluster into two further clusters.

[0096] Next, the length threshold L corresponding to the cluster diversity change amount dS3 that is next largest after the cluster diversity change amount dS2 TH3 is set, and the threshold L TH3 Edges with a length equal to or greater than this are deleted (third edge deletion). This separates the right cluster into two more clusters.

[0097] Here, an example of edge deletion according to this embodiment will be described. Fig. 10 shows an example of input data used for edge deletion in embodiment 1. The input data in this embodiment is distributed in six clusters, of which there are areas where three clusters are close to each other and areas where two clusters are close to each other, with the boundaries being unclear due to noise.

[0098] Figure 11 shows the network structure after edge deletion processing is performed on the input data in Figure 10. Before edge deletion, nodes corresponding to adjacent clusters are connected by edges, and it can be seen that the clusters cannot be separated. Furthermore, a cluster in an area where three clusters are close to each other and a cluster in an area where two adjacent clusters are close to each other are connected by a single edge. In contrast, after edge deletion, edges caused by noise have been properly deleted, and it can be seen that the six clusters have been separated.

[0099] As described above, according to this configuration, by deleting edges in stages, it is possible to perform more appropriate edge deletion processing for each cluster to which the edge of interest belongs. This makes it possible to suppress unnecessary edge deletion processing and to learn the distribution of input vectors more accurately.

[0100] Step SA37 If it is determined in step SA32 that there is no initial learning result that has not yet been selected as the selected learning result SR among the multiple initial learning results or there is no initial learning result that has not yet been selected as the selected learning result SR among the multiple learning results, the selection history of the initial learning results performed by the edge deletion processing unit 3 is reset. Then, the process proceeds to step SA4.

[0101] Step SA4 Distance matrix calculation unit 4 selects a pair of two clusters from the pair of intermediate learning results in which the two edges have been deleted, and calculates a distance matrix for the two selected clusters, in the following steps SA41 to SA45.

[0102] Step SA41 The distance matrix calculation unit 4 determines whether there is any pair of two intermediate learning results that has not yet been selected as a processing target, among the pairs of two intermediate learning results that have been arbitrarily selected from the intermediate learning results. If there is no pair of intermediate learning results that has not yet been selected, the process proceeds to step SA8.

[0103] Step SA42 If there are any unselected intermediate learning result pairs, the distance matrix calculation unit 4 selects one intermediate learning result pair from among them. Hereinafter, the intermediate learning result pair selected here is referred to as a selected learning result pair, and the intermediate learning results included in the selected learning result pair are SA and SB.

[0104] Step SA43 The distance matrix calculation unit 4 determines whether there is any pair of clusters that has not yet been selected as a processing target among pairs of clusters that are composed of one cluster arbitrarily selected from the intermediate learning result SA of the selected learning result pair and one cluster arbitrarily selected from the intermediate learning result SB. If there is no pair of clusters that has not yet been selected, the process proceeds to step SA6.

[0105] Step SA44 If there are unselected cluster pairs, the distance matrix calculation unit 4 selects one cluster pair from among them. Specifically, the distance matrix calculation unit 4 selects one cluster pair including a cluster A arbitrarily selected from p clusters CA1 to CAp (p is an integer equal to or greater than 1) included in the intermediate learning result SA from which edges have been deleted, and a cluster B arbitrarily selected from q clusters CB1 to CBq (q is an integer equal to or greater than 1) included in the intermediate learning result SB from which edges have been deleted. Hereinafter, the cluster pair selected here is referred to as a selected cluster pair.

[0106] Step SA45 The distance matrix calculation unit 4 calculates the distance d between all pairs of nodes a1 to am (ai, 1≦i≦m) included in cluster A and nodes b1 to bn (bj, 1≦j≦n) included in cluster B. aibj Then, the distance matrix D of the selected cluster pairs is calculated.

number

[0107] Step SA5 After step SA45, the evaluation index calculation unit 5 calculates an evaluation index indicating the degree of similarity between the two clusters of the selected cluster pair through the following steps SA51 to SA54.

[0108] Step SA51 The evaluation index calculation unit 5 calculates each row (d aib1 ~d aibn ), the minimum value min ai (i.e., min a1 ~min am ) and each column of the distance matrix D (d a1bj ~d ambj ), the minimum value minb j (i.e., min b1 ~min bn ) to get the

[0109] Step SA52 The evaluation index calculation unit 5 calculates the average distance d between nodes belonging to the same cluster. mean Calculate d mean is the average distance between nodes that are identified as being in the same cluster. If the distance is greater than this average, the nodes are identified as different, and if it is shorter than this average, the nodes are identified as being in the same cluster. mean For example, it can be written as follows:

number

number

[0110] Also, d mean may be set to the larger or smaller value of the values ​​shown in the following equations [9] and

[10] .

number

number

[0111] In addition, the minimum value of each row, min a1 ~min am and the minimum value of each column, min b1 ~min bn and separate d mean You can use the minimum value of each row, min a1 ~min am For example, as shown in equation

[11] , d calculated only for cluster A mean You can also use the minimum value of each column, min b1 ~min bn For example, as shown in equation

[12] , d calculated only for cluster B is used for mean may also be used.

number

number

[0112] In the above, we have described a network that includes multiple nodes. However, when the input data is trained using SOINN or the like, the network will include multiple nodes and multiple edges connecting two nodes. In this way, when edges exist between nodes, the average length of edges belonging to the same cluster is calculated as d mean It is desirable to use it as d. Since edges represent the relationship between nodes and the edge length represents the similarity between them, by using edges, the distance between nodes that happen to exist nearby due to the influence of noise can be calculated as d. meanIt is possible to prevent the average length from being used in the calculation of distance between neighboring nodes. In this embodiment, the arithmetic mean (arithmetic average) is used as the average length, but the present invention is not limited to this as long as it can numerically indicate the central tendency of the distance distribution between neighboring nodes. For example, another average index such as the square mean, the geometric mean, or the harmonic mean may be used as the average length, or another statistical representative value such as the median or the mode may be used. In the following, the term "average distance" is used to include the average index and statistical representative value described above in addition to the arithmetic mean.

[0113] Step SA53 The evaluation index calculation unit 5 calculates the minimum value min a1 ~min am The average distance d mean Count the number of items equal to or greater than NUMa, and find the minimum value min b1 ~min bn The average distance d mean Count the number of items above NUMb.

[0114] Step SA54 The evaluation index calculation unit 5 divides the sum of the acquired numbers NUMa and NUMb by the total number of nodes (m+n) in clusters A and B to calculate an evaluation index ES.

number

[0115] Here, the average value d mean By counting the number of the above, we can detect the number of dissimilar nodes between the two clusters. In addition, by dividing the number of dissimilar nodes by the total number of nodes in the two clusters in formula

[13] , we calculate the ratio of the number of dissimilar nodes to the total number of nodes. In other words, the evaluation index ES can be said to be a value that represents the dissimilarity; a large value indicates that the two clusters are dissimilar, and a small value indicates that the two clusters are similar.

[0116] In this embodiment, the evaluation index calculation unit 5 executes this for all pairs of any one of the p clusters CA1 to CAp included in the intermediate learning result SA and any one of the q clusters CB1 to CBq included in the intermediate learning result SB. Then, the evaluation index calculation unit 5 calculates the evaluation index for all pairs of two clusters.

[0117] Step SA6: Evaluation index matrix generation If it is determined in step SA43 that there is no pair of clusters that has not yet been selected as a processing target among the selected learning result pairs, evaluation index calculation unit 5 generates an evaluation index matrix EM with p rows and q columns. That is, the row numbers of the evaluation index matrix EM indicate the numbers of clusters included in the intermediate learning result SA, and the column numbers indicate the numbers of clusters included in the intermediate learning result SB.

number

[0118] Step SA7: Score calculation The score calculation unit 6 compares each evaluation index included in the evaluation index matrix EM with a threshold TH, and assigns a high score (e.g., score +1) to both clusters that constitute evaluation indexes ES that are equal to or less than the threshold TH (similar), and assigns a low score (e.g., score 0) to both clusters that constitute evaluation indexes ES that are greater than the threshold (dissimilar). Then, the total score of each cluster (CA1 to CAp) of the intermediate learning result SA is calculated. In other words, the total score of each row (ES i1 ,…,SE iq Calculate the sum of the points given to each row. (ES i1 ,…,ES iq The sum of the scores of the clusters CAi and CAi is the score of the cluster CAi. In addition, the sum of the scores of each cluster (CB1 to CBq) of the intermediate learning result SB is calculated. In other words, each column (ES 1j ,…,SE pj Calculate the sum of the points given to each column. (ES 1j ,…,SE pjThe sum of the scores of all the clusters CBj is the score of the cluster CBj. The method of calculating the score is not limited to this, and it is also possible to add a high score to the cluster with the lowest evaluation index ES (the most similar) in each row that is equal to or less than the threshold TH, or the cluster with the lowest evaluation index ES (the most similar) in each column.

[0119] In this embodiment, the above total score is calculated for all pairs of any two intermediate learning results selected from the multiple intermediate learning results. That is, the distance matrix calculation unit 4 and the evaluation index calculation unit 5 perform the above-mentioned distance matrix calculation, evaluation index calculation, and evaluation index matrix generation for all pairs of any two intermediate learning results selected from the multiple intermediate learning results. Then, the score calculation unit 6 obtains the total score for each cluster of each intermediate learning result based on these calculation results. After that, the process returns to step SA41.

[0120] Step SA8 If it is determined in step SA41 that there is no pair of two intermediate learning results that is not yet selected as a processing target among the pairs of two selectable intermediate learning results from the multiple intermediate learning results, the score calculation unit 6 generates a score matrix. Specifically, based on the acquired total score, the score calculation unit 6 generates a score matrix SM with r columns in which the row numbers are the cluster numbers, as shown in the following formula

[15] . Here, the element S of the matrix SM ij is the total score for the jth intermediate learning result (j is an integer between 1 and M). Specifically, it is the sum of all the total scores for cluster i of the jth intermediate learning result calculated by the score calculation unit 6 for all pairs of the jth intermediate learning result and all other intermediate learning results. Also, there are elements that do not exist in the matrix SM. For example, if cluster i of the jth intermediate learning result does not exist, S ij is empty and does not participate in calculations.

number

[0121] Step SA9 The pixel number calculation unit 7 calculates the pixel number by the following steps SA91 and SA92.

[0122] Step SA91 The score calculation unit 7 refers to the score matrix SM and calculates the minimum score S for each intermediate learning result. m1 ~S mr That is, the image score calculation unit 7 obtains the minimum score S m1 ~S mr Get the.

[0123] Step SA92 The image score calculation unit 8 calculates the minimum score S m1 ~S mr Based on this, the number of strokes UN is calculated. The number of strokes UN is calculated, for example, using the following formula. Note that N is the number of all learning results.

number

[16] , the minimum score S m1 ~S mr In other words, the pixel number UN is determined from the training results whose clusters are all similar to the clusters of the other training results.

[0124] A large value of the stroke-count UN means that in at least one training result, all clusters belonging to that training result have high scores, i.e., the clusters are very similar to the clusters of other training results. A small value of the stroke-count UN means that in every training result, at least one cluster has low scores, i.e., the clusters are not similar to the clusters of other training results.

[0125] The stroke number UN tends to decrease as the value of the edge deletion level M increases, i.e., as the number of times the edge deletion process is repeated increases, the edge deletion progresses and the clusters become more diverse. In other words, if the stroke number UN becomes smaller than a predetermined value by increasing the number of times the edge deletion process is repeated, it can be understood that the clusters have become more diverse and that dissimilar clusters have existed in each learning result. Therefore, the stroke number UN can be used to determine the timing to stop the repetition of the edge deletion process.

[0126] Fig. 12 shows an example of intermediate learning results and pixel UN obtained when two-stage edge deletion processing is performed. In Fig. 12, similar clusters are surrounded by dashed lines. As can be seen from Fig. 12, after the first stage of edge deletion processing, clusters that depend on the input data structure are generated by the edge deletion, but after the second stage of edge deletion processing, noise-like clusters that depend on the method and parameters of each learning result are generated. The results after the first stage of edge deletion processing are suitable for clustering, but the results after the second stage of edge deletion processing are not suitable for clustering.

[0127] In this example, the stroke count UN is the smallest score (S 01 ~S 04 ) is 3, which is 100%, and after the first edge deletion process, the minimum score in learning results 1 to 3 (S 11 ~S 13 ) is 2, which is 75%, and after the second edge deletion process, the minimum score (S 01 ~S 04 ) and becomes 0 and 25% at the same time. UN By setting it to 40%, the edge removal process can be stopped after the second edge removal process, when the clusters are deemed to be properly separated or close to being properly separated.

[0128] Step SB1 The end determination unit 8 sets the image number UN to a predetermined image number threshold TH UN Compare with.

[0129] Step SB2 The threshold value TH is UN If UN≧TH UN ), the end decision unit 8 increments the edge deletion level M by 1 (M=M+1), and returns the process to step SA32.

[0130] Step SB3 The threshold value TH is UN If it is smaller than (UN <TH UN ), the learning result selection unit 9 selects the final learning result based on the following steps SB31 to SB33.

[0131] Step SB31 The learning result selection unit 9 selects a group consisting of a plurality of intermediate learning results in which edges have been deleted at edge deletion level M-1. Which group to select may be determined appropriately depending on the application. Note that, although the learning result selection unit 9 has been described here as selecting a group consisting of a plurality of intermediate learning results in which edges have been deleted at edge deletion level M-1, it may also select a group consisting of a plurality of intermediate learning results in which edges have been deleted at edge deletion level M, if necessary.

[0132] Step SB32 The learning result selection unit 9 refers to the score matrix SM and obtains the minimum scores Smin1 to Sminr for each of the intermediate learning results.

[0133] Step SB33 The learning result selection unit 9 selects the intermediate learning result with the maximum value among the acquired minimum scores Smin1 to Sminr as the final learning result to be adopted. In other words, the learning result selection unit 9 extracts, for each intermediate learning result, the score of the cluster that is most dissimilar to the clusters included in the other intermediate learning results, and then selects the intermediate learning result having the most similar cluster from among them as the final learning result.

[0134] Step SB4 The clustering unit 10 performs class classification by referring to the nodes and edges of the selected final learning result. In the class classification, for example, processing such as assigning cluster numbers or labeling to the clusters included in the final learning result is performed.

[0135] As described above, according to this configuration, it is possible to repeat the edge deletion process an appropriate number of times by automatically determining the edge deletion level M. This makes it possible to accurately learn the structure of the input data by automatically performing the edge deletion process.

[0136] Other embodiments The present invention is not limited to the above embodiment, and can be modified as appropriate without departing from the spirit of the present invention. For example, as for the distance measure indicating the distance between nodes, any distance measure such as Euclidean distance, cosine distance, Manhattan distance, and fractional distance can be applied as the distance measure between nodes. In general, Euclidean distance is used when the input data is low-dimensional, and cosine distance, Manhattan distance, and fractional distance are used when the input data is high-dimensional.

[0137] In the above embodiment, the judgment of the magnitude of two values ​​has been described, but this is merely an example, and the case where the two values ​​are equal in the judgment of the magnitude of two values ​​may be handled as necessary. That is, for the judgment of whether the first value is equal to or greater than the second value or smaller than the second value, either may be adopted as necessary. For the judgment of whether the first value is equal to or less than the second value or larger than the second value, either may be adopted. In other words, when the magnitude judgment of two values ​​is performed to obtain two judgment results, the case where the two values ​​are equal may be included in either of the two judgment results as necessary.

[0138] In addition, in SOINN, the process of connecting edges is performed in the learning of input vectors. In this case, the length threshold L TH It is also possible to perform the same process as the edge deletion process described above by connecting only the following edges. That is, in a network, TH It is necessary to obtain a network in which only edges of the following length are connected to the nodes. The selection of edges connected to the nodes may be performed by the edge deletion process described above, or may be performed in the process of connecting edges performed in the process of learning input vectors in SOINN.

[0139] In the above-mentioned embodiment, the present invention has been described mainly as a hardware configuration, but the present invention is not limited to this, and any processing can be realized by making a CPU (Central Processing Unit) execute a computer program. In this case, the computer program can be stored using various types of non-transitory computer readable media and supplied to the computer. The non-transitory computer readable media includes various types of tangible storage media. Examples of the non-transitory computer readable media include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / Ws, and semiconductor memories (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs (random access memories)). The program may also be supplied to the computer by various types of transitory computer readable media. Examples of the transitory computer readable media include electrical signals, optical signals, and electromagnetic waves. The temporary computer-readable medium can supply the program to the computer via a wired communication path such as an electric wire or an optical fiber, or via a wireless communication path. [Explanation of symbols]

[0140] 1. Learning Processing Unit 2. Separation degree calculation section 3 Edge deletion processor 4 Distance matrix calculation section 5. Evaluation index calculation section 6. Score Calculation Section 7 stroke calculation section 8 End Judgment Section 9. Learning result selection section 10 Clustering Department 1000 Computers 1001 CPU 1002 ROM 1003 RAM 1004 Bus 1005 Input / Output Interface 1006 Input section 1007 Output section 1008 Storage section 1009 Communications Department 1010 Drive 1011 Magnetic disk 1012 Optical disc 1013 Flexible Disk 1014 Semiconductor memory 100 Information processing device

Claims

1. An information processing device that sequentially inputs an input vector and learns an input distribution structure of the input vector as a network structure in which a plurality of nodes described by a multidimensional vector and a plurality of edges connecting two of the nodes are arranged, a learning processing unit that acquires a plurality of initial learning results obtained by learning the same input data using a plurality of unsupervised learning methods; a separation degree calculation unit which sequentially selects one initial learning result from the plurality of initial learning results, sequentially selects each of a plurality of edges included in the selected initial learning result as an edge of interest, and calculates a degree of separation of clusters in a case where an edge longer than the edge of interest is deleted from the selected initial learning result and in a case where an edge having a length equal to or longer than the edge of interest is deleted from the selected initial learning result; an edge deletion processing unit that deletes edges having a length equal to or greater than a length threshold, the length of the edge of interest corresponding to the Mth largest degree of cluster separation among the degrees of cluster separation calculated for the plurality of edges, from the selected initial learning result; a distance matrix calculation unit that selects a learning result pair consisting of two selectable intermediate learning results from a plurality of intermediate learning results that are the plurality of initial learning results after the edge deletion by the edge deletion processing unit, selects a cluster pair consisting of two clusters selected one from each of the two intermediate learning results of the selected learning result pair, and calculates a distance matrix indicating the distance between a node belonging to one cluster and a node belonging to the other cluster for all of the selectable cluster pairs; an evaluation index calculation unit that obtains the minimum value of each column and each row for each distance matrix, and calculates an evaluation index for each distance matrix based on the number of the obtained minimum values ​​that satisfy a predetermined condition; a score calculation unit that calculates, for each cluster of each intermediate learning result, a score indicating a degree of similarity between one cluster included in one intermediate learning result and a cluster of another intermediate learning result, based on an evaluation index of the distance matrix calculated for all the learning result pairs; a stroke count calculation unit that obtains a minimum value of the scores calculated for each cluster of each intermediate learning result, and calculates a maximum value of the obtained minimum values ​​as a stroke count; a termination determination unit that adds 1 to M to update the value of M when the pixel count is greater than a pixel count threshold, and terminates the process of deleting edges when the pixel count is less than the pixel count threshold; a learning result selection unit that selects a final learning result from the plurality of intermediate learning results based on the score of each cluster of each intermediate learning result calculated by the score calculation unit when the termination determination unit determines that the process of deleting edges is terminated; A clustering unit that clusters the final learning result, Each time the value of M is updated, the edge deletion processing unit, the distance matrix calculation unit, the score calculation unit, the pixel number calculation unit, and the end determination unit repeat the process. Information processing device.

2. The degree of separation of the clusters is 0 when the cluster to which the edge of interest belongs is not separated into two clusters after the edge is deleted, and has a value other than 0 when the cluster is separated. The information processing device according to claim 1 .

3. The degree of separation of the clusters has a positive or negative value when the cluster to which the edge of interest belongs is separated into two clusters. The information processing device according to claim 2 .

4. When the cluster to which the edge of interest belongs is separated into two clusters, the closer the numbers of nodes in the two separated clusters are, the larger the degree of separation of the clusters becomes if the degree of separation is a positive value, and the smaller the degree of separation becomes if the degree of separation is a negative value. The information processing device according to claim 3 .

5. When the cluster to which the edge of interest belongs is separated into two clusters, the greater the number of nodes contained in the two separated clusters, the larger the degree of separation of the clusters becomes if the value is positive, and the smaller the value becomes if the value is negative.

5. The information processing device according to claim 3.

6. n ｋ１ The number of nodes in cluster k1 that exists when edges longer than the edge in question are deleted, n ｋ２ is the number of nodes in cluster k2 that exists when an edge longer than the edge of interest is deleted. The degree of separation D of the clusters is defined by the following formula: [0010] The information processing device according to claim 2 .

7. The score calculation unit: generating an evaluation index matrix composed of evaluation indexes for all pairs between two clusters selected one by one from each of the two intermediate learning results based on the evaluation indexes calculated for all distance matrices; calculating a score indicating whether a cluster included in one learning result is similar to a cluster of another learning result based on the evaluation indexes included in the evaluation index matrix; Generate a score matrix including scores for the clusters belonging to all the learning results; the learning result selection unit selects a final learning result from the plurality of intermediate learning results based on the score matrix. The information processing device according to claim 1 .

8. The evaluation index calculation unit is calculating an average distance between nodes belonging to each of the two clusters, and calculating an average value of the two calculated average distances; Calculating an evaluation index for each distance matrix based on the number of minimum values ​​equal to or greater than the average value among the minimum values ​​in each column and each row of each distance matrix; The information processing device according to claim 7.

9. The evaluation index is a value obtained by dividing the number of the minimum values ​​in each column and each row of each distance matrix that are equal to or greater than the average value by the total number of nodes belonging to the two clusters. The information processing device according to claim 8.

10. the score calculation unit compares each evaluation index included in the evaluation index matrix with a threshold, assigns a score based on a comparison result, and calculates a total score for each cluster.

9. The information processing device according to claim 7 or 8.

11. the learning result selection unit obtains a minimum value among the total scores of clusters belonging to each intermediate learning result, and selects the intermediate learning result corresponding to the maximum value among the obtained minimum values. The information processing device according to claim 10.

12. 1. An information processing method executed by a computer, comprising: inputting an input vector in sequence; and learning an input distribution structure of the input vector as a network structure in which a plurality of nodes described by a multidimensional vector and a plurality of edges connecting two of the nodes are arranged, the method comprising: The computer, Obtaining a plurality of initial learning results obtained by learning the same input data using a plurality of unsupervised learning methods; selecting one initial learning result from the plurality of initial learning results in order, and selecting one cluster from the selected one initial learning result in order; selecting each of a plurality of edges included in the selected cluster as an edge of interest in turn, and calculating the degree of separation of the clusters when an edge longer than the edge of interest is deleted from the network structure and when an edge longer than or equal to the edge of interest is deleted from the network structure; based on the calculated cluster separation degree for each edge of interest, delete edges having a length equal to or greater than a length threshold, the length of the edge of interest corresponding to the Mth largest cluster separation degree, from the selected cluster; selecting a learning result pair consisting of two selectable intermediate learning results from the intermediate learning results that are the initial learning results after the edge deletion, selecting a cluster pair consisting of two clusters, one selected from each of the two selected intermediate learning results, and calculating a distance matrix indicating the distance between a node belonging to one cluster and a node belonging to the other cluster for all of the selectable cluster pairs; obtaining a minimum value for each column and row of each distance matrix, and calculating an evaluation index for each distance matrix based on the number of the obtained minimum values ​​that satisfy a predetermined condition; calculating, for each cluster in each intermediate learning result, a score indicating a degree of similarity between one cluster included in one intermediate learning result and a cluster in another intermediate learning result, based on the evaluation index of the distance matrix calculated for all of the learning result pairs; A minimum value of the scores calculated for each cluster of each intermediate learning result is obtained, and a maximum value of the obtained minimum values ​​is calculated as a pixel count; if the pixel count is greater than a pixel count threshold, add 1 to M to update the value of M, and if the pixel count is less than a pixel count threshold, end the process of deleting edges; When it is determined that the process of deleting edges is to be terminated, a final learning result is selected from the multiple intermediate learning results for the previous period based on the calculated scores of each cluster of each intermediate learning result; Clustering the final learning result; Each time the value of M is updated, the deletion of the edge, the calculation of the distance matrix, the calculation of the score, the calculation of the pixel number, and the determination of the end are repeated. Information processing methods.

13. A program for causing a computer to perform a process of sequentially inputting an input vector and learning an input distribution structure of the input vector as a network structure in which a plurality of nodes described by a multidimensional vector and a plurality of edges connecting two of the nodes are arranged, the program comprising: A process of acquiring a plurality of initial learning results obtained by learning the same input data using a plurality of unsupervised learning methods; a process of sequentially selecting one initial learning result from the plurality of initial learning results and sequentially selecting one cluster from the selected one initial learning result; a process of sequentially selecting each of a plurality of edges included in the selected cluster as an edge of interest, and calculating a degree of separation of the cluster when an edge longer than the edge of interest is deleted from the network structure, and when an edge longer than the edge of interest is deleted from the network structure; a process of deleting edges from the selected cluster, the edges having a length equal to or greater than a length threshold, the length being the edge of interest corresponding to the Mth largest degree of cluster separation, based on the degree of cluster separation calculated for each edge of interest; a process of selecting a learning result pair consisting of two selectable intermediate learning results from a plurality of intermediate learning results that are the plurality of initial learning results after edge deletion, selecting a cluster pair consisting of two clusters selected one from each of the two selected intermediate learning results, and calculating a distance matrix indicating the distance between a node belonging to one cluster and a node belonging to the other cluster for all of the selectable cluster pairs; A process of obtaining the minimum value of each column and each row for each distance matrix, and calculating an evaluation index for each distance matrix based on the number of the obtained minimum values ​​that satisfy a predetermined condition; a process of calculating, for each cluster of each intermediate learning result, a score indicating a degree of similarity between one cluster included in one intermediate learning result and a cluster of another intermediate learning result, based on an evaluation index of the distance matrix calculated for all the learning result pairs; A process of obtaining a minimum value of the scores calculated for each cluster of each intermediate learning result, and calculating a maximum value of the obtained minimum values ​​as a pixel count; updating the value of M by adding 1 if the pixel count is greater than a pixel count threshold, and terminating the process of deleting edges if the pixel count is less than the pixel count threshold; a process of selecting a final learning result from the multiple intermediate learning results in the previous period based on the calculated scores of each cluster of each intermediate learning result when it is determined that the process of deleting edges is to be terminated; A process of clustering the final learning result. Each time the value of M is updated, the process of deleting the edge, the process of calculating the distance matrix, the process of calculating the score, the process of calculating the pixel count, and the process of determining whether the edge has been deleted are repeated. program.

Images