Information processing device, information processing method, and program
By generating persistent information and using Kullback-Leibler information for Bayesian optimization, the noise and "curse" effect problems in high-dimensional data reduction are solved, and efficient and robust dimensionality reduction effects are achieved.
Patent Information
- Application Number
- JP2023156459
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-09-21
- Publication Date
- 2025-04-03
AI Technical Summary
Existing high-dimensional data dimensionality reduction methods, such as t-SNE, are susceptible to the "curse" effect of high-dimensional data, and have difficulties in practical applications under the assumption of non-uniform distribution, making it difficult to effectively reduce the impact of noise.
By performing filtering processing, persistent information is generated, eigenvectors are extracted, and the Kullback-Leibler information is used as the objective function for Bayesian optimization, and the optimal dimensionality reduction matrix is calculated, thereby performing dimensionality reduction processing.
This method can effectively reduce the noise impact of high-dimensional data in the dimensionality reduction process, avoid the negative impact of the "curse" effect of high-dimensional data, and achieve practical dimensionality reduction without assuming the uniform distribution of data.
Smart Images
Figure 2025047765000001_ABST
Abstract
Description
[Technical field]
[0001] The present disclosure relates to an information processing device, an information processing method, and a program for performing dimensionality reduction. [Background technology]
[0002] In order to visualize high-dimensional data such as images and audio and understand the characteristics of the data, a method (dimension reduction) is known in which data is converted into a dimension (low-dimensional data) that can be understood by humans while retaining as much important information as possible. Non-Patent Document 1 is known as a related technology.
[0003] Non-Patent Document 1 discloses t-distributed Stochastic Neighbor Embedding (t-SNE) as a dimensionality reduction method. In Non-Patent Document 1, t-SNE is used to compress high-dimensional data into low dimensions and visualize features. [Prior art documents] [Non-patent literature]
[0004] [Non-Patent Document 1] Laurens van der Maaten, Geoffrey Hinton, “Visualizing data using t-SNE”, [online], Submitted 5 / 08; Revised 9 / 08; Published 11 / 08 2008, Journal of machine learning research 9(2008) 2579-2605, [Retrieved July 19, 2023], Internet<URL:https: / / www.jmlr.org / papers / volume9 / vandermaaten08a / vandermaaten08a.pdf> Summary of the Invention [Problem to be solved by the invention]
[0005] However, since t-SNE calculates the two-point similarity for all combinations of two points in high-dimensional data, it is susceptible to the effects of noise in high-dimensional data (the curse of dimensionality).
[0006] One example of the objective of the present disclosure is to reduce the influence of noise when reducing the dimension of high-dimensional data. [Means for solving the problem]
[0007] In order to achieve the above object, an information processing device according to one aspect of the present disclosure includes: a persistent information generating unit that performs a filtering process on high-dimensional learning data to generate first persistent information representing a persistent diagram; a feature vector extraction unit that performs vector conversion processing on each of the generated first persistent information to generate a first feature vector; an objective function generation unit that generates a Kullback-Leibler divergence representing the objective function using the generated first feature vector and a dimensionality reduction matrix representing variables of the objective function; an optimal solution calculation unit that performs a Bayesian optimization process on the generated objective function to calculate an optimal solution of the dimensionality reduction matrix in which the Kullback-Leibler divergence is minimized; a dimension reduction unit that reduces the dimension of the first feature vector by using the calculated optimal solution of the dimension reduction matrix; The present invention is characterized by having the following.
[0008] In order to achieve the above object, an information processing method according to one aspect of the present disclosure includes: An information processing device, Filtration is performed on the high-dimensional learning data to generate first persistence information representing a persistence diagram; performing a vector conversion process on each of the generated first persistent information to generate a first feature vector; generating a Kullback-Leibler divergence representing the objective function using the generated first feature vector and a dimensionality reduction matrix representing variables of the objective function; A Bayesian optimization process is performed on the generated objective function to calculate an optimal solution of the dimensionality reduction matrix that minimizes the Kullback-Leibler divergence; reducing the dimension of the first feature vector using the calculated optimal solution of the dimension reduction matrix; It is characterized by:
[0009] Furthermore, in order to achieve the above object, a program according to one aspect of the present disclosure includes: On the computer, Filtration is performed on the high-dimensional learning data to generate first persistence information representing a persistence diagram; performing a vector conversion process on each of the generated first persistent information to generate a first feature vector; generating a Kullback-Leibler divergence representing the objective function using the generated first feature vector and a dimensionality reduction matrix representing variables of the objective function; A Bayesian optimization process is performed on the generated objective function to calculate an optimal solution of the dimensionality reduction matrix that minimizes the Kullback-Leibler divergence; reducing the dimension of the first feature vector using the calculated optimal solution of the dimension reduction matrix; The present invention is characterized in that it causes the processing to be executed. Effect of the Invention
[0010] As described above, according to the present disclosure, it is possible to reduce the influence of noise in reducing the dimension of high-dimensional data. [Brief description of the drawings]
[0011] [Figure 1] FIG. 1 is a diagram illustrating an example of an information processing device according to the first embodiment. [Diagram 2]FIG. 2 is a diagram illustrating an example of a system including the information processing device according to the first embodiment. [Diagram 3] FIG. 3 is a diagram for explaining an example of the filtration process. [Figure 4] FIG. 4 is a diagram for explaining an example of the operation of the information processing device in the first embodiment. [Diagram 5] FIG. 5 is a diagram for explaining the quantitative evaluation results for each dimension reduction method. [Figure 6] FIG. 6 is a diagram for explaining the quantitative evaluation results (scatter plots and probability distributions) for each dimensionality reduction method. [Figure 7] FIG. 7 is a diagram illustrating an example of a system including an information processing device according to the second embodiment. [Figure 8] FIG. 8 is a diagram for explaining an example of the operation of the information processing device in the second embodiment. [Figure 9] FIG. 9 is a diagram illustrating an example of a computer that realizes the information processing device according to the first and second embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0012] First, an overview will be given to facilitate understanding of the embodiments described below. Even if a human tries to grasp the distribution of high-dimensional data (data with tens to a hundred or more dimensions), it is difficult to grasp the data distribution if the data remains high-dimensional.
[0013] It is known that high-dimensional data can be reduced in dimension to two-dimensional or three-dimensional data, and the two-dimensional or three-dimensional data can then be represented as a scatter plot to make it easier for humans to understand the data distribution. Examples of dimensionality reduction methods include PCA (Principal Component Analysis), t-SNE, and UMAP (Uniform Manifold Approximation and Projection for Dimension Reduction). For more information on UMAP, see reference [1].
[0014] References [1]: Leland McInnes, John Healy, James Melville, “UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction”, [online], September 21, 2020, [Retrieved July 19, 2023], Internet<URL:https: / / arxiv.org / abs / 1802.03426>
[0015] In addition, t-SNE, UMAP, etc. are used, for example, in medical image diagnosis. For a specific example of medical image diagnosis, please refer to reference [2].
[0016] References [2]: Mohammad Tariqul Islam, Jason W. Fleischer, “Distinguishing L and H phenotypes of COVID-19 using a single x-ray image”, [online], 03 May 2020, [Retrieved July 19, 2023], Internet<URL:https: / / www.medrxiv.org / content / 10.1101 / 2020.04.27.20081984v1>
[0017] However, as mentioned above, t-SNE calculates the two-point similarity for all combinations of two points in high-dimensional data, so it is susceptible to the effects of noise in high-dimensional data (the curse of dimensionality).
[0018] In addition, unlike t-SNE, UMAP can calculate similarity only from data within the k neighborhood of each point, but it assumes uniform distribution of data, and there are few practical cases where this assumption holds true. Also, it is difficult to determine an appropriate k.
[0019] In addition, t-SNE and UMAP do not use linear transformation and individually optimize the coordinates of each point after projection, so online learning is not possible. For example, when mapping new incremental data to a scatter plot, all calculations must be redone, making it difficult to instantly map new incremental data to a scatter plot.
[0020] Through this process, the inventors have discovered the above-mentioned problems in dimensionality reduction of high-dimensional data, and have also derived means for solving the problems.
[0021] That is, the inventor has derived a dimensionality reduction method (not using all two-point combinations) that is not susceptible to the influence of high-dimensional data (the curse of dimensionality) like (i) t-SNE. As a result, even if the dimension of high-dimensional data is reduced, it is not susceptible to the influence of noise.
[0022] In addition, the inventors have derived (ii) a method for realistic dimensionality reduction that does not require the assumption of uniform distribution, as in UMAP. In other words, by newly defining persistent density entropy, which uses the concept of topological data analysis as the amount of information, the assumption of uniform distribution is no longer necessary.
[0023] Furthermore, the inventors (iii) obtained a projection matrix (a matrix used in dimensionality reduction) that minimizes the Kullback-Leibler divergence (the difference between the probability distributions before and after dimensionality reduction) by Bayesian optimization (linear transformation), and derived a means for reducing the dimensionality using the projection matrix. As a result, it became possible to generate a scatter plot of the reduced-dimensional data (two-dimensional data or three-dimensional data).
[0024] Hereinafter, the first embodiment will be described with reference to the drawings. In the drawings described below, elements having the same or corresponding functions are denoted by the same reference numerals, and repeated description thereof may be omitted.
[0025] (Embodiment 1) The configuration of an information processing device in embodiment 1 will be described with reference to Fig. 1. Fig. 1 is a diagram for explaining an example of an information processing device in embodiment 1. In embodiment 1, learning for generating an optimal dimensionality reduction matrix using learning data (dataset) will be described.
[0026] [Device configuration] The information processing device shown in Fig. 1 is a device that makes it difficult to be affected by noise at least in the dimension reduction of high-dimensional data. Also, as shown in Fig. 1, the information processing device 10 has a persistent information generating unit 11, a feature vector extracting unit 12, an objective function generating unit 13, an optimal solution calculating unit 14, and a dimension reducing unit 15.
[0027] The persistent information generating unit 11 performs a filtration process on high-dimensional learning data (high-dimensional data) to generate persistent information (first persistent information) representing a persistent diagram.
[0028] The feature vector extraction unit 12 performs vector conversion processing on each piece of generated persistent information to generate a feature vector (first feature vector: high-dimensional data).
[0029] The objective function generating unit 13 generates a Kullback-Leibler divergence representing the objective function using the generated feature vector and a dimensionality reduction matrix representing the variables of the objective function.
[0030] The optimal solution calculation unit 14 executes a Bayesian optimization process on the generated objective function, and calculates an optimal solution of the dimension reduction matrix that minimizes the Kullback-Leibler divergence.
[0031] The dimension reduction unit 15 reduces the dimension of the feature vector using the calculated optimal solution of the dimension reduction matrix, and generates low-dimensional data (first low-dimensional data).
[0032] In this manner, in the first embodiment, (i) unlike t-SNE, since the distances between all two points are not used, it is possible to perform dimensionality reduction that is less susceptible to the influence of high-dimensional data (the curse of dimensionality).
[0033] In addition, (ii) since the persistent density entropy is defined using the concept of topological data analysis as the amount of information, it is not necessary to assume a uniform distribution as in UMAP, and realistic dimensionality reduction is possible.
[0034] Furthermore, (iii) an optimal solution (projection matrix used in dimensionality reduction) that minimizes the Kullback-Leibler divergence (difference in probability distribution before and after dimensionality reduction) is obtained by Bayesian optimization (linear transformation). Based on the finally obtained projection matrix (linear projection), the high-dimensional data is reduced in dimension, and a scatter plot can be generated using the reduced two-dimensional or three-dimensional data.
[0035] [System Configuration] The configuration of the information processing device 10 in the first embodiment will be described in more detail with reference to Fig. 2. Fig. 2 is a diagram for explaining an example of a system having the information processing device in the first embodiment. As shown in Fig. 2, the system 100 in the first embodiment has the information processing device 10, a storage device 20, and an output device 30. The information processing device 10, the storage device 20, and the output device 30 are connected via a network.
[0036] The information processing device 10 is, for example, an information processing device such as a CPU (Central Processing Unit), a programmable device such as an FPGA (Field-Programmable Gate Array), a GPU (Graphics Processing Unit), or a circuit equipped with any one or more of them, a server computer, a personal computer, a mobile terminal, or the like.
[0037] The storage device 20 is a database, a server computer, a circuit having a memory, etc. In the example of FIG. 2, the storage device 20 is provided outside the information processing device 10, but may be provided inside the information processing device 10.
[0038] The output device 30 acquires output information, which will be described later, that has been converted into an outputtable format, and outputs generated images and sounds based on the output information. The output device 30 is, for example, an image display device using a liquid crystal, an organic EL (Electro Luminescence), or a CRT (Cathode Ray Tube). Furthermore, the image display device may also include an audio output device such as a speaker. The output device 30 may also be a printing device such as a printer.
[0039] The network is a general communication network constructed using communication lines such as the Internet, a LAN (Local Area Network), a dedicated line, a telephone line, an in-house network, a mobile communication network, Bluetooth (registered trademark), or Wi-Fi (Wireless Fidelity) (registered trademark).
[0040] Detailed description of the information processing device The information processing device 10 in FIG. 2 includes a persistent information generation unit 11, a feature vector extraction unit 12, an objective function generation unit 13, an optimal solution calculation unit 14, a dimension reduction unit 15, a visualization unit 16, and an output information generation unit 17.
[0041] ●Generating persistent information The persistent information generator 11 acquires learning data stored in the storage device 20. The learning data is, for example, a handwritten character image, a medical diagnostic image, molecular structure data, urban traffic data, etc. The medical diagnostic image is, for example, a computed tomography (CT) diagnostic image, a magnetic resonance imaging (MRI) diagnostic image, etc. The molecular structure data is, for example, polymer data, etc. The urban traffic data is, for example, an image acquired from a camera, a sensor, etc.
[0042] Next, the persistent information generation unit 11 performs a filtration process on the acquired learning data, stores the process of creation (BIRTH) and destruction (DEATH) of geometric invariants (connected components and holes), and generates persistent information (connected component persistent information and hole persistent information).
[0043] Filtration processing is a process that creates and eliminates geometric invariants for an image. For example, it includes processing that expands each point included in the learning data with the center as the center, evolves the point cloud over time, and stores the diameters of geometric invariants (connected components and holes) when they appear and disappear. Figure 3 is a diagram for explaining an example of filtration processing.
[0044] Table 31 in Fig. 3 shows four examples of points included in the learning data, representing the case where each point is inflated (the diameter of the point is increased) by the filtration process. Note that the horizontal axis of the persistent diagram described in Table 31 represents BIRTH (the diameter when a connected component or a hole occurs), and the vertical axis represents DEATH (the diameter when a connected component or a hole disappears).
[0045] The connected component persistent information is information corresponding to the persistent diagram representing the process of generation of the connected component described above. The hole persistent information is information corresponding to the persistent diagram representing the process of disappearance of the hole described above.
[0046] As the filtration process, for example, the Vietoris-Rips filtration described in References [3][4] is used. Note that the conventional filtration cannot define Persistent Densitry Entropy, but the Vietrois-Rips filtration can define Persistent Density Entropy.
[0047] Reference [3]: Donald R. Sheehy, “Linear-size approximations to the Vietoris-Rips filtration”, [online], Proceedings of the twenty-eighth annual symposium on Computational geometry. June 2012. p. 239-248, [searched on July 19, 2023], Internet <URL:https: / / doi.org / 10.1145 / 2261250.<2261286>
[0048] References [4]: Jinzhong Xu, Xuzhi Li, Hongfei Wang, “Extracting Topological Features from Big Data Using Persistent Density Entropy”, [online], Journal of Physics: Conference Series (Vol. 1168, No. 3, p. 032017). IOP Publishing Ltd 12 March 2019, [Retrieved July 19, 2023], Internet<URL:https: / / iopscience.iop.org / article / 10.1088 / 1742-6596 / 1168 / 3 / 032017>
[0049] Next, we explain the persistent density entropy determined by the Vietoris-Rips filtration described above.
[0050] First, when N supervised data sets D as shown in Equation 1 are given, the joint probability and marginal probability can be defined as shown in Equation 2 and Equation 3.
[0051]
number
[0052] y in the above number 1 i x i represents the class to which it belongs, and N in number 1 represents a positive integer greater than or equal to 1.
[0053] Next, when a dataset D is given as in Equation 1, for each class y, (x i , y) ε (x i , y) can be defined as in Equation 2.
[0054]
number
[0055] Also, (x i ,y) ε (x i ), p(y) can be defined as in Equation 3.
[0056]
number
[0057] And x i Sets X and y i If the set Y of points is expressed as in Equation 4, the persistent density entropy H(X) for a Vietoris-Rips filtration constructed from a point group X can be defined as Equation 5.
[0058]
number
[0059]
number
[0060] In reference [4], the radius of the point x in the Vietoris-Rips filtration process is ε. i Probability density p ε is defined as Equation 6, and the persistent density entropy when ε>0 is defined as Equation 7.
[0061]
number
[0062] In addition, #B in number 6 ε is the point x i represents the number of points whose distance to is less than or equal to ε.
[0063]
number
[0064] However, in reference [4] X Since (ε) is not smooth, we defined persistent density entropy as shown in equation 6. Here, "not smooth" means that it is not a curve, but rather that the curve is jagged (broken line) due to many complex small changes. Therefore, differentiation is impossible, and it is difficult to find the minimum point (or maximum point). Also, there is no mention in reference [4] of the marginal probability p(y) (see equation 3) when class y is given.
[0065] In this way, by using persistent density entropy in Vietoris-Rips filtration as a filtration process, the following effects can be obtained. (1) It inherits the advantage of being strong against nonlinear problems, which is common to both t-SNE and UMAP. (2) There is no need to assume uniformity of the data as in UMAP. (3) t-SNE is susceptible to the curse of dimensionality because it uses all combinations between two points to define the probability, but as mentioned above, it is less susceptible to the curse of dimensionality because it defines the probability using only nearby data.
[0066] Generate feature vectors The feature vector extraction unit 12 first obtains the persistent information generated by the persistent information generation unit 11. Next, the feature vector extraction unit 12 generates a feature vector Z by performing a vector conversion process using the persistent information.
[0067] Specifically, the feature vector extraction unit 12 uses persistence weighted Gaussian kernel processing to generate the feature vector Z. For details about persistence weighted Gaussian kernel processing, see reference [6].
[0068] References [6]: Genki Kusano, Kenji Fukumizu, Yasuaki Hiraoka, “Persistence weighted Gaussian kernel for topological data analysis”, [online],: International conference on machine learning. PMLR, 2016. p. 2004-2013, arXiv:1601.01741v2 [math.AT] 26 Apr 2016, [Retrieved July 19, 2023], Internet<URL:https: / / arxiv.org / abs / 1601.01741>
[0069] ●Generate the objective function The objective function generation unit 13 first obtains the feature vector Z generated by the feature vector extraction unit 12. Next, the objective function generation unit 13 generates a Kullback-Leibler divergence as an objective function f(W) for dimension reduction, using the feature vector Z and the dimension reduction matrix W.
[0070] Specifically, the objective function generating unit 13 calculates the Kullback-Leibler divergence (Equation 8) using parameters ε and δ that determine the persistent density entropy for the data before and after the dimension reduction, respectively. Note that the Kullback-Leibler divergence is also defined using the calculation method of the persistent density entropy.
[0071] The parameter ε is a parameter (Equation 5) that determines the persistent density entropy for data before dimension reduction. The parameter δ is a parameter that determines the persistent density entropy for data after dimension reduction.
[0072] Next, we explain the Kullback-Leibler divergence. n →R m Given the Kullback-Leibler information between two evaluation distributions, D KL (Z1||Z2) can be defined as in Equation 8.
[0073]
number
[0074] The set Z1 is included in an n-dimensional Euclidean space, and the set Z2 is included in an m-dimensional Euclidean space, and they are expressed as in Equation 9.
[0075]
number
[0076] The persistent density entropy of sets X and Z is expressed as follows:
[0077]
number
[0078] Therefore, the Kullback-Leibler information D KL The smaller the value of (Z1||Z2), the more similar the geometric structures of the distributions of sets Z1 and Z2 are.
[0079] Next, the objective function f(W) will be explained. When dimension reduction is performed using a dimension reduction matrix W, the dimension reduction is performed by R n →R m :z→Wz Then, the objective function f(W) can be expressed as Equation 11.
[0080]
number
[0081] Here, the dimensionality reduction matrix W represents the variables of the objective function. Also, the dimensionality reduction matrix W is an m×n matrix (regarded as an m×n-dimensional vector) and belongs to R m×n (m < n) (W ∈ R m×n ).
[0082] ● Calculation of the optimal solution of the dimensionality reduction matrix The optimal solution calculation unit 14 first obtains the objective function f(W) generated by the objective function generation unit 13. Next, the optimal solution calculation unit 14 performs Bayesian optimization processing on the objective function f(W) to calculate the optimal solution W * (projection matrix) of the dimensionality reduction matrix W that minimizes the Kullback-Leibler information amount. Note that the Bayesian optimization processing can be expressed as in Equation (12). For details of the Bayesian optimization processing, refer to Reference [7].
[0083]
Equation
[0084] Reference [7]: Ziyu Wang, Masrour Zoghi, Frank Hutter, David Matheson, Nando de Freitas, “Bayesian optimization in high dimensions via random embeddings”, [online], Twenty-Third International Joint Conference on Artificial Intelligence. August 3 - 9, 2013, [searched on July 19, 2023], Internet <URL:https: / / www.ijcai.org / Proceedings / 13 / Papers / 263.pdf>
[0085] ● Dimensionality reduction of feature vectors The dimensionality reduction unit 15 first obtains the optimal solution W of the dimensionality reduction matrix W generated by the optimal solution calculation unit 14* Next, the dimension reduction unit 15 reduces the dimension of the feature vector Z (second high-dimensional data) by the projection matrix W * We use the reduction method to reduce the dimensional data Z LOW (2D data or 3D data) to convert low-dimensional data Z LOW The dimensionality reduction to can be expressed as follows:
[0086]
number
[0087] ●Visualization The visualization unit 16 uses the low-dimensional data generated by the dimension reduction unit 15 to generate visualization information that represents, for example, a scatter plot, a probability distribution diagram, or the like.
[0088] The output information generation unit 17 acquires the visualization information and generates output information for displaying a scatter plot, a probability distribution diagram, etc. on the output device 30. After that, the output information generation unit 17 outputs the output information to the output device 30.
[0089] [Device operation] Next, the operation of the information processing device in the first embodiment will be described with reference to FIG. 4. FIG. 4 is a diagram for explaining an example of the operation of the information processing device in the first embodiment. In the following description, the diagram will be referred to as appropriate. In the first embodiment, the information processing method is implemented by operating the information processing device. Therefore, the description of the information processing method in the first embodiment will be replaced with the following description of the operation of the information processing device.
[0090] As shown in FIG. 4, the persistent information generating unit 11 performs a filtration process on high-dimensional learning data (first high-dimensional data) to generate persistent information representing a persistence diagram (step A1).
[0091] Specifically, in step A1, the persistent information generator 11 first acquires learning data stored in the storage device 20.
[0092] Next, in step A1, the persistent information generation unit 11 performs a filtration process on the acquired learning data, stores the process of generation (BIRTH) and destruction (DEATH) of geometric invariants (connected components and holes), and generates persistent information (connected component persistent information and hole persistent information).
[0093] Next, the feature vector extraction unit 12 performs vector conversion processing on each piece of generated persistent information to generate a feature vector (second high-dimensional data) (step A2).
[0094] Specifically, in step A2, the feature vector extraction unit 12 first acquires the persistent information generated by the persistent information generation unit 11. Next, in step A2, the feature vector extraction unit 12 executes a vector conversion process using the persistent information to generate a feature vector Z.
[0095] Next, the objective function generating unit 13 generates a Kullback-Leibler divergence representing the objective function using the generated feature vector and a dimensionality reduction matrix representing the variables of the objective function (step A3).
[0096] Specifically, in step A3, the objective function generation unit 13 first acquires the feature vector Z generated by the feature vector extraction unit 12. Next, in step A3, the objective function generation unit 13 generates the Kullback-Leibler divergence as an objective function f(W) for dimension reduction, using the feature vector Z and the dimension reduction matrix W.
[0097] Next, in step A4, the optimal solution calculation unit 14 executes a Bayesian optimization process on the generated objective function, and calculates an optimal solution of the dimensionality reduction matrix that minimizes the Kullback-Leibler divergence (step A4).
[0098] Specifically, in step A4, the optimal solution calculation unit 14 first obtains the objective function f(W) generated by the objective function generation unit 13. Next, in step A4, the optimal solution calculation unit 14 executes a Bayesian optimization process on the objective function f(W) to obtain an optimal solution W of the dimensionality reduction matrix W that minimizes the Kullback-Leibler divergence. * Calculate the projection matrix.
[0099] Next, the dimension reduction unit 15 reduces the dimension of the feature vector using the calculated optimal solution of the dimension reduction matrix (step A5).
[0100] Specifically, in step A5, the dimension reduction unit 15 first calculates the optimal solution W of the dimension reduction matrix W generated by the optimal solution calculation unit 14. * Next, in step A5, the dimension reduction unit 15 reduces the dimension of the feature vector Z (second high-dimensional data) by the projection matrix W * We use the reduction method to reduce the dimensional data Z LOW (2D data or 3D data).
[0101] Next, the visualization unit 16 generates visualization information representing, for example, a scatter plot, a probability distribution diagram, or the like, using the low-dimensional data generated by the dimension reduction unit 15 (step A6).
[0102] The output information generating unit 17 acquires the visualization information, generates output information for displaying a scatter plot, a probability distribution diagram, or the like on the output device 30, and then outputs the generated output information to the output device 30 (Step A7).
[0103] [Effects of the First Embodiment] As described above, according to the first embodiment, (i) unlike t-SNE, all two-point combinations are not used, so that dimensionality reduction can be achieved that is less susceptible to the influence of high-dimensional data (the curse of dimensionality).
[0104] In addition, (ii) since the persistent density entropy is defined using the concept of topological data analysis as the amount of information, it is not necessary to assume a uniform distribution as in UMAP, and realistic dimensionality reduction is possible.
[0105] In addition, (iii) an optimal solution (projection matrix used in dimensionality reduction) that minimizes the Kullback-Leibler divergence (difference in probability distribution before and after dimensionality reduction) is obtained by Bayesian optimization (linear transformation). Based on the finally obtained projection matrix (linear projection), the high-dimensional data is reduced in dimension, and a scatter plot can be generated using the reduced two-dimensional or three-dimensional data.
[0106] Furthermore, the advantages of the first embodiment will be described with reference to Figs. 5 and 6. Fig. 5 is a diagram for explaining the quantitative evaluation results for each dimension reduction method. Fig. 6 is a diagram for explaining the quantitative evaluation results (scatter plot and probability distribution) for each dimension reduction method. Figs. 5 and 6 compare the dimension reduction method of the first embodiment with principal component analysis (PCA), t-SNE, and UMAP. In addition, quantitative evaluation was performed using Kullback-Lebler (KL) divergence, mutual information, purity, and Spearman using open data (iris data) in the data science field.
[0107] Kullback-Lebler (KL) divergence represents the difference in data distribution before and after dimensionality reduction. The smaller the evaluation result (closer to 0), the better. Mutual information represents the dependent relationship between explanatory variables and target variables. The larger the value, the better the characteristics of each class are expressed in the scatter plot. Purity represents the proportion of correct classes in clusters in the scatter plot. The larger the value (closer to 1), the better. It represents separation in the form of each cluster = each class distribution. Spearman represents the Spearman rank correlation coefficient. A larger value (closer to 1) indicates that there is no change in the rank order of the distance between two points before and after dimensionality reduction (the distribution of high-dimensional data is maintained).
[0108] 5A and 5B, the numbers with the best evaluation results are shown in bold. Therefore, it can be seen that the evaluation results using the dimension reduction method of the first embodiment (proposed method) are superior to the evaluation results using the other dimension reduction methods.
[0109] In the example of Fig. 6, A shows a scatter plot and a probability distribution diagram of PCA, B shows a scatter plot and a probability distribution diagram of t-SNE, C shows a scatter plot and a probability distribution diagram of UMAP, and D shows the dimension reduction method of the first embodiment (proposed method). In Fig. 6, the positional relationship and the probability distribution of three classes are shown.
[0110] [program] The program in the first embodiment may be a program that causes a computer to execute steps A1 to A7 shown in Fig. 4. By installing and executing this program in a computer, it is possible to realize the information processing device and information processing method in the first embodiment. In this case, the processor of the computer functions as a persistent information generating unit 11, a feature vector extracting unit 12, an objective function generating unit 13, an optimal solution calculating unit 14, a dimension reducing unit 15, a visualization unit 16, and an output information generating unit 17, and performs processing.
[0111] The program in the first embodiment may be executed by a computer system constructed by a plurality of computers. In this case, for example, each computer may function as any one of the persistent information generating unit 11, the feature vector extracting unit 12, the objective function generating unit 13, the optimal solution calculating unit 14, the dimension reducing unit 15, the visualizing unit 16, and the output information generating unit 17.
[0112] (Embodiment 2) In the second embodiment, during operation, dimension reduction of new data (high-dimensional data) is performed using the dimension reduction matrix of the optimal solution generated in the first embodiment. The new data is, for example, data added to the learning data.
[0113] [System Configuration] The configuration of the information processing device 10a in the second embodiment will be described in more detail with reference to Fig. 7. Fig. 7 is a diagram for explaining an example of a system having the information processing device in the second embodiment. As shown in Fig. 7, the system 100a in the second embodiment has the information processing device 10a, a storage device 20, and an output device 30. The information processing device 10a, the storage device 20, and the output device 30 are connected via a network.
[0114] 7, the information processing device 10a includes a persistent information generating unit 11, a feature vector extracting unit 12, a dimensionality reducing unit 15a, a visualization unit 16, and an output information generating unit 17. Note that the persistent information generating unit 11, the feature vector extracting unit 12, the visualization unit 16, and the output information generating unit 17 have been described in the first embodiment, and therefore detailed description thereof will be omitted.
[0115] The persistent information generating unit 11 first performs a filtration process on new high-dimensional data (high-dimensional data) to generate persistent information (second persistent information) representing a persistent diagram.
[0116] Next, the feature vector extraction unit 12 performs vector conversion processing on each piece of generated persistent information to generate a feature vector (second feature vector: high-dimensional data).
[0117] Next, the dimension reduction unit 15a reduces the dimension of the feature vector using the optimal solution of the dimension reduction matrix generated during learning, and generates low-dimensional data (second low-dimensional data).
[0118] Specifically, the dimension reduction unit 15a first calculates the optimal solution W * Next, the dimension reduction unit 15 reduces the dimension of the feature vector (second high-dimensional data) by the projection matrix W * The data is reduced using the method described above and converted into low-dimensional data (two-dimensional data or three-dimensional data).
[0119] Next, the visualization unit 16 uses the low-dimensional data generated by the dimension reduction unit 15 to generate visualization information that represents, for example, a scatter plot, a probability distribution diagram, or the like.
[0120] Next, the output information generation unit 17 acquires the visualization information and generates output information for displaying a scatter plot, a probability distribution diagram, etc. on the output device 30. Thereafter, the output information generation unit 17 outputs the output information to the output device 30.
[0121] [Device operation] Next, the operation of the information processing device in the second embodiment will be described with reference to FIG. 8. FIG. 8 is a diagram for explaining an example of the operation of the information processing device in the second embodiment. In the following description, the diagram will be referred to as appropriate. In the second embodiment, an information processing method is implemented by operating the information processing device. Therefore, the description of the information processing method in the second embodiment will be replaced with the description of the operation of the information processing device below.
[0122] As shown in FIG. 8, during operation, the persistent information generating unit 11 performs a filtration process on new high-dimensional data (first high-dimensional data) to generate persistent information representing a persistence diagram (step B1).
[0123] Specifically, in step B1, the persistent information generator 11 first acquires new data stored in the storage device 20.
[0124] Next, in step B1, the persistent information generation unit 11 performs a filtration process on the acquired new data, stores the process of creation (BIRTH) and destruction (DEATH) of geometric invariants (connected components and holes), and generates persistent information (connected component persistent information and hole persistent information).
[0125] Next, the feature vector extraction unit 12 performs vector conversion processing on each piece of generated persistent information to generate a feature vector (second high-dimensional data) (step A2).
[0126] Specifically, in step A2, the feature vector extraction unit 12 first acquires the persistent information generated by the persistent information generation unit 11. Next, in step A2, the feature vector extraction unit 12 executes a vector conversion process using the persistent information to generate a feature vector Z.
[0127] Next, the dimension reduction unit 15a reduces the dimension of the feature vector using the calculated optimal solution of the dimension reduction matrix (step B2).
[0128] Specifically, in step B2, the dimension reduction unit 15a calculates the optimal solution W of the dimension reduction matrix W generated in the first embodiment. * Next, in step B2, the dimension reduction unit 15a reduces the dimension of the feature vector Z (second high-dimensional data) to a projection matrix W * We use the reduction method to reduce the dimensional data Z LOW (2D data or 3D data).
[0129] Next, the visualization unit 16 generates visualization information representing, for example, a scatter plot, a probability distribution diagram, or the like, using the low-dimensional data generated during learning (step A6).
[0130] The output information generation unit 17 acquires the visualization information, generates output information for displaying a scatter plot, a probability distribution diagram, or the like on the output device 30, and then outputs the generated output information to the output device 30 (step A7).
[0131] [Effects of the second embodiment] As described above, according to the second embodiment, new incremental data can be visualized. In addition, (i) unlike t-SNE, since all two-point combinations are not used, dimensionality reduction that is less susceptible to the influence of high-dimensional data (the curse of dimensionality) can be achieved.
[0132] In addition, (ii) since the persistent density entropy is defined using the concept of topological data analysis as the amount of information, it is not necessary to assume a uniform distribution as in UMAP, and realistic dimensionality reduction is possible.
[0133] Furthermore, (iii) an optimal solution (projection matrix used in dimensionality reduction) that minimizes the Kullback-Leibler divergence (difference in probability distribution before and after dimensionality reduction) is obtained by Bayesian optimization (linear transformation). Based on the finally obtained projection matrix (linear projection), the high-dimensional data is reduced in dimension, and a scatter plot can be generated using the reduced two-dimensional or three-dimensional data.
[0134] [Physical configuration] Here, a computer that realizes the information processing device by executing the program in the first and second embodiments will be described with reference to Fig. 9. Fig. 9 is a diagram for explaining an example of a computer that realizes the information processing device in the first and second embodiments.
[0135] 9, the computer 110 includes a CPU (Central Processing Unit) 111, a main memory 112, a storage device 113, an input interface 114, a display controller 115, a data reader / writer 116, and a communication interface 117. These components are connected to each other via a bus 121 so as to be able to communicate data with each other. Note that the computer 110 may include a GPU or an FPGA in addition to the CPU 111 or instead of the CPU 111.
[0136] The CPU 111 loads a program in the embodiment, which is composed of a group of codes and is stored in the storage device 113, into the main memory 112, and executes each code in a predetermined order to perform various calculations. The main memory 112 is typically a volatile storage device such as a DRAM (Dynamic Random Access Memory).
[0137] Moreover, the programs in the first and second embodiments are provided in a state stored in a computer-readable recording medium 120. The programs in the first and second embodiments may be distributed on the Internet connected via the communication interface 117.
[0138] Specific examples of the storage device 113 include a hard disk drive and a semiconductor storage device such as a flash memory. The input interface 114 mediates data transmission between the CPU 111 and input devices 118 such as a keyboard and a mouse. The display controller 115 is connected to a display device 119 and controls the display on the display device 119.
[0139] Data reader / writer 116 mediates data transmission between CPU 111 and recording medium 120, reads programs from recording medium 120, and writes processing results in computer 110 to recording medium 120. Communication interface 117 mediates data transmission between CPU 111 and other computers.
[0140] Specific examples of recording medium 120 include general-purpose semiconductor storage devices such as CF (Compact Flash (registered trademark)) and SD (Secure Digital), magnetic recording media such as a flexible disk, or optical recording media such as a CD-ROM (Compact Disk Read Only Memory).
[0141] The information processing device in the first and second embodiments can be realized not by a computer with a program installed, but by hardware corresponding to each unit, for example, an electronic circuit. Furthermore, the information processing device may be partially realized by a program and the remaining part by hardware. In the embodiments, the computer is not limited to the computer shown in FIG.
[0142] Although the invention has been described above with reference to the embodiments, the invention is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the invention within the scope of the invention. [Industrial Applicability]
[0143] According to the above description, it is possible to reduce the influence of noise in the dimensionality reduction of high-dimensional data, and the present invention is useful in fields where dimensionality reduction is required. [Explanation of symbols]
[0144] 10, 10a Information processing device 11 Persistent information generation unit 12 Feature Vector Extraction Unit 13 Objective function generator 14 Optimal solution calculation section 15, 15a Dimension reduction part 16 Visualization part 17 Output information generation unit 20 Storage device 30 Output Device 100, 100a Systems 110 Computer 111 CPU 112 Main memory 113 Storage device 114 Input Interface 115 Display Controller 116 Data Reader / Writer 117 Communication Interface 118 Input Devices 119 Display device 120 Recording media 121 Bus
Claims
1. A persistent information generating means for performing a filtration process on high-dimensional learning data to generate first persistent information representing a persistent diagram; a feature vector extraction means for performing a vector conversion process on each of the generated first persistent information to generate a first feature vector; an objective function generating means for generating a Kullback-Leibler divergence representing the objective function using the generated first feature vector and a dimension reduction matrix representing variables of the objective function; an optimal solution calculation means for performing a Bayesian optimization process on the generated objective function to calculate an optimal solution of the dimensionality reduction matrix in which the Kullback-Leibler divergence is minimized; a dimension reduction means for reducing a dimension of the first feature vector using the calculated optimal solution of the dimension reduction matrix to generate first reduced-dimensional data; An information processing device having the above configuration.
2. The filtration process calculates the persistent density entropy determined by Vietoris-Rips filtration. The information processing device according to claim 1 .
3. The feature vector extraction means generates the first feature vector by using a persistence weighted Gaussian kernel process. The information processing device according to claim 1 .
4. Further, a visualization unit is provided for generating visualization information using the first low-dimensional data generated by the dimension reduction unit. The information processing device according to claim 1 .
5. The visualization information is information representing a scatter diagram, a probability distribution diagram, or the like. The information processing device according to claim 4.
6. The first low-dimensional data is two-dimensional data or three-dimensional data. The information processing device according to claim 4.
7. During operation, The persistent information generating means performs the filtration process on the new high-dimensional data to generate second persistent information; the feature vector extraction means performs the vector conversion process on each of the generated second persistent information to generate a second feature vector; the dimensionality reduction means reduces a dimension of the second feature vector using the calculated optimal solution of the dimensionality reduction matrix to generate second low-dimensional data; The information processing device according to claim 4.
8. The visualization means generates visualization information using the second low-dimensional data generated by the dimension reduction means. The information processing device according to claim 7.
9. An information processing device, Filtration is performed on the high-dimensional learning data to generate first persistence information representing a persistence diagram; performing a vector conversion process on each of the generated first persistent information to generate a first feature vector; generating a Kullback-Leibler divergence representing the objective function using the generated first feature vector and a dimensionality reduction matrix representing variables of the objective function; A Bayesian optimization process is performed on the generated objective function to calculate an optimal solution of the dimensionality reduction matrix that minimizes the Kullback-Leibler divergence; reducing the dimension of the first feature vector using the calculated optimal solution of the dimension reduction matrix; Information processing methods.
10. On the computer, Filtration is performed on the high-dimensional learning data to generate first persistence information representing a persistence diagram; performing a vector conversion process on each of the generated first persistent information to generate a first feature vector; generating a Kullback-Leibler divergence representing the objective function using the generated first feature vector and a dimensionality reduction matrix representing variables of the objective function; A Bayesian optimization process is performed on the generated objective function to calculate an optimal solution of the dimensionality reduction matrix that minimizes the Kullback-Leibler divergence; reducing the dimension of the first feature vector using the calculated optimal solution of the dimension reduction matrix; program.