Feature selection device, feature selection method, and program

Abstract

This feature amount selection device comprises: a feature amount data acquisition unit which acquires feature amount data including, for each of a plurality of samples, pairs of values of a plurality of feature amounts for the samples; a principal component analysis unit which performs, on the feature amount data, principal component analysis in a sample space that is a group of the plurality of feature amounts of the value pairs for each of the plurality of samples of the feature amounts; and a feature amount selection unit which selects a feature amount from among the plurality of feature amounts on the basis of the principal component analysis result performed by the principal component analysis unit.

Description

[Technical Field]

[0001] The present invention relates to a feature selection device, a feature selection method, and a program. This application claims priority based on Japanese Patent Application No. 2020-191502, filed on November 18, 2020, the contents of which are incorporated herein by reference. [Background technology]

[0002] In recent years, advances in measurement devices and sensors, such as next-generation sequencers and mass spectrometers, have made it possible to obtain large amounts of high-dimensional data. Therefore, there is a strong demand for effective data analysis techniques for dealing with such large amounts of high-dimensional data. High-dimensional data poses problems, such as increased computational complexity and a deterioration in prediction accuracy due to an excessive number of explanatory variables. Therefore, in analyzing high-dimensional data, the number of features used in the analysis is reduced by selecting some of the features. However, this feature reduction can result in the loss of information from the original data, resulting in a decrease in analytical accuracy. Therefore, it has been difficult to achieve a significant reduction while maintaining analytical accuracy.

[0003] Known conventional techniques for selecting features include the filter method and the wrapper method (see, for example, Patent Document 1). The filter method is a method for calculating statistical values (e.g., chi-squared value, Fisher information, ANNOVA test, variance of variables, etc.) for each feature and ranking them. The filter method may remove information obtained by combining multiple features. The wrapper method selects key features based on the accuracy of machine learning for many combinations of whether or not each feature is used. However, with the wrapper method, when there are a large number of features, the number of combinations increases, resulting in a huge amount of calculation, making it difficult to apply to large-scale data. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Lei Yu, Huan Liu, Feature Selection for High-Dimensional Data: A Fast Correlation-Based Filter Solution, Proceedings of the Twentieth International Conference on Machine Learning (ICML-2003), August 21, 2003, p.856-863 [Non-patent document 2] Andrew Butler, Paul Hoffman, Peter Smibert, Efthymia Papalexi, Rahul Satija, Integrating single-cell transcriptomic data across different conditions, technologies, and species, Nature Biotechnology, Nature America, Inc., April 2, 2018, Vol. 36, No. 5, pp. 411-420 [Non-patent document 3] “IGSR: The International Genome Sample Resource”, [online], EMBL-EBI, [Retrieved July 31, 2020], Internet <URL: http: / / www.1000genomes.org> [Non-patent document 4] Saori Sakaue,Jun Hirata,Masahiro Kanai,Ken Suzuki,Masato Akiyama,Chun Lai Too,Thurayya Arayssi,Mohammed Hammoudeh,Samar Al Emadi,Basel K. Masri,Hussein Halabi,Humeira Badsha,Imad W. Uthman,Richa Saxena,Leonid Padyukov,Makoto Hirata,Koichi Matsuda,Yoshinori Murakami,Yoichiro Kamatani,Yukinori Okada. Dimensionality reduction reveals fine-scale structure in the Japanese population with consequences for polygenic risk prediction. "NATURE COMMUNICATIONS", Springer Nature Limited, March 26, 2020, Volume 11, Issue 1569, pp. 1-11 [Non-Patent Document 5] "Gene Expression Omnibus", [online], National Center for Biotechnology Information, [searched on July 31, 2020], Internet <URL:https: / / www.ncbi.nlm.nih.gov / geo / > [Non-Patent Document 6] Tim Stuart, Andrew Butler, Paul Hoffman, Christoph Hafemeister, Efthymia Papalexi, William M. Mauck III, Yuhan Hao, Marlon Stoeckius, Peter Smibert, Rahul Satija. Comprehensive Integration of Single-Cell Data. "Cell", Elsevier Inc., June 13, 2019, Volume 177, pp. 1888-1902 [Summary of the Invention] [Problem to be solved by the invention]

[0005] The analysis of high-dimensional feature data has the problem that the accuracy of machine learning decreases due to the large number of explanatory variables. In addition, the analysis of high-dimensional feature data has the problem that the amount of calculation required is large, and the analysis calculation takes a huge amount of time. There is a demand for speeding up calculations without sacrificing analytical accuracy in the analysis of high-dimensional feature data.

[0006] The present invention has been made in view of the above points, and provides a feature selection device, a feature selection method, and a program that can speed up calculations without impairing analysis accuracy in analyzing high-dimensional feature data. [Means for solving the problem]

[0007] The present invention has been made to solve the above-mentioned problems, and one aspect of the present invention is a feature data acquisition unit that acquires feature data including a set of values of a plurality of feature amounts for each of a plurality of samples, and a principal component analysis unit that performs principal component analysis on the feature data in a sample space that is a collection of sets of values of the feature amounts for each of the plurality of samples, for the plurality of feature amounts; a distortion determination unit that determines whether or not there is distortion in the distribution of principal components obtained by the principal component analysis in the sample space; The principal component analysis performed by the principal component analysis unit Principal components that are determined to have no distortion in the distribution among the principal components obtained by and a feature selection unit that selects a feature from among the plurality of feature values based on the feature value.

[0009] In one aspect of the present invention, in the feature selection device, the feature selection unit selects a feature that is a large distance from the origin of the sample space for a principal component determined to have no distortion in the distribution.

[0010] Another aspect of the present invention is a feature data acquisition step of acquiring feature data including a set of values of a plurality of feature amounts for each of a plurality of samples; and a principal component analysis step of performing principal component analysis on the feature data in a sample space which is a collection of sets of values of the feature amounts for each of the plurality of samples, for the plurality of feature amounts. a distortion determination step of determining whether or not there is distortion in the distribution of principal components obtained by the principal component analysis in the sample space; The principal component analysis performed in the principal component analysis step Principal components that are determined to have no distortion in the distribution among the principal components obtained by a feature quantity selection step of selecting a feature quantity from the plurality of feature quantities based on The computer runs This is a feature selection method.

[0011] Also, one aspect of the present invention is a method for generating a plurality of feature vectors by a computer, the method comprising: a feature data acquisition step of acquiring feature vector data including a set of values of a plurality of feature vectors for each of a plurality of samples; and a principal component analysis step of performing principal component analysis on the feature vector data in a sample space which is a collection of sets of values of the feature vectors for each of the plurality of samples for the plurality of feature vectors; a distortion determination step of determining whether or not there is distortion in the distribution of principal components obtained by the principal component analysis in the sample space; The principal component analysis performed in the principal component analysis step Principal components that are determined to have no distortion in the distribution among the principal components obtained by and a feature quantity selection step of selecting a feature quantity from the plurality of feature quantities based on the feature quantity. [Effects of the Invention]

[0012] According to the present invention, it is possible to speed up calculations in the analysis of high-dimensional feature data without impairing the accuracy of the analysis. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a diagram illustrating an example of a configuration of a feature quantity selection system according to an embodiment of the present invention. [Figure 2] FIG. 2 is a diagram for explaining a definition of a feature space according to an embodiment of the present invention. [Figure 3] FIG. 2 is a diagram for explaining the definition of a sample space according to an embodiment of the present invention. [Figure 4]FIG. 4 is a diagram illustrating an example of a feature quantity selection process of the feature quantity selection device according to the embodiment of the present invention. [Figure 5] FIG. 2 is a diagram showing an example of artificial data according to the first embodiment of the present invention. [Figure 6] FIG. 4 is a diagram showing an example of the result of cluster analysis of a comparison target according to the first embodiment of the present invention. [Figure 7] FIG. 10 is a diagram showing an example of a first principal component and a second principal component in a sample space according to the first embodiment of the present invention. [Figure 8] FIG. 10 is a diagram showing an example of a third principal component and a fourth principal component in a sample space according to the first embodiment of the present invention. [Figure 9] FIG. 10 is a diagram showing an example of the distribution of the first principal component in the sample space according to the first example of the present invention. [Figure 10] FIG. 10 is a diagram showing an example of the distribution of a sixth principal component in a sample space according to the first example of the present invention. [Figure 11] FIG. 4 is a diagram showing an example of a result of dimensionality reduction in a feature space according to the first embodiment of the present invention. [Figure 12] FIG. 10 is a diagram showing an example of the result of cluster analysis of a comparison target according to the second embodiment of the present invention. [Figure 13] FIG. 10 is a diagram showing an example of a first principal component and a second principal component in a sample space according to a second embodiment of the present invention. [Figure 14] FIG. 10 is a diagram showing an example of the third principal component and the fourth principal component in the sample space according to the second embodiment of the present invention. [Figure 15] FIG. 10 is a diagram showing an example of the distribution of the first principal component in a sample space according to the second example of the present invention. [Figure 16] FIG. 10 is a diagram showing an example of the distribution of the fourth principal component in the sample space according to the second example of the present invention. [Figure 17] FIG. 10 is a diagram showing an example of a result of dimensionality reduction in a feature space according to the second embodiment of the present invention. [Figure 18] FIG. 10 is a diagram showing an example of the result of cluster analysis of a comparison target according to the third embodiment of the present invention. [Figure 19]FIG. 11 is a diagram showing an example of the first principal component and the second principal component in the sample space according to the third embodiment of the present invention. [Figure 20] FIG. 10 is a diagram showing an example of a third principal component and a fourth principal component in a sample space according to the third embodiment of the present invention. [Figure 21] FIG. 11 is a diagram showing an example of the distribution of the first principal component in a sample space according to the third example of the present invention. [Figure 22] FIG. 11 is a diagram showing an example of the distribution of the fifth principal component in the sample space according to the third example of the present invention. [Figure 23] FIG. 11 is a diagram showing an example of a result of dimensionality reduction in a feature space according to the third embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0014] (Embodiment) Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. FIG. 1 is a diagram showing an example of the configuration of a feature selection system 1 according to this embodiment. The feature selection system 1 performs principal component analysis in a sample space on high-dimensional feature data, and selects features based on the sample space principal components that contribute to cluster separation of samples. In conventional multivariate analysis, analysis such as principal component analysis is performed on a feature space. In contrast, the feature selection system 1 captures the relationships between multiple features in the sample space.

[0015] A sample space is a collection of sets of feature values for each of a plurality of samples of the feature. The sample space is visualized, for example, by plotting points corresponding to the plurality of feature values in a space having dimensions corresponding to each of the plurality of samples. Note that the feature space used in conventional analysis is a collection of sets of feature values for each of the samples.

[0016] Here, the definitions of the feature space and the sample space will be explained with reference to Figures 2 and 3. In the explanations of Figures 2 and 3, m and n each represent a natural number. In this embodiment, the feature space is defined as follows: Assume that the input data (the above-described high-dimensional feature data) includes n samples, each of which is composed of m features. Assume also that the input data is data in a table format. In this case, the input data has columns for each item of the samples and the m features. In other words, the input data is data in a table format consisting of rows and columns in which feature values are stored for each sample. If all m features are numerical values, each sample can be regarded as a point in an m-dimensional space. In this m-dimensional space, each of the m-dimensional coordinate axes corresponds to one of the m features. This m-dimensional space is called the feature space.

[0017] For example, in the table-format data shown in Figure 2(A), each sample (e.g., an individual indicated by "name") is composed of m features (e.g., four features: age, sex, blood glucose level, and "HbA1c"). As shown in Figure 2(B), principal components (e.g., the first and second principal components) are found from each feature by principal component analysis in the feature space. As shown in Figure 2(C), plotting each sample with the first and second principal components as axes shows the distance relationships between the samples in two-dimensional space.

[0018] Next, in this embodiment, the sample space is defined as follows: In the input data, which is the table-format data described above, rows and columns are swapped to generate new table-format data. The new table-format data has columns for each item of feature amounts and n samples. In other words, the new table-format data is table-format data consisting of rows and columns in which values for each feature amount are stored for the sample of that feature amount. In this case, the number of samples is n, and each feature amount can be considered as a point in n-dimensional space. In this n-dimensional space, each of the n-dimensional coordinate axes corresponds to one of the n samples. This n-dimensional space is called the sample space.

[0019] As an example, Fig. 3(A) shows the result of swapping the rows and columns of the table-format data shown in Fig. 2(A). As shown in Fig. 3(B), principal components (e.g., the first principal component and the second principal component) are found from each sample by principal component analysis in the sample space. As shown in Fig. 3(C), when each feature is plotted with the first principal component and the second principal component as axes, the distance relationship between the feature values is expressed in two-dimensional space. In this embodiment, feature values are selected based on the distance relationship between the feature values in the sample space.

[0020] Returning to FIG. 1, the description of the configuration of the feature selection system 1 will be continued. [Functional configuration of feature selection system 1] The feature selection system 1 includes a feature selection device 10, a feature data supply unit 20, and a presentation unit 30. The feature data supply unit 20 supplies high-dimensional feature data to the feature selection device 10. High-dimensional feature data is data including a set of feature values for each of a plurality of samples. Here, the dimension of a feature refers to the number of features. High dimension means that the number of features is equal to or greater than a predetermined number (e.g., several thousand). In the following description, the high-dimensional feature data will be simply referred to as feature data D. Note that the number of features included in the feature data D may be equal to or less than a predetermined number, for example, several to several hundred.

[0021] The feature data D is, for example, data in the form of a two-dimensional array consisting of rows and columns in which multiple feature values are stored for each sample. In this array, for example, the rows correspond to samples and the columns correspond to features. Therefore, in the feature data D, for example, the cell in the i-th row and j-th column stores the value of the j-th feature of the i-th sample. The feature may be either a feature represented as a categorical variable or a feature represented by a numerical value. Hereinafter, a feature represented as a categorical variable is also referred to as a categorical feature, and a feature represented using a numerical value is also referred to as a numerical feature.

[0022] The feature data supply unit 20 may be, for example, an information storage device such as a server, or a human interface device such as a keyboard, tablet, or scanner.

[0023] The feature selection device 10 includes a feature data acquisition unit 100, a preprocessing unit 101, a numerical feature normalization unit 102, a principal component analysis unit 103, a distortion determination unit 104, a feature selection unit 105, and an output unit 106. The feature selection device 10 is, for example, a personal computer (PC). Each functional unit included in the feature selection device 10 is realized by a central processing unit (CPU) reading a program from a read-only memory (ROM) and executing the program.

[0024] The feature amount data acquisition unit 100 acquires the feature amount data D supplied by the feature amount data supply unit 20 . The preprocessing unit 101 performs preprocessing on the feature amount data D. A specific example of the preprocessing will be described later. The numerical feature quantity conversion normalization unit 102 converts the pre-processed feature quantity data D into numerical feature quantities and normalizes the data. A specific example of the numerical feature quantity conversion and normalization process will be described later.

[0025] The principal component analysis unit 103 performs principal component analysis in the sample space on the feature data D. The principal components obtained by the principal component analysis in the sample space are called sample space principal components P. The sample space principal components P include as many principal components as the number of dimensions of the sample space, and these are called the first principal component, second principal component, etc. The distortion determination unit 104 determines whether or not there is distortion in the distribution of the sample space principal components P. Specific examples of distortion in the distribution of the sample space principal components P will be described later.

[0026] The feature selection unit 105 selects a feature from among the multiple feature values based on the result of the principal component analysis performed by the principal component analysis unit 103. As an example, the feature selection unit 105 selects a feature from among the multiple feature values based on a principal component determined to have no distribution distortion among the sample space principal components P. The output unit 106 outputs the feature quantity selection result R to the presentation unit 30. The feature quantity selection result R is information indicating the feature quantities selected by the feature quantity selection unit 105 from the feature quantities included in the feature quantity data D.

[0027] The presentation unit 30 presents the feature selection result R output from the output unit 106 provided in the feature selection device 10 by presentation means such as display or printing. The presentation unit 30 is, for example, a display or a printer. The presentation unit 30 may be a storage device such as a network server. In this case, the presentation unit 30 stores the feature selection result R output from the output unit 106 and supplies the stored feature selection result R to another device.

[0028] [Operation of the feature selection device 10] Next, a feature quantity selection process in which the feature quantity selection device 10 selects a feature quantity will be described with reference to Fig. 4. Fig. 4 is a diagram showing an example of the feature quantity selection process of the feature quantity selection device 10 according to this embodiment.

[0029] Step S10: The feature amount data acquisition unit 100 acquires the feature amount data D supplied by the feature amount data supply unit 20. The feature amount data acquisition unit 100 supplies the acquired feature amount data D to the preprocessing unit 101.

[0030] Step S20: The preprocessing unit 101 performs preprocessing on the feature data D supplied from the feature data acquisition unit 100. Here, if a predetermined percentage or more of the number of samples corresponding to a feature included in the feature data D is missing, the preprocessing unit 101 removes the feature from the feature data D. Furthermore, if a predetermined percentage or more of the number of samples corresponding to a feature included in the feature data D is missing, the preprocessing unit 101 removes the sample from the feature data D. Furthermore, the preprocessing unit 101 reduces the dimensions of the feature included in the feature data D based on a feature reduction method appropriate for the field to which the feature selection system 1 is applied. The preprocessing unit 101 supplies the preprocessed feature data D to the numerical feature normalization unit 102 .

[0031] Step S30: The numerical feature normalization unit 102 performs numerical feature conversion and normalization processing on the pre-processed feature data D. In the numerical feature conversion processing, the numerical feature normalization unit 102 converts categorical features into numerical features for the pre-processed feature data D. The numerical feature normalization unit 102 uses, for example, one-hot encoding or label encoding in the processing of converting categorical features into numerical features. The numerical feature normalization unit 102 also performs normalization processing on the feature data D. The numerical feature quantity normalization unit 102 supplies the feature quantity data D that has been subjected to the numerical feature quantity and normalization processes to the principal component analysis unit 103.

[0032] Step S40: The principal component analysis unit 103 performs principal component analysis in sample space on the feature data D supplied from the numerical feature normalization unit 102. As a result of the principal component analysis, the numerical feature normalization unit 102 generates sample space principal components P. The principal component analysis unit 103 supplies the generated sample space principal components P to the distortion determination unit 104 and the feature selection unit 105.

[0033] Step S50: The distortion determination unit 104 determines whether or not there is distortion in the distribution of the sample space principal components P, which are supplied from the principal component analysis unit 103. The distortion determination unit 104 performs this determination on the principal components included in the sample space principal components P, in order from the first principal component.

[0034] In this embodiment, the distortion of the distribution of the sample space principal components P refers to, for example, a deviation of the distribution from a normal distribution. The distortion determination unit 104 makes the determination based on, for example, skewness. The distortion determination unit 104 determines whether the distribution of the sample space principal components P deviates from a normal distribution. If the skewness of the distribution of the sample space principal components P deviates from 0 by a predetermined value, the distortion determination unit 104 determines that the distribution is skewed.

[0035] The distortion determining unit 104 may perform the determination based on kurtosis instead of skewness. The distortion determining unit 104 may perform the determination based on the arithmetic mean or standard deviation. Furthermore, the distortion determining unit 104 may perform the determination based on a combination of one or more of the arithmetic mean, standard deviation, skewness, and kurtosis.

[0036] In this embodiment, the distortion determination unit 104 determines the distortion of the distribution of the sample space principal component P as a deviation of the distribution from a normal distribution, but this is not limiting. The distortion determination unit 104 may make the determination based on the similarity between the distribution of the sample space principal component P and an asymmetric distribution. In this case, if the distribution of the sample space principal component P and the asymmetric distribution are not similar, the distortion determination unit 104 determines that the distribution of the sample space principal component P is not distorted. An asymmetric distribution is, for example, a distribution that is not axisymmetric about a center value.

[0037] The distortion determination unit 104 supplies the determination result of the distortion of the distribution of the sample space principal components P to the feature selection unit 105. Here, in Fig. 4, as an example, it is assumed that the first principal component to the Nth principal component of the sample space principal components P are determined to have distortion. In other words, it is assumed that the N+1th and subsequent principal components of the sample space principal components P are determined to have no distortion.

[0038] Step S60: The feature selection unit 105 selects a feature from among the multiple features included in the feature data D, based on the sample space principal components P supplied from the principal component analysis unit 103 and the determination result supplied from the distortion determination unit 104. Here, the feature selection unit 105 selects a feature that is far from the origin of the sample space for the sample space principal components P that are determined to have no distortion in their distribution.

[0039] The feature selection unit 105 selects feature quantities that are farthest from the origin of the sample space using K components from the (N+1)th principal component to the (N+K)th principal component that have been determined to have no distribution distortion. In other words, the feature selection unit 105 selects feature quantities that are farther from the origin of the sample space than a predetermined distance in a K-dimensional subspace corresponding to the (N+1)th principal component to the (N+K)th principal component of the sample space. Here, the feature selection unit 105 selects M feature quantities in descending order of distance from the origin of the sample space.

[0040] K is an integer equal to or greater than 0. M is an integer equal to or greater than 1. The values of K and M are supplied from the feature data supply unit 20 to the feature selection device 10 together with the feature data D. For example, values specified by a user are supplied from the feature data supply unit 20 as the values of K and M. Note that the feature selection unit 105 may use, as the values of K and M, predetermined values based on, for example, a cluster structure expected for features depending on the field to which the feature selection system 1 is applied.

[0041] The distance from the origin of the feature amount is, for example, the Euclidean distance. However, a distance other than the Euclidean distance may be used as the distance from the origin of the feature amount. The feature quantity selection unit 105 may select feature quantities whose distance from the origin of the sample space is greater than a predetermined distance, without setting an upper limit on the number of feature quantities to be selected in advance.

[0042] Here, features that are a short distance from the origin in the sample space are considered to be features that do not contribute to cluster separation of samples and correspond to noise. The distribution of these features tends to follow a normal distribution. The feature selection device 10 selects features that are a long distance from the origin based on the sample space principal components P, thereby removing features that correspond to noise from the feature data D.

[0043] Step S70: The feature quantity selection unit 105 outputs the feature quantity selection result R indicating the selected M feature quantities to the output unit . With this, the feature selection device 10 ends the feature selection process.

[0044] As described above, the feature selection unit 105 selects feature quantities from among a plurality of feature quantities based on some of the sample space principal components P obtained by principal component analysis in the sample space. While the present embodiment describes an example in which the feature selection unit 105 selects feature quantities based on principal components determined to have no distribution distortion among the sample space principal components P, the present invention is not limited to this. For example, the feature selection unit 105 may remove a predetermined number of principal components, starting with the first principal component, from the sample space principal components P. That is, although the above-described number N is determined based on the distribution distortion among the sample space principal components P in the present embodiment, a predetermined number may be used as the number N.

[0045] An example in which the feature selection system 1 according to this embodiment is applied will be described below. [First Example] In the first embodiment, artificial data D1, which is artificially generated data, is used as the feature data D. FIG. 5 is a diagram showing an example of the artificial data D1 according to this embodiment. The artificial data D1 stores 4,500 feature values for 1,000 samples. In the artificial data D1, five cluster structures are assumed, and the distribution of the features is given varying density. As shown in FIG. 5, the dense portion is given as five rectangles in the range from the first feature to approximately the 500th feature. Background noise is given to the portion other than the five rectangles, i.e., the portion other than the dense portion.

[0046] Before describing the analysis results obtained by the feature selection system 1 of this embodiment, a comparative example of the analysis results is shown. FIG. 6 is a diagram illustrating an example of the results of a cluster analysis performed for comparison with this embodiment. FIG. 6 illustrates the results of dimensionality reduction in a feature space when features are selected using a conventional feature selection technique. Conventional feature selection involves only the process equivalent to the preprocessing described above (step S20 shown in FIG. 4). In this comparative example, feature selection is performed through preprocessing, and 1,000 features are selected from 4,500 features included in the artificial data D1. FIG. 6 illustrates the results of cluster analysis of the data for the 1,000 features selected through preprocessing. In this cluster analysis, markers corresponding to each sample are displayed on a two-dimensional plane obtained by unsupervised dimensionality reduction for visualization. The markers form clusters for each class, and the characteristics of the samples are captured on the two-dimensional plane.

[0047] The following describes in detail the feature selection process performed by the feature selection system 1 of this embodiment, in association with the process shown in FIG. In the preprocessing of step S20, a method used in the analysis of gene expression data (for example, see Non-Patent Document 2) was applied. The preprocessing unit 101 reduces the 4500 features included in the artificial data D1 to 1000 features through the preprocessing.

[0048] In the process of converting the feature values into numerical features and normalizing them in step S30, all the feature values included in the artificial data D1 in this embodiment are numerical features, so there is no need to convert them into numerical features. The numerical feature conversion normalization unit 102 converts the feature values by normalizing them so that the average value of the feature values becomes 0 and the standard deviation becomes 1.

[0049] The results of the sample space principal components P obtained by the principal component analysis in the sample space in step S40 are shown in Figs. 7 and 8. Fig. 7 shows the first and second principal components in the sample space. Fig. 8 shows the third and fourth principal components in the sample space. In Figs. 7 and 8, each point corresponds to a feature amount.

[0050] In step S50, the distortion determination unit 104 determines which of the sample space principal components P deviate from a normal distribution. The distributions of the first and sixth principal components of the sample space principal components P, which are the principal components in the sample space, are shown in FIGS. 9 and 10, respectively. As shown in FIG. 9, the distribution of the first principal component deviates significantly from a normal distribution, indicating a distortion in the distribution. The distributions of the second through fifth principal components (not shown) are also distorted. As shown in FIG. 10, the distribution of the sixth principal component is close to a normal distribution, indicating no distortion in the distribution. The distributions of the seventh and subsequent components (not shown) are also distorted. Principal components of the sample space principal components P that deviate significantly from a normal distribution are excluded and are not used in feature selection. In this embodiment, the first through fifth principal components are excluded.

[0051] In selecting features in step S60, the feature selection unit 105 selects 200 features in descending order of Euclidean distance from the origin of the sample space using the sixth and subsequent principal components (excluding the first to fifth principal components) among the sample space principal components P. As described above, the number of features to be selected is predetermined as a parameter.

[0052] FIG. 11 shows the results of preprocessing (i.e., dimensionality reduction processing) similar to that used to obtain the plot in FIG. 6, using 200 features selected by the feature selection processing described above. FIG. 11 is a diagram showing the results of dimensionality reduction in feature space. Comparing the results using 200 features selected by the feature selection processing by feature selection system 1 (FIG. 11) with the results using 1,000 features selected by conventional feature selection technology (FIG. 6), it can be seen that almost the same cluster structure is obtained. In other words, it can be seen that the feature selection processing by feature selection system 1 is able to reproduce the results obtained by conventional feature selection technology while significantly reducing the dimensionality (number) of features. According to the results of this embodiment, the feature selection system 1 can reduce the dimension (number) of features while maintaining the cluster structure, compared to the results obtained using conventional feature selection techniques, thereby enabling faster calculations without sacrificing analysis accuracy.

[0053] [Second Example] In the second example, genotype data D2 based on whole genome sequencing published in Non-Patent Document 3 is used as feature data D. The genotype data D2 stores 20 million feature values for each of 600 samples.

[0054] Genotype data is data that represents the differences between the bases at each locus and a reference genome. Genotype data is used in research to classify samples (e.g., humans) into disease and non-disease groups and to discover genetic mutations that are specific to the disease group. The genotype data D2 used in this example is not data for two disease-related groups, but rather, in this example, an analysis is performed focusing on the genetic origins of ancestors based on the results of unsupervised dimensionality reduction.

[0055] As a comparison to this embodiment, the results of cluster analysis without feature selection, which is described in Patent Document 4, a conventional technique, are shown in Fig. 12. Fig. 12 is a diagram showing an example of the results of cluster analysis performed as a comparison to this embodiment. Fig. 12 shows the results of unsupervised dimensionality reduction of genotype data D2 into two dimensions in feature space after preprocessing has reduced 20 million features to 100,000 features.

[0056] Each marker represents a population, such as European ancestry, and a subpopulation within each population. Markers form clusters for each population, and the characteristics of the sample are captured on a two-dimensional plane through dimensionality reduction. This dimensionality reduction requires a considerable amount of computation time because it directly uses the 100,000 features obtained by preprocessing the genotype data D2.

[0057] The following describes in detail the feature selection process performed by the feature selection system 1 of this embodiment, in association with the process shown in FIG. In the preprocessing of step S20, the preprocessing unit 101 performs preprocessing on 20 million features included in the genotype data D2. Here, the preprocessing unit 101 removes features that are missing in 20 percent or more of the samples. The preprocessing unit 101 removes samples that have missing features in 20 percent or more of the samples. The preprocessing unit 101 removes features that are missing in 2 percent or more of the samples. The preprocessing unit 101 removes samples that have missing features in 2 percent or more of the samples. The preprocessing unit 101 removes features with a minor allele frequency of 5 percent or less.

[0058] In the process of converting to numerical features and normalizing in step S30, since all features included in the genotype data D2 are categorical features called genotypes, the numerical feature conversion normalization unit 102 converts categorical features into numerical features using label encoding. Note that in this embodiment, feature values are not normalized. The results of the cluster analysis shown in Figure 12 are the same as those obtained when the genotype data D2 is subjected to preprocessing in step S20, and the numerical feature and normalization processes in step S30, and dimension reduction is performed in the feature space.

[0059] The results of the sample space principal component P obtained by the principal component analysis in the sample space in step S40 are shown in FIGS. 13 and 14. FIG. 13 shows the first and second principal components in the sample space. FIG. 14 shows the third and fourth principal components in the sample space. In FIGS. 13 and 14, each point corresponds to a feature. In FIG. 13, the density of points is high in the ranges where the value of the first principal component is near -10 and near +10. The fact that the density of points is high in these ranges in FIG. 13 can be confirmed from the distribution shown in FIG. 15, which will be described later. In FIG. 14, the density of points is high in the ranges where the value of the first principal component is near 0 and the value of the second principal component is near 0, which can be confirmed from the distribution shown in FIG. 16, which will be described later.

[0060] In step S50, the distortion determination unit 104 determines which of the sample space principal components P deviate from a normal distribution. The distributions of the first and fourth principal components of the sample space principal components P, which are the principal components in the sample space, are shown in FIGS. 15 and 16, respectively. As shown in FIG. 15, the distribution of the first principal component deviates significantly from a normal distribution, indicating a distortion in the distribution. The distributions of the second to third principal components (not shown) are also distorted. As shown in FIG. 16, the distribution of the fourth principal component is close to a normal distribution, indicating no distortion in the distribution. The distributions of the fifth and subsequent components (not shown) are also distorted. Principal components of the sample space principal components P that deviate significantly from a normal distribution are excluded and are not used in feature selection. In this embodiment, the first to third principal components are excluded.

[0061] In selecting features in step S60, the feature selection unit 105 selects 1000 features in descending order of Euclidean distance from the origin of the sample space using the fourth and subsequent principal components excluding the first to third principal components from among the sample space principal components P. As described above, the number of features to be selected is predetermined as a parameter.

[0062] FIG. 17 shows the results of preprocessing (i.e., dimensionality reduction) similar to that used to obtain the plot in FIG. 12, using 1,000 features selected by the feature selection process described above. FIG. 17 illustrates the results of dimensionality reduction in feature space. Comparing the results (FIG. 17) using 1,000 features (corresponding to 1 percent of the original 100,000 features contained in genotype data D2) selected by the feature selection process using feature selection system 1 with the results (FIG. 12) using 100,000 features selected by conventional feature selection technology, it can be seen that nearly the same cluster structure is obtained. In other words, the feature selection process using feature selection system 1 is able to reproduce the results obtained by conventional feature selection technology while significantly reducing the dimensionality (number) of features. According to the results of this embodiment, the feature selection system 1 can reduce the dimension (number) of features while maintaining the cluster structure, compared to the results obtained using conventional feature selection techniques, thereby enabling faster calculations without sacrificing analysis accuracy.

[0063] [Third Example] In the third example, human gene expression data D3 published in Non-Patent Document 5 is used as the feature data D. The gene expression data D3 stores 6713 feature values for each of 3694 samples.

[0064] Gene expression data is data in which each feature represents the expression level of a specific (single) gene. In gene expression data, samples correspond to cells. In gene expression data, sample groups are classified into abnormal cell groups and normal cell groups, and are used in research to discover genes that are specifically over- or under-expressed in abnormal cell groups.

[0065] As a comparison to this example, the results of cluster analysis without feature selection, which is described in Patent Document 6, a conventional technique, are shown in Figure 18. Figure 18 is a diagram showing an example of the results of cluster analysis performed as a comparison in this example. Figure 18 shows the results of unsupervised dimensionality reduction of gene expression data D3 into two dimensions in feature space after preprocessing has reduced 6713 features to 2000 features. Each marker represents a cell type. The markers form clusters for each cell type, and the sample characteristics are captured on a two-dimensional plane through dimensionality reduction.

[0066] The following describes in detail the feature selection process performed by the feature selection system 1 of this embodiment, in association with the process shown in FIG. In the preprocessing of step S20, the preprocessing unit 101 performs preprocessing on the 6713 features included in the gene expression data D3. In the preprocessing of step S20, a method used in the analysis of gene expression data (see, for example, Non-Patent Document 2) is applied. Through the preprocessing, the preprocessing unit 101 reduces the 6713 features included in the gene expression data D3 to 2000 features.

[0067] In the process of converting the feature values into numerical features and normalizing them in step S30, all the feature values included in the gene expression data D3 in this embodiment are numerical features, so conversion into numerical features is not necessary. The numerical feature conversion normalization unit 102 converts the feature values by normalization so that the average value of the feature values becomes 0 and the standard deviation becomes 1. The results of the cluster analysis shown in Figure 18 are the same as those obtained when the gene expression data D3 is subjected to preprocessing in step S20, and the numerical feature and normalization processes in step S30, and dimension reduction is performed in the feature space.

[0068] The results of the sample space principal components P obtained by the principal component analysis in the sample space in step S40 are shown in Figs. 19 and 20. Fig. 19 shows the first and second principal components in the sample space. Fig. 20 shows the third and fourth principal components in the sample space. In Figs. 13 and 14, each point corresponds to a feature amount.

[0069] In step S50, the distortion determination unit 104 determines which of the sample space principal components P deviate from a normal distribution. The distributions of the first and fifth principal components of the sample space principal components P, which are the principal components in the sample space, are shown in FIGS. 21 and 22, respectively. As shown in FIG. 21, the distribution of the first principal component deviates significantly from a normal distribution, indicating a distortion in the distribution. The distributions of the second to third principal components (not shown) are also distorted. As shown in FIG. 22, the distribution of the fifth principal component is close to a normal distribution, indicating no distortion in the distribution. The distributions of the fourth and sixth components (not shown) and beyond are also distorted. Principal components of the sample space principal components P that deviate significantly from a normal distribution are excluded and are not used for feature selection. In this embodiment, the first to third principal components are excluded, and the fourth to tenth principal components are used for subsequent processing.

[0070] In selecting features in step S60, the feature selection unit 105 selects 300 features in descending order of Euclidean distance from the origin of the sample space using the fourth to tenth principal components (excluding the first to third principal components) of the sample space principal components P. As described above, the number of features to be selected is predetermined as a parameter.

[0071] Figure 23 shows the results of preprocessing (i.e., dimensionality reduction) similar to that used to obtain the plot in Figure 18, using 300 features selected by the feature selection process described above. Figure 23 illustrates the results of dimensionality reduction in feature space. Comparing the results using 300 features selected by the feature selection process using feature selection system 1 (Figure 23) with the results using 2,000 features selected by conventional feature selection technology (Figure 18), it can be seen that almost the same cluster structure is obtained. In other words, the feature selection process using feature selection system 1 is able to reproduce the results obtained by conventional feature selection technology while significantly reducing the dimensionality (number) of features. Since a cluster structure cannot be obtained when 300 features are directly selected using conventional feature selection technology (not shown), it can be said that feature selection system 1 is capable of preserving a cluster structure even with fewer features. According to the results of this embodiment, the feature selection system 1 can reduce the dimension (number) of features while maintaining the cluster structure, compared to the results obtained using conventional feature selection techniques, thereby enabling faster calculations without sacrificing analysis accuracy.

[0072] [summary] As described above, the feature selection device 10 according to this embodiment includes the feature data acquisition unit 100, the principal component analysis unit 103, and the feature selection unit 105. The feature data acquisition unit 100 acquires feature data D including a set of feature values for each of a plurality of samples. The principal component analysis unit 103 performs principal component analysis on the feature data D in a sample space, which is a collection of feature values for each of the plurality of samples. The feature selection unit 105 selects a feature from among the plurality of feature values based on the result of the principal component analysis performed by the principal component analysis unit 103.

[0073] With this configuration, the feature selection device 10 according to this embodiment can remove noise features and select key features, thereby speeding up calculations without sacrificing analytical accuracy in analyzing high-dimensional feature data. Here, speeding up means that the calculation time can be reduced compared to before the number (dimensionality) of features is reduced.

[0074] The analysis of high-dimensional feature data has the problem that the accuracy of machine learning decreases due to the large number of explanatory variables. In addition, the analysis of high-dimensional feature data has the problem that the amount of calculation required is large, and the analytical calculation takes a huge amount of time. Furthermore, with high-dimensional feature data, the analytical results of cluster analysis and regression analysis become difficult to interpret and explain. The feature selection device 10 according to this embodiment enables analysis using only a small number of key features, thereby shortening the time required for analysis. Since features that cause noise in the analysis can be removed, it is expected that the accuracy of the analysis will improve or knowledge that was not previously available will be obtained. Since the analysis results can be evaluated based on a small number of features, the interpretability and explainability of the analysis will improve.

[0075] By using the feature selection device 10 according to this embodiment, it is possible to narrow down features that are dominant in a specific sample population. The feature selection device 10 is preferably used, for example, to identify marker genes that exhibit a specific function from among a large number of genes.

[0076] The feature selection device 10 according to this embodiment further includes a distortion determination unit 104. The distortion determination unit 104 determines whether or not there is distortion in the distribution of principal components (sample space principal components P in this embodiment) obtained by principal component analysis in the sample space. The feature selection unit 105 selects a feature from among the multiple features based on a principal component (sample space principal components P in this embodiment) obtained by principal component analysis that has been determined to have no distortion in its distribution (the distribution of sample space principal components P in this embodiment).

[0077] With this configuration, the feature selection device 10 of this embodiment can make the distribution characteristics (closeness to a normal distribution) of feature quantities that contribute to noise among multiple feature quantities in a principal component that is determined to have no skew in its distribution more apparent than when selecting principal components without distinguishing between the presence or absence of skew in the distribution, and therefore can select feature quantities without reducing the analysis accuracy compared to when selecting principal components without distinguishing between the presence or absence of skew in the distribution.

[0078] Furthermore, in the feature selection device 10 according to this embodiment, the feature selection unit 105 selects features that are far from the origin of the sample space for principal components whose distribution (in this embodiment, the distribution of the sample space principal component P) is determined to be free of distortion.

[0079] With this configuration, the feature selection device 10 according to this embodiment can exclude features that contribute to noise based on the distance from the origin of the sample space for principal components whose distributions are determined to be free of distortion. This allows for feature selection without reducing analysis accuracy compared to when selection is not based on the distance.

[0080] Note that some of the feature selection device 10 in the above-described embodiment, such as the feature selection device 10, preprocessing unit 101, numerical feature normalization unit 102, principal component analysis unit 103, distortion determination unit 104, feature selection unit 105, and output unit 106, may be implemented by a computer. In this case, a program for implementing this control function may be recorded on a computer-readable recording medium, and the program may be loaded and executed by a computer system. Note that the term "computer system" as used herein refers to a computer system built into the feature selection device 10 and includes hardware such as an OS and peripheral devices. Furthermore, the term "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, as well as storage devices such as hard disks built into a computer system. Furthermore, the term "computer-readable recording medium" may also include devices that dynamically store programs for a short period of time, such as communication lines used when transmitting programs via networks such as the Internet or communication lines such as telephone lines, and devices that store programs for a fixed period of time, such as volatile memory within computer systems that serve as servers or clients in such cases. Furthermore, the above program may be one that realizes part of the functions described above, or may be one that can realize the functions described above in combination with a program already recorded in the computer system. Furthermore, part or all of the feature selection device 10 in the above-described embodiment may be realized as an integrated circuit such as an LSI (Large Scale Integration). Each functional block of the feature selection device 10 may be individually implemented as a processor, or part or all of the blocks may be integrated into a processor. The integrated circuit implementation method is not limited to LSI, and may be implemented using a dedicated circuit or a general-purpose processor. Furthermore, if an integrated circuit implementation technology that can replace LSI emerges due to advances in semiconductor technology, an integrated circuit based on that technology may be used.

[0081] One embodiment of the present invention has been described in detail above with reference to the drawings, but the specific configuration is not limited to that described above, and various design changes and the like are possible within the scope that does not deviate from the gist of the present invention. [Explanation of symbols]

[0082] 10... feature quantity selection device, 100... feature quantity data acquisition unit, 103... principal component analysis unit, 105... feature quantity selection unit, D... feature quantity data

Claims

1. a feature data acquisition unit that acquires feature data including a set of values of a plurality of feature amounts for each of the plurality of samples; a principal component analysis unit that performs principal component analysis on the feature data in a sample space that is a collection of sets of values for each of the plurality of samples of the feature; a distortion determination unit that determines whether or not there is distortion in the distribution of principal components obtained by the principal component analysis in the sample space; a feature selection unit that selects a feature from the plurality of feature amounts based on a principal component determined to have no distortion in distribution among the principal components obtained by the principal component analysis performed by the principal component analysis unit; A feature selection device comprising:

2. The feature selection unit selects a feature that is a large distance from the origin of the sample space for a principal component that is determined to have no distortion in the distribution. The feature selection device according to claim 1 .

3. a feature data acquisition step of acquiring feature data including a set of values of a plurality of feature amounts for each of the plurality of samples; a principal component analysis step of performing principal component analysis on the feature data in a sample space that is a collection of sets of values for each of the plurality of samples of the feature; a distortion determination step of determining whether or not there is distortion in the distribution of principal components obtained by the principal component analysis in the sample space; a feature selection step of selecting a feature from the plurality of feature values based on a principal component determined to have no distortion in distribution among the principal components obtained by the principal component analysis performed in the principal component analysis step; A feature selection method implemented by a computer.

4. On the computer, a feature data acquisition step of acquiring feature data including a set of values of a plurality of feature amounts for each of the plurality of samples; a principal component analysis step of performing principal component analysis on the feature data in a sample space that is a collection of sets of values for each of the plurality of samples of the feature; a distortion determination step of determining whether or not there is distortion in the distribution of principal components obtained by the principal component analysis in the sample space; a feature selection step of selecting a feature from the plurality of feature values based on a principal component determined to have no distortion in distribution among the principal components obtained by the principal component analysis performed in the principal component analysis step; A program to execute.

Images