Data analysis device, data analysis method and data analysis program

Abstract

This data analysis device (10) has a Gram matrix calculation unit (11) that calculates a Gram matrix of data, a Gram matrix norm calculation unit (12) that calculates a norm for each row of the Gram matrix, a snapshot acquisition unit (13) that acquires parameters during optimization, a parameter addition calculation unit (14) that adds parameters to an optimization problem, and an addition number count calculation unit (15) that counts the number of parameter additions.

Description

[Technical Field]

[0001] The present invention relates to a data analysis device, a data analysis method, and a data analysis program. [Background technology]

[0002] Traditionally, feature selection in data mining is a group of methods for selecting important features from data, and is one of the basic techniques. In feature selection, data features can have a group structure. For example, weather data for each region can be considered as data where each region corresponds to a group, and each group contains features such as "temperature," "humidity," "weather," and "wind direction."

[0003] In data with such a group structure, it is possible to analyze the data by selecting groups of important features (for example, groups corresponding to regions). Another typical example of data with groups of features is gene expression data.

[0004] A representative method for selecting feature groups is the Group Lasso, which is a method based on linear regression and can handle feature groups by imposing group constraints on the coefficients of a linear regression model.

[0005] Specifically, in Group Lasso, the features of the data correspond to the coefficients of the linear regression, and the coefficients are also grouped according to the feature groups. In this case, an optimization problem for estimating the coefficients can be solved using a solver, but if all of the grouped coefficients are zero, the feature group corresponding to that group does not contribute to the prediction and can be considered an unimportant feature group. In this way, Group Lasso finds important and unimportant feature groups.

[0006] However, Group Lasso has theoretical drawbacks in that the magnitude of the estimated coefficients may be biased and the consistency of the selected feature groups may not be guaranteed.

[0007] A method called Group SCAD was proposed to overcome this drawback. The theoretical drawback of Group Lasso is that the regularization term (penalty term) for the coefficients is a convex function, so Group SCAD overcomes this theoretical drawback by using a non-convex function as the regularization term. Group SCAD also selects important feature groups by solving an optimization problem. [Prior art documents] [Non-patent literature]

[0008] [Non-Patent Document 1] Wang, L., Chen, G. and Li, H., "Group SCAD regression analysis for microarray time course gene expression data.", Bioinform. 23(12): 1486-1494. [Non-patent document 2] Breheny, P. and Huang, J., "Group descent algorithms for nonconvex penalized linear and logistic regression models with grouped predictors.", Stat. Comput. 25(2): 173-187. Summary of the Invention [Problem to be solved by the invention]

[0009] However, conventional techniques have the problem that Group SCAD cannot be executed quickly on large-scale data. For example, when the size of the optimization problem executed by Group SCAD is large, the solver processing becomes heavy and the calculation becomes slow. In other words, it takes a long time to obtain results for large-scale data with a large number of feature groups. In addition, because Group SCAD has a complex regularization term that is non-convex and has a group structure, few speed-up methods have been proposed.

[0010] The present invention has been made in view of the above, and aims to provide a data analysis device, a data analysis method, and a data analysis program that can execute Group SCAD on large-scale data at high speed. [Means for solving the problem]

[0011] In order to solve the above-mentioned problems and achieve the object, the data analysis device of the present invention is a data analysis device that selects important feature groups from data whose features have a group structure, and is characterized by having: a Gram matrix calculation unit that calculates a Gram matrix of the data; a Gram matrix norm calculation unit that calculates the norm for each row of the Gram matrix calculated by the Gram matrix calculation unit; a snapshot acquisition unit that acquires parameters during optimization; a parameter addition calculation unit that adds the parameters acquired by the snapshot acquisition unit to the optimization problem; and an addition count calculation unit that counts the number of times parameters added by the parameter addition calculation unit are added. [Effects of the Invention]

[0012] According to the present invention, it becomes possible to execute Group SCAD on large-scale data at high speed. [Brief explanation of the drawings]

[0013] [Figure 1] FIG. 1 is a block diagram illustrating an example of a configuration of a data analysis apparatus according to an embodiment. [Figure 2]FIG. 2 is a block diagram illustrating an example of the configuration of a parameter addition calculation unit according to the embodiment. [Figure 3] FIG. 3 is a diagram showing an algorithm used by the data analysis apparatus shown in FIG. [Figure 4] FIG. 4 is a diagram showing an algorithm used by the data analysis apparatus shown in FIG. [Figure 5] FIG. 5 is a flowchart showing a processing procedure of data analysis according to the embodiment. [Figure 6] FIG. 6 is a diagram illustrating an example of a computer that executes a data analysis program. DETAILED DESCRIPTION OF THE INVENTION

[0014] Hereinafter, with reference to the drawings, embodiments of a data analysis device, a data analysis method, and a data analysis program according to the present application will be described in detail. Note that the present invention is not limited to these embodiments. In addition, in the description of the drawings, identical parts are denoted by the same reference numerals, and duplicated explanations will be omitted.

[0015] In the following, when "A^{_}" is written for A, which is a vector, matrix, or scalar, it is equivalent to "the symbol with "_" written directly above "A"." When "A_{_}" is written, it is equivalent to "the symbol with "_" written directly below "A"." Furthermore, when "~A" is written for A, which is a vector, matrix, or scalar, it is equivalent to "the symbol with "~" written directly above "A"." Furthermore, when A is a vector or matrix, AT represents the transpose of A.

[0016] [Conventional mathematical background] First, we will explain Group SCAD as background knowledge necessary for the following explanation.

[0017] Since Group SCAD is based on the linear regression model, we consider a linear regression problem. Let n be the number of data, and each data is represented by p-dimensional features. This allows the data to be represented by a matrix X∈R^{n×p}.

[0018] Since linear regression is a problem of predicting a response for each data, the response can be expressed as a vector y∈R^n with the number of dimensions equal to the number of data. Since linear regression makes predictions by calculating the dot product between the data and a coefficient vector, this coefficient vector is expressed as β∈R^{p}.

[0019] Under the above settings, Group SCAD extracts a group of important features by solving the optimization problems shown in the following equations (1) and (2).

[0020]

number

[0021]

number

[0022] In equations (1) and (2), X^(g)∈R^{n×pg} is a submatrix of matrix X, and pg is the size of the feature of the gth group. Similarly, β^{g} is the coefficient of the gth group. G represents the total number of groups. γ and λ are hyperparameters that are manually tuned.

[0023] There are various solvers that can be used to solve the optimization problems of equations (1) and (2), but a method called block coordinate descent (BCD) is known to be fast. The algorithm for this method is shown in Figure 3. Here, F(·,·,·) and z^(g) in Figure 3 are calculated using the following equations (3) and (4), respectively.

[0024]

number

[0025]

number

[0026] A in Figure 3 is the set of indexes for the group {1,...,G}. The function S(·,·) in equation (3) is calculated for the argument as in equation (5).

[0027]

number

[0028] [Mathematical background of this embodiment] Next, the mathematical background of this embodiment will be explained. In this embodiment, feature groups are divided into high, medium, low, and other groups according to their importance. First, β^(g) corresponding to the high-importance groups is optimized, and then the medium-importance groups are added to the optimization problem and optimized.

[0029] Then, add the less important groups to the problem and optimize them, and finally add the other groups and optimize the whole. By doing this, you can initially optimize only the number of small groups with high importance, avoiding optimizing all groups, so you can expect it to run quickly.

[0030] First, groups of high, medium, and low importance are extracted using equations (6), (7), and (8).

[0031]

number

[0032]

number

[0033]

number

[0034] Here, z^{_}^(g) is the upper bound of z^(g), and z_{_}^(g) is the lower bound of z^(g). These upper and lower bounds are calculated using equations (9) and (10), respectively.

[0035]

number

[0036]

number

[0037] Here, the Gram matrix of the data is K = XTX∈R^{p×p}. Λ(·,·) is calculated as shown in Equation (11).

[0038]

number

[0039] ~z^(g) and ~β^(l) are the values ​​(snapshots) corresponding to z^(g) and β^(l), respectively. These values ​​are updated at regular intervals during the iteration of the block coordinate descent.

[0040] The i-th element of ^K^(g)[l]∈R^(pg) is calculated as the L2 norm ||K(g,l)[i;]||2 of the i-th row when K(g,l)∈R^{pg×pl} is a submatrix of K. [Present embodiment]

[0041] The following describes a data analysis device 10 according to the present embodiment. The data analysis device 10 according to the present embodiment is a learning device for a linear regression model that uses Group SCAD to extract groups of important features from large-scale data.

[0042] 1 is a block diagram showing an example of the configuration of a data analysis apparatus 10 according to an embodiment. As shown in FIG. 1, the data analysis apparatus 10 according to the present embodiment includes a Gram matrix calculation unit 11, a Gram matrix norm calculation unit 12, a snapshot acquisition unit 13, a parameter addition calculation unit 14, an addition count calculation unit 15, and a solver unit 16.

[0043] The data analysis device 10 is realized, for example, by loading a predetermined program into a computer including a ROM (Read Only Memory), RAM (Random Access Memory), CPU (Central Processing Unit), etc., and the CPU executing the predetermined program.

[0044] The Gram matrix calculation unit 11 calculates the Gram matrix of the data. For example, the Gram matrix calculation unit 11 calculates the Gram matrix K=XTX∈R^{p×p} of the formulas (9) and (10).

[0045] The Gram matrix norm calculation unit 12 calculates the norm for each row of the Gram matrix calculated by the Gram matrix calculation unit 11. For example, the Gram matrix norm calculation unit 12 calculates the L2 norm ||K(g,l)[i;]||2 used in equations (9) and (10) based on the Gram matrix (lines 2-6 in FIG. 3).

[0046] The snapshot acquisition unit 13 acquires parameters during optimization. For example, the snapshot acquisition unit 13 acquires snapshots ~z^(g) and ~β^(l) of equations (9) and (10). These are acquired at regular intervals during the execution of the block coordinate descent solver, and may be acquired in a format such as that shown in FIG. 4.

[0047] The parameter addition calculation unit 14 adds the parameters acquired by the snapshot acquisition unit 13 to the optimization problem. For example, the parameter addition calculation unit 14 is a part that determines whether an arbitrary β^(l) is a parameter with high, medium, low, or other importance using equations (6), (7), and (8). The parameter addition calculation unit 14 first extracts parameters with high importance, adds them to the optimization target, and moves on to the next part. The parameter addition calculation unit 14 subsequently executes this process in a loop in order of importance.

[0048] 2 shows a block diagram of the parameter addition calculation unit 14. The parameter addition calculation unit 14 has a count determination unit 141, a first parameter (importance: high) calculation unit 142, a second parameter (importance: medium) calculation unit 143, a third parameter (importance: low) calculation unit 144, and a fourth parameter (importance: other) calculation unit 145.

[0049] The parameter addition calculation unit 14 adds the parameters to the optimization problem in descending order of importance, which are sorted according to the number of times the parameters have been added, counted by the addition number count calculation unit 15.

[0050] The count determination unit 141 branches the process depending on the number of times a parameter is added. The first parameter calculation unit 142 calculates parameters with high importance. The second parameter calculation unit 143 calculates parameters with medium importance. The third parameter calculation unit 144 calculates parameters with low importance.

[0051] The fourth parameter calculation unit 145 calculates other parameters, for example, parameters that could not be classified by importance.

[0052] Returning to the description of the data analysis device 10 using FIG. 1 again, the addition count calculation unit 15 counts the number of times a parameter is added by the parameter addition calculation unit 14. For example, the addition count calculation unit 15 increments the count by 1 when a parameter is added to the optimization target by the parameter addition calculation unit 14. The variable step in the algorithm diagram corresponds to this count. For example, the addition count calculation unit 15 assigns the importance of the parameter extracted by the parameter addition calculation unit 14 according to this count number. Specifically, the addition count calculation unit 15 assigns the importance of the parameter as follows: 0 indicates high importance, 1 indicates medium importance, 2 indicates low importance, and 3 indicates other.

[0053] The solver unit 16 optimizes the parameters added by the parameter addition calculation unit 14. For example, the solver unit 16 optimizes the parameters to be optimized that were added by the parameter addition calculation unit 14 and the parameters that have been added up to that point. At this time, since it is necessary to acquire snapshots to be used in equations (9) and (10) during optimization, a possible embodiment is, for example, block coordinate descent, which can acquire snapshots, as shown in FIG. 4. The solver unit 16 also determines whether optimization has been performed for all parameters.

[0054] The data analysis device 10 ends the process when the count reaches 3 and all parameters have been optimized by the solver unit 16 .

[0055] [Processing flow] Next, we will explain the algorithm used by data analysis apparatus 10 and the flow of processing executed by data analysis apparatus 10. Figures 3 and 4 are diagrams showing the algorithm used by data analysis apparatus 10 shown in Figure 1. Figure 5 is a flowchart showing the processing procedure of a data analysis method according to an embodiment.

[0056] First, the Gram matrix calculation unit 11 calculates the Gram matrix of the data (step S11). Next, the Gram matrix norm calculation unit 12 calculates the norm for each row of the Gram matrix of the given data (step S12). Then, the snapshot acquisition unit 13 acquires the parameters being optimized (step S13).

[0057] Next, the parameter addition calculation unit 14 adds the parameters acquired by the snapshot acquisition unit 13 to the optimization problem (step S14). Then, the addition count calculation unit 15 counts the number of times the parameters added by the parameter addition calculation unit 14 are added (step S15).

[0058] Next, the solver unit 16 performs optimization on the parameters added by the parameter addition calculation unit 14 (step S16). Here, the solver unit 16 determines whether or not all parameters have been optimized (step S17).

[0059] At this time, if the solver unit 16 determines that all parameters have not been optimized (step S17 "NO"), the processes of steps S14 to S16 are repeated for the next parameter.

[0060] Then, if the solver unit 16 determines that all parameters have been optimized ("YES" in step S17), the data analysis apparatus 10 ends this flow.

[0061] [Effects of the embodiment] As described above, data analysis device 10 according to this embodiment is a learning device for a linear regression model that uses Group SCAD to extract groups of important features from large-scale data.

[0062] Furthermore, the data analysis device 10 is expected to solve the optimization problem efficiently and speed up by sequentially optimizing parameters starting with the most important, rather than optimizing all parameters from the beginning as in conventional Group SCAD. This is because the data analysis device 10 can optimize parameters with high importance in a way that significantly contributes to prediction and significantly improves the value of the objective function. Furthermore, because the data analysis device 10 is an apparatus that ultimately optimizes all parameters, there is almost no degradation in accuracy. Therefore, according to this embodiment, Group SCAD can be executed accurately and quickly.

[0063] [System configuration of the embodiment] 1 are functional concepts and do not necessarily have to be physically configured as shown. In other words, the specific form of distribution and integration of the functions of data analysis apparatus 10 is not limited to that shown, and all or part of the functions can be functionally or physically distributed or integrated in any unit depending on various loads, usage conditions, etc.

[0064] Furthermore, all or any part of the processes performed in data analysis device 10 may be realized by a CPU and a program analyzed and executed by the CPU. Furthermore, each process performed in data analysis device 10 may be realized as hardware using wired logic.

[0065] Furthermore, among the processes described in the embodiments, all or part of the processes described as being performed automatically can be performed manually. Alternatively, all or part of the processes described as being performed manually can be performed automatically using a known method. In addition, the processing procedures, control procedures, specific names, and information including various data and parameters described above and illustrated can be changed as appropriate unless otherwise specified.

[0066] [program] 6 is a diagram showing an example of a computer in which a program is executed to realize data analysis apparatus 10. Computer 1000 includes, for example, memory 1010 and CPU 1020. Computer 1000 also includes hard disk drive interface 1030, disk drive interface 1040, serial port interface 1050, video adapter 1060, and network interface 1070. These components are connected by bus 1080.

[0067] The memory 1010 includes a ROM 1011 and a RAM 1012. The ROM 1011 stores a boot program such as a BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to a hard disk drive 1090. The disk drive interface 1040 is connected to a disk drive 1100. A removable storage medium such as a magnetic disk or optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to a mouse 1110 and a keyboard 1120, for example. The video adapter 1060 is connected to a display 1130, for example.

[0068] The hard disk drive 1090 stores, for example, an OS 1091, an application program 1092, a program module 1093, and program data 1094. That is, a program that defines each process of the data analysis apparatus 10 is implemented as a program module 1093 in which code executable by the computer 1000 is written. The program module 1093 is stored, for example, in the hard disk drive 1090. For example, a program module 1093 for executing the same process as the functional configuration of the data analysis apparatus 10 is stored in the hard disk drive 1090. The hard disk drive 1090 may be replaced by an SSD (Solid State Drive).

[0069] Furthermore, setting data used in the processing of the above-described embodiment is stored as program data 1094, for example, in memory 1010 or hard disk drive 1090. Then, CPU 1020 reads program module 1093 and program data 1094 stored in memory 1010 or hard disk drive 1090 into RAM 1012 as necessary and executes them.

[0070] The program module 1093 and program data 1094 are not limited to being stored in the hard disk drive 1090, but may also be stored in, for example, a removable storage medium and read by the CPU 1020 via the disk drive 1100 or the like. Alternatively, the program module 1093 and program data 1094 may be stored in another computer connected via a network (such as a local area network (LAN) or a wide area network (WAN)). The program module 1093 and program data 1094 may then be read by the CPU 1020 from the other computer via the network interface 1070.

[0071] Although the present invention has been described above as an embodiment, the present invention is not limited to the descriptions and drawings that form part of the disclosure of the present invention. In other words, other embodiments, examples, and operational techniques that can be made by those skilled in the art based on the present invention are all included in the scope of the present invention. [Explanation of symbols]

[0072] 10 Data analysis equipment 11 Gram matrix calculation unit 12 Gram matrix norm calculation unit 13 Snapshot acquisition section 14 Parameter addition calculation section 141 Count determination unit 142 First parameter calculation unit 143 Second parameter calculation unit 144 Third Parameter Calculation Unit 145 Fourth Parameter Calculation Unit 15 Addition count calculation section 16 Solver section

Claims

1. A data analysis device for selecting important feature groups from data having a group structure of feature quantities, a Gram matrix calculation unit for calculating a Gram matrix of the data; a Gram matrix norm calculation unit that calculates a norm for each row of the Gram matrix calculated by the Gram matrix calculation unit; a snapshot acquisition unit that acquires parameters during optimization; a parameter addition calculation unit that adds the parameters acquired by the snapshot acquisition unit to the optimization problem; an addition count calculation unit that counts the number of parameter additions added by the parameter addition calculation unit; A data analysis device comprising:

2. 2. The data analysis device according to claim 1, further comprising a solver unit that performs optimization on the parameters added by the parameter addition calculation unit.

3. 2. The data analysis apparatus according to claim 1, wherein the parameter addition calculation unit adds the parameters to the optimization problem in descending order of importance, the parameters being sorted according to the number of times the parameters have been added, counted by the addition count calculation unit.

4. A data analysis method executed by a data analysis device for selecting an important feature group from data having a group structure in feature quantities, comprising: a Gram matrix calculation step for calculating a Gram matrix of the data; a Gram matrix norm calculation step of calculating a norm for each row of the Gram matrix calculated by the Gram matrix calculation step; a snapshot acquisition step for acquiring parameters during optimization; a parameter addition calculation step of adding the parameters acquired by the snapshot acquisition step to the optimization problem; an addition count calculation step of counting the number of parameter additions added by the parameter addition calculation step; A data analysis method comprising:

5. A data analysis program for causing a computer to function as the data analysis device according to any one of claims 1 to 3.

Images