Data processing device
The data processing device accelerates feature selection in large datasets with tree structures by pruning unnecessary calculations, ensuring fast and accurate identification of important features.
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- NT T INC
- Filing Date
- 2024-11-26
- Publication Date
- 2026-06-04
AI Technical Summary
Tree lasso calculations become slow when dealing with large datasets, leading to prolonged processing times in feature selection for data with a tree structure.
A data processing device that includes a first calculation unit for row-by-row norm of the Gram matrix, a second unit for parameter norm, a third unit for upper bound calculation of node norms, an update unit for pruning unnecessary calculations, and a determination unit for convergence, to speed up the feature selection process by eliminating unnecessary calculations.
The device quickly selects important features from large-scale data with a tree structure, reducing overall computation time and maintaining accuracy by pruning unnecessary nodes in the tree structure.
Smart Images
Figure JP2024041884_04062026_PF_FP_ABST
Abstract
Description
Data processing device
[0001] This invention relates to a data processing device.
[0002] Feature selection is a group of techniques for selecting important features from data, and it is one of the fundamental techniques in data mining.
[0003] In feature selection, data features often have a tree structure. For example, Japanese weather data with features such as "temperature," "humidity," "weather," and "wind direction" can be divided at the "city / town / village" level, but it can also be divided at the higher level of "prefecture," and even further at the "seven regional divisions." This indicates that a hierarchical tree structure exists in the weather data between the seven regional divisions, prefectures, and cities / towns / villages.
[0004] As an example of a tree structure, under "Kanto Region" there are "Ibaraki Prefecture, Saitama Prefecture...Kanagawa Prefecture," and under "Saitama Prefecture" there are "Saitama City, Tokorozawa City...Honjo City." In a tree structure, "Kanto Region," "Saitama Prefecture," and "Saitama City" are called nodes. In data with such a tree structure of features, data analysis is sometimes performed by selecting important nodes and the features corresponding to these nodes (for example, features such as "temperature" and "wind direction" for the "Saitama Prefecture" node).
[0005] A representative method for selecting this feature group is tree lasso (Non-Patent Document 1). Tree lasso is a method based on linear regression, and by introducing a tree structure similar to that of features into the coefficients of the linear regression model, it is possible to handle tree-structured features.
[0006] Specifically, in tree lasso, there is a correspondence between the data features and the coefficients of the linear regression, and the coefficients have a tree structure similar to the tree structure of the features. In this case, tree lasso optimizes the coefficients corresponding to unnecessary nodes in the tree structure to zero. Since coefficients that become zero do not contribute to prediction or learning, the corresponding features can be considered unimportant features.
[0007] Thus, according to tree lasso, it is possible to identify unnecessary and important nodes within a tree structure and find the corresponding important and unimportant features.
[0008] J. Liu and J. Ye, “Moreau-Yosida Regularization for Grouped Tree Structure Learning”, In Advances in Neural Information Processing Systems (NeurIPS), pages 1459-1467, 2010.
[0009] However, there is a problem in that tree lasso calculations become slow when the size of the optimization problem is large. In other words, with large datasets, there is a problem in that it takes a long time to obtain the results when calculating tree lasso.
[0010] The present invention has been made in view of the above, and aims to provide a data processing device that can quickly select important features from data having a tree structure in its features.
[0011] To solve the above-mentioned problems and achieve the objective, the data processing device according to the present invention is a data processing device that selects features from data having a tree structure for its features, and is characterized by comprising: a first calculation unit that calculates the row-by-row norm of the Gram matrix of the given data; a second calculation unit that obtains the norm of the parameters being optimized; a third calculation unit that calculates an upper bound of the norm of the parameters corresponding to the nodes of the tree structure; an update unit that deletes unnecessary calculations among the calculations corresponding to the nodes of the tree structure and updates the parameters based on the calculation result of the upper bound by the third calculation unit; an assignment unit that assigns momentum to the parameters; and a determination unit that determines the convergence of the optimization based on the parameters.
[0012] According to the present invention, important features can be selected quickly from data that has a tree structure for its features.
[0013] Figure 1 is a block diagram showing an example of the configuration of a data processing device according to an embodiment. Figure 2 is a diagram illustrating the definition of an index tree. Figure 3 is a diagram illustrating an example of an index tree. Figure 4 is a diagram showing the algorithm of the FISTA (Fast Iterative Shrinkage Thresholding Algorithm) method. Figure 5 is a diagram showing the proximity mapping algorithm shown in Figure 4. Figure 6 is a diagram showing an example of an algorithm used by the data processing device shown in Figure 1. Figure 7 is a diagram showing an example of an algorithm used by the data processing device shown in Figure 1. Figure 8 is a flowchart showing the processing procedure of the data processing method according to an embodiment. Figure 9 is a diagram showing the number of calculations per depth using tree lasso and its total. Figure 10 is a diagram showing the number of calculations at each node per iteration in tree lasso. Figure 11 is a diagram showing an example of a computer in which a data processing device is realized when a program is executed.
[0014] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. However, the present invention is not limited to this embodiment. Furthermore, in the drawings, identical parts are denoted by the same reference numerals.
[0015] In the following, when "￣A" is written for a vector, matrix, or scalar A, it is equivalent to "the symbol "￣" placed directly above "A". Also, when "^A" is written for a vector, matrix, or scalar A, it is equivalent to "the symbol "^" placed directly above "A". Furthermore, for a vector or matrix A, A Ｔ This represents the transpose of A.
[0016] [Embodiment] First, this embodiment will be described. The data processing device according to this embodiment is a learning device for a linear regression model that uses tree lasso to select important features from large-scale data with a tree structure. Examples of large-scale data include weather data stratified by region, genetic data stratified by function, and sensor data from factories, etc., stratified by time. For example, the data processing device selects characteristic points from time-series sensor data to extract the causes of a decline in production volume at a chemical plant, performs data analysis, and supports a PDCA cycle that provides feedback for countermeasures.
[0017] The data processing unit 10 reduces the overall computation time and speeds up the process by pruning unnecessary nodes in the tree structure with minimal computation during tree lasso optimization. In a tree structure, the terminal nodes are called leaf nodes, and the other nodes are called internal nodes.
[0018] Figure 1 is a block diagram showing an example of the configuration of a data processing device according to an embodiment. The data processing device 10 according to this embodiment shown in Figure 1 is a tree lasso data processing device that selects important features from data that has a tree structure in its features.
[0019] The data processing device 10 includes a Gram matrix norm calculation unit 11 (first calculation unit), a parameter norm calculation unit 12 (second calculation unit), a node upper bound calculation unit 13 (third calculation unit), a node calculation pruning unit 14 (update unit), a momentum calculation unit 15 (assignment unit), and a convergence determination calculation unit 16 (determination unit).
[0020] The data processing device 10 is realized, for example, when a predetermined program is loaded into a computer including ROM (Read Only Memory), RAM (Random Access Memory), CPU (Central Processing Unit), etc., and the CPU executes the predetermined program.
[0021] The Gram matrix norm calculation unit 11 calculates the norm for each row of the Gram matrix of the given data.
[0022] The parameter norm calculation unit 12 obtains the norm of the parameter being optimized.
[0023] The node upper bound calculation unit 13 calculates an upper bound on the parameter norm corresponding to the node in the tree structure.
[0024] The node calculation pruning unit 14 deletes unnecessary calculations among the calculations corresponding to the nodes in the tree structure and updates the parameters, based on the results of the upper bound calculation by the node upper bound calculation unit 13.
[0025] The node calculation pruning unit 14 determines, based on the calculation results of the upper bounds of the node norms, that the calculations of leaf nodes whose upper bounds are below a predetermined threshold are unnecessary gradient calculations and delete them. The node calculation pruning unit 14 determines, based on the calculation results of the upper bounds of the node norms, that the calculations of internal nodes whose upper bounds are below a predetermined threshold are unnecessary proximity mapping calculations and delete them. The node calculation pruning unit 14 calculates the upper bounds of internal nodes by summing the upper bounds already calculated for the leaf nodes among the calculation results of the node upper bounds.
[0026] The momentum calculation unit 15 assigns momentum to the parameters.
[0027] The convergence determination calculation unit 16 determines the convergence of the optimization based on the parameters.
[0028] In conventional methods, the gradients necessary for optimization are calculated for all leaf nodes. In contrast, the data processing device 10 identifies and eliminates unnecessary gradient calculations by performing an upper bound calculation.
[0029] Furthermore, in conventional methods, a calculation called a proximity mapping is performed for all internal nodes. In contrast, the data processing device 10 identifies and removes unnecessary proximity mappings by calculating upper bounds, and calculates upper bounds for internal nodes by summing the upper bounds already calculated for leaf nodes. This allows for the fast calculation of upper bounds and the elimination of unnecessary proximity mapping calculations.
[0030] In this way, the data processing device 10 reduces the overall computation time and speeds up the process by pruning unnecessary nodes in the tree structure with minimal computational cost during tree lasso optimization.
[0031] [Mathematical Background] Here, as background knowledge, tree lasso will be explained.
[0032] Since the basis of tree lasso is a linear regression model, consider the linear regression problem. Let n be the number of data, and assume that each data is represented by a p-dimensional feature. Thus, the data can be represented by a matrix X ∈ R n×p .
[0033] Since linear regression is a problem of predicting a response for each data, the response can be represented as a vector y ∈ R n of the dimension of the number of data. Since linear regression makes predictions by calculating the inner product of the data and the coefficient vector, this coefficient vector is represented as β ∈ R p .
[0034] The tree structure in the features is defined using an index tree. FIG. 2 is a diagram illustrating the definition of the index tree.
[0035] As shown in Definition 1 of FIG. 2, in an index tree T of depth d, T i = {G i 1, G i 2, …, G i ni} includes all the nodes corresponding to depth i (where n0 = 1, G 0 1 = {1, 2, …, p} and n i ≥ 1, i = 1, 2, …, d). Each node G i j corresponds to a subset of {1, 2, …, p}. The nodes satisfy the following first condition. The first condition is that the indices of the nodes at the same depth level do not overlap.
[0036] FIG. 3 is a diagram explaining an example of the index tree.
[0037] Under the above settings, tree lasso extracts groups of important features by solving the optimization problems shown in the following equations (1) and (2).
[0038]
[0039]
[0040] In equations (1) and (2), ||・||2 is the L2 norm, and β∈R p λ is the parameter vector, λ>0 is the regularization constant, and φ(β) is the sparsity-induced regularization term of β. i j β[G] is a hyperparameter and is subject to manual tuning. i j ] is the coefficient vector corresponding to the j-th node at the i-th depth in the i-th depth of the i-th node in the i-th node of the i-th node.
[0041] A representative method for solving the optimization problems given by equations (1) and (2) is FISTA. Figure 4 shows the algorithm of the FISTA method. As shown in algorithm 1 of Figure 4, FISTA consists of parameter updates (4th row) and linear combination of parameters (momentum calculation) (5th to 6th rows).
[0042] Here, in Figure 4, t represents the number of FISTA iterations. Figure 4 prox φ λη The (•) (fourth line of Algorithm 1) is called a proximity map. The data processing device 10 calculates the proximity map using Algorithm 2 shown in Figure 5. Figure 5 is a diagram showing the algorithm for the proximity map shown in Figure 4.
[0043] The proximity mapping of Algorithm 2 in Figure 5 takes the argument from the fourth line of Algorithm 1 (Figure 4) as argument v. In the calculation of Algorithm 2, calculations are performed for all nodes of the index tree from the leaf nodes to the root node (lines 3 and 4). If the value of the norm calculated from the argument in line 5 of Algorithm 2 is less than or equal to the threshold, that node is set to zero in line 6 of Algorithm 2 and treated as an unnecessary feature. In other cases (line 7), calculations are performed for non-zero elements (line 8).
[0044] Algorithm 1's ∇f(^β t ), that is, the gradient is calculated by the following equation (3).
[0045]
[0046] Next, the updated parameter vector is the proximity map prox φ λη (・) is passed to β t+1 This is the result (fourth line of algorithm 1).
[0047] Based on the above, the fourth line of FISTA in algorithm 1 performs gradient calculation and coefficient updates using proximity mapping.
[0048] Line 6 of Algorithm 1 is a calculation called momentum calculation, which accelerates optimization by taking a weighted sum of past and current coefficients. Line 5 of Algorithm 1 calculates the weights for this momentum calculation. In Algorithm 1, the above process is repeated until the optimization converges.
[0049] [Mathematical Background of the Embodiment] Next, the mathematical background of this embodiment will be explained. This embodiment performs pruning between leaf nodes and internal nodes of an index tree.
[0050] In this embodiment, unnecessary gradient calculations (the fourth line of Algorithm 1) are identified and removed by pruning unnecessary leaf nodes using upper bound calculations. Furthermore, in this embodiment, unnecessary proximity mapping calculations (Algorithm 2) are removed by pruning unnecessary internal nodes using upper bound calculations. As a result, tree lasso is accelerated in this embodiment.
[0051] Furthermore, in this embodiment, the upper bounds of internal nodes can be calculated by summing the upper bounds already calculated for leaf nodes, thereby enabling even faster calculation of upper bounds.
[0052] [Pruning of leaf nodes] First, let's explain pruning of leaf nodes. In Algorithm 2 (Figure 5), leaf nodes are u d+1 This corresponds to Algorithm 2 (Figure 5), from the first row, u d+1 The argument on the fourth line of algorithm 1 is copied directly. From equation (3), this u d+1 When calculated, we get equation (4). In the t-th iteration of algorithm 1, u d+1 u d+1 tHowever, this is calculated using equation (4).
[0053]
[0054] In equation (4), M is given by equation (5).
[0055]
[0056] In the fifth line of algorithm 2, u d＋1 [G d j ] is the dth depth in the Idex tree, i.e., the u corresponding to the jth node of the leaf nodes. d+1 That is the case.
[0057] Therefore, the fifth line of algorithm 2 is u d＋1 [G d j This operation calculates the norm of [ ] and, if it is below the threshold, sets it to zero in the sixth row.
[0058] In this embodiment, this is used to prune leaf nodes. Figure 6 shows an example of the algorithm used by the data processing device 10 shown in Figure 1.
[0059] Specifically, the data processing device 10 uses the u used in the fifth line of algorithm 2. d＋1 [G d j Instead of the norm of ], as shown in the 5th line of algorithm 3 in Figure 6, u d＋1 [G d j ] Norm ||u d＋1 [G d j ]||Upper bound of 2￣u d＋1 [G d j The upper bound of this leaf node is calculated with minimal computational complexity. In Algorithm 3, d＋1 [G d j If the upper bound is below the threshold, this leaf node becomes zero (lines 8 and 9 of Algorithm 3). In other words, leaf nodes whose upper bound is below the threshold are pruned. Upper bound of a leaf node ￣u d＋1 [G d j This is calculated using formula (6).
[0060]
[0061] Here, ||M[G d j ]|| F Of the M calculated by equation (5), G d j This is the Frobenius norm of the row corresponding to the index. Also, Δ^β t , (^β t -^β t’ ) Also, let 0 ≤ t' < t.
[0062] [Pruning of Internal Nodes] Next, we will explain the pruning of internal nodes. The processing of internal nodes corresponds to the case when i≠d in Algorithm 3 (line 7 of Algorithm 3). Pruning of internal nodes is performed in the same way as leaf nodes, in lines 8 and 9 of Algorithm 3. However, the method for calculating the upper bound of internal nodes is different from that for leaf nodes.
[0063] Specifically, the upper bound of an internal node is calculated as shown in equation (7).
[0064]
[0065] Here, D i ｊ G i+1 k ⊆G i j This is the set of index k such that
[0066] Equation (7) is the node G that we are currently trying to calculate. i j The upper realm ￣u i+1 t [G i j ] is a child node G i+1 k This indicates that the calculation is performed using an upper bound. Since the outermost node is a leaf node, the upper bound of the leaf node is calculated using equation (6), and thereafter, the upper bounds can be calculated in the direction of the root node according to equation (7).
[0067] In Algorithm 3 of FIG. 6, the upper bounds of leaf nodes and internal nodes are calculated (lines 4 to 7), and it is determined whether to prune the nodes (lines 8 to 13). This calculation is performed in a bottom-up breadth-first order (lines 1 to 2). In Algorithm 3, when the node is a leaf node, the upper bound is calculated by Equation (6) (lines 4 to 5), and when the node is an internal node, the upper bound is calculated by Equation (7) (lines 6 to 7). In Algorithm 3, if the calculated upper bound is less than or equal to the threshold value λ i j (= λw i j η), this node is pruned (lines 8 to 9). If the upper bound is greater than the threshold value λ i j , the exact value is calculated (lines 10 to 13). In Algorithm 3, when the node is a leaf node, the value is calculated by Equation (4) only for the dimension corresponding to G i j (lines 11 to 12).
[0068] FIG. 7 is a diagram showing an example of an algorithm used by the data processing apparatus 10 shown in FIG. 1. Algorithm 4 shown in FIG. 7 is the pseudo-code of FISTA using Algorithm 3.
[0069] In Algorithm 4, first, ||M[G d j || F used for calculating the upper bound is pre-calculated (lines 3 to 4). The main loop of Algorithm 4 is from line 5 to line 14. The algorithm calculates ^β t’ and ||u d+1 t’ [G d j ||2 to calculate the upper bound of the leaf node (Equation (6)) (lines 6 to 8). These values are calculated at a fixed iteration interval r (>1) (line 9). For example, r = 2. Lines 10 to 13 are the same processing as the conventional FISTA (Algorithm 1) except for the calculation of the proximal mapping. Specifically, the proximal mapping is calculated using Algorithm 3.
[0070] [Computation cost] Let n u be the total dimension number of the vectors corresponding to the internal nodes that have not been pruned, and nl Let it be the total number of dimensions of the vectors corresponding to the leaf nodes that have not been pruned. When T is the total number of loops from the 5th to the 14th lines, the computational cost of Algorithm 4 is O(p 2 (n+(T / r))+p(n+T+n l ))+n u ) times.
[0071] Therefore, when n u and n l are small, it indicates that the processing time is shortened by Algorithm 4. The cost of the conventional FISTA by tree structure regularization is O(p 2 (n+T)+p(n+Td)) times. Therefore, according to Algorithm 4, the cost of the coefficient of p 2 is reduced.
[0072] [Processing Procedure of Data Processing Method] FIG. 8 is a flowchart showing the processing procedure of the data processing method according to the embodiment.
[0073] As shown in FIG. 8, the Gram matrix norm calculation unit 11 calculates the norm for each row of the Gram matrix of the given data (step S1). After calculating M in Equation (5), the Gram matrix norm calculation unit 11 calculates ||M[G d j || F in Equation (6). This calculation is performed only once before entering the optimization.
[0074] The parameter norm calculation unit 12 calculates the norm of the parameter during optimization (step S2). The parameter norm calculation unit 12 calculates the norm ||u[G d＋1 j ||2 of the parameter shown in Equation (6). This parameter norm is calculated every certain iteration, for example, every r iterations, in the iteration of FISTA.
[0075] The node upper bound calculation unit 13 calculates the upper bound of the norm of the parameter corresponding to the nodes of the tree structure (step S3). The node upper bound calculation unit 13 performs the upper bound calculation for the leaf nodes in Equation (6) and the upper bound calculation for the internal nodes in Equation (7).
[0076] The node calculation pruning unit 14 updates the parameters while deleting unnecessary calculations among the calculations corresponding to the nodes in the tree structure (step S4). The node calculation pruning unit 14 uses the upper bound calculated by the node upper bound calculation unit 13 to prune unnecessary calculations. In FISTA, in the 5th line of algorithm 2 (Figure 4), λ i j Because there is a threshold, the upper bound is λ i j The following conditions will result in pruning. If pruning is not performed, the normal calculation will be executed.
[0077] The momentum calculation unit 15 assigns momentum to the parameters (step S5). The momentum calculation unit 15 performs the calculations in lines 5-6 of algorithm 1 (Figure 4), similar to the conventional FISTA.
[0078] The convergence determination calculation unit 16 determines the convergence of the optimization based on the parameters (step S6). The convergence determination calculation unit 16 determines whether the coefficients to be optimized, i.e., the parameters, have converged. If the parameters have not converged (step S6: No), the data processing device 10 returns to step S3. If the parameters have converged (step S6: Yes), the processing of the data processing device 10 stops.
[0079] [Effects of the Embodiment] The data processing device 10 according to the embodiment is a learning device for a linear regression model that uses tree lasso to select important features from large-scale data with a tree structure.
[0080] Furthermore, the data processing unit 10 can efficiently solve optimization problems and achieve high speed by pruning unnecessary nodes in the tree structure, rather than optimizing all parameters as in conventional tree lasso. In addition, because the data processing unit 10 precisely prunes unnecessary calculations, no degradation in accuracy occurs.
[0081] In this embodiment, a fast proximal gradient method (PGM) was proposed for tree structure regularization. The bottlenecks of conventional methods are parameter updates, which include gradient calculation using a hierarchical structure (first bottleneck) and calculation of the proximal map (second bottleneck).
[0082] The data processing unit 10 resolved the first bottleneck by pruning the calculations of the leaf nodes of the tree; that is, the data processing unit 10 safely skipped the gradient calculation by identifying parameters that must be zero based on the upper bounds. Then, the data processing unit 10 resolved the second bottleneck by pruning the calculations of the internal nodes; that is, the data processing unit 10 hierarchically pruned the calculations of the internal nodes by summing the upper bounds of the child nodes.
[0083] Figure 9 shows the number of calculations per depth using tree lasso and the total number of calculations. In Figure 9, λw is shown as a representative example. i j =λ max To predict Tokyo's temperature for the case of / 10, the number of calculations per node was compared between the conventional method (FISTA) and pFISTA (data processing device 10). In pFISTA, the total number of calculations was reduced to 58.35% compared to FISTA. Specifically, the number of leaf node pruning was reduced to 66.37% as shown in depth 13 of Figure 9. The number of internal node pruning was reduced to 50.32% of the total number of calculations at depths 1 to 12. The trend was almost the same for other cities and methods. This demonstrated that the data processing device 10 can effectively prune unnecessary calculations.
[0084] In fact, in the demonstration experiment, the data processing device 10 was able to reduce processing time by 56% compared to the conventional method without compromising accuracy. Therefore, the data processing device 10 can handle approximately twice the amount of data in the same amount of time as the conventional method. Because the amount of data that can be handled in a single operation can be significantly increased compared to the conventional method, it is expected that advanced insights that could not be discovered before can be found can be discovered.
[0085] Next, we evaluated the number of calculations performed at each node in each iteration. Specifically, we counted the number of calculations at the 1st, 500th, 1000th, 2000th, and 2500th iterations. The algorithm stopped at the 2984th iteration.
[0086] Figure 10 shows the number of calculations per node per iteration in tree lasso. As shown in Figure 10, the data processing unit 10 prunes half of the calculations even at the start of the optimization. As a result, the data processing unit 10 was able to further reduce the number of calculations as it converged. Furthermore, the data processing unit 10 was able to guarantee the same optimization results and convergence rate as the conventional method.
[0087] Furthermore, since the data processing device 10 only modifies the calculation of proximity mappings in the conventional method, a variety of PGMs can be used. In addition, since the data processing device 10 can safely prune unnecessary calculations for zero parameters, the same optimization results and convergence rates as the conventional method can be ensured.
[0088] Therefore, the data processing device 10 can quickly select important features from data that has a tree structure for its features.
[0089] [System Configuration of the Embodiment] Each component of the data processing device 10 shown in Figure 1 is a functional concept and does not necessarily have to be physically configured as shown. In other words, the specific forms of distribution and integration of the functions of the data processing device 10 are not limited to those shown, and all or part of it can be configured by functionally or physically distributing or integrating in any unit according to various loads and usage conditions.
[0090] Furthermore, each process performed in the data processing device 10 may be implemented entirely or partially by a CPU and a program that is analyzed and executed by the CPU. Alternatively, each process performed in the data processing device 10 may be implemented as hardware using wired logic.
[0091] Furthermore, among the processes described in the embodiments, all or part of the processes described as being performed automatically can be performed manually. Alternatively, all or part of the processes described as being performed manually can be performed automatically by known methods. In addition, the processing procedures, control procedures, specific names, and information including various data and parameters described above and in the figures can be modified as appropriate unless otherwise specified.
[0092] [Program] Figure 11 shows an example of a computer in which a data processing device 10 is realized when a program is executed. The computer 1000 has, for example, a memory 1010 and a CPU 1020. The computer 1000 also has a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These components are connected by a bus 1080.
[0093] Memory 1010 includes ROM 1011 and RAM 1012. ROM 1011 stores, for example, a boot program such as a BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to the hard disk drive 1090. The disk drive interface 1040 is connected to the disk drive 1100. For example, a removable storage medium such as a magnetic disk or optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to, for example, a mouse 1110 and a keyboard 1120. The video adapter 1060 is connected to, for example, a display 1130.
[0094] The hard disk drive 1090 stores, for example, an OS (Operating System) 1091, an application program 1092, a program module 1093, and program data 1094. That is, the programs that define each process of the data processing device 10 are implemented as program modules 1093 in which code executable by the computer 1000 is written. The program modules 1093 are stored, for example, in the hard disk drive 1090. For example, a program module 1093 for performing processes similar to the functional configuration of the data processing device 10 is stored in the hard disk drive 1090. Note that the hard disk drive 1090 may be replaced by an SSD (Solid State Drive).
[0095] Furthermore, the configuration data used in the processing of the above-described embodiment is stored as program data 1094 in, for example, memory 1010 or hard disk drive 1090. The CPU 1020 then reads the program module 1093 and program data 1094 stored in memory 1010 or hard disk drive 1090 into RAM 1012 as needed and executes them.
[0096] Furthermore, the program module 1093 and program data 1094 are not limited to being stored in the hard disk drive 1090; for example, they may be stored in a removable storage medium and read by the CPU 1020 via a disk drive 1100 or the like. Alternatively, the program module 1093 and program data 1094 may be stored in another computer connected via a network (LAN (Local Area Network), WAN (Wide Area Network), etc.). The program module 1093 and program data 1094 may then be read by the CPU 1020 from the other computer via a network interface 1070.
[0097] Although embodiments applying the invention made by the present inventors have been described above, the present invention is not limited by the descriptions and drawings that constitute part of the disclosure of the present invention in these embodiments. That is, all other embodiments, examples, and operational techniques made by those skilled in the art based on these embodiments are included in the scope of the present invention.
[0098] 10 Data processing unit 11 Gram matrix norm calculation unit 12 Parameter norm calculation unit 13 Node upper bound calculation unit 14 Node calculation pruning unit 15 Momentum calculation unit 16 Convergence determination calculation unit
Claims
1. A data processing device for selecting features from data having a tree structure for its features, comprising: a first calculation unit that calculates the row-by-row norm of the Gram matrix of the given data; a second calculation unit that obtains the norm of the parameter being optimized; a third calculation unit that calculates an upper bound of the norm of the parameter corresponding to the node of the tree structure; an update unit that deletes unnecessary calculations from the calculations corresponding to the nodes of the tree structure and updates the parameter based on the calculation result of the upper bound by the third calculation unit; an assignment unit that assigns momentum to the parameter; and a determination unit that determines the convergence of the optimization based on the parameter.
2. The data processing device according to claim 1, characterized in that the update unit determines, based on the calculation result of the upper bound of the norm of the node, that the calculations for leaf nodes, which are terminal nodes, where the upper bound is less than or equal to a predetermined threshold, are unnecessary gradient calculations and deletes them.
3. The data processing device according to claim 1, characterized in that the update unit determines, based on the calculation result of the upper bound of the norm of the node, that the calculations of internal nodes other than leaf nodes (which are terminal nodes) for which the upper bound is less than or equal to a predetermined threshold are unnecessary proximity mapping calculations and delete them.
4. The data processing device according to claim 3, characterized in that the update unit calculates the upper bound of the internal node by summing the upper bounds already calculated at the leaf nodes from the calculation results of the upper bounds of the node.