Data processing device

Abstract

A data processing device (10) comprises a Gram matrix norm calculation unit (11), a lower bound calculation unit (12), a threshold value update unit (13), an upper bound calculation unit (14), and a candidate set update unit (15). The Gram matrix norm calculation unit (11) calculates a norm for each row of a Gram matrix of data including a plurality of parameters. The lower bound calculation unit (12) calculates a lower bound of an absolute value of each of the parameters on the basis of the norm. The threshold value update unit (13) updates a threshold value on the basis of the lower bound. The upper bound calculation unit (14) calculates an upper bound of the absolute value of each of the parameters on the basis of the norm. The candidate set update unit (15) updates a candidate set, which is at least a portion of the parameters, on the basis of the lower bound, the threshold value, and the upper bound.

Description

Data processing device

[0001] This invention relates to a data processing device.

[0002] Feature selection is a group of techniques for selecting important features from data, and it is one of the fundamental techniques in data mining. Lasso is a commonly used feature selection method (see, for example, Non-Patent Document 1).

[0003] On the other hand, Lasso requires tuning continuous hyperparameters to select important features, and the number of features selected is unknown until execution is complete, which can lead to usability issues depending on the use case.

[0004] One known method to solve this problem is iterative hard thresholding (IHT) (see, for example, Non-Patent Document 2). IHT is a method in which the number of features that the user wants to select can be specified in advance.

[0005] IHT is a linear regression-based method that is constrained so that the number of non-zero elements in the coefficients of the linear regression model is k. Here, k is the number of features that the user specifies to select. Since there is a one-to-one correspondence between the coefficients and features of the linear regression model, the k non-zero coefficients resulting from the learning process correspond to the k important features. The other features correspond to zero coefficients and therefore do not contribute to the prediction. Thus, features corresponding to zero coefficients can be considered unimportant features. In this way, IHT can identify important and unimportant features.

[0006] R. Tibshirani. Regression Shrinkage and Selection via the Lasso. Journal of the Royal Statistical Society, 58:267-288, 1996. T. Blumensath and M. E. Davies. Iterative Hard Thresholding for Compressed Sensing. Applied and Computational Harmonic Analysis, 27(3):265-274, 2009.

[0007] However, conventional techniques have the problem of slow IHT calculations. For example, when the optimization problem is large, i.e., when dealing with large datasets, it can take a long time to obtain the IHT results.

[0008] The present invention has been made in view of the above, and aims to speed up IHT calculations.

[0009] To solve the above-mentioned problems and achieve the objective, the data processing device according to the present invention is characterized by comprising: a Gram matrix norm calculation unit that calculates the row-by-row norm of a Gram matrix of data containing a plurality of parameters; a lower bound calculation unit that calculates a lower bound for each absolute value of the parameters based on the norm; a threshold update unit that updates a threshold based on the lower bound; an upper bound calculation unit that calculates an upper bound for each absolute value of the parameters based on the norm; and a candidate set update unit that updates a candidate set which is at least a part of the parameters based on the lower bound, the threshold, and the upper bound.

[0010] According to the present invention, IHT calculations can be accelerated.

[0011] Figure 1 shows an example configuration of a data processing device according to the first embodiment. Figure 2 shows the IHT algorithm. Figure 3 shows the algorithm for updating the candidate set. Figure 4 shows the algorithm for updating the threshold. Figure 5 is a flowchart showing the process flow for updating the threshold. Figure 6 is a flowchart showing the process flow for updating the candidate set. Figure 7 shows an example of a computer that executes a data processing program.

[0012] Hereinafter, one embodiment of the present invention will be described in detail with reference to the drawings. However, the present invention is not limited by this embodiment. Furthermore, in the drawings, the same parts are denoted by the same reference numerals.

[0013] [First Embodiment] [Configuration of Data Processing Device] Figure 1 is a diagram showing an example configuration of a data processing device according to the first embodiment. The data processing device 10 performs feature selection, selecting important feature quantities from data containing multiple feature quantities. The data processing device 10 performs feature selection using IHT. Furthermore, the data processing device 10 can perform IHT at a higher speed compared to the conventional technology.

[0014] The data processing device 10 may be used as a device for training a linear regression model that selects a specified number of important features from large-scale data using IHT.

[0015] The data processing device 10 reduces the overall computational load and speeds up the process by pruning unnecessary parameter updates and associated gradient calculations during parameter updates (optimization) using IHT. Parameters are values ​​included in the model and correspond one-to-one with features included in the data. For this reason, for example, features corresponding to parameters ultimately included in the candidate set described later can be considered as the ultimately selected features.

[0016] In IHT, after the parameters are updated in each iteration for parameter updates, k parameters are selected in descending order of importance, which is the absolute value of the parameters. This process is repeated in IHT until the convergence condition is met.

[0017] The data processing device 10 prepares a candidate set that holds k parameters that appear to be important. In each iteration, the data processing device 10 compares the absolute value of the smallest parameter in the candidate set with the upper bound of the absolute values ​​of the parameters that are not included in the candidate set.

[0018] Since parameters whose upper bound is less than or equal to the absolute value of the smallest parameter in the candidate set are unnecessary parameters, the data processing device 10 can remove those parameters based on the comparison results and prune unnecessary calculations before performing parameter updates and the associated gradient calculations.

[0019] Furthermore, parameters whose lower bound on the absolute value of a parameter not included in the candidate set is greater than or equal to the absolute value of the smallest parameter in the candidate set are guaranteed to be included in the candidate set. Using this, the data processing device 10 updates the absolute value of the smallest parameter in the candidate set to be as large as possible before pruning using the upper bound. This allows the data processing device 10 to increase the pruning rate.

[0020] As shown in Figure 1, the data processing device 10 includes a Gram matrix norm calculation unit 11, a lower bound calculation unit 12, a threshold update unit 13, an upper bound calculation unit 14, a candidate set update unit 15, a convergence determination unit 16, and a pruning rate calculation unit 17.

[0021] The Gram matrix norm calculation unit 11 calculates the row-by-row norm of the Gram matrix of the given data.

[0022] In the threshold update unit 13 and candidate set update unit 15 described below, parameters may be updated (optimized). When parameters are updated, feature selection is performed by referring to the updated parameters. The absolute value of the parameters is called importance. k parameters are selected in descending order of importance. Here, k is the number of features to be selected and is a positive integer predetermined as a hyperparameter.

[0023] The lower bound calculation unit 12 calculates a lower bound for the absolute value of parameters that are not included in the candidate set.

[0024] The threshold update unit 13 updates the threshold based on the lower bound.

[0025] The upper bound calculation unit 14 calculates an upper bound for the absolute values ​​of parameters that do not fall into the candidate set, based on the pruning rate.

[0026] The candidate set update unit 15 updates the candidate set to include important parameters. At this time, the candidate set update unit 15 updates the candidate set based on upper bounds and thresholds. The candidate set update unit 15 also optimizes the parameters.

[0027] The convergence determination unit 16 determines whether the program has converged or not. The pruning rate calculation unit 17 calculates the pruning rate.

[0028] In the following explanation, when A is a vector, matrix, or scalar, the notation "A^{_}" is equivalent to "the symbol with an underscore directly above "A". Similarly, when A is a vector, matrix, or scalar, the notation "A_{_}" is equivalent to "the symbol with an underscore directly below "A". Furthermore, when A is a vector, matrix, or scalar, the notation "^A" is equivalent to "the symbol with a ^ directly above "A". Also, for a vector or matrix A, AT represents the transpose of A.

[0029] [Conventional IHT] Here, we will explain conventional IHT along with its mathematical background.

[0030] Since IHT is based on a linear regression model, we consider a linear regression problem. Let n be the number of data points, and assume that each data point is represented by an m-dimensional feature. Thus, the data points are X ∈ R ｎ×ｍ It can be represented by a matrix. Linear regression is a problem of predicting the response for each data point, so the response is a vector y∈R with dimensions equal to the number of data points. ｎ It can be expressed as follows. Linear regression makes predictions by calculating the inner product of the data and the coefficient vector, so this coefficient vector is θ∈R ｐ It is expressed as follows.

[0031] Under the above settings, IHT extracts important features by solving the optimization problem shown in equation (1) below.

[0032]

[0033] In Equation (1), k is a hyperparameter and represents the number of feature quantities to be selected. For simplicity, hereinafter, the first term of Equation (1) is denoted as f(θ) as in Equation (2).

[0034]

[0035] According to IHT, Equation (1) can be approximately solved. The algorithm of IHT is shown in FIG. 2. FIG. 2 is a diagram showing the algorithm of IHT.

[0036] In FIG. 2, t represents the number of iterations of IHT. η is a hyperparameter that is the step size when updating the parameters. P ｋ (...) is a function that sorts the parameters specified as arguments in descending order of absolute value, retains the values of the top k parameters, and sets the rest to zero.

[0037] IHT is an algorithm that repeats parameter update by the gradient method (line 4) and parameter selection by P ｋ (...) until convergence.

[0038] The z in FIG. 2 ｔ is calculated by Equation (3).

[0039]

[0040] [IHT of the First Embodiment] Next, the processing of the first embodiment will be described together with the mathematical background. The data processing device 10 of the first embodiment introduces a candidate set D ｔ that manages k parameters in each iteration of IHT, and prunes parameters that clearly do not enter the candidate set, thereby accelerating the parameter update in Equation (3).

[0041] To achieve acceleration, the upper bound calculation unit 14 calculates z ｔ ^ {_} which is the upper bound of the j-th element of the absolute value of z at the t-th iteration ｔ ｊ as calculated by the following Equations (4) and (5).

[0042]

[0043]

[0044] G is the Gram matrix. ｊ This represents the jth row of G. ||G ｊ || represents the norm calculated by the Gram matrix norm calculation unit 11. θ ｔ＊ t ＊ This is the parameter for the iteration number. Here, t ＊ Let's call it <t. XT ｊ represents the jth row after transposing X. z^{_} ｔ ｊ z ｔ ｊ The value will be greater than or equal to the absolute value of t. ＊ Iteration (t ＊ The iteration number is t-t higher than the t-th iteration. ＊ It can be said that this is a past iteration, but only one time.

[0045] Candidate set D ｔ Let D be the set of indices of parameters that are non-zero at the beginning of the t-th iteration. ｔ z corresponding to ｔ Among them, the one with the smallest absolute value is z ｔ ｉｍｉｎ (However, min is a subscript of i.)

[0046] Here, the upper bound of equation (4) is |z ｔ ｉｍｉｎ If the value is less than or equal to | z | then the absolute value of the parameter corresponding to the upper bound is | z ｔ ｉｍｉｎ Since it cannot be greater than |, it can be seen that it will never be included in the candidate set. By utilizing this, the candidate set update unit 15 can update the candidate set and exclude the unnecessary parameter identified by the upper bound from the calculation of equation (3). Note that |・| represents the absolute value.

[0047] The method for updating the candidate set will be explained using Figure 3. Figure 3 is a diagram showing the algorithm for updating the candidate set. Note that in the initial state before the update (for example, t=0), the candidate set may contain all parameters.

[0048] The candidate set update unit 15 updates the candidate set D of the t-th iteration. ｔ For each parameter not included in (from the 4th line onwards), perform the following processing. Also, for each parameter z ｔ ｊ The upper bound of is calculated by equation (4). Also, at this point, the candidate set D of the t+1 iteration is ｔ＋１ D ｔ This is substituted directly (third line).

[0049] z ｔ ｊ The upper bound is | z ｔ ｉｍｉｎ | If the following is the case (5th line), the candidate set update unit 15 will, z ｔ ｊ Do not add z to the candidate set (line 6). Note that the candidate set update unit 15 is z ｔ ｊ By updating it to 0, it is excluded from the update target.

[0050] z ｔ ｊ The upper bound is | z ｔ ｉｍｉｎ If it is greater than | (7th line), the candidate set update unit 15 will ｔ ｊ This updates the parameter accurately (line 8). Note that `compute` means, for example, updating (optimizing) the parameter using equation (3).

[0051] Furthermore, |z ｔ ｊ |ga|z ｔ ｉｍｉｎ If it is greater than | (line 9), the candidate set update unit 15 will update the original z ｔ ｉｍｉｎ The corresponding index is D ｔ＋１ Remove from and add j (line 10) D ｔ＋１ no z ｔ＋１ ｉｍｉｎ Update (line 11).

[0052] The process in Figure 3 is z ｔ ｉｍｉｎ This can be described as pruning with z as the threshold. ｔ ｉｍｉｎIf it can be pre-updated to a larger value, the pruning rate is expected to increase. To execute this, the lower bound calculation unit 12 calculates z ｔ The lower bound z_{{}} ｔ ｊ Of the j-th element of the absolute value at the t-th iteration by the following equation (6).

[0053]

[0054] Using Fig. 4, the process of updating the threshold value z ｔ ｉｍｉｎ Will be described. Fig. 4 is a diagram showing the algorithm of the process for updating the threshold value.

[0055] The threshold value update unit 13 performs the following process for each parameter not included in the candidate set D ｔ At the t-th iteration (starting from the 4th line). Also, the lower bound of each parameter z ｔ ｊ Is calculated by equation (6). Also, the output candidate set D ｔ´ Contains D ｔ Is directly substituted (3rd line). Also, z ｔ´ ｉｍｉｎ , z ｔ´ , And D ｔ´ Are output as the updated threshold value, parameter, and candidate set.

[0056] The lower bound z_{{}} ｔ ｊ Is always less than or equal to the absolute value of z ｔ ｊ If z_{{}} ｔ ｊ Is greater than or equal to z ｔ ｉｍｉｎ It means that the absolute value of that parameter will always be included in the candidate set.

[0057] Therefore, the threshold value update unit 13 adds the parameter whose lower bound is greater than or equal to z ｔ ｉｍｉｎ To the candidate set D ｔ´ And deletes the original z ｔ ｉｍｉｎ From the candidate set D ｔ´ To update the candidate set.

[0058] At this time, z ｔ ｉｍｉｎ After the above values are included in the candidate set D ｔ´ and then z ｔ ｉｍｉｎ is calculated, the calculated z ｔ ｉｍｉｎ is found to be a value greater than or equal to the z before update ｔ ｉｍｉｎ Thus, the candidate set update unit 15 can update the threshold value z ｔ ｉｍｉｎ to a larger value. Therefore, by executing the process of FIG. 3 after the process of FIG. 4, the data processing apparatus 10 can increase the pruning rate in the process of FIG. 3.

[0059] In the IHT in the first embodiment, the higher the pruning rate, the faster the speed, and the lower the pruning rate, the slower the speed. The pruning rate changes depending on how the θ ｔ＊ in Equation (4) is acquired at intervals. This is because if θ ｔ＊ and θ ｔ are too far apart, the upper bound of Equation (4) and the lower bound of Equation (6) become loose, and pruning and threshold update do not work well. The larger the interval, the lower the pruning rate, and the smaller the interval, the higher the pruning rate. However, since calculation is required when acquiring θ ｔ＊ , if the interval is made too small, the processing time will increase.

[0060] Therefore, the pruning rate calculation unit 17 determines the acquisition interval of θ ｔ* according to the pruning rate. The pruning rate calculation unit 17 calculates the pruning rate p ｔ at the t-th iteration by Equation (7).

[0061]

[0062] u ｔ－１ is the number of parameters that could not be pruned at the (t - 1)-th iteration. Using this pruning rate, the pruning rate calculation unit 17 calculates the acquisition interval r of θ ｔ＊ as shown in Equation (8).

[0063]

[0064] In other words, if the pruning rate is 50% or higher, the acquisition interval is extended by one iteration. If the pruning rate is less than 50%, the acquisition interval is halved.

[0065] [Processing in the First Embodiment] Figure 5 is a flowchart showing the process for updating the threshold. The flowchart in Figure 5 corresponds to the algorithm described in Figure 4.

[0066] First, the threshold update unit 13 selects one unselected parameter from among the parameters not included in the candidate set (step S101). Then, the lower bound calculation unit 12 calculates a lower bound on the importance of the selected parameter (step S102).

[0067] Next, the threshold update unit 13 determines whether the threshold (the minimum importance value of the candidate set) is less than the lower bound (step S103). If the threshold is not less than the lower bound (step S103; No), the threshold update unit 13 proceeds to step S107.

[0068] If the threshold is less than the lower bound (step S103; Yes), the threshold update unit 13 adds the selected parameter to the candidate set (step S104). The threshold update unit 13 also optimizes the selected parameter (step S105). Furthermore, the threshold update unit 13 updates the threshold to the minimum importance value of the candidate set (step S106).

[0069] If there are any unselected parameters (step S107; Yes), the threshold update unit 13 returns to step S101 and repeats the process. If there are no unselected parameters (step S107; No), the threshold update unit 13 terminates the process of updating the threshold.

[0070] Figure 6 is a flowchart showing the process for updating the candidate set. The flowchart in Figure 6 corresponds to the algorithm described in Figure 3.

[0071] First, the candidate set update unit 15 selects one unselected parameter from among the parameters not included in the candidate set (step S201). Then, the upper bound calculation unit 14 calculates an upper bound on the importance of the selected parameter (step S202).

[0072] Next, the candidate set update unit 15 determines whether the threshold (the minimum importance value of the candidate set) is greater than or equal to the upper bound (step S203). If the threshold is greater than or equal to the upper bound (step S203; Yes), the candidate set update unit 15 updates the selected parameter to 0 (step S204) and proceeds to step S209.

[0073] If the candidate set update unit 15 finds that the threshold is not equal to or greater than the upper limit (step S203; No), it optimizes the selected parameters (step S205).

[0074] Next, the candidate set update unit 15 determines whether the threshold is less than the absolute value of the selected parameter (step S206). If the threshold is not less than the absolute value of the selected parameter (step S206; No), the candidate set update unit 15 updates the selected parameter to 0 (step S204) and proceeds to step S209.

[0075] If the candidate set update unit 15 finds that the threshold is less than the absolute value of the selected parameter (step S206; Yes), it adds the selected parameter to the candidate set (step S207). Then, the candidate set update unit 15 updates the threshold to the minimum importance value of the candidate set (step S208).

[0076] The data processing device 10 repeatedly executes iterations that include the processing shown in Figure 5 and Figure 6. Furthermore, the row-by-row norms of the Gram matrix of the data, calculated by the Gram matrix norm calculation unit 11, are used for calculating the upper and lower bounds. The calculation of the norms by the Gram matrix norm calculation unit 11 is performed once before the iteration begins.

[0077] The data processing device 10 repeatedly executes iterations until the convergence determination unit 16 determines that convergence has occurred. For example, the convergence determination unit 16 determines that convergence has occurred when the number of iterations reaches a predetermined number, when a predetermined amount of time has elapsed since the start of processing, or when the amount of parameter updates has become sufficiently small.

[0078] Furthermore, the pruning rate calculation unit 17 calculates the pruning rate in each iteration. The pruning rate is θ ｔ＊ This is used to determine the acquisition interval.

[0079] [Effects of the First Embodiment] As described above, the Gram matrix norm calculation unit 11 calculates the row-by-row norm of the Gram matrix of the data containing multiple parameters. The lower bound calculation unit 12 calculates the lower bound of the absolute value of each parameter based on the norm. The threshold update unit 13 updates the threshold based on the lower bound. The upper bound calculation unit 14 calculates the upper bound of the absolute value of each parameter based on the norm. The candidate set update unit 15 updates the candidate set, which is at least a part of the parameters, based on the lower bound, threshold, and upper bound. In this way, the data processing device 10 does not optimize all parameters, but rather prunes unnecessary parameters and their corresponding parameter updates and gradient calculations, thereby efficiently solving the optimization problem and speeding up the entire IHT. Furthermore, because the data processing device 10 accurately prunes unnecessary calculations, it can suppress the degradation of accuracy. Therefore, according to this embodiment, a fast and accurate IHT is realized.

[0080] Furthermore, the lower bound calculation unit 12 calculates a lower bound for each of the multiple iterators. The threshold update unit 13 updates the threshold for each of the multiple iterators. The upper bound calculation unit 14 calculates an upper bound for each of the multiple iterators. The candidate set update unit 15 updates the candidate set for each of the multiple iterators. In this way, the data processing device 10 can improve accuracy by repeatedly performing pruning and parameter optimization.

[0081] Furthermore, the pruning rate calculation unit 17 calculates the percentage of branches pruned (fifth row in Figure 3) by the candidate set update unit 15 in each of the multiple iterators. For example, the pruning rate is given by p shown in equation (7). ｔ The calculations are performed as follows: The lower bound calculation unit 12, the upper bound calculation unit 14, and the pruning rate calculation unit 17 acquire parameters from iterators that have been iterated in the past among multiple iterators at intervals corresponding to the pruning rate. The lower bound calculation unit 12 calculates the lower bound based on the acquired parameters. The upper bound calculation unit 14 calculates the upper bound based on the acquired parameters. As a result, the data processing device 10 can increase the pruning rate and speed up processing while reducing the processing load required to acquire past iterators.

[0082] [Program] In one embodiment, the data processing device 10 can be implemented by installing a data processing program that performs the above processing as packaged software or online software on a desired computer. For example, by having the above data processing program run on an information processing device, the information processing device can be made to function as the data processing device 10. The information processing device referred to here includes desktop or notebook personal computers. In addition, the information processing device also includes smartphones, tablet terminals, etc.

[0083] Figure 7 shows an example of a computer that executes a data processing program. Computer 1000 has, for example, memory 1010 and a CPU 1020. Computer 1000 also has a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These components are connected by a bus 1080.

[0084] Memory 1010 includes ROM (Read Only Memory) 1011 and RAM (Random Access Memory) 1012. ROM 1011 stores, for example, a boot program such as BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to the hard disk drive 1090. The disk drive interface 1040 is connected to the disk drive 1100. For example, a removable storage medium such as a magnetic disk or optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to, for example, a mouse 1110 and a keyboard 1120. The video adapter 1060 is connected to, for example, a display 1130.

[0085] The hard disk drive 1090 stores, for example, the OS 1091, application programs 1092, program modules 1093, and program data 1094. That is, the programs that define each process of the data processing device 10 are implemented as program modules 1093 in which executable code for the computer is written. The program modules 1093 are stored, for example, in the hard disk drive 1090. For example, a program module 1093 for performing processes similar to the functional configuration of the data processing device 10 is stored in the hard disk drive 1090. Note that the hard disk drive 1090 may be replaced by an SSD.

[0086] Furthermore, the configuration data used in the processing of the above-described embodiment is stored as program data 1094 in, for example, memory 1010 or hard disk drive 1090. The CPU 1020 then reads the program module 1093 and program data 1094 stored in memory 1010 or hard disk drive 1090 into RAM 1012 as needed and executes the processing of the above-described embodiment.

[0087] Furthermore, the program module 1093 and program data 1094 are not limited to being stored in the hard disk drive 1090; for example, they may be stored in a removable storage medium and read by the CPU 1020 via a disk drive 1100 or the like. Alternatively, the program module 1093 and program data 1094 may be stored in another computer connected via a network (LAN (Local Area Network), WAN (Wide Area Network), etc.). The program module 1093 and program data 1094 may then be read by the CPU 1020 from the other computer via a network interface 1070.

[0088] Although embodiments applying the invention made by the present inventors have been described above, the present invention is not limited by the descriptions and drawings that constitute part of the disclosure of the present invention in this embodiment. That is, all other embodiments, examples, and operational techniques made by those skilled in the art based on this embodiment are included in the scope of the present invention.

[0089] 10 Data processing unit 11 Gram matrix norm calculation unit 12 Lower bound calculation unit 13 Threshold update unit 14 Upper bound calculation unit 15 Candidate set update unit 16 Convergence determination unit 17 Pruning rate calculation unit

Claims

1. A data processing device comprising: a Gram matrix norm calculation unit that calculates the row-by-row norm of a Gram matrix of data containing multiple parameters; a lower bound calculation unit that calculates a lower bound for each absolute value of the parameters based on the norm; a threshold update unit that updates a threshold based on the lower bound; an upper bound calculation unit that calculates an upper bound for each absolute value of the parameters based on the norm; and a candidate set update unit that updates a candidate set which is at least a part of the parameters based on the lower bound, the threshold, and the upper bound.

2. The data processing apparatus according to claim 1, characterized in that the lower bound calculation unit calculates the lower bound in each of the multiple iterators that are repeated, the threshold update unit updates the threshold in each of the multiple iterators, the upper bound calculation unit calculates the upper bound in each of the multiple iterators, and the candidate set update unit updates the candidate set in each of the multiple iterators.

3. The data processing device according to claim 2, further comprising: a pruning rate calculation unit that calculates a pruning rate, which is the percentage of branches pruned by the candidate set update unit, in each of the plurality of iterators; and an acquisition unit that acquires parameters from iterators that have been iterated in the past among the plurality of iterators at intervals corresponding to the pruning rate, wherein the lower bound calculation unit calculates the lower bound based on the parameters acquired from iterators that have been iterated in the past among the plurality of iterators at intervals corresponding to the pruning rate, and the upper bound calculation unit calculates the upper bound based on the acquired parameters.

Images