Information processing device, information processing method, and information processing program

Abstract

An information processing device according to one embodiment comprises: an acquisition unit that acquires observation data which is composed of the number of data points and input value data at each of the data points; an inverse M-matrix processing unit that, on the basis of the observation data, selects, for a matrix which is calculated from the data points and the input value data at any data point, a non-negative scalar value and a positive definite kernel function in which all off-diagonal elements of an inverse matrix of the matrix becomes negative or zero; an estimation model learning unit that learns a parameter of an estimation model by solving a minimization problem using the observation data, the positive definite kernel function, and the non-negative scalar value; and an output control unit that outputs the positive definite kernel function, the non-negative scalar value, the learned parameter, and the input value data.

Description

Information processing device, information processing method, and information processing program

[0001] The present invention relates to an information processing device, an information processing method, and an information processing program.

[0002] There is known a technique for estimating a function whose output value is non-negative, such as the monthly sales amount of a product or the probability of an accident event occurring in a geographical space, from observed data. For example, in Non-Patent Document 1, a function is estimated by estimating a function whose output value is non-negative, such as the monthly sales amount of a product or the probability of an accident event occurring in a geographical space, from observed data. １ , x ２ , ..., x Ｎ ) is given, two types of estimation models (f １ (x) and f ２ (x)) have been proposed. These estimation models have been mathematically proven to take non-negative values ​​for any input x.

[0003] When it is known in advance that the output value is non-negative, designing an estimation model so that it takes only non-negative values, as in Non-Patent Document 1, has great practical advantages in that (i) the accuracy of the estimator is improved, and (ii) it can avoid situations in which errors occur on the computer when negative values ​​are taken, such as the logarithm of a probability value.

[0004] Marteau-Ferey, Bach, and Rudi, "Non-parametric Models for Non-negative Functions", Advances in Neural Information Processing Systems 32, 2020. Johnson, "Inverse M-Matrices", Linear Algebra and its Applications, 47:195-216, 1982.

[0005] The estimation model f in Non-Patent Document 1 １ Since (x) is a linear model for the positive definite kernel function k(x, x'), it has the advantage of low computational cost (linear cost relative to the number of data points). On the other hand, it has the problem of low accuracy of the estimated model because the function shapes that can be expressed are strongly limited.

[0006] The estimation model f in Non-Patent Document 2 ２ (x) is the estimated model f １ Unlike (x), there are fewer restrictions on the shape of the function that can be expressed, making this a highly accurate estimation model. However, because it is a quadratic model with respect to the positive definite kernel function k(x, x'), there is a problem in that it has a high computational cost (quadratic cost with respect to the number of data points).

[0007] The present invention has been made in light of the above circumstances, and aims to propose a technique that can realize an estimation model that achieves both low calculation costs and high accuracy.

[0008] In order to solve the above problem, an information processing device of one embodiment of the present invention includes an acquisition unit that acquires observation data consisting of the number of data points and input value data at each of the data points; an inverse M-matrix processing unit that selects, based on the observation data, a positive definite Kernel function and a non-negative scalar value for a matrix calculated from the data points and the input value data at an arbitrary data point, such that all off-diagonal components of the inverse matrix of the matrix are negative or zero; an estimation model learning unit that learns parameters of an estimation model by solving a minimization problem using the observation data, the positive definite Kernel function, and the non-negative scalar value; and an output control unit that outputs the positive definite Kernel function, the non-negative scalar value, the learned parameters, and the input value data.

[0009] According to one aspect of the present invention, a technique is provided that can realize an estimation model that achieves both low calculation cost and high accuracy.

[0010] Fig. 1 is a block diagram showing an example of a hardware configuration of an information processing device according to an embodiment. Fig. 2 is a block diagram showing a software configuration of the information processing device according to an embodiment in association with the hardware configuration shown in Fig. 1. Fig. 3 is a flowchart showing an example of a processing operation of the information processing device according to an embodiment.

[0011] Hereinafter, an information processing device, an information processing method, and an information processing program will be described in detail with reference to the drawings. In the following embodiments, parts with the same numbers perform the same operations, and redundant description will be omitted. For example, when there are multiple identical or similar elements, a common symbol may be used to describe each element without distinguishing between them, or a subnumber may be used in addition to the common symbol to describe each element with distinction between them.

[0012] [Estimation Models in Non-Patent Document 1] First, the two estimation models in Non-Patent Document 1 will be described. １ (x) and f ２ (x)) is defined by the following formula:

[0013]

[0014] where α is an N-dimensional column vector,

[0015]

[0016] is an (N × N) matrix B Ｎ is a positive definite matrix, and t represents the transpose operation of a matrix or vector. Parameter α and B Ｎ is determined based on observed data. These estimation models are mathematically guaranteed to take non-negative values ​​for any input x.

[0017] As described above, when it is known in advance that the output value is non-negative, there is a great practical advantage in designing an estimation model that takes only non-negative values.

[0018] [Embodiments] (Summary) Next, an outline of a method for solving the problem in one embodiment will be described. (I) In one embodiment, an estimation model is a method for solving the problem in one embodiment, where N data points (x １ , x ２ , ..., x Ｎ ) is given, and is defined by the following equation using a positive definite kernel function k(x, x') that takes a non-negative value.

[0019]

[0020] where the parameter α is an N-dimensional column vector, and σ ２ is a non-negative scalar value, and I Ｎ represents an (N × N) identity matrix, and an (N × N) matrix K Ｎ is the Gram matrix defined by the following equation:

[0021]

[0022] (II) N data points (x １ , x ２ , ..., x Ｎ ) and an (N+1)×(N+1) matrix calculated from an arbitrary data point x is expressed by the following formula:

[0023]

[0024] The matrix Ω Ｎ＋１ whereas the matrix Ω Ｎ＋１ A positive definite kernel function k(x, x') and a non-negative scalar value σ ２ The condition is expressed by the following formula:

[0025]

[0026] In this case, the matrix Ω Ｎ＋１ is a matrix called an Inverse M-Matrix (see, for example, Non-Patent Document 2). Due to the inherent properties of this Inverse M-Matrix, the estimation model f ＊ (x) is mathematically guaranteed to take a non-negative value for any input x.

[0027] Since equation (1) is a linear model for the positive definite kernel function k(x, x'), the calculation cost is low. Furthermore, the parameter α in equation (1) must satisfy the constraint ((K Ｎ +σ ２ I Ｎ ) α ≧ 0) is the estimation model f １ The parameter α in (x) is weaker than the constraint (α≧0) that must be satisfied. Therefore, the estimation model f defined in equation (1) ＊ (x) is the estimation model f１ Compared to (x), it can express a wider variety of functions, resulting in a highly accurate estimation model.

[0028] 1 is a block diagram showing an example of the hardware configuration of an information processing device 1 according to an embodiment. The information processing device 1 is realized by a computer such as a PC (Personal Computer). The information processing device 1 includes a control unit 11, an input / output interface 12, and a storage unit 13. The control unit 11, the input / output interface 12, and the storage unit 13 are connected to each other via a bus so as to be able to communicate with each other.

[0029] The control unit 11 controls the information processing device 1. The control unit 11 includes a hardware processor such as a central processing unit (CPU).

[0030] The input / output interface 12 is an interface that enables transmission and reception of information between the input device 2 and the output device 3. The input / output interface 12 may include a wired or wireless communication interface. That is, the information processing device 1, the input device 2, and the output device 3 may transmit and receive information via a network such as a LAN or the Internet.

[0031] The storage unit 13 is a storage medium. The storage unit 13 is configured by combining a nonvolatile memory that can be written to and read from at any time, such as a hard disk drive (HDD) or a solid state drive (SSD), a nonvolatile memory such as a read-only memory (ROM), and a volatile memory such as a random access memory (RAM). The storage unit 13 has a storage area including a program storage area and a data storage area. The program storage area stores an operating system (OS), middleware, and application programs required to execute various processes.

[0032] The input device 2 includes, for example, a keyboard, a pointing device, etc., which are used by the owner of the information processing device 1 (for example, a user, an administrator, or a supervisor) to input instructions to the information processing device 1. The input device 2 may also include a reader for reading data to be stored in the storage unit 13 from a memory medium such as a USB memory, or a disk device for reading such data from a disk medium. The input device 2 may also include an image scanner.

[0033] The output device 3 includes a display that displays output data to be presented to the owner from the information processing device 1, a printer that prints the output data, etc. The output device 3 may also include a writer that writes data to be input to another information processing device 1 such as a PC or a smartphone onto a memory medium such as a USB memory, and a disk device that writes such data onto a disk medium.

[0034] Fig. 2 is a block diagram showing the software configuration of the information processing device 1 according to an embodiment in relation to the hardware configuration shown in Fig. 1. The storage unit 13 includes an acquired data storage unit 131, a processing result storage unit 132, and a learning result storage unit 133.

[0035] The acquired data storage unit 131 stores various data acquired by the later-described acquisition unit 111 of the control unit 11. The data stored in the acquired data storage unit 131 is observation data, etc. The observation data includes at least the number of data points (e.g., N) and input value data at each data point.

[0036] The processing result storage unit 132 stores data processed by the inverse M matrix processing unit 112, which will be described later. The processed data is stored in the form of, for example, the above-mentioned matrix Ω Ｎ＋１ This includes positive definite kernel functions and non-negative scalar functions where is an Inverse M-Matrix.

[0037] The learning result storage unit 133 stores the parameters of the estimation model learned by the estimation model learning unit 113 (described later).

[0038] The control unit 11 includes an acquisition unit 111, an inverse M matrix processing unit 112, an estimation model learning unit 113, and an output control unit 114. These functional units are realized by the hardware processor executing an application program stored in the storage unit 13.

[0039] The acquisition unit 111 acquires observation data, such as observation data, which is data necessary for learning the parameter α of the estimation model, and stores the observation data in the acquired data storage unit 131 .

[0040] The inverse M matrix processing unit 112 calculates the matrix Ω calculated from the observation data acquired by the acquisition unit 111. Ｎ＋１ A positive definite kernel function and a non-negative scalar value that satisfy the condition that is an Inverse M-Matrix are selected. Details of this selection process will be described later.

[0041] The estimation model learning unit 113 learns the parameter α of the estimation model defined by equation (1) based on a positive definite kernel function and a non-negative scalar value. Details of the learning method will be described later.

[0042] The output control unit 114 displays the parameter α and the like learned by the estimation model learning unit 113 on the display of the output device 3. For example, the data displayed on the display of the output device 3 are a positive definite kernel function that characterizes the estimation model, a non-negative scalar value, parameters, and input value data.

[0043] 3 is a flowchart showing an example of a processing operation of the information processing device 1 according to an embodiment. For example, the control unit 11 of the information processing device 1 reads and executes a program stored in the storage unit 13, thereby realizing the operation of this flowchart.

[0044] The operation may be started at any timing, for example, when the owner of the information processing device 1 issues an instruction to learn an estimation model via the input device 2.

[0045] In step ST101, the acquisition unit 111 acquires observation data and stores the acquired observation data in the acquired data storage unit 131. Here, there are three possible examples of the acquired observation data:

[0046] Example 1: In the case of a regression problem of a non-negative function, the observed data is the number of data points (denoted as N), the input value data at each data point (x １ , x ２ , ..., x Ｎ ), and target value data (y １ , y ２ , ..., y Ｎ ) where the number of dimensions of the input value data is arbitrary, and the number of dimensions of the target value is one.

[0047] Example 2: In the case of a monotonically increasing function estimation problem, the observed data is the number of data points (denoted as N), the input value data at each data point (x １ , x ２ , ..., x Ｎ ), and target value data (y １ , y ２ , ..., y Ｎ ) where the number of dimensions of the input value data is arbitrary, and the number of dimensions of the target value is one.

[0048] Example 3: In the case of density distribution estimation problem, the observed data is the number of data points (denoted as N), the input value data at each data point (x １ , x ２ , ..., x Ｎ ) However, the number of dimensions of the input value data is arbitrary.

[0049] As can be seen from the above example, the observation data includes at least the number of data points and input value data. In addition to these data, the observation data may also include target value data, the number of dimensions of which is one.

[0050] In step ST102, the inverse M-matrix processing unit 112 performs inverse M-matrix processing. For example, the inverse M-matrix processing unit 112 acquires the observation data stored in the acquired data storage unit 131. Then, the inverse M-matrix processing unit 112 applies the ((N+1)×(N+1)) matrix Ω defined by the above-mentioned equation (3) to the observation data. Ｎ＋１ is an Inverse M-Matrix (equation (4)). ２ The inverse M matrix processing unit 112 selects the selected positive definite kernel function k(x, x'), the non-negative scalar value σ ２ , and the observation data are stored in the processing result storage unit 132.

[0051] For example, the inverse M-matrix processor 112 calculates a positive definite kernel function k(x, x′) and a non-negative scalar value σ as described below. ２ However, in one embodiment, the inverse M matrix processor 112 is not limited to these selections.

[0052] ・When the input data is one-dimensional, if the positive definite kernel function is an exponential kernel as shown in the following formula, any non-negative scalar value σ ２ For the matrix Ω Ｎ＋１ becomes an Inverse M-Matrix.

[0053]

[0054] Here, τ is the scale parameter of the positive definite kernel function. Regardless of the number of dimensions of the input data, the matrix Ω is obtained by combining the following positive definite kernel functions and non-negative scalar values: Ｎ＋１ becomes an Inverse M-Matrix.

[0055]

[0056]

[0057] In step ST103, the estimation model learning unit 113 learns the parameter α of the estimation model of equation (1) based on the observed data, the selected positive definite kernel function, and the non-negative scalar value. For example, the estimation model learning unit 113 acquires the observed data, the positive definite kernel function, and the non-negative scalar value stored in the processing result storage unit 132. The estimation model learning unit 113 learns the parameter α by solving a minimization problem using the acquired data. Then, the estimation model learning unit 113 learns the parameter α based on the input value data (x １ , x ２ , ..., x Ｎ ), a positive definite kernel function k(x, x'), a non-negative scalar value σ ２ The learned parameter α is stored in the learning result storage unit 133 .

[0058] Specifically, the estimation model learning unit 113 can learn the parameter α as in the example described below. However, it goes without saying that the method for learning the parameter α in one embodiment is not limited to this.

[0059] Example 1: Regression problem of non-negative function The estimation model learning unit 113 learns the parameter α by solving the minimization problem expressed by the following equation.

[0060]

[0061] where L(·,·) is a loss function, and γ is a weighting coefficient for the penalty term.

[0062] Example 2: In the case of an estimation problem of a monotonically increasing function The estimation model learning unit 113 learns the parameter α by solving a minimization problem expressed by the following equation.

[0063]

[0064] where L(·,·) is the loss function, γ is the weighting coefficient of the penalty term, and k ～ (x) is defined by the following formula:

[0065]

[0066] Example 3: Density Distribution Estimation Problem The estimation model learning unit 113 learns the parameter α by solving the minimization problem expressed by the following equation.

[0067]

[0068] Here, u is defined by the following formula:

[0069]

[0070] In step ST104, the output control unit 114 controls the display of the output device 3 to display the learned estimation model. The output data may include, for example, a positive definite kernel function k(x, x') that characterizes the estimation model of equation (1), a non-negative scalar value σ ２ , the learned parameter α, and the input data (x １ , x ２ , ..., x Ｎ )

[0071] (Effects of the embodiment) According to one embodiment, a parameter α is learned using an estimation model defined by formula (1). The constraints that the parameter α in formula (1) must satisfy are weaker than the constraints that the parameter α in the estimation model of Non-Patent Document 1 must satisfy, so formula (1) can express a wider variety of functions. Furthermore, formula (1) is a linear model with respect to the positive definite kernel function k(x, x'), so the calculation cost is low.

[0072] That is, the estimation model shown in equation (1) can achieve both low computational cost and high accuracy in estimating a function that takes a non-negative value.

[0073] [Other Embodiments] Note that the present invention is not limited to the above-described embodiment. For example, although three examples of observation data have been shown in the embodiments, the present invention is not limited to these examples. Furthermore, although the method for learning the parameter α of the estimation model has been described based on the observation data of the above three examples, the parameter α may be learned by other methods depending on the observation data.

[0074] The flow of each process described above is not limited to the procedures described, and the order of some steps may be changed, or some steps may be performed simultaneously in parallel. Furthermore, the series of processes described above do not need to be performed consecutively, and each step may be performed at any timing.

[0075] Furthermore, the techniques described in the above embodiments can be stored as a program (software means) that can be executed by a computer on a storage medium such as a magnetic disk (e.g., a floppy disk, a hard disk, etc.), an optical disk (e.g., a CD-ROM, a DVD, an MO, etc.), or a semiconductor memory (e.g., a ROM, a RAM, a flash memory, etc.), and can also be distributed by transmitting it via a communication medium. The program stored on the medium also includes a configuration program that configures the software means (including not only execution programs but also tables and data structures) that the computer executes. The computer that realizes this device loads the program stored on the storage medium and, in some cases, configures the software means using the configuration program, and executes the above-mentioned processing by controlling the operation of this software means. Note that the term "storage medium" as used herein is not limited to storage media for distribution, but also includes storage media such as magnetic disks and semiconductor memories installed inside the computer or in devices connected via a network.

[0076] In short, this invention is not limited to the above-described embodiments, and various modifications can be made in the implementation stage without departing from the spirit of the invention. Furthermore, the embodiments may be implemented in combination as appropriate as possible, and in such cases, the combined effects can be obtained. Furthermore, the above-described embodiments include inventions at various stages, and various inventions can be extracted by appropriately combining the disclosed multiple constituent elements.

[0077] REFERENCE SIGNS LIST 1... Information processing device 11... Control unit 111... Acquisition unit 112... Inverse M matrix processing unit 113... Estimation model learning unit 114... Output control unit 12... Input / output interface 13... Storage unit 131... Acquired data storage unit 132... Processing result storage unit 133... Learning result storage unit 2... Input device 3... Output device

Claims

1. An information processing device comprising: an acquisition unit that acquires observation data consisting of the number of data points and input value data at each of the data points; an inverse M-matrix processing unit that selects, based on the observation data, a positive definite Kernel function and a non-negative scalar value for a matrix calculated from the data points and the input value data at an arbitrary data point, such that all off-diagonal components of the inverse matrix of the matrix are negative or zero; an estimation model learning unit that learns parameters of an estimation model by solving a minimization problem using the observation data, the positive definite Kernel function, and the non-negative scalar value; and an output control unit that outputs the positive definite Kernel function, the non-negative scalar value, the learned parameters, and the input value data.

2. The information processing device according to claim 1, wherein the observation data further includes target value data at each data point, and the number of dimensions of the target value data is one.

3. An information processing method executed by a processor of an information processing device, comprising: acquiring observation data consisting of the number of data points and input value data at each of the data points; selecting, based on the observation data, a positive definite kernel function and a non-negative scalar value for a matrix calculated from the data points and the input value data at an arbitrary data point, such that the off-diagonal elements of the inverse matrix of the matrix are all negative or zero; learning parameters of an estimation model by solving a minimization problem using the observation data, the positive definite kernel function, and the non-negative scalar value; and outputting the positive definite kernel function, the non-negative scalar value, the learned parameters, and the input value data.

4. An information processing program comprising instructions to be executed by a processor of an information processing device, the instructions comprising: acquiring observation data consisting of the number of data points and input value data at each of the data points; selecting, based on the observation data, a positive definite kernel function and a non-negative scalar value for a matrix calculated from the data points and the input value data at an arbitrary data point, such that the off-diagonal elements of the inverse matrix of the matrix are all negative or zero; learning parameters of an estimation model by solving a minimization problem using the observation data, the positive definite kernel function, and the non-negative scalar value; and outputting the positive definite kernel function, the non-negative scalar value, the learned parameters, and the input value data.

Images