Learning program, learning method, and information processing device

Abstract

To provide a learning program capable of generating a generative model with high accuracy, a learning method, and an information processing device.SOLUTION: In learning a machine learning model capable of generating self-sample data on the basis of a probability distribution estimated for sample data, a learning program causes a computer to perform processing for: generating self-sample data generated from the machine learning model during learning; and learning the machine learning model so that a third loss function becomes small, which includes a first loss function for unsupervised learning based on the sample data and a second loss function for supervised learning based on the self-sample data.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a learning program, a learning method, and an information processing device. [Background technology]

[0002] Techniques for generating generative models by machine learning probability distributions have been disclosed (see, for example, Non-Patent Documents 1 and 2). [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Huang, L. and Wang “Accelerated monte carlo simulations with restricted boltzman machines” Physical Review B, 95(3):035105 [Non-patent document 2] Midgley, LI, Stimper, V., Simm, GN, Sch¨ olkopf, B., and Hern´andez-Lobato, JM (2022). Flow annealed importance sampling bootstrap. arXiv preprint arXiv:2208.01893. Summary of the Invention [Problem to be solved by the invention]

[0004] It is difficult to generate a generative model with high accuracy, whether it is data-based learning, where sample data is prepared, or data-less learning, where sample data is not prepared.

[0005] In one aspect, the present invention aims to provide a learning program, a learning method, and an information processing device that are capable of generating a generative model with high accuracy. [Means for solving the problem]

[0006] In one aspect, a learning program causes a computer to execute the following processes in training a machine learning model capable of generating self-sample data based on a probability distribution estimated for sample data: generating the self-sample data from the machine learning model during training; and training the machine learning model so as to reduce a third loss function including a first loss function for unsupervised learning based on the sample data and a second loss function for supervised learning based on the sample data and the self-sample data. [Effects of the Invention]

[0007] It is possible to generate generative models with high accuracy. [Brief explanation of the drawings]

[0008] [Figure 1] 1A is a functional block diagram showing the overall configuration of an information processing device, and FIG. 1B is a hardware configuration diagram of the information processing device. [Figure 2] 10 is a flowchart illustrating an example of an operation of the information processing device. [Figure 3] 10(a) and 10(b) are diagrams showing the learning results. DETAILED DESCRIPTION OF THE INVENTION

[0009] In the field of statistics, techniques have been proposed for sampling from a probability distribution under circumstances where a functional form other than the normalization constant (distribution function) of the probability distribution is given. For example, in the field of proteins, techniques for sampling from a probability distribution have been proposed. Specifically, in the following formula (1), p(x) is the probability distribution. In the following formula (1), Z is the normalization constant. In the following formula (1), it is difficult to evaluate the value of Z, and the energy function H(x) can be used within a realistic calculation time.

number

[0010] In the field of machine learning, technology (generative modeling) that models unknown complex probability distributions using a machine learning model q(x) is being developed. In particular, as shown in the following equation (2), the generative model q θ (x) is developing as the mainstream.

number

[0011] In recent years, advances have been made in the application of machine learning techniques to train generative models under conditions where a function form other than the normalization constant of the probability distribution is given. For example, frameworks have been proposed to speed up sampling under conditions where a function form other than the normalization constant of the probability distribution is given. Specific examples are explained below.

[0012] First, we will explain learning when sample data is prepared in advance (data-based learning). First, we will explain supervised learning, which is a type of data-based learning. In supervised learning, the target energy function H(x) and sample data D of its distribution are prepared. data Sample data D is available. data is expressed by the following formula (3).

number

[0013] Sample Data D data satisfies the following formula (4). Since "E" in the following formula (4) represents expectation, the left side is D data The expected values ​​obtained from the histogram are shown.

number

[0014] In supervised learning, a target energy function H(x) is used as training data to define an appropriate loss function, and the parameter θ is determined so that the loss function becomes small. For example, in Huang, L. and Wang "Accelerated Monte Carlo simulations with restricted Boltzman machines" Physical Review B, 95(3):035105, training data is used to update parameters by optimizing the following equation (7) using the forward f-divergence of the following equation (5) (f-divergence is given by the following equation (6)).

number

number

number

[0015] Next, we will explain unsupervised learning, a type of data-based learning. In unsupervised learning, the target energy function H(x) is not used as training data, so sample data expressed by the above formula (3) is prepared. Using the maximum likelihood method, the parameter θ is determined so that the loss function in the following formula (8) is small.

number

[0016] Next, we will explain learning when sample data is not prepared in advance (data-free learning). In this case, only the function form H(x) other than the normalization constant of the target probability distribution can be used. For example, in Midgley, LI, Stimper, V., Simm, GN, Schölkopf, B., and Hern´andez-Lobato, JM (2022). Flow annealed importance sampling bootstrap. arXiv preprint arXiv:2208.01893., the loss function is expressed by the following equations (9) and (10). Note that the following equation (10) represents self-sampled data. Note that the "x" in the following equation (10) μ ~q θ (x)" is q θ represents a random variable that follows (x).

number

number

[0017] Here, the advantages and disadvantages of the data-based learning and data-less learning will be summarized.

[0018] First, we will explain supervised learning, which is one of the data-based learning methods. In supervised learning, the function form H(x) is directly used, so the advantage is that the energy regression performance is high. On the other hand, the generalization performance is low, and the trained generative model q θ A drawback is that it is difficult to generate sample data from (x) using the following formula (11) in a realistic time. For example, a generative model q can be generated in a realistic time using a Markov chain Monte Carlo method or the like. θ (x) It is difficult to obtain sample data.

number

[0019] Next, we will explain unsupervised learning, which is one of the data-based learning methods. In unsupervised learning, learning is possible even when the function form H(x) is not required. In the process of learning the parameter θ, the generative model q θ (x) uses additional sample data (self-sampled data), so the implicitly easy-to-sample distribution q θ On the other hand, since the function form H(x) is not directly used, the regression performance is lower.

[0020] Next, we will explain data-less learning. In data-less learning, sample data D data One advantage of this method is that it can learn only the target energy function H(x), since it does not require the . However, it has the disadvantage of mode collapse, which means that it can only learn one mode of a probability distribution with multiple modes (a multi-peak probability distribution with multiple peaks).

[0021] For the reasons described above, it is difficult to generate a generative model in both data-based learning and data-less learning. Therefore, in the following examples, examples in which a generative model can be generated will be described. [Example]

[0022] First, the principle of this embodiment will be described.

[0023] By adding information about the target energy function H(x) as regularization to the unsupervised learning of data-based learning, a model with high energy regression performance and higher generalization performance is generated. Next, by using self-sample data in unsupervised learning, the generative model q θ We use implicit regularization to make (x) a distribution that is easy to sample, and add self-sampled data to the loss of supervised learning. Also, the self-sampled data D selfSince including only the loss in supervised learning can lead to mode collapse, as in data-free learning, we also improve the regression performance in the self-sampled domain generated by the model outside the sample data.

[0024] The above can be summarized as follows. Specifically, the premise is training of a machine learning model (generative model) that can generate additional sample data (self-sample data) based on a probability distribution estimated for sample data. The self-sample data generated by the machine learning model during training is obtained. Next, the machine learning model is trained so as to minimize a loss function that includes a loss function for unsupervised learning based on the sample data and a loss function for supervised learning based on the sample data and the additional sample data.

[0025] For example, the loss function of the following formula (12) is minimized. In the following formula (12), "t" represents the time that characterizes one step of the learning algorithm. L unsup (θ;D) represents the loss function for unsupervised learning. unsup represents the coefficient of the loss function in unsupervised learning. sup (θ;D,H) represents the loss function for supervised learning. sup (t) represents the coefficient of the loss function for supervised learning. "sup" is an abbreviation for Supervised, meaning "supervised." "unsup" is an abbreviation for Unsupervised, meaning "unsupervised."

number

[0026] By adopting this method, we can incorporate elements of supervised learning in data-based learning, and directly using the function form H(x) improves the energy regression performance. Next, we can incorporate elements of unsupervised learning in data-based learning, and by using self-sampled data, we can implicitly sample the distribution q θ(x) is learned. As a result, it becomes possible to generate a generative model with high accuracy.

[0027] Next, a device configuration for realizing the above solution principle will be described. Fig. 1(a) is a functional block diagram showing the overall configuration of an information processing device 100 according to this embodiment. The information processing device 100 is, for example, a server for optimization processing. As illustrated in Fig. 1(a), the information processing device 100 functions as a probability distribution storage unit 10, a generative model storage unit 20, a self sample generation unit 30, a self sample storage unit 40, a sample storage unit 50, a function calculation unit 60, a gradient calculation unit 70, a gradient storage unit 80, and the like.

[0028] Fig. 1(b) is a hardware configuration diagram of the information processing device 100. As illustrated in Fig. 1(b), the information processing device 100 includes a CPU 101, a RAM 102, a storage device 103, an input device 104, a display device 105, and the like.

[0029] The CPU (Central Processing Unit) 101 is a central processing unit. The CPU 101 includes one or more cores. The RAM (Random Access Memory) 102 is a volatile memory that temporarily stores programs executed by the CPU 101, data processed by the CPU 101, and the like. The storage device 103 is a non-volatile storage device. Examples of the storage device 103 include a read-only memory (ROM), a solid-state drive (SSD) such as a flash memory, and a hard disk driven by a hard disk drive. The storage device 103 stores a learning program. The input device 104 is a device for a user to input necessary information, such as a keyboard or a mouse. The display device 105 is a display device that displays learning results on a screen. The CPU 101 executes the learning program to realize each component of the information processing device 100. Each component of the information processing device 100 may be implemented using hardware such as a dedicated circuit.

[0030] 2 is a flowchart showing an example of the operation of information processing device 100 when a generative model is machine-learned. Machine learning of a generative model will be described below.

[0031] 2, the function calculation unit 60 initializes the generative model (step S1). Specifically, the function calculation unit 60 sets the model parameters stored in the generative model storage unit 20 to predetermined initial values.

[0032] Next, the function calculation unit 60 embeds the optimization problem (step S2). Specifically, first, the self sample generation unit 30 acquires the sample data (the above formula (3)) stored in the sample storage unit 50. Next, the self sample generation unit 30 generates the self sample data from the generative model (the generative model in which the model parameters are initial values) stored in the generative model storage unit 20. Next, the function calculation unit 60 generates the loss function of the above formula (12).

[0033] Next, the function calculation unit 60 calculates H(x) using the sample data and the self-sample data obtained in step S2 (step S3).

[0034] Next, the function calculation unit 60 uses H(x) obtained in step S3 to calculate the loss function L(θ) that minimizes the loss function of the above equation (12) (step S4).

[0035] Next, the gradient calculation unit 70 calculates the gradient of the loss function L(θ) (step S5). The gradient calculated in step S5 is stored in the gradient storage unit 80.

[0036] Next, the function calculation unit 60 updates the parameter θ using the gradient stored in the gradient storage unit 80 (step S6).

[0037] Next, the function calculation unit 60 determines whether or not the convergence condition is satisfied (step S7). For example, it determines whether or not the loss function L(θ) is no longer smaller than a specified value even when step S6 is repeatedly executed. If the determination in step S7 is "No," the process is executed again from step S3.

[0038] If the determination in step S7 is "Yes," execution of the flowchart ends. In this case, the generative model storage unit 20 stores the model parameters that result in the smallest loss function. In addition, the display device 105 may display the learning results, such as the model parameters stored in the generative model storage unit 20.

[0039] When actually using a machine-learned generative model, the self sample generation unit 30 acquires the sample data (the above formula (3)) obtained from the probability distribution stored in the probability distribution storage unit 10 and stored in the sample storage unit 50. Next, the self sample generation unit 30 generates self sample data from the generative model stored in the generative model storage unit 20. This makes it possible to use the generative model.

[0040] The effects of this embodiment will be verified below.

[0041] The generative model is the restricted Boltzmann machine H θ The loss function is expressed by the following equation (13).

number

[0042] Figure 3(a) shows the D self 3(a) and 3(b), the thick lines represent the true energy set (the energy distribution sampled from the target probability distribution) H data ={H(x)|x~p(x)}, and the thin line represents the energy set H self= {H(x)|x~qθ(x)}. In Fig. 3(a), H self represents the energy distribution of the sampled states from a generative model generated by general data-based learning. In Figure 3(b), H self represents the energy distribution sampled from the generative model generated by learning in this embodiment. In Fig. 3(a), the peak region indicated by the thick line and the peak region indicated by the thin line are separated. In contrast, in Fig. 3(b), the peak region indicated by the thick line and the peak region indicated by the thin line are almost identical.

[0043] Therefore, the distance of the following formula (14) was calculated. The distance of the following formula (14) is the KL divergence, and means the distance from the target probability distribution to the energy distribution in a sampled state. In general data-based learning, W1 = 184, and in learning according to this embodiment, W1 = 5.05. From this result, it can be seen that the energy distribution in a sampled state from the generative model generated by learning according to this embodiment has a distance from the target probability distribution that is 36.4 times better than the energy distribution in a sampled state from the generative model generated by general data-based learning.

number

[0044] In the above example, we use the restricted Boltzmann machine H θ Although the effectiveness was confirmed for the above, it can also be applied to other energy-based models, flow-based models, and autoregressive models whose likelihoods can be easily evaluated.

[0045] In the above embodiment, the self sample generation unit 30 is an example of a self sample generation unit that generates self sample data from a machine learning model in the middle of learning in learning a machine learning model that can generate self sample data based on a probability distribution estimated for sample data. The function calculation unit 60, the gradient calculation unit 70, and the gradient storage unit 80 are an example of a learning unit that trains the machine learning model so as to reduce a third loss function that includes a first loss function for unsupervised learning based on sample data and a second loss function for supervised learning based on sample data and self sample data.

[0046] Although the embodiments of the present invention have been described in detail above, the present invention is not limited to such specific embodiments, and various modifications and variations are possible within the scope of the gist of the present invention as defined in the claims. (Appendix 1) In training a machine learning model capable of generating self-sampled data based on a probability distribution estimated for sampled data, On the computer, A process of generating the self-sample data generated from the machine learning model during training; a process of training the machine learning model so as to reduce a third loss function including a first loss function of unsupervised learning based on the sample data and a second loss function of supervised learning based on the sample data and the self-sample data; A learning program characterized by executing the above. (Appendix 2) The learning program of claim 1, wherein the second loss function incorporates an energy function calculated from the self-sampled data as a penalty. (Appendix 3) In training a machine learning model capable of generating self-sampled data based on a probability distribution estimated for sampled data, The computer A process of generating the self-sample data generated from the machine learning model during training; a process of training the machine learning model so as to reduce a third loss function including a first loss function of unsupervised learning based on the sample data and a second loss function of supervised learning based on the sample data and the self-sample data; A learning method characterized by performing the following. (Appendix 4) 4. The learning method according to claim 3, wherein the second loss function incorporates an energy function calculated from the self-sampled data as a penalty. (Appendix 5) In learning a machine learning model capable of generating self sample data based on a probability distribution estimated for sample data, a self sample generation unit generates the self sample data generated from the machine learning model during learning; a learning unit that learns the machine learning model so as to reduce a third loss function that includes a first loss function of unsupervised learning based on the sample data and a second loss function of supervised learning based on the sample data and the self-sample data; An information processing device comprising: (Appendix 6) The information processing device according to claim 5, wherein the learning unit incorporates an energy function calculated from the self-sample data into the second loss function as a penalty. [Explanation of symbols]

[0047] 10 Probability distribution storage section 20 Generative model storage 30 Self-sample generation unit 40 Self-sample storage unit 50 Sample Storage Unit 60 Function Calculation Section 70 Gradient calculation unit 80 Gradient storage unit 100 Information processing device

Claims

1. In training a machine learning model capable of generating self-sampled data based on a probability distribution estimated for sampled data, On the computer, A process of generating the self-sample data generated from the machine learning model during training; a process of training the machine learning model so as to reduce a third loss function including a first loss function of unsupervised learning based on the sample data and a second loss function of supervised learning based on the sample data and the self-sample data; A learning program characterized by executing the above.

2. 2. The learning program according to claim 1, wherein the second loss function incorporates an energy function calculated from the self-sampled data as a penalty.

3. In training a machine learning model capable of generating self-sampled data based on a probability distribution estimated for sampled data, The computer A process of generating the self-sample data generated from the machine learning model during training; a process of training the machine learning model so as to reduce a third loss function including a first loss function of unsupervised learning based on the sample data and a second loss function of supervised learning based on the sample data and the self-sample data; A learning method characterized by performing the following.

4. In learning a machine learning model capable of generating self sample data based on a probability distribution estimated for sample data, a self sample generation unit generates the self sample data generated from the machine learning model during learning; a learning unit that learns the machine learning model so as to reduce a third loss function including a first loss function of unsupervised learning based on the sample data and a second loss function of supervised learning based on the sample data and the self-sample data; An information processing device comprising:

Images