Learning program, learning method and information processing unit

Abstract

To provide a learning program, a learning method and an information processing unit that can learn a complicated probability distribution.SOLUTION: A computer is caused to execute: first processing to generate a probability distribution model by learning a probability distribution having less peaks than a target probability distribution; and second processing to generate a new probability distribution model by learning a probability distribution closer to the target probability distribution than the probability distribution having been learnt by the use of parameters of the probability distribution model having been generated.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a learning program, a learning method, and an information processing device. [Background technology]

[0002] Techniques for learning a model have been disclosed (see, for example, Patent Documents 1 to 4). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-95600 A [Patent Document 2] Japanese Patent Application Publication No. 2023-129309 [Patent Document 3] U.S. Patent Publication No. 2022 / 8358373 [Patent Document 4] U.S. Patent Publication No. 2019 / 0347570 Summary of the Invention [Problem to be solved by the invention]

[0004] However, even if we try to model complex probability distributions using machine learning, it is difficult to generate models with high accuracy.

[0005] In one aspect, the present invention aims to provide a learning program, a learning method, and an information processing device that are capable of learning complex probability distributions. [Means for solving the problem]

[0006] In one aspect, the learning program causes a computer to perform a first process of generating a probability distribution model by learning a probability distribution having a simpler distribution than a target probability distribution, and a second process of generating a new probability distribution model by learning using parameters of the generated probability distribution model for a probability distribution that is closer to the target probability distribution than the learned probability distribution. [Effects of the Invention]

[0007] It is capable of learning complex probability distributions. [Brief explanation of the drawings]

[0008] [Figure 1] (a) is a diagram illustrating a simple probability distribution, and (b) is a diagram illustrating a complex probability distribution. [Figure 2] 1A is a functional block diagram showing the overall configuration of an information processing device, and FIG. 1B is a hardware configuration diagram of the information processing device. [Figure 3] 10 is a flowchart illustrating an example of an operation of the information processing device. [Figure 4] FIG. 10 is a diagram showing a learning result. DETAILED DESCRIPTION OF THE INVENTION

[0009] In the field of machine learning, technology (generative models) that model unknown complex probability distributions using machine learning has been developed. In particular, the generative model Q θ (x) is currently being developed as the mainstream. For example, generative model Q θ Examples of (x) include RBM: Restricted Boltzmann Machine, VAE: Variational Autoencoder, and GAN: Generative Adversarial Network.

[0010] In the field of statistics, techniques have been proposed for sampling from a probability distribution when a functional form other than the normalization constant of the probability distribution is given. For example, techniques for sampling from a probability distribution have been proposed in protein Bayesian statistical modeling. Specifically, a functional form other than the normalization constant is assumed to be given, as shown in the following equation (1). Note that in the following equation (1), Z is a normalization constant, making it difficult to evaluate the value of Z.

number

[0011] In recent years, a framework has been proposed that applies machine learning technology to speed up sampling in situations where a function form P other than the normalization constant of the probability distribution is given. A specific example will be explained below.

[0012] First, we will explain learning when training data is prepared in advance (data-based learning). In data-based learning, the generative model Q θ For (x), using the training data D of the following formula (2) prepared in advance and the function of the following formula (3) as a teacher, an appropriate loss function is defined, and the parameter θ is determined so that the loss function becomes small.

number

number

[0013] For example, in Huang, L. and Wang “Accelerated Monte Carlo simulations with restricted boltzman machines” Physical Review B, 95(3):035105, parameters are updated by optimizing the following equation (4) using training data.

number

[0014] Furthermore, when the function of the above formula (3) is not obtained in data-based learning, the parameters are determined by maximizing the logarithmic likelihood of the following formula (5) using only the learning data D.

number

[0015] Next, we will explain learning when no training data is prepared in advance (data-free learning). In this case, the generative model Q θ A sample sequence (self-sample) of the following equation (6) is generated from (x), and an appropriate loss is defined from the self-sample and the function of the following equation (7), and the parameter θ is determined so that the loss function becomes small.

number

number

[0016] For example, Albergo, MS, Kanwar, G., and Shanahan,PE(2019), “Flow-based generative models for markov chain Monte Carlo in lattice field theory”, Physical Review D, 100(3):034515, uses the self-samples in equation (6) above to determine parameters to minimize the KL divergence in equation (8) below.

number

[0017] Here, we will explain the probability distribution to be modeled, and summarize the advantages and disadvantages of the data-based learning and data-less learning described above.

[0018] First, we will explain the image of the probability distribution to be learned. Probability distributions include simple probability distributions, which have simple distributions, and complex probability distributions, which have complex distributions. Figure 1(a) is a diagram illustrating a simple probability distribution. Figure 1(b) is a diagram illustrating a complex probability distribution. As illustrated in Figure 1(b), a complex probability distribution is a multi-modal probability distribution with multiple peaks. As illustrated in Figure 1(a), a simple probability distribution is a probability distribution with fewer peaks than a complex probability distribution. As an example, a simple probability distribution is a probability distribution with only one peak.

[0019] Note that Figures 1(a) and 1(b) are explained in two dimensions for simplicity, but even in multidimensional spaces of three or more dimensions, a complex probability distribution is a multi-peaked probability distribution with multiple peaks, while a simple probability distribution is a probability distribution with fewer peaks than a complex probability distribution.

[0020] Data-based learning has the advantage that it can learn multi-peak distributions, even for complex probability distributions like the one shown in Figure 1(b), as long as training data for learning the regions of each peak is prepared. On the other hand, it has the disadvantage that training data is required in advance. Another disadvantage is that it is difficult to use the above equation (7) effectively, making it difficult to output an appropriate value for the parameter θ.

[0021] Next, data-less learning has the advantage that there is no need to prepare training data in advance. On the other hand, it has the disadvantage that it is difficult to learn only a simple probability distribution such as the example shown in Figure 1(a) because there is no training data to learn the regions of each peak. When trying to learn a multi-peak distribution such as the example shown in Figure 1(b) using data-less learning, it is typically difficult to learn only one peak.

[0022] For these reasons, learning complex probability distributions (for example, multi-modal distributions with a large number of clusters and multiple peaks) is difficult in both data-based and data-free learning.

[0023] Therefore, in the following embodiment, an example in which a complex probability distribution can be learned will be described. [Example]

[0024] First, the principle of this embodiment will be described.

[0025] It is assumed that the functional form of the desired probability distribution, P(x), or the functional form obtained by removing the normalization constant from the functional form, P(x), has been obtained. Since it does not have a significant effect on which functional form is obtained, hereafter we will refer to it as P(x) without distinction, even if the functional form obtained is one without the normalization constant.

[0026] In this embodiment, a probability distribution simpler than the target probability distribution to be modeled is trained in advance, and the parameters of the obtained model are used as initial values ​​to train the target probability distribution. However, since it is generally difficult to prepare a sufficiently simple distribution that is close to the target probability distribution, this method is performed in multiple stages, so that training is performed in order from the easiest distribution to the easiest distribution.

[0027] Specifically, we first introduce a parameter λ that represents the complexity of the probability distribution, and extend the target probability distribution P(x) to a parameterized probability distribution P(x;γ) that satisfies the following condition: ·0<γ≦1, and when γ=1, P(x;γ) matches the target probability distribution. For γ<γ', P(x;γ') is closer to the target probability distribution than P(x;γ).

[0028] Here, "close to the target probability distribution" means that the Wasserstein metric is small.

[0029] Next, γ K = 1, the monotonically increasing sequence γ0<γ1<…<γ K = 1. In learning P(x;γ0), a random value is set as the initial value of the parameter. The probability distribution P(x;γ K ) when learning with the goal of P(x;γK-1 ) are used as initial values ​​for the learning parameters of the model trained for this purpose.

[0030] By adopting such a method, the learning process starts with learning a simpler probability distribution. This increases the accuracy of machine learning even when learning without data. Furthermore, even if the learning target probability distribution approaches the target probability distribution and becomes more complex as a result, the parameters obtained by learning the simpler probability distribution can be used, thereby increasing the accuracy of machine learning. From the above, according to this embodiment, learning can be performed with high accuracy even when the target probability distribution is a complex probability distribution.

[0031] Next, a device configuration for realizing the above solution principle will be described. Fig. 2(a) is a functional block diagram showing the overall configuration of an information processing device 100 according to this embodiment. The information processing device 100 is, for example, a server for optimization processing. As illustrated in Fig. 2(a), the information processing device 100 functions as a function sequence creation unit 10, a learning unit 20, etc.

[0032] Fig. 2(b) is a hardware configuration diagram of the information processing device 100. As illustrated in Fig. 2(b), the information processing device 100 includes a CPU 101, a RAM 102, a storage device 103, an input device 104, a display device 105, and the like.

[0033] The CPU (Central Processing Unit) 101 is a central processing unit. The CPU 101 includes one or more cores. The RAM (Random Access Memory) 102 is a volatile memory that temporarily stores programs executed by the CPU 101, data processed by the CPU 101, and the like. The storage device 103 is a non-volatile storage device. As the storage device 103, for example, a solid-state drive (SSD) such as a ROM (Read Only Memory), a flash memory, a hard disk driven by a hard disk drive, or the like can be used. The storage device 103 stores a learning program. The input device 104 is a device for the user to input necessary information, such as a keyboard, a mouse, and the like. The display device 105 is a display device that displays the learning result of the learning unit 20 on the screen. By the CPU 101 executing the learning program, each part of the information processing device 100 is realized. Note that, as each part of the information processing device 100, hardware such as a dedicated circuit may be used.

[0034] FIG. 3 is a flowchart showing an example of the operation of the information processing device 100. As illustrated in FIG. 3, the learning unit 20 initializes the learning model (step S1).

[0035] Next, the function sequence creation unit 10 creates a function sequence representing a probability distribution, which is represented by the following formula (9), so as to satisfy the following (step S2). · For i < j, γ i < γ j becomes. · For γ < γ´, P(x; γ´) is closer to the target probability distribution than P(x; γ). Here, i takes any value of an integer from 0 to K. 0 < γ ≤ 1, and when γ = 1, P(x; γ) coincides with the target probability distribution.

Equation

[0036] Next, the learning unit 20 sets i = 0 (step S3). Thereby, the value of i is initialized.

[0037] Next, the learning unit 20 calculates P(x;γ i ) as a goal, and the model is trained (step S4).

[0038] Next, the learning unit 20 increments the value of i by 1 by setting i=i+1 (step S5).

[0039] Next, the learning unit 20 determines whether the value of i has become k+1 by determining whether i==k+1 (step S6). K It can be determined whether model learning is complete for all γ values ​​up to

[0040] If the determination in step S6 is "No", the process is executed again from step S4. If the determination in step S6 is "Yes", the execution of the flowchart ends. The display device 105 displays the learning result.

[0041] Next, a more specific example will be described. As the above probability distribution P(x;γ), an extended distribution such as that shown in the following formula (10) can be used.

number

[0042] Next, the parameters γ0<γ1<…<γ K Consider a situation where a sample sequence of P(x;γ) is given. First, learning is performed on data that follows P(x;γ), and then learning is performed on data that follows P(x;γ) using the obtained learning parameters as initial values. After that, learning is performed in the same way on data that follows P(x;γ) k-1 ) is used as the initial value for the learning parameters obtained by learning data that follows P(x;γ k ) is used for learning.

[0043] Next, another example will be explained. A probability distribution S that is easy to sample is introduced into the above probability distribution P(x;γ), and an extended distribution such as the following formula (11) is considered.

number

[0044] Next, the parameters γ0<γ1<…<γ K Consider a situation where a sample sequence of P(x;γ) is given. Note that "S" in the above formula (11) can be a uniform distribution, a Bernoulli distribution, a normal distribution, etc. First, learning is performed on data that follows P(x;γ0), and then learning is performed on data that follows P(x;γ1) using the obtained learning parameters as initial values. Thereafter, similarly, learning is performed on data that follows P(x;γ k-1 ) is used as the initial value for the learning parameters obtained by learning data that follows P(x;γ k ) is used for learning.

[0045] The effects of this embodiment will be verified below.

[0046] We performed PT (Parallel Tempering) on ​​Gset G1, a benchmark problem for MCP (Max Cut Problem), to prepare training data. In PT, we use the Boltzmann distribution at different temperatures to operate the MCMC (Markov Chain Monte Carlo) method, exchanging states between different temperatures at a fixed frequency. This allows for more efficient sampling at lower temperatures than conventional MCMC methods.

[0047] Next, a restricted Boltzmann machine (RBM) was trained using data-based training. In this case, the loss function MSE(θ) is the mean square error between the energy of the RBM and the energy of the MCP, as shown in the following equation (12). Note that θ is a training parameter, β is the inverse temperature, D is the training data, and F θ (x) is the energy of the RBM, and E(x) is the energy of the MCP.

number

[0048] Figure 4 shows the learning results. The horizontal axis of Figure 4 represents epochs, and the vertical axis represents MSE. Figure 4 also shows the results of general data-based learning. In data-based learning, 8,000 epochs of learning were performed using only the learning data obtained at the lowest temperature. In the above-mentioned PT, since the Boltzmann distribution generally becomes more complex as the temperature decreases, by performing PT so that the target probability distribution is the lowest temperature, it is possible to prepare a complex distribution from a simple distribution that approaches the target distribution, and to obtain learning data at each temperature. In the method of this example, learning was performed by approaching the value of γ to 1 every 500 epochs, starting from the highest temperature. Specifically, a total of 16 temperatures were prepared, with the lowest temperature set to 0.5 and the highest temperature set to 4.0. The intermediate temperatures were determined using N = 16 and the following equation (13).

number

[0049] 4, the MSE was 49.4 in the general data-based learning, whereas it was 37.4 in the method according to this embodiment. This result shows that the performance was improved in the method according to this embodiment.

[0050] In the above embodiment, the learning with γ0 corresponds to the first process of generating a probability distribution model by learning a probability distribution that is simpler than the target probability distribution. KThe learning in corresponds to a second process of generating a new probability distribution model by learning using parameters of the generated probability distribution model for a probability distribution that is closer to the target probability distribution than the learned probability distribution. The learning unit 20 corresponds to a learning unit that executes a first process of generating a probability distribution model by learning a probability distribution that has a simpler distribution than the target probability distribution, and a second process of generating a new probability distribution model by learning using parameters of the generated probability distribution model for a probability distribution that is closer to the target probability distribution than the learned probability distribution.

[0051] Although the embodiments of the present invention have been described in detail above, the present invention is not limited to such specific embodiments, and various modifications and variations are possible within the scope of the gist of the present invention as defined in the claims. (Appendix 1) On the computer, a first process of generating a probability distribution model by learning a probability distribution that has fewer peaks than a target probability distribution; and a second process of generating a new probability distribution model by learning using parameters of the generated probability distribution model for a probability distribution that is closer to the target probability distribution than the learned probability distribution. (Appendix 2) For P(x), which represents the target probability distribution, 0<γ≦1, P(x;γ)≡Pγ(x) / Z(γ), and Z(γ)=∫dxP γ A learning program according to claim 1, characterized in that a distribution (x) is prepared, and the value of γ during the second process is made larger than the value of γ during the first process. (Appendix 3) 2. The learning program according to claim 1, wherein the computer is caused to repeatedly execute the second process two or more times. (Appendix 4) For P(x), which represents the target probability distribution, 0<γ≦1, P(x;γ)≡P γ (x) / Z(γ), Z(γ)=∫dxP γA learning program according to claim 3, characterized in that a distribution (x) is prepared, the value of γ in the second process is made larger than the value of γ in the first process, and the value of γ is increased each time the second process is repeated. (Appendix 5) For P(x) representing the target probability distribution, 0<γ≦1, P(x;γ)≡Pγ(x)S 1-γ (x) / Z(γ), Z(γ)=∫dxP γ (x)S 1-γ A learning program according to claim 1, characterized in that a distribution (x) is prepared, and the value of γ during the second process is made larger than the value of γ during the first process. (Appendix 6) causing the computer to repeatedly execute the second process two or more times; For P(x), which represents the target probability distribution, 0<γ≦1, P(x;γ)≡P γ (x) / Z(γ), Z(γ)=∫dxP γ (x)S 1-γ A learning program according to claim 1, characterized in that a distribution (x) is prepared, the value of γ in the second process is made larger than the value of γ in the first process, and the value of γ is increased each time the second process is repeated. (Appendix 7) The computer a first process of generating a probability distribution model by learning a probability distribution that has fewer peaks than a target probability distribution; a second process of generating a new probability distribution model by learning using parameters of the generated probability distribution model for a probability distribution that is closer to the target probability distribution than the trained probability distribution. (Appendix 8) For P(x), which represents the target probability distribution, 0<γ≦1, P(x;γ)≡Pγ(x) / Z(γ), and Z(γ)=∫dxP γ The learning method described in Appendix 7, characterized in that a distribution (x) is prepared, and the value of γ in the second process is made larger than the value of γ in the first process. (Appendix 9) 8. The learning method according to claim 7, wherein the computer is caused to repeatedly execute the second process two or more times. (Appendix 10) For P(x), which represents the target probability distribution, 0<γ≦1, P(x;γ)≡P γ (x) / Z(γ), Z(γ)=∫dxP γ (x), the value of γ in the second process is made larger than the value of γ in the first process, and the value of γ is increased each time the second process is repeated. (Appendix 11) For P(x) representing the target probability distribution, 0<γ≦1, P(x;γ)≡Pγ(x)S 1-γ (x) / Z(γ), Z(γ)=∫dxP γ (x)S 1-γ The learning method described in Appendix 7, characterized in that a distribution (x) is prepared, and the value of γ in the second process is made larger than the value of γ in the first process. (Appendix 12) causing the computer to repeatedly execute the second process two or more times; For P(x), which represents the target probability distribution, 0<γ≦1, P(x;γ)≡P γ (x) / Z(γ), Z(γ)=∫dxP γ (x)S 1-γ (x), the value of γ in the second process is made larger than the value of γ in the first process, and the value of γ is increased each time the second process is repeated. (Appendix 13) An information processing device comprising: a learning unit that executes a first process of generating a probability distribution model by learning a probability distribution having fewer peaks than a target probability distribution; and a second process of generating a new probability distribution model by learning, using parameters of the generated probability distribution model, a probability distribution that is closer to the target probability distribution than the learned probability distribution. (Appendix 14) The learning unit is configured to set a target probability distribution P(x) such that 0<γ≦1, P(x;γ)≡Pγ(x) / Z(γ), and Z(γ)=∫dxPγ 14. The information processing device according to claim 13, wherein a distribution (x) is prepared, and the value of γ during the second processing is made larger than the value of γ during the first processing. (Appendix 15) The information processing device according to claim 13, wherein the learning unit repeatedly executes the second process two or more times. (Appendix 16) The learning unit is configured to set 0<γ≦1 and P(x;γ)≡P for P(x) representing the target probability distribution. γ (x) / Z(γ), Z(γ)=∫dxP γ The information processing device according to claim 15, characterized in that a distribution (x) is prepared, the value of γ during the second processing is made larger than the value of γ during the first processing, and the value of γ is increased each time the second processing is repeated. (Appendix 17) The learning unit is configured to set 0<γ≦1, P(x;γ)≡Pγ(x)S for P(x) representing the target probability distribution. 1-γ (x) / Z(γ), Z(γ)=∫dxP γ (x)S 1-γ 14. The information processing device according to claim 13, wherein a distribution (x) is prepared, and the value of γ during the second processing is made larger than the value of γ during the first processing. (Appendix 18) the learning unit repeatedly executes the second process two or more times; The learning unit is configured to set 0<γ≦1 and P(x;γ)≡P for P(x) representing the target probability distribution. γ (x) / Z(γ), Z(γ)=∫dxP γ (x)S 1-γ The information processing device according to claim 13, characterized in that a distribution (x) is prepared, the value of γ during the second processing is made larger than the value of γ during the first processing, and the value of γ is increased each time the second processing is repeated. [Explanation of symbols]

[0052] 10 Function sequence creation section 20 Learning Department 100 Information processing device 101 CPU 102 RAM 103 Storage device 104 Input Device 105 Display device

Claims

1. On the computer, a first process of generating a probability distribution model by learning a probability distribution that has fewer peaks than a target probability distribution; A learning program characterized by executing a second process of generating a new probability distribution model by learning using parameters of the generated probability distribution model for a probability distribution that is closer to the target probability distribution than the learned probability distribution.

2. For P(x) representing the target probability distribution, 0<γ≦1, P(x;γ)≡Pγ(x) / Z(γ), Z(γ)=∫dxP γ 2. The learning program according to claim 1, wherein a distribution of (x) is prepared, and the value of γ in the second process is made larger than the value of γ in the first process.

3. 2. The learning program according to claim 1, wherein the computer is caused to repeatedly execute the second process two or more times.

4. For P(x) representing the target probability distribution, 0<γ≦1, P(x;γ)≡P γ (x) / Z(γ), Z(γ)=∫dxP γ The learning program according to claim 3, characterized in that a distribution (x) is prepared, the value of γ in the second process is made larger than the value of γ in the first process, and the value of γ is increased each time the second process is repeated.

5. For P(x) representing the target probability distribution, 0<γ≦1, P(x;γ)≡Pγ(x)S １－γ (x) / Z(γ), Z(γ)=∫dxP γ (x)S １－γ 2. The learning program according to claim 1, wherein a distribution of (x) is prepared, and the value of γ in the second process is made larger than the value of γ in the first process.

6. causing the computer to repeatedly execute the second process two or more times; For P(x) representing the target probability distribution, 0<γ≦1, P(x;γ)≡P γ (x) / Z(γ), Z(γ)=∫dxP γ (x)S １－γ 2. The learning program according to claim 1, wherein a distribution (x) is prepared, the value of γ in the second process is made larger than the value of γ in the first process, and the value of γ is increased each time the second process is repeated.

7. The computer a first process of generating a probability distribution model by learning a probability distribution that has fewer peaks than a target probability distribution; a second process of generating a new probability distribution model by learning using parameters of the generated probability distribution model for a probability distribution that is closer to the target probability distribution than the learned probability distribution.

8. An information processing device comprising: a learning unit that executes a first process of generating a probability distribution model by learning a probability distribution having fewer peaks than a target probability distribution; and a second process of generating a new probability distribution model by learning a probability distribution that is closer to the target probability distribution than the learned probability distribution using parameters of the generated probability distribution model.

Images