Bayesian optimization device, Bayesian optimization method, and Bayesian optimization program

Abstract

A Bayesian optimization device comprising: a comparison result expansion unit that receives and expands the result from a comparison by a user; a comparison result storage unit that stores expanded comparison result; a model training unit that trains a model on the basis of the expanded comparison result; a model storage unit that stores the trained model; a candidate point selection unit that selects candidate points to be searched on the basis of the expanded comparison result and the trained model; and a candidate point presenting unit that presents candidate points to be searched to the user.

Description

[Technical field]

[0001] The present invention relates to a Bayesian optimization device, a Bayesian optimization method, and a Bayesian optimization program. [Background technology]

[0002] Personalization is often used in fields where it is easy to create mechanisms for collecting personal data. For example, products, content, services, and advertisements are personalized based on browsing / playback / click / purchase / rating logs.

[0003] On the other hand, personalization is not common in fields where it is difficult to create a system for collecting personal data. For example, personalization of how one feels about something, such as estimating emotions and favorability from facial expressions, gestures, and voices, or the smell, taste, and deliciousness of food and drink, is not common.

[0004] If personalization targeting such subjective data could be realized, it could be used, for example, to convert facial expressions to match Person A's emotion recognition characteristics in order to accurately convey emotions, or to prepare a dish that Person A finds most delicious.

[0005] For these reasons, a method is needed to efficiently collect subjective data in order to easily build a model related to the subjective data. [Prior art documents] [Non-patent literature]

[0006] [Non-Patent Document 1] B. Shahriari, K. Swersky, Z. Wang, R. Adams, and N. de Freitas : Taking the human out of the loop: A Review of Bayesian Optimization. Proc. of the IEEE,(1), 12 / 2015(2016) [Non-Patent Document 2] Google. “The makings of a smart cookie.” https: / / www.blog.google / technology / research / makings-smart-cookie / , 2017. [Non-Patent Document 3] Chu, Wei, et. al. "Preference learning with Gaussian processes."Proc. of the 22nd international conference on Machine learning, 2005. [Non-Patent Document 4] Niranjan Srinivas, et. al. “Information-theoretic regret bounds for gaussian process optimization in the bandit setting“, IEEE Transactions on Information Theory, vol. 58, no. 5, pp. 3250-3265, 2010. [Non-Patent Document 5] Jonas Mockus, et. al. “The application of Bayesian methods for seeking the extremum”, Towards Global Optimization, Vol. 2, pp. 117-129, 1978. [Non-Patent Document 6] Emile Contal, et. al. “Gaussian Process Optimization with Mutual Information”, International Conference on Machine Learning, pp. 253-261, 2014. Summary of the Invention [Problem to be solved by the invention]

[0007] Non-Patent Document 1 discloses Bayesian optimization as a method for building a model by collecting a small amount of data.

[0008] Bayesian optimization is a mechanism that uses a rating for a certain parameter as input and repeatedly learns to present parameters to be searched.

[0009] For example, when a user gives a score to a dish in a certain recipe, the next recipe to be rated is output, and the user then gives a score to that recipe, and this process is repeated until the recipe with the highest user rating is finally output.

[0010] Non-Patent Document 2 discloses an example of optimizing a cookie recipe using Bayesian optimization.

[0011] However, there is a problem with collecting subjective data, in that the user ratings tend to vary widely.

[0012] Non-Patent Document 3 proposes a method for solving the problem of inconsistent user ratings by inputting comparison results and using a Gaussian process for learning.

[0013] This method uses the results of comparing previously selected parameters with the currently presented parameters as input, and iteratively learns to present the parameters to be searched for. The previously selected parameters are the parameters with the highest evaluation in the past.

[0014] Here, the presented parameter is the parameter to be searched, and the previously selected parameter is the parameter to be compared with the presented parameter, i.e., the parameter to be searched, which will be referred to below as a candidate point to be searched, a new candidate point, or simply a candidate point, and the parameter to be compared with the parameter to be searched, which will be referred to below as a comparison point.

[0015] For example, when the user selects a dish from two recipes in turn, the next recipe to be compared is output, and the user then selects in turn from the recipe that was previously rated highly and the output recipe to be compared, and this process is repeated until the recipe with the highest user rating is finally output.

[0016] However, this method has problems in that it takes a long time to converge and the user has to make a large number of comparisons.

[0017] The present invention has been made in light of the above-mentioned circumstances, and an object of the present invention is to provide a Bayesian optimization device, a Bayesian optimization method, and a Bayesian optimization program in which learning converges quickly and the number of comparisons made by a user is reduced. [Means for solving the problem]

[0018] One aspect of the present invention is a Bayesian optimization device that includes a comparison result development unit that receives and develops a comparison result by a user, a comparison result storage unit that stores the developed comparison result, a model learning unit that learns a model based on the developed comparison result, a model storage unit that stores the learned model, a candidate point selection unit that selects candidate points to be searched for based on the developed comparison result and the learned model, and a candidate point presentation unit that presents the candidate points to be searched for to the user.

[0019] One aspect of the present invention is The computer runsA Bayesian optimization method includes receiving and expanding a comparison result by a user, storing the expanded comparison result, learning a model based on the expanded comparison result, storing the learned model, selecting candidate points to be searched for based on the expanded comparison result and the learned model, and presenting the candidate points to be searched for to the user.

[0020] A Bayesian optimization program according to one aspect of the present invention causes a computer to execute the functions of each of the components of the Bayesian optimization device described above. Effect of the Invention

[0021] According to the present invention, there are provided a Bayesian optimization device, a Bayesian optimization method, and a Bayesian optimization program that enable fast convergence of learning and reduce the number of comparisons performed by a user. [Brief description of the drawings]

[0022] [Figure 1] FIG. 1 is a block diagram showing an example of a functional configuration of a Bayesian optimization device according to an embodiment. [Diagram 2] FIG. 2 is a diagram showing an example of data of a comparison result received by a comparison result development unit in the Bayesian optimization device according to the embodiment. [Diagram 3] FIG. 3 is a diagram showing an example of data of the comparison result expanded by the comparison result expanding unit in the Bayesian optimization device according to the embodiment. [Figure 4] FIG. 4 is a schematic diagram showing how the comparison result is expanded by the comparison result expanding unit in the Bayesian optimization device according to the embodiment. [Diagram 5] FIG. 5 is a block diagram illustrating an example of a hardware configuration of a Bayesian optimization device according to an embodiment. [Figure 6] FIG. 6 is a flowchart illustrating the operation of the Bayesian optimization device according to the embodiment. [Figure 7] FIG. 7 is a block diagram showing an example of a functional configuration of a Bayesian optimization device according to a conventional example. [Figure 8] FIG. 8 is a diagram showing an example of data of a comparison result stored in a comparison result storage unit in a Bayesian optimization device according to a conventional example. [Figure 9] FIG. 9 is a schematic diagram showing a comparison in a Bayesian optimization device according to a conventional example. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0023] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. Fig. 1 is a block diagram showing an example of a functional configuration of a Bayesian optimization device 10 according to an embodiment.

[0024] As shown in FIG. 1, a Bayesian optimization device 10 has a comparison result development unit 11, a comparison result storage unit 12, a model learning unit 13, a model storage unit 14, a candidate point selection unit 15, and a candidate point presentation unit 16.

[0025] The comparison result development unit 11 receives the comparison result by the user, develops the received comparison result, and outputs the developed comparison result to the comparison result storage unit 12.

[0026] The comparison result storage unit 12 receives the expanded comparison results from the comparison result expansion unit 11 and stores them.

[0027] The model learning unit 13 reads the expanded comparison result from the comparison result storage unit 12, and also reads the model from the model storage unit 14. The model learning unit 13 learns the model based on the comparison result. The model learning unit 13 also outputs the learned model to the model storage unit 14.

[0028] In the initial state, the model storage unit 14 stores an unlearned model. Every time the model learning unit 13 executes model learning, the model storage unit 14 receives a learned model from the model learning unit 13 and stores it. In other words, the model storage unit 14 updates the stored model.

[0029] The candidate point selection unit 15 reads the comparison result from the comparison result storage unit 12 and reads the trained model from the model training unit 13. The candidate point selection unit 15 also selects candidate points to be searched for based on the comparison result and the trained model. The candidate point selection unit 15 also outputs the candidate points to be searched for to the candidate point presentation unit 16.

[0030] The candidate point presentation unit 16 receives the candidate points to be searched for from the candidate point selection unit 15 and presents them to the user.

[0031] Fig. 2 is a diagram showing an example of the data of the comparison result by the user input to the comparison result development unit 11. Fig. 3 is a diagram showing an example of the data of the comparison result developed and output by the comparison result development unit. In Fig. 2 and Fig. 3, the data of the comparison result is shown in a table format. In Fig. 2, for convenience, the data of the comparison result is drawn as a two-tiered table.

[0032] The comparison result data in Figures 2 and 3 is an example of cookie recipe data, and has multiple parameters. Each parameter has information on sugar, butter, ... The comparison result data also has information on the comparison result for each trial.

[0033] The comparison results by the user input to the comparison result development unit 11 have three or more parameters and their comparison results. The comparison result data in Fig. 2 is data when the number of parameters is three. Therefore, the comparison result data in Fig. 2 has parameter 1, parameter 2, and parameter 3. The comparison result data in Fig. 2 also has information on the first, second, and third place parameters for each number of trials as comparison results.

[0034] The comparison result data in Fig. 3 is data having two parameters, namely, parameter 1 and parameter 2. The comparison result data in Fig. 3 also has, as a comparison result, information on the parameter ranked first for each trial count.

[0035] The comparison result development unit 11 develops the comparison result data of Fig. 2 to create the comparison result data of Fig. 3. The comparison result development unit 11 first obtains three comparison results by mutually comparing parameter 1, parameter 2, and parameter 3 of the comparison result data of Fig. 2. The three comparison results are a first comparison result which is a comparison result of parameter 1 and parameter 2, a second comparison result which is a comparison result of parameter 2 and parameter 3, and a third comparison result which is a comparison result of parameter 3 and parameter 1. The comparison result development unit 11 then performs ranking based on these three comparison results to create the comparison result data of Fig. 3.

[0036] Next, we will explain the hardware configuration of the Bayesian optimization device 10. The Bayesian optimization device 10 is configured with a computer. For example, the Bayesian optimization device 10 is configured with a personal computer, a server computer, or the like.

[0037] Fig. 5 is a block diagram showing an example of a hardware configuration of a Bayesian optimization device 10 according to an embodiment. As shown in Fig. 5, the Bayesian optimization device 10 includes an input device 21, a CPU 22, a storage device 25, and an output device 28. In addition to these, the Bayesian optimization device 10 may further include other peripheral devices.

[0038] The input device 21, CPU 22, storage device 25, and output device 28 are electrically connected to one another via a bus 29, and exchange data and commands via the bus 29.

[0039] The input device 21 is a device that receives data and commands from a user. For example, the input device 21 is composed of a keyboard and a mouse. The input device 21 is not limited to these, and may be composed of any other input device.

[0040] The output device 28 is a device that presents data to a user. For example, the output device 28 is configured with a display. The output device 28 is not limited to this, and may be configured with any other output device.

[0041] The storage device 25 stores programs and data necessary for processing executed by the CPU 22. The CPU 22 reads out the necessary programs and data from the storage device 25 and executes them to perform various types of processing.

[0042] The storage device 25 includes a main storage device 26 and an auxiliary storage device 27. The main storage device 26 and the auxiliary storage device 27 exchange programs and data with each other.

[0043] The main storage device 26 stores programs and data temporarily required for processing by the CPU 22. For example, the main storage device 26 is configured with a volatile memory such as a RAM (Random Access Memory).

[0044] The auxiliary storage device 27 stores programs and data supplied via an external device or a network, and provides the programs and data temporarily required for processing by the CPU 22 to the main storage device 26. For example, the auxiliary storage device 27 is configured with a non-volatile memory such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive).

[0045] The CPU 22 is a processor, and is hardware that processes data and instructions. The CPU 22 includes a control device 23 and an arithmetic unit 24.

[0046] The control device 23 controls the input device 21, the arithmetic device 24, the storage device 25 and the output device .

[0047] The processor 24 reads programs and data from the main memory 26 , executes the programs to process the data, and provides the processed data to the main memory 26 .

[0048] In such a hardware configuration, the input device 21, the CPU 22, and the main memory device 26 constitute the comparison result development unit 11. In addition, the memory device 25 constitutes the comparison result storage unit 12 and the model storage unit 14. The CPU 22 and the main memory device 26 constitute the model learning unit 13 and the candidate point selection unit 15. The CPU 22, the main memory device 26, and the output device 28 constitute the candidate point presentation unit 16.

[0049] Next, the operation of the Bayesian optimization device 10 will be described with reference to Fig. 6. Fig. 6 is a flow chart showing the operation of the Bayesian optimization device 10.

[0050] In step S11, the comparison result development unit 11 waits for the input of the comparison result by the user. When the comparison result development unit 11 receives the comparison result by the user, the process proceeds to step S12. The comparison result by the user includes three or more parameters.

[0051] In step S12, the comparison result development unit 11 develops the comparison result. That is, the comparison result development unit 11 creates an expanded comparison result by comparing three or more parameters of the received comparison result with each other. For example, the comparison result development unit 11 receives a comparison result including three parameters as shown in FIG. 2, and creates an expanded comparison result including two parameters as shown in FIG. 3. The comparison result development unit 11 outputs the expanded comparison result to the comparison result storage unit 12.

[0052] In step S13, the comparison result storage section 12 receives the expanded comparison result from the comparison result expansion section 11 and stores it.

[0053] In step S14, the model learning unit 13 reads the expanded comparison results from the comparison result storage unit 12 and also reads the model from the model storage unit 14. The model learning unit 13 learns the model based on the expanded comparison results. The model learning unit 13 also outputs the learned model to the model storage unit 14 and the candidate point selection unit 15.

[0054] In step S15, the model storage unit 14 receives the trained model from the model training unit 13 and stores it.

[0055] In step S16, the candidate point selection unit 15 reads the expanded comparison results from the comparison result storage unit 12 and receives the trained model from the model training unit 13. The candidate point selection unit 15 also selects candidate points to be searched for based on the expanded comparison results and the trained model. The candidate point selection unit 15 outputs the candidate points to be searched for to the candidate point presentation unit 16.

[0056] In step S17, the candidate point presenting unit 16 receives the candidate points to be searched for from the candidate point selecting unit 15, and presents them to the user. After that, the Bayesian optimization device 10 returns to the process of step S11.

[0057] The Bayesian optimization device 10 according to the embodiment will be compared with a Bayesian optimization device 50 according to a conventional example. Fig. 7 is a block diagram showing an example of the functional configuration of the Bayesian optimization device 50 according to the conventional example. In Fig. 7, the functional configurations with the same reference symbols as those in Fig. 1 represent similar functional configurations.

[0058] As can be seen by comparing Fig. 7 with Fig. 1, the Bayesian optimization device 50 according to the conventional example has a configuration in which the comparison result development unit 11 is omitted from the Bayesian optimization device 10 according to the embodiment. Accordingly, in terms of hardware, the comparison result storage unit 12 according to the conventional example is configured to include an input device 21 and a CPU 22 in addition to a storage device 25.

[0059] Figure 8 is a diagram showing an example of comparison result data stored in comparison result storage unit 12 in Bayesian optimization device 50 of Figure 7. The comparison result data of Figure 8 is an example of cookie recipe data, similar to the comparison result data of Figures 2 and 3, and is represented in a table format.

[0060] The comparison result data in FIG. 8 has two parameters, namely, parameter 1 and parameter 2, and a comparison result. Each parameter has information on sugar, butter, .... The comparison result has information on the first-place parameter for each trial.

[0061] As can be seen by comparing FIG. 8 with FIG. 3, the data of the comparison result in FIG. 8 is similar to the data of the comparison result in FIG.

[0062] In a Bayesian optimization device 50 according to a conventional example, a comparison result by a user having two parameters is input to a comparison result storage unit 12. A model learning unit 13 learns a model based on the comparison result. A candidate point selection unit 15 selects candidate points to be searched for based on the comparison result and the learned model. A candidate point presentation unit 16 presents the candidate points to be searched for to the user. Thereafter, the comparison result storage unit 12 waits for the user to input the next comparison result. By repeating this process, Bayesian optimization is performed.

[0063] 9 is a schematic diagram showing a comparison in a Bayesian optimization device 50 according to a conventional example. The comparison result by the user input to the Bayesian optimization device 50 has two parameters, namely, parameter 1 and parameter 2. The comparison result obtained by comparing parameter 1 and parameter 2 is only one, comparison result C1.

[0064] In contrast, in the Bayesian optimization device 10 according to the embodiment, a comparison result by a user having three or more parameters is input to the comparison result development unit 11. The comparison result development unit 11 creates an expanded comparison result by performing parameter comparison between three or more parameters. The expanded comparison result is a comparison result having two parameters, as shown in FIG. 3. The expanded comparison result is stored in the comparison result storage unit 12. The model learning unit 13 learns a model based on the expanded comparison result. The candidate point selection unit 15 selects candidate points to be searched based on the comparison result and the learned model. The candidate point presentation unit 16 presents the candidate points to be searched to the user. After that, the comparison result development unit 11 waits for the input of the next comparison result by the user. By repeating this process, Bayesian optimization is performed.

[0065] FIG. 4 is a schematic diagram showing the state of comparison result expansion performed by the comparison result expansion unit 11 in the Bayesian optimization device 10 according to the embodiment. The comparison result by the user input to the comparison result expansion unit 11 has three or more parameters. FIG. 4 shows a case where the number of parameters is four. The comparison result expansion unit 11 compares parameters 1, 2, 3, and 4 with each other. As a result, the comparison result expansion unit 11: 4 C 2 =6 comparison results are obtained: C1, C2, C3, C4, C5, C6.

[0066] In this way, in the Bayesian optimization device 10 according to the embodiment, model learning is performed based on six comparison results obtained by comparing four parameters with each other, so model learning is expected to ideally converge six times faster than the Bayesian optimization device 50 according to the conventional example that performs model learning by comparing two parameters. In other words, it is expected that the number of comparisons made by the user will be reduced to 1 / 6.

[0067] In the following, a more generalized case will be discussed in which the comparison result by the user received by the comparison result development unit 11 includes N parameters, where N is a natural number equal to or greater than 3.

[0068] The comparison result development unit 11 compares the N parameters with each other to obtain N C 2 Obtain comparison results, N C 2 Based on the individual comparison results, ranking is performed to generate an expanded comparison result. Based on this expanded comparison result, the model learning unit 13 learns a model. This allows learning to converge quickly, and reduces the number of comparisons required by the user.

[0069] The comparison result development unit 11 selects M new candidate points and NM comparison points for N parameters of the comparison result by the user, where M is a natural number equal to or greater than 1 and less than N.

[0070] An example of how to select NM comparison points is as follows. (1) Select the NM best rated comparison points from past comparisons. (2) Randomly select NM comparison points from those previously compared. (3) Among the previously compared points, NM comparison points that are the furthest from the new candidate point in terms of distance (for example, squared distance) are selected.

[0071] Here, we will explain how to set the value of N. The larger N is, the more comparison results can be obtained. As a result, learning converges faster and the number of comparisons made by the user decreases. However, the burden on the user increases. Taking these factors into consideration, it is desirable for N to be 10 or less, for example.

[0072] Next, we will explain how to set the value of M. If M is made smaller, the number of new candidate points added at one time will be smaller. As a result, more comparison results between new candidate points and comparison points will be obtained, improving inference performance. However, the calculation cost will increase.

[0073] Therefore, the values ​​of N and M can be determined based on a trade-off between performance and cost.

[0074] The search for new candidate points in the comparison result development unit 11 and the candidate point selection unit 15 is performed using the Upper Confidence Bound algorithm disclosed in Non-Patent Document 4, the Expected Improvement algorithm disclosed in Non-Patent Document 5, the Mutual Information algorithm disclosed in Non-Patent Document 6, etc.

[0075] Next, a description will be given of the model and learning in the model learning unit 13. A Gaussian process is used as the model, and learning is performed using the following likelihood function.

[0076]

number

[0077] As described above, the Bayesian optimization device 10 according to the embodiment compares N parameters with each other. N C 2 The system obtains comparison results and trains the model based on the expanded comparison results, which allows the learning to converge quickly and reduces the number of comparisons required by the user.

[0078] Therefore, according to the embodiments, a Bayesian optimization device, a Bayesian optimization method, and a Bayesian optimization program are provided that enable learning to converge quickly and reduce the number of comparisons made by the user.

[0079] The present invention is not limited to the above-mentioned embodiment, and can be modified in various ways without departing from the gist of the present invention. The embodiments may be combined as appropriate, and in that case, the combined effect can be obtained. Furthermore, the above-mentioned embodiment includes various inventions, and various inventions can be extracted by combinations selected from the multiple components disclosed. For example, if the problem can be solved and the effect can be obtained even if some components are deleted from all the components shown in the embodiment, the configuration from which the components are deleted can be extracted as an invention. [Explanation of symbols]

[0080] 10...Bayesian optimization device according to an embodiment 11…Comparison result development section 12...Comparison result storage section 13…Model Learning Section 14…Model storage area 15...Candidate point selection section 16...Candidate point presentation section 21...Input device 22…CPU 23...Control device 24...Arithmetic device 25…Storage device 26…Main memory 27…Auxiliary storage device 28...Output device 29…Bus 50...Bayesian optimization device according to the prior art

Claims

1. a comparison result development unit that receives and develops the comparison result by the user; a comparison result storage unit for storing the expanded comparison result; a model learning unit that learns a model based on the deployed comparison results; A model storage unit for storing a trained model; a candidate point selection unit that selects candidate points to be searched for based on the deployed comparison results and a trained model; A candidate point presenting unit presents candidate points to be searched for to a user. Bayesian optimizer.

2. the comparison result development unit compares three or more parameters included in the comparison result by the user with each other to obtain three or more comparison results, and creates the developed comparison result based on the obtained three or more comparison results. The Bayesian optimization apparatus of claim 1 .

3. the comparison result development unit compares N parameters (N is a natural number equal to or greater than 3) included in the comparison result by the user with each other to obtain N comparison results, and creates the developed comparison result based on the obtained N comparison results. The Bayesian optimization apparatus of claim 2 .

4. The comparison result development unit selects M new candidate points (M is a natural number greater than or equal to 1 and less than N) and N-M comparison points for the N parameters. The Bayesian optimization apparatus of claim 3.

5. the comparison result development unit selects, as the N-M comparison points, any one of the N-M comparison points having the highest evaluation among the past comparisons, the N-M comparison points randomly selected among the past comparisons, and the N-M comparison points having the greatest distance from the new candidate point among the past comparisons.

5. The Bayesian optimization apparatus of claim 4.

6. The model learning unit uses a Gaussian process as a model, and learning is performed using the following likelihood function: [0010] A Bayesian optimization device according to any one of claims 1 to 5.

7. receiving and expanding the comparison results by the user; storing the expanded comparison results; Learning the model based on the deployed comparison results, Storing the trained model; Selecting candidate points to be searched based on the deployed comparison results and the trained model; and and presenting to the user candidate points to be searched. A computer implemented Bayesian optimization method.

8. A Bayesian optimization program that causes a computer to execute the functions of each component of the Bayesian optimization device according to any one of claims 1 to 3.

Images