Optimization device, optimization method, and program

Abstract

In order to address the problem for adjusting, to a more suitable value, an inverse temperature used in a non-linear optimization problem, this optimization device (100) comprises: an optimal variable candidate generation unit (101) which generates a plurality of optimal variable candidates on the basis of a fiducial distribution; an object function evaluation unit (102) which evaluates an object function with respect to each of the plurality of optimal variable candidates; an inverse temperature optimization unit (103) which uses an optimization technique and calculates an inverse temperature at which an input target valid sample has the same size as a valid sample of weights for the object function; a weight evaluation unit (104) which computes the weights for the object function on the basis of the inverse temperature; and a fiducial distribution update unit (105) which updates the fiducial distribution on the basis of the optimal variable candidates, the weights, and the fiducial distribution.

Description

[Technical Field]

[0001] The present invention relates to techniques for solving nonlinear optimization problems. [Background technology]

[0002] In nonlinear optimization problems, there are known techniques for approximately calculating optimal variables without using the gradient value of the objective function. For example, Non-Patent Documents 1 and 2 describe an example of such a technique, Information Theoretic Model Predictive Control (ITMPC). ITMPC (i) calculates a weight for Bayesian updating for each of multiple optimal variable candidates generated based on a belief distribution by referring to the objective function value and the inverse temperature, and (ii) updates the belief distribution by referring to the multiple optimal variable candidates and their respective weights. ITMPC also outputs an approximate solution by referring to the belief distribution updated by repeating the processes (i) and (ii). [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Grady Williams, et al., “Information Theoretic MPC for Model-Based Reinforcement Learning”, ICRA2017 [Non-patent document 2] Grady Williams, et al., “Information Theoretic Model Predictive Control: Theory and Applications to Autonomous Driving,” IEEE Transactions on Robotics (Volume: 34, Issue: 6, December 2018) Summary of the Invention [Problem to be solved by the invention]

[0004] Here, the inverse temperature is a parameter that determines the efficiency and accuracy of the optimization system. Furthermore, in Bayesian updating, the appropriate value of the inverse temperature may vary depending on the circumstances at any given time, such as the results of generating optimal variable candidates, the contents of the objective function, or the shape of the belief distribution. If the inverse temperature is inappropriate, the effective sample size becomes inappropriate, causing problems with Bayesian updating. The techniques described in Non-Patent Documents 1 and 2 have the problem that it is difficult to adjust the inverse temperature because the appropriate value of the inverse temperature is unknown.

[0005] One aspect of the present invention has been made in view of the above-mentioned problems, and one object of the present invention is to provide a technique for adjusting the inverse temperature used in a nonlinear optimization problem to a more appropriate value. [Means for solving the problem]

[0006] An optimization device according to one aspect of the present invention comprises an optimal variable candidate generation means for generating a plurality of optimal variable candidates based on a belief distribution; an objective function evaluation means for evaluating an objective function for each of the plurality of optimal variable candidates; an inverse temperature optimization means for calculating, using an optimization technique, an inverse temperature such that an input target effective sample size and an effective sample size of a weight for the objective function are approximately the same; a weight evaluation means for calculating a weight for the objective function based on the inverse temperature; and a belief distribution update means for updating the belief distribution based on the optimal variable candidates, the weight, and the belief distribution.

[0007] An optimization method according to one aspect of the present invention includes generating a plurality of optimal variable candidates based on a belief distribution, evaluating an objective function for each of the plurality of optimal variable candidates, calculating an inverse temperature using an optimization technique such that an input target effective sample size and an effective sample size of a weight for the objective function are approximately the same, calculating a weight for the objective function based on the inverse temperature, and updating the belief distribution based on the optimal variable candidates, the weight, and the belief distribution.

[0008] A program according to one aspect of the present invention is a program for causing a computer to function as an optimization device, and causes the computer to function as: optimal variable candidate generation means for generating a plurality of optimal variable candidates based on a belief distribution; objective function evaluation means for evaluating an objective function for each of the plurality of optimal variable candidates; inverse temperature optimization means for calculating, using an optimization technique, an inverse temperature such that an input target effective sample size and an effective sample size of a weight for the objective function are approximately the same; weight evaluation means for calculating a weight for the objective function based on the inverse temperature; and belief distribution update means for updating the belief distribution based on the optimal variable candidates, the weight, and the belief distribution. [Effects of the Invention]

[0009] According to one aspect of the present invention, the inverse temperature used in a nonlinear optimization problem can be adjusted to a more appropriate value. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a block diagram showing the configuration of an optimization device according to a first exemplary embodiment of the present invention. [Figure 2] FIG. 2 is a flow chart showing the flow of an optimization method according to the first exemplary embodiment of the present invention. [Figure 3] FIG. 10 is a block diagram showing the configuration of an optimization system according to a second exemplary embodiment of the present invention. [Figure 4] FIG. 10 is a flow chart showing the flow of an optimization method according to a second exemplary embodiment of the present invention. [Figure 5] FIG. 10 is a block diagram showing the configuration of an optimization system according to a third exemplary embodiment of the present invention. [Figure 6] FIG. 10 is a flow chart showing the flow of an optimization method according to a third exemplary embodiment of the present invention. [Figure 7] FIG. 1 is a block diagram showing a configuration of an optimization system according to an application example of the present invention. [Figure 8] 10A and 10B are schematic diagrams showing an example of a movable range of an operating lever in an application example of the present invention. [Figure 9]FIG. 10 is a flowchart showing the flow of an optimization method according to an application example of the present invention. [Figure 10] 10A and 10B are schematic diagrams for explaining a reference trajectory in an application example of the present invention. [Figure 11] 1 is a graph for comparing the related art with an application example of the present invention. [Figure 12] 10 is another graph for comparing the related art with the application of the present invention; [Figure 13] 1 is a table for comparing related art with application examples of the present invention. [Figure 14] 10 is another table for comparing the related art with the application of the present invention. [Figure 15] FIG. 1 is a block diagram showing a configuration of an optimization system according to a related art. [Figure 16] FIG. 16 is a flowchart showing the flow of processing by the optimization system shown in FIG. [Figure 17] 1A and 1B are diagrams illustrating examples of hardware configurations of devices in exemplary embodiments and application examples. DETAILED DESCRIPTION OF THE INVENTION

[0011] [Findings underlying the present invention] The present inventors have discovered that in Bayesian updating for solving nonlinear optimization problems, an appropriate value for the inverse temperature can be obtained by focusing on the correlation between the inverse temperature and the effective sample size. This discovery will be described in detail below.

[0012] First, ITMPC, a related technology described in Non-Patent Documents 1 and 2, will be described with reference to Figs. 15 and 16. Fig. 15 is a block diagram showing the configuration of an optimization system 9 that solves non-linear optimization problems using ITMPC. Fig. 16 is a flow diagram showing the flow of processing by the optimization system 9.

[0013] 15 , the optimization system 9 includes an optimization device 90, an input device 93, and an output device 94. The optimization device 90 includes a control unit 91 and a memory unit 92. The control unit 91 includes an optimal variable candidate generation unit 911, an objective function evaluation unit 912, a weight evaluation unit 914, and a belief distribution update unit 915. The memory unit 92 includes an optimal variable candidate memory unit 921, an objective function value memory unit 922, an inverse temperature memory unit 924, a weight memory unit 925, and a belief distribution memory unit 926.

[0014] The optimization system 9 operates, for example, as shown in Fig. 16. In step S91, the control unit 91 acquires an arbitrary inverse temperature input by the user via the input device 93 and records it in the inverse temperature memory unit 924. The control unit 91 also acquires an arbitrary belief distribution input by the user via the input device 93 and records it in the belief distribution memory unit 926.

[0015] Next, the control unit 91 repeatedly executes steps S92 to S96. In step S92, the optimal variable candidate generation unit 911 generates a plurality of optimal variable candidates based on the belief distribution recorded in the belief distribution storage unit 926, and records them in the optimal variable candidate storage unit 921. The belief distribution used for generation in the first iteration of the iterative process is the input initial belief distribution input in step S91. In addition, the belief distribution used for generation in the second and subsequent iterations of the iterative process is the belief distribution updated in step S95, which will be described later.

[0016] In step S93, the objective function evaluation unit 912 evaluates the objective function for each optimal variable candidate stored in the optimal variable candidate storage unit 921, and stores each evaluation value in the objective function value storage unit 922. Hereinafter, the evaluation value obtained by evaluating the objective function will also be referred to as the objective function value. The objective function value may also be simply referred to as the objective function.

[0017] In step S94, the weight evaluation unit 914 refers to each objective function value recorded in the objective function value memory unit 922 and the inverse temperature recorded in the inverse temperature memory unit 924, evaluates the weight in the Bayesian update for each optimal variable candidate, i.e., the amount obtained by dividing the likelihood by the marginal likelihood, and records it in the weight memory unit 925.

[0018] In step S95, the belief distribution update unit 915 refers to each weight recorded in the weight memory unit 925, each optimal variable candidate recorded in the optimal variable candidate memory unit 921, and the belief distribution recorded in the belief distribution memory unit 926, calculates an approximation of the posterior belief distribution as a new belief distribution, and records it in the belief distribution memory unit 926.

[0019] In step S96, the control unit 91 determines whether a predetermined termination condition is satisfied. The predetermined termination condition may be specified by the user. If the determination is true in this step, in step S97, the control unit 91 outputs the belief distribution recorded in the belief distribution storage unit 926 to the output device 94. The control unit 91 also adopts the optimal variable candidate that is the most frequent value of the belief distribution as an approximate solution to the target optimization problem, i.e., an approximate optimal variable, and outputs the approximate solution.

[0020] If it is determined to be false in step S96, the control unit 91 refers to the belief distribution recorded in the belief distribution storage unit 926 and repeats the process from step S92.

[0021] Here, the likelihood function L in ITMPC is defined by the following equation (A1).

[0022]

number

[0023] The inverse temperature λ can also be interpreted as a hyperparameter that adjusts the amount of fluctuation in the belief distribution per Bayesian update. Equation (A1) suggests that the smaller the inverse temperature λ, the greater the amount of fluctuation, and the larger λ, the smaller the amount of fluctuation. However, in practical applications, it is necessary to approximate the posterior belief distribution, and the smaller λ, the worse the approximation accuracy becomes, making it less accurate as an optimization method. In this way, the inverse temperature λ is a parameter that determines the efficiency and accuracy of the optimization system9, and its adjustment is important in applications.

[0024] The problem with this type of ITMPC is that although adjusting the inverse temperature is important to obtain high-quality approximate optimal variables, it is difficult to do so because the appropriate value of the inverse temperature varies depending on the circumstances at each time, such as the results of generating optimal variable candidates, the contents of the objective function, and the shape of the belief distribution in Bayesian updating.

[0025] If the inverse temperature is inappropriate, the effective sample size in importance sampling is likely to be inappropriate. For example, if the inverse temperature λ is too small, there will be many samples with a likelihood L(v) of zero, reducing the effective sample size. As a result, the error in Bayesian updating using sample approximation will increase. Also, if the inverse temperature λ is too large, there will be many samples with a likelihood L(v) of 1, and no difference will occur between the samples, preventing Bayesian updating from proceeding.

[0026] Therefore, the inventors of the present invention have found that in order to solve a nonlinear optimization problem with high accuracy, it is necessary to estimate the inverse temperature λ at which the number of effective sample sizes becomes a target value and use the estimated inverse temperature. Hereinafter, an embodiment of the present invention based on this finding will be described.

[0027] Exemplary Embodiment 1 A first exemplary embodiment of the present invention will be described in detail with reference to the drawings. This exemplary embodiment is a basic form of the exemplary embodiments described below.

[0028] <Configuration of optimization device 100> The configuration of an optimization device 100 according to this exemplary embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the configuration of the optimization device 100.

[0029] As shown in FIG. 1, the optimization device 100 includes an optimal variable candidate generation unit 101, an objective function evaluation unit 102, an inverse temperature optimization unit 103, a weight evaluation unit 104, and a belief distribution update unit 105. The optimal variable candidate generation unit 101 may realize, but is not limited to, the optimal variable candidate generation means described in the claims. The objective function evaluation unit 102 may realize, but is not limited to, the objective function evaluation means described in the claims. The inverse temperature optimization unit 103 may realize, but is not limited to, the inverse temperature optimization means described in the claims. The weight evaluation unit 104 may realize, but is not limited to, the weight evaluation means described in the claims. The belief distribution update unit 105 may realize, but is not limited to, the belief distribution update means described in the claims.

[0030] The optimal variable candidate generation unit 101 generates multiple optimal variable candidates based on a belief distribution. The objective function evaluation unit 102 evaluates the objective function for each of the multiple optimal variable candidates. The inverse temperature optimization unit 103 uses an optimization technique to calculate an inverse temperature such that the input target effective sample size and the effective sample size of the weight for the objective function are approximately the same. The weight evaluation unit 104 calculates the weight for the objective function based on the inverse temperature. The belief distribution update unit 105 updates the belief distribution based on the optimal variable candidates, the weight, and the belief distribution.

[0031] <Flow of optimization method M100> The optimization device 100 configured as above executes an optimization method M100 according to this exemplary embodiment. The flow of the optimization method M100 will be described with reference to Fig. 2. Fig. 2 is a flow diagram showing the flow of the optimization method M100. As shown in Fig. 2, the optimization method M100 includes steps S1001 to S1005.

[0032] In step S1001, the optimal variable candidate generation unit 101 generates multiple optimal variable candidates based on the belief distribution. In step S1002, the objective function evaluation unit 102 evaluates the objective function for each of the multiple optimal variable candidates. In step S1003, the inverse temperature optimization unit 103 uses an optimization method to calculate an inverse temperature such that the input target effective sample size and the effective sample size of the weight for the objective function are approximately the same. In step S1004, the weight evaluation unit 104 calculates the weight for the objective function based on the inverse temperature. In step S1005, the belief distribution update unit 105 updates the belief distribution based on the optimal variable candidates, the weight, and the belief distribution.

[0033] <Example of program implementation> When the optimization device 100 is configured by a computer, the following program is stored in a memory referenced by the computer. The program causes the computer to function as the optimization device 100, and causes the computer to function as an optimal variable candidate generation unit 101 that generates multiple optimal variable candidates based on a belief distribution, an objective function evaluation unit 102 that evaluates an objective function for each of the multiple optimal variable candidates, an inverse temperature optimization unit 103 that uses an optimization method to calculate an inverse temperature such that an input target effective sample size and an effective sample size of a weight for the objective function are approximately the same, a weight evaluation unit 104 that calculates a weight for the objective function based on the inverse temperature, and a belief distribution update unit 105 that updates the belief distribution based on the optimal variable candidates, the weight, and the belief distribution.

[0034] The computer reads the program from memory and executes it to implement the optimization method M100 described above.

[0035] <Advantages of this exemplary embodiment> As described above, according to this exemplary embodiment, a configuration is adopted in which multiple optimal variable candidates are generated based on a belief distribution, an objective function is evaluated for each of the multiple optimal variable candidates, an inverse temperature is calculated using an optimization method so that the input target effective sample size and the effective sample size of the weight for the objective function are approximately the same, weights for the objective function are calculated based on the inverse temperature, and the belief distribution is updated based on the optimal variable candidates, weights, and belief distribution.

[0036] This provides the effect of making it possible to adjust the inverse temperature used in the nonlinear optimization problem to a more appropriate value.

[0037] Exemplary Embodiment 2 A second exemplary embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in the first exemplary embodiment are given the same reference numerals, and their description will be omitted as appropriate.

[0038] <Configuration of Optimization System 1> The configuration of an optimization system 1 according to this exemplary embodiment will be described with reference to Fig. 3. Fig. 3 is a block diagram showing the configuration of the optimization system 1. As shown in Fig. 3, the optimization system 1 includes an optimization device 10, an input device 13, and an output device 14. The optimization device 10 includes a control unit 11 and a storage unit 12.

[0039] The control unit 11 controls each unit of the optimization device 10. The control unit 11 also includes an optimal variable candidate generation unit 111, an objective function evaluation unit 112, an inverse temperature optimization unit 113, a weight evaluation unit 114, and a belief distribution update unit 115. The control unit 11 also controls the memory unit 12, and refers to data in the memory unit 12 and records data in the memory unit 12.

[0040] The memory unit 12 includes an optimal variable candidate memory unit 121 , an objective function value memory unit 122 , a target effective sample size memory unit 123 , an inverse temperature memory unit 124 , a weight memory unit 125 , and a belief distribution memory unit 126 .

[0041] The input device 13 accepts an input operation by a user. The input device 13 may be, for example, but is not limited to, a keyboard, a mouse, or a touchpad. For example, the input device 13 accepts an operation to input information indicating a target effective sample size and an initial belief distribution.

[0042] The output device 14 outputs information under the control of the control unit 11. The output device 14 may be, for example, a liquid crystal display or a speaker, but is not limited to these. For example, the output device 14 outputs information indicating the belief distribution finally calculated by the control unit 11.

[0043] The optimal variable candidate generation unit 111 generates multiple optimal variable candidates based on the belief distribution in the belief distribution storage unit 126 and stores them in the optimal variable candidate storage unit 121. The optimal variable candidate generation unit 111 generates multiple optimal variable candidates based on the initial belief distribution input from the input device 13 or the belief distribution updated by the belief distribution update unit 115. For example, the initial belief distribution is referenced to generate multiple optimal variable candidates the first time a loop process described below is executed. The updated belief distribution is referenced to generate multiple optimal variable candidates the second time and thereafter the loop process is executed.

[0044] The objective function evaluation unit 112 evaluates the objective function for each optimal variable candidate in the optimal variable candidate storage unit 121 and records the result in the objective function value storage unit 122 .

[0045] The inverse temperature optimization unit 113 uses an optimization method to calculate an inverse temperature that makes the target effective sample size input from the input device 13 in the target effective sample size memory unit 123 and the effective sample size of the weight approximately the same, and records the calculated inverse temperature in the inverse temperature memory unit 124. Here, "the target effective sample size and the effective sample size of the weight approximately the same" may mean, for example, that they are equal. Also, "approximately the same" may mean, for example, that the difference between them is within a predetermined range. However, "approximately the same" is not limited to these.

[0046] The weight evaluation unit 114 evaluates the weight for each objective function value in the objective function value storage unit 122 based on the inverse temperature in the inverse temperature storage unit 124 , and records the weight in the weight storage unit 125 .

[0047] The belief distribution update unit 115 approximates the posterior belief distribution based on each optimal variable candidate in the optimal variable candidate memory unit 121, each weight in the weight memory unit 125, and the belief distribution in the belief distribution memory unit 126, and records this as a new belief distribution in the belief distribution memory unit 126.

[0048] <Flow of optimization method M10> The optimization system 1 configured as above executes an optimization method M10 according to this exemplary embodiment. The flow of the optimization method M10 will be described with reference to Fig. 4. Fig. 4 is a flow diagram showing the flow of the optimization method M10. As shown in Fig. 4, the optimization method M10 includes steps S1 to S8.

[0049] In step S1, the control unit 11 receives input of a target effective sample size and an initial belief distribution via the input device 13. The control unit 11 then records the acquired target effective sample size in the target effective sample size storage unit 123. The control unit 11 also records the acquired initial belief distribution in the belief distribution storage unit 126.

[0050] Thereafter, the control unit 11 repeats steps S2 to S8. The processing of steps S2 to S8 is also referred to as loop processing.

[0051] In step S2, the optimal variable candidate generation unit 111 generates a plurality of optimal variable candidates based on the belief distribution. Note that in the first loop processing, the belief distribution used for generation is the initial belief distribution in the belief distribution storage unit 126. In the second and subsequent loop processing, the belief distribution used for generation is the belief distribution updated in step S7 of the previous loop processing. Note that the updated belief distribution is recorded in the belief distribution storage unit 126. In addition, the optimal variable candidate generation unit 111 records the generated plurality of optimal variable candidates in the optimal variable candidate storage unit 121.

[0052] In step S3, the objective function evaluation unit 112 evaluates the objective function for each optimal variable candidate in the optimal variable candidate storage unit 121, and records the objective function value, which is the evaluation result, in the objective function value storage unit 122.

[0053] In step S4, the inverse temperature optimization unit 113 calculates an inverse temperature using an optimization method that makes the target effective sample size in the target effective sample size memory unit 123 equal to the effective sample size of the weight, and records it in the inverse temperature memory unit 124.

[0054] In step S 5 , the weight evaluation unit 114 evaluates the weight for each objective function value in the objective function value storage unit 122 based on the inverse temperature in the inverse temperature storage unit 124 , and records the weight in the weight storage unit 125 .

[0055] In step S6, belief distribution update unit 115 approximates a posterior belief distribution based on each optimal variable candidate in optimal variable candidate storage unit 121, each weight in weight storage unit 125, and the belief distribution in belief distribution storage unit 126, and records this as a new belief distribution in belief distribution storage unit 126. Note that in the first loop processing, the belief distribution from which the posterior belief distribution is approximated is the initial belief distribution in belief distribution storage unit 126. In the second and subsequent loop processing, the belief distribution from which the posterior belief distribution is approximated is the belief distribution updated in the corresponding step of the previous loop processing.

[0056] In step S7, the control unit 11 determines whether a predetermined termination condition is satisfied. The predetermined termination condition may be a termination condition specified by the user.

[0057] If the determination in step S7 is true, then in step S8, control unit 11 outputs the belief distribution to output device 14, and optimization method M10 ends.

[0058] If it is determined to be false in step S7, control unit 11 repeats the loop process of steps S2 to S8 based on the updated belief distribution.

[0059] <Advantages of this exemplary embodiment> According to this exemplary embodiment, the inverse temperature optimization unit 113 is configured to calculate the inverse temperature so that the target effective sample size and the effective sample size of the weight are approximately the same.

[0060] This allows the effective sample size to be fixed, making it possible to adjust the magnitude of the sampling error that occurs in the approximate calculation of the posterior belief distribution, and thus enabling stable updates. Furthermore, by setting the target effective sample size as small as possible within the range in which the sampling error is tolerable, it is possible to ensure a good balance between the stability and efficiency of updates. Alternatively, it is possible to prioritize the stability of updates by setting a larger target effective sample size. Overall, the difficulty of adjusting the inverse temperature can be improved by automatically adjusting the inverse temperature so as to fix the effective sample size.

[0061] Furthermore, according to this exemplary embodiment, the optimal variable candidate generation unit 111 is configured to generate multiple optimal variable candidates based on the initial belief distribution input from the input device 13 or the belief distribution updated by the belief distribution update unit 115.

[0062] Therefore, according to this exemplary embodiment, in addition to the same effect as exemplary embodiment 1, the inverse temperature used to calculate the weights used in the update can be adjusted to an appropriate value each time the belief distribution is updated.

[0063] Exemplary Embodiment 3 A third exemplary embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in the first and second exemplary embodiments are denoted by the same reference numerals, and their description will not be repeated.

[0064] <Configuration of Optimization System 2> The configuration of an optimization system 2 according to this exemplary embodiment will be described with reference to Fig. 5. Fig. 5 is a block diagram showing the configuration of the optimization system 2. As shown in Fig. 5, the optimization system 2 includes an optimization device 20, an input device 13, a control target 25, and a state observer 26. The optimization device 20 includes a control unit 21 and a storage unit 12. The storage unit 12 is as described in the exemplary embodiment 2. The control unit 21 is configured in almost the same manner as the control unit 11 in the exemplary embodiment 2, except that it includes an objective function evaluation unit 212 instead of the objective function evaluation unit 112, and that it newly includes a control input conversion unit 216 and a belief distribution processing unit 217.

[0065] The objective function evaluation unit 212 is basically configured in the same way as the objective function evaluation unit 112, but evaluates an objective function that depends on the state of the control object 25 observed by the state observation device 26 for each of a plurality of optimal variable candidates. For example, the objective function evaluation unit 212 may evaluate the objective function for each of a plurality of optimal variable candidates using information on the state of the control object 25 transmitted from the observation device 24. Note that when a plurality of states are observed for the control object 25, the objective function evaluation unit 212 may evaluate the objective function using a state according to the user's intended use. The user's intended use may be specified by input by the user.

[0066] The control input conversion unit 216 calculates a control input according to a predetermined conversion rule based on the belief distribution recorded in the belief distribution storage unit 126 by the belief distribution update unit 115, and transmits the calculated control input to the control object 25. Here, the conversion rule may vary depending on the user's intended use. For example, the predetermined conversion rule may be a conversion rule specified by a user input. Furthermore, the control input is information input to control the control object 25. For example, the control input may be an optimal variable candidate that obtains the most frequent value of the belief distribution.

[0067] The control object 25 receives a control input from the control input conversion unit 216 and operates in accordance with the control input. The control object 25 refers to any controllable device or system, such as, but not limited to, a robot, an automobile, a backhoe, a ship, a chemical plant system, a power plant system, a trading system, etc. The control object 25 may have a function to autonomously control its own device in accordance with the received control input. The control object 25 may also have a function to control its own device in accordance with the operation of an operator. In this case, the operator may control the control object 25 in accordance with the control input received by the control object 25.

[0068] The state observer 26 observes the state of the controlled object 25 and transmits the observed state to the objective function evaluator 212 .

[0069] The belief distribution processor 217 processes the belief distribution recorded in the belief distribution memory 126 by the belief distribution updater 115 for the next series of processes, i.e., the loop processes by the optimal variable candidate generator 111, the objective function evaluator 112, the inverse temperature optimizer 113, the weight evaluator 114, and the belief distribution updater 115, and stores the processed belief distribution in the belief distribution memory 126. Such processing is performed, for example, when it is necessary to modify the definition of an optimal variable and a belief distribution corresponding to the modified optimal variable must be prepared. Note that the belief distribution processor 217 may process the belief distribution in accordance with the user's intended use. The user's intended use may be specified by input from the user.

[0070] <Flow of optimization method M20> The optimization system 2 configured as above executes an optimization method M20 according to this exemplary embodiment. The flow of the optimization method M20 will be described with reference to Fig. 6. Fig. 6 is a flow chart showing the flow of the optimization method M20.

[0071] The optimization method M20 differs from the optimization method M10 according to the second exemplary embodiment in the following respects.

[0072] The first point is that steps S100 to S101 are executed after step S1 is executed and before step S2 is executed. The second point is that if step S7 is determined to be true, steps S108 to S110 are executed instead of step S8. Below, we will explain these steps that are different from optimization method M10, and will not repeat the explanation of the same steps.

[0073] In step S100 , the state observer 26 observes the state of the controlled object 25 and transmits the observed state to the objective function evaluator 212 .

[0074] In step S101, the control unit 21 determines whether or not a control end condition is satisfied. The control end condition may be specified by a user input.

[0075] If the result of step S101 is false, the optimization system 2 executes the loop process from step S2. If the result of step S101 is true, the optimization system 2 ends the optimization method M20.

[0076] If a plurality of states are observed in step S100, the state according to the user's intended use is used when the objective function evaluation unit 212 evaluates the objective function in step S3.

[0077] In step S108, the control input conversion unit 216 converts the belief distribution into a control input.

[0078] In step S109, the control input conversion unit 216 transmits the converted control input to the control target 25.

[0079] In step S110, belief distribution processing unit 217 processes the belief distribution updated by belief distribution update unit 115 in accordance with the user's intended use, and records the processed belief distribution in belief distribution storage unit 126. For example, the user's intended use is specified by input from the user.

[0080] <Advantages of this exemplary embodiment> In addition to the same configuration as in the first and second exemplary embodiments, this exemplary embodiment employs a configuration in which an objective function dependent on the state of the controlled object 25 observed by the state observer 26 is evaluated for each of a plurality of optimal variable candidates. Also, this exemplary embodiment employs a configuration in which a control input is calculated in accordance with a predetermined conversion rule based on the updated belief distribution, and the calculated control input is sent to the controlled object 25. Also, this exemplary embodiment employs a configuration in which the belief distribution updated in a certain step is processed for loop processing in the next step by the optimal variable candidate generator 111, the objective function evaluator 212, the inverse temperature optimizer 113, the weight evaluator 114, and the belief distribution updater 115.

[0081] In other words, according to this exemplary embodiment, the control input conversion unit 216 calculates a control input based on the belief distribution updated by the belief distribution update unit 115 in accordance with a conversion rule specified by the user, and transmits the calculated control input to the control object 25, so that the control object 25 operates in accordance with the control input. In addition, the state observer 26 observes the state of the control object 25, transmits the observed state to the objective function evaluator 212, and the belief distribution processor 217 processes the updated belief distribution according to the user's intended use for the next series of optimization processes.

[0082] Therefore, according to this exemplary embodiment, the user of the optimization system 2 can advantageously perform optimal control with automatic adjustment of inverse temperature, model predictive control, online optimization, and the like.

[0083] In particular, in these applications, it becomes more difficult to manually set an appropriate inverse temperature because changes in the objective function and objective variables generally occur as the state of the controlled object 25 changes. In contrast, in this exemplary embodiment, even in the face of such changes, the inverse temperature is automatically adjusted so that the effective sample size of the weights is always constant, thereby making it possible to maintain constant the efficiency and stability of the Bayesian updating.

[0084] [Application example of exemplary embodiment 3] The following describes an optimization system 2A, which is an application example of exemplary embodiment 3. The optimization system 2A is an example in which a hydraulic backhoe MV is applied as the control object 25 in the optimization system 2A described above. For example, the optimization system 2A can be used to automate the soil breaking-in operation using the bucket B of the hydraulic backhoe MV.

[0085] <Configuration of Optimization System 2A> The configuration of an optimization system 2A according to this application example will be described with reference to Fig. 7. Fig. 7 is a block diagram showing the configuration of the optimization system 2A. As shown in Fig. 7, the optimization system 2A has almost the same configuration as the optimization system 2, but includes a hydraulic excavator MV as a control target 25.

[0086] In this application example, an example will be described in which the optimization device 20 is configured by a computer. The computer configuring the optimization device 20 includes at least a processor, a memory, and a network interface. The optimization device 20 may also include a reading device, a magnetic storage device, etc. The reading device is a device for reading computer-readable recording media such as a USB (Universal Serial Bus) memory, a CD-ROM (Compact Disc Read Only Memory), etc.

[0087] The control unit 21 is configured with a processor. The control unit 21 loads program code received from a network interface into memory, or reads program code stored in a recording medium or magnetic storage device and loads it into memory. The processor then interprets and executes the loaded program code, causing the computer to function as an optimal variable candidate generation unit 111, an objective function evaluation unit 212, an inverse temperature optimization unit 113, a weight evaluation unit 114, a belief distribution update unit 115, a control input conversion unit 216, and a belief distribution processing unit 217.

[0088] For example, the optimization device 20 is a so-called personal computer (hereinafter referred to as a PC). This PC is equipped with a CPU (central processing unit) having a clock frequency of 3.20 [GHz (gigahertz)] and a GPU (graphical processing unit) equipped with 10,496 NVIDIA CUDA cores.

[0089] The storage unit 12 is configured by, for example, a memory or a magnetic storage device provided in the optimization device 20. The storage unit 12 includes an optimal variable candidate storage unit 121, an objective function value storage unit 122, a target effective sample size storage unit 123, an inverse temperature storage unit 124, a weight storage unit 125, and a belief distribution storage unit 126. In this application example, the storage unit 12 is a GPU memory with a storage capacity of 16 GB (gigabytes).

[0090] The input device 13 is a keyboard, a mouse, a touchpad, or the like connected to the optimization device 20 .

[0091] The hydraulic excavator MV is equipped with a remote control system. Hereinafter, the hydraulic excavator MV will also be simply referred to as the excavator MV. This remote control system is connected to the optimization device 20 via wireless communication such as WiFi (registered trademark). The remote control system receives control input from the optimization device 20 and remotely controls the operating lever of the excavator MV in accordance with the control input.

[0092] The movable range of the control lever in this application example will be described with reference to FIG. 8. FIG. 8 is a schematic diagram showing an example of the movable range of the control lever. In FIG. 8, an XY coordinate system is defined in which the horizontal direction of the rotation plane on which the bucket, arm, and boom of the excavator MV can rotate is the X axis and the vertical direction is the Y axis. As shown in FIG. 8, the excavator MV has a bucket B and a control lever (not shown). The movable range of the control lever is limited to a range corresponding to the rotational movement around the bucket axis a1, arm axis a2, and boom axis a3 of the excavator MV. The direction and strength of the rotational movement of each axis are determined by the tilt of the control lever, so the tilt of the control lever is specified by the control input. For convenience, the control input u at a discrete time t expressed by an arbitrary natural number is t is defined as the following equation (1).

[0093]

number

[0094] The state observer 26 observes the state of the excavator MV and transmits the observed state to the optimization device 20. In this application example, the state observer 26 is an inertial measurement unit (hereinafter referred to as IMU) provided on the excavator MV. The IMU observes the joint angles of the excavator MV, that is, the three angles θbucket, θarm, and θboom shown in FIG. 8, at discrete time t. θbucket indicates the angle around the bucket axis a1. θarm indicates the angle around the arm axis a2. θboom indicates the angle around the boom axis a3. In this application example, these three angles are used to represent the state x of the excavator MV at discrete time t. t and expressed as the following equation (2).

[0095]

number

[0096] <Flow of optimization method M20A> The optimization system 2A configured as described above executes an optimization method M20A. The optimization method M20A is a specific example in which the optimization method M20 is executed on an excavator MV. The optimization method M20A will be described with reference to FIG. 9. FIG. 9 is a flow diagram showing the flow of the optimization method M20A. Each step will be described in detail below. Note that the same details as those of the optimization method M20 will not be repeated, and the differences will be mainly described.

[0097] In step S1, the user inputs a target effective sample size and an initial belief distribution using the input device 13. Here, it is assumed that Nefftarget = 300 is input as the target effective sample size.

[0098] In step S100, the state observer 26 observes the state of the backhoe MV.

[0099] The control termination condition determined in step S101 is the observed state x t is true if the final target coordinate of the reference trajectory is reached. The reference trajectory and target coordinates will be described in detail later.

[0100] Here, to define the belief distribution, we first define the objective variable. In this application example, the prediction horizon in model predictive control is set to H = 20, and the objective variable is defined as the control input for H steps from the current discrete time t to t + H-1, and is expressed as in the following equation (3).

[0101]

number

[0102]

number

[0103] In step S2, the optimal variable candidate generation unit 111 generates a plurality of optimal variable candidates based on the initial belief distribution in the belief distribution storage unit 126 given by the input device 13 the first time and the belief distribution updated by the belief distribution update unit 115 in subsequent loop processing, and stores the generated optimal variable candidates in the optimal variable candidate storage unit 121. In this application example, the optimal variable candidates are generated using Monte Carlo sampling (MC sampling) with the number of optimal variable candidates generated set to K=64,000.

[0104] In step S3, the objective function evaluation unit 212 evaluates the objective function for each optimal variable candidate in the optimal variable candidate storage unit 121, and records the result in the objective function value storage unit 122. The objective function in this application example is t The state transition (trajectory) when the control inputs of vt:H are executed sequentially is predicted, and the predicted trajectory is evaluated. First, the state transition function is modeled as shown in the following equation (5).

[0105]

number

[0106]

number

[0107]

number

[0108] The reference trajectory in this application example will be described with reference to FIG. 10 . FIG. 10 is a schematic diagram for explaining the reference trajectory. As shown in FIG. 10 , the reference trajectory is constructed so that the height of the tip P of bucket B from the ground plane is maintained while the blade of bucket B is always kept horizontal. For example, the reference trajectory is constructed so that the height of the tip P of bucket B from the ground plane is maintained at 0.89 m and the reference trajectory moves horizontally between 0.63 m and 1.43 m in the X-axis direction from the swing axis of the excavator MV. Note that each arrow in FIG. 10 represents a target coordinate constituting the reference trajectory, with the arrow tip representing prefx and prefy and the arrow direction representing prefθ. Note that to avoid sudden starts and stops, the arrangement of the target coordinates is adjusted to have three phases: acceleration, constant velocity, and deceleration. Note that while FIG. 10 illustrates an example of a task of horizontal forward movement, a reference trajectory for a backward task is constructed using a similar concept. ax, ay, and aθ are coefficients that determine the cost weight of each term, and in this application example, they are set to 10000, 10000, and 10, respectively. The composite function of the total cost function and state transition function as described above is defined as the objective function S in this application example.

[0109] In step S4, the inverse temperature optimization unit 113 calculates an inverse temperature λ by an optimization method such that the target effective sample size Nefftarget in the target effective sample size memory unit 123 is equal to the effective sample size of the weight, and records the calculated value in the inverse temperature memory unit 124. The weight for each optimal variable candidate is the value obtained by dividing the likelihood by the marginal likelihood, and is expressed by the following equation (8).

[0110]

number

[0111]

number

[0112]

number

[0113] In step S6, belief distribution update unit 115 approximates the posterior belief distribution using the moment matching method based on each optimal variable candidate in optimal variable candidate storage unit 121, each weight in weight storage unit 125, and the belief distribution in belief distribution storage unit 126, and records this as a new belief distribution in belief distribution storage unit 126. Because the moment matching method is used, the approximate posterior belief distribution also becomes a Gaussian distribution as shown in equation (4), and its mean parameter (control input) ut:H is updated as shown in equation (11) below.

[0114]

number

[0115] On the other hand, if the user-specified termination condition is met (step S7 - true branch), step S108 is executed. In step S108, the control input conversion unit 216 converts the belief distribution updated in step S6 into a control input. The conversion is performed as follows.

[0116] In this application example, since the belief distribution is a Gaussian distribution, the optimal variable candidate with the maximum probability density coincides with the mean parameter ut:H of the Gaussian distribution, i.e., ut:H is the most promising optimal variable candidate. In this application example, application to model predictive control is assumed. Therefore, in step S109, the control input conversion unit 216 extracts only the element at the first time of the optimal variable candidate, that is, the element ut at discrete time t, and transmits this to the excavator MV.

[0117] In step S110, the belief distribution processor 217 processes the belief distribution and stores it in the belief distribution storage unit 126. Then, the loop process from step S100 is repeated again. In this application example, application to model predictive control is assumed. Therefore, the belief distribution is processed into a belief distribution shifted by one time step, that is, a belief distribution for the control input at discrete times t+1 to t+H. First, for the elements from discrete times t+1 to t+H-1, the elements of ut:H from discrete times t+1 to t+H-1 are used as is. For the elements at t+H, a three-dimensional zero vector is used, as when the initial belief distribution was set. The ut+1:H configured in this way is used as the parameters for the next initial belief distribution. Note that when proceeding to step S100 by the loop process, t←t+1.

[0118] <Effects of this application example> The effects of this application example will be described with reference to the graphs in Fig. 11 and Fig. 12 and the tables in Fig. 13 and Fig. 14. Figs. 11 to 14 are diagrams and tables for comparing the performance of the control system when the inverse temperature is fixed (related technologies in Non-Patent Documents 1 and 2) and when the inverse temperature is automatically adjusted (this application example). The graph in Fig. 11 and the table in Fig. 13 are evaluations of the task of horizontally moving the tip P of bucket B forward. The graph in Fig. 12 and the table in Fig. 14 are evaluations of the task of horizontally moving the tip P of bucket B backward.

[0119] This performance evaluation was conducted using a simulation of a MV excavator, and the average value and 1σ confidence interval were calculated based on 300 trials for each of the following settings. Note that the 1σ confidence intervals are shown in parentheses in the tables of Figures 13 and 14. A total of eight settings were used, with experiments conducted for fixed λ values ​​of 30, 100, 300, and 1000, as well as for automatic adjustment of the inverse temperature, with Nefftarget set to 30, 100, 300, and 1000, as in this application example. In the graphs of Figures 11 and 12 and the tables of Figures 13 and 14, the above settings will be represented as lam30, lam100, lam300, lam1000, ess30, ess100, ess300, and ess1000, respectively.

[0120] In this simulation, a pseudo-pulse-type disturbance is added every 20 time steps. This pulse-type disturbance changes the value of (θbucket,t, θarm,t, θboom,t) by an amount of (+4.5, -4.5, +7.5) [deg.] at the discrete time t when the disturbance occurs. Assuming that this disturbance cannot be predicted in advance, the objective function evaluation unit 212 is set not to take the disturbance into account when estimating the trajectory. In other words, when a disturbance occurs, the trajectory always deviates significantly from the expected trajectory, so it is necessary to quickly and accurately correct the belief distribution.

[0121] The Failure Rate in the graphs of Figures 11 and 12 and the tables of Figures 13 and 14 indicates that a failure occurs when any of the differences (|px-prefx|, |py-prefy|, |pθ-prefθ|) between the target coordinate and the current coordinate exceeds (0.1 m, 0.1 m, 10 deg.), and the task is terminated at that point. Failures are likely to occur when the belief distribution cannot be quickly and accurately corrected after a disturbance occurs. For ess300 and ess1000, the failure rate is shown as 0 in Figures 11 and 13 because there were no failures in all 300 trials. Regret represents the difference in total cost relative to the optimal control law. However, because it is difficult to know the optimal control law for this task setting in advance, this evaluation represents the difference from the lowest total cost among all settings and trials. Regret is evaluated only for tasks that did not fail. Therefore, in settings with a high failure rate, the sample size is small, resulting in a large 1σ confidence interval.,Furthermore, in the tables in Figures 13 and 14, the shaded characters and the characters with,diagonal lines represent the top two and third to fourth places in each,category of failure rate and regret.

[0122] According to the results shown in the graphs in Figures 11 and 12 and the tables in Figures 13 and 14, the settings that performed better overall in terms of failure rate and regret for both forward and backward tasks were the ESS series, and in particular, ESS300 and ESS1000 were in the top two in terms of failure rate and regret for both tasks.

[0123] In the forward task, the minimum failure rate for the LAM system was 3%, while all of the ESS system settings were below 3%, especially 0% for ESS300 and ESS1000.Regarding regret, the minimum regret for the ESS system (ESS1000, 13,933) was approximately 1.14 times better than the minimum regret for the LAM system (LAM300, 15,933).

[0124] In the backward task, the minimum failure rate in the LAM system setting was 1.7%, while it was 1% and 0.7% for the ESS300 and ESS1000, respectively.Regarding regret, the minimum regret in the ESS system (ESS300, 35,655) was approximately 2.59 times better than the minimum regret in the LAM system (LAM300, 92,227).

[0125] We also observed that the failure rate tends to decrease as the target effective sample size of the ESS system increases. This tendency is due to the fact that, in the optimization system 2A of this application example, the sampling error decreases as the target effective sample size increases. Furthermore, by setting the target effective sample size as small as possible within the range within which the sampling error is acceptable, we were able to ensure a good balance between update stability and efficiency, thereby maintaining a low failure rate and reducing regret, as in ESS300. Alternatively, by setting a larger target effective sample size, we could emphasize update stability and achieve a lower failure rate, as in ESS1000. Overall, the optimization system 2A of this application example can be used as an application of model predictive control, and by automatically adjusting the inverse temperature to maintain a constant effective sample size, we were able to improve the difficulty of adjusting the inverse temperature.

[0126] The exemplary embodiments and application examples described above are preferred embodiments of the present invention, and the scope of the present invention is not limited to only the exemplary embodiments and application examples. The present invention can be implemented in various modified forms without departing from the spirit of the present invention.

[0127] [Software implementation example] Some or all of the functions of the optimization devices 10 and 20 may be realized by hardware such as an integrated circuit (IC chip), or by software.

[0128] In the latter case, the optimization devices 10 and 20 are realized, for example, by a computer that executes instructions of a program, which is software that realizes each function. An example of such a computer (hereinafter referred to as computer C) is shown in FIG. 17. The computer C includes at least one processor C1 and at least one memory C2. The memory C2 stores a program P for operating the computer C as the optimization devices 10 and 20. In the computer C, the processor C1 reads and executes the program P from the memory C2, thereby realizing each function of the optimization devices 10 and 20.

[0129] The processor C1 may be, for example, a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a micro processing unit (MPU), a floating point number processing unit (FPU), a physics processing unit (PPU), a microcontroller, or a combination thereof. The memory C2 may be, for example, a flash memory, a hard disk drive (HDD), a solid state drive (SSD), or a combination thereof.

[0130] The computer C may further include a RAM (Random Access Memory) for expanding the program P during execution and for temporarily storing various data. The computer C may also include a communication interface for transmitting and receiving data to and from other devices. The computer C may also include an input / output interface for connecting input / output devices such as a keyboard, mouse, display, and printer.

[0131] Furthermore, the program P can be recorded on a non-transitory tangible recording medium M that can be read by the computer C. Such a recording medium M can be, for example, a tape, a disk, a card, a semiconductor memory, or a programmable logic circuit. The computer C can acquire the program P via such a recording medium M. The program P can also be transmitted via a transmission medium. Such a transmission medium can be, for example, a communication network or broadcast waves. The computer C can also acquire the program P via such a transmission medium.

[0132] [Appendix 1] The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. For example, embodiments obtained by appropriately combining the technical means disclosed in the above-described embodiments are also included in the technical scope of the present invention.

[0133] [Appendix 2] Some or all of the above-described embodiments can also be described as follows: However, the present invention is not limited to the following described aspects.

[0134] (Appendix 1) an optimal variable candidate generating means for generating a plurality of optimal variable candidates based on the belief distribution; an objective function evaluation means for evaluating an objective function for each of the plurality of optimal variable candidates; an inverse temperature optimization means for calculating, using an optimization method, an inverse temperature such that the input target effective sample size and the effective sample size of the weight for the objective function are approximately the same; a weight evaluation means for calculating a weight for the objective function based on the inverse temperature; a belief distribution update means for updating the belief distribution based on the optimal variable candidates, the weights, and the belief distribution; An optimization device comprising:

[0135] (Appendix 2) the optimal variable candidate generating means generates the plurality of optimal variable candidates based on the input initial belief distribution or the belief distribution updated by the belief distribution updating means; 2. The optimization apparatus of claim 1.

[0136] (Appendix 3) The objective function evaluation means evaluates the objective function, which depends on the state of the controlled object observed by a state observation device, for each of the plurality of optimal variable candidates. 3. The optimization device of claim 1 or 2.

[0137] (Appendix 4) a control input conversion means for calculating a control input in accordance with a predetermined conversion rule based on the belief distribution updated by the belief distribution update means, and transmitting the calculated control input to a controlled object; 4. The optimization device of any one of appendices 1 to 3, further comprising:

[0138] (Appendix 5) a belief distribution processing means for processing the belief distribution updated by the belief distribution updating means in a certain step for processing by the optimal variable candidate generating means, the objective function evaluating means, the inverse temperature optimization means, the weight evaluating means, and the belief distribution updating means in a next step; 5. The optimization device of any one of appendices 1 to 4, further comprising:

[0139] (Appendix 6) generating a plurality of optimal variable candidates based on the belief distribution; evaluating an objective function for each of the plurality of optimal variable candidates; Calculating an inverse temperature using an optimization method such that the input target effective sample size and the effective sample size of the weight for the objective function are approximately the same; calculating a weight for the objective function based on the inverse temperature; updating the belief distribution based on the optimal variable candidates, the weights, and the belief distribution; Optimization methods including:

[0140] (Appendix 7) Before the step of generating the plurality of optimal variable candidates, Accepting inputs of a target effective sample size and an initial belief distribution; 7. The optimization method of claim 6, further comprising:

[0141] (Appendix 8) After the updating step, outputting the updated belief distribution if a predetermined termination condition is met; If a predetermined termination condition is not satisfied, executing a loop process from the step of generating the plurality of optimal variable candidates; 8. The optimization method of claim 6 or 7, further comprising:

[0142] (Appendix 9) A program for causing a computer to function as an optimization device, the program comprising: an optimal variable candidate generating means for generating a plurality of optimal variable candidates based on the belief distribution; an objective function evaluation means for evaluating an objective function for each of the plurality of optimal variable candidates; an inverse temperature optimization means for calculating, using an optimization method, an inverse temperature such that the input target effective sample size and the effective sample size of the weight for the objective function are approximately the same; a weight evaluation means for calculating a weight for the objective function based on the inverse temperature; a belief distribution update means for updating the belief distribution based on the optimal variable candidates, the weights, and the belief distribution; A program that functions as a

[0143] [Appendix 3] Some or all of the above-described embodiments can also be expressed as follows.

[0144] The method includes at least one processor, and the processor generates a plurality of optimal variable candidates based on the belief distribution; an objective function evaluation process for evaluating an objective function for each of the plurality of optimal variable candidates; an inverse temperature optimization process that uses an optimization method to calculate an inverse temperature such that the input target effective sample size and the effective sample size of the weight for the objective function are approximately the same; a weight evaluation process for calculating a weight for the objective function based on the inverse temperature; a belief distribution update process for updating the belief distribution based on the optimal variable candidates, the weights, and the belief distribution.

[0145] The optimization device may further include a memory that stores a program for causing the processor to execute the optimal variable candidate generation process, the objective function evaluation process, the inverse temperature optimization process, the weight evaluation process, and the belief distribution update process. The program may also be recorded on a computer-readable, non-transitory, tangible recording medium. [Explanation of symbols]

[0146] 1, 2, 2A, 9 Optimization System 10, 20, 90, 100 Optimizer 11, 21, 91, 911 Control unit 12, 92 13, 93 Input devices 14, 94 Output Device 24 Observation equipment 125, 925 Weight storage unit 25 Control Objects 26 State Observation Device 101, 111, 911 Optimal variable candidate generation unit 102, 112, 212, 912 Objective function evaluation section 103, 113 Inverse temperature optimization section 104, 114, 914 Weight evaluation section 105, 115, 915 Belief distribution update part 121, 921 Optimal variable candidate memory section 122, 922 Objective function value storage unit 123 Target effective sample size memory 124, 924 Reverse temperature memory section 126, 926 Belief distribution memory section 216 Control Input Conversion Unit 217 Belief Distribution Processing Department C1 processor C2 Memory

Claims

1. an optimal variable candidate generating means for generating a plurality of optimal variable candidates based on the belief distribution; an objective function evaluation means for evaluating an objective function for each of the plurality of optimal variable candidates; an inverse temperature optimization means for calculating an inverse temperature such that an input target effective sample size and an effective sample size of the weight for the objective function are approximately the same; a weight evaluation means for calculating a weight for the objective function based on the inverse temperature; a belief distribution update means for updating the belief distribution based on the optimal variable candidates, the weights, and the belief distribution; An optimization device comprising:

2. the optimal variable candidate generating means generates the plurality of optimal variable candidates based on the input initial belief distribution or the belief distribution updated by the belief distribution updating means; The optimization device according to claim 1 .

3. The objective function evaluation means evaluates the objective function, which depends on the state of the controlled object observed by a state observation device, for each of the plurality of optimal variable candidates.

3. The optimization device according to claim 1 or 2.

4. a control input conversion means for calculating a control input in accordance with a predetermined conversion rule based on the belief distribution updated by the belief distribution update means, and transmitting the calculated control input to a controlled object; The optimization device according to claim 1 or 2, further comprising:

5. a belief distribution processing means for processing the belief distribution updated by the belief distribution updating means in a certain step for processing by the optimal variable candidate generating means, the objective function evaluating means, the inverse temperature optimization means, the weight evaluating means, and the belief distribution updating means in a next step; The optimization device according to claim 1 or 2, further comprising:

6. A method of generating a plurality of optimal variable candidates based on a belief distribution by a computer; evaluating an objective function for each of the plurality of optimal variable candidates; A computer calculates an inverse temperature such that the input target effective sample size and the effective sample size of the weight for the objective function are comparable; a computer calculating weights for the objective function based on the inverse temperatures; updating the belief distribution based on the optimal variable candidates, the weights, and the belief distribution; Optimization methods including:

7. The computer, prior to the step of generating the plurality of optimal variable candidates, accepting input of the target effective sample size and an initial belief distribution; The optimization method of claim 6 further comprising:

8. After the updating step, outputting the updated belief distribution when a predetermined termination condition is satisfied; the computer executes a loop process starting from the step of generating the plurality of optimal variable candidates when a predetermined termination condition is not satisfied; The optimization method according to claim 6 or 7, further comprising:

9. A program for causing a computer to function as an optimization device, the program comprising: an optimal variable candidate generating means for generating a plurality of optimal variable candidates based on the belief distribution; an objective function evaluation means for evaluating an objective function for each of the plurality of optimal variable candidates; an inverse temperature optimization means for calculating an inverse temperature such that an input target effective sample size and an effective sample size of the weight for the objective function are approximately the same; a weight evaluation means for calculating a weight for the objective function based on the inverse temperature; a belief distribution update means for updating the belief distribution based on the optimal variable candidates, the weights, and the belief distribution; A program that functions as a

Images