Information processing device, information processing method, and storage medium

Abstract

An information processing device 100 according to the present disclosure comprises: a weight selection unit 121 that selects one weight from among a plurality of weights that can be set in a machine learning model and are obtained by training the machine learning model under different conditions, the one weight being selected by using a selection model generated by machine learning; a weight synthesis unit 122 that generates a synthesized weight that is a new weight obtained by synthesizing a plurality of weights including the selected weight; an evaluation unit 123 that evaluates, with regard to one or more preset indices, the performance of the machine learning model in which the synthesized weight has been set; and an update unit 124 that updates the selection model by training the selection model by machine learning on the basis of the evaluation.

Description

Information processing device, information processing method, and storage medium

[0001] The present disclosure relates to an information processing device, an information processing method, and a storage medium.

[0002] In recent years, machine learning models have been used to make predictions in various fields, and there is a demand for improving the performance of machine learning models. It is known that for a specific index used to evaluate the performance of a machine learning model, an average weight obtained by averaging multiple weights under appropriate rules exhibits higher performance than selecting the single weight with the highest accuracy. For this reason, in Non-Patent Document 1, multiple weights (learning parameters) obtained by learning with different settings for a single machine learning model are stored in a weight pool, and several weights extracted from the weight pool are averaged.

[0003] Specifically, Non-Patent Document 1 performs weight averaging based on a greedy rule called greedy soup, which is a method that responds ad hoc without considering the consequences. This method extracts weights from a weight pool in descending order of performance, and adds them to the average only if they improve performance for a single accuracy evaluation metric.

[0004] The weight averaging device based on the greedy soup described above outputs an average weight for a single accuracy evaluation index that is more accurate than or at least as accurate as the weight at the top of the weight pool (the weight with the highest accuracy for that index). Therefore, the weight averaging process does not result in a decrease in accuracy compared to the original weight. Here, if the average weight is updated only once, the average weight output will match the weight at the top of the weight pool. This boils down to an operation of selecting the weight with the highest accuracy for the index from multiple weights given as a premise.

[0005] Mitchell Wortsman, Gabriel Ilharco, Samir Ya Gadre, Rebecca Roelofs, Raphael Gontijo-Lopes, Ari S Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, Ludwig Schmidt, Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time. In International Conference on Machine Learning, pp. 23965-23998, 2022.

[0006] However, in the technology described in Non-Patent Document 1, the extracted weights are considered to have two options: whether to use them for averaging or not, and therefore the variations in the output average weights are limited to powers of 2. Therefore, unless there are weights among these variations that can improve accuracy, the performance of the machine learning model cannot be improved. As a result, there is a problem in that further performance improvement of the machine learning model cannot be achieved.

[0007] Therefore, the purpose of the present disclosure is to solve the above-mentioned problem of being unable to further improve the performance of machine learning models.

[0008] An information processing device according to one aspect of the present disclosure includes: a weight selection unit that selects one weight using a selection model generated by machine learning from among a plurality of weights that can be set for a machine learning model obtained by training the machine learning model under different conditions; a weight combination unit that generates a composite weight that is a new weight by combining a plurality of weights including the selected weight; an evaluation unit that evaluates performance of the machine learning model to which the composite weight has been set with respect to at least one or more predetermined indicators; and an update unit that updates the selection model through machine learning based on the evaluation. Also, an information processing method according to one aspect of the present disclosure includes: selecting one weight using a selection model generated by machine learning from among a plurality of weights that can be set for the machine learning model obtained by training the machine learning model under different conditions; generating a composite weight that is a new weight by combining a plurality of weights including the selected weight; evaluating performance of the machine learning model to which the composite weight has been set with respect to at least one or more predetermined indicators; and updating the selection model through machine learning based on the evaluation. Furthermore, a program that is one form of the present disclosure is configured to cause a computer to execute the following processes: select one weight using a selection model generated by machine learning from multiple weights that can be set to the machine learning model obtained by training the machine learning model under different conditions; generate a composite weight that is a new weight by combining multiple weights including the selected weight; evaluate the performance of the machine learning model to which the composite weight has been set with respect to at least one or more predetermined indicators; and update the selection model through machine learning based on the evaluation.

[0009] By being configured as described above, the present disclosure can further improve the performance of machine learning models.

[0010] FIG. 1 is a block diagram showing a configuration of a weight averaging device according to the present disclosure. FIG. 2 is a flowchart showing a processing operation of a weight averaging device according to the present disclosure. FIG. 3 is a diagram showing a state of processing by a weight averaging device according to the present disclosure. FIG. 4 is a block diagram showing a configuration of a weight averaging device according to the present disclosure. FIG. 5 is a flowchart showing a processing operation of a weight averaging device according to the present disclosure. FIG. 6 is a block diagram showing a hardware configuration of an information processing device according to the present disclosure. FIG. 7 is a block diagram showing a configuration of an information processing device according to the present disclosure.

[0011] First Embodiment A first embodiment of the present disclosure will be described with reference to the drawings. Note that the drawings may be relevant to any embodiment.

[0012] The weight averaging device in this embodiment has a function of improving a weight selection strategy used to set optimal weights for a machine learning model that makes predictions on input data, in order to improve the performance of the machine learning model. In this embodiment, the machine learning model is, for example, a neural network trained by machine learning to perform multi-class classification, and the optimal weight set for the neural network is an average weight obtained by averaging multiple weights. In this case, the weight averaged as the average weight is selected from multiple weights by a weight selection strategy. The weight averaging device is configured to have a function of machine learning the weight selection strategy so that the weight selection strategy selects the optimal weight.

[0013] In this embodiment, the machine learning model is a neural network that performs multi-class classification, but the machine learning model may be any model that performs prediction. Furthermore, the weight set in the machine learning model is not necessarily limited to being the average of multiple weights, but may be a composite weight generated by combining multiple weights through some kind of calculation.

[0014] Specifically, the weight averaging device 1 is configured with one or more information processing devices each including a calculation device and a storage device. As shown in FIG. 1 , the weight averaging device 1 includes an average weight state input unit 11, an evaluation importance input unit 12, a weight selection unit 13, a weight input unit 14, a weight averaging unit 15, an accuracy evaluation unit 16, a composite reward calculation unit 17, an average weight update unit 18, a profit calculation unit 19, a weight selection update unit 20, and a policy output unit 21. These functions can be realized by the calculation device executing a program for realizing each function stored in the storage device. The weight averaging device 1 also includes a weight pool 31 and a composite reward pool 32, each of which is configured with a storage device.

[0015] Below, we will explain in detail each component of the weighted averaging device 1. First, we will explain the outline of the function of each component, and then we will explain in detail the processing operation of each component using specific examples.

[0016] The weight pool 31 stores two or more additional weights obtained by performing learning under different conditions for any one type of machine learning model. The data structure of the weight pool 31 is a structure that allows the added weights to be retrieved in any order, and may be a list structure or a tuple structure.

[0017] The average weight state input unit 11 inputs a state characterizing the current average weight. Here, the specific system of the state characterizing the average weight is not limited, and may be, for example, the current average weight itself, or a vector of the accuracy of the average weight for one or more evaluation indexes (a scalar when there is one index).

[0018] The evaluation importance input unit 12 inputs importance, which quantitatively expresses the degree of importance placed on improving the accuracy of one or more evaluation indices that evaluate the performance of the machine learning model. The specific system of importance is not limited, and may be, for example, an integer vector that expresses the importance of each index on a 10-point scale, or a real value vector (each element of which corresponds to an importance level between 0 and 1) normalized so that the sum is 1.

[0019] The weight selection unit 13 selects one weight from the weight pool 31 based on the current weight selection policy and extracts the selected weight by sampling with replacement (i.e., a copy of the extracted weight remains in the weight pool 31). Here, the weight selection policy of the weight selection unit 13 is defined as a function (selection model) that receives as input the average weight state and importance input from the average weight state input unit 11 and the evaluation importance input unit 12, and outputs an action of selecting a weight from the weight pool 31. The weight selection policy is assumed to be improved by machine learning, as described below, and may be, for example, a neural network or an agent in reinforcement learning.

[0020] The weight input unit 14 inputs the weight selected by the weight selection unit 13 and the current average weight.

[0021] The weighted average unit 15 (weight synthesis unit) performs an arithmetic average on the two types of weights input to the weight input unit 14, and outputs a new average weight.

[0022] The accuracy evaluation unit 16 (evaluation unit) performs accuracy evaluation for at least one or more accuracy evaluation indexes, using the new average weights output by the weighted average unit 15. Here, the specific system of the accuracy evaluation indexes is not limited, and may be multiple indexes for a single evaluation data set, or individual indexes for multiple evaluation data sets.

[0023] The composite reward calculation unit 17 (evaluation unit) calculates a composite reward based on the evaluation result for the new average weight obtained by the accuracy evaluation unit 16 and the importance of each index input by the evaluation importance input unit 12. Here, the definition of the composite reward will be explained in detail. First, when the accuracy of each index improves, a reward is defined according to the extent of the accuracy improvement. The specific system of the reward is not limited, and it may be defined as the accuracy improvement rate relative to a specific weight, or as the difference from the ideal accuracy. The composite reward is defined as the sum of the rewards calculated for each index, weighted by the importance of each index. Therefore, the more importance a user of the weight averaging device 1 places on improving the accuracy of an index, the more the accuracy improvement will contribute to an increase in the composite reward.

[0024] The average weight updating unit 18 overwrites and updates the current average weight with the new average weight output by the weight averaging unit 15 .

[0025] The profit calculation unit 19 calculates the profit of the composite reward obtained up to the point when the update of the average weight is completed. The specific method of calculating the profit is not limited, and may be a simple sum of the composite rewards or a time-decaying sum used in reinforcement learning.

[0026] The weight selection update unit 20 (update unit) improves the weight selection policy in the weight selection unit 13 based on the profit obtained by the profit calculation unit 19. When the weight selection policy is a neural network, parameters are modified in a direction that maximizes the profit of the composite reward. For parameter modification, for example, backpropagation used in machine learning such as deep learning may be used. When the weight selection policy is a policy in reinforcement learning, the value function and policy function are modified so that an action that maximizes profit is selected. For these modifications, for example, Temporal Difference (TD) learning, policy gradient method, etc. may be used.

[0027] The policy output unit 21 outputs the final weight selection policy after the improvement is completed. If the weight selection policy is a neural network, the unit outputs the learned parameters. If the weight selection policy is a policy in reinforcement learning, the unit outputs the learned value function or policy function.

[0028] Next, the operation of the weighted averaging device 1 according to this embodiment will be described in detail with specific examples with reference to Fig. 2. A conceptual diagram of the processing operation in the weighted averaging device 1 is shown in Fig. 3.

[0029] 2 is a flowchart showing an example of the operation of the weight averaging device 1 according to this embodiment. This embodiment corresponds to a case in which the weight averaging device 1 involves learning of a weight selection policy.

[0030] In this embodiment, as an example of a machine learning model to which the weight averaging device 1 is applied, the neural network M that has undergone multi-class classification training as described above is used. The neural network M has undergone N training cycles under different hyperparameter settings (e.g., learning rate, weight decay, etc.), and the weights obtained in each training cycle are represented as Wi (i = 1, ..., N). These weights are added to a weight pool 31 with a list structure in an arbitrary order.

[0031] Furthermore, it is assumed that there are L evaluation datasets (each dataset is generally a collection of evaluation data with a specific evaluation purpose) for evaluating the performance of the neural network M, and the class classification accuracy for each evaluation dataset is the accuracy evaluation index (i.e., it is assumed that there are L accuracy evaluation indexes). Note that, in order to prevent overlearning of the neural network M, it is assumed that the L evaluation datasets are prepared separately from the evaluation dataset for testing. Note that the accuracy evaluation index such as the class classification accuracy listed above is just an example, and any index may be used as long as it represents an evaluation of the performance of the machine learning model.

[0032] Hereinafter, the above-mentioned accuracy evaluation index will be referred to as the average weight state, and as a specific example, it is defined as a binary vector s that takes the value 1 if the data in the evaluation dataset is correctly classified into classes, and takes the value 0 if it is not. If the total number L of evaluation datasets is 2 or more, the vector obtained by concatenating the above binary vectors is defined as the average weight state s.

[0033] As a specific example of the importance of each index described above, each evaluation index is defined as a real value ranging from 0 to 1. A normalized vector with each importance as an element is defined as an importance vector g (see [w1, w2, ...] in Figure 3). Hereinafter, the importance vector g will be simply referred to as importance.

[0034] First, in the initialization process S1, the average weight, average weight state, and importance for each evaluation index are initialized. The initial value of the average weight may be, for example, a single weight arbitrarily selected from the weight pool 31. The initial value of the average weight state may be, for example, a constant vector and a constant tensor with all elements initialized to zero. The initial value of the importance is set to a random value sampled from an arbitrary probability distribution. For the average weight and average weight state, common initial values ​​are used hereafter throughout this embodiment. For the importance, a random value is reset at each initialization.

[0035] A specific example of the initial state of the average weight is a single weight arbitrarily selected from the weight pool 31. A specific example of the initial state of the average weight state s is a binary vector obtained by performing accuracy evaluation on one weight arbitrarily selected from the weight pool 31. A specific example of the initial state of the importance is to determine the importance value by a uniform random number ranging from 0 to 1.

[0036] Next, in the weight selection process S2, the weight selection unit 13 receives as input the current average weight state and the importance of each evaluation index from the average weight state input unit 11 and the evaluation importance input unit 12. Next, the weight selection unit 13 inputs the received average weight state and the importance of the evaluation index from the weight pool 31 to the weight selection strategy, and selects and extracts one weight based on the action obtained as the output.

[0037] As a specific example of the weight selection unit 13, the weight selection unit 13 is defined as an agent in value function-based reinforcement learning. The weight selection policy referenced by the agent is a function expressed as π:(s, g)→a. Here, a is a possible action of the agent, and in this case, it corresponds to the operation of selecting a weight from the weight pool 31. For example, a is defined as an N-dimensional one-hot vector, meaning that a selects a weight corresponding to a dimension whose value is 1. The specific structure of the weight selection policy π is arbitrary, and the ε-greedy policy, Softmax policy, etc. are used. Hereinafter, the tuple (s, g) summarizing the inputs of the weight selection policy π is simply referred to as the state. Note that the importance vector g is a supplementary input to the policy π, and a policy π:s→a that does not use g as an argument is also possible. If the importance g is not used, the state (s, g) is identified as the average weight state s.

[0038] The state-action value function referenced by the weight selection policy π is represented by Q. The state-action value function Q is defined as a neural network that takes state (s, g) and action a as input and outputs the value of taking action a in state (s, g) (i.e., the degree to which the reward profit will be high in the future), and Θ is the learning parameter of the state-action value function Q.

[0039] The weight selection unit 13 inputs the received average weight state s and importance g into the state-action value function Q to obtain the value Q(s, g, a) of each action a. The weight selection policy π selects the optimal action a* based on the value Q(s, g, a). For example, if the weight selection policy π is an ε-greedy policy, it normally selects the action a that maximizes the value Q(s, g, a), but with a certain probability, it randomly selects another action. The weight corresponding to the selected optimal action a* is extracted from the weight pool 31. Note that if importance g is not used, the input to the state-action value function Q is only the state s, and the output value Q(s, a) depends only on the state s and the action a.

[0040] Next, in weighted averaging processing S3, the weight input unit 14 receives the weight extracted in weight selection processing S2 and the current average weight, and passes each weight to the weight averaging unit 15. Subsequently, the weight averaging unit 15 performs an arithmetic average of the two types of input weights and outputs a new average weight. Note that the weighted averaging processing is not necessarily limited to averaging the weights, and it is also possible to calculate a composite weight by combining multiple weights using another preset calculation method, and use this composite weight as the average weight.

[0041] Next, in the accuracy evaluation process S4, the accuracy evaluation unit 16 receives the new average weights output in the weighted average process S3 and performs accuracy evaluation for at least one or more accuracy evaluation indexes using the received average weights.

[0042] Next, in the composite reward calculation process S5, first, a reward based on the accuracy improvement extent is calculated for each indicator based on the evaluation results obtained in the accuracy evaluation process S203. Next, the composite reward calculation unit 17 receives the importance from the evaluation importance input unit 12, and calculates the composite reward by weighting the calculated rewards by the importance and taking the sum. Finally, the calculated composite reward is added to the composite reward pool 32. If necessary, additional information other than the composite reward is also added to the composite reward pool 32 at the same time. For example, if the weight selection unit 13 is used as a reinforcement learning agent, a set of information necessary for policy improvement (i.e., the composite reward, the current average weight state, the new average weight state, and the action taken by the weight selection policy) is added.

[0043] As a specific example of the composite reward calculation process S5, first, a reward for each index is calculated based on the accuracy evaluation results obtained for each index. The reward is defined as the rate of improvement in accuracy of the new average weight when compared with the accuracy of the reference weight. Therefore, if accuracy deteriorates, the reward value becomes negative, which means that a penalty is given. The rewards for each index are summed up, weighted by importance, to calculate the composite reward r (see r in Figure 3). If importance g is not used, the sum of the rewards for each index is simply used as the composite reward r. The calculated composite reward r, the state (s, g), and the selected action a are added to the composite reward pool 32 in a queue structure.

[0044] Next, in the average weight updating process S6, the current average weight and average weight state are overwritten and updated with the average weight and average weight state calculated in the weighted average process S3.

[0045] Next, in conditional branch S7, it is determined whether the average weight update is complete. The specific criteria for this determination are not limited. For example, the process may end when the average weight has been updated a predetermined number of times, when the difference between the composite reward calculated in the composite reward calculation process S5 and the composite reward calculated in the previous iteration is less than a predetermined threshold, or when an action to terminate the update is selected as an action in the weight selection policy. If the termination condition for the update is met (Yes in S7), the process proceeds to the profit calculation process S8. If the termination condition is not met (No in S7), the process returns to the step immediately before the weight selection process S2.

[0046] In the profit calculation process S8, the composite reward (and additional information) accumulated in the composite reward pool 32 is extracted, and the profit is calculated by the composite reward calculation unit 17. As a method for extracting the composite reward from the composite reward pool 32, all the composite rewards up to the end of updating the average weight may be extracted, or only the composite rewards for a few updates may be extracted.

[0047] As a specific example of the profit calculation process S8, the composite reward r for n iterations is calculated from the composite reward pool. ｔ and state (S ｔ , g ｔ ), action a ｔ (t=0,...,n). Under the decay rate γ (0<γ<1), the profit R is calculated as Σ ｔ＝０,…,ｎ gamma ｔ r ｔ Calculated as follows.

[0048] Next, in the weight selection update process S9, the profit calculated in the profit calculation process S8 is received as an input, and the weight selection policy of the weight selection unit 13 is updated based on the update method defined in the weight selection update unit 20. As a specific example of the weight selection update process S9, the profit R and n states (S ｔ , g ｔ ), action a ｔThe n-step TD error is calculated using the algorithm, and the parameters Θ of the state-action value function Q are updated to minimize the squared loss or Huber loss. Θ is updated using the backpropagation algorithm typically used in deep learning, and optimization methods such as SGD (Stochastic Gradient Descent) or RMSProp (Root Mean Square Propagation) are used. This update changes the state-action value function Q so that it can correctly estimate the future value of selecting action a in a given state (s, g). The weight selection policy π is improved by updating the state-action value function Q so that a more optimal action a can be selected.

[0049] Next, in conditional branch S10, it is determined whether or not to terminate the update of the weight selection policy of the weight selector 13. The specific criteria for this determination are not limited, and for example, the process may terminate when the weight selection policy has been updated a predetermined number of times, or when the change in the value of the objective function that the weight selection updater 20 is attempting to maximize (or the loss function that the weight selection updater 20 is attempting to minimize) falls below a predetermined threshold. If the termination condition for the update is met (Yes in S10), the process proceeds to average weight output process S11, and if the termination condition is not met (No in S10), the process returns to just before initialization process S1.

[0050] Next, in the policy output process S11, the weight selection policy for which learning has been completed is output. As a specific example of the conditional branch S10 and the policy output process S11, if the number of updates of the state-action value function Q has not reached the predetermined number set by the user, the process returns to just before the initialization process. If the predetermined number of updates has been reached, the parameter Θ of the state-action value function for which learning has been completed is output, and the operation is completed.

[0051] In this way, the weight averaging device 1 outputs a weight averaging strategy that has been trained to maximize future profits from composite rewards under a given single machine learning model, its weight set, and evaluation index, and then its operation ends.

[0052] Next, the effects of the weighted averaging device 1 according to this embodiment will be described. As described above, according to this embodiment, the weight selector 13 selects weights from the weight pool 31 by sampling with replacement. In this respect, unlike the technology described in the prior art, it is possible to add the same weight originally contained in the weight pool 31 multiple times to the weighted averaging process. This is equivalent to the fact that, when the average weight is expressed as a weighted average of all weights in the weight pool 31, in the prior art the weight coefficient can only be a binary integer of 0 or 1, whereas in this embodiment the weight coefficient can be a continuous real value between 0 and 1. Therefore, the possible variations of the average weight are non-additively infinite, thereby solving one of the problems of the present disclosure.

[0053] Furthermore, in addition to the above-mentioned problems, the techniques of the prior art documents have two additional problems. The following will explain how the technique of the present disclosure can simultaneously solve these problems.

[0054] As a first further problem, the techniques in the prior art are greedy algorithms, and therefore cannot select optimal weights while taking into consideration future gains (i.e., until the final average weights are obtained). Therefore, the final average weights are generally a locally optimal solution, and are not guaranteed to be a globally optimal solution.

[0055] According to this embodiment, the weight selection policy in the weight selection unit 13 is improved in the weight selection update unit 20 so that the selection that maximizes the profit of the composite reward calculated by the profit calculation unit 19 is selected. Here, since the target to be maximized is not the composite reward per iteration but the profit of the composite reward for all iterations until the average weight update is completed, the policy improvement is implemented to maximize future gains. As a result, it is possible to move away from greedy optimization, and the above-mentioned problem is solved.

[0056] A second problem is that the prior art technology does not guarantee that it will output the optimal average weight when there are two or more evaluation indicators. The improvement in accuracy is only guaranteed for a single indicator, and the improvement in accuracy is not guaranteed for other indicators. In other words, the user of the prior art technology has no way to control the balance of accuracy between multiple indicators. Furthermore, when there are multiple evaluation indicators, the optimal average weight cannot be uniquely determined. This will be explained in detail using FIG. 4.

[0057] Figure 4 is a conceptual diagram showing the accuracy of three different accuracy evaluation indices versus two different average weights, expressed as a bar graph. The average weights on the left and right of Figure 4 both represent Pareto optimal states, meaning that it is not possible to improve the accuracy of one index without degrading the accuracy of the other index. Since there are an infinite number of Pareto-optimal average weights, the optimal average weight cannot be uniquely determined. Therefore, when simultaneously optimizing multiple different indices, the user of the weight averaging device must specify in advance which optimal state should be achieved.

[0058] According to this embodiment, it is necessary to define the importance of each evaluation index to be input to the evaluation importance input unit 12. By defining the importance, the average weight that the weight averaging device 1 should output is determined from among the countless average weights that exist in the Pareto-optimized state. This partially solves the above-mentioned problem. A means by which the user of the weight averaging device 1 controls the balance of accuracy changes among multiple indexes will be explained in the second embodiment described below, and thus the above-mentioned problem is ultimately solved.

[0059] Second Embodiment A second embodiment of the present disclosure will be described with reference to the drawings. Note that the drawings may be relevant to any embodiment.

[0060] The weight averaging device 1' in this embodiment has a configuration for performing inference using the trained weight selection strategy output by the weight averaging device 1 in embodiment 1. Therefore, in the following description, it is assumed that the machine learning model, weight pool 31, evaluation index, and accuracy evaluation method used in the weight averaging device 1' are the same as those used in the weight averaging device 1 in embodiment 1. Furthermore, in the following description, a description of the same configuration as the weight averaging device 1 according to embodiment 1 described above will be omitted, and differences will be mainly described.

[0061] As shown in Fig. 5, the weight averaging device 1' according to this embodiment differs from the weight averaging device 1 according to the first embodiment in that it does not include the composite reward calculation unit 17, the profit calculation unit 19, the weight selection and update unit 20, and the policy output unit 21, but instead includes a new average weight output unit 22. The average weight output unit 22 outputs the final average weight after the update is complete. The new average weight output unit 22 is realized by a calculation unit included in the weight averaging device 1' executing a program. The operation of the weight averaging device 1' configured as described above will now be described with reference to Fig. 6.

[0062] First, in the initialization process S21, the average weight, average weight state, and importance for each evaluation index are initialized. The average weight and average weight state are initialized with the same values ​​as those used for initialization during learning. Unlike during learning, the importance is initialized by the user of the weight averaging device 1' specifying an importance value for each index.

[0063] Next, in a weight selection process S22, the average weight state and the evaluation importance are input to the weight selection strategy in the weight selection unit 13 via the average weight state input unit 11 and the evaluation importance input unit 12. The weight selection unit 13 selects and extracts an optimal weight from the weight pool 31 using the learned weight selection strategy.

[0064] Next, in the weighted average process S23, the weighted average unit 15 performs an arithmetic average of the weight extracted from the weight pool 31 and the current average weight to obtain a new average weight.

[0065] Next, in the accuracy evaluation process S24, the accuracy evaluation unit 16 receives the new average weights output in the weighted average process S23, and performs accuracy evaluation for each accuracy evaluation index using the received average weights.

[0066] Next, in the average weight updating process S25, the current average weight and average weight state are overwritten and updated with the average weight and average weight state calculated in the weighted average process S23.

[0067] Next, in conditional branch S26, it is determined whether the average weight update is complete. The specific criteria for this determination are not limited, and the process may end, for example, when the average weight has been updated a predetermined number of times, when the change in accuracy is equal to or less than a predetermined threshold, or when an action to end the update is selected as an action in the weight selection strategy. If the update end condition is met (Yes in S26), the process proceeds to average weight output process S27. If the end condition is not met (No in S26), the process returns to the step immediately before weight selection process S22.

[0068] Subsequently, in an average weight output process S27, the updated average weight is output.

[0069] In this way, the weight averaging device 1' outputs average weights that Pareto-optimize the accuracy of each index for a given single machine learning model, its weight set, and evaluation index, under the constraints of importance set by the user, and then the operation ends.

[0070] Next, the effects of the weighted averaging device 1′ according to the second embodiment will be described. As described above, in the weighted averaging device 1′, the user predefines the importance of each evaluation index. Therefore, the user can control the balance of accuracy between indexes in the final output average weight. Prior art lacks a mechanism for controlling the balance of accuracy between indexes, which can result in an improvement in the accuracy of only one index while significantly degrading the accuracy of other indexes. In contrast, in the present embodiment, unless the user implements a unique setting (e.g., increasing the importance of only a specific index and setting the importance of other indexes to 0), the accuracy of all indexes is improved while maintaining a balance. This is because the weight selection strategy used in the present embodiment is trained to select weights that yield a high composite reward. However, in a situation where the accuracy of only some indexes improves while the accuracy of others significantly degrades, the composite reward value does not increase (i.e., the improvement and degradation in accuracy cancel each other out in the sum), and therefore the average weight is not updated in that direction. As described above, the user can control the balance of accuracy between multiple indexes, thereby resolving the aforementioned problem.

[0071] In this embodiment, when it is necessary to change the importance for each evaluation index, there is no need to retrain the weight selection policy. This is because the weight selection policy uses the importance initialized with a random value during training as input, and if it has been sufficiently trained, it can handle any value set for importance during inference. In this way, by simply changing the importance value input to the weight averaging device 1', it is possible to flexibly switch the average weight to be output without changing any other configuration.

[0072] As described above, when there are multiple indicators, the user can uniquely specify the desired average weight from among the countless Pareto-optimal average weights through the importance. The weight averaging device 1' synthesizes and outputs the average weight desired by the user from the weight pool 31. Even if the average weight desired by the user changes (i.e., even if the importance of each indicator changes), the weight averaging device 1' can quickly adapt to the changed conditions and output the expected average weight.

[0073] Third Embodiment Next, a third embodiment of the present disclosure will be described with reference to the drawings. This embodiment shows an outline of the configuration of the weighted averaging device described in the above-mentioned embodiment. Note that Figures 7 and 8 are diagrams for explaining the configuration, and these drawings may be relevant to any of the embodiments.

[0074] First, the hardware configuration of the information processing device 100 will be described with reference to Fig. 7. The information processing device 100 is configured as a general information processing device, and is equipped with the following hardware configuration, for example: CPU (Central Processing Unit) 101 (arithmetic unit); ROM (Read Only Memory) 102 (storage device); RAM (Random Access Memory) 103 (storage device); programs 104 loaded into RAM 103; storage device 105 storing programs 104; drive device 106 for reading and writing data from and to a storage medium 110 external to the information processing device; communication interface 107 for connecting to a communication network 111 external to the information processing device; input / output interface 108 for inputting and outputting data; and bus 109 for connecting the various components.

[0075] 7 shows an example of the hardware configuration of the information processing device 100, and the hardware configuration of the information processing device is not limited to the above-described case. For example, the information processing device may be configured with only a part of the above-described configuration, such as excluding the drive device 106. Furthermore, the information processing device may use a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), an MPU (Micro Processing Unit), an FPU (Floating Point Number Processing Unit), a PPU (Physics Processing Unit), a TPU (Tensor Processing Unit), a quantum processor, a microcontroller, or a combination thereof, instead of the above-described CPU.

[0076] The information processing device 100 can be equipped with the weight selection unit 121, weight synthesis unit 122, evaluation unit 123, and update unit 124 shown in FIG. 8 by having the CPU 101 acquire and execute the program group 104. The program group 104 is stored in advance in the storage device 105 or the ROM 102, for example, and is loaded into the RAM 103 and executed by the CPU 101 as needed. The program group 104 may be supplied to the CPU 101 via the communication network 111, or may be stored in advance in the storage medium 110, and the drive device 106 may read and supply the program to the CPU 101. However, the weight selection unit 121, weight synthesis unit 122, evaluation unit 123, and update unit 124 described above may be constructed using dedicated electronic circuits for realizing such means.

[0077] The weight selection unit 121 uses a selection model generated by machine learning to select one weight from among multiple weights that can be set for a machine learning model obtained by training the machine learning model under different conditions. The weight combination unit 122 generates a composite weight, which is a new weight obtained by combining multiple weights including the selected weight. The evaluation unit 123 evaluates the performance of the machine learning model to which the composite weight is set, using at least one or more predetermined indicators. The update unit 124 updates the selection model through machine learning based on the evaluation.

[0078] With the above-described configuration, the present disclosure can select weights that improve the performance of a machine learning model and set composite weights including such weights in the machine learning model, thereby further improving the performance of the machine learning model.

[0079] At least one or more of the functions of the weight selection unit 121, weight synthesis unit 122, evaluation unit 123, and update unit 124 described above may be executed by an information processing device installed and connected anywhere on the network, that is, they may be executed by so-called cloud computing.

[0080] The above-described program can be stored and supplied to a computer using various types of non-transitory computer-readable media. Non-transitory computer-readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-RWs, and semiconductor memories (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs (Random Access Memory)). The program can also be supplied to a computer by various types of transitory computer-readable media. Examples of transitory computer-readable media include electrical signals, optical signals, and electromagnetic waves. The transitory computer-readable media can be supplied to a computer via wired communication paths such as electric wires and optical fibers, or via wireless communication paths.

[0081] Although the present disclosure has been described above with reference to the above-described embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each of the above-described embodiments can be combined with other embodiments as appropriate.

[0082] <Supplementary Notes> Some or all of the above embodiments may be described as in the following supplementary notes. Below, an outline of the configurations of an information processing device, an information processing method, and a program according to the present disclosure will be described. However, the present disclosure is not limited to the following configurations. (Supplementary Note 1) An information processing device comprising: a weight selection unit that selects one weight using a selection model generated by machine learning from among a plurality of weights that can be set for a machine learning model obtained by training the machine learning model under different conditions; a weight combination unit that generates a composite weight that is a new weight by combining a plurality of weights including the selected weight; an evaluation unit that evaluates performance of the machine learning model to which the composite weight is set for at least one or more predetermined indicators; and an update unit that updates the selection model through machine learning based on the evaluation. (Supplementary Note 2) The information processing device according to Supplementary Note 1, wherein the evaluation unit sets a reward for an action of selecting the weight using the selection model based on the evaluation of the indicator of the machine learning model to which the composite weight that includes the selected weight is set, and the update unit updates the selection model through machine learning using the reward. (Supplementary Note 3) The information processing device according to Supplementary Note 2, wherein the evaluation unit sets the reward based on the evaluation for each of the multiple indicators of the machine learning model to which the composite weight is set, for the behavior of selecting the weight using the selection model, and the update unit updates the selection model by machine learning using the reward. (Supplementary Note 4) The information processing device according to Supplementary Note 3, wherein the evaluation unit sets the reward to a higher value as the evaluation for each of the multiple indicators of the machine learning model to which the composite weight is set is higher, and the update unit updates the selection model by machine learning so that the reward becomes higher.(Supplementary Note 5) The information processing device according to Supplementary Note 1, wherein the evaluation unit sets a reward for an action of selecting the weight using the selection model based on the evaluation for the index of the machine learning model to which the composite weight combined including the selected weight is set and the importance set for the index, and the update unit updates the selection model through machine learning using the reward. (Supplementary Note 6) The information processing device according to Supplementary Note 5, wherein the evaluation unit sets a reward for an action of selecting the weight using the selection model based on the evaluation for each of the multiple indexes of the machine learning model to which the composite weight is set and the importance set for each of the index, and the update unit updates the selection model through machine learning using the reward. (Supplementary Note 7) The information processing device according to Supplementary Note 6, wherein the evaluation unit sets the reward so that it becomes higher the higher the evaluation for each of the multiple indicators of the machine learning model to which the composite weight is set and the higher the importance, and the update unit updates the selection model through machine learning so as to increase the reward. (Supplementary Note 8) The information processing device according to Supplementary Note 1, wherein the evaluation unit sets the reward based on multiple evaluations of the machine learning model to which the composite weight, generated each time the weight is selected, is set, and the update unit updates the selection model through machine learning using the reward. (Supplementary Note 9) An information processing method comprising: selecting one weight from multiple weights that can be set in the machine learning model, obtained by training the machine learning model under different conditions, using a selection model generated by machine learning; generating a composite weight that is a new weight by combining the multiple weights including the selected weight; evaluating performance of the machine learning model to which the composite weight is set for at least one or more indicators that are set in advance; and updating the selection model through machine learning based on the evaluation.(Supplementary Note 10) The information processing method according to Supplementary Note 9, comprising: setting a reward for an action of selecting the weight using the selection model based on the evaluation for the index of the machine learning model to which the composite weight combined including the selected weight is set; and updating the selection model through machine learning using the reward. (Supplementary Note 10.1) The information processing method according to Supplementary Note 10, comprising: setting, for an action of selecting the weight using the selection model, the reward based on the evaluation for each of the multiple indexes of the machine learning model to which the composite weight is set, and updating the selection model through machine learning using the reward. (Supplementary Note 10.2) The information processing method according to Supplementary Note 10.1, comprising: an information processing device setting the reward to a higher value as the evaluation for each of the multiple indexes of the machine learning model to which the composite weight is set is higher; and updating the selection model through machine learning so that the reward is higher. (Supplementary Note 11) The information processing method according to Supplementary Note 9, wherein, for an action of selecting the weight using the selection model, a reward is set based on the evaluation for the indicator of the machine learning model to which the composite weight combined including the selected weight is set and the importance set for the indicator, and the selection model is updated through machine learning using the reward. (Supplementary Note 12) The information processing method according to Supplementary Note 11, wherein, for an action of selecting the weight using the selection model, a reward is set based on the evaluation for each of the multiple indicators of the machine learning model to which the composite weight is set and the importance set for each of the indicators, and the selection model is updated through machine learning using the reward. (Supplementary Note 13) The information processing method according to Supplementary Note 12, wherein, the higher the evaluation for each of the multiple indicators of the machine learning model to which the composite weight is set and the higher the importance, the higher the reward is set, and the selection model is updated through machine learning so that the reward becomes higher.(Supplementary Note 14) The information processing method according to Supplementary Note 9, comprising: setting a reward based on the plurality of evaluations of the machine learning model to which the composite weight generated each time the weight is selected is respectively set; and updating the selection model through machine learning using the reward. (Supplementary Note 15) A computer-readable storage medium storing a program that causes a computer to execute processes of: selecting one weight using a selection model generated by machine learning from a plurality of weights that can be set in the machine learning model, obtained by training the machine learning model under different conditions; generating a composite weight that is a new weight by combining a plurality of the weights including the selected weight; evaluating the performance of the machine learning model to which the composite weight is set for at least one or more predetermined indicators; and updating the selection model through machine learning based on the evaluation.

[0083] 1, 1' Weight averaging device 11 Average weight state input unit 12 Evaluation importance input unit 13 Weight selection unit 14 Weight input unit 15 Weight averaging unit 16 Accuracy evaluation unit 17 Composite reward calculation unit 18 Average weight update unit 19 Profit calculation unit 20 Weight selection update unit 21 Policy output unit 22 Average weight output unit 31 Weight pool 32 Composite reward pool 100 Information processing device 101 CPU 102 ROM 103 RAM 104 Program group 105 Storage device 106 Drive device 107 Communication interface 108 Input / output interface 109 Bus 110 Storage medium 111 Communication network 121 Weight selection unit 122 Weight synthesis unit 123 Evaluation unit 124 Update unit

Claims

1. An information processing device comprising: a weight selection unit that uses a selection model generated by machine learning to select one weight from multiple weights that can be set for a machine learning model obtained by training the machine learning model under different conditions; a weight synthesis unit that generates a composite weight, which is a new weight obtained by synthesizing multiple weights including the selected weight; an evaluation unit that evaluates the performance of the machine learning model to which the composite weight has been set, using at least one or more predetermined indicators; and an update unit that performs machine learning to update the selection model based on the evaluation.

2. An information processing device according to claim 1, wherein the evaluation unit sets a reward based on the evaluation of the index of the machine learning model to which the composite weight including the selected weight is set for the behavior of selecting the weight using the selection model, and the update unit uses the reward to machine learn and update the selection model.

3. An information processing device according to claim 2, wherein the evaluation unit sets the reward based on the evaluation of each of the multiple indicators of the machine learning model to which the composite weight is set for the behavior of selecting the weight using the selection model, and the update unit uses the reward to machine learn and update the selection model.

4. An information processing device according to claim 3, wherein the evaluation unit sets the reward such that the higher the evaluation for each of the multiple indicators of the machine learning model to which the composite weight is set, the higher the reward value; and the update unit performs machine learning to update the selection model so that the reward becomes higher.

5. An information processing device according to claim 1, wherein the evaluation unit sets a reward for the behavior of selecting the weight using the selection model based on the evaluation of the index of the machine learning model to which the composite weight including the selected weight is set, and the importance set for the index, and the update unit uses the reward to machine learn and update the selection model.

6. An information processing device according to claim 5, wherein the evaluation unit sets a reward for the behavior of selecting the weight using the selection model based on the evaluation of each of the multiple indicators of the machine learning model to which the composite weight is set and the importance set for each of the indicators, and the update unit uses the reward to machine learn and update the selection model.

7. An information processing device according to claim 6, wherein the evaluation unit sets the reward to a higher value the higher the evaluation and the importance of each of the multiple indicators of the machine learning model to which the composite weights are set, and the update unit performs machine learning to update the selection model so that the reward becomes higher.

8. An information processing device according to claim 1, wherein the evaluation unit sets a reward based on a plurality of evaluations of the machine learning model to which the composite weights generated each time the weights are selected are respectively set, and the update unit uses the rewards to machine learn and update the selection model.

9. An information processing method comprising: selecting one weight using a selection model generated by machine learning from among multiple weights that can be set for a machine learning model obtained by training the machine learning model under different conditions; generating a composite weight that is a new weight by combining multiple weights including the selected weight; evaluating the performance of the machine learning model to which the composite weight has been set in terms of at least one or more predetermined indicators; and updating the selection model through machine learning based on the evaluation.

10. An information processing method according to claim 9, comprising: setting a reward based on the evaluation of the index of the machine learning model to which the composite weight, which is a composite weight including the selected weight, is set for the action of selecting the weight using the selection model; and using the reward to machine-learn and update the selection model.

11. An information processing method according to claim 9, comprising: setting a reward for the behavior of selecting the weight using the selection model based on the evaluation of the index of the machine learning model to which the composite weight including the selected weight is set, and the importance set for the index; and updating the selection model through machine learning using the reward.

12. An information processing method according to claim 11, comprising: setting a reward for the behavior of selecting the weight using the selection model based on the evaluation of each of the multiple indicators of the machine learning model to which the composite weight is set and the importance set for each of the indicators; and using the reward to machine learn and update the selection model.

13. An information processing method according to claim 12, wherein the reward is set to a higher value the higher the evaluation and the importance of each of the multiple indicators of the machine learning model to which the composite weights are set, and the selection model is machine-learned and updated so that the reward becomes higher.

14. An information processing method according to claim 9, further comprising: setting a reward based on the evaluations of the machine learning model to which the composite weight generated each time the weight is selected is respectively set; and using the rewards to perform machine learning to update the selection model.

15. A computer-readable storage medium storing a program that causes a computer to execute the following processes: selecting one weight using a selection model generated by machine learning from multiple weights that can be set for a machine learning model obtained by training the machine learning model under different conditions; generating a composite weight that is a new weight by combining multiple weights including the selected weight; evaluating the performance of the machine learning model to which the composite weight has been set in terms of at least one or more predetermined indicators; and updating the selection model through machine learning based on the evaluation.

Images