Electronic apparatus for cost-sensitive multi-fidelity bayesian optimization using training curve extrapolation transfer

Abstract

The present invention relates to an electronic apparatus which may dynamically select a hyperparameter configuration via which the highest utility can be expected, by: for each of a plurality of virtual training conditions that assumes that a machine learning model is trained for each of a plurality of candidate epochs by using each of a plurality of hyperparameter configurations, calculating each predicted utility value for improvement in an actual performance value of the machine learning model by considering a training cost, and then performing an optimization procedure for selecting an optimal hyperparameter configuration from among the plurality of hyperparameter configurations on the basis of the predicted utility value and selecting an optimal epoch from among the plurality of candidate epochs.

Description

Electronic Devices for Cost-Sensitive Multi-Fidelity Bayesian Optimization Using Learning Curve Extrapolation Transfer

[0001] The present disclosure relates to an electronic device for cost-sensitive multi-fidelity Bayesian optimization utilizing learning curve extrapolation transfer, and more particularly, to an electronic device capable of dynamically selecting a hyperparameter configuration from which the highest utility can be expected by calculating a predicted utility value for an actual performance value improvement of a machine learning model by considering a learning cost for each of a plurality of virtual learning conditions in which a machine learning model is trained for each of a plurality of candidate epochs using each of a plurality of hyperparameter configurations, and then performing an optimization procedure for selecting an optimal hyperparameter configuration from among a plurality of hyperparameter configurations based on the predicted utility value and selecting an optimal epoch from among a plurality of candidate epochs.

[0002]

[0003] Machine learning (ML) and deep learning (DL) are increasingly gaining attention as core technologies in artificial intelligence. Recent CNNs can exceed 1,000 layers and contain over 100 million parameters.

[0004] As the number of parameters and layers continues to increase, training time, inference time, and model size increase rapidly, which can lead to longer design times, higher costs, and memory issues when resources are limited.

[0005] Automated ML (AutoML) has emerged to support efficient model building and hyperparameter tuning. Commonly used hyperparameter optimization methods include grid search (GS), random search (RS), Bayesian optimization (BO), gradient-based optimization (GO), evolutionary optimization, and population-based optimization.

[0006] However, conventional hyperparameter optimization methods suffer from the following problems: Grid search suffers from the curse of dimensionality, as the number of required function evaluations increases exponentially with the dimensionality of the configuration space.

[0007] Another problem with grid search is that increasing the resolution of the discretization significantly increases the number of function evaluations required.

[0008] Additionally, random search selects the most appropriate point from a randomly drawn set of points. While empirical and theoretical evidence suggests that random search is more practical and efficient than grid search, random search does not guarantee optimal search.

[0009] This means that a longer search increases the likelihood of finding optimal hyperparameters, but also consumes more resources.

[0010] Also, one of the most important things to keep in mind about Bayesian optimization is that once it finds a local maximum or minimum, it is easy to get stuck in a local maximum because it continuously samples from that area.

[0011] Bayesian optimization requires a lot of resources and time.

[0012] For high-dimensional, nonconvex functions with unknown smoothness and noise, Bayesian algorithms often have difficulty fitting and optimizing them, and Bayesian algorithms typically have very strong hypothesis conditions, which are generally difficult to meet.

[0013] There are also papers that show that Bayesian algorithms are not significantly better than random search.

[0014] Hyperparameter space is typically comprised of many discrete decisions, making it neither continuous nor differentiable. Therefore, gradient descent is typically not feasible in hyperparameter space.

[0015]

[0016] The present disclosure aims to solve all of the problems of the prior art described above.

[0017] In addition, the present disclosure provides an electronic device capable of dynamically selecting a hyperparameter configuration from which the highest utility can be expected by calculating a predicted utility value for an improvement in the actual performance value of a machine learning model by considering a learning cost for each of a plurality of virtual learning conditions in which a machine learning model is trained for each of a plurality of candidate epochs using each of a plurality of hyperparameter configurations, and then performing an optimization procedure for selecting an optimal hyperparameter configuration from among a plurality of hyperparameter configurations based on the predicted utility value and selecting an optimal epoch from among a plurality of candidate epochs.

[0018] The purposes of the present disclosure are not limited to those mentioned above, and other purposes and advantages of the present disclosure not mentioned above can be understood through the following description and will be more clearly understood through the embodiments of the present disclosure. Furthermore, it will be readily apparent that the purposes and advantages of the present disclosure can be realized by the means and combinations thereof set forth in the claims.

[0019]

[0020] An electronic device for cost-sensitive multi-fidelity Bayesian optimization utilizing learning curve extrapolation transfer according to the present disclosure comprises: a memory in which a machine learning model is stored; and a processor connected to the memory; wherein the processor calculates a predicted utility value for an improvement in an actual performance value of the machine learning model by considering a learning cost for each of a plurality of virtual learning conditions in which the machine learning model is trained for each of a plurality of candidate epochs using each of a plurality of hyperparameter configurations, and then performs an optimization procedure in which an optimal hyperparameter configuration is selected from among the plurality of hyperparameter configurations based on the predicted utility value, and an optimal epoch is selected from among the plurality of candidate epochs.

[0021] The above processor can output the predicted performance value of the machine learning model for each of the plurality of virtual learning conditions from the learning curve prediction model.

[0022] The processor can input the predicted performance value into an acquisition function that outputs the predicted utility value, and output the predicted utility value for each of the plurality of virtual learning conditions.

[0023] The processor may select a hyperparameter configuration corresponding to a virtual learning condition with a maximum predicted utility value among the plurality of virtual learning conditions as the optimal hyperparameter configuration, and select the candidate epoch as the optimal epoch.

[0024] The processor may calculate an actual performance value of the machine learning model trained for the optimal number of epochs using the optimal hyperparameter configuration, and input the actual performance value into a utility function that calculates an actual utility value for the actual performance value of the machine learning model by considering the learning cost, thereby outputting the actual utility value of the machine learning model trained for the optimal number of epochs using the optimal hyperparameter configuration.

[0025] The processor may set the largest actual utility value among the actual utility values ​​according to the previously performed optimization procedure as the maximum actual utility value, set the smallest actual utility value among the actual utility values ​​according to the previously performed optimization procedure as the minimum actual utility value, and input the largest actual performance value among the actual performance values ​​according to the previously performed optimization procedure into the utility function and set the output actual utility value as the previous actual utility value.

[0026] The above processor can calculate a utility residual rate as a ratio of a difference between a first utility value, which is a difference between the maximum actual utility value and the minimum actual utility value, and a difference between a second utility value, which is a difference between the maximum actual utility value and the previous actual utility value.

[0027] The above processor repeatedly performs the above optimization procedure, but if the utility residual rate exceeds the reference rate, it is determined that the utility residual rate satisfies the optimization termination condition, and the above optimization procedure may not be repeated.

[0028] The above processor repeatedly performs the above optimization procedure, but if the utility residual rate is lower than the reference rate, it is determined that the utility residual rate does not satisfy the optimization termination condition, and the above optimization procedure can be repeated.

[0029] According to the present disclosure, for each of a plurality of virtual learning conditions in which a machine learning model is trained for each of a plurality of candidate epochs using each of a plurality of hyperparameter configurations, a predicted utility value for an improvement in the actual performance value of the machine learning model is calculated by considering the learning cost, and then an optimization procedure is performed in which an optimal hyperparameter configuration is selected from among a plurality of hyperparameter configurations based on the predicted utility value and an optimal epoch is selected from among a plurality of candidate epochs, thereby dynamically selecting a hyperparameter configuration from which the highest utility can be expected.

[0030] FIG. 1 is a block diagram conceptually illustrating components of an electronic device according to one embodiment of the present disclosure.

[0031] FIG. 2 is a diagram illustrating a memory structure of an electronic device according to an embodiment of the present disclosure.

[0032] FIG. 3 is a diagram for explaining a process in which an electronic device according to an embodiment of the present disclosure outputs a predicted performance value using a learning curve prediction model.

[0033] FIG. 4 is a diagram illustrating an acquisition function used by an electronic device according to an embodiment of the present disclosure.

[0034] FIG. 5 is a diagram illustrating a utility function used by an electronic device according to an embodiment of the present disclosure.

[0035] FIG. 6 is a diagram illustrating a process of training a machine learning model used by an electronic device according to an embodiment of the present disclosure.

[0036] The present embodiments may be modified and have various embodiments. Therefore, specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the scope to specific embodiments, but should be understood to encompass various modifications, equivalents, and / or alternatives of the embodiments of the present disclosure. In connection with the description of the drawings, similar reference numerals may be used for similar components.

[0037] In describing the present disclosure, if it is determined that a specific description of a related known function or configuration may unnecessarily obscure the gist of the present disclosure, a detailed description thereof will be omitted.

[0038] Additionally, the following embodiments may be modified in various other forms, and the scope of the technical concepts of the present disclosure is not limited to the following embodiments. Rather, these embodiments are provided to further faithfully and completely convey the technical concepts of the present disclosure to those skilled in the art.

[0039] The terminology used in this disclosure is for the purpose of describing specific embodiments only and is not intended to limit the scope of the rights. Singular expressions include plural expressions unless the context clearly dictates otherwise.

[0040] In this disclosure, expressions such as “has,” “can have,” “includes,” or “may include” indicate the presence of a corresponding feature (e.g., a component such as a number, function, operation, or part), and do not exclude the presence of additional features.

[0041] In this disclosure, expressions such as “A or B,” “at least one of A and / or B,” or “one or more of A or / and B” can include all possible combinations of the listed items. For example, “A or B,” “at least one of A and B,” or “at least one of A or B” can all refer to (1) including at least one A, (2) including at least one B, or (3) including both at least one A and at least one B.

[0042] The expressions “first,” “second,” “first,” or “second,” etc., used in this disclosure can describe various components, regardless of order and / or importance, and are only used to distinguish one component from another, but do not limit the components.

[0043] When it is said that a component (e.g., a first component) is “(operatively or communicatively) coupled with / to” or “connected to” another component (e.g., a second component), it should be understood that the component may be directly coupled to the other component, or may be connected through another component (e.g., a third component).

[0044] On the other hand, when it is said that a component (e.g., a first component) is "directly connected" or "directly connected" to another component (e.g., a second component), it can be understood that no other component (e.g., a third component) exists between the component and the other component.

[0045] The expression "configured to" used in the present disclosure may be used interchangeably with, for example, "suitable for," "having the capacity to," "designed to," "adapted to," "made to," or "capable of." The term "configured to" may not necessarily mean only "specifically designed to" in terms of hardware.

[0046] In the embodiment, a 'module' or 'part' performs at least one function or operation and may be implemented as hardware or software or as a combination of hardware and software.

[0047] Meanwhile, the various elements and areas in the drawings are schematically drawn. Therefore, the technical concept of the present invention is not limited by the relative sizes or spacing depicted in the attached drawings.

[0048] Hereinafter, with reference to the attached drawings, embodiments according to the present disclosure will be described in detail so that a person having ordinary knowledge in the technical field to which the present disclosure pertains can easily implement the present disclosure.

[0049] FIG. 1 is a block diagram conceptually illustrating components of an electronic device according to an embodiment of the present disclosure, and FIG. 2 is a diagram illustrating a memory structure of an electronic device according to an embodiment of the present disclosure.

[0050] Referring to FIGS. 1 and 2, an electronic device (100) according to an embodiment of the present disclosure may be a device capable of performing an operation to obtain an operation result by performing an operation based on acquired data.

[0051] The electronic device (100) may be, for example, a computing device, a calculation device, a desktop PC, etc.

[0052] Additionally, the electronic device (100) may be a portable computing device and may include, but is not limited to, at least one of a smartphone, a tablet personal computer, a laptop personal computer, a netbook computer, a mobile device, and a wearable device.

[0053] In addition, the electronic device (100) may be a server device, for example, a computer that provides services to clients via a network. The server may be an FTP server, a web server, a database server, or a cloud-based server, and may be built with an operating system such as Linux.

[0054] The above-described electronic device (100) is not necessarily implemented as a single device, but may be implemented as multiple different devices each performing a function or operation.

[0055] An electronic device (100) according to one embodiment of the present disclosure may include a memory (110) and a processor (120).

[0056] The memory (110) temporarily or non-temporarily stores various programs or data, and transmits the stored information to the processor (120) upon a call from the processor (120). In addition, the memory (110) can store various information necessary for operations, processing, or control operations of the processor (120) in an electronic format.

[0057] The memory (110) may be implemented using a semiconductor storage medium such as ROM and / or RAM. The ROM may include, for example, conventional ROM, EPROM, EEPROM, and / or MASK-ROM. The RAM may include, for example, DRAM and / or SRAM. The memory may be implemented using at least one storage medium capable of permanently or semi-permanently storing data, such as a flash memory device, a Secure Digital (SD) card, a solid state drive (SSD), a hard disk drive (HDD), an optical recording medium such as a magnetic drum, a compact disc (CD), a DVD, or a laser disc, a magnetic tape, a magneto-optical disc, and / or a floppy disk.

[0058] For example, the memory (110) can store a machine learning model (ML), a learning curve prediction model (LCM), an acquisition function, a utility function, and an algorithm.

[0059] The processor (120) controls the overall operation of the electronic device (100). Specifically, the processor (120) is connected to the configuration of the electronic device (100) including the memory (110) as described above, and can control the overall operation of the electronic device (100) by executing at least one instruction stored in the memory (110) as described above. In particular, the processor (120) may be implemented as one processor or may be implemented as multiple processors.

[0060] The processor (120) may be implemented in various ways. For example, one or more processors may include one or more of a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), an Accelerated Processing Unit (APU), a Many Integrated Core (MIC), a Digital Signal Processor (DSP), a Neural Processing Unit (NPU), a hardware accelerator, or a machine learning accelerator. The one or more processors may control one or any combination of other components of the electronic device and perform operations related to communication or data processing. The one or more processors may execute one or more programs or instructions stored in a memory. For example, the one or more processors may perform a method according to an embodiment of the present disclosure by executing one or more instructions stored in a memory.

[0061] When a method according to an embodiment of the present disclosure includes multiple operations, the multiple operations may be performed by one processor or by multiple processors. For example, when a first operation, a second operation, and a third operation are performed by a method according to an embodiment, the first operation, the second operation, and the third operation may all be performed by the first processor, or the first operation and the second operation may be performed by the first processor (e.g., a general-purpose processor) and the third operation may be performed by the second processor (e.g., an artificial intelligence-specific processor).

[0062] One or more processors may be implemented as a single core processor including one core, or may be implemented as one or more multicore processors including multiple cores (e.g., homogeneous multicores or heterogeneous multicores). When one or more processors are implemented as a multicore processor, each of the multiple cores included in the multicore processor may include internal processor memory, such as on-chip memory, and a common cache shared by the multiple cores may be included in the multicore processor. In addition, each of the multiple cores (or some of the multiple cores) included in the multicore processor may independently read and execute a program instruction for implementing a method according to an embodiment of the present disclosure, or all (or some) of the multiple cores may be linked to read and execute a program instruction for implementing a method according to an embodiment of the present disclosure.

[0063] When a method according to an embodiment of the present disclosure includes a plurality of operations, the plurality of operations may be performed by one core among the plurality of cores included in a multi-core processor, or may be performed by the plurality of cores. For example, when a first operation, a second operation, and a third operation are performed by a method according to an embodiment, the first operation, the second operation, and the third operation may all be performed by a first core included in the multi-core processor, or the first operation and the second operation may be performed by a first core included in the multi-core processor, and the third operation may be performed by a second core included in the multi-core processor.

[0064] In embodiments of the present disclosure, the processor (120) may mean a system on a chip (SoC) in which one or more processors and other electronic components are integrated, a single-core processor, a multi-core processor, or a core included in a single-core processor or a multi-core processor, wherein the core may be implemented as a CPU, a GPU, an APU, a MIC, a DSP, an NPU, a hardware accelerator, or a machine learning accelerator, but embodiments of the present disclosure are not limited thereto.

[0065] The memory (110) can store an artificial intelligence model, output data obtained based on input data input to the artificial intelligence model, usage information obtained based on the output data, and reconstruction information for reconstructing the artificial intelligence model obtained based on the usage information.

[0066] The processor (20) can perform an operation using an output value using an artificial intelligence model.

[0067] AI models can be built based on factors such as the cognitive model's application area, the learning objective, or the device's computing power. Furthermore, AI models can be based on neural networks, for example.

[0068] An AI model can be designed to simulate the structure of the human brain on a computer. The AI ​​model can include multiple network nodes with weights, simulating the neurons of a human neural network. The multiple network nodes can form connections to simulate the synaptic activity of neurons exchanging signals through synapses.

[0069] Machine learning models may include, for example, neural network models or deep learning models developed from neural network models. In deep learning models, multiple network nodes are located at different depths (or layers) and can exchange data based on convolutional connections. For example, models such as deep neural networks (DNNs), recurrent neural networks (RNNs), and bidirectional recurrent deep neural networks (BRDNNs) can be used as artificial intelligence models, but are not limited thereto.

[0070] A deep learning model may include multiple artificial neural network layers. The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the foregoing examples, but is not limited thereto. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may also include a software structure.

[0071] Deep learning models may include learning algorithms. Examples include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning. Artificial intelligence models may include multiple artificial neural network layers.

[0072] FIG. 3 is a diagram for explaining a process in which an electronic device according to an embodiment of the present disclosure outputs a predicted performance value using a learning curve prediction model.

[0073] Referring to FIG. 3, when the optimization procedure starts, the processor (120) can output a predicted performance value (EP) of the machine learning model (ML) from the learning curve prediction model (LCM) for each of a plurality of virtual learning conditions that assume that the machine learning model (ML) is trained for each of a plurality of candidate epochs (E) using each of a plurality of hyperparameter configurations (HP).

[0074] For example, the plurality of virtual learning conditions may be first to ninth virtual learning conditions when there are three hyperparameter configurations (HP) and the maximum epoch (E) is 3, and the hyperparameter of the first virtual learning condition may be "x-1" and the epoch may be "1", the hyperparameter of the second virtual learning condition may be "x-1" and the epoch may be "2", the hyperparameter of the third virtual learning condition may be "x-1" and the epoch may be "3", the hyperparameter of the fourth virtual learning condition may be "x-2" and the epoch may be "1", the hyperparameter of the fifth virtual learning condition may be "x-2" and the epoch may be "2", the hyperparameter of the sixth virtual learning condition may be "x-2" and the epoch may be "3", and the hyperparameter of the seventh virtual learning condition may be "x-3". The epoch can be "1", the hyperparameter of the 8th virtual learning condition can be "x-3", the epoch can be "2", the hyperparameter of the 9th virtual learning condition can be "x-3", the epoch can be "3".

[0075] At this time, the learning curve prediction model (LCM) can be pre-trained to predict the predicted performance value (EP) according to the hyperparameter configuration (HP) and the epoch (E) using the learning curve data set as training data. To this end, the processor (120) can train the learning curve prediction model (LCM) by configuring the actual performance value (RP) according to the hyperparameter configuration (HP) and the epoch (E) as a learning curve data set.

[0076] Accordingly, the processor (120) can output the predicted performance value (EP) of the machine learning model (ML) for each of a plurality of virtual learning conditions from the learning curve prediction model (LCM).

[0077] FIG. 4 is a diagram illustrating an acquisition function used by an electronic device according to an embodiment of the present disclosure, and FIG. 5 is a diagram illustrating a utility function used by an electronic device according to an embodiment of the present disclosure.

[0078] Referring to FIGS. 4 and 5, the processor (120) can calculate a predicted utility value for an improvement in the actual performance value of a machine learning model (ML) by considering the learning cost for each of a plurality of virtual learning conditions.

[0079] To this end, the processor (120) can input the predicted performance value (EP) of each of the plurality of virtual learning conditions into an acquisition function that outputs the predicted utility value, thereby outputting the predicted utility value for each of the plurality of virtual learning conditions.

[0080] Specifically, the acquisition function can be expressed by the following mathematical expression 1.

[0081] <Mathematical Formula 1>

[0082]

[0083] Here, the acquisition function can be quantified by estimating the likelihood that the performance of a future machine learning model (ML) will improve for a specific hyperparameter configuration xn and the resulting expected improvement in the predicted utility value (Expected Improvement (EI).

[0084]

[0085] Thereafter, the processor (120) can select an optimal hyperparameter configuration (HP') from among a plurality of hyperparameter configurations (HP) based on the predicted utility value, and select an optimal epoch (E') from among a plurality of candidate epochs (E).

[0086] Specifically, the processor (120) may select a hyperparameter configuration (HP) corresponding to a virtual learning condition with a maximum predicted utility value among a plurality of virtual learning conditions as an optimal hyperparameter configuration (HP'), and select a candidate epoch (E) as an optimal epoch (E').

[0087] FIG. 6 is a diagram illustrating a process of training a machine learning model used by an electronic device according to an embodiment of the present disclosure.

[0088] Referring to FIG. 6, the processor (120) can train a machine learning model (ML) for an optimal number of epochs (E') using an optimal hyperparameter configuration (HP'), and calculate an actual performance value of the machine learning model (ML).

[0089] At this time, the actual performance value can be calculated as one of various types of indicators that can evaluate the performance of a machine learning model (ML).

[0090] Next, the processor (120) can calculate an actual utility value for the actual performance value of the machine learning model (ML) by considering the learning cost for the machine learning model (ML) trained for the optimal epoch (E') using the optimal hyperparameter configuration (HP').

[0091] To this end, the processor (120) inputs the actual performance value into a utility function that calculates the actual utility value for the actual performance value of the machine learning model (ML) by considering the learning cost, and outputs the actual utility value of the machine learning model (ML) trained for the optimal epoch (E') using the optimal hyperparameter configuration (HP').

[0092] Specifically, the utility function can be expressed by the following mathematical expression 2.

[0093] <Mathematical Formula 2>

[0094]

[0095] Here, b is the number of optimization steps performed, is the actual performance value produced by the corresponding optimization procedure, is the cost disadvantage coefficient.

[0096]

[0097] Next, the processor (120) can set the largest actual utility value among the actual utility values ​​according to the previously performed optimization procedure as the maximum actual utility value.

[0098] Additionally, the processor (120) can set the smallest actual utility value among the actual utility values ​​according to the previously performed optimization procedure as the minimum actual utility value.

[0099] Additionally, the processor (120) can input the largest actual performance value among the actual performance values ​​according to the previously performed optimization procedure into the utility function and set the output actual utility value as the previous actual utility value.

[0100] Thereafter, the processor (120) can calculate the ratio of the difference between the maximum actual utility value and the previous actual utility value, which is the difference between the first utility value difference, which is the difference between the maximum actual utility value and the minimum actual utility value, as the utility residual rate.

[0101] Finally, the processor (120) repeatedly performs the above-described optimization procedure, but if the utility residual rate exceeds the reference rate, it is determined that the utility residual rate satisfies the optimization termination condition and the optimization procedure may not be repeated.

[0102] Conversely, the processor (120) may perform the optimization procedure repeatedly, but if the utility residual rate is lower than the reference rate, it may be determined that the utility residual rate does not satisfy the optimization termination condition and the optimization procedure may be repeated.

[0103] At this time, the processor (120) can calculate and set the reference ratio using the following mathematical expression 3.

[0104] <Mathematical Formula 3>

[0105]

[0106] Here, is the standard ratio, is a control parameter that adjusts the sensitivity of the reference ratio, and is usually a real number between 0 and 1. A larger value means a stricter criterion, and a smaller value induces a faster termination. is defined as a sensitivity control coefficient involved in the calculation of the reference ratio and can generally be set to a real value greater than 0.

[0107]

[0108] Through this, the present disclosure can propose a new cost-sensitive multi-fidelity Bayesian optimization (CMBO) framework, taking into account the fact that the existing hyperparameter optimization (HPO) method fails to solve the problem of inefficient balance between computational resources and performance improvement.

[0109] In particular, by introducing a user-defined utility function, the trade-off between computational cost and performance can be clearly reflected.

[0110] The utility function is designed to have a higher value for lower cost and better performance, so that the hyperparameter configuration that can expect the highest utility can be dynamically selected for each optimization procedure.

[0111] In addition, the present disclosure adopts a freeze-thaw BO structure, which enables freezing or resuming a partially learned model, and can refine the extrapolation technique required to predict the learning curve of each configuration.

[0112] To this end, by utilizing a transfer learning strategy based on prior-fitted networks (PFNs) and data mixup, the accuracy of learning curve prediction can be improved even with a small number of samples.

[0113] According to one embodiment, the method according to the various embodiments disclosed in the present document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an application store (e.g., Play Store™) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product (e.g., a downloadable app) may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0114] Although the preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above, and various modifications may be made by a person having ordinary skill in the art to which the present disclosure pertains without departing from the gist of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical idea or prospect of the present disclosure.

Claims

1. In an electronic device for cost-sensitive multi-fidelity Bayesian optimization utilizing learning curve extrapolation transfer, Memory where machine learning models are stored; and a processor connected to the above memory; The above processor, An electronic device characterized in that, for each of a plurality of virtual learning conditions in which the machine learning model is trained for each of a plurality of candidate epochs using each of a plurality of hyperparameter configurations, a predicted utility value for an improvement in the actual performance value of the machine learning model is calculated by considering the learning cost, and then an optimization procedure is performed in which an optimal hyperparameter configuration is selected from among the plurality of hyperparameter configurations based on the predicted utility value, and an optimal epoch is selected from among the plurality of candidate epochs.

2. In paragraph 1, The above processor An electronic device characterized in that it receives the predicted performance value of the machine learning model for each of the plurality of virtual learning conditions from a learning curve prediction model.

3. In paragraph 2, The above processor An electronic device characterized in that the predicted performance value is input into an acquisition function that outputs the predicted utility value, and the predicted utility value is output for each of the plurality of virtual learning conditions.

4. In paragraph 1, The above processor An electronic device characterized in that, among the plurality of virtual learning conditions, a hyperparameter configuration corresponding to a virtual learning condition with the maximum predicted utility value is selected as the optimal hyperparameter configuration, and the candidate epoch is selected as the optimal epoch.

5. In paragraph 1, The above processor Using the above optimal hyperparameter configuration, the actual performance value of the machine learning model trained for the above optimal epoch is calculated, An electronic device characterized in that the actual performance value is input into a utility function that calculates an actual utility value for the actual performance value of the machine learning model by considering the learning cost, and the actual utility value of the machine learning model learned for the optimal number of epochs is output using the optimal hyperparameter configuration.

6. In paragraph 5, The above processor Among the actual utility values ​​according to the previously performed optimization procedure, the largest actual utility value is set as the maximum actual utility value, Among the actual utility values ​​according to the previously performed optimization procedure, the smallest actual utility value is set as the minimum actual utility value, An electronic device characterized in that the largest actual performance value among the actual performance values ​​according to the previously performed optimization procedure is input into the utility function and the output actual utility value is set as the previous actual utility value.

7. In paragraph 6, The above processor An electronic device characterized in that the ratio of the difference between the maximum actual utility value and the previous actual utility value, which is the difference between the first utility value difference between the maximum actual utility value and the minimum actual utility value, is calculated as a utility residual rate.

8. In paragraph 7, The above processor An electronic device characterized in that the above optimization procedure is performed repeatedly, but if the utility residual rate exceeds a reference rate, the utility residual rate is determined to satisfy the optimization termination condition, and the optimization procedure is not repeated.

9. In paragraph 7, The above processor An electronic device characterized in that the above optimization procedure is performed repeatedly, and if the utility residual rate is lower than the reference rate, the utility residual rate is determined to not satisfy the optimization termination condition, and the optimization procedure is repeated.

Images