Model learning system and method

Abstract

To recover delay of model training and avoid allocation of an excessive amount of resources.SOLUTION: A system acquires, every time a machine learning model is trained, for each of multiple kinds of computing resources, an amount of resources allocated, which is an amount of computing resources allocated for training of the machine learning model, and monitors, for each of the multiple kinds of computing resources, an amount of resources used, which is an amount of computing resources used for training of the machine learning model. The system estimates time required for current training of the machine learning model, and executes time determination to determine whether the estimated time is longer than time required for past training of the machine learning model. If the result of the time determination is true, the system increases the amount of resources allocated in the current training, regarding at least one of the multiple kinds of computing resources.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates generally to training machine learning models. [Background technology]

[0002] In computer systems, there is a service called IaaS (Infrastructure as a Service) that uses virtualization technology to provide digital infrastructure such as virtual servers and networks with hardware resources (CPU (Central Processing Unit) / memory / storage) on demand via the Internet, and it is common for users to be charged based on the amount of resources used and the duration of use.

[0003] Furthermore, in a computer system, the amount of resources can be increased (typically, scaled out and / or scaled up) and / or the amount of resources can be decreased (typically, scaled in and / or scaled down).

[0004] Furthermore, Patent Document 1 describes an information processing device that reduces resource usage and enables rapid switching of virtual machines. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 2020-64567 Summary of the Invention [Problem to be solved by the invention]

[0006] Recently, machine learning, especially deep learning using neural networks, has been widely used as a method for analyzing and predicting various types of data. When using a model obtained through machine learning to predict data, it is necessary to retrain the model using the latest data in order to maintain prediction accuracy.

[0007] As an example, consider the case where it takes a long time to retrain a machine learning model in IaaS. For example, in a machine learning model that requires retraining by inputting new data every week, if the retraining of the model is delayed and takes several days, opportunities to utilize the model will be lost during that time. While it may be possible to avoid delays by allocating a large amount of resources to model retraining in advance, another problem may arise: the allocation of excessive resources.

[0008] Patent Document 1 neither discloses nor suggests such a problem nor a means for solving the problem.

[0009] The present invention has been made in view of the above background, and aims to achieve both recovery from delays in model learning and avoidance of allocation of excessive amounts of resources. [Means for solving the problem]

[0010] A model learning system, each time a machine learning model is trained, acquires a resource allocation, which is the amount of computational resources allocated to training the machine learning model, for each of multiple types of computational resources, and monitors a resource usage, which is the amount of computational resources used for training the machine learning model, for each of the multiple types of computational resources. The model learning system estimates the time required for current training of the machine learning model and performs a time determination to determine whether the estimated time is longer than the time required for previous training of the machine learning model. If the result of the time determination is true, the model learning system increases the resource allocation for current training for at least one of the multiple types of computational resources.

[0011] Other problems and solutions disclosed in the present application will be made clear in the detailed description and drawings. [Effects of the Invention]

[0012] According to the present invention, it is possible to realize both recovery from delays in model learning and avoidance of allocation of excessive amounts of resources. [Brief explanation of the drawings]

[0013] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of a model learning system according to an embodiment of the present invention. [Figure 2] FIG. 10 is a diagram illustrating an example of a learning history table. [Figure 3] FIG. 10 is a diagram illustrating an example of a learning resource table. [Figure 4] FIG. 10 is a diagram illustrating an example of an execution cost table. [Figure 5] 10 is an example of a flowchart of a model learning process. [Figure 6] 10 is an example of a flowchart of a learning monitoring process. [Figure 7] 10 is a flowchart illustrating an example of a learning control process. [Figure 8] FIG. 10 is a diagram illustrating an example of a model information screen. DETAILED DESCRIPTION OF THE INVENTION

[0014] In the following description, an "interface apparatus" may refer to one or more interface devices, which may be at least one of the following: One or more I / O (Input / Output) interface devices. The I / O (Input / Output) interface devices are interface devices for at least one of the I / O devices and a remote display computer. The I / O interface device for the display computer may be a communications interface device. The at least one I / O device may be a user interface device, for example, either an input device such as a keyboard and a pointing device, or an output device such as a display device. One or more communication interface devices. The one or more communication interface devices may be one or more homogeneous communication interface devices (e.g., one or more NICs (Network Interface Cards)) or two or more heterogeneous communication interface devices (e.g., an NIC and an HBA (Host Bus Adapter)).

[0015] In the following description, "memory" refers to one or more memory devices, typically a primary storage device. At least one of the memory devices may be a volatile memory device or a non-volatile memory device.

[0016] In the following description, a "persistent storage device" refers to one or more persistent storage devices. A persistent storage device is typically a non-volatile storage device (e.g., an auxiliary storage device), specifically, for example, a hard disk drive (HDD) or a solid state drive (SSD).

[0017] In the following description, the term "storage device" may refer to at least one memory, including memory and persistent storage device.

[0018] In the following description, a "processor" refers to one or more processor devices. The at least one processor device is typically a microprocessor device such as a CPU (Central Processing Unit), but may also be another type of processor device such as a GPU (Graphics Processing Unit). The at least one processor device may be a single-core or multi-core. The at least one processor device may also be a processor core. The at least one processor device may also be a processor device in a broader sense, such as a hardware circuit (e.g., an FPGA (Field-Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit)) that performs part or all of the processing.

[0019] Furthermore, in the following description, functions are sometimes described using the expression "yyy unit." However, the functions may be realized by one or more computer programs executed by a processor, by one or more hardware circuits (e.g., FPGAs or ASICs), or by a combination thereof. When a function is realized by a program executed by a processor, the specified processing is performed using a storage device and / or an interface device, etc., as appropriate, and therefore the function may be at least a part of the processor. Processing described using a function as the subject may also be processing performed by a processor or a device having the processor. A program may be installed from a program source. The program source may be, for example, a computer from which the program is distributed or a computer-readable recording medium (e.g., a non-transitory recording medium). The description of each function is merely an example; multiple functions may be combined into one function, or one function may be divided into multiple functions.

[0020] In the following description, information that provides an output for an input may be described using expressions such as "xxx table." However, this information may be data of any structure (for example, structured data or unstructured data), or may be a neural network that generates an output for an input, or a learning model such as a genetic algorithm or random forest. Therefore, the "xxx table" may be referred to as "xxx information." In the following description, the table configuration is an example, and one table may be divided into two or more tables, or all or part of two or more tables may be one table.

[0021] In the following description, a "model" is a machine learning model, which may be, for example, a neural network.

[0022] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. [Example]

[0023] FIG. 1 shows an example of the configuration of a model learning system 1 according to one embodiment of the present invention.

[0024] As shown in the figure, model learning system 1 may be a physical computer system equipped with interface device 101, storage device 102, and processor 103, or may be a logical computer system based on a physical computer system (e.g., a virtual machine or cloud computing system). User device 3 is communicatively connected to interface device 101 via communication network 2. Communication network 2 is a wireless or wired communication means, such as a LAN (Local Area Network), WAN (Wide Area Network), the Internet, a dedicated line, or various public communication networks. User device 3 may be an information processing terminal such as a personal computer or smartphone. Model learning system 1 may be an example of a server, and user device 3 may be an example of a client.

[0025] The storage device 102 stores data accessed by the processor 103 and computer programs executed by the processor 103. Examples of data stored in the storage device 102 include a learning history table 400, a learning resource table 410, and an execution cost table 600. Examples of computer programs stored in the storage device 102 include a computer program for realizing the learning execution unit 12 described below, and a computer program for realizing the learning monitoring unit 10, learning control unit 11, and information output unit 13 described below. The computer program for realizing the learning monitoring unit 10 and learning control unit 11 and the computer program for realizing the information output unit 13 may be separate programs.

[0026] The processor 103 executes one or more computer programs stored in the storage device 102 to realize functions such as a learning monitor 10, a learning control unit 11, a learning execution unit 12, and an information output unit 13. The learning monitor 10 collects resource usage and execution time (learning time) for model learning. The learning control unit 11 performs control to recover from learning delays when there are delays in model learning. The information output unit 13 displays model information, which is information about the model, on the user device 3 (transmits information for displaying the model information to the user device 3).

[0027] The learning execution unit 12 executes model learning with resource usage within the range of the specified resource allocation. The learning execution unit 12 may be, for example, a digital infrastructure realized using IaaS. The learning execution unit 12 controls the resource usage required for model learning in accordance with an instruction to increase or decrease the amount of resources (for example, by increasing the CPU allocation or decreasing the memory allocation of a virtual machine in IaaS).

[0028] At least some of the interface device 101, the storage device 102, and the processor 103 may be examples of multiple types of computational resources used for model training. Specifically, for example, the storage device 102 may include a memory, and the processor 103 may include one or more CPUs having multiple CPU cores, and the multiple types of computational resources may be a memory and a CPU. Resources such as ports and communication bandwidths may also be adopted as computational resources used for model training.

[0029] Each time a model is trained, the learning monitor 10 acquires the resource allocation amount (the amount of computational resources allocated to the model's training) for each of multiple types of computational resources, and monitors the resource usage amount (the amount of computational resources used to train the model) for each of multiple types of computational resources. The learning control unit 11 estimates the time required for the current training of the model and performs a time determination to determine whether the estimated time is longer than the time required for previous training of the model (the time identified from the learning history table 400). If the result of the time determination is true, the learning control unit 11 identifies a bottleneck computational resource type based on the resource usage amounts of the multiple types of computational resources in the current training, and increases the resource allocation amount for the identified computational resource type in the current training (for example, instructs the learning execution unit 12 to increase the resource usage amount for the model).

[0030] According to this model learning system 1, if it is determined that the estimated time required for the current learning is longer than the previous learning time, the amount of resources allocated for learning is increased, thereby making it possible to avoid delays in model learning, and since such an increase in resource allocation is possible, there is no need to allocate excessive amounts of resources in advance.

[0031] The estimated learning time is compared with past learning time history, and one example of the technical significance of this is as follows: the size of the training data used to learn the same model is unlikely to change significantly during learning. Therefore, by using past learning time as a comparison target for the estimated learning time, it is possible to efficiently determine whether a delay will occur in the current learning.

[0032] This embodiment will be described in detail below. For convenience of explanation, the accuracy of the model will be referred to as "learning accuracy."

[0033] FIG. 2 shows an example of the learning history table 400.

[0034] As shown in the figure, the learning history table 400 shows the history of past learning for each model. For example, the learning history table 400 has a record for each model learning. The record holds information such as model name 401, Epoch 402, learning accuracy 403, date and time 404, time 405, number of data 406, CPU 407, and memory 408. Let's take one model learning as an example.

[0035] Model name 401 indicates the name of the model that is the target of model learning. Epoch 402 indicates the number of epochs in model learning. That is, in this embodiment, a group of training data is used for model learning, and one training data is repeatedly input. Learning that includes the input of one training data is an "epoch." Learning ends when a predetermined number of epochs (number of epochs) have been performed. Specifically, for example, the learning monitor 10 is configured to perform the following each time a model is learned. The start of the first epoch for the learning is determined as the start of the learning. After the start of the learning, if the specified number of epochs for the learning have not been completed, the model is determined to be in the middle of learning. After the learning starts, the completion of the specified number of epochs for the learning is determined to be the end of the learning of the model.

[0036] The learning accuracy 403 indicates the learning accuracy of a model that has completed learning. For example, at least one of the accuracy rate, loss, and other factors may be adopted as the elements of the learning accuracy. In this embodiment, the learning accuracy is defined by the accuracy rate and loss. Furthermore, "end of learning" may be any of the following: -The specified number of epochs is performed. The learning accuracy has converged (an example of a case where learning ends even if the specified number of epochs have not been performed).

[0037] Date and time 404 indicates the date and time when the record was stored (start date and time of model learning). Time 405 indicates the difference between the date and time when learning started and the date and time when learning ended, i.e., the time required for model learning. Number of data 406 indicates the number of data elements that make up the training data used for model learning. For example, if the training data is a collection of multiple image files, the number of data may be the number of image files.

[0038] CPU 407 represents the CPU allocation allocated to model learning. Memory 408 represents the memory allocation allocated to model learning. The CPU allocation and memory allocation are examples of resource allocation.

[0039] FIG. 3 shows an example of the learning resource table 410.

[0040] As shown in the figure, the learning resource table 410 represents the current learning status of each model. For example, the learning resource table 410 has a record for each epoch. The record holds information such as model name 411, date and time 419, epoch 412, learning accuracy 403, CPU 141, memory 415, time 416, delay 417, and resource 418. Take one epoch as an example.

[0041] Model name 401 indicates the name of the model to be trained. Epoch 402 indicates which epoch this is in the current training. Training accuracy 403 indicates the training accuracy of the model being trained (in this embodiment, the accuracy rate and loss). CPU 414 indicates the CPU usage (e.g., CPU usage rate and / or CPU usage time) involved in training the model. Memory 415 indicates the memory usage (e.g., memory usage rate and / or memory usage) involved in the training process. Time 416 indicates the difference in date and time between the start of training and the time of measurement, i.e., the time required for model training.

[0042] Delay 417 is a value that indicates whether the learning time (the time estimated to be required for current learning) estimated based on time 416 is longer (delayed) than the past learning time of the same model. For example, if there is a delay, the value is "Yes," and if there is no delay, the value is "No."

[0043] Resource 418 represents resource allocations such as CPU allocations and memory allocations for learning. For example, the value of resource 418 is a resource amount pair that is a pair of the number of CPU cores (an example of CPU allocation) and memory size (memory allocation).

[0044] FIG. 4 shows an example of the execution cost table 600 .

[0045] As shown in the figure, the execution cost table 600 shows the costs of multiple types of computing resources. For example, the execution cost table 600 has a record for each resource amount pair. The record holds information such as an allocated resource 601 and a cost 602. Take one resource amount pair as an example.

[0046] The allocated resources 601 indicate the CPU allocation amount and memory allocation amount for the pair, and the cost 602 indicates the resource usage cost (execution cost) per unit time for the pair.

[0047] Next, an example of processing performed in the model learning system 1 configured as above will be described.

[0048] FIG. 5 is a flowchart of the model learning process.

[0049] First, the learning execution unit 12 defines a model (e.g., a neural network) to be learned (step 801) in response to, for example, an operation from the user device 3. Here, the model may be defined by creating a model when learning is executed, or by importing a model that has already been created.

[0050] Next, the learning execution unit 12, for example, in response to an operation from the user device 3, specifies the number of epochs (number of times learning is repeated) for the model defined in step 801 (step 802). For example, in neural network learning, it is common to repeatedly learn models by inputting the same training data in order to improve learning accuracy. Model learning with one input of training data is called an "epoch." In step 802, the number of epochs is specified.

[0051] Next, the learning execution unit 12 executes model learning by inputting training data into the model to be learned (step 803). That is, the learning execution unit 12 executes one epoch and updates (increments by 1) the number of epochs (e.g., a count value stored in memory).

[0052] Next, the learning execution unit 12 determines whether the number of epochs after the update has reached the number of epochs specified in step 802 (step 804). If the determination result in step 804 is false, the process returns to step 803. If the determination result in step 804 is true, the learning of the model to be learned is terminated.

[0053] 6 is a flowchart of the learning monitoring process. The learning monitoring unit 10 is called by the learning execution unit 12 when the model learning process is executed, and starts the learning monitoring process. For example, the timing of the call is when the model learning process shown in FIG. 5 starts, when step 803 is completed (when the epoch is completed), and when the model learning process ends.

[0054] The learning monitor 10, called by the learning execution unit 12, first determines whether it is the start time of the model learning process (step 301).

[0055] If the determination result in step 301 is true, the learning monitor 10 records the date and time 419 (learning start date and time) for the model to be learned in the learning resource table 410 and stores the number of data (the number of data elements in the input training data) in memory (step 302). The learning monitor 10 instructs the learning execution unit 12 to set a resource allocation (step 308). Here, the "resource allocation" in step 308 may be a resource amount pair, such as "4 CPU cores" and "4 GB" of memory. The resource allocation may be selected from the perspective of execution cost, or may be selected based on an estimated value obtained from statistical calculations based on past learning results (e.g., regression analysis of the number of past data 406, CPU 407, and memory 408 for the model to be learned). Note that step 308 may be executed at the start of model learning, independently of the learning monitoring process.

[0056] If the determination result in step 301 is false, the learning monitor 10 acquires the resource usage (CPU usage and memory usage) and resource allocation (CPU allocation and memory allocation) of the model learning process from the learning execution unit 12 (step 303). Here, the resource usage may be the value at the time of acquisition, or may be a statistical value (e.g., average or maximum value) of the resource usage from the previous epoch to the time of measurement. If the statistical value is an average value, it is useful because it is closer to the actual resource usage of the learning process. The learning monitor 10 acquires information such as the name of the model being learned (model name), the current epoch number (count value), and learning accuracy from the learning execution unit 12 (step 304). The learning monitor 10 determines whether model learning has ended (step 305).

[0057] If the determination result in step 305 is true, the learning monitor 10 calculates the learning time of the model from the difference between the date and time 419 of the model to be learned and the current date and time, and further calculates statistical values ​​(e.g., average values) of the CPU allocation and memory allocation from the CPU 414 and memory 415 for each epoch of the model. The learning monitor 10 records the acquired model name 401, Epoch 402 (number of epochs), learning accuracy 403, and number of data 406, as well as the date and time 404, time 405 (calculated learning time), CPU 407 (calculated statistical value of CPU allocation), and memory (calculated statistical value of memory allocation) in the learning history table 400 (step 306).

[0058] If the judgment result in step 305 is false, the learning monitor 10 calculates the model learning time from the difference between the date and time 419 and the current date and time, and records the time 416 in the learning resource table 410 together with the acquired model name 411, Epoch 412, learning accuracy 413, resource usage (CPU 414 and memory 415), and resource 418 (step 307).

[0059] As described above, the learning monitor 10 monitors the model accuracy of the model being learned each time the model is learned (for example, acquires the model accuracy for each epoch). When learning is completed, the model accuracy, number of data, etc. of the model are recorded in the learning history table 400.

[0060] FIG. 7 is a flowchart of the learning control process.

[0061] The learning control unit 11 is called by the learning execution unit 12 when the model learning process is executed, similar to the learning monitoring unit 10. For example, the learning control unit 11 is called when the model learning process shown in Fig. 5 starts, when step 803 is completed (when an epoch is completed), and when the model learning process ends.

[0062] The learning control unit 11, called by the learning execution unit 12, acquires the time 416, Epoch 412, resource usage (CPU 414 and memory 415), and learning accuracy 413 for all records of the model being learned (model name 411) from the learning resource table 410 (step 501).

[0063] Next, the learning control unit 11 acquires the time 405 and learning accuracy 403 corresponding to the past model learning of the model from the learning history table 400 (step 502).

[0064] Next, the learning control unit 11 performs a convergence determination to determine whether the learning accuracy acquired in step 501 has converged (step 503). If the result of the convergence determination in step 503 is true, the learning control unit 11 instructs the learning execution unit 12 to stop (end) model learning (step 509). If the result of the convergence determination in step 503 is false, the learning control unit 11 decides to continue model learning (for example, does not send any special instruction to the learning execution unit 12) (step 508). Here, the learning accuracy having converged may mean that the fluctuations in the learning accuracy 413 have been kept small, and may be, for example, any of the following: The difference between the learning accuracy 413 (e.g., accuracy rate) corresponding to the latest epoch acquired in step 501 and the learning accuracy 413 (e.g., accuracy rate) corresponding to the previous epoch (Epoch 412 one step smaller than the latest Epoch 412) is smaller than a threshold value. The range (difference between the maximum and minimum values) of the learning accuracy 413 (e.g., accuracy rate) over the period from the most recent epoch to a pre-specified number of epochs back is smaller than a threshold. By terminating model learning when the learning accuracy has converged in this way (specifically, by terminating model learning without making the determination in step 505 described below), it is possible to avoid delays in model learning and reduce execution costs.

[0065] If the result of the convergence determination in step 503 is false, the learning control unit 11 estimates the learning time of the model being trained (step 504). Here, one method of estimating the learning time is to multiply the average learning time per epoch, obtained by dividing the latest time 416 acquired in step 501 by the latest epoch 412, by the latest epoch number (count value), and use this value as an estimate of the time required for the current model training. As described above, the learning control unit 11 performs the convergence determination (step 503) based on the learning accuracy 413 for each epoch in the current training, and estimates the time required for the current training based on the latest epoch 412 (count value) and the time required until the latest epoch 416. In this way, the information acquired for each epoch can be used for the convergence determination and learning time estimation.

[0066] Next, the learning control unit 11 performs a time determination to determine whether the learning time estimated in step 504 is longer than the past time 405 acquired in step 502, that is, to determine whether the learning process is delayed (step 505). If the result of the time determination in step 505 is true, the learning control unit 11 instructs the learning execution unit 12 to increase the resource allocation amount in order to increase the resource allocation amount allocated to the learning process (step 510).

[0067] Here, the past learning time compared with the estimated time required for the current learning may be based on at least one of all past times 405 of the model, for example, the longest time 405 of the model's past times 405, or the average value of the model's past times 405. Furthermore, an estimated learning time that falls within a statistically outlying range (such as μ+3σ or more) based on the average value μ and standard deviation σ of the model's past times 405 may be considered to be delayed.

[0068] In addition, the increase in resource allocation may be, for example, by increasing the allocation to a CPU or memory whose utilization rate is higher than a threshold by a preset value, or by increasing the allocation to a CPU or memory whose execution cost is lower. Increasing the resource allocation based on the execution cost may be performed by referring to the execution cost table 600.

[0069] Furthermore, here, if the result of the time determination in step 505 is true, the learning control unit 11 may identify the bottleneck computational resource type based on the resource usage of multiple types of computational resources in the current learning (in this embodiment, the CPU 414 and memory 415 corresponding to each epoch of the model being learned). For example, the computational resource corresponding to whichever of the CPU 414 and memory 415 exceeds a threshold, or if both the CPU 414 and memory 415 exceed their thresholds, the computational resource corresponding to the computational resource with the larger difference in usage rate from the threshold, may be the computational resource of the bottleneck computational resource type. In step 510, the learning control unit 11 may increase the resource allocation amount for the identified computational resource type in the current learning. This is expected to result in an efficient increase in resource allocation amount.

[0070] If the result of the time determination in step 505 is false, the learning control unit 11 performs an accuracy determination (step 506) to determine whether the latest learning accuracy 413 acquired in step 501 is higher than the learning accuracy in the past learning results acquired in step 502. If the result of the accuracy determination in step 506 is true, the learning control unit 11 instructs the learning execution unit 12 to reduce the amount of resource allocation in order to reduce the amount of resources allocated to the learning process (step 507). This is expected to optimize the amount of resource allocation.

[0071] Here, the past training accuracy to be compared with the latest training accuracy 413 may be based on at least one of all past training accuracies 403 of the model, and may be, for example, the highest training accuracy 403 in the past training records, or the average value of the training accuracies 403 in the past training records.

[0072] Furthermore, the resource allocation reduction may involve, for example, reducing the allocations of CPUs and memory whose utilization rates are lower than a threshold to a preset value, or reducing the allocation of the CPU or memory with a lower execution cost. Reducing the resource allocation based on the execution cost may be performed by referring to the execution cost table 600. For example, the learning control unit 11 may identify which type of computational resource, among multiple types, will most effectively reduce costs by maintaining its resource allocation, and then reduce the resource allocations of types of computational resources other than the identified type. Specifically, for example, if the current resource amount pair is "8-core CPU" and "8 GB" memory, the execution cost table 600 illustrated in FIG. 4 indicates that maintaining "8-core CPU" and reducing memory will result in lower costs than maintaining "8 GB" memory and reducing the number of CPU cores. Therefore, the instruction to reduce the resource allocation is to change "8-core CPU" and "8 GB" memory to "8-core CPU" and "4 GB" memory. This is expected to result in an appropriate reduction in the resource allocation.

[0073] After step 507, the learning control process ends via the above-mentioned step 508. Also, if the result of the accuracy determination in step 506 is false, the learning control process ends via the above-mentioned step 508.

[0074] In the learning control process, the learning control unit 11 may adjust at least one of the times compared in the time determination in step 505 based on the difference between the amount of training data used in the current learning and the amount of training data used in the previous learning. For example, the learning control unit 11 may adjust the time required for previous learning of the model (e.g., the statistical value of time 405) and / or the time estimated in step 504 based on the difference between the amount of training data used in the current learning (the number of pieces of data recorded in step 302 and acquired by the learning control unit 11) and the amount of training data used in the previous learning (e.g., the statistical value of data number 406). The time determination in step 505 may be a determination of whether the adjusted estimated time is longer than the previous learning time, whether the time estimated in step 504 is longer than the previous adjusted learning time, or whether the adjusted estimated time is longer than the previous adjusted learning time. The length of the learning time depends on the amount of data (amount of training data), but at least one of the learning times compared in the time judgment is adjusted based on the difference between the amount of data this time and the amount of data in the past, so we can expect to improve the accuracy of determining whether a learning delay is occurring.

[0075] FIG. 9 is a diagram showing an example of the model information screen.

[0076] The information output unit 13 displays the model information (information relating to the learning state of the model, including information based on the result of the time determination in step 505) (for example, by transmitting display information of the model information to the user device 3). The model information screen 700 is a display screen for the model information, and the model information represents the learning state of the model. The model information screen 700 allows the user to understand the learning state of the model.

[0077] Specifically, for example, the information output unit 13 generates and displays model information based on information acquired from the learning resource table 410 and the learning history table 400. The model information includes information such as a model name 701, a learning status 702, a scheduled end date and time 703, an execution status 704, and an execution control 705 for each of one or more models (for example, one or more models specified by the user).

[0078] The model name 701 indicates the model name.

[0079] The learning status 702 indicates the learning status of the model (for example, learning in progress, learning completed). The learning status 702 of a model whose model name is recorded in the learning history table 400 but whose model name is not recorded in the learning resource table 410 is "learning completed," and the learning status 702 of a model whose model name is recorded in the learning resource table 410 is "learning," regardless of whether the model name is recorded in the learning history table 400 or not.

[0080] The estimated end date and time 703 indicates the end date and time of the study time estimated based on the learning resource table 410 (the study time estimated in step 504).

[0081] The execution status 704 indicates whether the learning time of the model being trained is delayed. Specifically, if the result of the time determination in step 505 is true, the execution status 704 is "delayed," and if the result of the time determination in step 505 is false, the execution status 704 can be "smooth."

[0082] Execution control 705 is information when the resource allocation is increased or decreased by the learning control unit 11. For example, if step 510 is performed and the content is an increase in the CPU allocation, execution control 705 may be "CPU scale up," and if step 507 is performed and the content is a decrease in the CPU allocation, execution control 705 may be "CPU scale down."

[0083] The present invention is not limited to the above-described embodiments and includes various modifications. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those including all of the described configurations. Furthermore, it is possible to replace part of the configuration of one embodiment with the configuration of another embodiment, or to add the configuration of another embodiment to the configuration of one embodiment. Furthermore, it is possible to add, delete, or replace part of the configuration of each embodiment with other configurations.

[0084] Furthermore, the above-described configurations, functions, processing units, processing means, etc. may be partially or entirely implemented in hardware, for example, by designing them as integrated circuits. The above-described configurations, functions, etc. may also be implemented in software, with a processor interpreting and executing a program that implements each function. Information such as the programs, tables, and files that implement each function can be stored in a storage unit such as a memory, a hard disk, or an SSD (Solid State Drive), or on a recording medium such as an IC card, SD card, DVD, or BD.

[0085] In addition, the control lines and information lines shown are those that are considered necessary for the explanation, and do not necessarily show all the control lines and information lines in the product. In reality, it can be assumed that almost all components are interconnected. [Explanation of symbols]

[0086] 1: Model learning system 10: Learning monitoring unit 11: Learning control unit

Claims

1. a learning monitoring unit that, each time a machine learning model is learned, acquires a resource allocation amount, which is the amount of computational resources allocated to the learning of the machine learning model, for each of a plurality of types of computational resources, and monitors a resource usage amount, which is the amount of computational resources used for the learning of the machine learning model, for each of the plurality of types of computational resources; a learning control unit that estimates a time required for current learning of the machine learning model, performs a time determination to determine whether the estimated time is longer than a time required for past learning of the machine learning model, and increases a resource allocation amount for the current learning for at least one of the plurality of types of computing resources when the result of the time determination is true; A model learning system comprising:

2. the learning monitoring unit monitors the model accuracy of the machine learning model each time the machine learning model is learned; The learning control unit A convergence determination is performed to determine whether the model accuracy has converged in the current learning. If the result of the convergence determination is true, the current learning is terminated. If the result of the convergence determination is false, the time required for the current learning is estimated. The model learning system of claim 1 .

3. The learning monitoring unit, each time the machine learning model is learned, determining the start of the first epoch for the learning as the start of the learning; After the start of the learning, if the number of epochs specified for the learning has not been completed, the machine learning model is determined to be in the middle of learning; After the start of the learning, the completion of a specified number of epochs for the learning is determined as the end of learning of the machine learning model; the learning control unit determines the convergence based on the model accuracy for each epoch in the current learning, and estimates the time required for the current learning based on the number of latest epochs and the time required until the latest epoch. The model learning system of claim 2 .

4. The learning control unit If the result of the time determination is false, an accuracy determination is performed to determine whether the model accuracy in the current learning is higher than the model accuracy in past learning of the machine learning model; If the result of the accuracy determination is true, a computing resource type to be reduced in resource allocation amount is determined based on the costs of the plurality of types of computing resources, and the resource allocation amount in the current learning is reduced for the computing resources of the determined computing resource type. The model learning system of claim 1 .

5. If the result of the accuracy determination is true, the learning control unit identifies which type of computational resource, among the plurality of types of computational resources, should be maintained to reduce the cost most, and reduces the resource allocation of types of computational resources other than the identified type. The model learning system of claim 4 .

6. the learning control unit adjusts at least one of the times compared in the time determination based on a difference between an amount of training data used in the current learning and an amount of training data used in the previous learning. The model learning system of claim 1 .

7. an information output unit that displays model information that is information about the learning state of the machine learning model and that includes information based on the result of the time determination; The model learning system of claim 1 further comprising:

8. when the result of the time determination is true, the learning control unit identifies a bottleneck computing resource type based on the resource usage amounts of the plurality of types of computing resources in the current learning, and increases the resource allocation amount in the current learning for the computing resources of the identified computing resource type. The model learning system of claim 1 .

9. Each time a machine learning model is trained, the computer acquires, for each of a plurality of types of computational resources, a resource allocation amount that is the amount of computational resources allocated to the training of the machine learning model, and monitors, for each of the plurality of types of computational resources, a resource usage amount that is the amount of computational resources used for the training of the machine learning model; The computer estimates the time required for current learning of the machine learning model, The computer performs a time determination to determine whether the estimated time is longer than a time required for past learning of the machine learning model; If the result of the time determination is true, the computer increases the resource allocation amount for the current learning for at least one of the plurality of types of computing resources. Model learning methods.

10. Each time a machine learning model is trained, a resource allocation amount is acquired for each of a plurality of types of computational resources, which is the amount of computational resources allocated to the training of the machine learning model, and a resource usage amount is monitored for each of the plurality of types of computational resources, which is the amount of computational resources used for the training of the machine learning model; Estimate the time required for current learning of the machine learning model; performing a time determination to determine whether the estimated time is longer than the time required for past learning of the machine learning model; If the result of the time determination is true, increase the resource allocation amount in the current learning for at least one of the plurality of types of computing resources. A computer program that causes a computer to do something.

Images