Learning device, method, and program

Abstract

To generate a high-performance trained model.SOLUTION: A learning device according to the present embodiment includes a determination unit, and a re-training unit. The determination unit determines a re-training condition on the basis of a local result that is a result of training a local model and a global result that is a result of updating a parameter of a global model through federated learning. The re-training unit re-trains a model for re-training on the basis of the re-training condition.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] FIELD Embodiments of the present invention relate to a learning device, a learning method, and a learning program. [Background technology]

[0002] In machine learning, there is a training technique called federated learning. In federated learning, multiple local devices each train a machine learning model (local model) based on the training data they hold, and then send the parameters of the learned local model to a server. The server aggregates and integrates the parameters of each local model and updates the machine learning model (global model) stored on the server. The parameters of the updated global model are distributed to each of the multiple local devices. Federated learning is performed by repeating this series of processes.

[0003] In federated learning, training is performed on multiple local devices, which distributes the computational load. Furthermore, since only parameters are exchanged between the server and the trainee, the training data itself is not exchanged. This has the advantages of high privacy, the ability to utilize knowledge of the training data of other local devices, and low communication costs. Although federated learning can obtain a general-purpose global model, the characteristics of the data from each local device are lost. Therefore, one method is to use the parameters of the global model after federated learning as initial values ​​and fine-tune them using the training data held by each local device. However, when performing such fine-tuning, it is difficult to determine the retraining conditions for the adjustment parameters on the local device. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2023-042922 Summary of the Invention [Problem to be solved by the invention]

[0005] The present disclosure has been made to solve the above-mentioned problems, and aims to provide an information processing device, method, and program capable of generating a high-performance trained model. [Means for solving the problem]

[0006] The learning device according to this embodiment includes a determination unit and a retraining unit. The determination unit determines retraining conditions based on local results obtained by training a local model and global results obtained by updating parameters of a global model through federated learning. The retraining unit retrains a model for retraining based on the retraining conditions. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a block diagram showing a learning system according to an embodiment of the present invention. [Figure 2] FIG. 4 is a sequence diagram showing an example of the operation of the learning system according to the present embodiment. [Figure 3] 10 is a flowchart showing details of a process for setting retraining conditions for a local device. [Figure 4] FIG. 2 is a diagram showing an example of the hardware configuration of a learning device. DETAILED DESCRIPTION OF THE INVENTION

[0008] The learning device, method, and program according to the present embodiment will be described in detail below with reference to the drawings. In the following embodiments, parts with the same reference numerals perform similar operations, and redundant explanations will be omitted as appropriate.

[0009] A learning system 1 according to this embodiment will be described with reference to the block diagram of FIG. The learning system 1 according to this embodiment includes a local device 10A, a local device 10B, and a server 11, all of which are connected to each other via a network NW so that data can be transmitted and received. As an example, two local devices 10A and 10B are shown here, but three or more local devices 10 may be included. Furthermore, when describing the local devices in common, they will simply be referred to as local devices 10.

[0010] Each local device 10 includes a learning apparatus 100. The learning apparatus 100 includes a training unit 101, a determining unit 102, a retraining unit 103, a communication unit 104, and a storage unit 105.

[0011] The training unit 101 updates the local model by training it using the local data as training data. The local data is, for example, image data such as an inspection image. For example, a pair of image data related to the inspection image and a teaching label for distinguishing between good and bad products in the inspection image may be used as the training data. The local model may be, for example, a neural network, and is trained to perform a classification task, for example, classifying inspection images into good and bad products. Note that the training data is not limited to the example of inspection images, and may be appropriately selected so that the local device 10 can generate a target trained model.

[0012] The determination unit 102 determines retraining conditions for retraining the model based on the local results and the global results. The local results are the results of training the local model using training data in the local device 10. The local results include local model parameters, which are parameters of the trained local model, information on the performance of the trained local model, and training conditions for the local model. The local model parameters are, for example, a parameter set (weight coefficients, biases, etc.) of a neural network for sharing parameters with the global model. The global results include parameters of the global model, information on the performance of the global model, and training conditions for the global model. In this embodiment, "retraining" includes the concept of so-called fine tuning, in which trained parameters are used as initial values ​​for the model, the concept of so-called delta tuning, in which trained parameters for some parameters or some layers are used as initial values ​​and the values ​​of other parameters are fixed, and the concept of training a model by adding new parameters that are not present in the local model or the global model, such as a personalization layer.

[0013] The retraining unit 103 retrains a retraining model, which is a model for retraining, based on the retraining conditions, to generate a retrained model. The communication unit 104 transmits the local model parameters to the server 11. The communication unit 104 also receives the global results from the server 11. The storage unit 105 stores training data, local models, local results, global results, and the like.

[0014] The server 11 includes a storage unit 111 , an update unit 112 , and a communication unit 113 . The storage unit 111 stores global model and local model parameters. The global model is, for example, a neural network model. The update unit 112 updates the global model using the local model parameters received from the multiple local devices 10. The communication unit 113 receives each local model parameter from the multiple local devices 10. The communication unit 113 also transmits the updated global model to each local device 10.

[0015] Examples of local and global models include convolutional neural networks (CNNs), multilayer perceptrons (MLPs), recurrent neural networks (RNNs), Transformers, and Bidirectional Encoder Representations from Transformers (BERTs). Other common neural network models used in machine learning are also acceptable. Furthermore, federated learning is not limited to neural network models; it can be applied to any machine learning model to which federated learning can be applied. For example, machine learning models such as SVMs (Support Vector Machines) and Random Forests are also acceptable. Note that new adjustment parameters required for retraining are added as retraining conditions depending on the type of machine learning model. For example, adjustment parameters such as margins are added to SVMs, and the number of trees is added to Random Forests.

[0016] Next, an example of the operation of the learning system 1 according to this embodiment will be described with reference to the sequence diagram of Fig. 2. It is assumed that each of the multiple local devices 10 executes processing in accordance with the sequence diagram shown in Fig. 2.

[0017] In step SA1, the training unit 101 of the local device 10 trains the local model using the training data to generate a trained local model. The training of the local model is based on general machine learning training, and parameters of the neural network related to the local model are updated using, for example, backpropagation and stochastic gradient descent to minimize the training error. The determination of whether the training of the local model is complete may be made, for example, when the value of the training error is equal to or less than a threshold. The determination may also be made when the parameters have been updated a predetermined number of times. The determination may also be made when the absolute value or the sum of the absolute values ​​of the parameter update amounts reaches a certain value. The determination of whether the training is complete is not limited to the above example, and termination conditions commonly employed in machine learning may also be used.

[0018] In step SA2, the communication unit 104 of the local device 10 transmits the local model parameters to the server 11. The local model parameters may be, for example, parameter values ​​of a trained local model, or the amount of change before and after training (updating) the local model, for example, the difference between the parameter value before the update and the parameter value after the update. The communication unit 104 of the local device 10 may also compress data related to the parameter set to be transmitted to the server 11 and transmit the compressed data. The data compression process may be lossless compression or lossy compression. Transmitting the compressed data can reduce the amount of communication and the communication bandwidth. The data may also be encrypted before transmission, which can improve the confidentiality of the data.

[0019] In step SA3, the communication unit 113 of the server 11 receives the local model parameters from each local device 10, and the storage unit 111 of the server 11 stores the local model parameters.

[0020] In step SA4, the update unit 112 of the server 11 updates the global model based on the local model parameters. Whether the update of the global model is complete can be determined using a determination criterion used in general associative learning. For example, the update of the global model can be determined to be complete when the global model has been updated a predetermined number of times or when the update amount of the global model has converged to a threshold or less.

[0021] In step SA5, the communication unit 113 of the server 11 transmits the global result related to the global model to each local device 10. The global result may be the global model itself, the parameter value of the global model after updating, the amount of change (difference) due to the update, or a combination of these.

[0022] In step SA6, the communication unit 104 of the local device 10 receives the global results for the global model.

[0023] In step SA7, the decision unit 102 of the local device 10 decides retraining conditions based on the local results and the global results.

[0024] In step SA8, the retraining unit 103 of the local device 10 trains the retrained model in accordance with the retraining conditions determined by the determination unit 102, generating a retrained model. The determination unit 102 sets the parameters of the global model as the initial values ​​of the parameters of the retrained model. This allows training to begin based on the parameters of a global model that has average knowledge acquired from a large amount of data, compared to training that begins with random numbers as initial values, and provides reasonable initial values. Therefore, it is believed that the performance of the retrained local model will exceed that of the trained local model. The termination conditions for retraining may be set to the same conditions as for training a local model. That is, retraining may be terminated early if a pre-specified threshold for the difference in loss function, the difference in performance, or the number of epochs is exceeded. In step SA9, the storage unit 105 of the local device 10 stores the re-trained model.

[0025] Next, the process of setting the retraining conditions of the local device 10 in step SA7 will be described with reference to the flowchart of FIG.

[0026] In step SB1, the determination unit 102 uses the accuracy rate of image classification as an evaluation index for the classification task, compares the accuracy rate of the local model (also called the local accuracy rate) with the accuracy rate of the global model (also called the global accuracy rate), and determines which accuracy rate is higher. If the global accuracy rate is higher than the local accuracy rate, the process proceeds to step SB2; if the local accuracy rate is higher than the global accuracy rate, the process proceeds to step SB3.

[0027] In step SB2, the determination unit 102 determines the strength of the retraining conditions based on the local accuracy rate, the global accuracy rate, and the local training conditions so that the amount of update from the global model in the retraining model is small, in other words, so that the retraining model does not deviate from the global model. Specifically, as an example of determining the strength of the retraining condition, the retraining condition is set to a learning rate g s Assuming that the learning rate g s is defined by the following equation (1).

[0028] g s =f(k l / k g )*g l (1) where k l is the local accuracy rate, k g is the global accuracy rate, g lis the learning rate under local training conditions, and f is a function that is proportional to the learning rate. As shown in equation (1), when the global accuracy rate is higher than the local accuracy rate, that is, when information from other local devices 10 obtained through federated learning is effective (for example, when learning on the local device is difficult due to a lack of local data on the device itself or biased data), the learning rate g s is set small.

[0029] This allows the retrained model to be trained so that it does not deviate too much from the global model and so that the knowledge gained from effective associative learning is retained in the retrained model as much as possible. In other words, it is preferable to design the retraining conditions so as to limit the amount of updates from the global model.

[0030] On the other hand, in step SB3, the determination unit 102 determines the strength of the retraining conditions based on the local accuracy rate, the global accuracy rate, and the local training conditions so that the retraining model has a larger amount of update from the global model, in other words, so that the retraining model deviates from the global model.

[0031] If the local accuracy rate is higher than the global accuracy rate, that is, if the information from other local devices 10 obtained through federated learning is not effective (for example, if the characteristics (individuality) of the device itself are different from those of the other local devices 10, and an average global model integrating the data from all the local devices 10 does not sufficiently improve performance), the learning rate g s Just make it larger.

[0032] This allows the retrained model to be trained to deviate from the global model. In other words, by setting the retraining conditions so that the amount of update from the global model is as large as possible, the performance of the retrained model can be improved.

[0033] Note that any formula other than formula (1) may be used as long as it adjusts the strength of the retraining conditions so that the global model does not deviate significantly from the global model when the performance of the global model is higher than that of the local model, and so that the local model deviates from the global model when the performance of the local model is higher than that of the global model.

[0034] The retraining condition is not limited to the learning rate and may be other conditions. For example, the retraining condition may be the number of epochs. The number of epochs is an adjustment parameter that indicates the number of model updates, and the larger the number of epochs, the easier the training progresses. Therefore, the larger the number of epochs, the easier it is to diverge from the global model. Conversely, the smaller the number of epochs, the less likely it is to diverge from the global model. Therefore, the determination unit 102 may determine the number of epochs based on the comparison result of step SB1. Specifically, if the global result is superior to the local result (e.g., if the performance of the global model is higher than that of the local model), the determination unit 102 sets the number of epochs to be smaller than a predetermined value. On the other hand, if the local result is superior to the global result (e.g., if the performance of the local model is higher than that of the global model), the determination unit 102 sets the number of epochs to be larger than a predetermined value. This allows for smooth adjustment of the divergence between the retrained model and the global model (i.e., the amount of update from the global model) during retraining.

[0035] The retraining condition can also be the adjustment of regularization. For example, designing regularization such as weight decay, label smoothing, dropout, etc. is known to flatten the loss function landscape, making the model training stable and not changing rapidly. The determination unit 102 determines the strength of regularization based on the comparison result of step SB1. Specifically, when the global result is superior to the local result (for example, when the performance of the global model is higher than that of the local model), the determination unit 102 sets a large regularization ratio (regularization strength higher than a predetermined value) to reduce deviation from the global model. On the other hand, when the local result is superior to the global result (when the performance of the local model is higher than that of the global model), the determination unit 102 sets a small regularization ratio (regularization strength lower than a predetermined value), which makes the change in training more rapid and makes it easier for the retrained model to deviate from the global model. In this way, it is possible to control the deviation between the retrained model and the global model.

[0036] Furthermore, by distilling the global model as a teacher model during retraining, the retrained model and the global model can be prevented from diverging, and the output features can be made similar. Therefore, the determination unit 102 can more precisely control the similarity between the retrained model and the global model by setting the distillation strength (temperature parameter) based on the comparison result of step SB1. Specifically, when the global result is superior to the local result (e.g., when the performance of the global model is higher than that of the local model), the determination unit 102 sets the temperature parameter higher than a predetermined value. This makes it possible to utilize the features obtained by the global model, and is expected to improve retraining performance. On the other hand, when the local result is superior to the global result (when the performance of the local model is higher than that of the global model), the determination unit 102 may set the temperature parameter to a value lower than a predetermined value or to zero, since the influence of the features of the global model may be too strong.

[0037] Note that retraining does not necessarily require retraining all parameters. Especially for large-scale models, model training takes time, making retraining all parameters impractical. Therefore, the determination unit 102 may determine which layers and parameters to retrain based on the comparison results in step SB1 and the size of the retrained model. Specifically, the determination may be made using, for example, BitFit, which trains only the bias; PreEmbed, which relates to embedding vectors; PreLayer, which replaces the activation function of the Transformer layer with a trainable one; or adapter. Furthermore, if contribution results indicating which parameters or layers contribute most to inference are obtained as local or global results using a technique such as skill localization, the determination unit 102 may determine at least one of the layers and parameters to retrain based on the contribution results.

[0038] In addition to the existing parameters of the model, it is also possible to add a low-rank matrix using a personalization layer or LoRA (Low-Rank Adaptation). The initialization of the newly added parameters can be set to match the initialization method used when training the local model and global model, for example. By matching the initialization method in this way, the training status of the models will be similar, making it easier to compare whether the retrained model and the global model have diverged. Note that with LoRA, for example, the parameters of the introduced low-rank matrix can be initialized to zero. This takes the training data into account and is expected to speed up the model training speed.

[0039] Furthermore, because retraining is performed on the local device 10, an example has been described in which the learning rate for retraining is determined from the learning rate of the local model. However, global training conditions may also be used as retraining conditions. Because the final parameters of the global model are referenced when initializing the parameters of the retrained model, the retraining conditions can be appropriately set so as not to deviate from the global model or so as to deviate from the global model. Furthermore, because it is possible to avoid referring only to the local training conditions, it is possible to improve the performance of the retrained model so that it exceeds both local and global performance.

[0040] In this embodiment, the retraining conditions are described as learning rate, number of epochs, regularization, distillation, adjustment of model structure, and parameter initialization. However, the retraining conditions are not limited to these, and any method that has adjustment parameters in training may be used as the retraining conditions.

[0041] Furthermore, the determination unit 102 may use a method for calculating the degree of deviation between the retraining model itself and the global model itself, and adjust the strength of the retraining condition based on the degree of deviation.

[0042] For example, as a first technique, the determination unit 102 calculates the difference between the loss value calculated using the loss function of the global model and the loss value calculated using the loss function of the retraining model. The determination unit 102 may adjust the strength of the retraining conditions in proportion to the difference. In other words, if the difference in the loss values ​​is large, the determination unit 102 may adjust the strength of the retraining conditions to be higher.

[0043] As a second method, the determination unit 102 may calculate the difference in parameters between the global model and the retrained model. For example, the determination unit 102 calculates the difference between the sum of the absolute values ​​of each parameter of the global model and the sum of the absolute values ​​of each parameter of the retrained model. The determination unit 102 may adjust the strength of the retraining conditions to be proportional to the difference. Furthermore, the calculation is not limited to all parameters, and may be performed for each layer or module. Furthermore, the calculation is not limited to the sum of the absolute values ​​of each parameter, and may also calculate the difference in values ​​such as the L2 norm or the Frobenius norm.

[0044] As a third method, the determination unit 102 calculates a histogram of all parameters of the global model and a histogram of all parameters of the retrained model. The determination unit 102 approximates the two histograms with a Gaussian distribution and calculates the difference between the two distributions using KL-divergence or the like. The determination unit 102 may adjust the strength of the retraining conditions so that it is proportional to the difference. Of course, the calculation is not limited to all parameters, and may be performed for each layer, module, or the like.

[0045] As a fourth method, the determination unit 102 may use a Centered Kernel Alignment (CKA) method to calculate a similarity score between each expression obtained from a layer of the global model and a layer of the retrained model. The determination unit 102 may adjust the strength of the retraining condition in inverse proportion to the similarity score.

[0046] The intensity of the retraining condition may be adjusted by combining at least two of the first to fourth methods described above. Furthermore, weighting may be performed when combining the methods.

[0047] In this embodiment, a classification task is assumed, but any task, such as semantic segmentation, object detection, action recognition, anomaly detection, suspicious person detection, regression, or prediction, may be used. However, since the evaluation index adopted varies depending on the task, when comparing local results with global results, it is advisable to use the evaluation index adopted. Specifically, in semantic segmentation and object detection tasks, an index such as IoU is used to evaluate the amount of region error between the ground truth region and the estimated region, so IoU can be used as an evaluation index for the global model and the retrained model. Furthermore, in the case of a generation task, an index to evaluate generation quality, such as IS (Inception Score) or FID (Fréchet Inception Distance), can also be adopted.

[0048] Furthermore, the training data used in retraining may be the same as or different from the pre-trained local model or global model. For example, retraining performance can be improved by deleting noisy data or adding clean data without noise. In this case, the retraining conditions may be adjusted according to the data and changes in the data. For example, using clean data speeds up the convergence of training, which increases the degree of freedom to adjust the strength of the retraining conditions, for example, to be weaker. This makes it possible to efficiently adjust the retraining conditions to suit the data.

[0049] Furthermore, the training data and the input data when executing a trained local model are not limited to images, but may also be time-series data such as text, voice, machine sounds and other operational sounds, environmental sounds, acceleration data, instrument data, etc., and any data that can be handled by machine learning. Since the effects of retraining conditions vary depending on the type (modality) of data, the retraining conditions may be adjusted depending on the modality of the data.

[0050] For example, Batch Normalization (BN) is often used for normalization of images, while Layer Normalization (LN) is often used for normalization of text. Furthermore, in federated learning, BNs are not shared and are personalized for each local device 10, whereas LNs are often federated. Therefore, when an LN is initialized with parameters of a global model, the convergence speed is slower than that of a BN that has already been specialized (personalized) for data. Therefore, the determination unit 102 can adjust the strength of the retraining conditions so that it increases in proportion to, for example, the number of parameters of the LN. This allows the strength of the retraining conditions to be adjusted efficiently regardless of the modality of the data or the type of normalization method used.

[0051] During retraining, dynamics different from those at initialization occur. Therefore, the determination unit 102 may determine the retraining conditions again during the retraining process based on the degree of deviation and performance comparison between the retrained model currently being trained and the global model. For example, if the amount of update from the global model of the retrained model during retraining exceeds a threshold during retraining, the determination unit 102 may determine that the model deviates too much from the global model and set the intensity of the retraining conditions to a weaker level. This enables efficient adjustment during retraining in real time. Of course, if the performance of the retrained model adjusted in real time deteriorates, the determination unit 102 may change the intensity of the retraining conditions and the frequency of adjustment in inverse proportion to the deterioration.

[0052] According to the present embodiment described above, the retraining conditions are determined based on the local and global results, and the strength of the retraining conditions is adjusted. This allows efficient retraining while controlling the deviation between the retrained model and the global model, and ultimately allows a retrained model that achieves higher performance than the local and global results, regardless of the relative merits of the local and global results.

[0053] An example of the hardware configuration of the learning device 100 and the server 11 of the local device 10 according to the above embodiment is shown in the block diagram of FIG. The learning device 100 and the server 11 include a CPU (Central Processing Unit) 41, a RAM (Random Access Memory) 42, a ROM (Read Only Memory) 43, storage 44, a display device 45, an input device 46, and a communication device 47, each of which is connected by a bus.

[0054] The CPU 41 is a processor that executes arithmetic processing, control processing, etc. according to a program. The CPU 41 uses a predetermined area of ​​the RAM 42 as a working area and executes the processing of each part of the learning device 100 and the server 11 described above in cooperation with the programs stored in the ROM 43 and the storage 44.

[0055] The RAM 42 is a memory such as an SDRAM (Synchronous Dynamic Random Access Memory), and functions as a work area for the CPU 41. The ROM 43 is a memory that stores programs and various information in a non-rewritable manner.

[0056] The storage 44 is a device that writes and reads data to and from a magnetic recording medium such as a hard disk drive (HDD), a semiconductor storage medium such as a flash memory, or an optically recordable storage medium, etc. The storage 44 writes and reads data to and from the storage medium in accordance with control from the CPU 41.

[0057] The display device 45 is a display device such as an LCD (Liquid Crystal Display), etc. The display device 45 displays various information based on a display signal from the CPU 41. The input device 46 is an input device such as a mouse, a keyboard, etc. The input device 46 receives information input by a user as an instruction signal, and outputs the instruction signal to the CPU 41. The communication device 47 communicates with external devices via a network under the control of the CPU 41 .

[0058] The instructions shown in the processing procedures described in the above-described embodiments can be executed based on a software program. A general-purpose computer system can store this program in advance and, by loading this program, achieve effects similar to those achieved by the control operations of the learning system (local device and server) described above. The instructions described in the above-described embodiments can be recorded as a computer-executable program on a magnetic disk (such as a flexible disk or hard disk), an optical disk (such as a CD-ROM, CD-R, CD-RW, DVD-ROM, DVD±R, DVD±RW, or Blu-ray Disc), a semiconductor memory, or a similar recording medium. The recording medium may take any storage format as long as it is readable by a computer or embedded system. A computer can load the program from the recording medium and execute the instructions described in the program on a CPU based on the program, thereby achieving operations similar to those of the learning system (local device and server) described in the above-described embodiments. Of course, the computer may acquire or load the program via a network. In addition, an OS (operating system), database management software, network middleware, etc. running on a computer may execute some of the processes required to realize this embodiment based on instructions from a program installed on the computer or embedded system from a recording medium. Furthermore, the recording medium in this embodiment is not limited to a medium independent of a computer or an embedded system, but also includes a recording medium that stores or temporarily stores a program downloaded via a LAN, the Internet, or the like. Furthermore, the number of recording media is not limited to one, and cases where the processing in this embodiment is executed from multiple media are also included in the recording media in this embodiment, and the media may have any configuration.

[0059] The computer or embedded system in this embodiment is for executing each process in this embodiment based on a program stored on a recording medium, and may be configured as either a device consisting of a single device such as a personal computer or a microcomputer, or a system in which multiple devices are connected to a network. Furthermore, the computer in this embodiment is not limited to a personal computer, but also includes an arithmetic processing unit, a microcomputer, etc. included in information processing equipment, and is a general term for equipment or devices that can realize the functions in this embodiment by a program.

[0060] Although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, and are also included in the scope of the invention and its equivalents as defined in the claims. [Explanation of symbols]

[0061] 1···Learning system, 10···Local device, 11···Server, 100···Learning device, 101···Training unit, 102···Decision unit, 103···Retraining unit, 104, 113···Communication unit, 105···Storage unit, 111···Storage unit, 112···Update unit, 41···CPU, 42···RAM, 43···ROM, 44···Storage, 45···Display device, 46···Input device, 47···Communication device.

Claims

1. a decision unit that decides retraining conditions based on a local result obtained by training the local model and a global result obtained by updating parameters of the global model through federated learning; a retraining unit that retrains a model for retraining based on the retraining conditions; A learning device comprising:

2. The learning device according to claim 1 , wherein the determination unit determines the retraining condition based on a determination of whether or not the retraining model and the global model are to diverge.

3. The determination unit If the global result is better than the local result, determining the strength of the retraining condition such that the retrained model receives a smaller amount of update from the global model; The learning device according to claim 1 , wherein if the local result is superior to the global result, the strength of the retraining condition is determined so that the amount of update from the global model in the retraining model is large.

4. The learning device according to claim 1 , wherein the retraining conditions are training conditions having parameters that can be adjusted during model training.

5. The learning device according to claim 1 , wherein the retraining condition is at least one of a learning rate, a number of epochs, a regularization method, distillation, and adjustment of a model structure.

6. The determination unit If the global result is better than the local result, setting the learning rate of the retrained model to be less than a predetermined value; The learning device of claim 1 , wherein the learning rate is set to be greater than the predetermined value if the local result is better than the global result.

7. The determination unit If the global result is better than the local result, setting the number of epochs for retraining the retraining model to be less than a predetermined value; The learning device according to claim 1 , wherein the number of epochs is set to be greater than the predetermined value if the local result is superior to the global result.

8. The determination unit If the global result is better than the local result, setting a regularization strength for retraining the retrained model higher than a predetermined value; The learning device according to claim 1 , wherein the regularization strength is set lower than the predetermined value if the local result is better than the global result.

9. The determination unit If the global result is superior to the local result, a value of a temperature parameter related to the intensity of the distillation is set higher than a predetermined value in the distillation using the global model as a teacher model; The learning device of claim 1 , wherein if the local result is better than the global result, the value of the temperature parameter is set lower than the predetermined value.

10. The learning device according to claim 1 , wherein the determination unit sets parameters of the global model to initial values ​​of parameters of the retraining model.

11. The determination means determines a retraining condition based on a local result obtained by training the local model and a global result obtained by updating the parameters of the global model through federated learning; A learning method, wherein a retraining means retrains the model for retraining based on the retraining conditions.

12. Computer, a decision means for deciding a retraining condition based on a local result obtained by training the local model and a global result obtained by updating the parameters of the global model through federated learning; a learning program for functioning as a retraining means for retraining a model for retraining based on the retraining conditions;

Images