Learning systems and methods

Abstract

To realize federated learning in consideration of data, which is important regardless of the number of pieces of data.SOLUTION: A learning system comprises a plurality of local devices and a server. Each of the local devices includes a local generation section and a local communication section. The local generation section generates local data information relating to the local data included in a mini-batch used to update a local model and indicating information different from a label. The local communication section transmits a local model parameter relating to the local model and the local data information to the server. The server includes a global calculation section and a global update section. The global calculation section calculates an integrated parameter using the local data information acquired from each of the plurality of local devices. The global update section updates a global model using the integrated parameter and the local model parameter acquired from each of the local devices.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] Embodiments of the present invention relate to learning systems and methods. [Background technology]

[0002] One learning method called Federated Learning involves training a machine learning model (local model) based on training data acquired from multiple local devices, and then sending the parameters of the trained local models to a server. The server aggregates and integrates the parameters of each local model and updates the machine learning model (global model) on the server. The updated global model's parameters are then distributed to each of the multiple local devices. This series of processes is repeated. In federative learning, the computational load can be distributed because training is performed on multiple local devices. Furthermore, only parameters are exchanged with the server, so the training data itself is not exchanged. Therefore, it has the advantages of high privacy confidentiality and low communication costs. However, information based on a small amount of important data, such as anomalous data, on each local device belonging to each environment is diluted when it is integrated on the server, making it difficult to perform training that takes this small amount of important data into account. [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] Patent No. 6893874 [Overview of the Initiative] [Problems that the invention aims to solve]

[0004] This disclosure is made to solve the aforementioned problems and aims to provide a learning system and method that can achieve federative learning that takes important data into consideration, regardless of the amount of data. [Means for solving the problem]

[0005] The learning system according to this embodiment includes a plurality of local devices and a server. Each of the plurality of local devices includes a local selection unit, a local learning unit, a local generation unit, and a local communication unit. The local selection unit selects a minibatch from local data. The local learning unit updates the local model using the minibatch. The local generation unit generates local data information that indicates information different from labels regarding the local data contained in the minibatch. The local communication unit transmits local model parameters and the local data information regarding the local model to the server. The server includes a global calculation unit and a global update unit. The global calculation unit calculates integrated parameters using the local data information obtained from each of the plurality of local devices. The global update unit updates the global model using the local model parameters and integrated parameters obtained from each of the plurality of local devices. [Brief explanation of the drawing]

[0006] [Figure 1] A conceptual diagram showing the learning system according to this embodiment. [Figure 2] A flowchart illustrating the learning process of the learning system according to the first embodiment. [Figure 3] A flowchart illustrating the learning process of the learning system according to the second embodiment. [Figure 4] A block diagram showing an example of the hardware configuration of local devices and servers. [Modes for carrying out the invention]

[0007] The learning system and method according to this embodiment will be described in detail below with reference to the drawings. In the following embodiments, parts with the same reference numerals perform the same operation, and redundant explanations will be omitted as appropriate.

[0008] (First Embodiment) The learning system according to the first embodiment will be described with reference to the block diagram of FIG. 1. The learning system according to the first embodiment includes local devices 10A, local device 10B, and server 11, which are respectively connected via network NW so as to be able to transmit and receive data. Here, as an example, two local devices 10A and local device 10B are shown, but three or more local devices 10 may be included. Also, in the case of explanations common to each local device, it is simply referred to as local device 10.

[0009] Each local device 10 includes a local storage unit 101, a local selection unit 102, a local learning unit 103, a local generation unit 104, and a local communication unit 105.

[0010] The local storage unit 101 stores local data, which is training data, local labels assigned to the local data, attribute labels, and local models. In this embodiment, the local data is assumed to be inspection images of factory-produced products. The local label is, for example, if the local data is an inspection image, the category classification of defects (scratches, stains, deformations, etc.) associated with the inspection image. The attribute label is information accompanying the local data that is different from the local label. For example, it is the importance of an event individually assigned to the local data. For example, a label to which information regarding a good product with particularly excellent performance or a defective product with poor performance among manufactured products is assigned becomes the attribute label. The local model is, for example, a neural network and is trained to be able to perform a classification task of classifying inspection images into good products and defective products. Note that the task of the local model is not limited to the classification task, and can be any task such as object detection, semantic segmentation, action recognition, anomaly detection, suspicious person detection, regression, prediction, etc. Also, the training data and the input data when executing the trained local model are not limited to images, and can also be time series data such as sound, operation sounds such as machine sounds, environmental sounds, acceleration data, instrument data, etc., and any data that can be handled by machine learning is acceptable.

[0011] The local selection unit 102 selects a mini-batch from the local data. For example, the mini-batch may be selected from the local data by random sampling. The local learning unit 103 updates the local model by training the local model using the mini-batch.

[0012] The local generation unit 104 generates local data information indicating information different from the label regarding the local data included in the mini-batch. The local data information is, for example, information representing the characteristics of the local data included in the mini-batch used for updating the local model. Note that the local generation unit 104 may generate information regarding the label of the local data such as the frequency distribution of the labels instead of the local data information. The local communication unit 105 transmits the local model parameters and the local data information regarding the local model to the server 11. The local model parameters are a set of parameters (such as weight coefficients and biases) of the neural network for sharing parameters with the global model. The local communication unit 105 receives the global model from the server 11.

[0013] The server 11 includes a global storage unit 111, a global calculation unit 112, a global update unit 113, a global control unit 114, and a global communication unit 115.

[0014] The global storage unit 111 stores global model and local data information. The global model is, for example, a neural network model. The global calculation unit 112 calculates integrated parameters using local data information received from multiple local devices 10.

[0015] The global update unit 113 updates the global model using local model parameters and local data received from multiple local devices 10. The global control unit 114 controls the storage of local data history data in the global storage unit 111, as well as the operation of the server 11. The global communication unit 115 receives local model parameters and local data information from multiple local devices 10. The global communication unit 115 transmits the updated global model to each local device 10.

[0016] Examples of local and global models include convolutional neural networks (CNNs), multilayer perceptrons (MLPs), recurrent neural networks (RNNs), Transformers, and BERT (Bidirectional Encoder Representations from Transformers), and any other neural network commonly used in machine learning is acceptable. Furthermore, it is not limited to neural network models, but is applicable to all machine learning models to which associative learning can be applied. For example, models such as SVM (Support Vector Machine) and Random Forest are also acceptable.

[0017] Next, the learning process of the learning system 1 according to the first embodiment will be described with reference to the sequence diagram in Figure 2. Unless otherwise specified, each of the multiple local devices 10 will execute the process according to the sequence diagram.

[0018] In step SA1, the local communication unit 105 of the local device 10 receives the global model from the server 11. The global model is, for example, a set of parameters of a neural network shared by the local device 10. The parameters are values that are randomly initialized at the start of learning or values that are pre-trained using the oven dataset. In step SA2, the local selection unit 102 of the local device 10 selects a mini-batch. Here, the local selection unit 102 selects a subset by random sampling from the local data, and selects the local label and the attribute label associated with the data selected as the subset as the mini-batch.

[0019] In step SA3, the local learning unit 103 of the local device 10 updates the local model by training the local model using the mini-batch. Specifically, the input image input to the local model is x → ij (i = 1, 2, 3, …, j = 1, …, N i ) Let it be. Here, the upper arrow indicates that the data to which the arrow is attached is tensor data. i is the serial number identifying the local device 10, j is the serial number of the training data, and N i is a natural number of 2 or more representing the number of training data sampled by the i-th local device 10. Also, the input image x → ij is a set of pixels with a horizontal width W and a vertical height H, and is two-dimensional tensor data.

[0020] The target label for the input image x → ij is represented as t → ij The target label t → ij is an M-dimensional vector in which the corresponding element is 1 and the other elements are zero. M is the number of classification types and is a natural number of 2 or more.

[0021] The input to the local model is the input image x →ij , the output of the local model is y → ij Therefore, it can be expressed by equation (1).

[0022] y → ij =f(x → ij )···(1) Here, f() represents the neural network function relating to the local model.

[0023] Also, the learning error L ij This is expressed by equation (2). L ij =-t → ij T ln(y → ij )···(2) Here, the learning error L ij This is calculated using cross-entropy. In each local device 10, the local learning unit 103 calculates the average of the learning errors of each of the multiple input images related to a minibatch as the loss, and updates the parameters of the neural network for the local model, for example, by backpropagation and stochastic gradient descent, in order to minimize this loss.

[0024] Furthermore, when updating local models, to absorb differences in data distribution (data properties) among multiple local devices 10, personalization techniques for each local device 10, such as introducing individualization layers, meta-learning, or distillation, may be employed. For example, the input layer of each local model may be set as a layer with parameters unique to each local model, or part of the intermediate layer may be set as a layer with parameters unique to each local model. Also, if each local model includes a normalization layer, the normalization layer may be set as a layer with parameters unique to each local model.

[0025] Thus, instead of directly copying the received global model to the local model, the global model may be transformed according to the personalization method, and the local model may be updated accordingly.

[0026] In step SA4, the local generation unit 104 of the local device 10 generates local data information from the selected mini-batch. The local data information includes, in this case, at least one of the following regarding the local data contained in the mini-batch: the frequency distribution of attribute labels, the frequency distribution of losses for the local model, the frequency distribution of losses for the global model, and statistical values. For example, the mean, maximum value, and median may be used as statistical values.

[0027] In step SA5, the local learning unit 103 of the local device 10 determines whether the learning of the local model has finished. For example, it may be determined that learning has finished when the parameters have been updated a predetermined number of times, or when the absolute value or sum of the absolute values ​​of the parameter updates reaches a certain value. Note that the determination of whether learning has finished is not limited to the above examples, but may also be made using termination conditions commonly adopted in machine learning. If learning has finished, the process proceeds to step SA6; otherwise, it returns to step SA2 and the same process is repeated.

[0028] In step SA6, the local communication unit 105 of the local device 10 sends local parameters and local data information related to the local model whose training has been completed to the server 11. The local model parameters may be, for example, the updated parameter values, or the amount of change due to the update, for example, the difference between the parameter values ​​before the update and the parameter values ​​after the update. The local communication unit 105 may also compress the data related to the parameter set to be sent to the server 11 before sending it. The data compression process may be lossless or lossy. By compressing the data before sending it, communication volume and communication bandwidth can be saved. The data may also be encrypted before sending it, which can improve data confidentiality.

[0029] In step SA7, the global communication unit 115 of the server 11 receives local model parameters and local data information from each local device 10. In step SA8, the global calculation unit 112 of the server 11 generates integrated parameters based on the local data information received from each local device 10. The integrated parameters can be calculated, for example, by determining the weight of each local model parameter when updating by weighted averaging of local model parameters. For example, if the local data information is attribute labels and is the maximum or sum of importance values ​​assigned to each local data in a mini-batch, then a value proportional to that value can be calculated as the integrated parameter.

[0030] Furthermore, if frequency distribution information of local labels is received instead of local data information, the integrated parameters are calculated so that the sum of products of the integrated parameters and the frequency distribution is uniform. Specifically, if the first local device 10 is updated using 10 good sample data and the second local device 10 is updated using 5 defective sample data, the local model parameters of the first local device 10 are multiplied by 1 and the local model parameters of the second local device 10 are multiplied by 2, and the integrated parameters are calculated by averaging the respective local model parameters. This avoids the result that the recognition rate of local labels with low occurrence frequency is not adequately considered. Note that the target frequency distribution does not have to be uniform and may be designed to follow a predetermined prior distribution. In this way, even with a small amount of local data, it is possible to update the model while considering important local data. Furthermore, by calculating a value proportional to the loss statistics for the local model or global model as an integrated parameter, the influence of local model parameters trained with a large amount of hard-to-identify local data can be greatly increased, and it is expected that the training of both the global model and the local model will converge more quickly.

[0031] In step SA9, the global update unit 113 of the server 11 updates the global model based on the integration parameters. The global model update can be achieved by replacing the local model parameters of each local device with a weighted average value obtained by the integration parameters, or by using a moving average of the weighted average value and the global model before the update. In step SA10, the global update unit 113 of the server 11 determines whether federated learning has finished. Whether federated learning has finished can be determined, for example, when the performance of the local models of each local device 10, such as recognition rate, accuracy, precision, and recall, reaches the target value, when the global model has been updated a predetermined number of times, or when the update range of the global model has converged to below a threshold. If federated learning has finished, the learning process is terminated; otherwise, the process proceeds to step SA11.

[0032] In step SA11, the global communication unit 115 of the server 11 transmits information about the global model to each local device 1. The information about the global model includes, for example, the global model itself, the updated parameter values, and at least one of the amounts of change due to the update. In step SA12, the local communication unit 105 of the local device 10 receives information about the global model from the server 11. After that, the processes from step SA2 to step SA11 should be repeated until federated learning is complete.

[0033] According to the first embodiment described above, during federated learning, the local device calculates local data information related to the mini-batch, and the server updates the global model taking the local data information into consideration. This allows even small amounts of data to have a significant impact on model training if they are important, and enables federated learning that considers important data regardless of the amount of data.

[0034] (Second Embodiment) In the first embodiment, the global model is updated by local model parameters unilaterally transmitted from the local device, and therefore information bias is not controlled. The second embodiment differs from the first embodiment in that it transmits selection request information regarding the request for selected information from the server to each local device.

[0035] The learning process of the learning system according to the second embodiment will be explained with reference to the flowchart in Figure 3. Note that the processing of local device 10 (steps SA3 to SA10) is the same as in the first embodiment, so its explanation is omitted here.

[0036] In step SB1, the local communication unit 105 of the local device 10 receives the global model and selection request information. The selection request information is information for controlling the local data information. For example, if the local data information is the frequency distribution of local labels within a mini-batch, and the local labels are information about good and defective products, then the selection request information could be the mixing ratio of good and defective products, or the specific mixing ratio of defective types. The selection request information may also represent a set of mini-batches used in the past, and may be an index of the number of communications between the server 11 and the local device 10, that is, an index showing the number of times parameters related to model updates have been exchanged.

[0037] In step SB2, the local selection unit 102 of the local device 10 selects a mini-batch from the training data based on the selection request information. Specifically, if the selection request information is the frequency distribution of local labels within the mini-batch, the local selection unit 102 selects a subset of the training data to form a mini-batch that matches the label distribution indicated in the selection request information. If the selection request information is an index indicating the number of communications, the local selection unit 102 simply selects the corresponding training data.

[0038] In step SB3, the global control unit 114 of server 11 calculates selection request information for each local device 10 for the next update. For example, if the received local data information includes a frequency distribution of local labels, and this frequency distribution is biased, and the information desired by server 11 is missing, server 11 may not be able to calculate the integration parameters that result in the target frequency distribution. In such cases, the global control unit 114 calculates the target frequency distribution as selection request information, thereby enabling the local device 10 to select a mini-batch of local data that results in the target frequency distribution. Directions for what kind of learning to perform for the next update can be determined, for example, by looking at the local data information stored in the global storage unit 111. If, for example, a frequency distribution of local labels is included, local labels with a small number of data points can be identified. Therefore, it is sufficient to generate selection request information that includes an instruction to include the desired local labels in a mini-batch.

[0039] In step SB4, the global communication unit 115 of the server 11 transmits information about the global model and selection request information to each local device 10. In step SB5, the local communication unit 105 of the local device 10 receives information about the global model and selection request information. Subsequently, steps SB1, SB2, SA3 through SA10, SB3, and SB4 can be repeatedly executed until federated learning is completed.

[0040] Furthermore, the global control unit 114 of the server 11 may, after sending the selection request information to the local device 10, determine whether the local data information sent from the local device 10 satisfies the request. For example, if the local data information sent from the local device 10 differs from the selection request information, for instance, if the difference between the frequency distribution of the selection request information and the frequency distribution of the local data information is greater than or equal to a threshold, the weights of the local model parameters from the local device 10 may be reduced to generate integrated parameters. This allows the server 11 to appropriately control the training trend (direction of updates) of the model based on the selection request information.

[0041] According to the second embodiment described above, in addition to enabling federated learning that considers important data regardless of the amount of data, it is possible to train a local model on a local device with a mini-batch containing the desired data type and to control the direction of model updates, thereby enabling effective federated learning.

[0042] In the above embodiment, the local data was assumed to be inspection images from a factory, and the task was assumed to be a classification task to identify good and defective products. However, the local data is not limited to this, and the task could also be an OCR (Optical Character Recognition) task in which handwritten character images are used to output text. In this case, the local data information should include type information such as the type of field, for example, a postal code which is relatively easy to recognize, an address which is relatively difficult to recognize, or free text, as well as information about a flag indicating whether the user has corrected the character recognition result recognized by a general machine learning model. Furthermore, local data may simply be document data, in which case the local data information should include flag information such as importance associated with the document and keywords associated with the document.

[0043] Furthermore, local data may be medical-related data, such as medical images. The local model may be a model that takes medical-related data as input and outputs image processing results and image recognition results. In this case, the local data information may include, for example, device information indicating the performance of the equipment, such as whether it is a high-end or low-end model, information on discrepancies between image findings and biological examinations, and information on the course of prognosis.

[0044] Furthermore, local data may be sales data for products, such as customer data or product purchase data. The local model may be a model that takes sales data as input and outputs recommendation information, predictive information, etc., for products and services. In this case, the local data information may include attribute information such as the season, region, and age group in which the product was purchased.

[0045] Furthermore, the local data may also be video data from a dashcam. The local model may be a model that takes video acquired by a camera and acceleration information of the vehicle body acquired by an accelerometer as input and outputs the result of the accident determination. In this case, the local data information may include, for example, information such as the driver's age group, driving period, insurance coverage amount, remaining points on the license, location, and time of day. Furthermore, local data may also consist of camera images from the robot's picking process. In this case, the local data information should include information such as past picking success rates, weight, size, price, fragility, and inventory levels of the target product.

[0046] Furthermore, local data may also include camera images and sensor data from the vehicle's autonomous driving. In this case, the local data information should include information such as the accident rate at the driving location, the time of year (season, time of day, etc.), weather, road surface information, number of lanes, number of vehicles, and latitude / longitude / altitude. Furthermore, local data may also be video data from surveillance cameras. In this case, the local data information should include information such as the time of year, weather, and, in the case of surveillance cameras installed in stores, the level of crowding, sales data, and whether or not an event is taking place. Furthermore, local data may also be access data from websites or applications accessed via PCs or smartphones. In this case, local data information may include information such as the number of views, browsing history, and, in the case of websites or applications that play videos, the number of views, playback history, age group, access time, period, and global affairs.

[0047] Furthermore, in the above embodiment, it is assumed that the local model of the local device 10 and the global model of the server 11 are neural networks with basically the same structure. However, the local models of multiple local devices 10 may each be scalable neural networks that share some parameters with the neural network of the global model. A scalable neural network is a neural network that allows the model size, such as the number of convolutional layers in the network model, to be adjusted according to the required computational load or performance. For example, if the local models are different in each local device 10, this indicates that the model structure, model size, number of parameters such as weight coefficients and biases will be different. The server 11 can update the parameters of the global model and send a portion of the global model to each local device 10 according to the scale of each local model. The global control unit 114 of the server 11 may also include information regarding how much computational load is required to train the local model in the selection request information and send it to the local device 10. On the other hand, the local generation unit 104 of the local device 10 may generate local data information based on the computational cost during training, and the local communication unit 105 may send this local data information to the server 11.

[0048] Here, an example of the hardware configuration of the local device 10 and server 11 according to the above embodiment is shown in the block diagram of Figure 4. The local device 10 and the server 11 each include a CPU (Central Processing Unit) 41, RAM (Random Access Memory) 42, ROM (Read Only Memory) 43, storage 44, display device 45, input device 46, and communication device 47, and are connected to each other by bus.

[0049] The CPU 41 is a processor that performs arithmetic and control processing according to a program. The CPU 41 uses a predetermined area of ​​the RAM 42 as a working area and, in cooperation with programs stored in the ROM 43 and storage 44, executes the processing of each part of the local device 10 and server 11 described above.

[0050] RAM42 is a type of memory such as SDRAM (Synchronous Dynamic Random Access Memory). RAM42 functions as a workspace for the CPU41. ROM43 is a memory that stores programs and various information in a non-rewritable format.

[0051] The storage device 44 is a device that writes and reads data from magnetic recording media such as HDDs (Hard Disk Drives), semiconductor storage media such as flash memory, or optically recordable storage media. The storage device 44 writes and reads data from the storage media in response to control from the CPU 41.

[0052] The display device 45 is a display device such as an LCD (Liquid Crystal Display). The display device 45 displays various information based on display signals from the CPU 41.

[0053] The input device 46 is an input device such as a mouse and a keyboard. The input device 46 receives information input by the user as an instruction signal and outputs the instruction signal to the CPU 41.

[0054] The communication device 47 communicates with external devices via a network in response to control from the CPU 41.

[0055] The instructions shown in the processing procedure described in the above-described embodiment can be executed based on a software program. A general-purpose computer system can store this program in advance and, by reading this program, can obtain effects similar to those of the control operation of the learning system (local device and server) described above. The instructions described in the above-described embodiment are recorded as a program that can be executed by a computer on a magnetic disk (flexible disk, hard disk, etc.), optical disk (CD-ROM, CD-R, CD-RW, DVD-ROM, DVD±R, DVD±RW, Blu-ray® Disc, etc.), semiconductor memory, or similar recording medium. Any storage format is acceptable as long as it is a recording medium that can be read by a computer or embedded system. The computer can read the program from this recording medium and, based on this program, have the CPU execute the instructions described in the program, thereby achieving operations similar to the control of the learning system (local device and server) in the above-described embodiment. Of course, when the computer acquires or reads the program, it may do so via a network. Furthermore, an operating system (OS) running on a computer, a database management software, a network, or other middleware (MW) operating on a computer, based on instructions from a program installed on a computer or embedded system from a recording medium, may execute some of the processes necessary to realize this embodiment. Furthermore, the recording medium in this embodiment is not limited to a medium independent of the computer or embedded system, but also includes a recording medium that stores or temporarily stores a program that has been downloaded via a LAN, the Internet, or the like. Furthermore, the recording medium is not limited to one; even when the processing in this embodiment is performed from multiple media, these are also included as recording media in this embodiment, and the configuration of the media may be any configuration.

[0056] In this embodiment, the computer or embedded system is used to execute each process in this embodiment based on a program stored on a recording medium, and may be configured as any of the following: a single device such as a personal computer or microcontroller, or a system in which multiple devices are connected via a network. Furthermore, the term "computer" in this embodiment is not limited to personal computers, but also includes arithmetic processing units, microcontrollers, and the like included in information processing equipment, and refers collectively to any equipment or device capable of realizing the functions of this embodiment through a program.

[0057] While several embodiments of the present invention have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These novel embodiments can be carried out in a variety of other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents. [Explanation of symbols]

[0058] Learning system…1, 10, 10A, 10B…Local devices, 11…Server, 101…Local storage unit, 102…Local selection unit, 103…Local learning unit, 104…Local generation unit, 105…Local communication unit, 111…Global storage unit, 112…Global calculation unit, 113…Global update unit, 114…Global control unit, 115…Global communication unit, 41…CPU, 42…RAM, 43…ROM, 44…Storage, 45…Display device, 46…Input device, 47…Communication device

Claims

1. A learning system including multiple local devices and a server, Each of the multiple local devices is: A local selection section for selecting a mini-batch from local data, A local learning unit that updates the local model using the aforementioned mini-batch, A local generation unit generates local data information that indicates information different from the label regarding the local data included in the mini-batch, The system comprises a local communication unit that transmits local model parameters and local data information relating to the local model to the server, The aforementioned server, A global calculation unit that uses the local data information acquired from each of the aforementioned multiple local devices to calculate integrated parameters for increasing the influence of important local data, A global update unit updates the global model using the local model parameters and integrated parameters obtained from each of the aforementioned multiple local devices, A global control unit generates selection request information for controlling the training trend of the local model based on the history of the local data information, A global communication unit that transmits the selection request information to the plurality of local devices, A learning system equipped with the following features.

2. The learning system according to claim 1, wherein the local data information is at least one of the frequency distribution of attribute information, the frequency distribution of loss for the local model, the frequency distribution of loss for the global model, and statistical values ​​relating to the local data included in the minibatch.

3. The learning system according to claim 1, wherein the global calculation unit generates the integrated parameters by weighting and averaging the local model parameters based on the local data information.

4. Each of the multiple local devices is: The learning system according to claim 1, wherein the local selection unit selects a mini-batch based on the selection request information.

5. The aforementioned local model is a scalable neural network with adjustable computational cost, The local learning unit trains the local model, The learning system according to claim 1, wherein the local generation unit calculates local data information including the computational complexity of the local model based on the number of parameters of the local model.

6. The aforementioned server, The system further comprises a global communication unit that transmits request information regarding computation costs to a designated local device. The predetermined local device is The learning system according to claim 5, which trains the local model based on the requested information.

7. A learning method relating to a learning system including multiple local devices and a server, In the aforementioned multiple local devices, Select a mini-batch from local data, The local model is updated using the aforementioned mini-batch. Local data information is generated that shows information different from the label regarding the local data included in the mini-batch. The local model parameters and local data information relating to the local model are transmitted to the server. In the aforementioned server, Using the local data information acquired from each of the aforementioned multiple local devices, an integrated parameter is calculated to increase the influence of important local data. The global model is updated using the local model parameters and integrated parameters obtained from each of the aforementioned multiple local devices. Based on the history of the local data information, select request information is generated to control the training trend of the local model. A learning method that transmits the selection request information to the multiple local devices.

Images