Method for training global model by using training contribution of client in federated learning

Abstract

A server for training on the basis of federated learning, according to the present invention, may comprise: a communication unit that receives, from each of a plurality of clients, information regarding a weight of a trained model and the number of pieces of training data; and a processor that calculates a corresponding training contribution of each of the clients, and calculates a weight of a global model on the basis of the corresponding weight received from each of the clients, the number of pieces of corresponding training data, and the corresponding training contribution calculated for each of the clients.

Description

How to train a global model using client learning contributions in federated learning.

[0001] The present invention relates to federated learning, and more particularly, to a method for learning a global model using the learning contributions of clients in federated learning.

[0002] Federated learning is a distributed machine learning technique that enables AI models to be trained collaboratively without directly sharing data stored in multiple locations. Federated learning is one method for training deep learning models. Data quality and quantity are crucial for deep learning model training. Typically, data is collected on a single server and refined for use in deep learning model training. However, for data that is difficult to export externally due to its nature, or for data related to an individual's private life, moving it directly to a server poses a privacy risk.

[0003] Federated learning is a method for training deep learning models using this data, allowing models to be trained without leaking data from users' devices.

[0004] Since only information from the local model and global model is exchanged between the server and the local client, data collected by the local client never leaves the management of the local client, so sensitive information related to the user can be protected.

[0005] However, as the number of clients participating in federated learning increases, there are problems such as increasing network costs for exchanging models and learning parameters, and bottlenecks caused by differences in performance among clients, which slows down the learning progress.

[0006] However, little research has been conducted to address these issues.

[0007] The technical task to be achieved in the present invention is to provide a server with a federated learning structure.

[0008] Another technical challenge to be achieved by the present invention is to provide a method for a server in a federated learning structure to update a global model.

[0009] Another technical problem to be achieved by the present invention is to provide a computer-readable recording medium having recorded thereon a program for executing a method of learning based on federated learning by a server on a computer.

[0010] The technical problems to be achieved in the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned can be clearly understood by a person having ordinary skill in the technical field to which the present invention belongs from the description below.

[0011] In order to achieve the above technical task, a server of a federated learning structure according to the present invention may include a communication unit that receives information on the weights of a learned model and the number of learning data from a plurality of clients; and a processor that calculates a corresponding learning contribution for each client and calculates a weight of a global model based on the corresponding weights, the corresponding number of learning data, and the corresponding learning contributions calculated for each client.

[0012] The processor may calculate the corresponding learning contribution based on the accuracy ratio or loss rate of the weights of the learning model of the corresponding client. The processor may update the global model based on the calculated global model weights.

[0013] The server further includes a memory electrically connected to the processor and storing the updated global model, wherein the communication unit can transmit the weights of the updated global model to each client.

[0014] The above processor can calculate the weight of the global model by using the total sum obtained by multiplying the corresponding learning contribution calculated for each client by the number of corresponding learning data.

[0015] The above processor can calculate the weight of the global model based on the following mathematical expressions A and B.

[0016] [Mathematical Formula A]

[0017]

[0018] [Equation B]

[0019]

[0020] Here, K is the number of clients participating in learning, n k is the number of client learning data, w k is the weight of the client learning model, r k is the client's learning contribution, w g is the weight of the global model.

[0021] The above processor can calculate the corresponding learning contribution based on the following mathematical formulas C and D.

[0022] [Mathematical Formula C]

[0023]

[0024] [Mathematical Formula D]

[0025]

[0026] Here, K is the number of clients participating in learning, a k is the accuracy of the client learning model weights, r k is the client's learning contribution.

[0027] The above processor can calculate the corresponding learning contribution based on the following mathematical formulas E and F.

[0028] [Mathematical formula E]

[0029]

[0030] [Mathematical Formula F]

[0031]

[0032] Here, K is the number of learning participating clients, l k is the loss rate of the client model weights, r k is the client's learning contribution.

[0033] In order to achieve the above-described other technical task, a method for a server to update a global model in a federated learning structure may include the steps of: receiving information on the weights and the number of learning data of a model learned from each of a plurality of clients; calculating a corresponding learning contribution for each client; and calculating a weight of the global model based on the corresponding weights, the corresponding number of learning data, and the corresponding learning contribution calculated for each client received for each client.

[0034] The step of calculating the learning contribution may be calculated based on the accuracy ratio or loss rate of the weight of the learning model of the client.

[0035] The step of calculating the weight of the above global model may include a step of multiplying the corresponding learning contribution calculated for each client by the number of corresponding learning data and then adding them all together.

[0036] The step of calculating the weight of the above global model can be calculated based on the following mathematical formulas A and B.

[0037] [Mathematical Formula A]

[0038]

[0039] [Equation B]

[0040]

[0041] Here, K is the number of clients participating in learning, nk is the number of client learning data, w k is the weight of the client learning model, r k is the client's learning contribution, w g is the weight of the global model.

[0042] In the present invention, the server improves learning efficiency by replacing the FedAVG algorithm model's equalization method with an equalization method that compensates for the learning contribution of each client.

[0043] Increasing the weight of models of clients with high learning contributions during the global model leveling process can improve learning efficiency and enhance the performance of the entire federated learning system by reducing the number of rounds.

[0044] As learning efficiency improves, network costs can be reduced by reducing the total number of rounds required for learning or by reducing the number of clients participating in learning.

[0045] Additionally, by calculating the learning contribution of each client, it is possible to secure a basis for providing compensation based on contribution.

[0046] The effects that can be obtained from the present invention are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by a person having ordinary skill in the art to which the present invention belongs from the description below.

[0047] Figure 1 is a diagram illustrating the layer structure of an artificial neural network.

[0048] Figure 2 is a diagram illustrating an example of a deep neural network.

[0049] Figure 3 is an example diagram for explaining the server-client federated learning structure.

[0050] Figure 4 is a drawing illustrating a block for explaining the function of a server (400).

[0051] Figure 5 is an exemplary diagram for explaining a server-client federated learning method according to the present invention.

[0052] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. The detailed description set forth below, together with the accompanying drawings, is intended to illustrate exemplary embodiments of the present invention and is not intended to represent the only embodiments in which the present invention may be practiced. The following detailed description includes specific details to provide a thorough understanding of the present invention. However, one of ordinary skill in the art will appreciate that the present invention may be practiced without these specific details.

[0053] In some cases, to avoid ambiguity in the concepts of the present invention, well-known structures and devices may be omitted or illustrated in block diagram form focusing on the core functions of each structure and device. Furthermore, the same components are described using the same reference numerals throughout this specification.

[0054] The present invention is susceptible to various modifications and embodiments. Specific embodiments are illustrated and described in detail in the drawings. However, this is not intended to limit the present invention to specific embodiments, but rather to encompass all modifications, equivalents, and alternatives falling within the spirit and technical scope of the present invention.

[0055] When a component is referred to as being "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but that there may be other components intervening. Conversely, when a component is referred to as being "directly connected" or "connected" to another component, it should be understood that there are no other components intervening.

[0056] Terms such as first, second, etc. may be used to describe various components, but these components should not be limited by these terms. These terms are used solely to distinguish one component from another.

[0057] The terminology used herein is for the purpose of describing specific embodiments only and is not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this specification, it should be understood that the terms "comprises" or "has" indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but do not exclude in advance the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0058] In addition, it is to be understood that the components of the embodiments described with reference to each drawing are not limited to the specific embodiments, but may be implemented to be included in other embodiments within the scope in which the technical idea of ​​the present invention is maintained, and that multiple embodiments may be re-implemented as a single integrated embodiment even if a separate description is omitted.

[0059] Additionally, terms such as “part,” “unit,” “module,” and “device” described in the specification mean a unit that processes at least one function or operation, which may be implemented by hardware, software, or a combination of hardware and software.

[0060] The present invention proposes a method for learning a global model based on the contribution of local parameters learned from multiple distributed devices or servers in federated learning. Before describing the present invention, we briefly review the deep learning models used to train the global model.

[0061] deep learning

[0062] Artificial neural networks, another algorithm developed by early machine learning researchers, were inspired by the biological properties of the human brain, specifically the interconnected structure of its neurons. However, unlike the brain, where any physically adjacent neurons can be interconnected, artificial neural networks have a fixed layer connection and data propagation direction.

[0063] For example, if an image is sliced ​​into numerous tiles and input into the first layer of a neural network, the neurons in that tile will repeatedly pass the data on to the next layer until the final output is generated at the last layer. Each neuron is then assigned a weight representing the accuracy of the input based on the task it performs, and the final output is then determined by adding all the weights together. In the case of a stop sign, the characteristics of the image, such as the octagonal shape, the red color, the characters displayed, the size, and the presence of movement, are sliced ​​and "examined" by the neurons, and the neural network's task is to identify whether it is a stop sign. Here, a "probability vector" is utilized, which predicts the outcome based on the weights based on sufficient data.

[0064] Deep learning is a form of artificial intelligence developed from artificial neural networks. It utilizes input-output layers of information, similar to neurons in the brain, to learn from data. However, even basic neural networks require enormous computational loads, hindering the commercialization of deep learning from the beginning. Despite this, researchers continued their research and, using supercomputers, successfully parallelized algorithms that proved the concept of deep learning. The advent of GPUs, optimized for parallel computing, dramatically accelerated neural network computation, ushering in the emergence of true deep learning-based artificial intelligence.

[0065] Deep learning is a type of artificial neural network (ANN) that utilizes the theory of the human neural network (Neural Network). It is a set of machine learning models or algorithms that refer to a deep neural network (DNN) that is structured in a layer structure and has one or more hidden layers (hereinafter referred to as intermediate layers) between the input layer and the output layer. Simply put, deep learning can be said to be an artificial neural network with a deep layer.

[0066] In a deep learning model, a layer is a layer that generates different output values ​​when receiving computer data, such as images. This algorithm stacks multiple layers to perform complex nonlinear modeling. Each layer has parameters, and the output values ​​vary depending on these parameters. By updating these parameters using input data, the deep learning model learns.

[0067] The human brain is estimated to be composed of 25 billion nerve cells. The brain is made up of nerve cells, and each nerve cell (neuron) refers to a single nerve cell that forms a neural network. A nerve cell contains a cell body, an axon (or nucleus), and usually multiple dendrites (or protoplasmic processes). Information is transmitted between these nerve cells through synapses, the connections between nerve cells. While a single nerve cell appears very simple, when these nerve cells come together, they are capable of human intelligence. Dendrites are the part that receives signals from other nerve cells (input), and the axon is the very long part that extends from the cell body and transmits signals to other nerve cells (output). Synapses, the connections between axons and dendrites that transmit signals between nerve cells, do not transmit signals unconditionally. Instead, they only transmit signals if the signal strength exceeds a certain value (threshold). In other words, not only does each synapse have a different connection strength, but it also determines whether or not a signal will be transmitted.

[0068] Artificial neural networks (ANNs), a branch of artificial intelligence, are mathematical models modeled after the structure of the biological (typically human) brain. In other words, ANNs mimic the information processing and transmission processes of biological neurons. Similar to how the human brain solves problems, ANNs exhibit excellent parallelism because each neuron operates independently. Furthermore, because information is distributed across numerous connections, problems in a few neurons do not significantly impact the overall system. Consequently, ANNs are robust to a certain level of error and possess the ability to learn from a given environment.

[0069] Deep neural networks (DNNs) are the latest evolution of artificial neural networks, transcending existing limitations and achieving success in areas where numerous AI technologies have previously failed. Looking at the modeling of artificial neural networks based on biological neural networks, biological neurons are modeled as nodes, and connections are modeled as weights (synapses), as shown in Table 1.

[0070] Biological neural network Artificial neural network Cell body Node Dendrite Input Axon Output Synapse Weight

[0071] Figure 1 is a diagram illustrating the layer structure of an artificial neural network.

[0072] Just as human biological neurons are connected in groups rather than in isolation to perform meaningful tasks, artificial neural networks are also connected in multiple layers by connecting individual neurons to each other through synapses, and the connection strength between each layer can be updated using weights.

[0073] In this way, it is used in the fields of learning and cognition with a multi-layered structure and connection strength.

[0074] Each node is connected by weighted links, and the entire model learns by repeatedly adjusting the weights. Weights are the basic means of long-term memory and express the importance of each node. Simply put, an artificial neural network trains the entire model by initializing these weights and updating and adjusting them with the training data set. After training is complete, when a new input value is received, the appropriate output value is inferred. The learning principle of an artificial neural network can be viewed as a process in which intelligence is formed through the generalization of experience and is performed in a bottom-up manner. In Figure 1, when there are two or more intermediate layers (i.e., 5 to 10), it is considered deep and is called a deep neural network. Learning and inference models achieved through such a deep neural network can be referred to as deep learning.

[0075] While artificial neural networks can perform to some extent with a single intermediate layer (commonly referred to as a hidden layer) beyond the input and output layers, as the problem complexity increases, the number of nodes or layers must be increased. While increasing the number of layers to achieve a multilayered model is effective, its application is limited due to the impossibility of efficient learning and the large computational load required to train the network.

[0076] However, by overcoming these limitations, artificial neural networks have been able to achieve deep structures. This has enabled the construction of complex, highly expressive models, leading to groundbreaking results in diverse fields such as speech recognition, face recognition, object recognition, and character recognition.

[0077] Figure 2 is a diagram illustrating an example of a deep neural network.

[0078] A deep neural network (DNN) is an artificial neural network (ANN) with multiple hidden layers between the input layer and the output layer. It is a collection of machine learning models or algorithms that refer to deep neural networks (DNNs) with one or more hidden layers between the input layer and the output layer. The connections in the neural network are made from the input layer to the hidden layer, and from the hidden layer to the output layer.

[0079] Deep neural networks, like typical artificial neural networks, can model complex non-linear relationships. For example, in a deep neural network architecture for object recognition, each object can be represented as a hierarchical structure of basic image elements. Additional layers can then gradually aggregate features from lower layers. This characteristic of deep neural networks allows them to model complex data with a smaller number of units (nodes) compared to similarly implemented artificial neural networks.

[0080] While previous deep neural networks were typically designed as feed-forward neural networks, recent research has successfully applied deep learning structures to recurrent neural networks (RNNs). For example, deep neural network architectures have been applied to language modeling. Convolutional neural networks (CNNs) have been successfully applied to computer vision, with each successful application well documented. More recently, CNNs have been applied to acoustic modeling for Automatic Speech Recognition (ASR), and are considered more successful than existing models. Deep neural networks can be trained using the standard error backpropagation algorithm, with weights updated via stochastic gradient descent.

[0081] Federated learning is a distributed learning technique that allows devices and organizations to collaborate and train AI models without directly sharing data stored in multiple locations. Rather than sending each client's individual training data to a central server, federated learning sends AI models from the cloud to each client, where they learn from each client's data. The local models trained on each client are then transferred to the cloud, where they are normalized to create a global model. By repeating this process, AI models can be generalized and efficiency in terms of analysis time and network costs can be significantly improved.

[0082] FIG. 3 is an exemplary diagram for explaining a federated learning structure of a server-client, FIG. 4 is an exemplary diagram of a block for explaining the function of a server (400), and FIG. 5 is an exemplary diagram for explaining a federated learning method of a server-client according to the present invention.

[0083] Referring to Fig. 3, federated learning is one of the methods for training a deep learning model without data leakage from an individual's device. A common deep learning model is transmitted from a server (which can be referred to in various ways, such as a central server or a global server) (400) to (local) clients (individual devices). The local model (i.e., trained weight parameters) (W1, W2) trained on each client is transmitted to the server (400), and training is performed by updating the parameters of the global deep learning model in the server (400) (e.g., calculating the average value).

[0084] As a result, the server (400) can train a model without directly accessing data, and since it does not directly access personal information, personal information can be protected.

[0085] Referring to Fig. 3, the federated learning structure is largely divided into server-client, sequential model, peer-to-peer, and clustering model, but only the server-client structure is described here for convenience of explanation. Fig. 3 schematically illustrates the server-client structure. When the server (400) distributes the global model (or learning parameters) to the clients, each client performs learning with local data and then transmits the learning parameters (e.g., weights) to the server (400). The server (400) updates the global model using the learning parameters received from each client and a federated learning algorithm such as FedAVG (Federated Averaging) or FedSGD (Federated stochastic gradient descent). That is, the server (400) updates the global model based on the received learning parameters. The server (400) transmits the learning parameters of the global model to the clients and repeats the same process.

[0086] For convenience of explanation, the present invention only describes the FedAVG algorithm model. The FedAVG algorithm model transmits the weights, which are the learning parameters of the model after each client has repeatedly learned a certain number of times, to the server, and the server normalizes the collected weights. The formulas for obtaining the weights of the global model by normalizing the weights of the clients on the server are as follows: Equations 1 and 2. When calculating the average of the weights, it is not a simple average of the client weights, but a weighted average that reflects the number of client training data. In other words, the normalized weights reflect higher weights for clients with more training data, and reflect lower weights for clients with less training data.

[0087]

[0088]

[0089] Here, K is the number of learning participating clients, n k is the number of client learning data,

[0090] w k is the client's model weight, w g is the global model weight.

[0091] Federated learning allows for training global models without sharing training data held by different organizations, and research on federated learning is being conducted across various industries. In particular, the energy and healthcare sectors can leverage personal data to create personalized business models based on federated learning while maintaining privacy protection. However, as the number of clients participating in federated learning increases, network costs for exchanging models and training parameters increase, and bottlenecks due to performance differences among clients can occur, slowing down the learning process. To address these issues, the present invention proposes that the server (400) evaluate the trained weights sent by each client to calculate the learning contribution. Furthermore, the present invention proposes that the server (400) replace the conventional FedAVG algorithm model's equalization method with an equalization method that compensates for the learning contribution of each client.

[0092] Referring to FIG. 4, the server (400) may include a processor (410), a communication unit (420), and a memory (430). The processor (410) calculates a learning contribution for each client in a federated learning environment, calculates a correction weight for each client, and then determines a weight of a global model based on the correction weight calculated for each client. The communication unit (420) may exchange information (e.g., a global deep learning model, weight parameters updated through learning, the number of learning data, etc.) with each client through transmission and reception. The memory (430) may be electrically coupled to the processor (410) and store various information necessary for computing, such as information necessary for the processor (520) to estimate a result, information about the estimated result, and the like. Hereinafter, a method for learning a global model based on contributions of local parameters learned from multiple distributed clients or servers in federated learning as illustrated in FIG. 5 will be described with reference to this.

[0093] The communication unit (420) of the server (400) can transmit a global (deep learning) model for learning to a plurality of clients (client A (100), client B (200)) (S510). Here, the plurality of clients are only exemplified as client A (100), client B (200)) for convenience of explanation, and the number is not limited. Client A (100), client B (200)) each performs learning of the global model (S520). Hereinafter, client A (100), client B (200)) can each transmit learning parameters (e.g., weights of the learned global model, number of learning data, etc.) of the learned global model to the server (400) (S530).

[0094] Then, the processor (410) of the server (400) can calculate the corresponding learning contribution for each client (100, 200) (S540). In the present invention, a new algorithm that improves the leveling method of the existing FedAVG algorithm model is applied to calculate the learning contribution. The processor (410) uses the improved FedAVG algorithm model, and the improved FedAVG algorithm model is a leveling method that additionally reflects the number of learning data of the clients who participated in learning and the learning contribution of each client. The learning contribution can be calculated based on the loss rate or accuracy evaluated with test data and the weight of the client's model.

[0095] In the FedAVG algorithm, accuracy evaluated on test data is a metric used to measure the performance of the entire model trained on distributed clients. Accuracy indicates how well the entire model generalizes by incorporating learning from local data on distributed clients. Accuracy evaluated on test data indicates how accurately the entire model predicts on test data, which is used to measure the model's generalization ability. Higher test data accuracy indicates better generalization and more accurate predictions on new data. The goal of the FedAVG algorithm is to effectively train models on distributed data and integrate these models to improve generalization performance. Therefore, accuracy evaluated on test data is a crucial metric.

[0096] Specifically, the accuracy evaluated with test data for the weights of the client's model can be calculated by the following mathematical expressions 3 and 4.

[0097]

[0098]

[0099] Here, K is the number of clients participating in learning, ak is the accuracy of the client learning model weights, and rk is the client's learning contribution.

[0100]

[0101] Since the higher the accuracy, the higher the learning contribution, the processor (410) can first calculate the sum of the accuracies of all clients as in Equation 3, and then calculate the learning contribution as the ratio of each client's accuracy to the sum. Equation 5 means that the sum of the learning contributions of all clients is 1.

[0102] The FedAVG algorithm uses the term "loss rate" to represent the value of the model's loss function for local data on each client (or client model). This loss function measures the difference between the model's predictions and the actual labels, and is an indicator of how well the model is currently performing. Each client trains a model on its local data, and the trained model attempts to minimize the loss function for that client's training data. This loss is an indicator of the model's performance. The loss function typically measures the difference between the model's predictions and the actual labels. The loss of the model trained on each client is transmitted to the server. The server can then average the losses of the trained models on these clients to obtain the overall model loss.

[0103] In other words, "loss rate in the training data model" refers to the loss of the model trained on each client. This loss rate is averaged on the server and used as a loss function value representing the performance of the overall model. The FedAVG algorithm aims to improve the performance of the overall model while adjusting the model on the server by taking distributed data into account. Therefore, this loss is used to adjust the global model update. This can be expressed in equations 6, 7, and 8 below.

[0104]

[0105]

[0106]

[0107] Here, K is the number of learning participating clients, l k is the loss rate of the client model weights, r k is the client's learning contribution.

[0108] Since the lower the loss rate, the higher the learning contribution, the processor (410) can first obtain the sum of all reciprocals of the loss rates as in Equation 6, and then calculate the learning contribution as the ratio of the reciprocal of the loss rate of each client to the sum as in Equation 7. Equation 8 means that the sum of the learning contributions of all clients is 1.

[0109] In this way, the processor (410) can calculate the learning contribution for each client (100, 200) based on the accuracy or loss rate in the learning data model (S540). The processor (410) calculates the weight (w) of the global model for each client (100, 200) based on the corresponding weight received for each client (100, 200), the corresponding number of learning data, and the corresponding learning contribution calculated for each client (100, 200). g ) can be produced (S550).

[0110] The processor (410) can calculate the learning contribution of the client by reflecting it in the equalization formula of the FedAVG algorithm as in the following mathematical formulas 9 and 10.

[0111]

[0112]

[0113] Here, K is the number of learning participating clients, n k is the number of client learning data, w k is the weight of the client model, r kis the client's learning contribution, w g is the global model weight.

[0114] The processor (410) can calculate the N value as in Equation 9 by multiplying the number of learning data by the learning contribution for each client and then adding them up. The processor (410) calculates a weighted average by multiplying the number of learning data by the learning contribution for each client. That is, the processor (410) divides the product of the number of learning data, the learning contribution, and the weight for each client by the N, and then adds them up to calculate the weight of the global model, which can be referred to as a weighted average.

[0115] As a result, the equalized weights calculated by the above mathematical expressions 9 and 10 reflect higher weights for clients with more learning data and higher learning contributions, and reflect lower weights for clients with less learning data and lower learning contributions.

[0116] The processor (410) can update the global model by applying the weights of the calculated global model to the global model (S560). The communication unit (420) can transmit the updated weights of the global model to each client (100, 200). Each client then applies the information about the weights of the global model received from the server (400) to re-perform learning.

[0117] As discussed above, by applying a new algorithm based on learning contribution to the federated learning process, learning efficiency can be improved. This improved learning efficiency can reduce network costs by reducing the total number of rounds required for learning or the number of participating clients. This reduction in network costs and the number of participating clients can improve the overall performance of the federated learning system, including reducing bottlenecks and accelerating learning. Furthermore, calculating the learning contribution of each client provides a basis for rewarding contributions based on their contribution.

[0118] Here, as an example, the server (400) can compensate each client in proportion to the learning contribution calculated for each client, and such compensation can be performed based on a blockchain. To compensate each client for their learning contribution, the server (400) performs verification or authentication for each client and then continuously records the details of the calculated learning contribution in block units on the blockchain. Each client can verify and confirm the contribution details, so that the contribution details can be managed transparently. Regarding the server (400) in the present invention compensating each client based on their learning contribution, the contribution compensation of a network server described in Korean Patent No. 10-2016-0166101 can be applied to the present invention and includes the spirit of the invention.

[0119] The devices described above may be implemented as hardware components, software components, and / or a combination of hardware components and software components. For example, the devices and components described in the embodiments may be implemented using one or more general-purpose computers or special-purpose computers, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing instructions and responding to them. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing device is sometimes described as being used alone; however, one of ordinary skill in the art will recognize that the processing device may include multiple processing elements and / or multiple types of processing elements. For example, a processing unit may include multiple processors, or a processor and a controller. Other processing configurations, such as parallel processors, are also possible.

[0120] Software may include a computer program, code, instructions, or a combination of one or more of these, and may configure a processing device to perform a desired operation or, independently or collectively, command the processing device. The software and / or data may be permanently or temporarily embodied in any type of machine, component, physical device, virtual equipment, computer storage medium or device, or transmitted signal wave, for interpretation by the processing device or for providing instructions or data to the processing device. The software may also be distributed across network-connected computing devices and stored or executed in a distributed manner. The software and data may be stored on one or more computer-readable recording media.

[0121] The method according to the embodiment may be implemented in the form of program commands that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program commands, data files, data structures, etc., alone or in combination. The program commands recorded on the medium may be those specially designed and configured for the embodiment or may be those known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CDROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program commands, such as ROMs, RAMs, and flash memories. Examples of the program commands include not only machine language codes generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter, etc. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiment, and vice versa.

[0122] The embodiments described above are combinations of components and features of the present invention in a predetermined form. Each component or feature should be considered optional unless explicitly stated otherwise. Each component or feature may be implemented without being combined with other components or features. Furthermore, it is also possible to form an embodiment of the present invention by combining some components and / or features. The order of operations described in the embodiments of the present invention may be changed. Some components or features of one embodiment may be included in another embodiment or may be replaced with corresponding components or features of another embodiment. It is self-evident that claims that do not have an explicit citation relationship in the patent claims may be combined to form an embodiment or may be incorporated as a new claim through a post-application amendment.

[0123] In the present invention, the processor (410) may be implemented by hardware, firmware, software, or a combination thereof. When implementing an embodiment of the present invention using hardware, application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), etc. configured to perform the present invention may be provided in the processor (410).

[0124] The method of learning a global model using the learning contributions of clients in federated learning is industrially applicable to improve the learning efficiency of AI federated learning and reduce network costs.

Claims

1. In the server of the federated learning structure, A communication unit that receives information about the weights of the learned model and the number of learning data from each of multiple clients; and Calculate the learning contribution for each client above, A server including a processor that calculates weights of a global model based on the corresponding weights received for each client, the corresponding number of learning data, and the corresponding learning contribution calculated for each client.

2. In paragraph 1, The above processor is a server that calculates the corresponding learning contribution based on the accuracy ratio or loss rate of the weight of the learning model of the corresponding client.

3. In paragraph 1, The server, wherein the processor updates the global model based on the weights of the calculated global model.

4. In paragraph 3, Further comprising a memory electrically connected to the processor and storing the updated global model, The above communication unit is a server that transmits the weights of the updated global model to each client.

5. In paragraph 1, The above processor is a server that calculates the weight of the global model by multiplying the corresponding learning contribution calculated for each client by the number of corresponding learning data and then using the total sum.

6. In paragraph 5, The above processor calculates the weights of the global model based on the following mathematical formulas A and B: [Mathematical Formula A] [Equation B] Here, K is the number of clients participating in learning, n k is the number of client learning data, w k is the weight of the client learning model, r k is the client's learning contribution, w g is the weight of the global model.

7. In paragraph 2, The above processor calculates the corresponding learning contribution based on the following mathematical formulas C and D: [Mathematical Formula C] [Mathematical Formula D] Here, K is the number of clients participating in learning, a k is the accuracy of the client learning model weights, r k is the client's learning contribution.

8. In paragraph 2, The above processor calculates the corresponding learning contribution based on the following mathematical formulas E and F: [Mathematical formula E] [Mathematical Formula F] Here, K is the number of learning participating clients, l k is the loss rate of the client model weights, r k is the client's learning contribution.

9. In the federated learning structure, how the server updates the global model: A step of receiving information about the weights of the learned model and the number of learning data from each of multiple clients; A step of calculating the learning contribution for each client above; and A method comprising a step of calculating the weight of the global model based on the corresponding weight received for each client, the corresponding number of learning data, and the corresponding learning contribution calculated for each client.

10. In paragraph 9, A method in which the step of calculating the learning contribution is calculated based on the accuracy ratio or loss rate of the weight of the learning model of the client.

11. In paragraph 10, A method in which the step of calculating the weight of the global model includes the step of multiplying the corresponding learning contribution calculated for each client by the number of corresponding learning data and then summing the total.

12. In paragraph 11, The step of calculating the weight of the above global model is calculated based on the following mathematical formulas A and B: [Mathematical Formula A] [Equation B] Here, K is the number of clients participating in learning, n k is the number of client learning data, w k is the weight of the client learning model, r k is the client's learning contribution, w g is the weight of the global model.

13. A computer-readable recording medium recording a program for executing the method described in any one of Articles 9 to 12 on a computer.

Images