Calculation device
By optimizing data transfer policies for IoT devices using reinforcement learning or genetic algorithms, the accuracy of machine learning models is enhanced in federated learning systems while respecting bandwidth constraints.
Patent Information
- Application Number
- JP2024061474
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2024-04-05
- Publication Date
- 2025-10-17
AI Technical Summary
Existing federated learning technologies fail to determine an optimal data transfer policy for IoT devices to improve machine learning model accuracy while adhering to bandwidth constraints.
A computing device determines parameters for data transfer to minimize generalization error by solving an optimization problem considering bandwidth constraints and resource block usage, using reinforcement learning or genetic algorithms to specify the optimal server and resource block for each IoT device.
Improves the accuracy of machine learning models trained collaboratively by multiple servers by optimizing data transfer policies for IoT devices within bandwidth limitations.
Smart Images
Figure 2025158685000001_ABST
Abstract
Description
[Technical Field]
[0001] The present invention relates to a technology in which data generated by IoT (Internet of Things) devices is collected in a distributed manner by multiple servers, and machine learning models are learned collaboratively by these servers. [Background technology]
[0002] In recent years, federated learning has been proposed as a method for training machine learning models in a distributed manner, in which model parameters are trained individually on each server and then periodically synchronized (see Non-Patent Document 1). In federated learning, the amount and distribution of data collected by each server affects the trained model that is ultimately obtained. On the other hand, when attempting to regularly send data generated by each IoT device to a server, it is necessary to consider the wireless bandwidth. [Prior art documents] [Non-patent literature]
[0003] [Non-Patent Document 1] Y. Zhao, M. Li, L. Lai, N. Suda, D. Civin, and V. Chandra, "Federated learning with non-iid data," arXiv preprint arXiv:1806.00582, 2018. Summary of the Invention [Problem to be solved by the invention]
[0004] To improve the accuracy of machine learning models trained by federated learning while satisfying bandwidth constraints, it is necessary to determine a transfer policy for determining which server the data generated by IoT devices should be transferred to. However, there is no conventional technology for determining an appropriate transfer policy.
[0005] The present invention has been made in consideration of the above points, and aims to provide a technology for determining to which server each IoT device should transfer data so as to improve the accuracy of the machine learning model when multiple servers collaboratively train a machine learning model using data generated from multiple IoT devices. [Means for solving the problem]
[0006] According to the disclosed technology, in a system in which a model is learned by performing collaborative learning among a plurality of servers using data generated by a plurality of devices, a computing device determines parameters indicating to which server each device should transfer data, the computing device comprising: a control unit that determines the parameters so as to minimize the generalization error of the model obtained by the collaborative learning; A computing device is provided, comprising: [Effects of the Invention]
[0007] According to the disclosed technology, when multiple servers collaboratively train a machine learning model using data generated from multiple IoT devices, it is possible to determine to which server each IoT device should transfer data so as to improve the accuracy of the machine learning model. [Brief explanation of the drawings]
[0008] [Figure 1] 1 is a diagram showing the overall configuration of a learning system according to an embodiment of the present invention; [Figure 2] FIG. 1 is a diagram illustrating a configuration of an IoT device 10. [Figure 3] FIG. 2 is a diagram showing the configuration of a server 20. [Figure 4] FIG. 2 is a diagram showing the configuration of a server 20. [Figure 5] FIG. 2 is a diagram showing the configuration of a calculation device 30. [Figure 6] 10 is a flowchart illustrating the operation of the learning system. [Figure 7] FIG. 2 illustrates an example of a hardware configuration of the apparatus. DETAILED DESCRIPTION OF THE INVENTION
[0009] Hereinafter, an embodiment of the present invention (the present embodiment) will be described with reference to the drawings. The embodiment described below is merely an example, and the embodiment to which the present invention is applied is not limited to the following embodiment.
[0010] In the text of this specification (other than the mathematical formulas inserted in images), normal font is used for characters that represent sets, vectors, matrices, etc. It is clear from the context that they represent sets, vectors, matrices, etc. Also, in the text of this specification, a hat (^), intended to be placed above a letter, is written before the letter.
[0011] (Outline of the embodiment) First, we will explain the problem in more detail and then provide an overview of the technology to solve it. When data generated by IoT devices is distributed and aggregated across multiple servers and machine learning models are trained collaboratively on those servers, improving training accuracy requires each server to transfer as much data as possible from the IoT devices while also homogenizing the data across the multiple servers. However, with conventional technology, IoT devices communicate with, for example, the nearest server (with good radio wave conditions), which allows for the transfer of large amounts of data. However, because a server can only communicate with one IoT device at a time, IoT devices do not necessarily transfer data to the optimal server, making it impossible to improve training accuracy.
[0012] To solve the above problems, in this embodiment, under the constraints that the communication bandwidth between the server and the IoT device is limited by the distance between them and that the server can only communicate with one IoT device at a time, parameters indicating which server and which resource block each IoT device should use for data transfer are obtained by solving an optimization problem in order to improve the accuracy of the model. Based on these parameters, the server and resource block to which each IoT device should transfer data are specified.
[0013] (System configuration example) FIG. 1 shows an example of the overall configuration of a learning system according to this embodiment. As shown in FIG. 1, this learning system includes multiple IoT devices 10 and multiple servers 20. Each IoT device 10 is, for example, a sensor device, but is not limited to a specific device and may be any device that generates and transmits (transfers) data. Each server 20 includes a wireless base station (communication device) and is capable of communicating with each IoT device 10 and each other server 20.
[0014] Each server 20 receives data from one or more IoT devices 10, uses the data to individually learn the model parameters of the machine learning model, and periodically synchronizes the model parameters with other servers 20 to advance the learning.
[0015] The transfer policy of "to which server each IoT device 10 should transfer data" may be determined by one of the servers 20, or may be determined by a computing device 30 shown in Fig. 1, which is a device other than the server 20. Note that the server 20 that determines the transfer policy may also be called the "computing device 30."
[0016] Below, we will first explain how to determine the transfer policy, and then explain the configuration of each device in the learning system. Note that in the explanation of how to determine the transfer policy, the entity that determines the transfer policy will be the "computing device 30." Also, for the sake of convenience, the IoT devices and servers will be explained without the reference numerals 10 and 20.
[0017] (How forwarding policies are determined) Assume that there is a set I={1,2,...,N} of N IoT devices and a set B={1,2,...,M} of M servers with attached wireless base stations. Each IoT device shares a set R={1,2,...,K} of K resource blocks (RBs) when performing uplink communication to transfer data to one of the servers. Each IoT device uses one of the resource blocks to transfer data to the server.
[0018] In this embodiment, two parameters to be optimized in determining the transfer policy are U={u i}={u ij}∈{0,1} N×M ,V={v i}={v ik}∈{0,1} N×K Let's say. ij =1 indicates that IoT device i is communicating with server j and transferring data it has generated, and v ik =1 indicates that IoT device i is communicating using RBk.
[0019] The following four constraints are required from general constraints regarding communication between IoT devices and servers and the use of RBs: The following constraints indicate that "each IoT device communicates with one server using one resource block when transferring data, and one resource block can only be used by one IoT device at a time."
[0020]
number
[0021]
number
[0022]
number
[0023]
number
[0024]
number
[0025]
number
[0026]
number
[0027] IoT device i is constantly i Assuming that the amount of data generated by server j is RBk, the bandwidth constraints for being able to transfer this data to server j are given as follows: Equation (8) below shows that the bandwidth when IoT device i attempts to communicate with server j using RBk is equal to or greater than the amount of data.
[0028]
number
[0029] In this embodiment, a supervised classification problem of C types of labels is considered as the problem to be solved by the machine learning model to be trained, and the loss function used for training is the cross-entropy loss. Regarding the data generated by IoT device i, the data generation distribution of the classification labels is defined as p i (c)=p ic Let c∈C={1, ,C} and ∀i∈IΣ c=1 C p ic = 1). In this case, the label distribution of the data collected by server j is expressed by the following formula (9). Formula (9) corresponds to the average of the label distribution for all data collected by server j.
[0030]
number
[0031]
number
[0032]
number
[0033]
number
[0034]
number
[0035] Generally, models trained by collecting data in one place are more accurate than models trained by federated learning. Therefore, in order to reduce the (A) term, we need to reduce the model h F The parameters w included in F A Model H C The parameter w C It is necessary to get closer to
[0036] According to Non-Patent Document 1 (Y. Zhao, M. Li, L. Lai, N. Suda, D. Civin, and V. Chandra, "Federated learning with non-iid data," arXiv preprint arXiv:1806.00582, 2018.), wF and w C The distance between j (c), p(c), |D j |, |D| is related to |D| as shown in the following equation (14).
[0037]
number
[0038] Next, we will explain term (B) in equation (13). We use the idea of PAC-Bayesian to explain the difference between the generalization error and empirical loss of the federated learning model shown in term (B). According to the reference (DA McAllester, "PAC-Bayesian stochastic model selection," Machine Learning, vol. 51, pp. 5-21, 2003.), there is a relationship between the generalization error and empirical loss as shown in the following equation (15).
[0039]
number
[0040] In this embodiment, the amount of data collected by each server is the subject of consideration, so we extract only the part of equation (15) related to the amount of data, and assume that term (B) in equation (13) depends on √(log|D| / 2|D|).
[0041] From the above, it is considered that the accuracy of the model learned by federated learning in the problem setting in this embodiment depends on the following equation (16).
[0042]
number
[0043] The proposed formula in this embodiment for improving the accuracy of a model trained by federated learning is as follows, taking into account the connection between the IoT device and the server, the use of RB, bandwidth constraints, etc. The proposed formula below indicates that the calculation device 30 determines U and V that minimize equation (16) under constraints (1) to (4) and (8).
[0044]
number
[0045] Furthermore, in this embodiment, when solving an optimization problem using reinforcement learning, a genetic algorithm, or the like, in order to find a solution that satisfies the constraints, the following function may be used, which is obtained by adding an equation that takes into account the penalty in the case of violating the constraints to equation (16): The calculation device 30 can calculate an appropriate solution that takes the constraints into account by minimizing the following function:
[0046]
number
[0047] Here, cost is the predetermined penalty strength, and max(0,U T The 0s and 1s in V-1) are j×k matrices with all elements being 0 and 1, respectively, and the max function is taken element by element. H(·) is also a step function. A step function is a function that outputs 0 when the input value to the function is less than 0, and outputs 1 when the input value is 0 or greater.
[0048] (Example) As an embodiment, the configuration and operation of each device shown in Fig. 1 will be described. Fig. 2 shows the configuration of an IoT device 10. As shown in Fig. 2, the IoT device 10 has a data generation unit 11 and a data transfer unit 12.
[0049] 3 shows the configuration of the server 20. As shown in FIG. 3, the server 20 includes a database 21, a data receiving unit 22, a local learning unit 23, a model transfer unit 24, and an inference unit 25.
[0050] FIG. 4 shows the configuration of the server 20 when the server 20 performs a process for determining a transfer policy (specifically, a solution process for finding parameters that minimize Equation (18)). As shown in FIG. 4, the server 20 in this case includes a control unit 26 in addition to the configuration shown in FIG. 3. The control unit 26 may also be included in a computing device 30 separate from the server 20, and the configuration of the computing device 30 in that case is shown in FIG. 5. As mentioned above, the server 20 including the control unit 26 may also be called the "computing device 30."
[0051] The computing device 30 is a device that determines parameters indicating to which server 20 each IoT device 10 should transfer data in a system that learns a model by performing collaborative learning among multiple servers 20 using data generated in multiple IoT devices 10. The control unit 26 determines the parameters so as to minimize the generalization error of the model obtained by collaborative learning.
[0052] More specifically, the control unit 26 determines the parameters so as to minimize the sum of a first value (e.g., the first and second terms on the right-hand side of equation (16)) based on the difference in empirical loss between a model obtained when data generated by multiple IoT devices 10 is virtually collected in one place and learned and a model obtained by collaborative learning, and a second value (e.g., the third term on the right-hand side of equation (16)) based on the difference between the generalization error and empirical loss in the model obtained by collaborative learning.
[0053] The control unit 26 can calculate the second value using the total amount of data collected in all servers. The control unit 26 can also determine the parameters so as to minimize a function obtained by adding, as a penalty, constraints on the communication between each IoT device 10 and each server 20 and the use of resource blocks (the second and third terms on the right side of equation (18)) to a function corresponding to the sum of the first value and the second value (e.g., the first term on the right side of equation (18)).
[0054] The operation of the system will be described according to the procedure of the flowchart shown in Fig. 6. In S1 (step 1), the control unit 26 of the computing device 30 (or a specific server 20) executes a process for solving the optimization problem (minimization problem of equation (18)) described above, and calculates parameters. In S2, the control unit 26 transmits instructions to each IoT device 10 regarding the data transfer destination and the RB to be used, based on the obtained parameters.
[0055] In S3, the data generation unit 11 of each IoT device 10 generates data, and the data transfer unit 12 transfers the data to the server 20 in accordance with instructions from the control unit 26. In S4, the data reception unit 22 in each server 20 arranges (stores) the data constantly sent from the IoT device 10 in the database 21.
[0056] In S5, the local learning unit 23 and the model transfer unit 24 learn a machine learning model by federated learning using the data received from the IoT device 20 during the period T and stored in the database 21. The trained model (specifically, model parameters) is stored in the database 21. In S6, when a request for inference processing is made to the server 20, the inference unit 25 performs inference on the desired data using the trained model.
[0057] The data used by the control unit 26 of the computing device 30 (or a specific server 20) to find a solution that minimizes equation (18) may be input into the computing device 30 (or a specific server 20) in advance and stored in a storage unit such as a memory in the computing device 30 (or a specific server 20), or the computing device 30 (or a specific server 20) may collect the data from each server 20.
[0058] (Example of hardware configuration) Any of the devices described in this embodiment (such as the IoT device 10, the server 20, and the computing device 30) can be realized by, for example, causing a computer to execute a program. This computer may be a physical computer or a virtual machine on the cloud.
[0059] That is, the device can be realized by executing a program corresponding to the processing performed by the device using hardware resources such as a CPU and memory built into a computer. The program can be recorded on a computer-readable recording medium (such as a portable memory) and stored or distributed. The program can also be provided via a network such as the Internet or email.
[0060] Fig. 7 is a diagram showing an example of the hardware configuration of the computer. The computer in Fig. 7 includes a drive device 1000, an auxiliary storage device 1002, a memory device 1003, a CPU 1004, an interface device 1005, a display device 1006, an input device 1007, an output device 1008, and the like, all of which are interconnected via a bus B. The computer may further include a GPU.
[0061] A program for realizing processing on the computer is provided by a recording medium 1001 such as a CD-ROM or a memory card. When the recording medium 1001 storing the program is set in the drive device 1000, the program is installed from the recording medium 1001 to the auxiliary storage device 1002 via the drive device 1000. However, the program does not necessarily have to be installed from the recording medium 1001, but may be downloaded from another computer via a network. The auxiliary storage device 1002 stores the installed program as well as necessary files, data, etc.
[0062] The memory device 1003 reads and stores a program from the auxiliary storage device 1002 when an instruction to start the program is received. The CPU 1004 realizes the functions related to the device in accordance with the program stored in the memory device 1003. The interface device 1005 is used as an interface for connecting to a network, etc. The display device 1006 displays a GUI (Graphical User Interface) or the like according to the program. The input device 1007 is composed of a keyboard, mouse, buttons, a touch panel, etc., and is used to input various operation instructions. The output device 1008 outputs the results of calculations.
[0063] (Effects of the embodiment) As described above, the technology described in this embodiment makes it possible, when multiple servers collaboratively train a machine learning model using data generated from multiple IoT devices, to determine to which server each IoT device should transfer data so as to improve the accuracy of the machine learning model.
[0064] The following additional notes are provided regarding the above-described embodiments.
[0065] <Additional Notes> (Additional note 1) In a system for learning a model by performing collaborative learning with a plurality of servers using data generated by a plurality of devices, a computing device is provided for determining parameters indicating to which server each device should transfer data, the computing device comprising: a control unit that determines the parameters so as to minimize the generalization error of the model obtained by the collaborative learning; A computing device comprising: (Additional note 2) The control unit determines the parameters so as to minimize the sum of a first value based on a difference in empirical loss between a model obtained when learning is performed by virtually gathering data generated by a plurality of devices in one place and a model obtained by the collaborative learning, and a second value based on a difference between a generalization error and an empirical loss in the model obtained by the collaborative learning. Item 1. The computing device of item 1. (Additional note 3) The control unit calculates the second value using a total amount of data collected by all servers. Item 2. The computing device of item 2. (Additional note 4) The control unit determines the parameters so as to minimize a function obtained by adding, as a penalty, constraints on communication between each device and each server and use of resource blocks to a function corresponding to the sum of the first value and the second value. Item 2. The computing device of item 2. (Additional note 5) A non-transitory storage medium storing a program for causing a computer to function as the computing device described in any one of appendixes 1 to 4.
[0066] Although the present embodiment has been described above, the present invention is not limited to such a specific embodiment, and various modifications and changes are possible within the scope of the gist of the present invention described in the claims. [Explanation of symbols]
[0067] 10 IoT devices 11 Data Generation Unit 12 Data transfer section 20 servers 21 Databases 22 Data receiving unit 23 Local Learning Department 24 Model Transfer Unit 25 Reasoning part 26 Control Unit 30 Computing equipment 1000 Drive Device 1001 Recording media 1002 Auxiliary storage device 1003 Memory device 1004 CPU 1005 Interface device 1006 Display device 1007 Input Device 1008 Output Device
Claims
1. In a system for learning a model by performing collaborative learning with a plurality of servers using data generated by a plurality of devices, a computing device is provided for determining parameters indicating to which server each device should transfer data, the computing device comprising: a control unit that determines the parameters so as to minimize the generalization error of the model obtained by the collaborative learning; A computing device comprising:
2. The control unit determines the parameters so as to minimize the sum of a first value based on a difference in empirical loss between a model obtained when learning is performed by virtually collecting data generated by a plurality of devices in one place and a model obtained by the collaborative learning, and a second value based on a difference between a generalization error and an empirical loss in the model obtained by the collaborative learning. The computing device of claim 1 .
3. The control unit calculates the second value using a total amount of data collected by all servers.
3. The computing device of claim 2.
4. The control unit determines the parameters so as to minimize a function obtained by adding, as a penalty, constraints on communication between each device and each server and use of resource blocks to a function corresponding to the sum of the first value and the second value.
3. The computing device of claim 2.