Information processing method, server, information processing program, and learned model related to associative learning

Abstract

To provide an information processing method, a server, an information processing program, and a learned model related to associative learning that can generate highly accurate models while protecting the privacy of learning data and reducing at least one of a load on a terminal device, an amount of data communication, and a learning time.SOLUTION: An information processing method is executed by a server 1 and a plurality of terminal devices 10, 20, and 30 communicatively connected to the server 1 via communication networks N1, N2, and N3. The information processing method includes a first associative learning process of executing associative learning using a distilled model m of a global model M, and a second associative learning process of executing associative learning using the global model M.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present disclosure relates to an information processing method, a server, an information processing program, and a trained model through federated learning using a distillation model of a global model. [Background technology]

[0002] In a centralized learning method in which learning data from multiple terminal devices is aggregated on a server and machine learning is performed on the server, problems arise with protecting the privacy of the learning data and increasing data communication volume and learning time. To address this, federated learning has been proposed, which protects the privacy of the learning data and reduces data communication volume and learning time by having the terminal devices perform machine learning (for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2023-128056 Summary of the Invention [Problem to be solved by the invention]

[0004] There is a need for a machine learning method that can generate highly accurate models while protecting the privacy of training data and reducing at least one of the load on the terminal device, data communication volume, and training time. [Means for solving the problem]

[0005] A first information processing method according to one aspect of the present disclosure includes: An information processing method executed by a server and a plurality of terminal devices communicably connected to the server via a communication network, comprising: a first associative learning step of performing associative learning using a distilled model of the global model; a second associative learning step of performing the associative learning using the global model; The first associative learning step The server replicates the distilled model as a first local model and transmits the first local model to each of the plurality of terminal devices; a step in which each of the plurality of terminal devices learns the first local model to obtain a first learned parameter; a step in which each of the plurality of terminal devices transmits the first learned parameter to the server; The server calculates a first integrated parameter from the plurality of first learned parameters, and updates the distillation model using the first integrated parameter; The second associative learning step the server replicating the global model as a second local model and transmitting the second local model to each of the plurality of terminal devices; each of the plurality of terminal devices learning the second local model to obtain second learned parameters; a step in which each of the plurality of terminal devices transmits the second learned parameter to the server; The method includes a step in which the server calculates a second integrated parameter from the plurality of second learned parameters and updates the global model using the second integrated parameter.

[0006] A second information processing method according to another aspect of the present disclosure includes: An information processing method executed by a server communicatively connected to a plurality of terminal devices via a communication network, comprising: a first associative learning step of performing associative learning using a distilled model of the global model; a second associative learning step of performing the associative learning using the global model; The first associative learning step replicating the distilled model as a first local model and transmitting the first local model to each of the plurality of terminal devices; receiving, from each of the plurality of terminal devices, first learned parameters acquired by training the first local model; calculating a first integrated parameter from the plurality of first learned parameters and updating the distillation model using the first integrated parameter; The second associative learning step replicating the global model as a second local model and transmitting the second local model to each of the plurality of terminal devices; receiving, from each of the plurality of terminal devices, second learned parameters acquired by training the second local model; and calculating a second integrated parameter from the plurality of second learned parameters and updating the global model using the second integrated parameter.

[0007] A server according to one aspect of the present disclosure includes: a communication interface communicatively connected to a plurality of terminal devices via a communication network; and a processor; The processor: a first associative learning step of performing associative learning using a distilled model of the global model; a second associative learning step of performing the associative learning using the global model; The first associative learning step replicating the distilled model as a first local model and transmitting the first local model from the communication interface to each of the plurality of terminal devices; receiving, from each of the plurality of terminal devices, first learned parameters acquired by training the first local model via the communication interface; calculating a first integrated parameter from the plurality of first learned parameters and updating the distillation model using the first integrated parameter; The second associative learning step replicating the global model as a second local model and transmitting the second local model from the communication interface to each of the plurality of terminal devices; receiving, from each of the plurality of terminal devices, second learned parameters acquired by training the second local model via the communication interface; and calculating a second integrated parameter from the plurality of second learned parameters and updating the global model using the second integrated parameter.

[0008] A computer program according to one aspect of the present disclosure includes: A computer program executable by a processor installed in a server communicatively connected to a plurality of terminal devices via a communication network, When executed, the server a first associative learning step of performing associative learning using a distilled model of the global model; a second associative learning step of performing the associative learning using the global model; The first associative learning step replicating the distilled model as a first local model and transmitting the first local model to each of the plurality of terminal devices; receiving, from each of the plurality of terminal devices, first learned parameters acquired by training the first local model; calculating a first integrated parameter from the plurality of first learned parameters and updating the distillation model using the first integrated parameter; The second associative learning step replicating the global model as a second local model and transmitting the second local model to each of the plurality of terminal devices; receiving, from each of the plurality of terminal devices, second learned parameters acquired by training the second local model; and calculating a second integrated parameter from the plurality of second learned parameters and updating the global model using the second integrated parameter.

[0009] A trained model according to one aspect of the present disclosure includes: This is a trained model trained using the first information processing method or the second information processing method. [Effects of the Invention]

[0010] According to the present disclosure, it is possible to provide an information processing method, server, information processing program, and trained model related to federated learning that can generate highly accurate models while protecting the privacy of training data and reducing at least one of the load on the terminal device, the amount of data communication, and the training time. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is an explanatory diagram illustrating a configuration example of an associative learning system according to a first comparative example. [Figure 2] 10 is a flowchart of learning in the associative learning system according to Comparative Example 1. [Figure 3] FIG. 10 is an explanatory diagram showing a configuration example of an associative learning system according to a second comparative example. [Figure 4] 10 is a flowchart of learning in an associative learning system according to a second comparative example. [Figure 5] FIG. 1 is an explanatory diagram illustrating an example of the configuration of a federated learning system according to an embodiment of the present invention. [Figure 6] 10 is a graph showing the accuracy of the trained global model over the rounds. [Figure 7] FIG. 1 is an explanatory diagram of the highest accuracy of a trained global model. [Figure 8] 10 is a graph showing the accuracy of a trained global model versus the training time. [Figure 9] FIG. 1 is an explanatory diagram of the time it takes for a trained global model to reach 70% accuracy. [Figure 10]10 is a graph showing accuracy versus data traffic required for federated learning of a trained global model. [Figure 11] This is an explanatory diagram of the accuracy of the trained global model when the data communication volume required for federated learning was 500 MB. [Figure 12] 10 is a graph showing the learning time versus the accuracy achieved by the trained global model. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. For the sake of convenience, the description of components having the same reference numerals as components already described in the description of the embodiments will be omitted.

[0013] First, in order to explain the associative learning system according to this embodiment shown in FIG. 5, associative learning systems according to comparative examples 1 and 2 will be explained using FIGS.

[0014] [Comparative Example 1] FIG. 1 is an explanatory diagram showing an example of the configuration of an associative learning system 100 according to a first comparative example. As shown in FIG. 1, a federated learning system 100 according to Comparative Example 1 includes a server 1 and multiple terminal devices (clients) 10, 20, and 30. The server 1 and the multiple terminal devices 10, 20, and 30 are communicatively connected via communication networks N1 to N3. The communication networks N1 to N3 are, for example, local area networks (LANs) and are configured using either or both wired and wireless networks. The communication networks N1 to N3 may also use a general-purpose communication protocol such as TCP / IP (Transmission Control Protocol / Internet Protocol). The server 1 has a global model M and transmits local models M1 to M3, which are copies of the global model M, to the multiple terminal devices (clients) 10, 20, and 30, respectively, and performs machine learning on the multiple terminal devices 10, 20, and 30, thereby performing federated learning. Here, federated learning includes cross-device learning and cross-silo learning. In the case of cross-device learning, the multiple terminal devices 10, 20, and 30 are, for example, IoT devices such as smartphones. In the case of cross-silo learning, the multiple terminal devices 10, 20, and 30 are servers within an organization such as a hospital. The global model M and local models M1 to M3 are neural networks configured with multiple layers. Although the example in FIG. 1 shows three terminal devices 10, 20, and 30, the number of terminal devices is not limited to three.

[0015] 2 is a flowchart of learning in the federated learning system according to Comparative Example 1. First, the server 1 replicates the global model M as local models M1 to M3 (S101). Next, the server 1 transmits the local model M1 to the terminal device 10, the local model M2 to the terminal device 20, and the local model M3 to the terminal device 30 (S102). Upon receiving the local model M1 (S103), the terminal device 10 trains the local model M1 using the training data held by the terminal device 10, acquires trained parameters (S104), and transmits the trained parameters to the server 1 (S105). Similarly, upon receiving the local model M2 (S103), the terminal device 20 trains the local model M2 using the training data held by the terminal device 20, acquires trained parameters (S104), and transmits the trained parameters to the server 1 (S105). Similarly, upon receiving the local model M3 (S103), the terminal device 30 trains the local model M3 using the training data held by the terminal device 30, acquires trained parameters (S104), and transmits the trained parameters to the server 1 (S105). Here, the trained parameters are, for example, weight parameters of a neural network.

[0016] When the server 1 receives the learned parameters from each of the terminal devices 10, 20, and 30 (S106), it calculates an integrated parameter from the learned parameters (S107). Here, the integrated parameter may be calculated as a weighted average of the learned parameters. Next, the server 1 updates the global model M using the integrated parameter (S108).

[0017] By repeatedly executing the above steps S101 to S108 (hereinafter also referred to as round R) and repeatedly updating the global model M, it is possible to improve the accuracy of the global model M. However, in the federated learning system 100 according to Comparative Example 1, when the model size of the global model M is large, the model sizes of the local models M1 to M3 also become large, which may increase the learning calculation cost of the local models M1 to M3 in the multiple terminal devices 10, 20, and 30, the amount of data communication between the server 1 and the multiple terminal devices 10, 20, and 30, and the learning time for federated learning.

[0018] Comparative Example 2 FIG. 3 is an explanatory diagram showing an example of the configuration of an associative learning system 200 according to a second comparative example. As shown in FIG. 3, the federated learning system 200 according to Comparative Example 2 has the same hardware configuration as the federated learning system 100 according to Comparative Example 1 shown in FIG. 1. In the federated learning system 200 according to Comparative Example 2, the server 1 generates a distilled model m from a global model M using learning data stored in the server 1. The server 1 then transmits local models m1 to m3, which are copies of the distilled model m, to multiple terminal devices 10, 20, and 30, respectively, and performs machine learning on the multiple terminal devices 10, 20, and 30, thereby performing federated learning. Here, the distilled model m is a pseudo model with a smaller model size than the global model M, which is created by training a new model using input / output data (e.g., learning data and prediction results) of the global model M. Note that the model size refers to the number of layers and nodes in the neural network.

[0019] 4 is a flowchart of learning in a federated learning system according to Comparative Example 2. First, the server 1 generates a distilled model m from the global model M using learning data held by the server 1 (S201), and replicates the distilled model m as local models m1 to m3 (S202). Next, the server 1 transmits the local model m1 to the terminal device 10, the local model m2 to the terminal device 20, and the local model m3 to the terminal device 30 (S203). Upon receiving the local model m1 (S204), the terminal device 10 trains the local model m1 using the learning data held by the terminal device 10, acquires learned parameters (S205), and transmits the learned parameters to the server 1 (S206). Similarly, upon receiving the local model m2 (S204), the terminal device 20 trains the local model m2 using the learning data held by the terminal device 20, acquires learned parameters (S205), and transmits the learned parameters to the server 1 (S206). Similarly, when the terminal device 30 receives the local model m3 (S204), it trains the local model m3 using the learning data that the terminal device 30 has, obtains learned parameters (S205), and transmits the learned parameters to the server 1 (S206).

[0020] When the server 1 receives trained parameters from each of the multiple terminal devices 10, 20, and 30 (S207), it calculates an integrated parameter from the multiple trained parameters (S208). Here, the integrated parameter may be calculated by a weighted average of the multiple trained parameters. Next, the server 1 updates the distilled model m using the integrated parameter (S209), and generates a global model M from the updated distilled model m by reverse distillation (S210). Here, reverse distillation means training a new model using input and output data of the distilled model m (e.g., training data and prediction results) to generate a model with a larger model size than the distilled model m.

[0021] By repeatedly executing the above steps S201 to S210 (hereinafter also referred to as rounds R) and repeatedly updating the global model M, the accuracy of the global model M can be improved. Compared to the federated learning system 100 of Comparative Example 1, the federated learning system 200 of Comparative Example 2 can reduce the computational costs of the multiple terminal devices 10, 20, and 30, the amount of data communication between the server 1 and the multiple terminal devices 10, 20, and 30, and the learning time for federated learning. However, because the federated learning system 200 of Comparative Example 2 trains a distilled model with a small model size, the accuracy of the global model M may be reduced compared to the federated learning system 100 of Comparative Example 1, and the processing load of distillation and reverse distillation on the server 1 may be increased. [Example]

[0022] Next, a description will be given of a federated learning system according to this embodiment shown in Fig. 5. Fig. 5 is an explanatory diagram showing an example of the configuration of a federated learning system according to this embodiment. As shown in FIG. 5, the federated learning system 300 according to this embodiment has the same hardware configuration as the federated learning system 100 according to Comparative Example 1 shown in FIG. 1 and the federated learning system 200 according to Comparative Example 2 shown in FIG. 3. In the federated learning system 300 according to this embodiment, first, as shown in the upper part of FIG. 5, similar to the federated learning system 200 according to Comparative Example 2 shown in FIG. 3, the server 1 generates a distilled model m from the global model M using learning data held by the server 1. Furthermore, the server 1 transmits local models m1 to m3, which are copies of the distilled model m, to each of the multiple terminal devices 10, 20, and 30, and causes each of the multiple terminal devices 10, 20, and 30 to perform machine learning, thereby performing federated learning (hereinafter also referred to as first federated learning). Next, after the server 1 has performed the first associative learning up to a predetermined round Ra, as shown in the lower part of Figure 5, similar to the associative learning system 100 of comparison example 1 shown in Figure 1, the server 1 performs associative learning by transmitting local models M1 to M3, which are copies of the global model M, to each of the multiple terminal devices 10, 20, and 30, and having each of the multiple terminal devices 10, 20, and 30 perform machine learning (hereinafter also referred to as second associative learning).

[0023] In this way, in the federated learning system 300 according to this embodiment, the first federated learning is executed up to a predetermined round Ra, and the second federated learning is executed after the predetermined round Ra. This makes it possible to improve the accuracy of the global model M and reduce the processing load of distillation and reverse distillation on the server 1 while suppressing the calculation costs of the multiple terminal devices 10, 20, and 30, the amount of data communication between the server 1 and the multiple terminal devices 10, 20, and 30, and the learning time of the federated learning.

[0024] [Performance evaluation of trained models] Next, using Figures 6 to 9, we will explain performance evaluation of the trained global model obtained by the federated learning system 100 according to Comparative Example 1, the trained global model obtained by the federated learning system 200 according to Comparative Example 2, and the trained global model obtained by the federated learning system 300 according to an embodiment of the present disclosure.

[0025] <Conditions for evaluating the performance of trained models> Specifically, in the federated learning, 200 rounds of image recognition tasks were performed using the CIFAR-10 (Canadian Institute for Advanced Research-10) image dataset as training data. In each round R, when server 1 generates distilled model m from global model M (see S201 in Figure 4), it used training data consisting of a 20,000-image dataset, and when multiple terminal devices 10, 20, and 30 train local models, they each used training data consisting of a 10,000-image dataset. In addition, each time global model M was updated in each round R, the accuracy A of the global model was measured using test data consisting of a 10,000-image dataset. In addition, the global model is a neural network consisting of 12 layers, and the distillation model is a neural network consisting of 6 layers.

[0026] <Trained model for performance evaluation> The performance evaluation targets were a trained global model produced by the federated learning system 100 according to Comparative Example 1 (hereinafter also referred to as the trained global model M_ref1), a trained global model produced by the federated learning system 200 according to Comparative Example 2 (hereinafter also referred to as the trained global model M_ref2), and a trained global model produced by the federated learning system 300 according to this embodiment (hereinafter also referred to as the trained global model M_emb). For the trained global model M_emb, the predetermined round Ra was set to 20, 40, 60, and 80, and performance evaluation was performed for each model (hereinafter the trained global models with the predetermined round Ra set to 20, 40, 60, and 80 will also be referred to as the trained global model M_emb_20, the trained global model M_emb_40, the trained global model M_emb_60, and the trained global model M_emb_80, respectively).

[0027] Furthermore, for the trained global model M_ref2 and the trained global model M_emb, the number of image data pieces in the training data used for learning in each round R is 50,000 in total: 20,000 images used when the server 1 generates the distilled model m from the global model M, and 30,000 images used by the multiple terminal devices 10, 20, and 30. In contrast, for the trained global model M_ref1, the server 1 does not generate the distilled model m from the global model M, so the number of image data pieces in the training data used for learning in each round R is 30,000 images used by the multiple terminal devices 10, 20, and 30. Therefore, it is necessary to match the number of image data pieces in the training data in each round R to evaluate the performance of the trained global model. Therefore, in the federated learning system 100 according to Comparative Example 1, a trained global model (hereinafter also referred to as the trained global model M_ref1′) in which the number of image data pieces in the training data used by the multiple terminal devices 10, 20, and 30 is 50,000 images was also used as a target model for performance evaluation.

[0028] <Performance evaluation results of trained models> FIG. 6 is a graph showing accuracy A versus round R of a trained global model. The horizontal axis represents round R, and the vertical axis represents accuracy A. Dashed lines L1 to L4 respectively represent the predetermined round Ra (=20) of the trained global model M_emb_20, the predetermined round Ra (=40) of the trained global model M_emb40, the predetermined round Ra (=60) of the trained global model M_emb_60, and the predetermined round Ra (=80) of the trained global model M_emb_80. FIG. 7 is an explanatory diagram of the maximum accuracy Amax of a trained global model. Note that maximum accuracy Amax means the highest accuracy A among rounds 1 to 200.

[0029] As shown in Figure 6, the accuracy A of each trained global model improves as the round R increases. Furthermore, from round 80 onwards, the accuracy A of the trained global model M_emb is higher than the accuracy A of the trained global model M_ref2. This is because the trained global model M_ref2 performs associative learning (second associative learning) of the distilled model m, which has a small model size, in all rounds R. In contrast, the trained global model M_emb performs associative learning (first associative learning) of the global model M, which has a large model size, from a predetermined round Ra onwards.

[0030] As shown in Figure 7, the highest accuracy Amax of each of the trained global models M_emb (trained global model M_emb_20, trained global model M_emb_40, trained global model M_emb_60, trained global model M_emb_80) is higher than the highest accuracy Amax of the trained global model M_ref2. Furthermore, among the trained global models M_emb, the highest accuracy Amax of the trained global models M_emb_60 and M_emb_80 is higher than the highest accuracy Amax of the trained global models M_emb_20 and M_emb_40. This is because the number of rounds for performing the first associative learning of the trained global models M_emb_60 and M_emb_80 is greater than the number of rounds for performing the first associative learning of the trained global models M_emb_20 and M_emb_40.

[0031] FIG. 8 is a graph comparing the accuracy of trained global models against the learning time. The horizontal axis represents learning time T, and the vertical axis represents accuracy A. Dashed lines L1 to L4 respectively represent the learning time T for the trained global model M_emb_20 up to the predetermined round Ra (=20), the learning time T for the trained global model M_emb40 up to the predetermined round Ra (=40), the learning time T for the trained global model M_emb60 up to the predetermined round Ra (=60), and the learning time T for the trained global model M_emb_80 up to the predetermined round Ra (=80). Dashed line L5 indicates the boundary where the accuracy A of the trained global model reaches 70%. FIG. 9 is an explanatory diagram of the time t for the trained global model to reach 70% accuracy.

[0032] As shown in Figure 8, the accuracy A of each trained global model improves as the learning time T increases. Furthermore, when the learning time T is 2000 seconds or more, the accuracy A of the trained global model M_emb is higher than the accuracy A of the trained global model M_ref2.

[0033] As shown in Figure 9, the time t to reach 70% accuracy for the trained global model M_emb_40, the trained global model M_emb_60, and the trained global model M_emb_80 is shorter than the time t to reach 70% accuracy for the trained global model M_ref1' and the trained global model M_ref2. This is because, in the early stages of learning when accuracy A is low (e.g., accuracy below 50%), the associative learning of the distilled model m with a small model size (second associative learning) can improve accuracy A in a shorter learning time than the associative learning of the global model M with a large model size (first associative learning), but at the stage when accuracy A is high (e.g., accuracy above 60%), the associative learning of the global model M with a large model size (first associative learning) can improve accuracy A in a shorter learning time than the associative learning of the distilled model m with a small model size (second associative learning).

[0034] Figure 10 is a graph showing accuracy A versus the amount of data communication required for federated learning of a trained global model. The horizontal axis represents data communication volume, and the vertical axis represents accuracy A. Dashed lines L1 to L4 respectively represent the data communication volume CA of the trained global model M_emb_20 up to the predetermined round Ra (=20), the data communication volume CA of the trained global model M_emb40 up to the predetermined round Ra (=20), the data communication volume CA of the trained global model M_emb60 up to the predetermined round Ra (=20), and the data communication volume CA of the trained global model M_emb_80 up to the predetermined round Ra (=20). Dashed line L6 indicates the boundary for the 500MB data communication volume CA. Dashed line L7 indicates the boundary for the 800MB data communication volume CA. Figure 11 is an explanatory diagram of the accuracy A of a trained global model when the data communication volume CA is 500MB for federated learning.

[0035] As shown in Figure 10, the accuracy A of each trained global model improves as the data communication volume CA increases. Furthermore, when the data communication volume CA is 800 MB or less (see dashed line L7), the accuracy A of the trained global model M_emb is higher than the accuracy A of the trained global model M_ref1'. This is because the trained global model M_ref1' performs federated learning (second federated learning) of the global model M, which has a large model size, in all rounds R, resulting in an increase in the data communication volume CA. In contrast, the trained global model M_emb performs federated learning (first federated learning) of the distilled model m, which has a small model size, up to a predetermined round Ra, thereby suppressing the data communication volume CA.

[0036] As shown in Figure 11, at a data communication volume of 500 MB (see L6 in Figure 10), the accuracy A of the trained global model M_emb_80 is higher than the accuracy A of the trained global model M_ref2. This is because, for example, at an accuracy level of 65% or higher, the federated learning of the global model M with a large model size (first federated learning) can improve the accuracy A with a smaller data communication volume CA than the federated learning of the distilled model m with a small model size (second federated learning).

[0037] Figure 12 is a graph showing the learning time T versus the accuracy achievement level A' of the trained global model. The horizontal axis represents the accuracy achievement level A', and the vertical axis represents the learning time T. Note that the accuracy achievement level A' represents the ratio of the accuracy A of each trained global model to the maximum accuracy Amax of the trained global model M_ref1'.

[0038] As shown in Figure 12, the accuracy achievement rate A' of the trained global model M_ref2 only reaches 88%. In contrast, the accuracy achievement rate A' of the trained global model M_emb is always 95.0% or higher, reaching a maximum of 97.4%. This is because the trained global model M_ref2 performs associative learning (second associative learning) of the distilled model m, which has a small model size, in all rounds R. In contrast, the trained global model M_emb performs associative learning (first associative learning) of the global model M, which has a large model size, from a predetermined round Ra onwards.

[0039] Furthermore, the trained global models M_emb_40, M_emb_60, and M_emb_80 have improved accuracy A in a shorter learning time T than the trained global model M_ref1'. This is because the associative learning of the distilled model m, which has a small model size (second associative learning), can improve accuracy A in a shorter learning time than the associative learning of the global model M, which has a large model size (first associative learning).

[0040] Each process in the federated learning system 300 according to the embodiment of the present disclosure can be realized as a computer program that runs on the server 1 and the multiple terminal devices 10, 20, and 30. That is, the server 1 and the multiple terminal devices 10, 20, and 30 each include a processor such as a CPU or GPU, a memory, and a communication interface that is connected to enable communication.

[0041] A computer program can be stored in a non-transitory computer-readable medium and read by a computer. Examples of non-transitory computer-readable media include magnetic recording media, magneto-optical recording media, CD-ROMs, CD-Rs, CD-R / Ws, and semiconductor memories (including EPROMs and flash ROMs). A computer program can also be read by a computer using various types of temporary computer-readable media. Examples of temporary computer-readable media include electrical signals, optical signals, and electromagnetic waves. A temporary computer-readable medium can provide a computer with a computer program via a wired communication path such as an electric wire or optical fiber, or via a wireless communication path.

[0042] As described above, the information processing method, server, information processing program, and trained model related to federated learning according to this embodiment can generate highly accurate models while protecting the privacy of the training data and reducing at least one of the load on the terminal device, the amount of data communication, and the training time.

[0043] Although the embodiments of the present invention have been described above, it goes without saying that the technical scope of the present invention should not be construed as being limited by the description of the present embodiments. The present embodiments are merely examples, and it will be understood by those skilled in the art that various modifications of the embodiments are possible within the scope of the invention described in the claims. The technical scope of the present invention should be determined based on the scope of the invention described in the claims and its equivalents.

[0044] For example, in the associative learning system 300 according to this embodiment, as shown in Fig. 5, the associative learning is performed in the order of first associative learning and second associative learning, but the associative learning may be performed in the order of second associative learning and first associative learning, or the first associative learning and second associative learning may be performed by switching between them multiple times, such as first associative learning, second associative learning, and first associative learning. Furthermore, in the first associative learning, the model size of the distilled model m may be changed as the round R increases. [Explanation of symbols]

[0045] 1: Server 10, 20, 30: Terminal device (client) M: Global model M1, M2, M3: Local models m: Distillation model m1, m2, m3: local models N1, N2, N3: Communication networks 100, 200, 300: Federated Learning Systems

Claims

1. An information processing method executed by a server and a plurality of terminal devices communicably connected to the server via a communication network, comprising: a first associative learning step of performing associative learning using a distilled model of the global model; a second associative learning step of performing the associative learning using the global model; The first associative learning step The server replicates the distilled model as a first local model and transmits the first local model to each of the plurality of terminal devices; a step in which each of the plurality of terminal devices learns the first local model to obtain a first learned parameter; a step in which each of the plurality of terminal devices transmits the first learned parameter to the server; The server calculates a first integrated parameter from the plurality of first learned parameters, and updates the distillation model using the first integrated parameter; The second associative learning step the server replicating the global model as a second local model and transmitting the second local model to each of the plurality of terminal devices; each of the plurality of terminal devices learning the second local model to obtain second learned parameters; a step in which each of the plurality of terminal devices transmits the second learned parameter to the server; and a step of calculating, by the server, a second integrated parameter from the plurality of second learned parameters, and updating the global model using the second integrated parameter.

2. The information processing method according to claim 1 , wherein the second associative learning step is executed after the first associative learning step is executed.

3. the size of the distilled model used in the (N+1)th first combined learning step is larger than the size of the distilled model used in the Nth first combined learning step; The information processing method according to claim 1 , wherein N is a natural number equal to or greater than 1.

4. The information processing method according to claim 3 , wherein the server calculates the first integrated parameter by taking a weighted average of a plurality of the first learned parameters, and calculates the second integrated parameter by taking a weighted average of a plurality of the second learned parameters.

5. An information processing method executed by a server communicatively connected to a plurality of terminal devices via a communication network, comprising: a first associative learning step of performing associative learning using a distilled model of the global model; a second associative learning step of performing the associative learning using the global model; The first associative learning step replicating the distilled model as a first local model and transmitting the first local model to each of the plurality of terminal devices; receiving, from each of the plurality of terminal devices, first learned parameters acquired by training the first local model; calculating a first integrated parameter from the plurality of first learned parameters and updating the distillation model using the first integrated parameter; The second associative learning step replicating the global model as a second local model and transmitting the second local model to each of the plurality of terminal devices; receiving, from each of the plurality of terminal devices, second learned parameters acquired by training the second local model; calculating a second integrated parameter from the plurality of second learned parameters, and updating the global model using the second integrated parameter.

6. a communication interface communicatively connected to a plurality of terminal devices via a communication network; and a processor; The processor: a first associative learning step of performing associative learning using a distilled model of the global model; a second associative learning step of performing the associative learning using the global model; The first associative learning step replicating the distilled model as a first local model and transmitting the first local model from the communication interface to each of the plurality of terminal devices; receiving, from each of the plurality of terminal devices, first learned parameters acquired by training the first local model via the communication interface; calculating a first integrated parameter from the plurality of first learned parameters and updating the distillation model using the first integrated parameter; The second associative learning step replicating the global model as a second local model and transmitting the second local model from the communication interface to each of the plurality of terminal devices; receiving, from each of the plurality of terminal devices, second learned parameters acquired by training the second local model via the communication interface; calculating a second integrated parameter from the plurality of second learned parameters and updating the global model using the second integrated parameter.

7. A computer program executable by a processor installed in a server communicatively connected to a plurality of terminal devices via a communication network, When executed, the server a first associative learning step of performing associative learning using a distilled model of the global model; a second associative learning step of performing the associative learning using the global model; The first associative learning step replicating the distilled model as a first local model and transmitting the first local model to each of the plurality of terminal devices; receiving, from each of the plurality of terminal devices, first learned parameters acquired by training the first local model; calculating a first integrated parameter from the plurality of first learned parameters and updating the distillation model using the first integrated parameter; The second associative learning step replicating the global model as a second local model and transmitting the second local model to each of the plurality of terminal devices; receiving, from each of the plurality of terminal devices, second learned parameters acquired by training the second local model; calculating a second integrated parameter from the plurality of second learned parameters, and updating the global model using the second integrated parameter.

8. A trained model trained using the information processing method according to claim 1 or 5.

Images