Association learning system, client device, server device, association learning method, and program

Abstract

To detect an anomaly in association learning while balancing information security and reduced calculation cost.SOLUTION: An integration processing unit generates global learning model parameters by performing integration processing of association learning using multiple local learning model parameters. A server difference calculation unit performs server difference calculation processing that shows the differences between each of the multiple local learning model parameters and the global learning model parameters. A learning unit performs learning processing of local learning models using global learning model parameters and learning data. A local difference calculation unit performs a local difference calculation process that indicates the differences between global learning model parameters and local learning model parameters. A detection unit performs detection processing to detect abnormalities in the server device, client device, or communication channel by comparing the server difference and the local difference.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] Embodiments of the present invention relate to a federated learning system, a client device, a server device, a federated learning method, and a program.

Background Art

[0002] In recent years, in organizations such as companies, the amount of various data collected in internal operations and provided services has been increasing with the spread of the Internet and digital technologies, and the various data collected are being treated as big data. These data have attracted attention for their potential to create new value. Under such circumstances, there is a technology called federated learning as a technology for realizing data sharing and utilization between different organizations.

Prior Art Documents

Non-Patent Documents

[0003]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the conventional technology, it has been difficult to detect anomalies in federated learning while achieving both the security of information security and the suppression of calculation costs.

Means for Solving the Problems

[0005] The federated learning system of the embodiment includes a plurality of client devices and a plurality of server devices connected to the plurality of client devices. Each of the server devices includes a server receiving unit, an integrated processing unit, a server difference calculation unit, and a server transmitting unit. The server receiving unit receives local learning model parameters of a plurality of local learning models transmitted from the plurality of client devices via a plurality of communication paths. The integrated processing unit generates global learning model parameters of a global learning model by performing integrated processing of federated learning using the plurality of local learning model parameters. The server difference calculation unit executes a calculation process of a server difference indicating a difference between each of the plurality of local learning model parameters and the global learning model parameters. The server transmitting unit transmits the global learning model parameters to each of the plurality of client devices. Each of the plurality of client devices includes a client receiving unit, a learning unit, a local difference calculation unit, and a client transmitting unit. The client receiving unit receives the global learning model parameters transmitted from the server device via the communication path. The learning unit executes a learning process of the local learning model using the global learning model parameters and learning data stored in the client device. The local difference calculation unit executes a calculation process of a local difference indicating a difference between the global learning model parameters received from the server device and the local learning model parameters. The client transmitting unit transmits the local learning model parameters to the server device. At least one of the plurality of client devices and the plurality of server devices includes a detection unit that executes a detection process of detecting that there is an abnormality in any one of the server device, the client device, or the communication path by comparing the server difference and the local difference. When the detection process is executed in the client device, the server transmitting unit further transmits the server difference to the client device in which the detection process is executed. When the detection process is executed in the server device, the client transmitting unit further transmits the local difference to the server device in which the detection process is executed. [Brief explanation of the drawings]

[0006]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

[0007] Hereinafter, embodiments of a federated learning system, a client device, a server device, a federated learning method, and a program will be described in detail with reference to the accompanying drawings.

[0008] Federated learning is an effective technique for improving the accuracy of machine learning based on data across multiple organizations. In federated learning, a global learning model is distributed to learning data stored in a distributed manner at each site. The global learning model is updated using learning data stored independently at each site. The global learning model is then updated by returning the differences between the pre-update and post-update models to the global learning model.

[0009] That is, in federated learning, only the data of the learning results is provided from the client terminal to the integrated server device. Therefore, since there is no need to provide learning data that requires privacy consideration to the integrated server device, there is an advantage that leakage of learning data during data transmission and from the integrated server can be prevented. Furthermore, since only the model parameters are transmitted and received, it is possible to reduce the communication data volume.

[0010] However, since it is assumed that the integrated server device and client terminals may become attackers in federated learning, there are privacy threats such as the learning data being inferred from the information obtained during the learning process. To solve such problems, a method for detecting attacks on learning models as shown in Non-Patent Document 1 has been reported, but the computational cost is high.

[0011] In the following embodiments, embodiments of a federated learning system, a client device, a server device, a federated learning method, and a program for detecting anomalies in federated learning while achieving both the security of information security and the suppression of computational cost will be described.

[0012] (First Embodiment) First, an example of the device configuration of the federated learning system 1 according to the first embodiment will be described.

[0013] [Example of Device Configuration] FIG. 1 is a diagram showing an example of the device configuration of the federated learning system 1 according to the first embodiment. The federated learning system 1 according to the first embodiment includes a plurality of integrated server devices 10 (examples of server devices) and a plurality of client terminals 20 (examples of client devices).

[0014] The federated learning system 1 according to the first embodiment includes a plurality of integrated server devices 10. This is related to using a secret sharing method as an encryption method in the first embodiment.

[0015] The client terminals 20 shown in FIG. 1 are each installed on the network of a client, such as inside a factory or a store. Each client terminal 20 is a computer existing within a network that can access the data held by each client.

[0016] In the first embodiment, it is assumed that the data held by each client has a network configuration that cannot be accessed from other client terminals 20. Also, it is assumed that each client terminal 20 exists for each data to be learned by the machine learning model.

[0017] To distinguish each of the plurality of client terminals, the plurality of client terminals are denoted as C1, C2, …, C5 as shown in FIG. 1. Let the total number of client terminals be N ca When ca is an integer of 2 or more. Each client terminal 20 includes a management program associated with the execution of local learning in federated learning.

[0018] Similarly, to distinguish the plurality of integrated server devices 10, the plurality of integrated server devices 10 are denoted as T1, T2, T3 as shown in FIG. 1.

[0019] [Example of Functional Configuration] FIG. 2 is a diagram showing an example of the functional configuration of the integrated server device 10 and the client terminal 20 according to the first embodiment. In FIG. 2, a pair of integrated server devices 10 and client terminals 20 included in the federated learning system 1 according to the first embodiment will be described as an example.

[0020] The integrated server device 10 according to the first embodiment includes a storage unit 101, a synchronization processing unit 102, an integration processing unit 103, a communication unit 104, a difference calculation unit 105, and a detection unit 106.

[0021] The storage unit 101 stores information on the global learning model (for example, global learning model parameters, etc.).

[0022] Before the cooperative processing unit 102 causes each of the plurality of client terminals 20 to train its respective local learning model, it synchronizes the local learning model on the client terminal 20 side with the global learning model. The synchronization process performed before learning is performed in federated learning to eliminate the need to adjust matching parameters between the local learning model and the global learning model based on the information of the global learning model held by the integrated server device

[10] .

[0023] Based on the learning results (local learning model parameters) received from each client terminal 20, the integrated processing unit 103 generates a global learning model by performing integrated processing of federated learning.

[0024] The communication unit 104 executes communication processes such as receiving the respective learning results from the plurality of client terminals and transmitting the global learning model parameters to each client terminal. Further, when transmitting the global learning model parameters to each client terminal, the communication unit 104 also encrypts the global learning model parameters by means of a secret sharing method.

[0025] The difference calculation unit 105 executes a calculation process of server differences indicating the differences between each of the plurality of local learning model parameters and the global learning model parameters.

[0026] The detection unit 106 executes a detection process of detecting an abnormality in the integrated server device 10, the client terminal 20, or the communication path between the integrated server device 10 and the client terminal 20 by comparing the server differences with local differences (described later) calculated at each client terminal 20.

[0027] The client terminal 20 according to the first embodiment includes a storage unit 201, a learning unit 202, an encryption unit 203, a decryption unit 204, a communication unit 205, a difference calculation unit 206, and a detection unit 207.

[0028] The storage unit 201 stores the local learning model parameters of the local learning model, the learning data used for the machine learning of the local learning model, and the like.

[0029] The learning unit 202 performs machine learning of the local learning model using the learning data.

[0030] The encryption unit 203 encrypts the local learning model parameters of the local learning model obtained by learning.

[0031] The decryption unit 204 decrypts the encrypted global learning model parameters received from the integrated server device 10.

[0032] The communication unit 205 executes communication processes such as transmitting the encrypted local learning model parameters to the integrated server device 10 and receiving the global learning model parameters transmitted from the integrated server device 10.

[0033] The difference calculation unit 206 executes a calculation process of local difference indicating the difference between the global learning model parameters received from the integrated server device 10 and the local learning model parameters.

[0034] The detection unit 207 executes a detection process of detecting an abnormality in the integrated server device 10, the client terminal 20, or the communication path between the integrated server device 10 and the client terminal 20 by comparing the server difference and the local difference.

[0035] Next, the operation of the first embodiment will be described.

[0036] In the first embodiment, data is encrypted using secret sharing, and the secret sharing uses a secret sharing method based on Shamir's threshold secret sharing method. Shamir's threshold secret sharing method converts one piece of secret information into n distributed values and distributes them to n devices. Also, if a predetermined number of the distributed information is gathered from the distributed information, the original secret information can be restored. The secret sharing method of the first embodiment generates distributed information obtained by distributing the original data into n pieces for two or more integers n and k satisfying k ≤ n, and has the property of being able to restore the original data even if n - k pieces of distributed information are lost, and the confidentiality that the original data cannot be restored at all from less than k pieces of distributed information.

[0037] The encryption unit 203 encrypts the local learning model parameters generated by the client terminal 20 using the above secret sharing method. Therefore, the number of integrated server devices 10 is the number n of distribution destinations when performing encryption by secret sharing. Also, the number of data required for decrypting the global learning model obtained by the integrated server device 10 is k, and k is an integer satisfying 1 < k ≤ n.

[0038] For example, a neural network is used as the algorithm of the learning model to be subjected to anomaly detection in the first embodiment. A neural network is composed of an input layer, an output layer, and one or more hidden layers, includes a plurality of nodes in each layer, and the nodes are interconnected. Regarding the network parameters including the weights and biases constituting the neural network, a learning model reflecting the learning data in each client terminal 20 is updated between the client terminal 20 and the integrated server device 10.

[0039] Also, the learning algorithm is not limited to a neural network with a specific structure, and any structure of neural network including a convolutional neural network (CNN) and a recurrent neural network (RNN) may be used.

[0040] Hereinafter, an example of an anomaly detection method by the federated learning system 1 according to the first embodiment will be described.

[0041] [Examples of Anomaly Detection Methods] FIG. 3 is a flowchart showing an example of the anomaly detection method according to the first embodiment. The example in FIG. 3 shows a series of flows in the processing of each round. In the series of flows, first, the client terminal 20 receives the global learning model and updates the local learning model. The integrated server device 10 receives the updated local learning model from the client terminal 20 and updates the global learning model. Then, the integrated server device 10 transmits the global learning model to each client terminal 20.

[0042] In the first embodiment, anomaly detection is performed by the client terminal 20 during each round to detect a malicious integrated server device 10 or to perform anomaly detection of the learning model in communication via the communication path between the client terminal 20 and the integrated server device 10.

[0043] Here, let the number of client terminals 20 used in the federated learning system 1 of the first embodiment be Nca, the number of integrated server devices 10 be Nsa, and both Nca and Nsa be integers of 2 or more. Also, let t be the number of rounds, and t = 0 at the start of step S001.

[0044] First, the communication unit 205 of each client terminal 20 receives the initial value of the global learning model from the integrated server device 10 or the like before starting learning (steps S001 and S003). The initial value of the global learning model may be provided by the integrated server device 10 or may be provided by another server device that only prepares the initial value.

[0045] Also, in the first embodiment, the flowchart is described on the premise that encryption is performed to provide the initial value of the global learning model to each client terminal 20 more securely. However, the encryption method of the initial value of the global learning model is not limited to secret sharing and may be arbitrary.

[0046] In step S002, when the round number t is 1 or less, it branches to step S003, and when the round number is 2 or more, it branches to step S004. That is, at the 0th round, which is the time of transmitting the initial value of the global learning model immediately after step S001, since the local learning model has not been generated at the client terminal 20, the detection steps S004 to S011 are not executed.

[0047] Up to the time of receiving the initial value of this global learning model is regarded as the 0th round. When the count number of the round number is incremented by 1 in step S012, it becomes the 1st round. Therefore, after the 1st round, step S012 serves as the starting point for the round number switch.

[0048] (1st round) The operation of the 1st round will be described from here. First, the decoding unit 204 decodes the global learning model (step S013). Specifically, in the 1st round, in step S003, each client terminal 20 receives the initial value of the global learning model. After that, the decoding unit 204 decodes the received global learning model. As described above, since the encryption method for the initial value of the global learning model is not limited, a decoding method along with the used encryption method is used.

[0049] Next, the learning unit 202 inputs the learning dataset held in the storage unit 201 in each client terminal 20, and updates (learns) the local learning model with the decoded global learning model as the initial value (step S014). Let the local learning model parameter obtained in step S014 be m i t Here, i is the client number, satisfying 1 ≦ i ≦ Nca. The local learning model parameter of the local learning model thus obtained is encrypted by secret sharing by the encryption unit 203 (step S015). Hereinafter, the value encrypted by secret sharing is called a share. Also, the share of the local learning model parameter transmitted to the integrated server device x is [m i,x tIt is denoted as []. Here, the share of the plaintext z is denoted as [z]. The number x that identifies the integrated server device 10 satisfies 1 ≤ x ≤ Nsa.

[0050] Next, the communication unit 205 transmits the share [m i,x t of the local learning model parameters generated in step S015 to the integrated server device x (step S016).

[0051] Next, the communication unit 104 of the integrated server device x receives the share [m i,x t of the local learning model parameters from each client terminal 20 (step S017). Then, in step S018, the number of rounds is confirmed, and step S019 is skipped in the first round. Step S019 is a process of calculating parameters for detection at the client terminal 20. In the first round, the global learning model parameters [m x t-1 from the previous round used here do not exist.

[0052] Next, the integration processing unit 103 uses the shares of the local learning models received from each client terminal 20 to generate the global learning model parameters [M x t of the global learning model (step S020). The operation of model integration is performed by secure computation, and all operations of model integration and parameter calculation used for detection in the second and subsequent rounds are also performed by secure computation. For example, the integration processing unit 103 calculates FedAvg from the local learning model parameters and the number of data used for learning in each client terminal 20 to generate a global learning model. Here, FedAvg is taken as an example of the model integration operation, but other integration algorithms may also be used.

[0053] Next, the communication unit 104 sends the global learning model parameters [M i,x tTransmit (step S021). However, the transmission in step S021 is only [M i,x t only when the round number t = 1. After t = 2, in addition to [M i,x t , the detection parameter ΔD global is also transmitted together. Regarding ΔD global , it will be explained in the operation description after the second round.

[0054] Also, [M i,x t and [M x t represent the same global learning model, and the number i indicates the number for identifying the client terminal 20. [M i,x t indicates that it is the global learning model parameter of the global learning model sent to the client terminal i. Considering the possibility that the value of the global learning model parameter may change when received by the client terminal i due to attacks such as model tampering or other reasons between the client terminal i and the integrated server device x, the notations are distinguished.

[0055] Next, return to step S002. Since t = 1, proceed to step S003 in this branch, and receive the share [M i,x t of the global learning model transmitted from the integrated server device x in step S021 above (step S003). Then, after this step S003, proceed to step S012, increment the round number by 1 so that t changes from 1 to 2, and the operation of the first round ends.

[0056] (After the second round) Next, the operation after the second round will be explained. The operation after the second round starts from step S013. The decoding unit 204 decodes the share [M i,x tUse it to decode the global learning model (step S013). The subsequent steps S014 to S017 are the same as those in the first round.

[0057] At the branch in step S018, in the subsequent rounds from the second round, the next step is step S019. The difference calculation unit 105 calculates the difference ΔD global (i, x) between the global learning model and the local learning model, which are the parameters used for anomaly detection at the client terminal i (step S019).

[0058] ΔD global (i, x) is global ΔD x t-1 (i, x)=[M i,x t -[m x t-1 . One item of [M x 1 represents the share of the global learning model generated at the integrated server device x in the (t - 1)th round. For example, in the second round, [M x t-1 obtained in the first round corresponds to [M i,x t . The second item of [m

[0059] is the share of the local learning model generated by the client terminal i at the integrated server device x in the tth round and received in step S017. global This ΔD global (i, x) is performed at the client terminal 20 participating in the integration process. For example, when all client terminals 20 participate in the integration process, at each integrated server device 10, Nca pieces of ΔD

[0060] (i, x) are generated. i,x tand ΔD global (i, x) are transmitted to each client terminal i (step S021).

[0061] Next, the process returns to step S002. Since the branch condition t > 1 is satisfied, the process proceeds to step S004. That is, the communication unit 205 of the client terminal i receives the parameters ([M i,x t and ΔD global (i, x)) transmitted from the integrated server device x in step S021 (step S004). At this time, ΔD global (i, x) is received from all the integrated server devices x that participated in the (t - 1)-th round. For example, when all Nsa integrated server devices 10 participate in the integration process, Nsa ΔD global (i, x)) are received.

[0062] Steps S005 to S011 are the processes of the detection step, which are performed by the difference calculation unit 206 or the detection unit 207. First, the detection unit 207 calculates the difference ΔD local (i, x) between the global learning model and the local learning model, which are the parameters used for detection at the client terminal i (step S005).

[0063] ΔD local (i, x) is defined as ΔD local (i, x)=[M i,x t-1 -[m i,x t . One item of [M i,x t-1 indicates the share of the global learning model generated at the integrated server device x in the (t - 1)-th round. That is, one item of [M i,x t-1 is the [M i,x t-1 received in step S004 in the (t - 1)-th round or, when t - 1 = 1, in step S003. Also, the two items of [m i,x tis the share of the local learning model generated in step S014 of the t-th round and encrypted in step S015. ΔD local [m used for the calculation of (i, x) i,x t has never been sent outside from the client terminal i that generated the [m i,x t .

[0064] Next, the difference calculation unit 206 calculates the difference between ΔD local (i, x) calculated in step S005 and ΔD global (i, x) received in step S004 (step S006).

[0065] ΔD local (i, x) and each term of ΔD global (i, x) are calculated in the same process respectively. Here, one item of ΔD local (i, x) of [M i,x t-1 is the value received in step S004 of the (t - 1)-th round or step S003 if t - 1 = 1. On the other hand, one item of ΔD global (i, x) of [M i,x t-1 is the value used for the calculation of ΔD global (i, x) received in step S004 of the t-th round. Although the timing of transmission from the integrated server device x to the client terminal i is different, the values of these two [M i,x t-1 should be the same value originally.

[0066] Also, two items of ΔD local (i, x) of [m i,x t are the values held in the internal storage unit 201 etc. of the client terminal i that generated the [m i,x t after being generated in step S015 of the t-th round. On the other hand, two items of ΔD global (i, x) of [m i,x tis the value received by the integrated server device x from the client terminal i in step S017 of the t-th round and used for the calculation of ΔD global (i, x) in step S019. Therefore, the values of these two [m i,x t should be the same value originally.

[0067] Thus, unless there is forgery by an attacker, an abnormality in network communication or server processing, etc., ΔD local (i, x) - ΔD global (i, x) = 0 (the differences ΔD local and ΔD global match). Here, forgery by an attacker may occur between the client terminal i and the integrated server device x and at the integrated server device x.

[0068] Next, the detection unit 207 counts the number of integrated server devices that satisfy ΔD local (i, x) - ΔD global (i, x) = 0 (step S007), and sets the count value as Ns. Also, at this time, the detection unit 207 of the client terminal i records whether the value of ΔD local (i, x) - ΔD global (i, x) is 0 or not for each number x that identifies the integrated server device 10, and holds the result as a list L.

[0069] Next, the detection unit 207 determines whether the count value Ns is equal to or greater than the number k required for decoding the global learning model [M i,x t (step S008).

[0070] If Ns ≥ k (step S008, Yes), the detection unit 207 selects the shares of the integrated server devices to be used for decoding the global learning model (step S011). For example, the detection unit 207 selects, from the list L, the shares [M local (i, x) - ΔD global (i, x) = 0 of k integrated server devices x for the global learning model i,x tRandomly select

[0071] On the other hand, when Ns < k (step S008, No), the detection unit 207 returns the share of the global learning model and the list L to the state at the (t - 1)-th round (step S009). Specifically, the detection unit 207 of the client terminal i replaces the share [M of the global learning model at the t-th round i,x t with the share [M of the global learning model that satisfied Ns ≥ k and was adopted at the (t - 1)-th round i,x t-1 . That is, [M i,x t =[M i,x t-1 . This replacement is performed on all the server Nsa. Further, the detection unit 207 also replaces the value of the above-mentioned list L with the corresponding value according to the replacement of the share of the global learning model.

[0072] Next, the detection unit 207 counts again the number Ns of integrated server devices that satisfy ΔD local -ΔD global = 0 (step S010). Assume that at the (t - 1)-th round, Ns ≥ k was satisfied in step S008, and the process of step S011 is performed here.

[0073] The share [M of the global learning model selected in this way i,x t is decoded in the next round. Therefore, the operation up to the selection of this share [M of the global learning model i,x t is the operation of the t-th round.

[0074] The above is the processing flow of anomaly detection in the first embodiment.

[0075] (Second Embodiment) Next, the second embodiment will be described. In the description of the second embodiment, the same explanations as those in the first embodiment will be omitted, and the differences from the first embodiment will be described. In the second embodiment, the case where the integrated server device 10 executes anomaly detection will be described.

[0076] [Examples of Anomaly Detection Methods] FIG. 4 is a flowchart showing an example of the anomaly detection method of the second embodiment. The example in FIG. 4 shows a series of flows in the processing of each round. In the series of flows, first, the client terminal 20 receives the global learning model and updates the local learning model. The integrated server device 10 receives the updated local learning model from the client terminal 20 and updates the global learning model. Then, the integrated server device 10 transmits the global learning model to each client terminal 20.

[0077] In the second embodiment, during each round, the integrated server device 10 executes anomaly detection to detect a malicious client terminal 20 or to perform anomaly detection of the learning model in the communication via the communication path between the client terminal 20 and the integrated server device 10.

[0078] Let t be the number of rounds, and assume t = 0 at the start of step S101.

[0079] Step S101 is the same as step S001 in the first embodiment, so the description is omitted.

[0080] In step S102, when the number of rounds t is less than 1, the process branches to step S103, and when the number of rounds is 1 or more, the process branches to step S104. This branch is due to the fact that for the received global learning model, the encryption method is not limited in the case of the initial value, and after round 1, the received global learning model is a share [M encrypted by secret sharing i,x t received as.

[0081] The period until the initial value of this global learning model is received is regarded as the 0th round. When the count of the round number is incremented by 1 in step S105, it becomes the 1st round. Therefore, after the 1st round, step S105 serves as the starting point for the switching of the round number.

[0082] (1st round) Next, the operation in the 1st round will be described. First, the decoding unit 204 decodes the global learning model (step S106). Specifically, in the 1st round, in step S103, each client terminal 20 receives the initial value of the global learning model. Thereafter, the decoding unit 204 decodes the received global learning model. As described above, since the encryption method for the initial value of the global learning model is not limited, the decoding method in accordance with the used encryption method is used.

[0083] Steps S107 and S108 are the same as steps S014 and S015 in the first embodiment, so the description is omitted.

[0084] In step S109, when the round number is 1 or less, the process proceeds to step S110. Step S110 is the same as step S016 in the first embodiment, so the description is omitted.

[0085] Next, in the process of the integrated server device x, when the round number is 1, in the branch of step S113, the process proceeds to step S114, and thereafter, step S119 is executed. The processes of steps S114 and S119 are the same as steps S017 and S020 in the first embodiment, so the description is omitted. Next, the communication unit 104 transmits the global learning model parameter [M i,x t to each client terminal 20 (step S120).

[0086] Also, similar to the first embodiment, [M i,x t and [M x trepresents the same global learning model, and the number i represents the number identifying the client terminal 20. [M i,x t indicates that it is the global learning model parameter of the global learning model sent to the client terminal i.

[0087] Next, returning to step S102, since t = 1, in this branch, it proceeds to step S104, and receives the share [M i,x t of the global learning model transmitted from the integrated server device x in the above-mentioned step S120 (step S104). Then, after this step S104, it proceeds to step S105, and by adding 1 to the number of rounds, t changes from 1 to 2, and the operation of the first round ends.

[0088] (After the second round) Next, the operation after the second round will be described. The operation after the second round starts from step S105. The step S106 in the second round is the same as the step S013 in the second round in the first embodiment, so the description is omitted. Also, the subsequent steps S107 and S108 have the same operations as in the first round.

[0089] In the branch of step S109, after the second round, the next step is step S111.

[0090] The difference calculation unit 105 calculates the difference ΔD local (i, x) between the global learning model, which is a parameter used when the integrated server device x performs anomaly detection, and the local learning model (step S111). The definition of ΔD local (i, x) is the same as that in the first embodiment. One item of ΔD local (i, x) of [M i,x t-1 is the [M i,x t-1 received in step S104 of the (t - 1)-th round. Also, two items of ΔD local (i, x) of [m i,xt is the share of the local learning model generated in step S107 of the t-th round and encrypted in step S108.

[0091] Next, the communication unit 205 transmits ΔD local (i, x) and the share [m i,x t of the local learning model parameters to the integration server device x (step S112).

[0092] Then, on the integration server device x side, since the number of rounds is 2 or more here, the process proceeds to step S115, and the communication unit 104 receives ΔD local (i, x) and the share [m i,x t of the updated local learning model from each client terminal i (step S115).

[0093] Next, the difference calculation unit 105 calculates the difference ΔD global (i, x) between the global learning model and the local learning model, which are parameters used for anomaly detection in the integration server device x (step S019). The definition of ΔD global (i, x) is the same as that in the first embodiment. Note that the two items of [m global (i, x) i,x t are the shares of the local learning model generated by the client terminal i in the t-th round and received in step S115.

[0094] This ΔD global (i, x) is performed by the 20 client terminals participating in the integration process. For example, when all 20 client terminals participate in the integration process, Nca ΔD global (i, x) are generated in each integration server device 10.

[0095] Next, the difference calculation unit 105 calculates the ΔD local (i, x) received in step S115 and the ΔD calculated in step S116global (i, x) difference is calculated (step S117). ΔD local (i, x) and ΔD global Each term of (i, x) is a value calculated in the same process. Here, ΔD local One item of [M in (i, x) i,x t-1 was received in step S104 of the (t - 1)-th round and is the value of ΔD received in step S115 of the t-th round local and is the value used in the calculation of (i, x). On the other hand, ΔD global One item of [M in (i, x) i,x t-1 is the value calculated in step S119 of the (t - 1)-th round and then stored in the storage unit 101 of the integrated server device x. ΔD global One item of [M in (i, x) i,x t-1 was transmitted from the integrated server device x to the client terminal i in the (t - 1)-th round and should be the same value as the value used in the calculation of ΔD in step S111 local (i, x).

[0096] Also, the second item of [m in ΔD received in step S115 local (i, x) is generated using [M i,x t received in step S104 of the (t - 1)-th round in step S108 of the t-th round. On the other hand, [m in ΔD i,x t-1 (i, x) is the value received by the integrated server device x in step S115 of the t-th round. The values of these two [m global (i, x) i,x t should be the same value. i,x t (i, x).

[0097] Therefore, unless there is forgery by an attacker, an abnormality in network communication or server processing, etc., occurs, ΔD local (i, x) - ΔD global(i, x) = 0. Here, tampering by the attacker may occur between the integrated server device x and the client terminal i, and at the client terminal i.

[0098] Next, the detection unit 106 selects the share [m i,x t of the local learning model used for the integrated processing (step S118). Specifically, the detection unit 106 counts the number of client terminals that satisfy ΔD local (i, x) - ΔD global (i, x) = 0, and sets the count value as Nc. Also, at this time, the detection unit 106 of the integrated server device x records whether the value of ΔD local (i, x) - ΔD global (i, x) is 0 or not for each number i identifying the client terminal 20, and holds the result as a list L. Also, this list L is created at each integrated server device x and shared among the integrated server devices 10. Then, the detection unit 106 selects the share [m local (i, x) - ΔD global (i, x) = 0 received from the client terminals 20 determined to satisfy the condition in all the integrated server devices 10. i,x t of the local learning model.

[0099] Next, the integration processing unit 103 performs model integration using the share [m i,x t of the local learning model selected in step S118 by a function such as FedAvg, and generates the share [M x t of the global learning model (step S119).

[0100] Step S120 is the same as step S120 in the first round, so the description is omitted.

[0101] And here, since the number of rounds is 2 or more according to step S102, it proceeds to step S104, and the communication unit 205 transmits the share [M i,xt receive it (step S104). This is the operation of the second round. The processing in one round after the third round is the same as the operation of the second round.

[0102] (Modification Example 1 of the Second Embodiment) Next, Modification Example 1 of the second embodiment will be described. In the description of Modification Example 1, the same description as that of the second embodiment will be omitted, and the parts different from the second embodiment will be described. In Modification Example 1, after the second round in the second embodiment, in step S117 of the detection process performed by the integrated server device x, for t1 consecutive times (consecutive for a predetermined number of times), ΔD local (i,x) - ΔD global client terminal i for which (i,x) ≠ 0 is not included in the subsequent integration process. The consecutive number of times t1 is set to any value of 1 or more. Also, the number of client terminals required for integration is 2 or more. When this condition is satisfied, the learning of the next round is performed.

[0103] (Modification Example 2 of the Second Embodiment) Next, Modification Example 2 of the second embodiment will be described. In the description of Modification Example 2, the same description as that of the second embodiment will be omitted, and the parts different from the second embodiment will be described. In Modification Example 2, after the second round in the second embodiment, in step S117 of the detection process performed by the integrated server device x, in total t2 times (predetermined total number of times), ΔD local (i,x) - ΔD global client terminal i for which (i,x) ≠ 0 is not included in the subsequent integration process. The total number of times t2 is set to any value of 1 or more. Also, the number of client terminals required for integration is 2 or more. When this condition is satisfied, the learning of the next round is performed.

[0104] As described in the above first and second embodiments, the detection process for detecting that there is an abnormality in any one of the integrated server device 10, the client terminal 20, or the communication (communication path) between the integrated server device 10 and the client terminal 20 may be performed by the detection unit 106 of the integrated server device 10, or may be performed by the detection unit 207 of the client terminal 20.

[0105] That is, in each of the integrated server devices 10 (examples of server devices), the communication unit 104 (example of a server reception unit) receives local learning model parameters of a plurality of local learning models transmitted from a plurality of client terminals 20 (examples of client devices) via a plurality of communication paths. The integrated processing unit 103 generates global learning model parameters of the global learning model by executing the integration process of federated learning using the plurality of local learning model parameters. The difference calculation unit 105 (example of a server difference calculation unit) executes a calculation process of a server difference (ΔD as described above) indicating the difference between each of the plurality of local learning model parameters and the global learning model parameters. Then, the communication unit 104 (example of a server transmission unit) transmits the global learning model parameters to each of the plurality of client terminals 20. global )

[0106] Also, in each of the plurality of client terminals 20, the communication unit 205 (example of a client reception unit) receives the global learning model parameters transmitted from the integrated server device 10 via the communication path. The learning unit 202 executes a learning process of the local learning model using the global learning model parameters and the learning data stored in the client terminal 20. The difference calculation unit 206 (example of a local difference calculation unit) executes a calculation process of a local difference (ΔD as described above) indicating the difference between the global learning model parameters received from the integrated server device 10 and the local learning model parameters. Then, the communication unit 205 (example of a client transmission unit) transmits the local learning model parameters to the integrated server device 10. local )

[0107] At least one of the plurality of client terminals 20 and the plurality of integrated server devices 10 includes a detection unit 106 (206) that executes a detection process for detecting an abnormality in any one of the integrated server device 10, the client terminal 20, or the communication (communication path) between the integrated server device 10 and the client terminal 20 by comparing the server difference and the local difference.

[0108] When the detection process is executed by the client terminal 20, the communication unit 104 (an example of a server transmission unit) further transmits the server difference to the client terminal 20 where the detection process is executed. When the detection process is executed by the integrated server device 10, the communication unit 205 (an example of a client transmission unit) further transmits the local difference to the integrated server device 10 where the detection process is executed.

[0109] In the above-described first and second embodiments, the above-described integration process and the above-described calculation process of the server difference are performed by secret calculation in which each of a plurality of local learning model parameters and the global learning model parameter are calculated in a state of encrypted shares ([m i,x t and [M i,x t ). Further, the local learning model is updated based on parameters decrypted from the share [M i,x t of the encrypted global learning model parameter.

[0110] According to the above-described first or second embodiment, it is possible to detect an abnormality in federated learning while achieving both the security of information security and the suppression of calculation costs. Specifically, the above-described shares [m i,x t and [M i,x tEncrypt the model parameters, and calculate at each of the client terminal 20 and the integrated server device 10 in an encrypted state the difference between the local learning model generated at the client terminal 20 and the global learning model generated by the integrated processing of the integrated server device 10, and determine the presence or absence of forgery based on whether the values match. Thereby, it is possible to ensure higher security than the conventional federated learning detection method and reduce the arithmetic cost required for detection.

[0111] Also, in the above-described first or second embodiment, the value used for detection is the share [m i,x t of the local learning model encrypted by secret sharing and the share [M i,x t-1 of the global learning model. This difference is ΔD local and ΔD global calculated at each of the client terminal 20 and the integrated server device 10. Even if the share [m i,x t , the share [M i,x t-1 , ΔD local and ΔD global are stolen when transmitted to the client terminal 20 or the integrated server device 10, it is necessary to collect the corresponding values in the number required for decryption, and in addition, the operations of ΔD local and ΔD global for detection are performed without decryption, so it is possible to achieve both information security and suppression of arithmetic cost.

[0112] (Third Embodiment) Next, the third embodiment will be described. In the description of the third embodiment, the same description as in the first embodiment will be omitted, and the parts different from the first embodiment will be described. In the third embodiment, the detection process is performed at the client terminal, and an operation method for more limiting the location where an abnormality occurs will be described based on the operation of the detection unit 207 in the first embodiment.

[0113] [Examples of Abnormality Detection Methods] FIG. 5 is a flowchart showing an example of the abnormality detection method according to the third embodiment.

[0114] (First round) The operation in the first round is the same as that in the first embodiment, so the description is omitted.

[0115] (After the second round) The second round starts from step S216 after the round number is incremented in step S215. Here, steps S216 to S220 correspond to steps S013 to S016 in the first embodiment, respectively, and perform the same operations, so the description is omitted. Also, steps S220 to S224, which are operations in the integrated server device 10, perform the same operations as steps S017 to S021 in the first embodiment, so the description is omitted.

[0116] Next, returning to step S202, if t is 2 or more, the process proceeds to the process of step S204, and the processes of steps S205 to S207 are executed. Steps S205 to S207 perform the same operations as steps S005 to S007 in the first embodiment, so the description is omitted.

[0117] Next, the detection unit 207 determines whether the conditional expression Ns = Nsa is satisfied (step S208). Here, Ns is ΔD local (i,x) - ΔD global (i,x) = 0, indicating the number of integrated server devices 10 that satisfy the condition. Nsa indicates the number of integrated server devices 10 used in the federated learning system 1 according to the third embodiment.

[0118] When Ns = Nsa (step S208, Yes), the decoding unit 204 arbitrarily selects k shares required for decoding from Nsa shares (shares of all integrated server devices 10) as the shares used for decoding the global learning model (step S212).

[0119] On the other hand, when Ns ≠ Nsa (step S208, No), the decoding unit 204 obtains the shares of the global learning model from Nsa integrated server devices 10 [M i,xt From the k pieces necessary for decoding, the global learning model is decoded (step S209).

[0120] Then, the detection unit 207 compares the global learning model decoded in step S209 with the global learning model in the (t - 1) - th round (step S210). If, as a result of the comparison, there are elements of model parameters that change by d% or more, the detection unit 207 determines that the share [M i,x t includes an outlier.

[0121] When it is determined that the share [M i,x t includes an outlier, the detection unit 207 particularly checks, among the Nsa pieces, the share [M local (i, x) - ΔD global (i, x) = 0 that did not satisfy the conditional expression in step S207. For example, by changing the combination for selecting the share [M i,x t of the global learning model and performing decoding multiple times, the location of the abnormality can be specified more precisely. Specifically, by selecting k - 1 pieces from the Ns pieces of share [M i,x t that satisfied ΔD local (i, x) - ΔD global (i, x) = 0 in step S207 and one share that did not satisfy ΔD local (i, x) - ΔD global (i, x) = 0 and performing the combination of the global learning model, the abnormality can be detected more efficiently. For example, an abnormality in the integrated server device x and an abnormality in the communication path can be detected.

[0122] Also, for the share [M local (i, x) - ΔD global (i, x) = 0 of the global learning model of the integrated server device x that did not satisfy the condition in step S207 i,x t] may be used for decryption without any problems. In this case, if an attacker converts some of the parameters sent and received on the communication channel, i,x t ] and ΔD global Of these, ΔD global It is highly likely that only the data has been tampered with.

[0123] Conversely, if the decoding result in step S210 is judged to contain an abnormal value, [M i,x t ] and ΔD global Among them, [M i,x t Even if only the [] is tampered with, there is a high possibility that some of the parameters sent and received on the communication channel have been changed.

[0124] On the other hand, if the results of steps S207 and S210 both indicate an abnormality, it is highly likely that an abnormality has already occurred at the time of model integration. i,x t ] and ΔD global The local learning model [M i,x t ] is likely to have been tampered with along the communication path from client terminal i to integrated server device x. Alternatively, it is likely that integrated server device x itself is outputting an inaccurate value due to an attack. In addition, a list of numbers x that identify Ns' normal integrated server devices 10 that were determined to be normal in step S210 is recorded, and the result is held as list L'.

[0125] Next, the detection unit 207 determines whether the conditional expression Ns'≧k is satisfied (step S211). Here, Ns' indicates the number of normal integrated server devices x that are determined to be normal in step S210. k indicates the share [M i,x t ] indicates the number of

[0126] When Ns’ ≥ k (Yes in step S211), the decoding unit 204 arbitrarily selects k shares, which are the number of shares required for decoding, from the Ns’ shares (shares of the normal integrated server device 10) used for decoding the global learning model from the list L’ (step S212).

[0127] On the other hand, when Ns’ < k (No in step S211), the detection unit 207 returns the shares of the global learning model, and the lists L and L’ to the state of the (t - 1)-th round (step S213). Specifically, the detection unit 207 of the client terminal i replaces the share [M i,x t of the global learning model in the t-th round with the share [M i,x t-1 of the global learning model that satisfied Ns’ ≥ k and was adopted in the (t - 1)-th round. That is, [M i,x t =[M i,x t-1 . Further, the detection unit 207 also replaces the values of the above-mentioned list L with the corresponding values in accordance with the replacement of the shares of this global learning model.

[0128] Next, the detection unit 207 counts again the number Ns’ of integrated server devices that satisfy ΔD local -ΔD global = 0 (step S214). In the (t - 1)-th round, it is assumed that Ns ≥ k was satisfied in step S211, and the process of step S212 is performed.

[0129] The share [M i,x t of the global learning model selected in this way is decoded in the next round. Therefore, the operation up to the selection of this share [M i,x t is the operation in the t-th round.

[0130] The above is the processing flow of anomaly detection in the third embodiment.

[0131] (Fourth Embodiment) Next, the fourth embodiment will be described. In the description of the fourth embodiment, the same description as that of the first embodiment will be omitted, and the parts different from the first embodiment will be described. In the fourth embodiment, an operation example of reading out the global learning model decoded in the (t - 1)-th round when Ns < k after the second round in the round t will be described.

[0132] [Example of anomaly detection method] FIG. 6 is a flowchart showing an example of the anomaly detection method according to the fourth embodiment. In the example of FIG. 6, in the first embodiment, the process in the case of Ns < k in step S008 performed after the second round is changed to the process in step S311. In step S311 of the fourth embodiment, instead of using the share [M i,x t of the global learning model in the (t - 1)-th round, the global learning model (plaintext) decoded in the (t - 1)-th round is read out.

[0133] (Fifth embodiment) Next, the fifth embodiment will be described. In the description of the fifth embodiment, the same description as that of the first embodiment will be omitted, and the parts different from the first embodiment will be described. In the fifth embodiment, an operation example of stopping the use of the integrated server device x in which anomalies are detected multiple times will be described.

[0134] [Example of anomaly detection method] FIG. 7 is a flowchart showing an example of the anomaly detection method according to the fifth embodiment. In the example of FIG. 7, in the fifth embodiment, the client terminal i performs the detection process. The detection unit 207 counts the number of times when ΔD local -ΔD global = 0 is not satisfied for each pair of the client terminal i and the integrated server device x. When the count number is r1(i, x), the integrated server device x satisfying r1(i, x) ≥ a is not used for updating the global learning model.

[0135] (First round) The operation in the first round is the same as that in the first embodiment.

[0136] (After the second round and later) The second round starts from step S416 after the round number is incremented in step S415. Here, steps S416 to S419 respectively correspond to steps S013 to S016 in the first embodiment and perform the same operations. Also, steps S420 to S424, which are operations in the integrated server device 10, are the same as steps S017 to S021 in the first embodiment, so the description is omitted.

[0137] Next, returning to step S402, when t is 2 or more, the process proceeds to the process of step S404, and the processes of steps S405 to S407 are executed. Steps S405 to S407 are the same as steps S005 to S007 in the first embodiment, so the description is omitted.

[0138] Next, the detection unit 207 of the client terminal i is ΔD local (i, x) - ΔD global (i, x) ≠ 0 for the integrated server device x's identification number list Lng t is generated (step S408). This identification number list Lng t records, among the Nsa integrated server devices x, which integrated server device x has an abnormality (an error in the verification) in the connection with the client terminal i at the t-th round.

[0139] Next, the detection unit 207, based on this identification number list Lng t counts, for each integrated server device x, the number of times that ΔD local (i, x) - ΔD global (i, x) ≠ 0 has occurred up to the t-th round (step S409), and sets the count number as r1(i, x).

[0140] Next, the detection unit 207 counts the integrated server devices x for which the count number r1(i,x) in step S409 satisfies r1(i,x) ≥ a (step S410), and sets the count number to Nngs.

[0141] Next, the detection unit 207 determines whether the value obtained by subtracting the count value Nngs from the total number of integrated server devices Ns (Ns - N ngs ) is equal to or greater than the number k required to decrypt the share of the global learning model (step S411). That is, the share of the global learning model parameters received from the integrated server device x for which r1(i,x) ≥ a (where a is an example of a frequency threshold) is not used for decrypting the global learning model parameters.

[0142] Ns - N ngs If Ns - N ngs ≥ k (step S411, Yes), the decryption unit 204 arbitrarily selects k shares necessary for decryption from Ns - N

[0143] shares (shares of the integrated server device x that satisfies r1(i,x) < a) (step S414). ngs <k (step S411, No), since the number of shares required for decryption is insufficient, the detection unit 207 returns the share [M ngs i,x t of the global learning model and the list L to the state where Ns - N ngs

[0144] ≥ k was satisfied in the (t - 1)-th round (step S412). Then, with the values replaced again to the values in the (t - 1)-th round, the process returns to step S407 and proceeds through the steps in order. ngs Note that in round t - 1, if it can be confirmed that Ns - N tIt is also replaced with the value in the (t-1)th round. That is, Lng t is set to Lng t-1 .

[0145] Step S414 becomes the last step in the process of the t-th round, and thereafter it becomes the (t+1)-th round.

[0146] (Modification of the Fifth Embodiment) Next, a modification of the fifth embodiment will be described. In the description of the modification, the same descriptions as those in the fifth embodiment will be omitted, and the parts different from the fifth embodiment will be described. In the modification, an operation of determining how many integrated server devices 10 to allow attacks on will be described.

[0147] In the modification, in step S410 of the fifth embodiment, when the share [M local (i,x)-ΔD global (i,x) of the integrated server device x where a or more times ΔD i,x t (i,x)≠0 is not used for decrypting the global learning model. Specifically, when reaching a times in the t-th round, the decryption unit 204 does not use the share [M i,x t transmitted from the integrated server device x for decrypting the global learning model after the (t+1)-th round.

[0148] At that time, when the number of shares that can be used for decryption, that is, Nsa-N ngs is less than the number k of shares required for decrypting the global learning model, the federated learning system 1 is stopped.

[0149] (Sixth Embodiment) Next, the sixth embodiment will be described. In the description of the sixth embodiment, the same descriptions as those in the second embodiment will be omitted, and the parts different from the second embodiment will be described. In the sixth embodiment, an operation example of suspending (stopping) the operation of the integrated server device x will be described.

[0150] [Example of Abnormality Detection Method] FIG. 8 is a flowchart showing an example of the abnormality detection method according to the sixth embodiment. In the example of FIG. 8, after the second round in the second embodiment, the operations after step S117 of the detection process performed by the integrated server device x (in the example of FIG. 8, the operations after step S517) are different. Specifically, the detection unit 106 counts the number of client terminals Nc that satisfy ΔD local -ΔD global =0, and steps S519 and S523 for stopping the operation when the count is less than a certain value b according to the count are added.

[0151] Here, b (an example of the client number threshold) can be arbitrarily set within the range of Nca≧b≧2 for all the client terminal numbers Nca.

[0152] Finally, an example of the hardware configuration of the integrated server device 10 and the client terminal 20 according to the first to sixth embodiments will be described.

[0153] [Example of Hardware Configuration] FIG. 9 is a diagram showing an example of the hardware configuration of the integrated server device 10 and the client terminal 20 according to the first to sixth embodiments. The integrated server device 10 and the client terminal 20 according to the first to sixth embodiments include a processor 301, a main storage device 302, an auxiliary storage device 303, a display device 304, an input device 305, and a communication device 306. The processor 301, the main storage device 302, the auxiliary storage device 303, the display device 304, the input device 305, and the communication device 306 are connected via a bus 310.

[0154] Note that the integrated server device 10 and the client terminal 20 may not include some of the above configurations. For example, when the integrated server device 10 can use the input function and the display function of an external device, the integrated server device 10 may not be provided with the display device 304 and the input device 305.

[0155] The processor 301 executes the program read from the auxiliary storage device 303 into the main storage device 302. The main storage device 302 is a memory such as a ROM and a RAM. The auxiliary storage device 303 is an HDD (Hard Disk Drive), a memory card, or the like.

[0156] The display device 304 is, for example, a liquid crystal display or the like. The input device 305 is an interface for operating the integrated server device 10 and the client terminal 20. Note that the display device 304 and the input device 305 may be realized by a touch panel or the like having a display function and an input function. The communication device 306 is an interface for communicating with other devices.

[0157] For example, the programs executed on the integrated server device 10 and the client terminal 20 are files in an installable format or an executable format, and are recorded on a computer-readable storage medium such as a memory card, a hard disk, a CD-RW, a CD-ROM, a CD-R, a DVD-RAM, and a DVD-R, and are provided as a computer program product.

[0158] Also, for example, the programs executed on the integrated server device 10 and the client terminal 20 may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network.

[0159] Also, for example, the programs executed on the integrated server device 10 and the client terminal 20 may be configured to be provided via a network such as the Internet without being downloaded. Specifically, it may be configured to execute the process by a so-called ASP (Application Service Provider) type service in which the transfer of the program is not performed from the server computer and the processing function is realized only by the execution instruction and the result acquisition.

[0160] For example, the programs of the integrated server device 10 and the client terminal 20 may be configured to be provided by being pre - incorporated into a ROM or the like.

[0161] The programs executed by the integrated server device 10 and the client terminal 20 have a module configuration including functions that can also be realized by the programs among the above - described functional configurations. Each of these functions, as actual hardware, is such that when the processor 301 reads the program from the storage medium and executes it, the above - mentioned functional blocks are loaded onto the main storage device 302. That is, the above - mentioned functional blocks are generated on the main storage device 302.

[0162] Note that some or all of the above - described functions may be realized by hardware such as an IC (Integrated Circuit) instead of being realized by software.

[0163] Also, each function may be realized using a plurality of processors 301. In that case, each processor 301 may realize one of the functions, or may realize two or more of the functions.

[0164] Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the gist of the invention. These embodiments and their modifications are included in the scope and gist of the invention, and are included in the invention described in the claims and its equivalent scope.

Explanation of Reference Numerals

[0165] 1 Federated learning system 10 Integrated server device 20 Client terminal 101 Storage unit 102 Synchronization processing unit 103 Integration processing unit 104 Communication unit 105 Difference calculation unit 106 Detection unit 201 Memory unit 202 Learning unit 203 Encryption unit 204 Decryption unit 205 Communication unit 206 Difference calculation unit 207 Detection unit 301 Processor 302 Main memory device 303 Auxiliary storage device 304 Display device 305 Input device 306 Communication device 310 Bus

Claims

1. A system comprising a plurality of client devices and a plurality of server devices connected to the plurality of client devices, wherein each of the server devices comprises: a server receiving unit configured to receive local learning model parameters of a plurality of local learning models transmitted from the plurality of client devices via a plurality of communication channels; an integration processing unit configured to generate global learning model parameters of a global learning model by performing integration processing of federated learning using the plurality of local learning model parameters; a server difference calculation unit configured to perform a calculation process of a server difference indicating a difference between each of the plurality of local learning model parameters and the global learning model parameters; a server transmission unit configured to transmit the global learning model parameters to each of the plurality of client devices; wherein each of the plurality of client devices comprises: a client receiving unit configured to receive the global learning model parameters transmitted from the server device via the communication channel; a learning unit configured to perform learning processing of the local learning model using the global learning model parameters and learning data stored in the client device; a local difference calculation unit configured to perform a calculation process of a local difference indicating a difference between the global learning model parameters received from the server device and the local learning model parameters; a client transmission unit configured to transmit the local learning model parameters to the server device; wherein at least one of the plurality of client devices and the plurality of server devices comprises: a detection unit configured to perform a detection process of detecting an abnormality in any one of the server device, the client device, or the communication channel by comparing the server difference and the local difference; when the detection process is performed by the client device, the server transmission unit further transmits the server difference to the client device in which the detection process is performed; when the detection process is performed by the server device, the client transmission unit further transmits the local difference to the server device in which the detection process is performed; a federated learning system.

2. The integrated processing and the calculation processing of the server difference are executed by secret calculation that calculates each of the plurality of local learning model parameters and the global learning model parameters in a state of encrypted shares. The local learning model is updated based on parameters decrypted from the shares of the encrypted global learning model parameters. The federated learning system according to claim 1.

3. When the server difference and the local difference do not match, the detection unit detects the abnormality. The federated learning system according to claim 1.

4. The detection unit is provided in the client device. The detection unit identifies the location of the abnormality by changing a combination of the plurality of shares of the global learning model parameters used when the global learning model parameters are decrypted from the shares of the plurality of global learning model parameters. The federated learning system according to claim 2 or 3.

5. The client device The detection unit and a decryption unit that decrypts the global learning model parameters from the shares of the global learning model parameters received from the server device where the server difference and the local difference match. As a result of performing the detection in the t-th round, when the number of server devices where the server difference and the local difference match is less than the number of shares required for decrypting the global learning model parameters, the detection unit sets the shares of the global learning model parameters to those received from the server device in the detection in the (t - 1)-th round. The federated learning system according to claim 2 or 3.

6. As a result of performing detections in a plurality of rounds, the decryption unit does not use the shares of the global learning model parameters received from the server devices where the number of times the server difference and the local difference do not match is equal to or greater than the number of times threshold for decrypting the global learning model parameters. The federated learning system according to claim 5.

7. The detection unit is provided in the client device. When the number of the server devices in which the server difference and the local difference match is less than the number of shares required for decrypting the global learning model parameters, the detection unit stops the federated learning system. The federated learning system according to claim 2 or 3.

8. The detection unit is provided in the server device. When the number of the client devices in which the server difference and the local difference match is less than the client number threshold, the detection unit stops the federated learning system. The federated learning system according to claim 2 or 3.

9. The detection unit is provided in the server device. The integration processing unit generates shares of the global learning model parameters by executing integration processing of federated learning using shares of a plurality of the local learning model parameters. The shares of the local learning model parameters received from the client devices in which the server difference and the local difference do not match continuously for a predetermined number of times are not used for the integration processing by the detection unit. The federated learning system according to claim 2 or 3.

10. The detection unit is provided in the server device. The integration processing unit generates shares of the global learning model parameters by executing integration processing of federated learning using shares of a plurality of the local learning model parameters. The shares of the local learning model parameters received from the client devices in which the number of times the server difference and the local difference do not match reaches a predetermined total number of times are not used for the integration processing. The federated learning system according to claim 2 or 3.

11. A server reception unit that receives local learning model parameters of a plurality of local learning models transmitted from a plurality of client devices via a plurality of communication paths; An integration processing unit that generates global learning model parameters of a global learning model by executing integration processing of federated learning using the plurality of local learning model parameters; A server difference calculation unit that executes a calculation process of a server difference indicating a difference between each of the plurality of local learning model parameters and the global learning model parameters; And a server transmission unit that transmits the global learning model parameters to each of the plurality of client devices. The server receiving unit receives a local difference indicating a difference between the global learning model parameters and the local learning model parameters from each of the plurality of client devices, a detection unit that executes a detection process for detecting that there is an abnormality in either the own device, the client device, or the communication path by comparing the server difference and the local difference; A server device comprising:

12. A client receiving unit that receives global learning model parameters transmitted from a server device via a communication path; a learning unit that executes a learning process of a local learning model using the global learning model parameters and learning data; a local difference calculation unit that executes a calculation process of a local difference indicating a difference between the global learning model parameters received from the server device and the local learning model parameters of the local learning model; a client transmission unit that transmits the local learning model parameters to the server device, The client receiving unit receives a server difference indicating a difference between the global learning model parameters and the local learning model parameters from the server device, a detection unit that executes a detection process for detecting that there is an abnormality in either the own device, the server device, or the communication path by comparing the server difference and the local difference; A client device comprising:

13. A federated learning method for a federated learning system including a plurality of client devices and a plurality of server devices connected to the plurality of client devices, each of the server devices receiving local learning model parameters of a plurality of local learning models transmitted from the plurality of client devices via a plurality of communication paths; each of the server devices generating global learning model parameters of a global learning model by executing an integration process of federated learning using the plurality of local learning model parameters; each of the server devices executing a calculation process of a server difference indicating a difference between each of the plurality of local learning model parameters and the global learning model parameters; each of the server devices transmitting the global learning model parameters to each of the plurality of client devices; Each of the plurality of client devices receives the global learning model parameters transmitted from the server device via the communication path; Each of the plurality of client devices executes learning processing of the local learning model using the global learning model parameters and learning data stored in the client device; Each of the plurality of client devices executes a calculation process of a local difference indicating a difference between the global learning model parameters received from the server device and the local learning model parameters; Each of the plurality of client devices transmits the local learning model parameters to the server device; At least one of the plurality of client devices and the plurality of server devices executes a detection process of detecting that there is an abnormality in any one of the server device, the client device, or the communication path by comparing the server difference and the local difference; When the detection process is executed by the client device, each of the server devices further transmits the server difference to the client device in which the detection process is executed; When the detection process is executed by the server device, each of the plurality of client devices further transmits the local difference to the server device in which the detection process is executed; Federated learning method.

14. A server device, A server receiving unit that receives local learning model parameters of a plurality of local learning models transmitted from a plurality of client devices via a plurality of communication paths; An integration processing unit that generates global learning model parameters of a global learning model by executing integration processing of federated learning using the plurality of local learning model parameters; A server difference calculation unit that executes a calculation process of a server difference indicating a difference between each of the plurality of local learning model parameters and the global learning model parameters; Function as a server transmission unit that transmits the global learning model parameters to each of the plurality of client devices, The server receiving unit receives a local difference indicating a difference between the global learning model parameter and the local learning model parameter from each of the plurality of client devices. A detection unit that executes a detection process for detecting that there is an abnormality in any one of the server device, the client device, or the communication path by comparing the server difference and the local difference. A program for causing the above to function.

15. A client device A client receiving unit that receives a global learning model parameter transmitted from a server device via a communication path; A learning unit that executes a learning process of a local learning model using the global learning model parameter and learning data; A local difference calculation unit that executes a calculation process of a local difference indicating a difference between the global learning model parameter received from the server device and the local learning model parameter of the local learning model; Function as a client transmission unit that transmits the local learning model parameter to the server device. The client receiving unit receives a server difference indicating a difference between the global learning model parameter and the local learning model parameter from the server device. A detection unit that executes a detection process for detecting that there is an abnormality in any one of the server device, the client device, or the communication path by comparing the server difference and the local difference. A program for causing the above to function.

Images