Secret global model calculation device, secret global model calculation system configuration method, and program

Abstract

Provided is technology for efficiently learning a model in federated learning, by devices that configure a secret computation system that computes a global model being provided with a function for learning a local model. According to the present invention, an arbitrary N secret global model computation devices from among K secret global model computation devices can configure a secret global model computation system in which a global model is secretly computed from N local models, where K is an integer equal to or greater than 3 and N is an integer satisfying 3≤N≤K. Moreover, each secret global model computation device comprises: a selection unit for selecting, from among the other K-1 secret global model computation devices, N-1 secret global model computation devices having large availability of computation resources; and a system configuration unit for configuring a secret global model computation system from N secret global model computation devices including the secret global model computation device and the N-1 selected secret global model computation devices.

Description

[Technical Field]

[0001] The present invention relates to a federated learning technology, and more particularly to a technology for efficiently learning a model by providing a function for learning a local model in a device that constitutes a secure computing system that calculates a global model. [Background technology]

[0002] Federated learning is a technique for learning without consolidating learning data in a single device. For example, FedAVG, described in Non-Patent Document 1, is an example of a federated learning technique.

[0003] FIG. 1 shows the basic configuration of a federated learning system 90. The federated learning system 90 includes M (M is an integer of 2 or greater) local model learning devices 7001, ..., 700 M and a global model calculation device 900. The basic operation of the federated learning system 90 is as follows: Local model learning devices 7001, ..., 700 M After the learning is completed, the local model learning devices 7001, ..., 700 M The local model computation device 900 transmits the local model to the global model computation device 900 via the network 800. The global model computation device 900 computes the global model using the received local model. After the computation is completed, the global model computation device 900 transmits the global model to the local model learning devices 7001, ..., 700 via the network 800. M The local model learning devices 7001, ..., 700 M The global model computing device 900 then uses the received global model to re-learn the local model. By repeating this operation, the federated learning system 90 advances model learning. At this time, the global model computing device 900 manages the parameters of the local model using a local model management table.

[0004] When using federated learning technology, the learning data is not taken out of the local model learning device, which eliminates concerns about data taking out and also enables faster parallel learning. M There is a risk that the training data may be inferred if the parameters of the model being trained are tracked and leaked during the communication process between the global model computation device 900. In order to avoid such a risk, it is conceivable to use secure computation for the calculation of the global model.

[0005] Secure computation is a method of obtaining the result of a specified computation without restoring the encrypted numerical value (see, for example, Reference Non-Patent Document 1). In the method of Reference Non-Patent Document 1, encryption is performed by distributing multiple pieces of information from which a numerical value can be restored to three secure computing devices, and the results of addition / subtraction, constant sum, multiplication, constant multiplication, logical operations (negation, logical product, logical sum, exclusive logical sum), and data format conversion (integer, binary number) can be stored in a distributed state, i.e., encrypted, among the three secure computing devices without restoring the numerical value. In general, the number of shares is not limited to 3 and can be N (N is an integer equal to or greater than 3), and a protocol that realizes secure computation through cooperative computation by N secure computing devices is called a multi-party protocol. (Reference Non-Patent Document 1: Koji Senda, Hiroki Hamada, Dai Igarashi, Katsumi Takahashi, “Rethinking Lightweight Verifiable Three-Party Secure Function Computation,” In CSS, 2010.) [Prior art documents] [Non-patent literature]

[0006] [Non-Patent Document 1] McMahan, B., E. Moore, D. Ramage, et al, “Communication-efficient learning of deep networks from decentralized data,” In Artificial Intelligence and Statistics, pp.1273-1282, 2017. Summary of the Invention [Problem to be solved by the invention]

[0007] In general, the number of devices that learn a local model tends to be excessively large compared to the number of devices that make up the secure computation system that calculates the global model. In such cases, the process of registering a local model in the secure computation system takes time, which results in a problem of an overall increase in the time required for model learning.

[0008] Therefore, an object of the present invention is to provide a technology for efficiently learning a model in federated learning by providing a function for learning a local model to a device that constitutes a secure computing system that calculates a global model. [Means for solving the problem]

[0009] One aspect of the present invention is a secret global model computation device in a federated learning system including K secret global model computation devices that learn local models using learning data, where K is an integer greater than or equal to 3 and N is an integer satisfying 3≦N≦K, and any N of the K secret global model computation devices can constitute a secret global model computation system that secretly computes a global model from the N local models, and includes a selection unit that sends a query to K-1 secret global model computation devices excluding itself to check the availability of computational resources, and selects N-1 secret global model computation devices with the largest availability of computational resources from the K-1 secret global model computation devices, and a system configuration unit that constitutes a secret global model computation system with the N secret global model computation devices including the selected N-1 secret global model computation devices and itself.

[0010] One aspect of the present invention is a secret global model calculation device in a federated learning system including K secret global model calculation devices that learn local models using learning data, where K is an integer equal to or greater than 3, P is an integer equal to or greater than 2, and N is an integer satisfying 3≦N≦K / P, and any N secret global model calculation devices out of the K secret global model calculation devices can constitute a secret global model calculation system that secretly calculates a global model from the N local models, and K−(p−1)N (where p is an integer satisfying 1≦p≦P) that are not selected for configuring the secret global model calculation system are used. a selection unit that transmits a query to check availability of computational resources to K-(p-1)N-1 secret global model computation devices excluding itself among the K-(p-1)N-1 secret global model computation devices (number of secret global model computation devices) and selects N-1 secret global model computation devices with the largest availability of computational resources from the K-(p-1)N-1 secret global model computation devices; a system configuration unit that configures a secret global model computation system (hereinafter referred to as the p-th secret global model computation system) with N secret global model computation devices including the selected N-1 secret global model computation devices and itself; ..., P}, where L is an integer equal to or greater than 1 and p is an integer satisfying σ(p)>1, the local model learning unit learns the local model using the parameters of the global model of the p'th secret global model calculation system (where p' is an integer satisfying σ(p')=σ(p)-1) in one learning session out of L sessions, and the parameters of the global model of the pth secret global model calculation system as the initial values of the parameters of the local model in the remaining L-1 learning sessions out of L sessions, and when p is an integer satisfying σ(p)=1, the local model learning unit learns the local model using the parameters of the global model of the pth secret global model calculation system as the initial values of the parameters of the local model in the second and subsequent learning sessions. [Effects of the Invention]

[0011] According to the present invention, in federated learning, the devices constituting the secure computing system that computes the global model are provided with the function of learning the local model, thereby enabling efficient model learning. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a diagram showing the basic configuration of a federated learning system 90. [Figure 2] FIG. 1 is a block diagram showing the configuration of an associative learning system 10. [Figure 3] FIG. 2 is a block diagram showing the configuration of a secret global model calculation device 100n. [Figure 4] 1 is a flowchart showing the operation of the associative learning system 10. [Figure 5] FIG. 2 is a block diagram showing a configuration of a local model generating unit 130n. [Figure 6] 10 is a flowchart showing the operation of a local model generation unit 130n. [Figure 7] FIG. 2 is a diagram illustrating an example of the functional configuration of a computer that realizes each device according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, embodiments of the present invention will be described in detail. Components having the same functions are given the same numbers, and duplicated explanations will be omitted.

[0014] Before describing each embodiment, the notation used in this specification will be explained.

[0015] ^ (caret) represents a superscript, e.g., x y^z Yes z is a superscript to x, and x y^z Yes z is a subscript to x. Also, _ (underscore) represents a subscript. For example, x y_z Yes z is a superscript to x, and xy_z Yes z is a subscript to x.

[0016] The superscripts "^" and "~" such as ^x and ~x for a certain letter x should be written directly above the "x", but due to restrictions on the notation in the specification, they are written as ^x and ~x. <Technical background> <<Secure calculation>> The secure computation in the present invention is constructed by combining existing secure computation operations. The operations required for this secure computation include, for example, concealment, addition, subtraction, multiplication, division, logical operations (negation, logical AND, logical OR, exclusive OR), and comparison operations (=, <, >, ≦, ≧). Below, we will explain some of the operations, including their notations. [Redacted] Let [[x]] be the value of x concealed by secret sharing (hereinafter referred to as a share of x). Any secret sharing method can be used. For example, 61 -1) Shamir secret sharing on Z2 and replication secret sharing on Z2 can be used.

[0017] Multiple secret sharing methods may be combined within a single algorithm. In this case, they will be converted into each other as appropriate.

[0018] Also, N-dimensional vector → x=(x1, …, x N ) for [[ → x]]=([[x1]], …, [[x N ]]). That is, [[ → x]] is → The nth element of x, x n Share [[x n ]] is a vector with n-th element. Similarly, M×N matrix A=(a m,n )(1≦m≦M, 1≦n≦N), let [[A]] be the (m, n)th element a of A. m,n Share of [[a m,n Let ]] be the matrix whose element is (m, n).

[0019] Note that x is called the plaintext of [[x]].

[0020] Specific examples of a method for obtaining [[x]] from x (concealment) and a method for obtaining x from [[x]] (restoration) are described in Reference Non-Patent Document 1 and Reference Non-Patent Document 2. (Reference Non-Patent Document 2: Shamir, A., "How to share a secret", Communications of the ACM, Vol. 22, No. 11, pp. 612-613, 1979.) [Addition, Subtraction, Multiplication, Division] Secure addition [[x]]+[[y]] takes inputs [[x]] and [[y]] and outputs [[x+y]]. Secure subtraction [[x]]-[[y]] takes inputs [[x]] and [[y]] and outputs [[xy]]. Secure multiplication [[x]]×[[y]] (sometimes expressed as mul([[x]], [[y]])) takes inputs [[x]] and [[y]] and outputs [[x×y]]. Secure division [[x]] / [[y]] (sometimes expressed as div([[x]], [[y]])) takes inputs [[x]] and [[y]] and outputs [[x / y]].

[0021] Specific methods for addition, subtraction, multiplication, and division include those described in Reference Non-Patent Documents 3 and 4. (Reference Non-Patent Document 3: Ben-Or, M., Goldwasser, S. and Wigderson, A., “Completeness theorems for non-cryptographic fault-tolerant distributed computation”, Proceedings of the twentieth annual ACM symposium on Theory of computing, ACM, pp. 1-10, 1988.) (Reference Non-Patent Document 4: Gennaro, R., Rabin, MO and Rabin, T., “Simplified VSS and fast-track multiparty computations with applications to threshold cryptography”, Proceedings of the seventeenth annual ACM symposium on Principles of distributed computing, ACM, pp.101-111, 1998.) [Logical Operations] Secure negation not[[x]] takes [[x]] as input and outputs [[not(x)]]. Secure logical conjunction and([[x]], [[y]]) takes [[x]], [[y]] as input and outputs [[and(x, y)]]. Secure logical sum or([[x]], [[y]]) takes [[x]], [[y]] as input and outputs [[or(x, y)]]. Secure exclusive OR xor([[x]], [[y]]) takes [[x]], [[y]] as input and outputs [[xor(x, y)]].

[0022] Note that logical operations can be easily configured by combining addition, subtraction, multiplication, and division. [Comparison operation] Equality determination by secret calculation =([[x]], [[y]]) (sometimes denoted as equal([[x]], [[y]])) takes [[x]] and [[y]] as inputs and outputs [[1]] if x = y and [[0]] otherwise. Comparison by secret calculation <([[x]], [[y]]) takes [[x]] and [[y]] as inputs and outputs [[1]] if x < y and [[0]] otherwise. Comparison by secret calculation >([[x]], [[y]]) takes [[x]] and [[y]] as inputs and outputs [[1]] if x > y and [[0]] otherwise. Comparison by secret calculation ≦([[x]], [[y]]) takes [[x]] and [[y]] as inputs and outputs [[1]] if x ≦ y and [[0]] otherwise. Comparison by secret calculation ≧([[x]], [[y]]) takes [[x]] and [[y]] as inputs and outputs [[1]] if x ≧ y and [[0]] otherwise.

[0023] Note that the comparison operation can be easily configured by combining logical operations. <First Embodiment> Hereinafter, the federated learning system 10 will be described with reference to FIGS. 2 to 4. FIG. 2 is a block diagram showing the configuration of the federated learning system 10. The federated learning system 10 includes K (K is an integer of 3 or more) secret global model calculation devices 1001,..., 100 K which are included. The secret global model calculation devices 1001,..., 100 K are connected to the network 800 and can communicate with each other. The network 800 may be, for example, a communication network such as the Internet or a broadcast communication path. FIG. 3 is a block diagram showing the configuration of the secret global model calculation device 100 n (1 ≦ n ≦ K). FIG. 4 is a flowchart showing the operation of the federated learning system 10.

[0024] As shown in FIG. 3, the secret global model calculation device 100 n includes a selection unit 110 n and a system configuration unit 120n and the local model generation unit 130 n and the parameter share registration unit 140 n and the learning start condition determination unit 150 n and the global model calculation unit 160 n and the transmitting / receiving unit 180 n and recording unit 190 n The selection unit 110 n and system configuration unit 120 n and the local model generation unit 130 n and the parameter share registration unit 140 n and the transmitter / receiver 180 n and Recording Section 190 n Excluding the secret global model computing device 100 n Each of the components is configured to be able to execute operations required to realize the function of each component, such as concealment, addition, subtraction, multiplication, division, logical operation, and comparison operation, among the operations required for calculation of the global model. In the present invention, the specific functional configuration for realizing each operation is sufficient if it is a configuration that can execute existing algorithms, and since these are conventional configurations, detailed description will be omitted. In addition, the recording unit 190 n The secret global model calculation device 100 n The recording unit 190 is a component that records information necessary for the processing of the n For example, the global model records training data, local model parameters, and global model parameters. The training data is updated as needed. The global model has the same structure as the local model.

[0025] Any N (N is an integer satisfying 3≦N≦K) secret global model calculation devices out of the K secret global model calculation devices can constitute a secret global model calculation system that secretly calculates a global model from N local models.

[0026] By the collaborative calculation by any N secret global model calculation devices among the K secret global model calculation devices, the secret global model calculation system configured by the N secret global model calculation devices realizes the secret calculation of the global model, which is a multi-party protocol. Therefore, the N secret global model calculation devices are referred to as the secret global model calculation devices 100. i_1 , …, 100 i_N (where i1, …, i N is 1≦i1< … N ≦K), the learning start condition determining means 150 (not shown) of the secret global model computation system configured by the N secret global model computation devices is a learning start condition determining unit 150 i_1 , …, 150 i_N The global model calculation means 160 (not shown) is a global model calculation unit 160 i_1 , …, 160 i_N It consists of:

[0027] The operation of the associative learning system 10 will be described below with reference to FIG.

[0028] In S110, the secret global model calculation device 100 n One secret global model calculation device 100 out of (1≦n≦K) k_0 (where k0 is an integer satisfying 1≦k0≦K) selection unit 110 k_0 The transmitting / receiving unit 180 k_0 The secret global model computing device 100 sends a query to check the availability of computing resources to K-1 secret global model computing devices excluding itself using the above, and selects N-1 secret global model computing devices with the largest availability of computing resources from the K-1 secret global model computing devices. k_0 The N secret global model calculation devices, which are the sum of the selected N-1 secret global model calculation devices and the master secret global model calculation device, are called secret global model calculation devices 100. i_1 , …, 100 i_N ​(where i1, …, i N is 1≦i1< … N ≦K). Therefore, k0 is an integer that satisfies i1, …, i N where k0=i ~n (where ∼n is an integer that satisfies 1≦∼n≦N). Furthermore, the computational resources refer to, for example, the CPU and communication unit of a general-purpose computer used to configure the secret global model computation device.

[0029] In S120, the secret global model computing device 100, which is the master secret global model computing device, i_~n System configuration section 110 i_~n The transmitting / receiving unit 180 i_~n Using the secret global model calculation device 100 i_1 , …, 100 i_~n-1 , 100 i_~n+1 , …, 100 i_N and mutually exchange data required to configure the secret global model calculation system with the N secret global model calculation devices 100. i_1 , …, 100 i_N The secret global model computation system is configured with the above data. Data required to configure the secret global model computation system is, for example, the network address (for example, IP address) of the secret global model computation device that configures the secret global model computation system.

[0030] In S130, the secret global model calculation device 100 n (n=i1, …, i N ) local model generation unit 130 n generates a local model.

[0031] Hereinafter, the local model generation unit 130 will be described with reference to FIGS. 5 and 6. n FIG. 5 shows the local model generation unit 130 n 6 is a block diagram showing the configuration of the local model generation unit 130. n ​5 is a flowchart showing the operation of the local model generation unit 130. n (1≦n≦K) is the local model learning unit 131 n and the parameter share calculation unit 132 n and the global model acquisition unit 133 n and the parameter calculation unit 134 n and the learning start condition determination unit 135 n Includes:

[0032] Hereinafter, the local model generation unit 130 will be described with reference to FIG. n (n=i1, …, i N ) operation will be explained.

[0033] S131 n In the local model learning unit 131 n is the recording unit 190 n In the first learning of the local model, the local model learning unit 131 n is stored in advance in the recording unit 190 n The initial values of the parameters of the local model may be set using the initial values recorded in the local model learning unit 131, or may be set using initial values generated using random numbers. n is S133, which will be described later. n The initial values of the parameters of the local model are set using the share of the parameters of the global model obtained in step 2.

[0034] S132 n In the parameter share calculation unit 132 n is S131 n The parameter share calculation unit 132 calculates the parameter share of the local model from the parameters of the local model learned in the step 132. n When the calculation is completed, the transmitting / receiving unit 180 n A secret global model computing device 100 excluding itself using i_1 , …, 100 i_n-1 , 100 i_n+1, …, 100 i_N Send a share of the local model parameters to

[0035] S133 n In the global model acquisition unit 133 n is S132 n After the processing of S135 n After a predetermined time has elapsed since the end of the process, the transmitting / receiving unit 180 n A secret global model computing device 100 excluding itself using i_1 , …, 100 i_n-1 , 100 i_n+1 , …, 100 i_N Obtain the parameter shares of the global model from

[0036] S134 n In the parameter calculation unit 134 n is S133 n The parameter calculation unit 134 calculates the parameters of the global model from the share of the parameters of the global model acquired by the parameter calculation unit 134 and the share of the parameters of the global model calculated by the parameter calculation unit 134. n The parameters of the calculated global model are recorded in the recording unit 190. n The recording unit 190 n At least two global model parameters, namely, the global model parameters obtained in the current calculation and the global model parameters obtained in the previous calculation, are recorded in .

[0037] S135 n In the learning start condition determination unit 135 n is S134 n The parameters of the global model calculated in step S131 are compared with the parameters of the global model obtained in the previous calculation, and if the parameters of the two global models are different, it is determined that the learning start condition is satisfied, and step S131 n Otherwise, it is determined that the learning start condition is not satisfied, and the process proceeds to S133. n Return to processing.

[0038] In S140, the secret global model calculation device 100 n (n=i1, …, i N ) parameter share registration unit 140 n The transmitting / receiving unit 180 n The secret global model computing device 100 receives the secret global model excluding itself using i_1 , …, 100 i_n-1 , 100 i_n+1 , …, 100 i_N The local model learning device 190 receives the parameter share of a local model learned by one of the local model learning devices or the parameter share of a local model learned by itself as an input, and registers the parameter share of the local model in a local model management table. Here, the local model management table is a table for managing the parameter shares of N local models, and is stored in the recording unit 190. n It is recorded in.

[0039] In S150, if the number of newly registered local models since the last global model calculation exceeds or becomes equal to a predetermined value (the value is between 1 and N), the learning start condition determination means 150 determines that the learning start condition is satisfied and executes the processing of S160; otherwise, it determines that the learning start condition is not satisfied and returns to the processing of S140.

[0040] In S160, the global model calculation means 160 calculates the parameter share of the global model using the parameter shares of the local models managed in the local model management table. For example, the global model calculation means 160 determines the parameter share of the global model to be the average of the parameter shares of the corresponding first to Nth local models.

[0041] By periodically executing the process of S110, it is possible to periodically change the combination of the private global model computation devices that constitute the private global model computation system.

[0042] According to an embodiment of the present invention, in federated learning, the devices constituting the secure computing system that computes the global model have the function of learning the local model, thereby enabling efficient model learning. Also, by having the secret global model computing device have the local model learning function and configuring the secret global model computing system using secret global model computing devices with available computing resources, it is possible to reduce the amount of calculation and communication and learn the model efficiently. Furthermore, by periodically changing the combination of secret global model computing devices that constitute the secret global model computing system, it is possible to reduce the risk of leakage due to collusion among operators. Second Embodiment In the first embodiment, one secret global model computation system is configured, but the number of configured secret global model computation systems may be two or more. In this case, each of the two or more secret global model computation systems forms one layer, and the federated learning system as a whole learns the model by using the share of parameters of the global model calculated by the secret global model computation system that constitutes a certain layer to set initial values when the secret global model computation system that constitutes the next layer learns the local model, like calculations in a neural network.

[0043] Below, an example will be described in which the federated learning system 10 comprises two secret global model computation systems.

[0044] In this case, any N (N is an integer satisfying 3≦N≦K / 2) secret global model calculation devices out of the K secret global model calculation devices can constitute a secret global model calculation system that secretly calculates a global model from N local models.

[0045] The operation of the associative learning system 10 will be described below with reference to FIG.

[0046] In S110, the secret global model calculation device 100 nOne secret global model calculation device 100 out of (1≦n≦K) k_0 (where k0 is an integer satisfying 1≦k0≦K) selection unit 110 k_0 The transmitting / receiving unit 180 k_0 The secret global model computing device 100 sends a query to check the availability of computing resources to K-1 secret global model computing devices excluding itself using the above, and selects N-1 secret global model computing devices with the largest availability of computing resources from the K-1 secret global model computing devices. k_0 The N secret global model calculation devices, which are a combination of the selected N-1 secret global model calculation devices and the first master secret global model calculation device, are referred to as secret global model calculation devices 100. i_1 , …, 100 i_N (where i1, …, i N is 1≦i1< … N ≦K). Therefore, k0 is an integer that satisfies i1, …, i N where k0=i ~n (where ~n is an integer that satisfies 1≦~n≦N).

[0047] Similarly, the secret global model calculation device 100 n One secret global model calculation device 100 out of (1≦n≦K) k_1 (where k1 is an integer that satisfies 1≦k1≦K, and i1, …, i N ), that is, KN secret global model computing devices 100 that are not selected for the secret global model computing system configuration. n One of the secret global model computing devices 100 k_1 Selection unit 110 k_1 The transmitting / receiving unit 180 k_1 Using yourself and the secret global model computing device 100 i_1 , …, 100 i_N KN-1 secret global model calculation devices 100 excluding n ​a query to check the availability of computing resources to the KN-1 secret global model computing devices 100 n , N-1 secret global model computing devices with the most available computing resources are selected. k_1 The N secret global model calculation devices, which are a combination of the selected N-1 secret global model calculation devices and the second master secret global model calculation device, are referred to as secret global model calculation devices 100. i_N+1 , …, 100 i_2N (However, i N+1 , …, i 2N is 1≦i N+1 < … 2N ≦K, where i1, …, i N Therefore, k1 is different from i N+1 , …, i 2N where k1=i ^n (where ^n is an integer that satisfies 1≦^n≦N). Note that the secret global model calculation device 100 k_1 is, for example, the secret global model calculation device 100 i_~n The secret global model calculation device 100 i_1 , …, 100 i_N It is assumed that it is known in advance that the device cannot be selected.

[0048] In S120, the secret global model computing device 100, which is the first master secret global model computing device, i_~n System configuration section 110 i_~n The transmitting / receiving unit 180 i_~n Using the secret global model calculation device 100 i_1 , …, 100 i_~n-1 , 100 i_~n+1 , …, 100 i_N and mutually exchange data required to configure the secret global model calculation system with the N secret global model calculation devices 100. i_1 , …, 100 i_N ​This constitutes a secret global model calculation system (hereinafter referred to as the first secret global model calculation system).

[0049] Similarly, the secret global model computing device 100, which is the second master secret global model computing device, i_^n System configuration section 110 i_^n The transmitting / receiving unit 180 i_^n Using the secret global model calculation device 100 i_N+1 , …, 100 i_^n-1 , 100 i_^n+1 , …, 100 i_2N and mutually exchange data required to configure the secret global model calculation system with the N secret global model calculation devices 100. i_N+1 , …, 100 i_2N This constitutes a secret global model calculation system (hereinafter referred to as the second secret global model calculation system).

[0050] In S130, the secret global model calculation device 100 n (n=i1, …, i N ) local model generation unit 130 n generates a local model.

[0051] Similarly, the secret global model calculation device 100 n (n=i N+1 , …, i 2N ) local model generation unit 130 n generates a local model.

[0052] Hereinafter, the local model generation unit 130 will be described with reference to FIG. n (n=i N+1 , …, i 2N ) operation will be explained.

[0053] S131 n In the local model learning unit 131 n is the recording unit 190 nThe local model is learned using the learning data recorded in the local model learning unit 131. n is S133, which will be described later. n The initial values of the parameters of the local model are set using the share of the parameters of the global model obtained in step 2.

[0054] S132 n In the parameter share calculation unit 132 n is S131 n The parameter share calculation unit 132 calculates the parameter share of the local model from the parameters of the local model learned in the step 132. n When the calculation is completed, the transmitting / receiving unit 180 n A secret global model computing device 100 excluding itself using i_N+1 , …, 100 i_n-1 , 100 i_n+1 , …, 100 i_2N Send a share of the local model parameters to

[0055] S133 n In the global model acquisition unit 133 n is S132 n After the processing of S135 n After a predetermined time has elapsed since the end of the process, the transmitting / receiving unit 180 n Using the secret global model calculation device 100 i_1 , …, 100 i_N , or the secret global model computing device 100 excluding itself i_N+1 , …, 100 i_n-1 , 100 i_n+1 , …, 100 i_2N In the first acquisition, the share of the global model parameters is acquired from the secret global model calculation device 100 i_1 , …, 100 i_N From the second acquisition onwards, the share of the global model parameters is acquired from the secret global model calculation device 100 in one acquisition out of L attempts (where L is an integer equal to or greater than 2). i_1 , …, 100 i_Nand obtains a share of the global model parameters from the secret global model calculation device 100 by obtaining the remaining L-1 times out of L times. i_N+1 , …, 100 i_n-1 , 100 i_n+1 , …, 100 i_2N Obtain the parameter shares of the global model from

[0056] Therefore, the local model learning unit 131 n In one of the L learning times, the local model is learned using the share of parameters of the global model of the first secret global model calculation system as the initial value of the parameters of the local model, and in the remaining L-1 learning times out of the L learning times, the share of parameters of the global model of the second secret global model calculation system as the initial value of the parameters of the local model.

[0057] S134 n In the parameter calculation unit 134 n is S133 n Share of global model parameters obtained in S133 n The parameter calculation unit 134 calculates the parameters of the global model from the share of the parameters of the global model acquired by the parameter calculation unit 134 and the share of the parameters of the global model calculated by the parameter calculation unit 134. n The parameters of the calculated global model are recorded in the recording unit 190. n The recording unit 190 n At least two global model parameters, namely, the global model parameters obtained in the current calculation and the global model parameters obtained in the previous calculation, are recorded in .

[0058] S135 n In the learning start condition determination unit 135 n is S134 n The parameters of the global model calculated in step S131 are compared with the parameters of the global model obtained in the previous calculation, and if the parameters of the two global models are different, it is determined that the learning start condition is satisfied, and step S131 nOtherwise, it is determined that the learning start condition is not satisfied, and the process proceeds to S133. n Return to processing.

[0059] Above S130 n Secret global model computing device 100 in n (n=i1, …, i N ) local model generation unit 130 n and Secret Global Model Computing Device 100 n (n=i N+1 , …, i 2N ) local model generation unit 130 n In the description of the operation of the secret global model computing device 100 n (n=i N+1 , …, i 2N ) Global Model Acquisition Section 133 n Secret Global Model Computing Device 100 n (n=i1, …, i N ) Global Model Acquisition Section 133 n However, the reverse may be true. That is, the secret global model calculation device 100 n (n=i1, …, i N ) Global Model Acquisition Section 133 n Secret Global Model Computing Device 100 n (n=i N+1 , …, i 2N ) Global Model Acquisition Section 133 n may obtain a share of the parameters of the global model generated by the

[0060] In S140, the secret global model calculation device 100 n (n=i1, …, i N ) parameter share registration unit 140 n The transmitting / receiving unit 180 n The secret global model computing device 100 receives the secret global model excluding itself using i_1 , …, 100 i_n-1 , 100 i_n+1 , …, 100i_N The local model learning device inputs the parameter share of a local model learned by one of the local model learning devices or the parameter share of a local model learned by itself, and registers the parameter share of the local model in a local model management table.

[0061] Similarly, the secret global model calculation device 100 n (n=i N+1 , …, i 2N ) parameter share registration unit 140 n The transmitting / receiving unit 180 n The secret global model computing device 100 receives the secret global model excluding itself using i_N+1 , …, 100 i_n-1 , 100 i_n+1 , …, 100 i_2N The local model learning device inputs the parameter share of a local model learned by one of the local model learning devices or the parameter share of a local model learned by itself, and registers the parameter share of the local model in a local model management table.

[0062] In S150, the learning start condition determination means 150 of the first secret global model calculation device and the learning start condition determination means 150 of the second secret global model calculation device each determine that the learning start condition is satisfied if the number of newly registered local models since the previous global model calculation has exceeded a predetermined value (the value is 1 to N inclusive) or has become equal to or greater than the predetermined value, and execute the processing of S160; otherwise, they determine that the learning start condition is not satisfied, and return to the processing of S140.

[0063] In S160, the global model calculation means 160 of the first secret global model calculation device and the global model calculation means 160 of the second secret global model calculation device each calculate the parameter share of the global model using the parameter share of the local model managed in the local model management table.

[0064] Next, a brief description will be given of a case where federated learning system 10 constitutes P secret global model computation systems, where P is an integer equal to or greater than 2 (see FIG. 4).

[0065] In this case, any N (N is an integer satisfying 3≦N≦K / P) secret global model calculation devices out of the K secret global model calculation devices can constitute a secret global model calculation system that secretly calculates a global model from N local models.

[0066] In S110, one secret global model computing device 100 out of K-(p-1)N (where p is an integer satisfying 1≦p≦P) secret global model computing devices that have not been selected for the secret global model computing system configuration is selected. n (hereinafter referred to as the p-th master secret global model calculation device) n The transmitting / receiving unit 180 n The N-1 secret global model computing devices with the largest available computing resources are selected from the K-(p-1)N-1 secret global model computing devices using the above.

[0067] In S120, the pth master secret global model calculation device 100 n System configuration section 120 n The transmitting / receiving unit 180 n The N-1 secret global model calculation devices selected in S110 and the N secret global model calculation devices including itself are mutually exchanged data required to construct a secret global model calculation system using the above, and the N secret global model calculation devices form a secret global model calculation system (hereinafter referred to as the pth secret global model calculation system).

[0068] In S130, the secret global model calculation device 100 constituting the p-th secret global model calculation system n The local model generation unit 130n generates a local model. At this time, the local model generation unit 130 n Local model learning unit 131 included in n The local model learning unit 131 learns a local model using the learning data. Specifically, σ is a permutation of the set {1, ..., P}, L is an integer equal to or greater than 2, and n If p is an integer that satisfies σ(p) > 1, then in one of the L learning times, the local model is trained using the parameters of the global model of the p'th secret global model computation system (where p' is an integer that satisfies σ(p') = σ(p) - 1), and in the remaining L-1 learning times out of the L learning times, the local model is trained using the parameters of the global model of the pth secret global model computation system as the initial values of the parameters of the local model; and if p is an integer that satisfies σ(p) = 1, then in the second and subsequent learning times, the local model is trained using the parameters of the global model of the pth secret global model computation system as the initial values of the parameters of the local model. Note that σ(p') = σ(p) - 1 shows that p' is an integer different from p that satisfies 1 ≦ p' ≦ P.

[0069] In S140, the secret global model calculation device 100 constituting the p-th secret global model calculation system n Parameter share registration unit 140 n The secret global model calculation device 100 constituting the p-th secret global model calculation system n The parameter shares of the learned local model are input, and the parameter shares of the local model are registered in the local model management table.

[0070] In S150, if the number of newly registered local models since the previous global model calculation exceeds a predetermined value (the value is 1 to N inclusive) or becomes equal to or greater than the predetermined value, the learning start condition determination means 150 of the pth master secret global model calculation device determines that the learning start condition is satisfied and executes the processing of S160; otherwise, it determines that the learning start condition is not satisfied and returns to the processing of S140.

[0071] In S160, the global model calculation means 160 of the p-th master secret global model calculation device calculates the parameter shares of the global model using the parameter shares of the local models managed in the local model management table.

[0072] According to an embodiment of the present invention, in federated learning, the devices constituting the secure computing system that computes the global model are provided with a function for learning a local model, thereby enabling efficient model learning. Furthermore, by providing the secret global model computing device with a local model learning function and configuring the secret global model computing system using secret global model computing devices with available computing resources, it is possible to reduce the amount of calculation and communication traffic and efficiently learn the model. Furthermore, each of two or more secret global model computing systems forms one layer, and the federated learning system as a whole learns the model in a manner similar to calculations in a neural network, thereby enabling efficient model learning by reusing the global model. <Additional Notes> The processing of each unit of each of the above-mentioned devices may be realized by a computer, in which case the processing content of the functions that each device should have is described by a program. Then, by loading this program into the recording unit 2020 of the computer 2000 shown in Fig. 7 and operating the arithmetic processing unit 2010, the input unit 2030, the output unit 2040, the auxiliary recording unit 2025, etc., various processing functions of each of the above-mentioned devices are realized on the computer.

[0073] The device of the present invention may, for example, be a single hardware entity, having an input unit capable of inputting signals from outside the hardware entity, an output unit capable of outputting signals to outside the hardware entity, a communication unit to which a communication device (e.g., a communication cable) can be connected for communication with outside the hardware entity, a CPU (which may also include a central processing unit, cache memory, registers, etc.) as an arithmetic processing unit, RAM and ROM as memories, an external storage device such as a hard disk, and buses connecting these input unit, output unit, communication unit, CPU, RAM, ROM, and external storage device so as to enable data exchange. If necessary, the hardware entity may also be provided with a device (drive) capable of reading and writing to a recording medium such as a CD-ROM. An example of a physical entity equipped with such hardware resources is a general-purpose computer.

[0074] The external storage device of the hardware entity stores the programs required to realize the above-mentioned functions and the data required for processing these programs (the programs may be stored in a ROM, which is a read-only storage device, for example, instead of an external storage device). Data obtained by processing these programs is stored in RAM, the external storage device, etc. as appropriate.

[0075] In the hardware entity, each program stored in an external storage device (or ROM, etc.) and data required for processing each program are loaded into memory as needed, and interpreted, executed, and processed by the CPU as appropriate. As a result, the CPU realizes predetermined functions (each component represented as the above, "... unit," "... means," etc.). In other words, each component in the embodiments of the present invention may be configured by a processing circuitry.

[0076] As described above, when the processing functions of the hardware entities (apparatuses of the present invention) described in the above embodiments are realized by a computer, the processing contents of the functions that the hardware entities should have are described by a program. Then, by executing this program on a computer, the processing functions of the hardware entities are realized on the computer.

[0077] The program describing the processing contents can be recorded on a computer-readable recording medium, such as a non-transitory recording medium, specifically a magnetic recording device, an optical disk, or the like.

[0078] The program may be distributed, for example, by selling, transferring, lending, etc. a portable recording medium such as a DVD or CD-ROM on which the program is recorded. Furthermore, the program may be stored in a storage device of a server computer, and then transferred from the server computer to another computer via a network, thereby distributing the program.

[0079] A computer that executes such a program, for example, first stores the program recorded on a portable recording medium or transferred from a server computer in its own non-transitory storage device, the auxiliary storage unit 2025. Then, when executing a process, the computer loads the program stored in its own non-transitory storage device, the auxiliary storage unit 2025, into the storage unit 2020 and executes processing in accordance with the loaded program. Alternatively, as another execution mode of this program, the computer may load the program directly from a portable recording medium into the storage unit 2020 and execute processing in accordance with the program. Furthermore, each time a program is transferred from a server computer to this computer, the computer may execute processing in accordance with the received program. Alternatively, the server computer may not transfer the program to this computer, but may instead execute the processing function by issuing an execution instruction and obtaining the results, thereby executing the above-described processing through a so-called ASP (Application Service Provider) type service. Note that the program in this embodiment includes information used for processing by a computer that is equivalent to a program (such as data that is not a direct instruction to a computer but has properties that define computer processing).

[0080] Furthermore, in this embodiment, the device is configured by executing a predetermined program on a computer, but at least a part of the processing contents may be realized by hardware.

[0081] The present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the spirit of the present invention.

Claims

1. Let K be an integer of 3 or more, P be an integer of 2 or more, and N be an integer satisfying 3≦N≦K / P, A secret global model calculation device in a federated learning system including K secret global model calculation devices that learn local models using learning data, Any N secret global model calculation devices among the K secret global model calculation devices can constitute a secret global model calculation system that secretly calculates a global model from the N local models, a selection unit that transmits a query to check the availability of computational resources to K-(p-1)N-1 secret global model computing devices excluding itself among K-(p-1)N (where p is an integer satisfying 1≦p≦P) secret global model computing devices that have not been selected for configuring the secret global model computing system, and selects N-1 secret global model computing devices with the largest availability of computational resources from the K-(p-1)N-1 secret global model computing devices; a system configuration unit that configures a secret global model computation system (hereinafter referred to as the p-th secret global model computation system) using N secret global model computation devices, including the selected N-1 secret global model computation devices and itself; Let σ be a permutation of the set {1, …, P}, and L be an integer greater than or equal to 2. a local model learning unit that learns a local model using the parameters of the global model of the p'th secret global model calculation system (where p' is an integer that satisfies σ(p')=σ(p)-1) as the initial values of the parameters of the global model in one of L learning times when p is an integer that satisfies σ(p)>1, and the parameters of the global model of the pth secret global model calculation system as the initial values of the parameters of the local model in the remaining L-1 learning times out of L learning times, and that learns a local model using the parameters of the global model of the pth secret global model calculation system as the initial values of the parameters of the local model in the second and subsequent learning times when p is an integer that satisfies σ(p)=1; A secret global model calculation device including:

2. Let K be an integer of 3 or more, P be an integer of 2 or more, and N be an integer satisfying 3≦N≦K / P, A method for configuring a secret global model computation system, in which a federated learning system including K secret global model computation devices that learn local models using learning data configures the secret global model computation system, comprising: Any N secret global model calculation devices among the K secret global model calculation devices can constitute a secret global model calculation system that secretly calculates a global model from the N local models, a selection step in which one secret global model computing device (hereinafter referred to as the p-th master secret global model computing device) among the K-(p-1)N (where p is an integer satisfying 1≦p≦P) secret global model computing devices not selected for configuring the secret global model computing system sends a query to the K-(p-1)N-1 secret global model computing devices excluding itself to check the availability of computing resources, and selects N-1 secret global model computing devices with the largest availability of computing resources from the K-(p-1)N-1 secret global model computing devices; a system configuration step in which the p-th master secret global model computation device configures a secret global model computation system (hereinafter referred to as the p-th secret global model computation system) with N secret global model computation devices including the selected N-1 secret global model computation devices and itself; Let σ be a permutation of the set {1, …, P}, and L be an integer greater than or equal to 2. a local model learning step in which, if p is an integer satisfying σ(p)>1, the p-th master secret global model calculation device learns a local model using parameters of the global model of the p'-th secret global model calculation system (where p' is an integer satisfying σ(p')=σ(p)-1) as initial values of parameters of the local model in one learning among L times, and parameters of the global model of the p-th secret global model calculation system as initial values of parameters of the local model in the remaining L-1 learnings among L times, and if p is an integer satisfying σ(p)=1, learns a local model using parameters of the global model of the p-th secret global model calculation system as initial values of parameters of the local model in second and subsequent learnings; A method for configuring a secret global model calculation system, including:

3. A program for causing a computer to function as the secret global model calculation device according to claim 1.

Images