Model learning device, federated learning system, their methods, and programs

Abstract

This model learning device: acquires information identifying an aggregate model, or confidential information for the information identifying the aggregate model, from a secret federated learning device; obtains a worker model by updating the aggregate model through machine learning using local learning data; obtains confidential information for information identifying the worker model; and provides the confidential information to the secret federated learning device. The secret federated learning device: acquires confidential information for information identifying a plurality of worker models from a plurality of model learning devices; performs secret calculation using this confidential information to obtain confidential information for information identifying an aggregate model, which is an aggregation of the plurality of worker models, without obtaining the plurality of worker models; and provides the obtained confidential information to the plurality of model learning devices.

Description

[Technical Field]

[0001] The present invention relates to machine learning techniques, and more particularly to federated learning techniques. [Background technology]

[0002] Federated learning is known, in which machine learning is performed in a distributed state without aggregating learning data (see, for example, Non-Patent Document 1). In federated learning, multiple model learning devices perform machine learning using the learning data they hold to generate worker models (local models) and transmit the generated worker models to the federated learning device. The federated learning device generates an aggregated model (global model) by aggregating the worker models sent from the multiple model learning devices and transmits the generated aggregated model to the multiple model learning devices. The multiple model learning devices that receive the aggregated model update the aggregated model through machine learning using the learning data they hold, generate new worker models, and transmit the generated worker models to the federated learning device. By repeating this process, each model learning device can obtain an aggregated model in which the learning data held in the multiple model learning devices is reflected in the machine learning, without having to externally transmit the learning data it holds. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] C. He, S. Li, J. So, X. Zeng, M. Zhang, et al., “FedML: A Research Library and Benchmark for Federated Machine Learning”, [online], January 27, 2020, arXiv: 2007.13518, [Retrieved February 17, 2022], Internet<https: / / arxiv.org / abs / 2007.13518> Summary of the Invention [Problem to be solved by the invention]

[0004] However, in conventional federated learning, the federated learning device receives plaintext worker models from each model learning device, allowing the federated learning device to understand the trends in the learning data held by each model learning device based on the difference between the aggregate model it sent and the worker model it received.

[0005] The present invention has been made in view of the above points, and has as its object to improve the security of federated learning. [Means for solving the problem]

[0006] The model learning device obtains information identifying the aggregate model or confidential information identifying the aggregate model from the secret associative learning device, updates the aggregate model through machine learning using local learning data stored in the memory unit to obtain a worker model, obtains confidential information identifying the worker model, and provides the confidential information identifying the worker model to the secret associative learning device.

[0007] The secret federated learning device obtains confidential information for information identifying multiple worker models from multiple model learning devices, and by secret calculation using the confidential information for information identifying the multiple worker models, obtains confidential information for information identifying an aggregated model that aggregates the multiple worker models without obtaining the multiple worker models, and provides the information identifying the aggregated model or the confidential information for information identifying the aggregated model to the multiple model learning devices. [Effects of the Invention]

[0008] This can improve the safety of federated learning. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram illustrating the configuration of a federated learning system according to the first and second embodiments. [Figure 2] FIG. 2 is a block diagram illustrating the configuration of the model learning device according to the first and second embodiments. [Figure 3] FIG. 3 is a block diagram illustrating the configuration of the secret association learning device according to the first, third, and fourth embodiments. [Figure 4] FIG. 4 is a block diagram illustrating the configuration of the secret association learning device according to the second, third, and fourth embodiments. [Figure 5] FIG. 5 is a block diagram illustrating the configuration of a federated learning system according to the third and fourth embodiments. [Figure 6] FIG. 6 is a block diagram illustrating the configuration of a model learning device according to the third and fourth embodiments. [Figure 7] FIG. 7 is a block diagram illustrating a hardware configuration of the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. "First embodiment" In this embodiment, among the functions of the federated learning device, the collection of worker models (local models) and the generation of aggregate models are executed in a confidential manner.

[0011] <Configuration> As illustrated in FIG. 1 , the federated learning system 1 of this embodiment includes N model learning devices 11-1, ..., 11-N that perform model learning, M secret federated learning devices 12-1, ..., 12-M that perform federated learning using secure computation, and a control device 13 that controls the federated learning. There are no limitations on the secure computation method. For example, this secure computation method may be a multi-party computation method that performs secure computation using secretly shared shares, or a homomorphic encryption method that performs secure computation using homomorphic encryption. N is an integer of 1 or greater, for example, N is an integer of 2 or greater. M is an integer of 1 or greater, for example, M is an integer of 2 or greater. However, if the secure computation method is the multi-party computation method, M is an integer of 2 or greater. If the secure computation method is the homomorphic encryption method, M is an integer of 1 or greater, for example, M=1.

[0012] As illustrated in FIG. 2, the model learning device 11-n of this embodiment includes a storage unit 111-n, an acquisition unit 112-n, a learning unit 113-n, a concealment unit 114-n, a provision unit 115-n, and a control unit 116-n. The model learning device 11-n executes each process under the control of the control unit 116-n, and input information and information obtained in each process are stored in the storage unit 111-n and are read out and used as needed. Here, n is a positive integer, n=1,...,N. Unless otherwise specified, the configuration and processing related to n are the same for all n=1,...,N. However, the content of the data (information) handled may differ depending on the value of n.

[0013] As illustrated in FIG. 3, the secret associative learning device 12-m of this embodiment includes an acquisition unit 121-m, a secret aggregation processing unit 122-m, a provision unit 123-m, a control unit 126-m, and a storage unit 127-m. The secret associative learning device 12-m executes each process under the control of the control unit 126-m, and input information and information obtained in each process are stored in the storage unit 127-m and are read out and used as needed. Here, m is a positive integer, m=1,...,M. Unless otherwise specified, the configuration and processing related to m are the same for all m=1,...,M. However, the content of the data (information) handled may differ depending on the value of m.

[0014] <Pre-processing> The storage unit 111-n of the model learning device 11-n stores local learning data Dn of each model learning device 11-n. The local learning data Dn is learning data for machine learning, and may be learning data for supervised learning or learning data for unsupervised learning. The local learning data Dn may be updated. If the secure computation method used is homomorphic encryption, an encryption key and a decryption key are stored in the storage unit 111-n of the model learning device 11-n.

[0015] <Learning process> The learning process of this embodiment will be exemplified below. The learning unit 113-n of each model learning device 11-n (FIG. 2) reads the local learning data Dn stored in the memory unit 111-n, updates the latest aggregate model by machine learning using the local learning data Dn to obtain a worker model, and outputs information WM-n (e.g., a model parameter group) that identifies the worker model. If the model learning device 11-n has not yet obtained an aggregate model, the initially set machine learning model is the "latest aggregate model." The initially set machine learning model may be provided by the control device 13. The initially set model is, for example, a machine learning model to which an initial model parameter group is set. If the model learning device 11-n has obtained information GM that identifies an aggregate model as described below, the latest aggregate model among the aggregate models identified by the information GM is the "latest aggregate model." In the latter case, the learning unit 113-n identifies the latest aggregate model based on the information GM read from the memory unit 111-n. The aggregate model and worker model are known machine learning models. There are no limitations on the aggregation model and worker model, and they may be, for example, models based on deep learning, hidden Markov models, support vector machines, or linear prediction. However, all aggregation models and worker models used in the federated learning system 1 are models based on the same method. Information WM-n identifying the worker model is sent to the concealment unit 114-n (step S113-n).

[0016] The concealment unit 114-n receives input of information WM-n that identifies a worker model. The concealment unit 114-n conceals the information WM-n that identifies a worker model using a method that allows the above-mentioned secure computation, and obtains and outputs the concealed information [WM-n] of the information WM-n that identifies the worker model. For example, if the above-mentioned secure computation method is a multi-party computation method, the concealment unit 114-n secretly distributes the information WM-n into M shares, [WM-n]1, ..., [WM-n] MFor example, if the above-mentioned secure computation method is a homomorphic encryption method, the concealment unit 114-n uses the encryption key read from the storage unit 111-n to encrypt the information WM-n in accordance with the homomorphic encryption method, and outputs M (for example, 1) ciphertexts [WM-n]1, ..., [WM-n] M and obtain the ciphertext [WM-n]1,...,[WM-n] M is output as secret information [WM-n]. Secret information [WM-n]={[WM-n]1,...,[WM-n] M} is sent to the providing unit 115-n (step S114-n).

[0017] The provider 115-n stores confidential information [WM-n]={[WM-n]1,...,[WM-n] of information WM-n that identifies a worker model. M The providing unit 115-n receives the confidential information [WM-n] of the information WM-n that identifies the worker model. m to secret associative learning device 12-m (FIG. 3) (where m=1, . . . , M). Furthermore, providing unit 115-n transmits (provides) secret information [WM-n] from model learning device 11-n to secret associative learning device 12-m. m (The model learning device 11-n has completed learning of the worker model, and the confidential information [WM-n] of the worker model WM-n has been transmitted.) m The control device 13 then sends synchronization information indicating that the secret association learning device 12-m has received the synchronization information (step S115-n).

[0018] The acquisition unit 121-m of the secret federated learning device 12-m (FIG. 3) acquires the worker model-specific information WM-n sent from the model learning device 11-n. m Receive the confidential information [WM-n] m That is, the acquisition unit 121-m stores the confidential information [WM-n] of the information WM-n that identifies the plurality of worker models from the plurality of model learning devices 11-n. m is obtained and stored in the storage unit 127-m (step S121-m).

[0019] Based on the synchronization information, the control device 13 controls all the model learning devices 11-1, ..., 11-N to transmit the secret information [WM-n]1, ..., [WM-n] to all the secret associative learning devices 12-1, ..., 12-M. M Here, all the model learning devices 11-n (where n=1, ..., N) transmit the secret information [WM-n]1, ..., [WM-n] to all the secret associative learning devices 12-1, ..., 12-M (step S131). M If it is determined that the secret information [WM-n]1, ..., [WM-n] is not transmitted and if it is determined that the predetermined time has not elapsed since the reference time point (timeout has not occurred), the control device 13 performs the determination in step S131 at regular intervals. Meanwhile, all the model learning devices 11-n (where n=1, ..., N) transmit the secret information [WM-n]1, ..., [WM-n] to all the secret associative learning devices 12-1, ..., 12-M. M has been transmitted, or it is determined that a predetermined time has elapsed since the reference time point (timeout), the control device 13 sends a command to the secret association learning devices 12-1, ..., 12-M to instruct them to start a secret aggregation process. Note that the reference time point for the timeout may be any time point; for example, the reference time point may be the start or end of the previous secret aggregation process, or if the secret aggregation process has not yet been executed, the reference time point may be the start of the learning process (step S132).

[0020] The command to start the secret aggregation process is received by the acquisition unit 121-m of the secret associative learning device 12-m (where m = 1, ..., M) (Fig. 3) and input to the control unit 126-m. The control unit 126-m, which has received the command to start the secret aggregation process, instructs the secret aggregation processing unit 122-m to start the secret aggregation process. In response to this, the secret aggregation processing unit 122-m reads multiple pieces of secret information [WM-n] (where n ∈ {1, ..., N}) (secret information of information identifying multiple worker models) from the storage unit 127-m, and performs secure computation using these pieces of secret information to obtain, without obtaining the multiple worker models, secret information [GM] of information GM identifying an aggregate model obtained by aggregating the multiple worker models. mFor example, the information WM-n that identifies the worker model is obtained and output as a set of model parameters {p1(n), ..., p K (n)}, then {n1,…,n max} ⊆ {1,…,N}, the set of model parameters {p1(n1),…,p K (n1)},…,{p1(n max ),…,p K (n max )}, the aggregated set of model parameters {p1,…,p K} is the information GM that specifies the aggregation model. For example, p k HAp k (n1),…,p k (n max ) is a function value such as a weighted linear combination value or an average value of the worker model parameters. Here, k is an index k=1,...,K that identifies the model parameters, and K is a positive integer. The secret aggregation processing unit 122-m can obtain the secret information [GM] of the information GM that identifies the aggregation model without recovering the information WM-n that identifies the worker model or the information GM that identifies the aggregation model by secure computation. m The information that identifies the aggregation model is the confidential information of the GM [GM]. m is sent to the providing unit 123-m (step S122-m).

[0021] Confidential information provided by 123-m [GM] m The providing unit 123-m receives the secret information [GM] via the control device 13. m to a plurality of model learning devices 11-n (where n∈{1, ..., N}). For example, the providing unit 123-m transmits (provides) the secret information [GM] via the control device 13. m is transmitted (provided) to all model learning devices 11-1, . . . , 11-N (step S123-m).

[0022] Confidential Information [GM] m (where m∈{1, ..., M}) is sent to the model learning device 11-n (FIG. 2), and the acquisition unit 112-n of the model learning device 11-n (FIG. 2) acquires the secret information [GM] mThe acquisition unit 112-n receives the secret information [GM] (the secret information of the information GM that identifies the aggregate model provided from the secret associative learning device 12-m). m For example, when the secure computation method is a multi-party computation method, the acquisition unit 112-n acquires a plurality of mutually different secret information [GM] required for the restoration. m(1) ,…,[GM] m(max) (where {m(1),...,m(max)}⊆{1,...,M}) to restore the information GM. When the secure computation method is a homomorphic encryption method, the acquisition unit 112-n restores the secret information [GM] using the decryption key read from the storage unit 111-n. m The information GM that identifies the aggregation model is stored in the storage unit 111-n (step S112-n).

[0023] The control unit 116-n determines whether the termination condition of the federated learning is satisfied (step S116-n). If the termination condition is not satisfied, the process returns to step S113-n. In this case, the processes from step S113-n to step S116-n described above (i.e., S113-n, S114-n, S115-n, S121-m, S131, S132, S122-m, S123-m, S112-n, S116-n) are executed again. On the other hand, if the termination condition is satisfied, the process is terminated. Note that any termination condition may be used. For example, the control unit 13 may send a command to the model learning device 11-n to terminate the federated learning when the number of updates, update amount, update time, etc. of the aggregation model reach specified values, and the reception of this command may be the termination condition. Alternatively, the control unit 116-n may determine whether the number of updates, the amount of updates, the update time, etc. of the aggregate model have reached a specified value, and the condition for termination may be that the specified value has been reached.

[0024] <Features of this embodiment> In this embodiment, multiple model learning devices provide secret information for identifying a worker model to a secret federated learning device, and the secret federated learning device uses the secret information to obtain secret information for identifying an aggregate model that aggregates the multiple worker models without obtaining the multiple worker models. In this case, the secret federated learning device cannot obtain the worker models themselves, and therefore cannot know the trends in the learning data held by each model learning device based on the differences between the worker models and the aggregate model. This improves the security of federated learning.

[0025] [Modification 1 of the First Embodiment] Based on the synchronization information, the control device 13 controls all the model learning devices 11-1, ..., 11-N to transmit the secret information [WM-n]1, ..., [WM-n] to all the secret associative learning devices 12-1, ..., 12-M. M In step S131, the control device 13 determines whether a predetermined number of model learning devices have transmitted secret information to a predetermined number of secret associative learning devices. However, this does not limit the present invention. For example, instead of this, the control device 13 may determine whether a predetermined number of model learning devices have transmitted secret information to a predetermined number of secret associative learning devices. If it is determined that a predetermined number of model learning devices have not transmitted secret information to a predetermined number of secret associative learning devices and if it is determined that a predetermined time has not elapsed since the reference time point, the control device 13 may perform this determination at certain intervals. If it is determined that a predetermined number of model learning devices have transmitted secret information to a predetermined number of secret associative learning devices or if it is determined that a predetermined time has elapsed since the reference time point, the control device 13 may send a command to the secret associative learning devices 12-1, ..., 12-M to instruct them to start a secret aggregation process. Alternatively, the control device 13 may simply send a command to the secret associative learning devices 12-1, ..., 12-M to instruct them to start a secret aggregation process when a predetermined time has elapsed since the reference time point.

[0026] [Second embodiment] This embodiment is a modification of the first embodiment or modification 1 of the first embodiment, in which a specific secret associative learning device 12-1 receives secret information [GM]1, ..., [GM]M The information GM that identifies the aggregation model is restored from the data, and the information GM that identifies the aggregation model is provided to each model learning device 11-n. Hereinafter, the differences from the matters that have been explained so far will be mainly explained, and the same reference numbers will be used to simplify the explanation of matters that have already been explained.

[0027] <Configuration> As illustrated in FIG. 1, the federated learning system 2 of this embodiment includes N model learning devices 21-1, ..., 21-N that perform model learning, M secret federated learning devices 22-1, 12-2, ..., 12-M that perform federated learning using secure computation, and a control device 13 that controls the federated learning.

[0028] 2, the model learning device 21-n of this embodiment includes a storage unit 111-n, an acquisition unit 212-n, a learning unit 113-n, a concealment unit 114-n, a provision unit 115-n, and a control unit 116-n. The model learning device 21-n executes each process under the control of the control unit 116-n, and input information and information obtained in each process are stored in the storage unit 111-n and are read out and used as needed.

[0029] 4, secret associative learning device 22-1 of this embodiment has acquisition unit 121-1, secret aggregation processing unit 122-1, provision unit 223-1, control unit 126-1, and storage unit 127-1. Secret associative learning device 22-1 executes each process under the control of control unit 126-1, and input information and information obtained in each process are stored in storage unit 127-1 and are read out and used as needed. The configuration of secret associative learning devices 12-2, ..., 12-M is the same as that of the first embodiment.

[0030] <Pre-processing> The pre-processing in this embodiment is the same as that in the first embodiment, except that when the secure computation method used is the homomorphic encryption method, the decryption key is stored in the memory unit 127-1 of the secure association learning device 22-1, rather than in the memory unit 111-n of the model learning device 11-n.

[0031] <Learning process> The learning process of this embodiment will be exemplified below. First, instead of model learning device 11-n, model learning device 21-n executes the processing of steps S113-n, S114-n, and S115-n described in the first embodiment, instead of secret associative learning device 12-1, secret associative learning device 22-1 executes the processing of step S121-1, secret associative learning device 12-m' (where m'∈{2, ..., M}) executes the processing of step S121-m', control device 13 executes the processing of steps S131 and S132, instead of secret associative learning device 12-1, secret associative learning device 22-1 executes the processing of step S122-1, and secret associative learning device 12-m' (where m'∈{2, ..., M}) executes the processing of step S122-m'. However, in step S122-1, secret information [GM]1 of information GM that identifies the aggregation model is sent to provider 223-1 of secret associative learning device 22-1 (FIG. 4) instead of provider 123-1 of secret associative learning device 12-1. Furthermore, if M≧2, provider 123-m′ (where m′∈{2,...,M}) of secret associative learning device 12-m′ further sends secret information [GM] m’ The secret information [GM] is transmitted to provider 223-1 of secret associative learning device 22-1 (FIG. 4). m (where m∈{1, ,M}) is input. The providing unit 223-1 receives the input confidential information [GM] m For example, when the secure computation method is a multi-party computation method, the providing unit 223-1 obtains information GM that identifies the aggregation model by restoring the secret information [GM]. m(1) ,…,[GM] m(max) (where {m(1),...,m(max)}⊆{1,...,M}) to restore the information GM. When the secure computation method is a homomorphic encryption method, the providing unit 223-1 restores the secret information [GM] using the decryption key read from the storage unit 127-1. mand obtains information GM. Providing unit 223-1 transmits (provides) information GM to a plurality of model learning devices 21-1, ..., 21-N (where n∈{1, ..., N}) via control device 13. For example, providing unit 223-1 transmits (provides) information GM to all model learning devices 21-1, ..., 21-N via control device 13 (step S223-1).

[0032] The acquisition unit 212-n of the model learning device 21-n (FIG. 2) to which the information GM has been sent receives the information GM (information identifying the aggregation model). That is, the acquisition unit 212-n obtains the information GM identifying the aggregation model from the secret associative learning device 12-1. The information GM identifying the aggregation model is stored in the storage unit 111-n (step S212-n). The control unit 116-n determines whether or not the termination condition of the associative learning is satisfied (step S116-n). If the termination condition is not satisfied, the process returns to step S113-n. In this case, the processes from steps S113-n to S116-n (i.e., S113-n, S114-n, S115-n, S121-m, S131, S132, S122-m, S223-1, S212-n, S116-n) described above are executed again. On the other hand, if the termination condition is satisfied, the process ends. Other matters are as described in the first embodiment. In the second embodiment, the same modifications as the first modification of the first embodiment may be made.

[0033] <Features of this embodiment> In this embodiment, multiple model learning devices also provide secret information for identifying a worker model to the secret federated learning device, and the secret federated learning device uses the secret information to obtain secret information for identifying an aggregate model that aggregates the multiple worker models without obtaining the multiple worker models. In this case, the secret federated learning device cannot obtain the worker models themselves, and therefore cannot know the trends in the learning data held by each model learning device based on the differences between the worker models and the aggregate model. This improves the security of federated learning.

[0034] [Third embodiment] This embodiment is a modification of the first embodiment. In the third embodiment, the model learning device determines whether it is necessary to update the acquired aggregate model and obtain a new worker model. If it determines that this is necessary, the model learning device updates the aggregate model and obtains a new worker model. However, if it determines that this is not necessary, the model learning device does not update the aggregate model and obtain a new worker model, and instead obtains confidential information identifying a new aggregate model from the secret federated learning device after a waiting time has elapsed. Furthermore, in the third embodiment, the secret federated learning device determines whether it has obtained confidential information identifying the worker model from a specific model learning device. If it determines that it has obtained confidential information identifying the worker model from the specific model learning device, the secret federated learning device obtains confidential information identifying the aggregate model obtained by aggregating the worker model by secure computation using the confidential information identifying the worker model. This improves the security of federated learning without using the control device 13.

[0035] <Configuration> As illustrated in FIG. 5, the federated learning system 3 of this embodiment includes N model learning devices 31-1, ..., 31-N that perform model learning, and M secret federated learning devices 32-1, ..., 32-M that perform federated learning using secure computation.

[0036] 6, the model learning device 31-n of this embodiment includes a storage unit 111-n, an acquisition unit 312-n, a determination unit 317-n, a learning unit 113-n, a concealment unit 114-n, a provision unit 115-n, and a control unit 116-n. The model learning device 31-n executes each process under the control of the control unit 116-n, and input information and information obtained in each process are stored in the storage unit 111-n and are read out and used as needed.

[0037] 3, secret associative learning device 32-m of this embodiment includes acquisition unit 121-m, determination unit 328-m, secret aggregation processing unit 322-m, provision unit 123-m, control unit 126-m, and storage unit 127-m. Secret associative learning device 32-m executes each process under the control of control unit 126-m, and input information and information obtained in each process are stored in storage unit 127-m and are read out and used as needed.

[0038] <Pre-processing> The pre-processing in this embodiment is the same as that in the first embodiment.

[0039] <Learning process> The learning process of this embodiment will be exemplified below. First, model learning device 31-n (FIG. 6) executes the processes of steps S113-n, S114-n, and S115-n described in the first embodiment in place of model learning device 11-n, and secret associative learning device 32-m (FIG. 3) executes the process of step S121-m in place of secret associative learning device 12-m. However, since control device 13 is not provided in this embodiment, providing unit 115-n of model learning device 31-n does not transmit the above-mentioned synchronization information to control device 13 in step S115-n.

[0040] In addition, the determination unit 328-m of the secret associative learning device 32-m determines, at a predetermined opportunity, the secret information [WM-n] stored in the memory unit 127-m. m For example, the determination unit 328-m may make this determination periodically, or may refer to the confidential information [WM-n] m The determination may be made by being triggered by the fact that the worker model has been registered in the storage unit 127-m. max (where {n1,…,n max}={1,…,N}), secret information for identifying the new worker model [WM-n1] m ,…,[WM-n max ] mThat is, the determination unit 328-m obtains the predetermined model learning device 31-n1, ..., 31-n max From the confidential information identifying the new worker model [WM-n1] m ,…,[WM-n max ] m The predetermined model learning devices 31-n1, ..., 31-n max is the number of all model learning devices 31-1,...,31-N (i.e., {n1,...,n max}={1,...,N}), or a part of the model learning devices 31-n1,...,31-n max (i.e., {n1,…,n max} ⊂ {1,…,N}). Also, the secret information for identifying the new worker model [WM-n1] m ,…,[WM-n max ] m is secret information that has not yet been used in secret aggregation processing [WM-n1] m ,…,[WM-n max ] m or secret information obtained after the previous secret aggregation process [WM-n1] m ,…,[WM-n max ] m However, confidential information [WM-n] m is a share in a secret sharing scheme or a ciphertext in a homomorphic encryption scheme, so the secret information [WM-n] m From the confidential information [WM-n] m In such a case, the determination unit 328-m may be unable to identify the model learning device 31-n that provided the secret information [WM-n] stored in the storage unit 127-m. m For example, the determination unit 328-m may determine whether the registration of the worker model is complete based on the total amount of data in the storage unit 127-m. m The total amount of data is determined by the predetermined model learning devices 31-n1, ..., 31-n max Confidential information provided by [WM-n1] m ,…,[WM-n max ] mIf the total data amount of the worker model matches the total data amount of the worker model, it may be determined that the registration of the worker model is complete, and if not, it may be determined that the registration of the worker model is not complete. m The total number of worker models corresponding to the given model learning devices 31-n1, ..., 31-n max The total number of worker models is n max If the worker model registration is completed, it is determined that the worker model registration is complete. If the worker model registration is not completed, it is determined that the worker model registration is not complete. For example, if the information WM-n that identifies the worker model is a model parameter group and the number of model parameters included in one worker model is N, MP and the secret information [WM-n] stored in the memory unit 127-m m The total amount of data is N records R If so, confidential information [WM-n] m The total number of worker models provided is N R / N MP In this case, the determination unit 328-m determines whether n max =N R / N MP If the result is "YES", it may be determined that the registration of the worker model is complete, and if not, it may be determined that the registration of the worker model is not complete. Here, if it is determined that the registration of the worker model is not complete and a predetermined time has not elapsed since the reference time point (timeout has not occurred), the determination unit 328-m determines again at a predetermined opportunity whether the registration of this worker model is complete. For example, the determination unit 328-m may make the determination again after a predetermined waiting time has elapsed, or may make the determination again after any of the confidential information [WM-n] has elapsed. m The determination may be made again when the worker model registration is determined to be completed or when a predetermined time has elapsed since the reference time point (timeout has occurred), the determination unit 328-m sends a command to the control unit 126-m to instruct the control unit 126-m to start the secret aggregation process. An example of the reference time point for timeout is as described in the first embodiment (step S328-m).

[0041] The control unit 126-m, which has received the command to start the secret aggregation process, instructs the secret aggregation processing unit 322-m to start the secret aggregation process. The secret aggregation processing unit 322-m, which has received this command, reads multiple pieces of secret information [WM-n] (where n∈{1,...,N}) (secret information of information that identifies multiple worker models) from the storage unit 127-m, and performs secure computation using these pieces of secret information to calculate the secret information [GM] of information GM that identifies an aggregate model that aggregates the multiple worker models, without obtaining the multiple worker models. m That is, the secret aggregation processing unit 322-m obtains and outputs the secrets of the predetermined model learning devices 31-n1, . . . , 31-n max Confidential information for identifying the worker model from [WM-n1] m ,…,[WM-n max ] m If it is determined that the worker model has been obtained, confidential information for identifying the worker model [WM-n1] m ,…,[WM-n max ] m Confidential information of GM [GM], which identifies the aggregate model that aggregates the multiple worker models by secret computation using m The information that identifies the aggregation model is the confidential information of the GM [GM]. m is sent to the providing unit 123-m (step S322-m).

[0042] The acquisition unit 312-n of the model learning device 31-n (FIG. 6) accesses the provider 123-m of the secret associative learning device 32-m (where m∈{1, ..., M}) (FIG. 3) at a predetermined opportunity, and acquires the confidential information [GM] of the information GM that identifies the aggregate model from the provider 123-m. m The acquisition unit 312-n acquires the acquired confidential information [GM]. m The information GM specifying the aggregation model is obtained by restoring the data GM. The information GM specifying the aggregation model is stored in the storage unit 111-n (step S312-n).

[0043] The determination unit 317-n determines whether it is necessary to update the aggregation model corresponding to the information GM stored in the storage unit 111-n to obtain a new worker model. In other words, the determination unit 317-n determines whether it is necessary to update the latest aggregation model identified by the information GM to obtain a worker model through machine learning using the local learning data Dn. For example, if the aggregation model is identical to or similar to the "latest aggregation model" already used to generate the worker model (step S113-n), the determination unit 317-n determines that it is not necessary to update the aggregation model to obtain a new worker model; otherwise, it determines that it is necessary to update the aggregation model to obtain a new worker model. Note that two aggregation models being similar may mean, for example, that the distance between their model parameters is equal to or less than a predetermined value, or that the difference in output distributions between the two aggregation models for a predetermined input group is equal to or less than a predetermined value (step S317a-n).

[0044] Here, if it is determined that there is no need to update the aggregation model and obtain a new worker model, the control unit 116-n determines whether or not the termination condition of the federated learning is met. If the termination condition is met, the process is terminated. On the other hand, if the termination condition is not met, the learning unit 113-n does not update the aggregation model and obtain a new worker model, and after the waiting time has elapsed, the acquisition unit 312-n acquires the secret information [GM] of the information identifying the new aggregation model from the secret federated learning device 32-m (where m∈{1,...,M}) (FIG. 3). m That is, without the learning unit 113-n obtaining a new worker model, the acquisition unit 312-n accesses the provision unit 123-m after the waiting time has elapsed, and obtains the confidential information [GM] of the information GM that identifies the aggregate model from the provision unit 123-m. m The acquisition unit 312-n acquires the acquired confidential information [GM]. m is restored to obtain information GM that identifies the aggregation model, which is stored in the storage unit 111-n, and the process returns to step S317a-n (step S317b-n).

[0045] On the other hand, if it is determined that it is necessary to update the aggregation model and obtain a new worker model, the process returns to step S113-n (step S317c-n). That is, the learning unit 113-n reads the local learning data Dn and the latest information GM stored in the storage unit 111-n, updates the latest aggregation model identified by the information GM through machine learning using the local learning data Dn, obtains a worker model, and outputs information WM-n that identifies the worker model (step S113-n). Thereafter, the processes from step S114-n onwards described in this embodiment are executed again.

[0046] <Features of this embodiment> In this embodiment, multiple model learning devices also provide secret information for identifying a worker model to the secret federated learning device, and the secret federated learning device uses the secret information to obtain secret information for identifying an aggregate model that aggregates the multiple worker models without obtaining the multiple worker models. In this case, the secret federated learning device cannot obtain the worker models themselves, and therefore cannot know the trends in the learning data held by each model learning device based on the differences between the worker models and the aggregate model. This improves the security of federated learning.

[0047] Furthermore, the model learning device of this embodiment determines whether it is necessary to update the aggregate model and obtain a new worker model. If it is determined that it is not necessary to update the aggregate model and obtain a new worker model, the model learning device acquires confidential information identifying the new aggregate model after the waiting time has elapsed, without updating the aggregate model and obtaining a new worker model. On the other hand, if it is determined that it is necessary to update the aggregate model and obtain a new worker model, the model learning device updates the aggregate model through machine learning using local learning data and obtains the worker model. The secret federated learning device also determines whether it has obtained confidential information identifying the worker model from a specific model learning device. If it is determined that confidential information identifying the worker model has been obtained from the specific model learning device, the secret federated learning device obtains confidential information identifying the aggregate model obtained by aggregating the worker model through secure computation using the confidential information identifying the worker model. These features improve the security of federated learning without using the control device 13.

[0048] Furthermore, in this embodiment, the model learning device does not notify the outside that it has completed learning of the worker model, which prevents information about the performance and processing time of each model learning device from leaking out.

[0049] [Modification 1 of the third embodiment] In the third embodiment, when the determination unit 328-m of the secret federated learning device 32-m (FIG. 3) determines that the registration of the worker model has been completed or that a predetermined time has elapsed since the reference point in time, it sends a command to the control unit 126-m to instruct the start of the secret aggregation process (step S328-m). However, the determination unit 328-m may send a command to the control unit 126-m to instruct the start of the secret aggregation process when it determines that the registration of the worker model has been completed without determining whether the predetermined time has elapsed since the reference point in time.

[0050] [Modification 2 of the third embodiment] In the third embodiment, the acquisition unit 312-n of the model learning device 31-n (FIG. 6) acquires, at a predetermined opportunity, confidential information [GM] of information GM that identifies an aggregate model from the provision unit 123-m of the secret associative learning device 32-m (FIG. 3). m Obtained and obtained confidential information [GM] m

[0073] However, acquisition unit 312-n of model learning device 31-n may acquire information GM that identifies the aggregate model from provider 223-1 of a specific secret associative learning device 32-1 (FIG. 4) at a predetermined opportunity. In this case, as described in the second embodiment, in step S322-m, confidential information [GM]1 of information GM that identifies the aggregate model is sent to provider 223-1 of secret associative learning device 32-1 (FIG. 4). Furthermore, if M≧2, provider 323-m' (where m'∈{2,...,M}) of secret associative learning device 32-m' further acquires confidential information [GM] m’ The secret information [GM] is transmitted to the provider 223-1 of the secret associative learning device 32-1 (FIG. 4). m (where m∈{1, ,M}) is input. The providing unit 223-1 receives the input confidential information [GM] m The acquisition unit 112-n of the model learning device 31-n acquires the information GM that identifies the aggregation model from the provision unit 223-1 of the secret associative learning device 32-1 (FIG. 4) at a predetermined opportunity. The rest is the same as in the third embodiment.

[0051] [Fourth embodiment] This embodiment is a modification of the third embodiment. In the fourth embodiment, the model learning device further provides plaintext synchronization information indicating that it has provided the secret information identifying the worker model to the secret federated learning device. The secret federated learning device acquires the plaintext synchronization information indicating that the model learning device has provided the secret information identifying the worker model to the secret federated learning device, and uses the synchronization information to determine whether it has obtained the secret information identifying the worker model from a specific model learning device. This makes communication more efficient and improves performance.

[0052] <Configuration> As illustrated in FIG. 5, the federated learning system 4 of this embodiment includes N model learning devices 41-1, ..., 41-N that perform model learning, and M secret federated learning devices 42-1, ..., 42-M that perform federated learning using secure computation.

[0053] 6, the model learning device 41-n of this embodiment includes a storage unit 111-n, an acquisition unit 412-n, a determination unit 317-n, a learning unit 113-n, a concealment unit 114-n, a provision unit 415-n, and a control unit 116-n. The model learning device 41-n executes each process under the control of the control unit 116-n, and input information and information obtained in each process are stored in the storage unit 111-n and are read out and used as needed.

[0054] 3, the secret associative learning device 42-m of this embodiment includes an acquisition unit 421-m, a determination unit 428-m, a secret aggregation processing unit 322-m, a provision unit 423-m, a control unit 126-m, and a storage unit 127-m. The secret associative learning device 42-m executes each process under the control of the control unit 126-m, and input information and information obtained in each process are stored in the storage unit 127-m and are read out and used as needed.

[0055] <Pre-processing> The pre-processing in this embodiment is the same as that in the first embodiment.

[0056] <Learning process> The learning process of this embodiment will be exemplified below. First, model learning device 41-n (FIG. 6) instead of model learning device 11-n executes the processes of steps S113-n and S114-n described in the first embodiment.

[0057] The secret information obtained in the process of step S114-n is [WM-n]={[WM-n]1, ..., [WM-n] M} (confidential information of information WM-n that identifies the worker model) is input to the providing unit 415-n of the model learning device 41-n (FIG. 6). The providing unit 415-n provides the confidential information [WM-n] of the information WM-n that identifies the worker model. m to the secret associative learning device 42-m (FIG. 3) (where m=1, . . . , M). Furthermore, the providing unit 415-n transmits (provides) the worker model information [WM-n] of the worker model WM-n, which is information that the model learning device 41-n transmits to the secret associative learning device 42-m. m The model learning device 41-n transmits (provides) synchronization information syn-n to the secret federated learning device 42-m (where m = 1, ..., M) indicating that the model learning device 41-n has completed learning the worker model. The synchronization information syn-n is in plain text (step S415-n).

[0058] The acquisition unit 121-m of the secret associative learning device 42-m (FIG. 3) receives the secret information [WM-n] sent from the model learning device 41-n. m and synchronization information syn-n, and the secret information [WM-n] m and synchronization information syn-n are stored in the storage unit 127-m. That is, the acquisition unit 421-m acquires confidential information [WM-n] of information WM-n that identifies a plurality of worker models from a plurality of model learning devices 41-n. m , and the model learning device 41-n transmits to the secret federated learning device 42-m information identifying the worker model WM-n, secret information [WM-n] of the worker model WM-n. m The synchronization information syn-n indicating that the synchronization information syn-n has been transmitted is acquired, and stored in the storage unit 127-m (step S421-m).

[0059] In addition, the determination unit 428-m of the secret federated learning device 42-m determines, at a predetermined opportunity, whether or not the registration of the worker model has been completed, using the synchronization information syn-n stored in the storage unit 127-m. For example, the determination unit 428-m may perform this determination periodically, or may perform this determination by periodically updating each secret information [WM-n]. mAs explained in the third embodiment, the determination may be triggered by the fact that the worker model registration is completed when the predetermined model learning devices 41-n1, ..., 41-n max (where {n1,…,n max} ⊆ {1,…,N}), secret information to identify the new worker model [WM-n1] m ,…,[WM-n max ] m By using the synchronization information syn-n, the determination unit 428-m determines which model learning device 41-n has sent the secret information [WM-n] to the secret associative learning device 42-m. m has been provided. Therefore, by using this synchronization information syn-n, the determination unit 428-m can accurately determine whether or not the registration of the worker model has been completed. Here, if it is determined that the registration of the worker model has not been completed and that a predetermined time has not elapsed since the reference time point (timeout has not occurred), the determination unit 428-m determines again, at a predetermined opportunity, whether or not the registration of this worker model has been completed. On the other hand, if it is determined that the registration of the worker model has been completed or that a predetermined time has elapsed since the reference time point (timeout has occurred), the determination unit 428-m sends a command to the control unit 126-m to instruct the start of secret aggregation processing. Note that an example of the reference time point for timeout is as described in the first embodiment (step S421-m).

[0060] The control unit 126-m, which has received the command to start the secret aggregation process, instructs the secret aggregation processing unit 322-m to start the secret aggregation process. The secret aggregation processing unit 322-m, which has received this command, reads multiple pieces of secret information [WM-n] (where n∈{1,...,N}) (secret information of information that identifies multiple worker models) from the storage unit 127-m, and performs secure computation using these pieces of secret information to calculate the secret information [GM] of information GM that identifies an aggregate model that aggregates the multiple worker models, without obtaining the multiple worker models. m The secret information of the GM that identifies the aggregation model is obtained and output (secret aggregation process). mis sent to providing unit 423-m (step S322-m).

[0061] Confidential information provided by 423-m [GM] m The providing unit 423-m receives the secret information [WM-n] received by the acquiring unit 121-m from the model learning device 41-n. m and confidential information [GM] of information GM that identifies the aggregation model as a return value for the synchronization information syn-n (step S421-m). m is transmitted (provided) to the model learning device 41-n (step S423-m).

[0062] The acquisition unit 412-n of the model learning device 41-n (FIG. 6) receives the secret information [GM] provided by the secret associative learning device 42-m (where m∈{1, ..., M}) as a return value. m The acquisition unit 412-n receives the acquired secret information [GM] m The information GM specifying the aggregation model is obtained by restoring the data GM. The information GM specifying the aggregation model is stored in the storage unit 111-n (step S412-n).

[0063] The determination unit 317-n determines whether it is necessary to update the aggregation model corresponding to the information GM stored in the storage unit 111-n to obtain a new worker model. In other words, the determination unit 317-n determines whether it is necessary to update the latest aggregation model identified by the information GM to obtain a worker model by machine learning using the local learning data Dn (step S317a-n).

[0064] Here, if it is determined that there is no need to update the aggregate model and obtain a new worker model, the control unit 116-n of the model learning device 41-n (FIG. 6) determines whether or not the termination condition of the federated learning is met. If the termination condition is met, the process is terminated. On the other hand, if the termination condition is not met, the learning unit 113-n does not update the aggregate model and obtain a new worker model, and after the waiting time has elapsed, the acquisition unit 412-n acquires the secret information [GM] of the information identifying the new aggregate model from the secret federated learning device 42-m (where m∈{1,...,M}) (FIG. 3). m The acquisition unit 412-n acquires the acquired confidential information [GM]. m is restored to obtain information GM that identifies the aggregation model, which is stored in the storage unit 111-n, and the process returns to step S317a-n (step S417b-n).

[0065] On the other hand, if it is determined that it is necessary to update the aggregate model and obtain a new worker model, the process returns to step S113-n (step S317c-n). After that, the process from step S114-n onwards described in this embodiment is executed again.

[0066] <Features of this embodiment> In this embodiment, multiple model learning devices also provide secret information for identifying a worker model to the secret federated learning device, and the secret federated learning device uses the secret information to obtain secret information for identifying an aggregate model that aggregates the multiple worker models without obtaining the multiple worker models. In this case, the secret federated learning device cannot obtain the worker models themselves, and therefore cannot know the trends in the learning data held by each model learning device based on the differences between the worker models and the aggregate model. This improves the security of federated learning.

[0067] Furthermore, the model learning device of this embodiment determines whether it is necessary to update the aggregate model and obtain a new worker model. If it is determined that it is not necessary to update the aggregate model and obtain a new worker model, the model learning device acquires confidential information identifying the new aggregate model after the waiting time has elapsed, without updating the aggregate model and obtaining a new worker model. On the other hand, if it is determined that it is necessary to update the aggregate model and obtain a new worker model, the model learning device updates the aggregate model through machine learning using local learning data and obtains the worker model. The secret federated learning device also determines whether it has obtained confidential information identifying the worker model from a specific model learning device. If it is determined that confidential information identifying the worker model has been obtained from the specific model learning device, the secret federated learning device obtains confidential information identifying the aggregate model obtained by aggregating the worker model through secure computation using the confidential information identifying the worker model. These features improve the security of federated learning without using the control device 13.

[0068] Furthermore, the model learning device of this embodiment further provides plaintext synchronization information to the secret federated learning device, indicating that the model learning device has provided the secret information identifying the worker model to the secret federated learning device.The secret federated learning device uses the synchronization information to determine whether it has obtained the secret information identifying the worker model from a specific model learning device.This makes it possible to accurately determine whether the registration of the worker model has been completed.As a result, communication can be made more efficient and performance can be improved.

[0069] [Modification 1 of the Fourth Embodiment] In the fourth embodiment, when the determination unit 428-m of the secret federated learning device 42-m (FIG. 3) determines that the registration of the worker model has been completed or that a predetermined time has elapsed since the reference point in time, it sends a command to the control unit 126-m to instruct the start of the secret aggregation process (step S428-m). However, the determination unit 428-m may send a command to the control unit 126-m to instruct the start of the secret aggregation process when it determines that the registration of the worker model has been completed without determining whether the predetermined time has elapsed since the reference point in time.

[0070] [Modification 2 of the Fourth Embodiment] In the fourth embodiment, the acquisition unit 412-n of the model learning device 41-n (FIG. 6) receives, as a return value, confidential information [GM] of the information GM that identifies the aggregate model from the provision unit 423-m of the secret associative learning device 42-m (FIG. 3). m Obtained and obtained confidential information [GM] m

[0073] However, the acquisition unit 112-n of the model learning device 31-n may acquire the information GM that identifies the aggregation model from the provider 223-1 of a specific secret associative learning device 42-1 (FIG. 4) as a return value. In this case, as described in the second embodiment, in step S322-m, the confidential information [GM]1 of the information GM that identifies the aggregation model is sent to the provider 223-1 of the secret associative learning device 42-1 (FIG. 4). Furthermore, if M≧2, the provider 423-m' (where m′∈{2,...,M}) of the secret associative learning device 42-m' further acquires the confidential information [GM] m’ This transmits the secret information [GM] to provider 223-1 of secret associative learning device 42-1 (FIG. 4). m (where m∈{1, ,M}) is input. The providing unit 223-1 receives the input confidential information [GM] m The acquisition unit 412-n of the model learning device 41-n acquires the information GM that identifies the aggregation model from the provider 223-1 of the secret federated learning device 42-1 (FIG. 4) as a return value. The rest is the same as in the fourth embodiment.

[0071] [Hardware configuration] The model learning devices 11-n, 21-n, 31-n, and 41-n and the secret associative learning devices 12-m, 22-1, 32-m, and 42-m in each embodiment are devices configured by a general-purpose or dedicated computer having a processor (hardware processor) such as a central processing unit (CPU) and memories such as random-access memory (RAM) and read-only memory (ROM) executing a predetermined program. That is, the model learning devices 11-n, 21-n, 31-n, and 41-n and the secret associative learning devices 12-m, 22-1, 32-m, and 42-m in each embodiment have, for example, processing circuitry configured to implement each of the components possessed by the device. The computer may have one processor and memory, or multiple processors and memories. The program may be installed on the computer or may be pre-recorded on a ROM or the like. Furthermore, some or all of the processing units may be configured using electronic circuits that independently realize processing functions, rather than electronic circuits that realize functional configurations by loading programs like a CPU.Furthermore, an electronic circuit that configures one device may include multiple CPUs.

[0072] 7 is a block diagram illustrating the hardware configuration of the model learning devices 11-n, 21-n, 31-n, and 41-n and the secret associative learning devices 12-m, 22-1, 32-m, and 42-m in each embodiment. As illustrated in FIG. 7, the model learning devices 11-n, 21-n, 31-n, and 41-n and the secret associative learning devices 12-m, 22-1, 32-m, and 42-m of this example each include a central processing unit (CPU) 10a, an input unit 10b, an output unit 10c, a random access memory (RAM) 10d, a read-only memory (ROM) 10e, an auxiliary storage device 10f, a communication unit 10h, and a bus 10g. The CPU 10a of this example includes a control unit 10aa, a calculation unit 10ab, and a register 10ac, and executes various calculation processes according to various programs loaded into the register 10ac. The input unit 10b is an input terminal, keyboard, mouse, touch panel, etc. through which data is input. The output unit 10c is an output terminal, display, etc. through which data is output. The communication unit 10h is a LAN card, etc., controlled by the CPU 10a that has loaded a predetermined program. The RAM 10d is a static random access memory (SRAM), a dynamic random access memory (DRAM), etc., and has a program area 10da in which a predetermined program is stored and a data area 10db in which various data is stored. The auxiliary storage device 10f is a hard disk, a magneto-optical disc (MO), a semiconductor memory, etc., and has a program area 10fa in which a predetermined program is stored and a data area 10fb in which various data is stored. The bus 10g connects the CPU 10a, the input unit 10b, the output unit 10c, the RAM 10d, the ROM 10e, the communication unit 10h, and the auxiliary storage device 10f so that information can be exchanged. The CPU 10a writes the program stored in the program area 10fa of the auxiliary storage device 10f into the program area 10da of the RAM 10d in accordance with the loaded OS (Operating System) program.Similarly, CPU 10a writes various data stored in data area 10fb of auxiliary storage device 10f to data area 10db of RAM 10d. The addresses in RAM 10d where the programs and data are written are stored in register 10ac of CPU 10a. Control unit 10aa of CPU 10a sequentially reads these addresses stored in register 10ac, reads the programs and data from the areas in RAM 10d indicated by the read addresses, causes calculation unit 10ab to sequentially execute the operations indicated by the programs, and stores the calculation results in register 10ac. This configuration realizes the functional configuration of model learning devices 11-n, 21-n, 31-n, and 41-n and secret associative learning devices 12-m, 22-1, 32-m, and 42-m.

[0073] The above-mentioned program can be recorded on a computer-readable recording medium. Examples of computer-readable recording media include non-transitory recording media. Examples of such recording media include magnetic recording devices, optical disks, magneto-optical recording media, and semiconductor memories.

[0074] This program may be distributed, for example, by selling, transferring, or lending a portable recording medium, such as a DVD or CD-ROM, on which the program is recorded. Furthermore, the program may be distributed by storing the program in a storage device of a server computer and transferring the program from the server computer to other computers via a network. As described above, a computer that executes such a program may, for example, first temporarily store the program recorded on a portable recording medium or transferred from the server computer in its own storage device. Then, when executing a process, the computer reads the program stored in its own storage device and executes processing in accordance with the read program. Alternatively, the program may be executed by a computer that reads the program directly from a portable recording medium and executes processing in accordance with the program. Furthermore, the computer may execute processing in accordance with the received program each time a program is transferred from the server computer to the computer. Alternatively, the server computer may not transfer the program to the computer, but may instead execute the processing function simply by issuing an execution instruction and obtaining the results, thereby executing the processing described above through a so-called ASP (Application Service Provider) type service. In this embodiment, the program includes information used for processing by an electronic computer that is equivalent to a program (such as data that is not a direct instruction to a computer but has properties that dictate computer processing).

[0075] In each embodiment, the device is configured by executing a predetermined program on a computer, but at least a part of the processing contents may be realized by hardware.

[0076] [Other variations] Note that the present invention is not limited to the above-described embodiment. For example, in the above-described embodiment, all or some of the model learning devices may complete learning of the worker models using a consensus building method, agree to have provided confidential information identifying the worker models to the secret federated learning device, and provide this information to the secret federated learning device. In this way, the secret federated learning device can receive the confidential information identifying the worker models of all or some of the model learning devices and then obtain the confidential information identifying the aggregated model into which the worker models have been aggregated.

[0077] Furthermore, the various processes described above may not only be executed in chronological order as described, but may also be executed in parallel or individually depending on the processing capabilities of the devices executing the processes or as necessary. Needless to say, other modifications are possible within the scope of the present invention. [Explanation of symbols]

[0078] 1~4 Federated Learning System 11-n, 21-n, 31-n, 41-n model learning device 111-n Storage section 112-n,212-n,312-n,412-n Acquisition part 113-n Learning Department 114-n Secret Department 115-n,415-n Providing Department 317-n Judgment section 12-m, 22-1, 32-m, 42-m Secret Associative Learning Device 121-m,421-m Acquisition Department 122-m, 322-m Secret aggregation processing unit 123-m,223-1,423-m Provided by 127-m storage section 328-m,428-m Judgment section

Claims

1. a storage unit for storing local training data; an acquisition unit that acquires information specifying an aggregation model or secret information of the information specifying the aggregation model from a secret associative learning device; a learning unit that updates the aggregate model by machine learning using the local learning data to obtain a worker model; a concealment unit for obtaining concealment information of information for identifying the worker model; a providing unit that provides confidential information of the information that identifies the worker model to the secret associative learning device; a determination unit that determines whether or not it is necessary to update the aggregate model and newly obtain the worker model; When it is determined that there is no need to update the aggregation model and obtain a new worker model, the learning unit does not update the aggregation model and obtain a new worker model, and after a waiting time has elapsed, the acquisition unit acquires information identifying a new aggregation model or confidential information of the information identifying the new aggregation model from the secret federated learning device, A model learning device in which, when it is determined that the aggregate model needs to be updated and a new worker model needs to be obtained, the learning unit updates the aggregate model through machine learning using the local learning data to obtain the worker model.

2. 2. The model learning device of claim 1, The providing unit further provides plaintext synchronization information to the secret federated learning device, which indicates that the model learning device has provided the secret information identifying the worker model to the secret federated learning device.

3. A method for learning a network comprising: a plurality of model learning devices and a secret association learning device; Each of the plurality of model learning devices a storage unit for storing local training data; an acquisition unit that acquires information specifying an aggregation model or secret information of the information specifying the aggregation model from the secret associative learning device; a learning unit that updates the aggregate model by machine learning using the local learning data to obtain a worker model; a concealment unit for obtaining concealment information of information for identifying the worker model; a providing unit that provides confidential information of the information that identifies the worker model to the secret associative learning device; a determination unit that determines whether or not it is necessary to update the aggregate model and newly obtain the worker model; When it is determined that there is no need to update the aggregation model and obtain a new worker model, the learning unit does not update the aggregation model and obtain a new worker model, and after a waiting time has elapsed, the acquisition unit acquires information identifying a new aggregation model or confidential information of the information identifying the new aggregation model from the secret federated learning device, when it is determined that it is necessary to update the aggregate model and obtain a new worker model, the learning unit updates the aggregate model by machine learning using the local learning data to obtain the worker model; The secret association learning device an acquisition unit that acquires confidential information of information that identifies a plurality of worker models from the plurality of model learning devices; a secret aggregation processing unit that obtains secret information for identifying the aggregate model obtained by aggregating the plurality of worker models, without obtaining the plurality of worker models, by secret computation using secret information for identifying the plurality of worker models; a providing unit that provides information that identifies the aggregate model or confidential information that identifies the aggregate model to the plurality of model learning devices; A federated learning system with.

4. The associative learning system of claim 3, The acquisition unit further includes a determination unit that determines whether or not confidential information for identifying the worker model has been obtained from a predetermined model learning device, A federated learning system in which, when it is determined that confidential information identifying the worker model has been obtained from the specified model learning device, the secret aggregation processing unit obtains confidential information identifying the aggregated model that aggregates the worker model by secret calculation using the confidential information identifying the worker model.

5. 5. The associative learning system of claim 4, the acquisition unit acquires plaintext synchronization information indicating that the model learning device has provided the secret information for identifying the worker model to the secret federated learning device; A federated learning system, wherein the determination unit uses the synchronization information to determine whether confidential information for identifying the worker model has been obtained from the specified model learning device.

6. A model learning method by a model learning device, an acquisition step in which an acquisition unit acquires information specifying an aggregate model or secret information of the information specifying the aggregate model from a secret associative learning device; a learning step in which, in a learning unit, the aggregate model is updated by machine learning using local learning data stored in a storage unit to obtain a worker model; a concealment step of obtaining, in a concealment unit, concealment information of information that identifies the worker model; a providing step of providing, in a providing unit, confidential information of information identifying the worker model to the secret federated learning device; a determination step in which a determination unit determines whether or not it is necessary to update the aggregation model and newly obtain the worker model; When it is determined that there is no need to update the aggregation model and obtain a new worker model, the learning unit does not update the aggregation model and obtain a new worker model, and after a waiting time has elapsed, the acquisition unit acquires information identifying a new aggregation model or confidential information of the information identifying the new aggregation model from the secret federated learning device, A model learning method in which, when it is determined that the aggregate model needs to be updated and a new worker model needs to be obtained, the learning unit updates the aggregate model by machine learning using the local learning data to obtain the worker model.

7. A federated learning method using a plurality of model learning devices and a secret federated learning device, comprising: Each of the plurality of model learning devices Local learning data is stored in the memory section, an acquisition step in which an acquisition unit acquires information specifying an aggregation model or secret information of the information specifying the aggregation model from the secret associative learning device; a learning step in which, in a learning unit, the aggregate model is updated by machine learning using the local learning data to obtain a worker model; a concealment step of obtaining, in a concealment unit, concealment information of information that identifies the worker model; a providing step of providing, in a providing unit, confidential information of information identifying the worker model to the secret federated learning device; a determination step in which a determination unit determines whether or not it is necessary to update the aggregate model and newly obtain the worker model; When it is determined that there is no need to update the aggregation model and obtain a new worker model, the learning unit does not update the aggregation model and obtain a new worker model, and after a waiting time has elapsed, the acquisition unit acquires information identifying a new aggregation model or confidential information of the information identifying the new aggregation model from the secret federated learning device, when it is determined that it is necessary to update the aggregate model and obtain a new worker model, the learning unit updates the aggregate model by machine learning using the local learning data to obtain the worker model; The secret association learning device an acquisition step in which confidential information for identifying a plurality of worker models is acquired from a plurality of model learning devices in an acquisition unit; a secret aggregation processing step in which, in a secret aggregation processing unit, secret information for identifying the plurality of worker models is obtained by secret computation using secret information for identifying the plurality of worker models, without obtaining the plurality of worker models; a providing step of providing, in a providing unit, information that identifies the aggregation model or confidential information that identifies the aggregation model to the plurality of model learning devices.

8. A program for causing a computer to function as the model learning device of claim 1 or 2.

Images