Associative Learning Systems and Associative Learning Methods

Abstract

To perform multiple encrypting of a plurality of feature quantities and decrypting of the multiple-encrypted feature quantities that are multiple-encrypted, in multiple stages using different servers.SOLUTION: A federated learning system includes: one or more data transmitting devices that calculate a plurality of feature quantities from a learning model and multiple pieces of learning data, multiple-encrypt the plurality of feature quantities, and generate a plurality of encrypted feature quantities as transmission information; a feature quantity aggregation server that accepts input of the transmission information generated by the data transmitting devices, aggregates the plurality of encrypted feature quantities input, and generates an encrypted aggregated feature quantity; and a learning model update server that updates the learning model based on information received from the feature quantity aggregation server. The feature quantity aggregation server executes the first decryption process among a plurality of decryption processes that decrypts the encrypted aggregated feature quantity in multiple stages. The learning model update server executes the last decryption process among the plurality of decryption processes.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to a federated learning system and a federated learning method.

Background Art

[0002] The spread of AI (Artificial Intelligence) has been progressing in various fields such as finance, healthcare, and manufacturing. Among them, in order to develop more accurate AI, there are movements for multiple organizations to cooperate. However, directly sharing the data held by each organization is not desirable because the risk of information leakage increases. Therefore, federated learning has been proposed as a method for developing a common AI without directly sharing the data held by each organization.

[0003] Federated learning is an information processing method executed in a system including a plurality of data transmission devices and a central server. Each data transmission device calculates feature amounts necessary for updating the learning model using the common learning model distributed from the central server and the data it holds. Then, the feature amounts are sent from each data transmission device to the central server, and the central server aggregates the feature amounts received from each data transmission device and updates the learning model. In a system adopting federated learning, each data transmission device shares only the feature amounts with the central server, so it is considered to have high privacy protection performance. However, in recent years, attacks for restoring data used for learning from the feature amounts have been reported.

[0004] As a countermeasure against attacks for restoring data from feature amounts, a method of concealing feature amounts using encryption technology has been proposed. As technologies for applying encryption technology to federated learning, there are Non-Patent Document 1 and Non-Patent Document 2. The technology described in Non-Patent Document 1 is a method applying symmetric key encryption technology and secret sharing technology to federated learning, and the technology described in Non-Patent Document 2 is a method applying public key encryption technology to federated learning.

Prior Art Documents

Non-Patent Documents

[0005] [Non-Patent Document 1] Keith Bonawitz and 8 others, “Practical Secure Aggregation for Privacy-Preserving Machine Learning.”, ACM SIGSAC Conference on Computer and Communications Security, Pages:1175-1191, October 30, 2017. [Non-Patent Document 2] Le Trieu Phong and 4 others, “Privacy-Preserving Deep Learning via Additively Homomorphic Encryption”, December 29, 2017, IEEE Transactions on Information Forensics and Security (Volume:13, Issue:5, pages:1333-1345) [Overview of the Initiative] [Problems that the invention aims to solve]

[0006] In the federated learning described in Non-Patent Document 1, the computational and communication costs of each data transmission device depend on the number of data transmission devices due to the effects of key sharing and secret sharing processes. Therefore, efficiency may decrease in large-scale systems with many data transmission devices. On the other hand, in the federated learning described in Non-Patent Document 2, the computational and communication costs of each data transmission device are independent of the number of data transmission devices. However, the configuration involves data transmission devices obtaining aggregated features, making it difficult to apply to systems where a central server obtains aggregated features.

[0007] Therefore, even if the method described in Non-Patent Document 2 is applied to a system in which a central server has the function of aggregating encrypted feature quantities from each data transmission device and the function of decrypting the aggregated feature quantities to update the learning model, the central server can decrypt the aggregated feature quantities, and there is a risk that the feature quantities will be decrypted at the central server that aggregates the feature quantities.

[0008] Therefore, the objective of the present invention is to realize a secure federated learning system that allows aggregated features to be obtained from a central server and is applicable to large-scale systems. [Means for solving the problem]

[0009] To solve the above problems, the present invention comprises one or more data transmission devices that calculate each of a plurality of features from a learning model and each of a plurality of learning data, and multiple times encrypt each of the calculated plurality of features to generate a plurality of encrypted features as transmission information; a feature aggregation server that receives the transmission information generated by the data transmission device via a communication network, and aggregates each of the plurality of encrypted features belonging to the input transmission information to generate an encrypted aggregate feature; and a learning model update server that transmits and receives information with the feature aggregation server via the communication network and updates the learning model based on the information received from the feature aggregation server, wherein the feature aggregation server executes the first of a plurality of decryption processes that decrypt the encrypted aggregate feature in multiple stages, and the learning model update server executes the last of the plurality of decryption processes. The feature aggregation server sequentially inputs the encrypted feature quantities from two or more of the data transmission devices, monitors a first timing when the input number of encrypted feature quantities reaches a set number, and a second timing when a set time has elapsed from the input start time for inputting the encrypted feature quantities from the data transmission devices, and aggregates the input encrypted feature quantities at least at the first timing or the second timing. It is characterized by the following: [Effects of the Invention]

[0010] According to the present invention, a secure federated learning system can be realized that allows aggregated features to be obtained from a central server and is applicable to large-scale systems.

[0011] Other issues, configurations, and effects not mentioned above will be clarified by the following description of the embodiments. [Brief explanation of the drawing]

[0012] [Figure 1] This is a block diagram showing an example configuration of a federative learning system according to Embodiment 1 of the present invention. [Figure 2] This is a schematic diagram showing an overview of the processing of the federated learning system according to Embodiment 1 of the present invention. [Figure 3]This is a block diagram showing an example of the hardware configuration of a feature aggregation server according to Embodiment 1 of the present invention. [Figure 4] This is a block diagram showing an example of the hardware configuration of a learning model update server according to Embodiment 1 of the present invention. [Figure 5] This is a block diagram showing an example of the hardware configuration of a data transmission device according to Embodiment 1 of the present invention. [Figure 6] This is a block diagram showing an example of the hardware configuration of a data receiving device according to Embodiment 1 of the present invention. [Figure 7] This sequence diagram shows an example of the processing of the federative learning system according to Embodiment 1 of the present invention. [Figure 8] This flowchart shows an example of processing by a data transmission device according to Embodiment 1 of the present invention. [Figure 9] This is a schematic diagram showing an overview of the processing of the federated learning system according to Embodiment 2 of the present invention. [Figure 10] This sequence diagram shows an example of the processing of the federative learning system according to Embodiment 2 of the present invention. [Modes for carrying out the invention]

[0013] Embodiments of the present invention will be described below with reference to the attached drawings. In this embodiment, the same components will be denoted by the same reference numerals in principle, and repeated descriptions will be omitted. It should be noted that this embodiment is merely one example for realizing the present invention and does not limit the technical scope of the present invention. In this embodiment, a system for updating a learning model based on features obtained from learning data while protecting the security of the learning data held by the data transmission device will be described. [Examples]

[0014] FIG. 1 is a block diagram showing a configuration example of the federated learning system according to Embodiment 1 of the present invention. In FIG. 1, the federated learning system 10 includes a feature aggregation server 100, a learning model update server 200, a data transmission device 300, a data reception device 400, and a communication network 500. The data transmission device 300 holds, for example, a plurality of learning data (learning data), calculates a plurality of feature amounts using the plurality of learning data and a learning model, encrypts each calculated feature amount, and generates a plurality of encrypted feature amounts as transmission information respectively. As the learning data, for example, image data such as a photograph of an animal or an X-ray photograph, log data including stock trading data, etc. are used. As the learning model, for example, a Deep Neural Network used in DeepLearning, a random forest, etc. are used. As the feature amount, when the learning model is a Deep Neural Network, real values corresponding to the number of parameters of the Deep Neural Network are calculated.

[0015] When the feature aggregation server 100 receives a plurality of encrypted feature amounts generated by the data transmission device 300, it aggregates each of the received encrypted feature amounts to generate an encrypted aggregated feature amount, and performs primary decryption of the generated encrypted aggregated feature amount. The learning model update server 200 performs secondary decryption of the encrypted aggregated feature amount that has been primary decrypted by the feature aggregation server 100, and updates the learning model based on the secondary decrypted aggregated feature amount. The data reception device 400 receives the learning model updated by the learning model update server 200 from the learning model update server 200.

[0016] The feature aggregation server 100, the learning model update server 200, the data transmission device 300, and the data reception device 400 transmit and receive information to and from each other via a communication network 500 such as the Internet. Details of the configurations of the feature aggregation server 100, the learning model update server 200, the data transmission device 300, and the data reception device 400 will be described later. Also, some or all of the devices included in the federated learning system 10 may be integrated. For example, the feature aggregation server 100 and the learning model update server 200 may be integrated, or the data transmission device 300 and the data reception device 400 may be integrated.

[0017] FIG. 2 is a schematic diagram showing an outline of the processing of the federated learning system according to Embodiment 1 of the present invention. In FIG. 2, the data transmission device 300 is composed of, for example, a plurality of units or a plurality of information processing systems (encryption processing systems). When the data transmission device 300 is composed of a plurality of units, each data transmission device 300 holds, for example, the learning model 320 transmitted from the learning model update server 200 and the learning data 321 obtained by learning using the learning model 320 by each data transmission device 300, and generates unique feature amounts 322 for each data transmission device 300 based on the learning model 320 and the learning data 321. After that, each data transmission device 300 encrypts each generated feature amount 322 multiplicatively. At this time, each data transmission device 300 encrypts each feature amount 322 with a first public key 323 and a second public key 324, respectively, generates each double-encrypted encrypted feature amount 325, and transmits information on each generated encrypted feature amount 325 to the feature aggregation server 100. Also, when the data transmission device 300 is composed of a plurality of information processing systems (encryption processing systems), in the data transmission device 300, a plurality of feature amounts 322 are generated from the learning model 320 to be developed and the plurality of learning data 321 by the processing of the plurality of information processing systems (encryption processing systems), and a plurality of encrypted feature amounts 324 are generated from each generated feature amount 322.

[0018] The feature aggregation server 100 is configured as a feature aggregation device that, upon receiving information on each encrypted feature 325 from each data transmission device 300, aggregates each received encrypted feature 325 to generate an encrypted aggregate feature 120, decrypts the generated encrypted aggregate feature 120 using the primary secret key 121 to generate a primary decrypted encrypted aggregate feature 122, and transmits the information of the generated primary decrypted encrypted aggregate feature 122 to the learning model update server 200. In other words, the feature aggregation server 100 is equipped with the function of aggregating each encrypted feature 325 and the function of decrypting the encrypted aggregate feature 120 obtained by aggregating each encrypted feature 325 using the primary secret key 121. At this time, the feature aggregation server 100 executes the first decryption process (first stage decryption process) of multiple decryption processes that decrypt the encrypted aggregate feature 325 in multiple stages.

[0019] The learning model update server 200 is configured as a learning model update device that, upon receiving information on the primary decrypted encrypted aggregate feature 122 from the feature aggregation server 100, decrypts the received primary decrypted encrypted aggregate feature 122 using the secondary secret key 220 to generate an aggregate feature 221, updates the learning model 223 (the learning model corresponding to learning model 320) based on the generated aggregate feature 221, and transmits the updated learning model 223 information to the data receiving device 400. In other words, the learning model update server 200 is equipped with the function of decrypting the primary decrypted encrypted aggregate feature 122 received from the feature aggregation server 100 using the secondary secret key 220 to generate an aggregate feature 221, and updating the learning model 223 based on the generated aggregate feature 221. At this time, the learning model update server 200 executes the final decryption process (the second stage of decryption) among multiple decryption processes that decrypt the encrypted aggregate feature 325 in multiple stages. When the data receiving device 400 receives information about the updated learning model 223 from the learning model update server 200, it stores the received information about the learning model 223 in memory or elsewhere.

[0020] Figure 3 is a block diagram showing an example of the hardware configuration of a feature aggregation server according to Embodiment 1 of the present invention. In Figure 3, the feature aggregation server 100 is composed of a computer having, for example, a processor (CPU) 101, an auxiliary storage device 102, a memory 103, a display device 105, an input / output interface 106, and a communication interface 107, and each device is connected to the others via an internal signal line 104 such as a bus.

[0021] The processor 101 is a processing unit that executes programs stored in the memory 103 and controls each device. The memory 103 includes a non-volatile, non-temporary memory element called ROM (Read Only Memory) and a volatile memory element called RAM (Random Access Memory). ROM stores immutable programs (such as the BIOS). RAM is a high-speed, volatile memory element such as DRAM (Dynamic Random Access Memory) and temporarily stores programs executed by the processor 101 and data used during program execution.

[0022] The auxiliary storage device 102 is a large-capacity, non-volatile, non-temporary storage device such as a magnetic storage device (HDD) or flash memory (SSD), and stores the program executed by the processor 101 and the data used when the program is executed. In other words, the program is read from the auxiliary storage device 102, loaded into memory 103, and then executed by the processor 101.

[0023] The input / output interface (I / F) 106 is connected to input devices such as a keyboard and mouse, and is an interface for inputting information from the operator's operations. The input / output interface 106 is also connected to output devices such as a display device 105 and a printer, and is an interface for outputting the program execution results in a format that the operator can view. The display device 105 displays the program execution results output from the input / output interface 106.

[0024] The communication interface (I / F) 107 is a network interface that controls communication with other devices via the communication network 500 according to a predetermined protocol. The communication interface 107 may also include a serial interface, such as USB (Universal Serial Bus).

[0025] Some or all of the program executed by the processor 101 may be provided to the feature aggregation server 100 via the communication network 500 from a removable media (such as a CD-ROM or flash memory) or an external computer equipped with a non-temporary storage device, and stored in a non-volatile auxiliary storage device 102, which is also a non-temporary storage device. For this reason, the feature aggregation server 100 can be configured to have an interface for reading data from the removable media. The same applies to the learning model update server 200, the data transmission device 300, and the data receiving device 400.

[0026] The feature aggregation server 100 is a computer system that operates on a single physical computer or on multiple logically or physically configured computers. It may operate on the same computer in separate threads, or on a virtual computer built on multiple physical computing resources. The same applies to the learning model update server 200, the data transmission device 300, and the data reception device 400.

[0027] The processor 101 includes, for example, a key generation unit 111, a feature aggregation unit 112, and a decoding processing unit 113 as software resources. For example, the processor 101 functions as a key generation unit 111 by operating according to a key generation program loaded into memory 103, functions as a feature aggregation unit 112 by operating according to a feature aggregation program loaded into memory 103, and functions as a decoding processing unit 113 by operating according to a decoding processing program loaded into memory 103.

[0028] The key generation unit 111 generates a primary public key for encrypting the feature quantity and a primary secret key for decrypting the feature quantity, based on information about the encryption method used for encryption in each data transmission device 300. In this case, the key generation unit 111 uses, for example, additive homomorphic encryption as the encryption technique. Additive homomorphic encryption is a technique that allows ciphertexts to be added together while remaining encrypted.

[0029] The following explains additive homomorphism using mathematical formulas. Let M1 and M2 be arbitrary values, the encryption key be pk, and the encryption function be Enc. For example, Enc(M,pk) refers to the ciphertext obtained by encrypting the value M with the encryption key pk. Furthermore, let * be the operation for performing addition while the values ​​are encrypted. In this case, due to the additive homomorphism, the following equation (1) holds.

[0030] Enc(M1,pk)*Enc(M2,pk)=Enc(M1+M2,pk)...(1)

[0031] From equation (1) above, a ciphertext of the sum M1+M2 can be generated from the ciphertexts of M1 and M2. Furthermore, among additive homomorphic encryption schemes, there are encryption schemes that allow for the addition of public keys. If pk1 and pk2 are the public keys, M is an arbitrary value, and * is the operation for public key addition, the addition of public keys can be expressed mathematically as shown in equation (2) below.

[0032] Enc(M,pk1)*Enc(M,pk2)=Enc(M,pk1+pk2)···(2)

[0033] By utilizing the additivity of public keys, the computational cost of cryptographic processing can be reduced. Here, as a cryptographic technique to be applied to the federated learning system 10, for example, the application of Elliptic ElGamal cryptography, which is an additive homomorphic encryption scheme having the above-mentioned additivity of public keys, can be considered.

[0034] In this process, the feature aggregation unit 112 performs addition or averaging as aggregation of encrypted feature quantities 325 received from each data transmission device 300, for example, to generate encrypted aggregate feature quantities 120. The decryption processing unit 113 uses the secret key 121 generated by the key generation unit 111 to decrypt the encrypted aggregate feature quantities 120 generated by the feature aggregation unit 112 to generate primary decrypted encrypted aggregate feature quantities 122.

[0035] Figure 4 is a block diagram showing an example of the hardware configuration of a learning model update server according to Embodiment 1 of the present invention. In Figure 4, the learning model update server 200 is composed of a computer having, for example, a processor (CPU) 201, an auxiliary storage device 202, a memory 203, a display device 205, an input / output interface 206, and a communication interface 207, and each device is connected to the others via an internal signal line 204 such as a bus.

[0036] The hardware configurations of the processor 201, auxiliary storage device 202, memory 203, internal signal lines 204, display device 205, input / output interface 206, and communication interface 207 are the same as those of the processor 101, auxiliary storage device 102, memory 103, internal signal lines 104, display device 105, input / output interface 106, and communication interface 107, so their descriptions are omitted.

[0037] The processor 201 includes a key generation unit 211, a learning model update unit 212, and a decryption processing unit 213 as software resources. In this case, for example, the processor 201 functions as a key generation unit 211 by operating according to a key generation program loaded into memory 203, functions as a learning model update unit 212 by operating according to a learning model update program loaded into memory 203, and functions as a decryption processing unit 213 by operating according to a decryption processing program loaded into memory 103.

[0038] The key generation unit 211 generates a secondary public key for encrypting feature quantities and a secondary secret key for decrypting feature quantities, based on information about the encryption method used for encryption in each data transmission device 300. In this case, the key generation unit 111 uses additive homomorphic encryption as the encryption technique. The decryption processing unit 213 decrypts the primary decrypted encrypted aggregate feature quantities 122 received from the feature quantity aggregation server 100 using the secondary secret key generated by the key generation unit 211 to generate aggregate feature quantities 221. The learning model update unit 212 updates the learning model 223 using the aggregate feature quantities 221 generated by the decryption processing unit 213 and the learning model 223, for example by addition or averaging.

[0039] Figure 5 is a block diagram showing an example of the hardware configuration of a data transmission device according to Embodiment 1 of the present invention. In Figure 5, the data transmission device 300 is composed of a computer having, for example, a processor (CPU) 301, an auxiliary storage device 302, a memory 303, a display device 305, an input / output interface 306, and a communication interface 307, and each device is connected to the others via an internal signal line 304 such as a bus.

[0040] The hardware configurations of the processor 301, auxiliary storage device 302, memory 303, internal signal lines 304, display device 305, input / output interface 306, and communication interface 307 are the same as those of the processor 101, auxiliary storage device 102, memory 103, internal signal lines 104, display device 105, input / output interface 106, and communication interface 107, so their descriptions are omitted.

[0041] The processor 301 includes, for example, a feature calculation unit 311 and an encryption processing unit 312 as software resources. In this case, for example, the processor 301 functions as a feature calculation unit 311 by operating according to a feature calculation program loaded into memory 303, and functions as an encryption processing unit 312 by operating according to an encryption processing program loaded into memory 303.

[0042] The auxiliary storage device 302 of the data transmission device 300 holds the training data 321 and the training model 320 obtained by the processor 301 using the training model 320 under development. The training model 320 and training data 321 may also be stored in the memory 303 of the data transmission device 300.

[0043] The feature calculation unit 311 calculates feature quantities 322 using the training data 321 and the training model 320. The cryptography processing unit 312 encrypts the feature quantities 322 using, for example, the primary public key 323 generated by the key generation unit 111 of the feature aggregation server 100 and the secondary public key 324 generated by the key generation unit 211 of the training model update server 200 to generate encrypted feature quantities 325. At this time, the data transmission device 300 can input information about the training model 320, the primary public key 323, and the secondary public key 324 by sending and receiving information with the feature aggregation server 100 and the training model update server 200. If the data transmission device 300 is configured as multiple units, the processing of multiple data transmission devices 300 will generate multiple feature quantities 322 from the training model 320 under development and multiple training data 321, and multiple encrypted feature quantities 325 will be generated from each of the generated feature quantities 322. Furthermore, if the data transmission device 300 is composed of multiple information processing systems (encryption processing systems), the feature calculation unit 311 and the encryption processing unit 312 constitute multiple information processing systems (encryption processing systems).

[0044] Figure 6 is a block diagram showing an example of the hardware configuration of a data receiving device according to Embodiment 1 of the present invention. In Figure 6, the data receiving device 400 is composed of a computer having, for example, a processor (CPU) 401, an auxiliary storage device 402, a memory 403, a display device 405, an input / output interface 406, and a communication interface 407, and each device is connected to the others via an internal signal line 404 such as a bus.

[0045] The hardware configurations of the processor 401, auxiliary storage device 402, memory 403, internal signal lines 404, display device 405, input / output interface 406, and communication interface 407 are the same as those of the processor 101, auxiliary storage device 102, memory 103, internal signal lines 104, display device 105, input / output interface 106, and communication interface 107, so their descriptions are omitted.

[0046] Furthermore, when the communication interface 407 receives information about the updated learning model 223 from the learning model update server 200, the processor 401 stores the received information in the auxiliary storage device 402 and also displays it on the screen of the display device 405.

[0047] Figure 7 is a sequence diagram showing an example of the processing of a federated learning system according to Embodiment 1 of the present invention. In Figure 7, the learning model update server 200 first generates a secondary public key 324 as part of the initial phase P701 processing (S701), transmits the learning model 320 to be developed, the generated secondary public key 324, and cryptographic parameter information to the data transmission device 300 via the communication network 500 (S702), and further transmits the cryptographic parameter information to the feature aggregation server 100 via the communication network 500 (S703). The feature aggregation server 100 generates a primary public key 323 based on the received cryptographic parameter information (S704), and transmits the generated primary public key 323 information to multiple data transmission devices 300 via the communication network 500 (S705).

[0048] In the key generation process by the learning model update server 200 (S701) and the key generation process by the feature aggregation server 100 (S704), for example, Elgamal elliptic cryptography, a type of additive homomorphic encryption, is used. In this case, the encryption parameters transmitted in steps S702 and S703 include information such as the use of Elgamal elliptic cryptography, the elliptic curve to be used, and the base point. The setting of the encryption parameters may be performed by either the learning model update server 200 or the feature aggregation server 100. For example, machine learning models such as decision trees and Deep Learning may be used as the learning model 320 to be developed.

[0049] Each data transmission device 300 calculates features 322 using its own training data 321 and the training model 320 received from the training model update server 200 (S706). Before calculating each feature 322, each data transmission device 300 needs to determine the number of times it will perform training locally, called the number of epochs. The number of epochs may be determined individually by each data transmission device 300, or it may be determined in advance by the feature aggregation server 100 or the training model update server 200. The method for calculating each feature 322 may be linear regression or logistic regression in the case of a decision tree, or statistical gradient descent in the case of Deep Learning.

[0050] Next, each data transmission device 300 encrypts the feature quantity 322 using the secondary public key 324 received in step S702 and the primary public key 323 received in step S705, generating an encrypted feature quantity 325 (S707). When encrypting the feature quantity 322, for example, one could encrypt the feature quantity as many times as there are public keys, or one could add up the public keys first and then encrypt only once using the calculated added public key.

[0051] Each data transmission device 300 transmits the information of the encrypted feature quantities 325 generated in step S707 to the feature quantity aggregation server 100 via the communication network 500 (S708).

[0052] When the feature aggregation server 100 receives encrypted feature information 325 from each data transmission device 300, it aggregates the information of each encrypted feature 325 to generate an encrypted aggregated feature 120 (S709). Possible methods for aggregating the information of each encrypted feature 325 include addition and averaging. By using additive homomorphic encryption, it is possible to perform addition while the data remains encrypted. Regarding the timing of aggregation, aggregation may be performed after receiving encrypted feature information 325 from all data transmission devices 300, or a threshold related to time or the number of encrypted feature information 325 may be set, and aggregation may be performed after a certain period of time has elapsed or after receiving a certain number of encrypted feature information 325 or more.

[0053] For example, the feature aggregation server 100 sequentially inputs encrypted features 325 from each of the multiple data transmission devices 300, monitors a first timing (first time) when the number of input encrypted features 325 reaches a set number, and a second timing (second time) when a set time has elapsed from the input start time when encrypted features 325 are input from each of the multiple data transmission devices 300, and aggregates the input encrypted features 325 at least at the first timing or the second timing. This allows the feature aggregation server 100 to start the process of aggregating the input encrypted features 325 from the set first timing or the second timing.

[0054] The feature aggregation server 100 uses the primary secret key 121 generated in step S704 to decrypt the encrypted aggregated feature 120 generated in step S709, generating a primary decrypted encrypted aggregated feature 122 (S710). Here, since the feature aggregation server 100 does not possess the secondary secret key 220 generated by the learning model update server 200 in step S701, it cannot completely decrypt the encrypted aggregated feature 120. In this way, since the feature aggregation server 100 performs primary decryption after aggregation, even if the encrypted feature 325 before aggregation is leaked, it is encrypted multiple times, so the learning model update server 200, which only possesses the secondary secret key 220, cannot completely decrypt it, and the security of the original feature 322 is ensured.

[0055] After this, the feature aggregation server 100 transmits the information of the primary decrypted encrypted aggregate feature 122 generated in step S710 to the learning model update server 200 via the communication network 500 (S711).

[0056] When the learning model update server 200 receives information on the primary decrypted encrypted aggregate feature 122, it uses the secondary secret key 220 generated in step S701 to decrypt the primary decrypted encrypted aggregate feature 122 and generate the aggregate feature 221 (S712).

[0057] Next, the learning model update server 200 updates the learning model 223 using the aggregated features 221 generated in step S712 (S713). When updating the learning model 223, possible methods include adding the aggregated features 221 to the learning model 223, or dividing the aggregated features 221 by the number of data transmission devices 300 and then adding them.

[0058] Subsequently, the learning model update server 200 transmits the updated learning model 223 information to the data receiving device 400 via the communication network 500 (S714).

[0059] If steps S701 to S705 are designated as the initial phase P701, and steps S706 to S714 as the repeating phase P702, then after completing the processing of the initial phase P701, the processing of the repeating phase P702 may be repeated. That is, after step S714, the same procedure may be repeated starting from step S706. Also, after step S713, the processing of step S714 may be skipped, and the same procedure may be repeated starting from step S706.

[0060] Here, we will explain the specific processing in steps S707 to S712, for example, when ElGamal elliptic curve encryption is used as the encryption method. In this case, let G be the base point on the elliptic curve, which is one of the encryption parameters, a be the integer value of the primary secret key, b be the integer value of the secondary secret key, aG be the primary public key, and bG be the secondary public key. Let M1 and M2 be the two points on the elliptic curve to be encrypted. Furthermore, we consider the case where, in encryption, the primary public key aG and the secondary public key bG are first added together to calculate the added public key (a+b)G, and encryption is performed using the above added public key. At this time, the ciphertexts of points M1 and M2 on the elliptic curve are (r1G,r1(a+b)G+M1) and (r2G,r2(a+b)G+M2) respectively, using random integer values ​​r1 and r2 as random numbers (S707). These ciphertexts are sent as encrypted feature quantities 325 from each data transmission device 300 to the feature quantity aggregation server 100 (S708). If integer values ​​r1 and r2 are not used as random numbers, the same data will be converted into the same ciphertext, so the integer values ​​r1 and r2 will be generated each time.

[0061] The feature aggregation server 100 aggregates each ciphertext to calculate the encrypted aggregate feature 120, which is ((r1+r2)G,(r1+r2)(a+b)G+M1+M2) (S709). Furthermore, it performs the following calculation using the primary secret key a and the encrypted aggregate feature 120 (S710). ((r1+r2)(a+b)G+M1+M2)―a((r1+r2)G)=b(r1+r2)G+M1+M2...(3)

[0062] At this point, the feature aggregation server 100 sends the calculated ((r1+r2)G,b(r1+r2)G+M1+M2) as the primary decrypted encrypted aggregate feature 122 to the learning model update server 200 (S711).

[0063] When the learning model update server 200 receives the primary decrypted encrypted aggregate feature 122, it performs the following calculation using the secondary secret key b and the primary decrypted encrypted aggregate feature 122 (S712). (b(r1+r2)G+M1+M2)―b((r1+r2)G)=M1+M2...(4)

[0064] Based on the above calculation, the learning model update server 200 generates M1+M2 as aggregated feature 221 through the decryption process according to equation (4). However, in the federative learning system 10, it is necessary to encrypt feature 322 in step S707. In this case, since feature 322 is a real number and points on the elliptic curve are integers, when using ElGamal elliptic encryption, a mechanism is needed to convert feature 322 to points on the elliptic curve. As a method of conversion, for example, a conversion table between n bits and points on the elliptic curve can be created in advance, and after converting the feature to a binary representation of n bits, it can be converted to points on the elliptic curve using the above conversion table.

[0065] Furthermore, when encrypting feature 322, a method of rounding feature 322 can be employed to efficiently perform encryption and decryption. One method of rounding feature 322 is to set a value n and perform rounding up or rounding down on feature 322 to unify it to an n-bit representation. In other words, if feature 322 is encrypted without rounding, the amount of information in the lookup table (correspondence table) referenced when decrypting the encrypted aggregated feature becomes large, and decryption takes time, so a method of rounding feature 322 is adopted. In this case, each data transmission device 300 performs rounding up or rounding down on feature 322 before multiplexing it, unifying the value of feature 322 to the set bit information. This allows the feature aggregation server 100 and the learning model update server 200 to efficiently decrypt the features.

[0066] Here, as a concrete example, if we define a feature as a real number satisfying 0 < or = feature < or = 1, and the value after rounding the feature as f, then when n=2, since it is 2 bits, the feature can be rounded to the following four values. In this case, the following methods are used to round the feature.

[0067] f=0 if 0 < OR =feature < 0.25 ... (a) f = 0.33 if 0.25 < OR = feature < 0.5 ... (b) f = 0.67 if 0.5 < OR = feature < 0.75 ... (c) f=1 if 0.75 < or =feature < or =1 ... (d) In the case of n=4, since it is 4 bits, the feature can be rounded to 16 different values.

[0068] Figure 8 is a flowchart showing an example of the processing of a data transmission device according to Embodiment 1 of the present invention. This processing is a detailed representation of steps S706 and S708 in Figure 7. In this case, the number of epochs is assumed to be determined by the data transmission device 300.

[0069] In Figure 8, each data transmission device 300 inputs training data 321 (S801), and then specifies the epoch number (a number indicating how many times one training data 321 will be repeated for training) (S802).

[0070] Next, each data transmission device 300 calculates and outputs feature quantities 322 from the training data 321 and the training model 320 if the input training data 321 satisfies the specified number of epochs (S803), encrypts the calculated feature quantities 322, and determines whether or not to send the encrypted feature quantities 322 as encrypted feature quantities 325 to the feature quantity aggregation server 100 (S804).

[0071] After this, if each data transmission device 300 obtains a Yes result in step S804, it sends the encrypted feature 325 to the feature aggregation server 100 (S805), and then terminates processing in this routine. On the other hand, if each data transmission device 300 obtains a No result in step S804, it then terminates processing in this routine.

[0072] In this embodiment, the feature aggregation server 100 decrypts the encrypted aggregated feature 120 after aggregating the encrypted feature 325, but it is also possible to decrypt the encrypted aggregated feature 120 before aggregating the encrypted feature 325.

[0073] In this embodiment, instead of the learning model update server 200 generating the secondary secret key 220, the data transmission device 300 may generate the secondary secret key 220, and the learning model update server 200 may receive the secondary secret key 220 generated by the data transmission device 300 from the data transmission device 300. In other words, as long as the learning model update server 200's secondary secret key 220 is not leaked to the feature aggregation server 100, the feature aggregation server 100 cannot decrypt the primary decrypted encrypted aggregated feature 120.

[0074] In this embodiment, when the data transmission device 300 generates a secret key, multiple data transmission devices 300 may each generate a secret key, and the value derived from the secret keys generated by each data transmission device 300 may be used as the secondary secret key 220. For example, the sum of n secret keys may be used as the secondary secret key 220.

[0075] Furthermore, although this embodiment uses two types of private key and public key pairs (primary and secondary), there are no limitations on the number of encryption and decryption types. For example, each data transmission device 300 may encrypt the feature quantities five times, the feature quantity aggregation server 100 may decrypt the encrypted feature quantities twice, and the model update server 200 may decrypt the encrypted feature quantities three times.

[0076] According to this embodiment, multiple feature quantities can be encrypted multiple times, and these multiple encrypted feature quantities can be aggregated and decrypted in multiple stages using different devices. As a result, the risk of feature quantities being decrypted during the multi-stage decryption process can be reduced, thereby ensuring high information security.

[0077] Furthermore, according to this embodiment, each data transmission device 300 encrypts the feature quantity 322 multiple times (double) using a public-key cryptography scheme with a primary public key 323 and a secondary public key 324. Therefore, the computational and communication costs of each data transmission device 300 do not depend on the number of data transmission devices 300, and the federated learning system 10 can be applied to large-scale systems with a large number of data transmission devices.

[0078] Furthermore, according to this embodiment, since each data transmission device 300 encrypts the feature quantities 322 using an additive homomorphic encryption scheme, the feature quantity aggregation server 100 can aggregate the encrypted feature quantities 325 from each data transmission device 300 at any time. In other words, the feature quantity aggregation server 100 can generate a primary decrypted encrypted aggregated feature quantity 122 and send it to the learning model update server 200, at its own discretion. As a result, for example, even if communication with one of the data transmission devices 300 is interrupted, the federated learning system 10 can continue federated learning by aggregating the encrypted feature quantities 325 that have been successfully received from the other data transmission devices 300 to generate a primary decrypted encrypted aggregated feature quantity 122 and sending it to the learning model update server 200.

[0079] Furthermore, according to this embodiment, since the learning model update server 200 possesses the secondary secret key 220, the learning model update server 200 can use the secondary secret key 220 to decrypt the primary decrypted encrypted aggregate feature 122 and generate the aggregate feature 221.

[0080] Furthermore, provided that the feature aggregation server 100 does not leak feature 322, and the learning model update server 200 does not decrypt any features other than those that have already been decrypted, it is possible to achieve the same level of security as when feature 322 is double-encrypted (encrypted using both the primary and secondary public keys) by simply single-encrypting feature 322 (encrypting using only the primary public key). [Examples]

[0081] Example 2 involves arranging feature aggregation servers in a multi-layered structure and distributing the aggregation process across multiple feature aggregation servers, thereby reducing the computational and communication load on each feature aggregation server. This approach makes it applicable to a hierarchical organizational structure for federative learning systems.

[0082] Figure 9 is a schematic diagram showing an overview of the processing of a federated learning system according to Embodiment 2 of the present invention. In Figure 9, the federated learning system 20 according to Embodiment 2 comprises multiple data devices 300, for example, four or more data devices 300, two or more first-layer feature aggregation servers 600, one second-layer feature aggregation server 700, a learning model update server 200, and a data receiving device 400. In this case, the second-layer feature aggregation server 700 holds the primary secret key 701, and the learning model update server 200 holds the secondary secret key 220.

[0083] Each data transmission device 300 holds, for example, the learning model 320 transmitted from the learning model update server 200 and the learning data 321 obtained by each data transmission device 300 through learning using the learning model 320, and generates feature quantities unique to each data transmission device 300 based on the learning model 320 and the learning data 321. After this, each data transmission device 300 encrypts each generated feature quantity with the primary public key 323 and the secondary public key 324 to generate each encrypted feature quantity 325, and transmits the information of each generated encrypted feature quantity 325 to the designated first-layer feature quantity aggregation server 600 among the multiple first-layer feature quantity aggregation servers 600 that are the target of communication.

[0084] When each first-layer feature aggregation server 600 receives encrypted feature information 325 from two or more data transmission devices 300, it aggregates the received encrypted feature information 325 to generate an encrypted aggregate feature 601 as the first-layer encrypted aggregate feature, and transmits the information of the generated encrypted aggregate feature 601 to the second-layer feature aggregation server 700.

[0085] When the second-layer feature aggregation server 700 receives encrypted aggregate feature 601 information from multiple first-layer feature aggregation servers 600, it aggregates the received encrypted aggregate feature 601 to generate encrypted aggregate feature 701, decrypts the generated encrypted aggregate feature 701 using the primary secret key 702 (a secret key corresponding to the primary secret key 121) to generate a primary decrypted encrypted aggregate feature 703 (a primary decrypted encrypted aggregate feature corresponding to the primary decrypted encrypted aggregate feature 122), and transmits the information of the generated primary decrypted encrypted aggregate feature 703 to the learning model update server 200.

[0086] When the learning model update server 200 receives information on the primary decrypted encrypted aggregate feature 703 from the second layer feature aggregation server 700, it decrypts the received primary decrypted encrypted aggregate feature 703 using the secondary secret key 220 to generate an aggregate feature 221, updates the learning model 223 based on the generated aggregate feature 221, and transmits the updated learning model 223 information to the data receiving device 400.

[0087] When the data receiving device 400 receives information about the updated learning model 223 from the learning model update server 200, it stores the received information about the learning model 223 in memory or elsewhere.

[0088] Figure 10 is a sequence diagram showing an example of the processing of the federated learning system according to Embodiment 2. In Figure 10, the learning model update server 200 first generates a secondary public key 324 as part of the initial phase P701 processing (S701), and transmits information on the learning model 320 under development, the generated secondary public key 324, and cryptographic parameters to the data transmission device 300 via the communication network 500 (S702). Furthermore, it transmits information on cryptographic parameters to the second layer feature aggregation server 700 via the communication network 500 (S703). The second layer feature aggregation server 700 generates a primary public key 323 based on the received cryptographic parameter information (S704), and transmits information on the generated primary public key 323 to multiple data transmission devices 300 via the communication network 500 (S705).

[0089] In the key generation process by the learning model update server 200 (S701) and the key generation process by the second-layer feature aggregation server 700 (S704), for example, Elgamal elliptic cryptography, a type of additive homomorphic encryption, is used. In this case, the encryption parameters transmitted in steps S702 and S703 include information such as the use of Elgamal elliptic cryptography, the elliptic curve to be used, and the base point. The setting of the encryption parameters may be performed by either the learning model update server 200 or the second-layer feature aggregation server 700. For example, machine learning models such as decision trees and Deep Learning can be used as the learning model 320 to be developed.

[0090] Each data transmission device 300 calculates features 322 using its own training data 321 and the training model 320 received from the training model update server 200 (S706). Before calculating each feature 322, each data transmission device 300 needs to determine the number of times it will perform training locally, called the number of epochs. The number of epochs may be determined individually by each data transmission device 300, or it may be determined in advance by the feature aggregation server 100 or the training model update server 200. The method for calculating each feature may be, for example, linear regression or logistic regression in the case of a decision tree, or statistical gradient descent in the case of Deep Learning.

[0091] Next, each data transmission device 300 encrypts the feature quantity 322 using the secondary public key 324 received in step S702 and the primary public key 323 received in step S705, generating an encrypted feature quantity 325 (S707). When encrypting the feature quantity 322, for example, one could encrypt the feature quantity as many times as there are public keys, or one could add up the public keys first and then encrypt only once using the calculated added public key.

[0092] Each data transmission device 300 transmits the information of the encrypted feature quantities 325 generated in step S707 to the first-layer feature quantity aggregation server 600, which is the target of communication, via the communication network 500 (S708).

[0093] When each first-layer feature aggregation server 600 receives encrypted feature information 325 from multiple data transmission devices 300, it aggregates the received encrypted feature information 325 to generate an encrypted aggregated feature 601 (S709), and transmits the generated encrypted aggregated feature 601 to the second-layer feature aggregation server 700 via the communication network 500 (S1001).

[0094] Possible methods for aggregating the information from each encrypted feature 325 include addition and averaging. By using additive homomorphic encryption, it becomes possible to perform addition while the data remains encrypted. Regarding the timing of aggregation, aggregation may be performed after receiving all encrypted feature 325 from the data transmission device 300, or a threshold related to time or the number of encrypted feature 325 may be set, and aggregation may be performed after a certain period of time has elapsed or after receiving a certain number of encrypted feature 325 or more.

[0095] When the second-layer feature aggregation server 700 receives information on encrypted aggregated features 601 from multiple first-layer feature aggregation servers 600, it aggregates each received encrypted aggregated feature 601 to generate an encrypted aggregated feature 701 (S1002), decrypts the generated encrypted aggregated feature 701 using the primary secret key 702 to generate a primary decrypted encrypted feature 703 (S710), and transmits the information of the generated primary decrypted encrypted feature 703 to the learning model update server 200 via the communication network 500 (S711).

[0096] Here, the second-layer feature aggregation server 700 does not possess the secondary secret key 220 generated by the learning model update server 200, and therefore cannot completely decrypt the encrypted aggregated feature 701.

[0097] When the learning model update server 200 receives information on the primary decrypted encrypted aggregate feature 703, it uses the secondary secret key 220 to decrypt the primary decrypted encrypted aggregate feature 703 and generates an aggregate feature 221 (S712).

[0098] Next, the learning model update server 200 updates the learning model 223 using the aggregated features 221 generated in step S712 (S713). When updating the learning model 223, possible methods include adding the aggregated features to the learning model, or dividing the aggregated features by the number of transmitting devices and then adding them.

[0099] Subsequently, the learning model update server 200 transmits the updated learning model 223 information to the data receiving device 400 via the communication network 500 (S714).

[0100] If steps S701 to S705 are designated as the initial phase P701 and steps S706 to S714 as the repeating phase P702, then after completing the processing of the initial phase P701, the processing of the repeating phase P702 may be repeated. That is, after step S714, the same procedure may be repeated starting from step S706. Also, after step S713, the processing of step S714 may be omitted, and the same procedure may be repeated starting from step S706.

[0101] In this embodiment, the feature aggregation servers are arranged in two layers, but it is also possible to arrange the feature aggregation servers in three, four, or other multi-layer configurations. Furthermore, in this embodiment, key generation is performed by the second-layer feature aggregation server 700 and the learning model update server 200, but any one of the first-layer feature aggregation server 600, the second-layer feature aggregation server 700, or the learning model update server 200 may perform key generation. In this case, the feature aggregation server 600 or 700, or the learning model update server 200 that performs key generation, holds the generated secret key and performs decryption processing using the held secret key.

[0102] In this embodiment, we have described a configuration in which the second-layer feature aggregation server 700 and the learning model update server 200 each possess two secret keys 702 and 220 for decrypting the encrypted feature 325. However, the first-layer feature aggregation server 600 may generate a tertiary secret key, and the data transmission device 300 may further encrypt it with the tertiary public key. Furthermore, if the feature aggregation servers are multi-layered, the number of public and private key pairs may be increased by the number of layers, or a small number of secret keys may be shared and used by the feature aggregation servers.

[0103] Furthermore, in this embodiment, key generation can be performed by any feature aggregation server and the learning model update server 200. In this case, there may be feature aggregation servers that do not perform key generation. In that case, if a feature aggregation server that does not perform key generation encrypts the encrypted aggregated features, it will receive the secret key information from the feature aggregation server that performs key generation.

[0104] This embodiment can achieve the same effects as in Embodiment 1. Furthermore, in this embodiment, the feature aggregation servers 600 and 700 are arranged in a multi-layered configuration, and the aggregation process is distributed to multiple feature aggregation servers 600 and 700, thereby reducing the computational and communication load per feature aggregation server. Moreover, this embodiment makes it possible to apply the federated learning system 20 to a hierarchical organizational structure. In addition, in this embodiment, since each data transmission device 300 encrypts the feature quantities 322 using additive homomorphic encryption, each first-layer feature aggregation server 600 can aggregate the encrypted feature quantities 325 from each data transmission device 300 at any time. In other words, each first-layer feature aggregation server 600 can aggregate the encrypted features 325 to generate an encrypted aggregate feature 601 under its authority, and the second-layer feature aggregation server 700 can aggregate the encrypted aggregate feature 601 to generate a primary decrypted encrypted aggregate feature 703 under its authority, and then send the generated primary decrypted encrypted aggregate feature 703 to the learning model update server 200. As a result, for example, even if communication with one of the data transmission devices 300 is interrupted, the federated learning system 10 can continue federated learning by sending the primary decrypted encrypted aggregate feature 703, which is generated by aggregating the encrypted features 325 that were successfully received from other data transmission devices 300, to the learning model update server 200.

[0105] It should be noted that the present invention is not limited to the embodiments described above, and various modifications are included. For example, the embodiments described above are described in detail to make the present invention easier to understand, and are not necessarily limited to those having all the configurations described. It is also possible to replace parts of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add configurations from other embodiments to the configuration of one embodiment. Furthermore, it is possible to add, delete, or replace parts of the configuration of each embodiment with other configurations.

[0106] Furthermore, each of the above configurations and functional means may be implemented in hardware, either partially or entirely, by designing them as an integrated circuit, for example. Alternatively, each of the above configurations and functions may be implemented in software by having the processor interpret and execute a program that implements each function. Information such as programs, tables, and files that implement each function can be stored in memory, a recording device such as a hard disk or SSD (Solid State Drive), or a recording medium such as an IC card, SD card, or DVD.

[0107] Furthermore, the control lines and information lines shown are those deemed necessary for explanatory purposes, and not all control lines and information lines are necessarily shown in the actual product. In reality, it is safe to assume that almost all components are interconnected. [Explanation of Symbols]

[0108] 100 Feature aggregation server, 200 Learning model update server, 300 Data transmission device, 400 Data reception device, 101 Processor, 102 Auxiliary storage device, 103 Memory, 107 Communication interface, 111 Key generation unit, 112 Feature aggregation unit, 113 Decryption processing unit, 201 Processor, 202 Auxiliary storage device, 203 Memory, 207 Communication interface, 211 Key generation unit, 212 Learning model update unit, 213 Decryption processing unit, 301 Processor, 302 Auxiliary storage device, 303 Memory, 307 Communication interface, 311 Feature calculation unit, 312 Cryptography processing unit, 401 Processor, 402 Auxiliary storage device, 403 Memory, 407 Communication interface, 600 1st layer feature aggregation server, 700 2nd layer feature aggregation server

Claims

1. One or more data transmission devices that calculate multiple features from a learning model and each of multiple training data, and encrypt each of the calculated multiple features in multiple ways to generate multiple encrypted features as transmission information, A feature aggregation server inputs the transmission information generated by the data transmission device via a communication network, and aggregates each of the multiple encrypted feature quantities belonging to the input transmission information to generate an encrypted aggregated feature quantity. The system comprises a feature aggregation server and a learning model update server that transmits and receives information via the communication network and updates the learning model based on the information received from the feature aggregation server, The aforementioned feature aggregation server, The first of several decryption processes that decrypt the aforementioned encrypted aggregated features in multiple stages is executed. The aforementioned learning model update server is: The last of the aforementioned multiple decoding processes is executed. The aforementioned feature aggregation server, A federated learning system characterized by sequentially inputting the encrypted feature quantities from two or more of the data transmission devices, monitoring a first timing when the input encrypted feature quantities reach a set number, and a second timing when a set time has elapsed from the input start time for inputting the encrypted feature quantities from the data transmission devices, and aggregating the input encrypted feature quantities at least at the first timing or the second timing.

2. In the associative learning system described in claim 1, The aforementioned feature aggregation server, The aforementioned encrypted aggregate feature quantity is decrypted once to generate a first-decrypted encrypted aggregate feature quantity. The aforementioned learning model update server is: A federated learning system characterized by generating aggregated features by secondary decrypting the primary decrypted encrypted aggregated features generated by the feature aggregation server, and updating the learning model with the generated aggregated features.

3. In the associative learning system described in claim 2, The aforementioned feature aggregation server, Based on information regarding the encryption method used for encryption in the data transmission device, primary public key and primary private key information are generated, the generated primary public key information is transmitted to the data transmission device via the communication network, and the generated primary private key is used to decrypt the encrypted aggregate feature quantity. The aforementioned learning model update server is: A federated learning system characterized by generating information on a secondary public key and a secondary private key based on information on the encryption method used for encryption in the data transmission device, transmitting the generated secondary public key information to the data transmission device via the communication network, and decrypting the primary decrypted encrypted aggregate feature quantity using the generated secondary private key.

4. In the associative learning system described in claim 3, The aforementioned data transmission device is A federated learning system characterized by single-encrypting the features with the primary public key received from the feature aggregation server and double-encrypting the features with the secondary public key received from the learning model update server.

5. In the associative learning system described in claim 1, The aforementioned data transmission device is A federated learning system characterized by performing a first encryption process that generates an encrypted feature by encrypting each of the aforementioned multiple feature quantities multiple times with multiple public keys, or a second encryption process that generates an added public key by adding the aforementioned multiple public keys, and then encrypts each of the aforementioned multiple feature quantities with the generated added public key to generate the encrypted feature.

6. In the associative learning system described in claim 3, The aforementioned feature aggregation server, Multiple first-layer feature aggregation servers that receive the encrypted feature quantities from two or more of the data transmission devices and aggregate the received encrypted feature quantities to generate first-layer encrypted aggregate feature quantities, A federated learning system characterized by including a second-layer feature aggregation server that receives the first-layer encrypted aggregate features generated by each of the plurality of first-layer feature aggregation servers as input, and aggregates the input first-layer encrypted aggregate features to generate the encrypted aggregate features.

7. In the associative learning system described in claim 6, The aforementioned second-layer feature aggregation server is: The encrypted aggregate feature quantity is first decrypted using the primary secret key to generate the first decrypted encrypted aggregate feature quantity. The aforementioned learning model update server is: A federated learning system characterized in that, upon receiving information of the primary decrypted encrypted aggregated features from the second-layer feature aggregation server, the received primary decrypted encrypted aggregated features are secondary decrypted using the secondary secret key to generate the aggregated features.

8. A method in a federated learning system comprising one or more data transmission devices, a feature aggregation server, and a learning model update server, The data transmission device performs a transmission information generation step in which it calculates each of several features from the learning model and each of several learning data, encrypts each of the calculated several features multiple times, and generates several encrypted features as transmission information. The feature aggregation server inputs the transmission information generated by the data transmission device via the communication network, and aggregates each of the plurality of encrypted features belonging to the input transmission information to generate an encrypted aggregate feature, in an encrypted aggregate feature generation step, The learning model update step involves the learning model update server sending and receiving information with the feature aggregation server via the communication network, and updating the learning model based on the information received from the feature aggregation server. The feature aggregation server performs a first decryption step in which it executes the first of a plurality of decryption processes that decrypt the encrypted aggregated feature in multiple stages, The learning model update server includes a second decoding step in which it performs the last decoding process among the plurality of decoding processes, The aforementioned feature aggregation server, In the above-mentioned cryptographic aggregate feature generation step, A federated learning method characterized by sequentially inputting the encrypted feature quantities from two or more data transmission devices, monitoring a first timing when the input amount of encrypted feature quantities reaches a set number, and a second timing when a set time has elapsed from the input start time for inputting the encrypted feature quantities from the data transmission devices, and aggregating the input encrypted feature quantities at least at the first timing or the second timing.

9. In the associative learning method described in claim 8, The aforementioned feature aggregation server, In the first decryption step described above, the encrypted aggregate feature is first decrypted to generate a first decrypted encrypted aggregate feature, The aforementioned learning model update server is: The federated learning method is characterized in that, in the second decryption step, the primary decrypted encrypted aggregate features generated by the feature aggregation server are secondary decrypted to generate aggregate features, and the learning model is updated with the generated aggregate features.

10. In the associative learning method described in claim 9, The aforementioned feature aggregation server, In the first decryption step, information on a primary public key and a primary private key are generated based on information on the encryption method used for encryption in the data transmission device, the generated primary public key information is transmitted to the data transmission device via the communication network, and the encrypted aggregate feature quantity is decrypted using the generated primary private key. The aforementioned learning model update server is: The federated learning method is characterized in that, in the second decryption step, information on a secondary public key and information on a secondary private key are generated based on information on the encryption method used for encryption of the data transmission device, the generated secondary public key information is transmitted to the data transmission device via the communication network, and the primary decrypted encrypted aggregate feature quantity is decrypted secondary using the generated secondary private key.

11. In the associative learning method described in claim 10, The aforementioned data transmission device is The federated learning method is characterized in that, in the transmission information generation step, each of the multiple features is single-encrypted with the primary public key received from the feature aggregation server, and each of the multiple features is double-encrypted with the secondary public key received from the learning model update server.

12. In the associative learning method described in claim 8, The aforementioned data transmission device is The federated learning method is characterized in that, in the transmission information generation step, a first encryption process is performed to generate an encrypted feature by encrypting each of the multiple feature quantities multiple times with multiple public keys, or a second encryption process is performed to generate an added public key by adding the multiple public keys, and then encrypt each of the multiple feature quantities with the generated added public key to generate the encrypted feature.

13. In the associative learning method described in claim 10, The aforementioned feature aggregation server, Multiple first-layer feature aggregation servers that receive the encrypted feature quantities from two or more data transmission devices in the first decryption step and aggregate the received encrypted feature quantities to generate first-layer encrypted aggregate feature quantities, A federated learning method characterized by including a second-layer feature aggregation server that, in the first decryption step, receives the first-layer encrypted aggregate features generated by each of the plurality of first-layer feature aggregation servers, and aggregates the input first-layer encrypted aggregate features to generate the encrypted aggregate features.

14. In the associative learning method described in claim 13, The aforementioned second-layer feature aggregation server is: In the first decryption step described above, the encrypted aggregate feature is first decrypted with the first secret key to generate the first decrypted encrypted aggregate feature, The aforementioned learning model update server is: The federated learning method is characterized in that, in the second decryption step, when information of the primary decrypted encrypted aggregate feature is received from the second layer feature aggregation server, the received primary decrypted encrypted aggregate feature is secondary decrypted with the secondary secret key to generate the aggregate feature.

Images