Associative Learning Systems and Associative Learning Methods

Abstract

To provide an association learning system and an association learning method, which update a learning model by a purpose-oriented aggregated feature amount aggregated according to a purpose for updating the learning model.SOLUTION: An association learning system includes: a plurality of data transmission devices connected to a communication network; one or a plurality of data reception devices connected to a feature amount aggregation server via the communication network; and the feature amount aggregation server for transmitting / receiving information to / from each device via the communication network. Each data transmission device generates a tagged feature amount indicating a feature amount according to a first query on the basis of a learning model and each of a plurality of learning data under a condition of input of the first query including a data extraction requirement to transmit transmission information including the tagged feature amount to the feature amount aggregation server. The feature amount aggregation server aggregates the tagged feature amount belonging to the transmission information to generate a purpose-oriented aggregated feature amount to transmit it to the data reception device. The data reception device updates the learning model on the basis of the purpose-oriented aggregated feature amount.SELECTED DRAWING: Figure 6

Description

Technical Field

[0001] The present invention relates to a federated learning system and a federated learning method.

Background Art

[0002] The spread of AI (Artificial Intelligence) is progressing in various fields such as finance, healthcare, and manufacturing. Among them, in order to develop more accurate AI, there is a movement for multiple organizations to cooperate. However, directly sharing the data held by each organization is not desirable because the risk of information leakage increases. Therefore, federated learning has been proposed as a method for developing a common AI without directly sharing the data held by each organization.

[0003] Federated learning is an information processing method executed in a system including a plurality of data transmission devices and a central server. Each data transmission device calculates feature quantities necessary for updating the learning model using the common learning model distributed from the central server and the data it holds. Then, the feature quantities are sent from each data transmission device to the central server, and the central server aggregates the feature quantities received from each data transmission device and updates the learning model. In a system adopting federated learning, the data held by multiple organizations can be used, so the development of high-precision AI is expected. However, due to differences in the data distributions of each organization, phenomena such as the performance of AI not improving or the AI not being suitable for a certain organization may occur.

[0004] As a countermeasure to the above problems caused by differences in data distribution, a method called Personarized Federated Learning has been developed. In Non-Patent Document 1, three methods are proposed: user clustering in which data transmission devices are clustered and FL is performed for each cluster, data complementation in which a data transmission device calculates feature quantities using its own held data and the held data of other data transmission devices at an appropriate ratio, and model complementation in which a data transmission device updates the model using its own calculated feature quantities and the feature quantities calculated by other data transmission devices at an appropriate ratio. [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] Yishay Mansour, et al., “Three Approaches for Personalization with Applications to Federated Learning”, [online], February 25, 2020, Internet, < https: / / arxiv.org / pdf / 2002.10619v1.pdf > [Overview of the project] [Problems that the invention aims to solve]

[0006] In the federated learning described in Non-Patent Literature 1, learning is performed using a fixed ratio of data and features from other data transmitting devices. As a result, each data transmitting device cannot calculate features specific to its purpose, making it difficult to develop AI specialized for specific purposes. For example, when each data transmitting device learns using a learning model, it cannot meet the requirement of learning data with a ratio of 3:7 between data for those under 30 and data for those 30 and over, and therefore cannot calculate age-specific features.

[0007] Therefore, the objective of the present invention is to update the learning model with purpose-specific aggregated features that are aggregated in accordance with the purpose of updating the learning model. [Means for solving the problem]

[0008] To solve the above problems, the present invention provides a plurality of data transmission devices connected to a communication network, and the plurality of data transmitted via the communication network. sendThe system comprises a feature aggregation server that transmits and receives information with each of the devices, and one or more data receiving devices connected to the feature aggregation server via the communication network, wherein each data transmission device generates tagged features that indicate features according to the first query based on a learning model and each of a plurality of learning data, subject to input of a first query including data extraction requirements, and transmits transmission information including the generated tagged features to the feature aggregation server, and when the feature aggregation server receives the transmission information from each of the data transmission devices, it aggregates the tagged features belonging to the received transmission information to generate purpose-specific aggregated features, and transmits the generated purpose-specific aggregated features to the data receiving device, and when the data receiving device receives the purpose-specific aggregated features from the feature aggregation server, it updates the learning model based on the received purpose-specific aggregated features. [Effects of the Invention]

[0009] According to the present invention, a learning model can be updated with purpose-specific aggregated features that are aggregated in accordance with the purpose of updating the learning model.

[0010] Other issues, configurations, and effects not mentioned above will be clarified by the following description of the embodiments. [Brief explanation of the drawing]

[0011] [Figure 1] This is a block diagram showing an example configuration of a federative learning system according to Embodiment 1 of the present invention. [Figure 2] This is a block diagram showing an example of the hardware configuration of a feature aggregation server according to Embodiment 1 of the present invention. [Figure 3] This is a block diagram showing an example of the hardware configuration of a learning model update server according to Embodiment 1 of the present invention. [Figure 4] This is a block diagram showing an example of the hardware configuration of a data transmission device according to Embodiment 1 of the present invention. [Figure 5] This is a block diagram showing an example of the hardware configuration of a data receiving device according to Embodiment 1 of the present invention. [Figure 6] This sequence diagram shows an example of the processing of the federative learning system according to Embodiment 1 of the present invention. [Figure 7] This sequence diagram shows an example of the processing of the federative learning system according to Embodiment 2 of the present invention. [Figure 8] This sequence diagram shows an example of the processing of the federative learning system according to Embodiment 3 of the present invention. [Figure 9] This sequence diagram shows an example of the processing of the federative learning system according to Embodiment 4 of the present invention. [Modes for carrying out the invention]

[0012] Embodiments of the present invention will be described below with reference to the attached drawings. In these embodiments, the same components are generally denoted by the same reference numerals, and repeated descriptions are omitted. It should be noted that these embodiments are merely one example of how to realize the present invention and do not limit the technical scope of the present invention. In these embodiments, a federated learning system that enables AI development in line with the purpose of a data receiving device will be described. [Examples]

[0013] This embodiment provides a federated learning system with a common AI learning phase and a purpose-specific AI learning phase. In the common AI learning phase, the following processes are sequentially repeated: a feature calculation process that calculates multiple features using multiple training data and a learning model; an aggregate feature generation process that aggregates the multiple features calculated in the feature calculation process to generate aggregate features; and a learning model update process that updates the learning model using the aggregate features generated in the aggregate feature generation process. In the purpose-specific AI learning phase, a tagged feature calculation process is executed based on the input of a query, using multiple training data and a learning model to calculate multiple tagged features that indicate features according to the query; a purpose-specific aggregate feature generation process that aggregates the multiple tagged features calculated in the tagged feature calculation process to generate purpose-specific aggregate features; and a purpose-specific learning model update process that updates the learning model using the purpose-specific aggregate features generated in the purpose-specific aggregate feature generation process. The specific details of the federated learning system will be described below. Note that, due to the layout of the drawings, the configuration of other embodiments may be described in Embodiment 1.

[0014] Figure 1 is a block diagram showing an example configuration of a federated learning system according to Embodiment 1 of the present invention. In Figure 1, the federated learning system 10 is a system comprising a common AI learning phase and a purpose-specific AI learning phase, and includes a feature aggregation server 100, a learning model update server 200, a data transmission device 300, a data receiving device 400, and a communication network 500. The feature aggregation server 100, the learning model update server 200, the data transmission device 300, and the data receiving device 400 send and receive information to each other via a communication network 500 such as the Internet. Note that the data transmission device 300 and the data receiving device 400 may be configured in multiple units.

[0015] The data transmission device 300 holds, for example, a learning model to be developed, and also holds a plurality of learning data (data for learning) obtained by learning using the learning model. In the common AI learning phase, the data transmission device 300 calculates a plurality of feature amounts using the plurality of learning data and the learning model, and generates each feature amount as transmission information. Further, in the AI learning phase for each purpose, the data transmission device 300 calculates a plurality of tagged feature amounts indicating feature amounts according to a query using the plurality of learning data and the learning model on the condition that a query is input, and generates each tagged feature amount as transmission information. As the learning data, for example, image data such as photographs of animals and X-ray photographs, log data including stock trading data, etc. are used. As the learning model, for example, a Deep Neural Network used in DeepLearning, a random forest, etc. are used. As the feature amount, when the learning model is a Deep Neural Network, real values corresponding to the number of parameters of the Deep Neural Network are calculated.

[0016] In the common AI learning phase, when the feature amount aggregation server 100 receives a plurality of feature amounts generated by the data transmission device 300, the feature amount aggregation server 100 aggregates each of the received plurality of feature amounts to generate an aggregated feature amount, and transmits the generated aggregated feature amount to the learning model update server 200. Further, in the AI learning phase for each purpose, when the feature amount aggregation server 100 receives a plurality of tagged feature amounts generated by the data transmission device 300, the feature amount aggregation server 100 aggregates each of the received plurality of tagged feature amounts to generate a plurality of purpose-specific aggregated feature amounts, and transmits, to the data reception device 400, the purpose-specific aggregated feature amount suitable for the query received from the data reception device 400 among the generated plurality of purpose-specific aggregated feature amounts.

[0017] In the common AI learning phase, the learning model update server 200 updates the learning model based on the aggregated feature amount received from the feature amount aggregation server 100, and transmits the updated learning model to the data reception device 400. Further, in the AI learning phase for each purpose, the learning model update server 200 transmits the learning model to the data transmission device 300.

[0018] In the common AI learning phase, the data receiving device 400 receives the updated learning model from the learning model update server 200. In the AI learning phase for each purpose, the data receiving device 400 sends a query to the learning model update server 200, receives the purpose-specific aggregated feature amounts corresponding to the query from the feature amount aggregation server 100, and updates the learning model based on the received purpose-specific aggregated feature amounts.

[0019] Details of the feature amount aggregation server 100, the learning model update server 200, the data transmission device 300, and the data receiving device 400 will be described later. Also, some or all of the devices included in the federated learning system 10 may be integrated. For example, the feature amount aggregation server 100 and the learning model update server 200 may be integrated, or the data transmission device 300 and the data receiving device 400 may be integrated.

[0020] FIG. 2 is a block diagram showing a hardware configuration example of the feature amount aggregation server according to Embodiment 1 of the present invention. In FIG. 2, the feature amount aggregation server 100 is constituted by, for example, a computer having a processor (CPU) 101, an auxiliary storage device 102, a memory 103, a display device 105, an input / output interface 106, and a communication interface 107, and each device is connected to each other via an internal signal line 104 such as a bus. Note that FIG. 2 shows the hardware and software resources used in Embodiments 1 to 4.

[0021] The processor 101 is a processing device that executes the program stored in the memory 103 and comprehensively controls each device. The memory 103 includes a non-volatile non-temporary storage element, a ROM (Read Only Memory), and a volatile storage element, a RAM (Random Access Memory). The ROM stores invariant programs (such as BIOS). The RAM is a high-speed and volatile storage element such as a DRAM (Dynamic Random Access Memory), and temporarily stores the program executed by the processor 101 and the data used during the execution of the program.

[0022] The auxiliary storage device 102 is a large-capacity, non-volatile, non-temporary storage device such as a magnetic storage device (HDD) or flash memory (SSD), and stores the program executed by the processor 101 and the data used when the program is executed. In other words, the program is read from the auxiliary storage device 102, loaded into memory 103, and then executed by the processor 101.

[0023] The input / output interface (I / F) 106 is connected to input devices such as a keyboard and mouse, and is an interface for inputting information from the operator's operations. The input / output interface 106 is also connected to output devices such as a display device 105 and a printer, and is an interface for outputting the program execution results in a format that the operator can view. The display device 105 displays the program execution results output from the input / output interface 106.

[0024] The communication interface (I / F) 107 is a network interface that controls communication with other devices via the communication network 500 according to a predetermined protocol. The communication interface 107 may also include a serial interface, such as USB (Universal Serial Bus).

[0025] Some or all of the program executed by the processor 101 may be provided to the feature aggregation server 100 via the communication network 500 from a removable media (such as a CD-ROM or flash memory) or an external computer equipped with a non-temporary storage device, and stored in a non-volatile auxiliary storage device 102, which is also a non-temporary storage device. For this reason, the feature aggregation server 100 can be configured to have an interface for reading data from the removable media. The same applies to the learning model update server 200, the data transmission device 300, and the data receiving device 400.

[0026] The feature aggregation server 100 is a computer system that operates on a single physical computer or on multiple logically or physically configured computers. It may operate on the same computer in separate threads, or on a virtual computer built on multiple physical computing resources. The same applies to the learning model update server 200, the data transmission device 300, and the data reception device 400.

[0027] The processor 101 includes, for example, a key generation unit 111, a feature aggregation unit 112, a decoding processing unit 113, and a key replacement unit 114 as software resources. For example, the processor 101 functions as a key generation unit 111 by operating according to a key generation program loaded into memory 103, functions as a feature aggregation unit 112 by operating according to a feature aggregation program loaded into memory 103, functions as a decoding processing unit 113 by operating according to a decoding processing program loaded into memory 103, and functions as a key replacement unit 114 by operating according to a key replacement program loaded into memory 103.

[0028] The key generation unit 111, for example, in the initial phase, generates a primary public key for encrypting the feature quantity and a primary secret key for decrypting the feature quantity, based on information about the encryption method used for encryption by the data transmission device 300. In this case, the key generation unit 111 uses, for example, additive homomorphic encryption as the encryption technique. Additive homomorphic encryption is a technique that allows ciphertexts to be added together while remaining encrypted.

[0029] The following explains additive homomorphism using mathematical formulas. Let M1 and M2 be arbitrary values, the encryption key be pk, and the encryption function be Enc. For example, Enc(M,pk) refers to the ciphertext obtained by encrypting the value M with the encryption key pk. Furthermore, let * be the operation for performing addition while the values ​​are encrypted. In this case, due to the additive homomorphism, the following equation (1) holds. Enc(M1,pk)*Enc(M2,pk)=Enc(M1+M2,pk)...(1)

[0030] From equation (1) above, a ciphertext of the sum M1+M2 can be generated from the ciphertexts of M1 and M2. Furthermore, among additive homomorphic encryption schemes, there are encryption schemes that allow for the addition of public keys. If pk1 and pk2 are the public keys, M is an arbitrary value, and * is the operation for public key addition, the addition of public keys can be expressed mathematically as shown in equation (2) below. Enc(M,pk1)*Enc(M,pk2)=Enc(M,pk1+pk2)···(2)

[0031] By utilizing the additivity of public keys, the computational cost of cryptographic processing can be reduced. Here, as a cryptographic technique to be applied to the federated learning system 10, for example, the application of Elliptic ElGamal cryptography, which is an additive homomorphic encryption scheme having the above-mentioned additivity of public keys, can be considered.

[0032] The feature aggregation unit 112 generates aggregated features or encrypted features by performing addition or averaging as aggregation on features or encrypted features received from the data transmission device 300, for example, during the common AI learning phase. Furthermore, the aggregation unit 112 generates purpose-specific aggregated features or encrypted purpose-specific aggregated features by performing addition or averaging as aggregation on features, tagged features, or encrypted tagged features received from the data transmission device 300, for example, during the purpose-specific AI learning phase.

[0033] The decryption processing unit 113, for example, in the common AI learning phase, uses the primary secret key generated by the key generation unit 111 to decrypt (primarily decrypt) the encrypted aggregated features generated by the feature aggregation unit 112 to generate primary decrypted encrypted aggregated features.

[0034] The key replacement unit 114, for example, in the purpose-specific AI learning phase, generates key-replaced encrypted purpose-specific aggregate features that can be decrypted with the individual secret key held by the data receiving device 400, from the key replacement key received from the learning model update server 200 and the calculated encrypted purpose-specific aggregate features.

[0035] The key change in the key change unit 114 will be explained below using mathematical formulas. Let M1 be an arbitrary value, the encryption key be pk, the key change key be rk, the encryption function be Enc, and the key change function be ReEnc. For example, ReEnc(M,pk) refers to the ciphertext obtained by encrypting the value M with the encryption key pk, and (M,rk) refers to the ciphertext obtained by re-encrypting the value M with the key change key be rk. If the decryption key for the re-ciphertext obtained by re-encrypting pk with rk is rsk and the decryption function is Dec, then the following equation holds. Dec(ReEnc(Enc(M,pk),rk),rsk)=M···(3)

[0036] In other words, a ciphertext encrypted with pk and then re-encrypted with rk can be decrypted with the secret key rsk. However, if the secret key corresponding to pk is sk, then the re-ciphertext cannot be decrypted with sk.

[0037] Figure 3 is a block diagram illustrating an example of the hardware configuration of a learning model update server according to Embodiment 1 of the present invention. In Figure 3, the learning model update server 200 is composed of a computer having, for example, a processor (CPU) 201, an auxiliary storage device 202, a memory 203, a display device 205, an input / output interface 206, and a communication interface 207, and each device is connected to the others via an internal signal line 204 such as a bus. Note that Figure 3 illustrates the hardware and software resources used in Embodiments 1 to 4 for illustrative purposes.

[0038] The hardware configurations of the processor 201, auxiliary storage device 202, memory 203, internal signal lines 204, display device 205, input / output interface 206, and communication interface 207 are the same as those of the processor 101, auxiliary storage device 102, memory 103, internal signal lines 104, display device 105, input / output interface 106, and communication interface 107, so their descriptions are omitted.

[0039] The processor 201 includes a key generation unit 211, a learning model update unit 212, and a decryption processing unit 213 as software resources. In this case, for example, the processor 201 functions as a key generation unit 211 by operating according to a key generation program loaded into memory 203, functions as a learning model update unit 212 by operating according to a learning model update program loaded into memory 203, and functions as a decryption processing unit 213 by operating according to a decryption processing program loaded into memory 103.

[0040] The key generation unit 211 generates, for example, information on the encryption method used for encryption by the data transmission device 300, in an initial phase before the common AI learning phase, a secondary public key for encrypting the feature quantities and information on a secondary secret key for decrypting the feature quantities. Furthermore, in an initial phase before the purpose-specific AI learning phase, the key generation unit 211 generates information on a secondary public key for encrypting the feature quantities and information on an individual secret key for transmission to the data receiving device 400, and also generates information on a key replacement key for use by the feature quantity aggregation server 100 for key replacement. In this case, the key generation unit 211 uses, for example, additive homomorphic encryption as the encryption technology.

[0041] For example, in the common AI learning phase, when the decryption processing unit 213 receives primary decrypted encrypted aggregated features from the feature aggregation server 100, it decrypts (decrypts) the received primary decrypted encrypted aggregated features using the secondary secret key generated by the key generation unit 211 to generate aggregated features.

[0042] The learning model update unit 212, for example, in the common AI learning phase, updates the learning model 320 using the aggregated features generated by the decoding processing unit 213 and the learning model 320, for example, by addition or averaging.

[0043] Figure 4 is a block diagram showing an example of the hardware configuration of a data transmission device according to Embodiment 1 of the present invention. In Figure 4, the data transmission device 300 is composed of a computer having, for example, a processor (CPU) 301, an auxiliary storage device 302, a memory 303, a display device 305, an input / output interface 306, and a communication interface 307, and each device is connected to the others via an internal signal line 304 such as a bus. Note that Figure 4 illustrates the hardware and software resources used in Embodiments 1 to 4 for illustrative purposes.

[0044] The hardware configurations of the processor 301, auxiliary storage device 302, memory 303, internal signal lines 304, display device 305, input / output interface 306, and communication interface 307 are the same as those of the processor 101, auxiliary storage device 102, memory 103, internal signal lines 104, display device 105, input / output interface 106, and communication interface 107, so their descriptions are omitted.

[0045] The processor 301 includes, for example, a feature calculation unit 311 and an encryption processing unit 312 as software resources. In this case, for example, the processor 301 functions as a feature calculation unit 311 by operating according to a feature calculation program loaded into memory 303, and functions as an encryption processing unit 312 by operating according to an encryption processing program loaded into memory 303.

[0046] The auxiliary storage device 302 of the data transmission device 300 holds the training data 321 and the training model 320 obtained by the processor 301 using the training model 320 under development. The training model 320 and training data 321 may also be stored in the memory 303 of the data transmission device 300.

[0047] The feature calculation unit 311 calculates multiple features using multiple training data 321 and the training model 320 during the common AI learning phase, and generates each feature as transmission information. The data transmission device 300, during the purpose-specific AI learning phase, calculates multiple tagged features indicating features according to the query using multiple training data 321 and the training model 320, based on the input of a query, and generates each tagged feature as transmission information.

[0048] The cryptographic processing unit 312, for example, in the common AI learning phase, encrypts the features using the primary public key generated by the key generation unit 111 of the feature aggregation server 100 and the secondary public key generated by the key generation unit 211 of the learning model update server 200 to generate encrypted features. In addition, in the purpose-specific AI learning phase, the cryptographic processing unit 312 encrypts the tagged features using the secondary public key generated by the key generation unit 211 of the learning model update server 200 to generate encrypted tagged features. At this time, the data transmission device 300 can input information about the learning model 320, the primary public key 324, and the secondary public key 325 by sending and receiving information with the feature aggregation server 100 and the learning model update server 200.

[0049] Furthermore, if the data transmission device 300 is configured with multiple units, the processing of the multiple data transmission devices 300 generates multiple feature quantities or tagged feature quantities from the learning model 320 under development and multiple training data 321, and multiple encrypted feature quantities or encrypted tagged feature quantities are generated from each of the generated feature quantities or tagged feature quantities. In addition, if the data transmission device 300 is configured with multiple information processing systems (encryption processing systems), the feature calculation unit 311 and the encryption processing unit 312 constitute multiple information processing systems (encryption processing systems).

[0050] Figure 5 is a block diagram showing an example of the hardware configuration of a data receiving device according to Embodiment 1 of the present invention. In Figure 5, the data receiving device 400 is composed of a computer having, for example, a processor (CPU) 401, an auxiliary storage device 402, a memory 403, a display device 405, an input / output interface 406, and a communication interface 407, and each device is connected to the others via an internal signal line 404 such as a bus. Note that Figure 5 illustrates the hardware and software resources used in Embodiments 1 to 4 for illustrative purposes.

[0051] The hardware configurations of the processor 401, auxiliary storage device 402, memory 403, internal signal lines 404, display device 405, input / output interface 406, and communication interface 407 are the same as those of the processor 101, auxiliary storage device 102, memory 103, internal signal lines 104, display device 105, input / output interface 106, and communication interface 107, so their descriptions are omitted.

[0052] The processor 401 includes, for example, a learning model update unit 411 and a decoding processing unit 412 as software resources. In this case, for example, the processor 401 functions as a learning model update unit 411 by operating according to a learning model update program loaded into memory 403, and functions as a decoding processing unit 412 by operating according to a decoding processing program loaded into memory 403.

[0053] Furthermore, when the communication interface 407 receives information about the updated learning model 320 from the learning model update server 200, the processor 401 stores the received information in the auxiliary storage device 402 and also displays it on the screen of the display device 405.

[0054] The decryption processing unit 412, for example, in the objective-specific AI learning phase, receives key-replaced encrypted objective-specific aggregated features from the feature aggregation server 100, decrypts (decrypts) the received key-replaced encrypted objective-specific aggregated features using an individual secret key to generate objective-specific aggregated features. The learning model update unit 411, for example, in the objective-specific AI learning phase, receives objective-specific aggregated features from the feature aggregation server 100, and updates the learning model 320 using the received objective-specific aggregated features and the learning model 320, for example by addition or averaging. Furthermore, in the objective-specific AI learning phase, if objective-specific aggregated features are generated by the decryption processing unit 412, the learning model update unit 411 updates the learning model 320 using the objective-specific aggregated features generated by the decryption processing unit 412 and the learning model 320, for example by addition or averaging.

[0055] Figure 6 is a sequence diagram showing an example of the processing of a federated learning system according to Embodiment 1 of the present invention. First, the processing of the common AI learning phase P601 will be explained. In Figure 6, the learning model update server 200 transmits information of the learning model 320 to be developed to each data transmission device 300 via the communication network 500 (S601). For example, machine learning models such as decision trees and Deep Learning can be used as the learning model 320 to be developed.

[0056] Each data transmission device 300 calculates features (common learning features) using its own training data 321 and the training model 320 received from the training model update server 200, provided there is no query input (S602). Here, each data transmission device 300 needs to determine the number of times it will perform local training, called the number of epochs, before calculating each feature. The number of epochs may be determined individually by each data transmission device 300, or it may be determined in advance by the feature aggregation server 100 or the training model update server 200. The method for calculating each feature may be, for example, linear regression or logistic regression in the case of a decision tree, or statistical gradient descent in the case of Deep Learning.

[0057] Each data transmission device 300 transmits the feature information generated in step S620 to the feature aggregation server 100 via the communication network 500 (S603).

[0058] When the feature aggregation server 100 receives feature information from each data transmission device 300, it aggregates the information from each feature to generate aggregated features (common learning aggregated features) (S604). Possible methods for aggregating the information from each feature include addition and averaging. Regarding the timing of aggregation, aggregation may be performed after receiving features from all data transmission devices 300, or a threshold related to time and features may be set, and aggregation may be performed after a certain period of time has elapsed or after receiving a certain number of features.

[0059] For example, the feature aggregation server 100 sequentially inputs features from each of the multiple data transmission devices 300, monitors a first timing (first time) when the number of input features reaches a set number, and a second timing (second time) when a set time has elapsed from the input start time when features are input from each of the multiple data transmission devices 300, and aggregates the input features at least at the first timing or the second timing. This allows the feature aggregation server 100 to start the process of aggregating the input features from the set first timing or the second timing.

[0060] After this, the feature aggregation server 100 transmits the aggregated feature information generated in step S604 to the learning model update server 200 via the communication network 500 (S605).

[0061] Next, the learning model update server 200 updates the learning model 320 using the aggregated features (common learning aggregated features) received in step S605 (S606). When updating the learning model 320, possible methods include adding the aggregated features to the learning model 320, or dividing the aggregated features by the number of data transmission devices 300 before adding them.

[0062] Subsequently, the learning model update server 200 transmits the updated learning model 320 information to the data receiving device 400 via the communication network 500 (S607).

[0063] The above steps S601 to S606 constitute the common AI learning phase 601. By repeating the process from steps S601 to S606 multiple times, the accuracy of the learning model 320 can be improved.

[0064] Next, we will explain the processing of the purpose-specific AI learning phase P602. In Figure 6, the learning model update server 200 transmits information about the learning model 320 under development to the data transmission device 300 via the communication network 500 (S601). For example, machine learning models such as decision trees and deep learning can be used as the learning model 320 under development.

[0065] The feature aggregation server 100 transmits information about queries (first queries) that include tags to be added to features, such as query information that defines data extraction requirements (extraction requirements such as gender and age), to each data transmission device 300 (S608).

[0066] Each data transmission device 300 calculates multiple tagged features (features that satisfy data extraction requirements such as gender and age) according to the query, using the training data 321 it possesses and the training model 320 received from the training model update server 200, based on the query information received in S608 (S609). For example, if the data extraction requirement of the query specifies that the ratio of data under 30 years old to data 30 years old and over 7, then the training data used will be data with a ratio of 3:7 between data under 30 years old and data 30 years old and over 7, and the features according to the query will be calculated based on this training data and the training model 320. In addition, each data transmission device 300 needs to determine the number of times it will perform local training, called the epoch count, before calculating each tagged feature. The epoch count may be determined individually by each data transmission device 300, or it may be determined in advance by the feature aggregation server 100 or the training model update server 200. Methods for calculating each tagged feature include, for example, linear regression and logistic regression in the case of decision trees, and statistical gradient descent in the case of deep learning.

[0067] Each data transmission device 300 transmits the tagged feature information generated in step S609 to the feature aggregation server 100 via the communication network 500 (S610).

[0068] When the feature aggregation server 100 receives information on multiple tagged features from each data transmission device 300, it aggregates the information on each tagged feature to generate multiple aggregated features for specific purposes (S611). As a method for aggregating the information on each tagged feature, for example, one could set requirements such as gender or age, and perform addition or averaging only on tagged features that satisfy these requirements. The timing of aggregation can be the same as in the common AI learning phase.

[0069] Next, when the learning model update server 200 receives a query (second query) from the data receiving device 400 (S612), it transmits information on the purpose-specific aggregate feature suitable for the received query from among the multiple purpose-specific aggregate features generated to the data receiving device 400 via the communication network 500 (S613). In this case, if the query (second query) specifies data aggregation requirements such as gender and age, purpose-specific aggregate features for gender and age are generated as purpose-specific aggregate features suitable for the query. Note that the first and second queries can be, for example, queries that are in line with the purpose of updating the learning model and specify data aggregation requirements such as the data distribution of the data to be used for learning (data to be used as learning data). It is also possible to make the content of the first query and the second query different. In this case, the content of the second query should have a narrower scope of requirements than the content of the first query. For example, the data extraction requirement for the first query could be data of all ages, and the data aggregation requirement for the second query could be data of a specific age group.

[0070] The data receiving device 400 updates the learning model 320 (the learning model 320 updated in the common AI learning phase P601) using the purpose-specific aggregated features received from the learning model update server 200 (S614). When updating the learning model 320, possible methods include adding the purpose-specific aggregated features to the learning model 320, or dividing the purpose-specific aggregated features by the number of data transmitting devices 300 before adding them.

[0071] Subsequently, the data receiving device 400 performs feedback processing to send information regarding the performance of the updated learning model 320, etc., to the learning model update server 200 and the feature aggregation server 100 via the network 500 (615).

[0072] Steps S601, S602, S608 to S615 constitute the purpose-specific AI learning phase P602.

[0073] According to this embodiment, the learning model can be updated with purpose-specific aggregated features that are aggregated according to the purpose of updating the learning model. In other words, according to this embodiment, the learning model can be updated in the common AI learning phase in the same way as in normal federated learning, and in the purpose-specific AI learning phase, an updated model can be generated in accordance with the query (second query) of the data receiving device 400 by collecting tagged features with the feature aggregation server 100, and as a result, it becomes possible to develop purpose-specific AI in accordance with the query of the data receiving device 400.

[0074] Furthermore, in this embodiment, during the purpose-specific AI learning phase, the feature aggregation server 100 creates a list of calculated purpose-specific aggregated features, and the data receiving device 400 sends a query for the necessary purpose-specific aggregated features from the list to the feature aggregation server 100, thereby obtaining the purpose-specific aggregated features that the data receiving device 400 queries. [Examples]

[0075] In this embodiment, during the purpose-specific AI learning phase, the feature aggregation server 100 sends a query (first query) to each data transmission device 300 upon receiving a query (second query) from the data receiving device 400. The hardware configuration and some of the software resources are the same as in Embodiment 1.

[0076] Figure 7 is a sequence diagram showing an example of the processing of the federated learning system according to Embodiment 2 of the present invention. The processing of the common AI learning phase P601 in this embodiment is the same as in Embodiment 1.

[0077] Next, we will explain the processing of the purpose-specific AI learning phase P701. In Figure 7, the learning model update server 200 transmits information about the learning model 320 under development to the data transmission device 300 via the communication network 500 (S601). For example, machine learning models such as decision trees and Deep Learning can be used as the learning model 320 under development.

[0078] Next, the data receiving device 400 sends information about a query (second query) to the feature aggregation server 100, for example, a query that specifies the data distribution of the data to be used for training as a data aggregation requirement (S701). Upon receiving the query (second query), the feature aggregation server 100 generates a query (first query) based on the received query, which includes tags to be added to the features, for example, a query that specifies the data distribution to be used for training as a data extraction requirement, and sends information about the generated query to the data transmission device 300 (S608). In this case, if the feature aggregation server 100 receives a query (second query) from the data receiving device 400, it can also generate a first query with different content from the second query. In this case, the content of the first query can be broadened to include a wider range of requirements than the content of the second query.

[0079] After this, the processes from S609 to S615 are executed as part of the purpose-specific AI learning phase P701. However, the process from steps S609 to S615 is the same as in Example 1, except that the process in step S612 is omitted, so the explanation of these steps will be omitted.

[0080] According to this embodiment, in the common AI learning phase, the learning model can be updated in the same way as in normal federative learning, and in the purpose-specific AI learning phase, by collecting tagged features with the feature aggregation server 100, an updated model can be generated that conforms to the query (second query) of the data receiving device 400. As a result, it becomes possible to develop purpose-specific AI that conforms to the query of the data receiving device 400.

[0081] Furthermore, according to this embodiment, in the purpose-specific AI learning phase, the feature aggregation server 100 obtains and aggregates tagged features from each data transmission device 300 corresponding to the query (second query) received from the data receiving device 400, which concerns the distribution of data to be used for learning. This enables the generation of an updated model that is more suitable for the query (second query) from the data receiving device 400, and as a result, it becomes possible to develop a purpose-specific AI that is more suitable for the query from the data receiving device 400. [Examples]

[0082] In this embodiment, a key generation process necessary for data encryption is performed as an initial phase before the common AI learning phase, and then, in the common AI learning phase, the features are encrypted using the key generated in the initial phase. In other words, this embodiment is a method that enhances resistance to data leakage attacks, such as attacks that attempt to reconstruct the learning data held by the data transmission terminal from the features, by encrypting the features. Note that the hardware configuration and some of the software resources are the same as in Embodiment 1.

[0083] Figure 8 is a sequence diagram showing an example of the processing of a federated learning system according to Embodiment 3 of the present invention. In Figure 8, the learning model update server 200 first generates a secondary public key as part of the initial phase P801 before common AI learning (S801), transmits the learning model 320 to be developed, the generated secondary public key, and cryptographic parameter information to each data transmission device 300 via the communication network 500 (S802), and further transmits the cryptographic parameter information to the feature aggregation server 100 via the communication network 500 (S803).

[0084] The feature aggregation server 100 generates a primary public key based on the received cryptographic parameter information (S804), and transmits the generated primary public key information to each data transmission device 300 via the communication network 500 (S805).

[0085] In the key generation process by the learning model update server 200 (S801) and the key generation process by the feature aggregation server 100 (S804), for example, Elgamal elliptic cryptography, a type of additive homomorphic encryption, is used. In this case, the encryption parameters transmitted in steps S802 and S803 include information such as that Elgamal elliptic cryptography is to be used, the elliptic curve to be used, and the base point. The setting of the encryption parameters may be performed by either the learning model update server 200 or the feature aggregation server 100. For example, machine learning models such as decision trees and Deep Learning may be used as the learning model 320 to be developed.

[0086] Next, as part of the processing in the common AI learning phase P802, each data transmission device 300 calculates features (common learning features) using the learning data 321 it possesses and the learning model 320 received from the learning model update server 200 (S602). Here, each data transmission device 300 needs to determine the number of times it will perform learning locally, called the number of epochs, before calculating each feature. The number of epochs may be determined individually by each data transmission device 300, or it may be determined in advance by the feature aggregation server 100 or the learning model update server 200. The method for calculating each feature may be, for example, linear regression or logistic regression in the case of a decision tree, or statistical gradient descent in the case of Deep Learning.

[0087] Next, each data transmission device 300 encrypts the feature vectors using the secondary public key received in step S802 and the primary public key received in step S805 to generate encrypted feature vectors (common learning encrypted feature vectors) (S806). When encrypting the feature vectors, for example, one could encrypt the feature vectors as many times as there are public keys, or one could add up the public keys first and then encrypt them only once using the calculated added public key.

[0088] Each data transmission device 300 transmits the encrypted feature information generated in step S806 to the feature aggregation server 100 via the communication network 500 (S807).

[0089] When the feature aggregation server 100 receives encrypted feature information from each data transmission device 300, it aggregates the information of each encrypted feature to generate an encrypted aggregate feature (common learning encrypted aggregate feature) (S808). Possible methods for aggregating the information of each encrypted feature include addition and averaging. By using additive homomorphic encryption, it is possible to perform addition while the data remains encrypted. Regarding the timing of aggregation, aggregation may be performed after receiving encrypted features from all data transmission devices 300, or thresholds related to time or the number of encrypted features may be set, and aggregation may be performed when a certain amount of time has elapsed or when a certain number of encrypted features have been received.

[0090] For example, the feature aggregation server 100 sequentially inputs encrypted features from each of the multiple data transmission devices 300, monitors a first timing (first time) when the input number of encrypted features reaches a set number, and a second timing (second time) when a set time has elapsed from the input start time when encrypted features are input from each of the multiple data transmission devices 300, and aggregates the input encrypted features at least at the first timing or the second timing. This allows the feature aggregation server 100 to start the process of aggregating the input encrypted features from the set first timing or the second timing.

[0091] The feature aggregation server 100 uses the primary secret key generated in step S804 to decrypt the encrypted aggregated features generated in step S808, generating primary decrypted encrypted aggregated features (S809). Here, since the feature aggregation server 100 does not possess the secondary secret key generated by the learning model update server 200 in step S801, it cannot completely decrypt the encrypted aggregated features. In this way, since the feature aggregation server 100 performs primary decryption after aggregation, even if the encrypted features before aggregation are leaked, the learning model update server 200, which only possesses the secondary secret key, cannot completely decrypt the encrypted features because the encrypted features are encrypted multiple times, and the security of the original features is ensured.

[0092] After this, the feature aggregation server 100 transmits the information of the primary decrypted encrypted aggregated features generated in step S809 to the learning model update server 200 via the communication network 500 (S810).

[0093] When the learning model update server 200 receives information on the primary decrypted encrypted aggregate features, it uses the secondary secret key generated in step S701 to decrypt (decrypt) the primary decrypted encrypted aggregate features and generate aggregate features (S811).

[0094] Next, the learning model update server 200 updates the learning model 320 using the aggregated features generated in step S811 (S606). When updating the learning model 320, possible methods include adding the aggregated features to the learning model 320, or dividing the aggregated features by the number of data transmission devices 300 before adding them.

[0095] Subsequently, the learning model update server 200 transmits the updated learning model 320 information to the data receiving device 400 via the communication network 500 (S607).

[0096] If steps S801 to S805 are designated as the initial phase (before the common AI learning phase) P801, and steps S601, S602, S806 to S811 are designated as the common AI learning phase P802, then after completing the processing of the initial phase P801, the processing of the common AI learning phase P802 may be repeated. That is, after step S607 of the common AI learning phase P802, the same procedure may be repeated starting from step S601 of the common AI learning phase P802. Alternatively, after step S606, the processing of step S607 may be omitted, and the same procedure may be repeated starting from step S601 of the common AI learning phase 802.

[0097] Here, we will explain the specific processing in steps S806 to S811, for example, when ElGamal elliptic curve encryption is used as the encryption method. In this case, let G be the base point on the elliptic curve, which is one of the encryption parameters; let a be the integer value of the primary secret key; let b be the integer value of the secondary secret key; let aG be the primary public key; and let bG be the secondary public key. Let M1 and M2 be the two points on the elliptic curve to be encrypted. Furthermore, we consider the case where, in encryption, the primary public key aG and the secondary public key bG are first added together to calculate the added public key (a+b)G, and then encryption is performed using the above added public key. At this time, the ciphertexts of points M1 and M2 on the elliptic curve are (r1G,r1(a+b)G+M1) and (r2G,r2(a+b)G+M2), respectively, using random integer values ​​r1 and r2 as random numbers (S806). These ciphertexts are sent as encrypted features from each data transmission device 300 to the feature aggregation server 100 (S807). If the integer values ​​r1 and r2 are not used as random numbers, the same data will be converted into the same ciphertext, so the integer values ​​r1 and r2 will be generated each time.

[0098] The feature aggregation server 100 aggregates each ciphertext to calculate the encryption aggregate feature ((r1+r2)G,(r1+r2)(a+b)G+M1+M2) (S808), and then performs the following calculation using the primary secret key a and the encryption aggregate feature (S809). ((r1+r2)(a+b)G+M1+M2)―a((r1+r2)G)=b(r1+r2)G+M1+M2...(3)

[0099] At this point, the feature aggregation server 100 sends the calculated ((r1+r2)G,b(r1+r2)G+M1+M2) as the primary decrypted encrypted aggregate feature to the learning model update server 200 (S810).

[0100] When the learning model update server 200 receives the primary decrypted encrypted aggregate features, it performs the following calculation using the secondary secret key b and the primary decrypted encrypted aggregate features (S811). (b(r1+r2)G+M1+M2)―b((r1+r2)G)=M1+M2...(4)

[0101] Based on the above calculation, the learning model update server 200 generates M1+M2 as aggregated features by decryption processing according to equation (4). However, in the federative learning system 10, it is necessary to encrypt the features in step S806. In this case, since the features are real values ​​and the points on the elliptic curve are integer values, when using ElGamal elliptic encryption, a mechanism is needed to convert between the features and the points on the elliptic curve. As a method of conversion, for example, a conversion table between n bits and points on the elliptic curve can be created in advance, and after converting the features to a binary representation of n bits, the conversion table can be used to convert them to points on the elliptic curve.

[0102] Furthermore, when encrypting features, a method of rounding the features can be employed to efficiently perform encryption and decryption. One method of rounding features is to set a value n and perform rounding up or rounding down on the features to unify them into an n-bit representation. In other words, if features are encrypted without rounding, the amount of information in the lookup table (correspondence table) referenced when decrypting the encrypted aggregated features increases, and decryption takes time, so a method of rounding the features is adopted. In this case, each data transmission device 300 performs rounding up or rounding down on the features before encrypting them multiple times, unifying the value of the features to the set bit information. This allows the feature aggregation server 100 and the learning model update server 200 to perform feature decryption efficiently.

[0103] Here, as a concrete example, if we define a feature as a real number satisfying 0 < or = feature < or = 1, and the value after rounding the feature as f, then when n=2, since it is 2 bits, the feature can be rounded to the following four values. In this case, the following methods are used to round the feature.

[0104] f=0 if 0 < OR =feature < 0.25 ... (a) f = 0.33 if 0.25 < OR = feature < 0.5 ... (b) f = 0.67 if 0.5 < OR = feature < 0.75 ... (c) f=1 if 0.75 < or =feature < or =1 ... (d)

[0105] In the case of n=4, since it is 4 bits, the feature can be rounded to 16 different values.

[0106] In this embodiment, the feature aggregation server 100 decrypts the encrypted aggregated features after aggregation, but it is also possible to decrypt the encrypted features before aggregation.

[0107] In this embodiment, instead of the learning model update server 200 generating the secondary secret key, the data transmission device 300 may generate the secondary secret key, and the learning model update server 200 may receive the secondary secret key generated by the data transmission device 300 from the data transmission device 300. In other words, as long as the learning model update server 200's secondary secret key is not leaked to the feature aggregation server 100, the feature aggregation server 100 cannot decrypt (decrypt) the primary decrypted encrypted aggregated features.

[0108] In this embodiment, when the data transmission device 300 generates a secret key, multiple data transmission devices 300 may each generate a secret key, and the value derived from the secret keys generated by each data transmission device 300 may be used as the secondary secret key. For example, the sum of n secret keys may be used as the secondary secret key.

[0109] Furthermore, although this embodiment uses two types of private key and public key pairs (primary and secondary), there are no limitations on the number of encryption and decryption methods. For example, each data transmission device 300 may encrypt the feature quantities five times, the feature quantity aggregation server 100 may decrypt the encrypted feature quantities twice, and the model update server 200 may decrypt the encrypted feature quantities three times.

[0110] According to this embodiment, multiple feature quantities can be encrypted multiple times, and these multiple encrypted feature quantities can be aggregated and decrypted in multiple stages using different devices. As a result, the risk of feature quantities being decrypted during the multi-stage decryption process can be reduced, thereby ensuring high information security.

[0111] Furthermore, according to this embodiment, in the common AI learning phase, since the learning model update server 200 holds the secondary secret key, the learning model update server 200 can use the secondary secret key to decrypt (decrypt) the primary decrypted encrypted aggregate feature to generate aggregate feature.

[0112] Furthermore, provided that the feature aggregation server 100 does not leak features and the learning model update server 200 does not decrypt (decrypt) anything other than the primary decrypted encrypted aggregated features, it is possible to ensure the same level of security as when features and tagged features are double-encrypted (encrypted using both the primary and secondary public keys) simply by using single encryption (encryption using only the primary public key) for features and tagged features. [Examples]

[0113] In this embodiment, a key generation process necessary for data encryption is performed as an initial phase before the purpose-specific AI learning phase, and then the tagged features are encrypted using the key generated in the key generation process during the purpose-specific AI learning phase. In other words, this embodiment is a method that enhances resistance to data leakage attacks, such as attacks that attempt to reconstruct the learning data held by the data transmission terminal from the tagged features, by encrypting the tagged features. Note that the hardware configuration and some of the software resource configurations are the same as in Embodiment 1. Furthermore, it is assumed that there are multiple data receiving devices 400.

[0114] Figure 9 is a sequence diagram showing an example of the processing of a federated learning system according to Embodiment 4 of the present invention. In Figure 9, the learning model update server 200 first generates a secondary public key, individual secret keys used by each data receiving device 400 to decrypt (decrypt) aggregated features for the purpose of key replacement encryption, a key replacement key used by the feature aggregation server 100 for key replacement, and information on cryptographic parameters as part of the initial phase P901 before common AI learning (S901).

[0115] Subsequently, the learning model update server 200 transmits information about the learning model 320 under development, the generated secondary public key, and cryptographic parameters to each data transmission device 300 via the communication network 500 (S902), and also transmits information about each individual private key to each data receiving device 400 (S903). Furthermore, it transmits information about the cryptographic parameters and key replacement key to the feature aggregation server 100 via the communication network 500 (S904).

[0116] In the key generation process (S901) by the learning model update server 200, for example, Elgamal elliptic cryptography, which is a type of additive homomorphic encryption, is used. In this case, the encryption parameters transmitted in steps S902 and S904 include information such as that Elgamal elliptic cryptography is to be used, the elliptic curve to be used, and the base point.

[0117] Next, as part of the processing in the objective-specific AI learning phase P902, steps S601, S608, S609, S905-S907, S612, S908-S910, and S614-S615 are executed. In this case, steps S601, S608, S609, S612, and S614 are the same as in the objective-specific AI learning phase of Example 1, so the processing that differs from the processing in the objective-specific AI learning phase of Example 1 (S905-S910) will be explained below.

[0118] After step S609, each data transmission device 300 encrypts the tagged feature calculated in step S609 using the secondary public key received in step S902, generating encrypted tagged feature (S905), and transmits the information of the generated encrypted tagged feature to the feature aggregation server 100 (S906). At this time, as a method of encrypting the tags, for example, the tags may be encrypted using the secondary public key, or the tags may not be encrypted at all.

[0119] When the feature aggregation server 100 receives information on encrypted tagged features from each data transmission device 300, it aggregates the information on each encrypted tagged feature to generate multiple aggregated features for different encryption purposes (S907). Possible methods for aggregating the information on each encrypted tagged feature include addition and averaging. By using additive homomorphic encryption, it is possible to perform addition while the data remains encrypted. Regarding the timing of aggregation, aggregation may be performed after receiving encrypted tagged features from all data transmission devices 300, or a threshold related to time or the number of encrypted tagged features may be set, and aggregation may be performed when a certain amount of time has elapsed or when a certain number of encrypted tagged features have been received.

[0120] Next, when the feature aggregation server 100 receives a query (second query) from the data receiving device 400 (S612), it performs a key swapping process using the target encryption purpose aggregate feature from the multiple encryption purpose aggregate features generated and the key swapping key received in step S904 to generate a key swapped encryption purpose aggregate feature (S908), and sends the information of the generated key swapped encryption purpose aggregate feature to the data receiving device 400 (S909). Here, after the key swapping, only the data receiving device 400, which has an individual secret key corresponding to the key swapping key, can recover the key swapped encryption purpose aggregate feature. For this reason, by selecting an appropriate key swapping key in step S908, the feature aggregation server 100 can generate the information of the key swapped encryption purpose aggregate feature as ciphertext that can only be recovered by the data transmitting device 300 that sent the query in step S612.

[0121] Subsequently, the data receiving device 400 decrypts the key-replaced encrypted purpose-specific aggregate feature quantity received in step S909 using the individual secret key received in step S903 to generate purpose-specific aggregate feature quantities (S910). At this time, if the key-replaced encrypted purpose-specific aggregate feature quantity is an encrypted message sent to another data receiving terminal 400, the individual secret key held by the data receiving device 400, which is not the target of the query, cannot decrypt the key-replaced encrypted purpose-specific aggregate feature quantity.

[0122] Here, we will explain the specific process of key swapping and decryption (decryption) in steps S907 to S910, for example, when ElGamal elliptic curve encryption is used as the encryption method. In this case, let G be the base point on the elliptic curve, which is one of the encryption parameters; b be the integer value of the secondary secret key; c be the integer value of the individual secret key; c / b be the integer value of the key swapping key; and bG be the secondary public key. Note that c / b is a division over a finite field defined by the encryption parameters, and let M be the point on the elliptic curve to be encrypted. In this case, the ciphertext of point M on the elliptic curve is (brG, rG+M) (S907), where r is a random integer. (brG, rG+M) corresponds to the aggregated features for the purpose of encryption. If the integer value r is not used as a random number, the same data will be converted to the same ciphertext, so the integer value r will be generated each time.

[0123] Next, the feature aggregation server 100 calculates (brG)*(c / b)=crG using the key replacement key and the encryption purpose-specific aggregated features (S908), and transmits (crG,rG+M) to the data receiving device 400 as the key-replaced encryption purpose-specific aggregated features (S909).

[0124] The data receiving device 400 performs the following calculation using the received key-replaced encryption purpose-specific aggregated feature quantities and individual secret keys. (rG+M)―(crG / c)=M···(5)

[0125] Based on the above calculation, the data receiving device 400 generates M as a purpose-specific aggregated feature by decrypting it according to equation (5). However, in the federative learning system 10, it is necessary to encrypt the tagged feature in step S905. In this case, since the tagged feature is a real value and the points on the elliptic curve are integer values, when using ElGamal elliptic encryption, a mechanism is needed to convert between the tagged feature and the points on the elliptic curve. As a method of conversion, for example, one could create a conversion table between n bits and points on the elliptic curve in advance, convert the feature to a binary representation of n bits, and then convert it to a point on the elliptic curve using the above conversion table.

[0126] In this embodiment, the feature aggregation server 100 decrypts (decrypts) the encrypted tagged features after aggregation of the encrypted tagged features, but it is also possible to decrypt the encrypted tagged features before aggregation of the encrypted tagged features.

[0127] According to this embodiment, the same effects as in Embodiment 1 can be achieved, and in the purpose-specific AI learning phase, since the data receiving device 400 holds an individual secret key, the data receiving device 400, which holds an individual secret key corresponding to the key replacement key, can use the individual secret key to decrypt (decode) the key-replaced encrypted purpose-specific aggregate feature to generate purpose-specific aggregate feature.

[0128] It should be noted that the present invention is not limited to the embodiments described above, and various modifications are included. For example, the embodiments described above are described in detail to make the present invention easier to understand, and are not necessarily limited to those having all the configurations described. It is also possible to replace parts of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add configurations from other embodiments to the configuration of one embodiment. Furthermore, it is possible to add, delete, or replace parts of the configuration of each embodiment with other configurations.

[0129] Furthermore, each of the above configurations and functional means may be implemented in hardware, either partially or entirely, by designing them as integrated circuits, for example. Alternatively, each of the above configurations and functions may be implemented in software by having the processor interpret and execute programs that implement each function. Information such as programs, tables, and files that implement each function can be stored in memory, a recording device such as a hard disk or SSD (Solid State Drive), or a recording medium such as an IC (Integrated Circuit) card, an SD (Secure Digital) card, or a DVD (Digital Versatile Disc).

[0130] Furthermore, the control lines and information lines shown are those deemed necessary for explanatory purposes, and not all control lines and information lines are necessarily shown in the actual product. In reality, it is safe to assume that almost all components are interconnected. [Explanation of Symbols]

[0131] 100 Feature aggregation server, 200 Learning model update server, 300 Data transmission device, 400 Data reception device, 101 Processor, 102 Auxiliary storage device, 103 Meri, 107 Communication interface, 111 Key generation unit, 112 Feature aggregation unit, 113 Decryption processing unit, 114 Key replacement unit, 201 Processor, 202 Auxiliary storage device, 203 Memory, 207 Communication interface, 211 Key generation unit, 212 Learning model update unit, 213 Decryption processing unit, 301 Processor, 302 Auxiliary storage device, 303 Memory, 307 Communication interface, 311 Feature calculation unit, 312 Cryptography processing unit, 401 Processor, 402 Auxiliary storage device, 403 Memory, 407 Communication interface, 411 Learning model update unit, 412 Decryption processing unit

Claims

1. Multiple data transmission devices connected to a communication network, A feature aggregation server that sends and receives information with each of the multiple data transmission devices via the aforementioned communication network, The system comprises one or more data receiving devices connected to the feature aggregation server via the aforementioned communication network, Each of the aforementioned data transmission devices, Based on the input of a first query including data extraction requirements, tagged features indicating features according to the first query are generated based on the learning model and each of the multiple learning data, and transmission information including the generated tagged features is sent to the feature aggregation server. The aforementioned feature aggregation server, When the transmission information is received from each of the aforementioned data transmission devices, the tagged feature quantities belonging to the received transmission information are aggregated to generate purpose-specific aggregated feature quantities, and the generated purpose-specific aggregated feature quantities are transmitted to the data receiving device. The aforementioned data receiving device is A federated learning system characterized in that, when it receives the purpose-specific aggregated features from the feature aggregation server, it updates the learning model based on the received purpose-specific aggregated features.

2. In the associative learning system described in claim 1, The aforementioned data receiving device is The information of the second query, including the data aggregation requirements, is transmitted to the feature aggregation server via the communication network. The aforementioned feature aggregation server, A federated learning system characterized in that, upon receiving the second query, it transmits the purpose-specific aggregated feature corresponding to the second query from the purpose-specific aggregated feature quantities to the data receiving device.

3. In the associative learning system described in claim 1, The aforementioned data receiving device is The information of the second query, including the data aggregation requirements, is transmitted to the feature aggregation server via the communication network. The aforementioned feature aggregation server, A federated learning system characterized by generating a first query based on the received second query, and transmitting the generated first query to each of the data transmission devices via the communication network, on the condition that the second query has been received.

4. In the associative learning system described in claim 1, The aforementioned data receiving device is The information of the second query, including the data aggregation requirements, is transmitted to the feature aggregation server via the communication network. Each of the aforementioned data transmission devices, The tagged feature quantities are encrypted multiple times to generate encrypted tagged feature quantities, and the generated encrypted tagged feature quantities are sent to the feature quantity aggregation server. The aforementioned feature aggregation server, When the second query is received, the encrypted tagged feature quantities transmitted from each of the data transmission devices are aggregated to generate an aggregated feature quantity for encryption purposes, the key is changed on the aggregated feature quantity for encryption purposes corresponding to the second query from the generated aggregated feature quantity for encryption purposes to generate a key-changed aggregated feature quantity for encryption purposes, and the generated key-changed aggregated feature quantity for encryption purposes is transmitted to a specific data receiving device among the data receiving devices that is the source of the second query. The aforementioned specific data receiving device is A federated learning system characterized by receiving the aforementioned key-replaced encrypted purpose-specific aggregated feature quantity, decrypting the received key-replaced encrypted purpose-specific aggregated feature quantity, and generating the purpose-specific aggregated feature quantity.

5. In the associative learning system described in claim 4, The system further comprises a learning model update server that generates information on the learning model, public key, key replacement key, individual private key, and cryptographic parameters, respectively. The aforementioned learning model update server is: The learning model, the public key, and the cryptographic parameters are transmitted to each data transmission device via the communication network; the key replacement key and the cryptographic parameters are transmitted to the feature aggregation server via the communication network; and the individual secret key is transmitted to the specific data receiving device via the communication network. Each of the aforementioned data transmission devices, Based on the public key and cryptographic parameters received from the learning model update server, the tagged feature is encrypted multiple times to generate the encrypted tagged feature. The aforementioned feature aggregation server, Based on the cryptographic parameters and key replacement key received from the learning model update server, the key replacement is performed on the encryption purpose-specific aggregate feature corresponding to the second query to generate the key-replaced encryption purpose-specific aggregate feature. The aforementioned specific data receiving device is A federated learning system characterized by decrypting the key-replaced encrypted aggregated features based on the individual secret keys received from the learning model update server.

6. In the associative learning system according to claim 2 or 3, A federated learning system characterized in that at least one of the data extraction requirements and the data aggregation requirements specifies the data distribution of the data used for the training data.

7. In the associative learning system described in claim 1, The system further comprises a learning model update server that transmits information about the learning model to each of the data transmission devices via the communication network, Each of the aforementioned data transmission devices, Conditional on the absence of input for the first query, common learning features are generated based on the learning model and each of the multiple learning data, and sent to the feature aggregation server. The aforementioned feature aggregation server, The common learning features transmitted from each of the data transmission devices are aggregated to generate a common learning aggregate feature, and the generated common learning aggregate feature is transmitted to the learning model update server via the communication network. The aforementioned learning model update server is: A federated learning system characterized by receiving the aggregated common learning features, updating the learning model based on the received aggregated common learning features, and transmitting the updated learning model to the data receiving device via the communication network.

8. In the associative learning system described in claim 7, The aforementioned learning model update server is: The learning model, the first public key, the first private key, and the cryptographic parameters are generated, the generated learning model, the first public key, and the cryptographic parameters are transmitted to each data transmission device via the communication network, and the generated cryptographic parameters are transmitted to the feature aggregation server via the communication network. The aforementioned feature aggregation server, When the learning model update server receives the cryptographic parameters, it generates a second public key based on the received cryptographic parameters and sends it to each of the data transmission devices. Each of the aforementioned data transmission devices, Based on the first public key and cryptographic parameters received from the learning model update server and the second public key received from the feature aggregation server, the common learning features are encrypted multiple times to generate common learning encrypted features. The aforementioned feature aggregation server, The aforementioned common learning encryption features are aggregated to generate a common learning encryption aggregate feature, the generated common learning encryption aggregate feature is decrypted using a second secret key to generate a primary decrypted encryption aggregate feature, and the generated primary decrypted encryption aggregate feature is sent to the learning model update server. The aforementioned learning model update server is: A federated learning system characterized in that, upon receiving the primary decrypted encrypted aggregate feature quantity, it decrypts the received primary decrypted encrypted aggregate feature quantity with the first secret key to generate the common learning aggregate feature quantity.

9. Multiple data transmission devices connected to a communication network, A feature aggregation server that sends and receives information with each of the multiple data transmission devices via the aforementioned communication network, A method in a federated learning system comprising one or more data receiving devices connected to the feature aggregation server via the aforementioned communication network, Each of the aforementioned data transmission devices, upon input of a first query including data extraction requirements, generates tagged features indicating features according to the first query based on the learning model and each of the multiple learning data, and transmits transmission information including the generated tagged features to the feature aggregation server in a first learning step. When the feature aggregation server receives the transmission information from each of the data transmission devices, it aggregates the tagged features belonging to the received transmission information to generate purpose-specific aggregated features, and transmits the generated purpose-specific aggregated features to the data receiving device in a second learning step. A federated learning method characterized by comprising: a third learning step in which, when the data receiving device receives the purpose-specific aggregated features from the feature aggregation server, the learning model is updated based on the received purpose-specific aggregated features.

10. In the associative learning method described in claim 9, The aforementioned data receiving device is Before executing the third learning step described above, information of the second query, including data aggregation requirements, is sent to the feature aggregation server via the communication network. The aforementioned feature aggregation server, The federated learning method is characterized in that, in the second learning step, when the second query is received, the purpose-specific aggregate feature corresponding to the second query from the purpose-specific aggregate feature is transmitted to the data receiving device.

11. In the associative learning method described in claim 9, The aforementioned data receiving device is Before executing the third learning step described above, information of the second query, including data aggregation requirements, is sent to the feature aggregation server via the communication network. The aforementioned feature aggregation server, The federated learning method is characterized in that, in the second learning step, upon receiving the second query, a first query is generated based on the received second query, and the generated first query is transmitted to each of the data transmission devices via the communication network.

12. In the associative learning method described in claim 9, The aforementioned data receiving device is Before executing the third learning step described above, information of the second query, including data aggregation requirements, is sent to the feature aggregation server via the communication network. Each of the aforementioned data transmission devices, In the first learning step, the generated tagged features are encrypted multiple times to generate encrypted tagged features, and the generated encrypted tagged features are sent to the feature aggregation server. The aforementioned feature aggregation server, In the second learning step, upon receiving the second query, the encrypted tagged feature quantities transmitted from each data transmission device are aggregated to generate an aggregated feature quantity for each encryption purpose, the key is changed on the aggregated feature quantity for each encryption purpose corresponding to the second query from the generated aggregated feature quantity for each encryption purpose to generate a key-changed aggregated feature quantity for each encryption purpose, and the generated key-changed aggregated feature quantity for each encryption purpose is transmitted to a specific data receiving device among the data receiving devices that is the source of the second query. The aforementioned specific data receiving device is The third learning step is characterized by a federated learning method in which, upon receiving the key-replaced encrypted purpose-specific aggregated feature quantity, the received key-replaced encrypted purpose-specific aggregated feature quantity is decrypted to generate the purpose-specific aggregated feature quantity.

13. In the associative learning method described in claim 12, The aforementioned federated learning system further comprises a learning model update server that generates information on the learning model, public key, key-replacement key, individual private key, and cryptographic parameters, respectively. As a step preceding the first learning step, the learning model update server includes an initial learning step in which it transmits information of the learning model, the public key, and the cryptographic parameters to each of the data transmission devices via the communication network, transmits information of the key replacement key and the cryptographic parameters to the feature aggregation server via the communication network, and transmits information of the individual secret key to the specific data receiving device via the communication network. Each of the aforementioned data transmission devices, In the first learning step, the tagged feature is encrypted multiple times based on the public key and cryptographic parameters received from the learning model update server in the initial learning step to generate the encrypted tagged feature. The aforementioned feature aggregation server, In the second learning step, based on the cryptographic parameters and key replacement key received from the learning model update server in the initial learning step, the key replacement is performed on the encryption purpose-specific aggregate feature corresponding to the second query to generate the key-replaced encryption purpose-specific aggregate feature. The aforementioned specific data receiving device is The federated learning method is characterized in that, in the third learning step, the key-replaced encrypted purpose-specific aggregated feature quantities are decrypted based on the individual secret keys received from the learning model update server in the initial learning step.

14. In the associated learning method according to claim 10 or 11, A federated learning method characterized in that at least one of the data extraction requirements and the data aggregation requirements specifies the data distribution of the data to be used as training data.

15. In the associative learning method described in claim 9, The aforementioned federated learning system further comprises a learning model update server that generates information about the learning model, The learning model update server performs a first common learning step in which it transmits information about the learning model to each of the data transmission devices via the communication network, A second common learning step in which each data transmission device generates common learning features based on the learning model and each of the multiple learning data, and transmits them to the feature aggregation server, provided that there is no input for the first query. A third common learning step in which the feature aggregation server aggregates the common learning features transmitted from each of the data transmission devices to generate a common learning aggregated feature, and transmits the generated common learning aggregated feature to the learning model update server via the communication network, A federated learning method characterized by comprising: a fourth common learning step in which, when the learning model update server receives the common learning aggregate features, the learning model updates based on the received common learning aggregate features, and transmits the updated learning model to the data receiving device via the communication network.

Images