Federated learning system, federated learning method and federated learning program

Abstract

To provide a suitable model for each client terminal that participates in federated learning.SOLUTION: A federated learning system has a plurality of client terminals having a learning data set and a server, the learning data set has a data sample including a client ID, a first explanatory variable and a first objective variable, and the system executes calculation processing through the client terminals, and executes first federated learning processing that repeats first learning processing through the client terminals and first integration processing through the server. In the calculation processing, the client terminal inputs the data sample in a similarity calculation mode and calculates similarity between the data sample and a plurality of learning data sets. In the first learning processing, the client terminal learns an individual analysis model on the basis of similarity between a specific learning data set and specific similarity. In the first integration processing, the server integrates first learning results from the plurality of client terminals, and generates first integration information about an integrated individual analysis model.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a federated learning system, a federated learning method, and a federated learning program for performing federated learning. [Background technology]

[0002] With the advancement of digitalization of healthcare information, secondary utilization of healthcare information managed by local governments, medical institutions, individuals, etc. (hereafter referred to as clients) is increasing. In particular, federated learning, which can train models in a distributed environment without centralizing and managing information on a server, is attracting attention from the perspective of personal information protection.

[0003] The following Patent Document 1 discloses a concept of federated learning that specifies clients and measures the similarity of training data in a manner more suitable for application in a federated learning environment. The apparatus for federated learning disclosed in the following Patent Document 1 receives parameterization updates related to a predetermined parameterization of a neural network from multiple clients, and performs federated learning of the neural network according to the similarity between the parameterization updates. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2020 / 229684 Summary of the Invention [Problem to be solved by the invention]

[0005] However, in the above-mentioned Patent Document 1, the similarity is determined based on parameters calculated from client data, and the similarity for each data sample of each client is not taken into consideration. Therefore, the device of Patent Document 1 determines whether to use data for integrated learning on a client-by-client basis, and cannot determine whether to use data on a client-by-client data sample basis, and therefore cannot control the magnitude of the influence on integrated learning on a client-by-client data sample basis.

[0006] An object of the present invention is to provide a model suitable for each client terminal participating in federated learning. [Means for solving the problem]

[0007] A federated learning system according to one aspect of the invention disclosed in the present application includes a plurality of client terminals each having a learning dataset, and a server capable of communicating with the plurality of client terminals, and performs federated learning in which each of the plurality of client terminals learns a model using the learning dataset, and the server repeats a process of integrating the models of the plurality of client terminals using a learning result, wherein the learning dataset includes one or more data samples including a client ID that identifies the client terminal, a first explanatory variable, and a first objective variable, and the system performs a calculation process by each of the plurality of client terminals, and also performs a first federated learning process in which a first learning process by each of the plurality of client terminals and a first integration process by the server are repeated until a first termination condition is met, and in the calculation process, Each of the client terminals calculates the similarity between the data sample and the multiple learning datasets by inputting the data sample into a similarity calculation model that calculates the similarity between the data sample and the multiple learning datasets; in the first learning process, each of the multiple client terminals learns the individual analysis model based on an individual analysis model that calculates a predicted value of the first dependent variable from the first explanatory variable, and the first explanatory variable, the first dependent variable, and a specific similarity with a specific learning dataset calculated by each of the multiple client terminals by the calculation process; and in the first integration process, the server integrates multiple first learning results from the first learning process from the multiple client terminals and generates first update information regarding an integrated individual analysis model that integrates the individual analysis models of each of the multiple client terminals. [Effects of the Invention]

[0008] According to a representative embodiment of the present invention, it is possible to provide a model suitable for each client terminal participating in federated learning. Problems, configurations, and effects other than those described above will become clear from the description of the following examples. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is an explanatory diagram illustrating an example of associative learning according to this embodiment. [Figure 2] FIG. 2 is an explanatory diagram illustrating an example of a training data set. [Figure 3] FIG. 3 is an explanatory diagram showing an example of associative learning of a similarity calculation model. [Figure 4] FIG. 4 is an explanatory diagram showing an example of federated learning of individual analysis models. [Figure 5] FIG. 5 is an explanatory diagram showing a specific example of the learning weight calculation shown in FIG. [Figure 6] FIG. 6 is an explanatory diagram showing a specific example of the weighted model learning 411t shown in FIG. [Figure 7] FIG. 7 is an explanatory diagram showing a specific example of the weighted model learning 412t shown in FIG. [Figure 8] FIG. 8 is an explanatory diagram showing a specific example of the weighted model learning 413t shown in FIG. [Figure 9] FIG. 9 is a sequence diagram of federated learning in the federated learning system. [Figure 10] FIG. 10 is an explanatory diagram showing an example of a management screen. [Figure 11] FIG. 11 is an explanatory diagram showing an example of an information display screen. [Figure 12] FIG. 12 is a block diagram showing an example of the hardware configuration of the server and the client terminal. DETAILED DESCRIPTION OF THE INVENTION

[0010] <Example of associative learning> FIG. 1 is an explanatory diagram showing an example of federated learning according to this embodiment. The federated learning system 100 has a server S and multiple client terminals C1 to C3 (three in FIG. 1 as an example). When there is no need to distinguish between them, they are referred to as client terminals Ck (k=1, 2, 3, ..., K). The number of client terminals Ck is not limited to three, and may be two or more. In this example, K=3. The server S and the client terminals Ck are connected to each other so as to be able to communicate with each other via a network such as the Internet, a LAN (Local Area Network), or a WAN (Wide Area Network).

[0011] The client terminals C1 to C3 have training data sets D1 to D3. When there is no need to distinguish between them, they are referred to as training data set Dk. The training data set Dk is a combination of training data that serves as explanatory variables and correct answer data that serves as target variables. It is assumed that the training data set Dk is prohibited from being taken out of the client terminal Ck or the base where the client terminal Ck is installed.

[0012] The client terminal Ck is a computer that provides the learning data set Dk to a prediction model to learn it individually, and transmits the learning results, such as the model parameters of the learned prediction model or their gradients, to the server S each time learning is performed.

[0013] The server S is a computer that uses the learning results from the client terminals Ck to integrate the prediction models for each client terminal Ck to generate an integrated prediction model and transmits the integrated prediction model to the client terminal Ck. The client terminal Ck applies the learning dataset Dk to the integrated prediction model from the server S to learn the prediction model. By repeating this learning process, the federated learning system 100 performs federated learning.

[0014] In this embodiment, the associative learning system 100 performs two types of associative learning: associative learning FL1 of a similarity calculation model, and associative learning FL2 of an individual analysis model.

[0015] The federated learning FL1 of the similarity calculation model is federated learning that generates an integrated similarity calculation model M1 that integrates the similarity calculation models from each client terminal Ck by performing the above-mentioned federated learning using the similarity calculation model as a prediction model. The similarity calculation model is a prediction model that, when focusing on a learning dataset Dk (hereinafter referred to as a learning dataset Dj to distinguish it from Dk) of a client terminal Ck (hereinafter referred to as a client terminal Cj to distinguish it from Ck) that is the learning target among the client terminals C1 to CK, calculates the similarity with the learning dataset Dk for each data sample (hereinafter referred to as data sample i) of the learning dataset Dj (i is an integer that satisfies 1≦i≦Nj, and Nj is the total number of data samples of the learning dataset Dj).

[0016] Specifically, for example, the similarity calculation model is a model that calculates a propensity score with a client ID that uniquely identifies a client terminal Ck as an assigned variable.

[0017] Hereinafter, k will be used for any client terminal and its learning data set, but j will be used for the client terminal and its learning data set that are the learning target.

[0018] By generating the integrated similarity calculation model M1 prior to the associative learning FL2 of the individual analysis models, it becomes possible to adjust the influence of each data sample i on the associative learning in the associative learning FL2 of the individual analysis models.

[0019] The individual analytical model federated learning FL2 is federated learning that generates individual analytical models M21 to M23 by integrating the individual analytical models for the client terminal Cj obtained from each of the client terminals C1 to C3.

[0020] Specifically, for example, individual analytical model M21 is a prediction model that integrates individual analytical models for client terminal C1 (j=1) from client terminals C1 to C3, individual analytical model M22 is a prediction model that integrates individual analytical models for client terminal C2 (j=2) from client terminals C1 to C3, and individual analytical model M23 is a prediction model that integrates individual analytical models for client terminal C3 (j=3) from client terminals C1 to C3.

[0021] By executing federated learning FL2 of the individual analytical models, appropriate individual analytical models M21 to M23 are generated for each client terminal C.

[0022] <Learning dataset Dk> FIG. 2 is an explanatory diagram showing an example of a training dataset Dk. The training dataset Dk has, as fields, a client ID (sometimes referred to as "CID" in the following drawings) 201, a data ID 202, an explanatory variable 203 (sometimes referred to as explanatory variable X), and a response variable 204 (sometimes referred to as response variable y). A combination of values ​​in each field on the same line becomes an entry that defines one data sample. Each of the training datasets D1 to D3 is, for example, a collection of data samples of patient groups from each hospital.

[0023] The client ID 201 is identification information that uniquely identifies the client terminal Ck. The value of the client ID 201 is expressed as Ck. The data ID 202 is identification information that uniquely identifies a data sample. The value of the data ID 202 is expressed as Dki, where i is a number unique to the data sample. The data ID 202, for example, identifies a patient. The explanatory variables 203 are learning data used in the federated learning FL2 of the individual analysis model, and include one or more features x1, x2, ... (when these are not distinguished, they are simply referred to as feature x). The feature x is, for example, the height, weight, blood pressure, etc. of the patient identified by the data ID 202.

[0024] The objective variable 204 is the correct answer data used in the associative learning FL2 of the individual analysis model. The objective variable 204 indicates, for example, whether or not a patient identified by the data ID 202 has a disease. y=1 indicates the presence of a disease, and y=0 indicates the absence of a disease.

[0025] <Associative learning of similarity computation model FL1> 3 is an explanatory diagram showing an example of federated learning FL1 of the similarity calculation model. In federated learning FL1 of the similarity calculation model, the combination of explanatory variables 203 and objective variables 204 of the learning dataset Dk becomes explanatory variables 301, and the client ID 201 becomes objective variable 302.

[0026] The server S has a base similarity calculation model (hereinafter referred to as a base similarity calculation model) M10. The base similarity calculation model M10 may be an untrained neural network, and has model parameters φ such as weights and biases. k t Alternatively, it may be a trained neural network in which the following has been set: t is a natural number in ascending order starting from 1 that indicates the number of times the associative learning FL1 of the similarity calculation model has been executed. The server S transmits the base similarity calculation model M10 to the client terminals C1 to C3.

[0027] If the client terminal Ck has an untrained neural network, the server S can set the model parameters φ k t The client terminal Ck receives the model parameters φ k t The base similarity calculation model M10 may be constructed by setting the above to an untrained neural network.

[0028] The base similarity calculation model M10 becomes the learning object similarity calculation model M11t of the first associative learning FL1 in the client terminal Ck.

[0029] The client terminal Ck executes model learning 31kt (311t to 313t) in the t-th federated learning FL1. Specifically, for example, the client terminal Ck provides the explanatory variables 301 and the objective variables 302 of the learning data set D1 to the learning object similarity calculation model M1kt (M11t to M13t) and performs individual learning. The client terminal Ck uses the model parameter φ k t or its gradient gs k t The learning result 32kt (321t to 323t) is sent to the server S.

[0030] The server S executes integrated learning 330t in the t-th associative learning FL1 using the learning result 32kt, and generates an integrated similarity calculation model M1(t+1) for the next time. Specifically, for example, the server S generates an integrated similarity calculation model M1(t+1) using an integrated result obtained by integrating the learning result 32kt, and calculates the integrated similarity calculation model M1(t+1) or its integrated model parameter φ t+1 to the client terminal Ck. As a result, the learning object similarity calculation model M1kt for the next round of associative learning FL1 is set in the client terminal Ck.

[0031] In this way, the federated learning FL1 is repeatedly executed. When the number of executions t reaches a predetermined threshold T1 or the accuracy of the integrated similarity calculation model M1t reaches the target accuracy, the server S ends the federated learning FL1 and outputs the latest integrated similarity calculation model M1(t+1) as the integrated similarity calculation model M1, and transmits it to the client terminals C1 to C3.

[0032] The formulas used in the associative learning FL1 of the similarity calculation model are defined below.

[0033] [p j ] t =h(X,y;φ j t )···(1)

[0034] The above formula (1) is a calculation formula that defines the similarity calculation model, and is executed by the model learning 31jt of the client terminal Cj. The function h is a function of the explanatory variables 301, which are a combination of the explanatory variables 203 (X) and the objective variables 204 (y), and the model parameters φ j t The learning object similarity calculation model M1jt (M11t to M13t) is defined as follows. [p j ] t is a predicted probability indicating which learning data set Dk each data sample i in the learning data set Dj of the client terminal Cj is similar to in the t-th federated learning FL1.

[0035] Let N be the total number of data samples i in the training dataset Dj. j Then, the predicted probability [p j ] t is N j ×K matrix. That is, the row vector [p ji ] t are the predicted probabilities indicating the similarity between data sample i of the learning dataset Dj and the learning dataset D1, the predicted probabilities indicating the similarity between data sample i of the learning dataset Dj and the learning dataset D2, the predicted probabilities indicating the similarity between data sample i of the learning dataset Dj and the learning dataset D3, ..., the predicted probabilities indicating the similarity between data sample i of the learning dataset Dj and the learning dataset DK.

[0036] Also, the predicted probability [p j ] t The kth column of the matrix j A column vector [p j k ] t is the predicted probability indicating the similarity between the first (i=1) data sample i in the training data set Dj and the training data set Dk, the predicted probability indicating the similarity between the second (i=2) data sample i in the training data set Dj and the training data set Dk, the predicted probability indicating the similarity between the third (i=3) data sample i in the training data set Dj and the training data set Dk, ..., N of the training data set Djj th (i=N j ) is the predicted probability indicating the similarity between data sample i and the training data set Dk.

[0037]

number

[0038] The above formula (2) is the loss function H(φ j t ) is p ji is the similarity between the data sample i in the training data set Dj and the training data sets D1 to D3. ji For example, the value ranges from 0.0 to 1.0, and the larger the value, the more similar it is.

[0039] If j=1, then p ji = (1.0, 0, 0), which indicates that data sample i is a data sample in the training data set D1. When j = 2, p ji = (0, 1.0, 0), which indicates that data sample i is a data sample in the training dataset D2. When j = 3, p ji =(1.0,0,0), indicating that the data sample i is a data sample in the training data set D3.

[0040] [p ji ] t is the predicted probability [p j ] t is a row vector that is a predicted probability indicating the degree to which data sample i is similar to the learning data sets D1 to D3 in the matrix indicated by

[0041] N j is the total number of data samples i in the training dataset Dj. The function loss is the error function of data sample i, and the average value of the error function loss for each data sample i is the loss function H(φ jt )

[0042]

number

[0043] The above equation (3) is the model parameter φ j t The gradient of gs j t is a calculation formula that defines the gradient gs, which is executed by the model learning 31jt of the client terminal Cj. η is the learning rate. J is the total number of client terminals Cj that are the learning targets, and J=K. When applying the following formula (4), the client terminal Cj j t is sent to server S as the learning result 32jt.

[0044]

number

[0045] The above equation (4) is the integrated model parameter φ j t The integrated model parameters φ j t+1 The server S receives the gradient gs from the client terminal Cj. j t is received as the learning result 32jt and the above equation (4) is executed. Note that the integrated model parameter φ t is the integrated model parameter calculated as a result of executing the above formula (4) in the previous associative learning FL1.

[0046]

number

[0047] The above equation (5) is the model parameter φ j t The model parameter φ jt+1 This is a calculation formula for updating the model parameter φ after updating, which is executed by the integrated learning 330t of the server S. It is executed by the model learning 31jt of the client terminal Cj. When applying the above formula (5), the client terminal Cj updates the model parameter φ after updating. j t+1 is sent to server S as the learning result 32jt.

[0048] The above equation (6) is the updated model parameter φ of the above equation (5). j t+1 Using the integrated model parameters φ t+1 is a formula for calculating the sum of the data samples i of the client terminals C1 to CJ, and is executed by the integrated learning 330t of the server S. N is the total number of data samples i of the client terminals C1 to CJ.

[0049] The federated learning system 100 updates the integrated model parameter φ using either the update method according to the above formula (4) or the update method according to formulas (5) and (6). t+1 The server S calculates the integrated similarity calculation model M1t (the base similarity calculation model M10 when t=0) using the integrated model parameter φ t+1 and generate the integrated similarity calculation model M1(t+1).

[0050] The server S calculates the integrated similarity calculation model M1(t+1) or its integrated model parameters φ t+1 is transmitted to the client terminals C1 to C3, thereby updating the above formula (1).

[0051] [p j ] t+1 =h(X,y;φ t+1 )···(7)

[0052] The above formula (6) is a calculation formula that defines the integrated similarity calculation model M1(t+1), which is generated by the integrated learning 330 of the server S, sent to the client terminal Ck, and becomes the learning target similarity calculation model M1kt in the next associative learning FL1.

[0053] [p j]=h(X,y;φ) (8)

[0054] The above formula (8) is a calculation formula that defines the integrated similarity calculation model M1 when associative learning FL1 is completed. Since it is the same formula as the above formula (7) at the end of associative learning FL1, "t+1" has been deleted from the above formula (7).

[0055] <Associative learning of individual analytical models FL2> 4 is an explanatory diagram showing an example of federated learning FL2 of an individual analysis model. A client terminal C1 executes a learning weight calculation 401. Specifically, for example, a client terminal Cj inputs the explanatory variables 301 of the learning data set Dj into the integrated similarity calculation model M1 (the above formula (8)) generated in FIG. 3, thereby calculating a predicted probability [p j ]. The client terminal Cj calculates the calculated predicted probability [p j ], and the learning weights w j Calculate.

[0056] The server S has a base individual analysis model (hereinafter referred to as a base individual analysis model) M20. The base individual analysis model M20 may be an untrained neural network, and has model parameters θ such as weights and biases. t Alternatively, the server S may transmit the base individual analysis model M20 to the client terminal Ck.

[0057] If the client terminal Ck has an untrained neural network, the server S will t is sent to the client terminal Ck, and the client terminal Ck receives the model parameters θ t A base individual analysis model M20 may be constructed by setting the above to an untrained neural network.

[0058] The base individual analytical model M20 becomes the individual analytical model M2j-kt to be learned in the first federated learning FL2 in the client terminal Ck. t is an ascending natural number starting from 1 that indicates the number of times the federated learning FL2 of the individual analytical model is executed.

[0059] When j=1, client terminal C1 holds learning subject individual analytical models M21-1t to M21-3t. When j=2, client terminal C2 holds learning subject individual analytical models M22-1t to M22-3t. When j=3, client terminal C3 holds learning subject individual analytical models M23-1t to M23-3t.

[0060] The client terminal Cj executes weighted model learning 41jt. Specifically, for example, the client terminal Cj inputs the explanatory variables 203 of the learning data set Dj into each of the individual analytical models M2j-kt to be learned, thereby obtaining the predicted value [y j k ]. The client terminal Cj calculates the learning weight w j and the response variable y and the predicted value [y j k ] and model parameters θ j kt As a result, the loss function F(θ j kt ) is calculated.

[0061] Then, the client terminal Cj calculates the model parameters θ j kt The gradient of ga j kt is calculated as the individual analysis model learning result 42j-kt, and the model parameters θ j kt The model parameters θ j k(t+1) The model parameters θ j k(t+1) is also included in the individual analysis model learning results 42j-kt.

[0062] When j=1, the client terminal C1 generates individual analytical model learning results 421-1t to 421-3t. The individual analytical model learning results 421-1t are generated by updating the model parameters θ1 1t or its gradient ga1 1t The individual analysis model learning result 421-2t is the model parameter θ1 updated in the learning target individual analysis model M21-2t. 2t or its gradient ga1 2t The individual analysis model learning result 421-3t is the model parameter θ1 updated in the learning target individual analysis model M21-3t. 3t or its gradient ga1 3t is.

[0063] When j=2, the client terminal C2 generates individual analytical model learning results 422-1t to 422-3t. The individual analytical model learning results 422-1t are generated based on the model parameters θ2 1t or its gradient ga2 1t The individual analysis model learning result 422-2t is the model parameter θ2 updated in the learning target individual analysis model M22-2t. 2t or its gradient ga2 2t The individual analysis model learning result 422-3t is the model parameter θ2 updated in the learning target individual analysis model M22-3t. 3t or its gradient ga2 3t is.

[0064] When j=3, the client terminal C3 generates individual analytical model learning results 423-1t to 423-3t. The individual analytical model learning results 423-1t are generated based on the model parameters θ3 1t or its gradient ga3 1t The individual analysis model learning result 423-2t is the model parameter θ3 updated in the learning target individual analysis model M23-2t. 2t or its gradient ga3 2tThe individual analysis model learning result 423-3t is the model parameter θ3 updated in the learning target individual analysis model M23-3t. 3t or its gradient ga3 3t is.

[0065] The server S receives the individual analytical model learning results 42j-kt from the client terminal Ck. Specifically, for example, the server S receives the individual analytical model learning results 421-1t to 421-3t from the client terminal C1, the individual analytical model learning results 422-1t to 422-3t from the client terminal C2, and the individual analytical model learning results 423-1t to 423-3t from the client terminal C3.

[0066] The server S executes integrated learning 430t in the t-th federated learning FL2 to generate an individual analytical model M2k(t+1) for the next round. Specifically, for example, the server S integrates the individual analytical model learning results 421-1t, 422-1t, and 423-1t to generate an individual analytical model M21(t+1). The server S also integrates the individual analytical model learning results 421-2t, 422-2t, and 423-2t to generate an individual analytical model M22(t+1). The server S also integrates the individual analytical model learning results 421-3t, 422-3t, and 423-3t to generate an individual analytical model M23(t+1).

[0067] The server S calculates the individual analytical model M2j(t+1) or its integrated model parameters θ j t+1 to the client terminal Ck. As a result, the learning target individual analytical model M2j-kt in the next round of federated learning FL2 is set in the client terminal Ck.

[0068] Specifically, for example, the server S may generate an individual analytical model M21(t+1) or its integrated model parameters θ1 t+1to the client terminals C1 to C3. Using this received information, client terminal C1 sets a learning object individual analytical model M21-1t in the next round of associative learning FL2, client terminal C2 sets a learning object individual analytical model M22-1t in the next round of associative learning FL2, and client terminal C3 sets a learning object individual analytical model M23-1t in the next round of associative learning FL2.

[0069] In addition, the server S calculates the individual analytical model M22(t+1) or its integrated model parameters θ2 t+1 to client terminals C1 to C3. Using this received information, client terminal C1 sets a learning object individual analytical model M21-2t in the next round of associative learning FL2, client terminal C2 sets a learning object individual analytical model M22-2t in the next round of associative learning FL2, and client terminal C3 sets a learning object individual analytical model M23-2t in the next round of associative learning FL2.

[0070] In addition, the server S calculates the individual analytical model M23(t+1) or its integrated model parameters θ3 t+1 to client terminals C1 to C3. Using this received information, client terminal C1 sets a learning object individual analytical model M21-3t in the next round of associative learning FL2, client terminal C2 sets a learning object individual analytical model M22-3t in the next round of associative learning FL2, and client terminal C3 sets a learning object individual analytical model M23-3t in the next round of associative learning FL2.

[0071] In this way, federated learning FL2 is repeatedly executed. When the number of executions t reaches a predetermined threshold T2, or when the accuracies of all of the individual analytical models M21(t+1) to M2K(t+1) reach the target accuracy, the server S ends federated learning FL2, outputs the latest individual analytical model M2k(t+1) as the individual analytical model M2k, and transmits it to the client terminal Ck.

[0072] Specifically, for example, the server S outputs the individual analytical model M21(t+1) as the individual analytical model M21 and transmits it to the client terminal C1. The server S also outputs the individual analytical model M22(t+1) as the individual analytical model M22 and transmits it to the client terminal C2. The server S also outputs the individual analytical model M23(t+1) as the individual analytical model M23 and transmits it to the client terminal C3.

[0073] <Learning weight calculation 40k> Fig. 5 is an explanatory diagram showing a specific example of the learning weight calculation 40k (401 to 403) shown in Fig. 4. In Fig. 5, the learning data set D1 has D11 and D12 as data samples i, the learning data set D2 has D21 and D22 as data samples i, and the learning data set D3 has D31 and D32 as data samples i.

[0074] The client terminal Cj inputs the explanatory variables 301 of the learning data set Dj into the integrated similarity calculation model M1 and calculates the predicted probability [p j ] is calculated. j ], the CID=k column is the predicted probability [p j k ] is a column vector showing the predicted probability [p ji ] is a row vector indicating the predicted probability that a data sample i in the training data set Dj is similar to each training data set Dk.

[0075] Specifically, for example, when j=1, the client terminal C1 inputs the explanatory variables 301 of the learning data set D1 into the integrated similarity calculation model M1 to calculate the predicted probability [p1]. By inputting the explanatory variables 301 of the data sample D11 into the integrated similarity calculation model M1, the predicted probability [p 11 ] is calculated, and by inputting the explanatory variables 301 of the data sample D12 into the integrated similarity calculation model M1, the predicted probability [p 12] is calculated. In addition, the column of predicted probabilities [p1] (CID=1 to 3) indicates how similar each data sample i of the training data set D1 is to the training data sets D1 to D3. 1 ], [p1 2 ], [p1 3 ]. The client terminal C1 uses the predicted probability [p1 1 ], [p1 2 ], [p1 3 ], the learning weight w1 1 , w1 2 , w1 3 Calculate.

[0076] Furthermore, when j=2, the client terminal C2 inputs the explanatory variables 301 of the learning data set D2 into the integrated similarity calculation model M1 to calculate the predicted probability [p2]. By inputting the explanatory variables 301 of the data sample D21 into the integrated similarity calculation model M1, the predicted probability [p 21 ] is calculated, and by inputting the explanatory variables 301 of the data sample D22 into the integrated similarity calculation model M1, the predicted probability [p 22 ] is calculated. In addition, the column of CID (1 to 3) of the predicted probability [p2] indicates how similar each data sample i of the training data set D2 is to the training data sets D1 to D3. 1 ], [p2 2 ], [p2 3 ]. The client terminal C2 uses the predicted probability [p2 1 ], [p2 2 ], [p2 3 ], the learning weight w2 1 , w2 2 , w2 3 Calculate.

[0077] Furthermore, when j=3, the client terminal C3 inputs the explanatory variables 301 of the learning data set D3 into the integrated similarity calculation model M1 to calculate the predicted probability [p3]. By inputting the explanatory variables 301 of the data sample D31 into the integrated similarity calculation model M1, the predicted probability [p31 ] is calculated, and by inputting the explanatory variables 301 of the data sample D32 into the integrated similarity calculation model M1, the predicted probability [p 32 ] is calculated. In addition, the column of CID (1 to 3) of the predicted probability [p3] indicates how similar each data sample i of the training data set D3 is to the training data sets D1 to D3. 1 ], [p3 2 ], [p3 3 ]. The client terminal C3 uses the predicted probability [p3 1 ], [p3 2 ], [p3 3 ], the learning weight w3 1 , w3 2 , w3 3 Calculate.

[0078] Below, we use the learning weights w in the federated learning FL2 of the individual analysis model. j k This specifies the formula to be used in the calculation of

[0079]

number

[0080] W on the left side ji k is the learning weight applied to the similarity relationship between data sample i of the learning data set Dj and the learning data set Dk. ji k ] is the predicted probability that data sample i in the training data set Dj is similar to the training data set Dk. The learning weights w ji k The set of learning weights w j k is.

[0081] <Model learning by weighting 41kt> Next, a specific example of the weighted model learning 41kt (411t to 413t) shown in FIG. 4 will be described.

[0082] 6 is an explanatory diagram showing a specific example of the weighted model learning 411t shown in FIG. 4. The client terminal C1 receives the explanatory variables 203 and the objective variables 204 of the learning data set D1 and the learning weights w1 1 and updates the learning target individual analytical model M21-1t using the above, and outputs the individual analytical model learning result 421-1t.

[0083] The client terminal C2 uses the explanatory variables 203 and the objective variables 204 of the learning data set D2 and the learning weights w2 1 and updates the learning target individual analytical model M22-1t using the above, and outputs the individual analytical model learning result 422-1t.

[0084] The client terminal C3 uses the explanatory variables 203 and the objective variables 204 of the learning dataset D3 and the learning weights w3 1 The learning object individual analytical model M23-1t is updated using the above and the individual analytical model learning result 423-1t is output.

[0085] The server S integrates the individual analytical model learning results 421-1t, 422-1t, and 423-1t in the integrated learning 430t to generate the next individual analytical model M21(t+1). The server S generates the individual analytical model M21(t+1) or its integrated model parameters θ1 t+1 to client terminals C1 to C3. Using this received information, client terminal C1 sets an individual analytical model M21-1t to be learned in the next round of federated learning FL2, client terminal C2 sets an individual analytical model M22-1t to be learned in the next round of federated learning FL2, and client terminal C3 sets an individual analytical model M23-1t to be learned in the next round of federated learning FL2. The individual analytical model M21 is the individual analytical model of client terminal C1 that was finalized at the end of federated learning FL2.

[0086] 7 is an explanatory diagram showing a specific example of the weighted model learning 412t shown in FIG. 4. The client terminal C1 receives the explanatory variables 203 and the objective variables 204 of the learning data set D1 and the learning weights w1 2 and updates the learning target individual analytical model M21-2t using the above, and outputs the individual analytical model learning result 421-2t.

[0087] The client terminal C2 uses the explanatory variables 203 and the objective variables 204 of the learning data set D2 and the learning weights w2 2 and updates the learning target individual analytical model M22-2t using the above, and outputs the individual analytical model learning result 422-2t.

[0088] The client terminal C3 uses the explanatory variables 203 and the objective variables 204 of the learning dataset D3 and the learning weights w3 2 The learning target individual analytical model M23-2t is updated using the above and the individual analytical model learning result 423-2t is output.

[0089] The server S integrates the individual analytical model learning results 421-2t, 422-2t, and 423-2t in the integrated learning 430t to generate the next individual analytical model M22(t+1). The server S generates the individual analytical model M22(t+1) or its integrated model parameters θ2 t+1 to client terminals C1 to C3. Using this received information, client terminal C1 sets an individual analytical model M21-2t to be learned in the next round of federated learning FL2, client terminal C2 sets an individual analytical model M22-2t to be learned in the next round of federated learning FL2, and client terminal C3 sets an individual analytical model M23-2t to be learned in the next round of federated learning FL2. The individual analytical model M22 is the individual analytical model of client terminal C2 that was finalized at the end of federated learning FL2.

[0090] 8 is an explanatory diagram showing a specific example of the weighted model learning 413t shown in FIG. 4. The client terminal C1 calculates the explanatory variables 203 and the objective variables 204 of the learning data set D1 and the learning weights w1 3and updates the learning target individual analytical model M21-3t using the above, and outputs the individual analytical model learning result 421-3t.

[0091] The client terminal C2 uses the explanatory variables 203 and the objective variables 204 of the learning data set D2 and the learning weights w2 3 The learning target individual analytical model M22-3t is updated using the above and the individual analytical model learning result 422-3t is output.

[0092] The client terminal C3 uses the explanatory variables 203 and the objective variables 204 of the learning dataset D3 and the learning weights w3 3 The learning target individual analytical model M23-3t is updated using the above and the individual analytical model learning result 423-3t is output.

[0093] The server S integrates the individual analytical model learning results 421-3t, 422-3t, and 423-3t in the integrated learning 430t to generate the next individual analytical model M23(t+1). The server S generates the individual analytical model M23(t+1) or its integrated model parameters θ3 t+1 to client terminals C1 to C3. Using this received information, client terminal C1 sets an individual analytical model M21-3t to be learned in the next round of federated learning FL2, client terminal C2 sets an individual analytical model M22-3t to be learned in the next round of federated learning FL2, and client terminal C3 sets an individual analytical model M23-3t to be learned in the next round of federated learning FL2. Individual analytical model M23 is the individual analytical model of client terminal C3 that was finalized at the end of federated learning FL2.

[0094] Below we define the formulas used in federated learning FL2 of the individual analysis model.

[0095] [y j ]=f(x j ;θ j )···(10)

[0096] The above equation (10) defines the individual analytical model M2j. xj is the explanatory variable 203 of the training data set Dj, and θ j is the integrated model parameter of the individual analysis model M2j. [y j ] is the predicted value of the objective variable 204 of the training data set Dj.

[0097]

number

[0098] The above formula (11) is the loss function F(θ j kt ) is specified. ji is the objective variable 204 of a data sample i in the training data set Dj, and [y ji ] is the predicted value. [y ji ] is [y j ]. w ji k is the learning weight calculated by the above equation (9).

[0099]

number

[0100] The above formula (12) is the model parameter θ j kt The gradient of ga j kt The gradient ga j kt is calculated on the client terminal Cj.

[0101]

number

[0102] The above formula (13) is the model parameter θ j ktThis is a calculation formula for updating the weighted model learning 41jt of the client terminal Cj. j t As shown in the above formula (14), P of the client terminal Ck in the t-th federated learning FL2 j kt It is the sum of P j kt As shown in the above formula (15), the predicted probability [p ji k ] t is the sum of the above.

[0103] The client terminal Cj uses the updated model parameters θ j 1(t+1) ~θ j 3(t+1) The server S transmits the individual analysis model learning results 421-1t (updated model parameters θ1 1(t+1) ), Individual analysis model learning result 422-1t (updated model parameters θ2 1(t+1) ), individual analysis model learning results 423-1t (updated model parameters θ3 1(t+1) ) are integrated, for example, by averaging, to obtain the updated integrated model parameters θ1 for the client terminal C1. (t+1) is calculated and an individual analysis model M21(t+1) is generated.

[0104] In addition, the server S generates the individual analysis model learning result 421-2t (updated model parameters θ2 1(t+1) ), individual analysis model learning results 422-2t (updated model parameters θ2 2(t+1) ), Individual analysis model learning results 423-2t (updated model parameters θ3 2(t+1) ) are integrated, for example, by averaging, to obtain the updated integrated model parameters θ2 for the client terminal C2. (t+1) is calculated to generate the individual analysis model M22(t+1).

[0105] In addition, the server S generates the individual analysis model learning result 421-3t (updated model parameters θ3 1(t+1) ), Individual analysis model learning results 422-3t (updated model parameters θ2 3(t+1) ), Individual analysis model learning results 423-3t (updated model parameters θ3 3(t+1) ) are integrated, for example, by averaging, to obtain the updated integrated model parameters θ3 for the client terminal C2. (t+1) is calculated to generate the individual analysis model M23(t+1).

[0106]

number

[0107] The above formula (16) is the gradient ga from the client terminal Cj as the individual analysis model learning result 42j-kt. j kt and predicted probability [p ji k ] t When receiving the tth federated learning FL2, each model parameter θ j kt This is a calculation formula for updating the parameter .DELTA..times ...

[0108] The above formula (17) is the gradient ga from the client terminal Cj as the individual analysis model learning result 42j-kt. j kt and predicted probability [p ji k ] t When receiving the integrated model parameter θ j t This is a calculation formula for updating the integrated model parameter θ1 after updating, and is executed by the integrated learning 430t of the server S. Specifically, for example, the server S updates the integrated model parameter θ1 after updating. t+1 The individual analysis model M21(t+1) is generated using the updated integrated model parameters θ2 t+1The individual analysis model M22(t+1) is generated using the updated integrated model parameters θ3 t+1 The individual analysis model M23(t+1) is generated using

[0109] <Associative learning sequence> 9 is a sequence diagram of federated learning in the federated learning system 100. First, prior to federated learning FL1 and FL2, the server S transmits a client ID 201 to each client terminal Ck (step S901). The client terminal Ck associates the client ID 201 with the learning dataset Dk.

[0110] Next, the associative learning system 100 executes associative learning FL1 of the similarity calculation model (step S902), and executes associative learning FL2 of the individual analysis model (step S903).

[0111] In the federated learning FL1 of the similarity calculation model (step S902), the server S transmits the similarity calculation model information (integrated similarity calculation model M1(t+1) or its integrated model parameters φ t+1 ) is sent (step S921).

[0112] The client terminal Ck learns the learning target similarity calculation model using the learning data set Dj and the similarity calculation model information (step S922).

[0113] The client terminal Ck transmits the learning result 32jt obtained in step S922 to the server S (step S923).

[0114] The server S executes the integrated learning 330t of the similarity calculation model (step S924). The server S determines whether or not the termination condition of the integrated learning 330t of the similarity calculation model (step S924) is met (step S925).

[0115] If the termination condition is not satisfied, the server S transmits the updated similarity calculation model information to the client terminal Ck (step S921).

[0116] If the termination condition is satisfied, the server S transmits the updated similarity calculation model information to the client terminal Ck (step S926). As a result, the client terminal Ck sets the integrated similarity calculation model M1 using the updated similarity calculation model information, and terminates the federated learning FL1 of the similarity calculation model (step S902).

[0117] In the federated learning FL2 of the individual analysis model (step S903), the client terminal Cj executes the learning weight calculation 40j using the learning data set Dj and the integrated similarity calculation model M1 to calculate the learning weight wj (step S930).

[0118] In addition, the server S transmits the individual analytical model information (the individual analytical model M2j(t+1) or its integrated model parameters θ j t+1 ) is sent (step S931).

[0119] The client terminal Cj executes the weighted model learning 41jt and transmits the individual analysis model learning result 42j-kt to the server S (step S933).

[0120] The server S uses the individual analytical model learning results 42j-kt to perform integrated learning 430t of the individual analytical models (step S934).

[0121] The server S determines whether or not the termination condition for the integrated learning 430t of the individual analytical model (step S934) is met (step S925).

[0122] If the termination condition is not satisfied, the server S transmits the updated individual analytical model information to the client terminal Ck (step S931).

[0123] If the termination condition is met, the server S transmits the updated individual analytical model information to the client terminal Ck (step S936). As a result, the client terminal Ck generates an individual analytical model M2j using the updated individual analytical model information and terminates the individual analytical model federated learning FL2 (step S903).

[0124] <Display screen example> 10 is an explanatory diagram showing an example of a management screen. The management screen 1000 is displayed on the server S. The management screen 1000 displays a trained model list 1001, a model parameter list 1002, and a client learning contribution list 1003. The trained model list 1001 is table data that associates the client ID 201 with the ID of the individual analysis model M2j (individual analysis model ID).

[0125] The model parameter list 1002 includes the model parameters θ of the individual analysis model M2j. j The client learning contribution list 1003 is table data that associates the client ID 201 with the contribution 1030 of the client terminal Cj. The contribution 1030 is calculated by the following formula. The contribution 1030 is an index value that indicates how much the learning dataset Dk of the client terminal Ck contributes to the learning of the individual analysis model.

[0126]

number

[0127] R on the left side of the above formula (18) j k is the contribution 1030 of the client terminal Cj. j is the P of the client terminal Ck at the end of federated learning FL2. j k (See equation (14) above.) P j k is the predicted probability [p j k] (see formula (14) above). The contribution 1030 of the client terminal Cj is calculated by the server S.

[0128] 11 is an explanatory diagram showing an example of an information display screen. The information display screen 1100 is a screen that displays individual analysis model information for a client terminal Cj, and is displayed on the server S or the client terminal Cj. FIG. 11 shows an example where j=1.

[0129] The information display screen 1100 displays the client learning contribution list 1003, a data sample i (D11 in FIG. 11 as an example) designated from the learning dataset D1 of the client terminal C1, and a predicted probability distribution 1101 of the designated data sample D11 of the client terminal C1. Note that the designation of the data sample i is performed by the server S or the user operation of the client terminal C1.

[0130] The horizontal axis of the predicted probability distribution 1101 is the client ID 201, and the vertical axis is the predicted probability between the specified data sample D11 and the learning data set Dk identified by the client ID 201. The circular dots are data samples i in the learning data set D1 of the client terminal C1, and indicate the predicted probability with the client terminal Ck. Note that the data sample i specified by the cursor 1102 is the data sample D11, and indicates the predicted probability with the learning data set D1.

[0131] <Example of hardware configuration of server S and client terminal Ck> FIG. 12 is a block diagram showing an example of the hardware configuration of the server S and the client terminal Ck (hereinafter, computer 1200). The computer 1200 includes a processor 1201, a storage device 1202, an input device 1203, an output device 1204, and a communication interface (communication IF) 1205. The processor 1201, the storage device 1202, the input device 1203, the output device 1204, and the communication IF 1205 are connected via a bus 1206. The processor 1201 controls the computer 1200. The storage device 1202 serves as a working area for the processor 1201. The storage device 1202 is a non-transitory or temporary recording medium that stores various programs and data. Examples of the storage device 1202 include a read-only memory (ROM), a random access memory (RAM), a hard disk drive (HDD), and a flash memory. The input device 1203 inputs data. The input device 1203 includes, for example, a keyboard, a mouse, a touch panel, a numeric keypad, a scanner, a microphone, and a sensor. The output device 1204 outputs data. The output device 1204 includes, for example, a display, a printer, and a speaker. The communication IF 1205 connects to a network and transmits and receives data.

[0132] As described above, according to this embodiment, it is possible to provide an individual analysis model M2k suitable for each client terminal Ck participating in the federated learning. Furthermore, when generating the integrated similarity calculation model M1, the client terminal Ck can predict the similarity with other learning data sets Dk without passing the data sample i to other client terminals Ck or the server S, thereby preventing the data sample i itself from being leaked.

[0133] In the above-described embodiment, server S participated in federated learning FL1 and FL2. However, server S may participate only in federated learning FL2 without participating in federated learning FL1. In this case, client terminal Ck may obtain the results of federated learning FL1 (integrated similarity calculation model M1 or learning weights wk) from the other servers. Conversely, server S may participate in federated learning FL1 but not in federated learning FL2. In this case, client terminal Ck obtains the results of federated learning FL1 (integrated similarity calculation model M1 or learning weights wk) from server S. In this case, integrated learning 430t will be performed by the other servers.

[0134] Furthermore, the server S can transmit a program to be executed by the client terminal Ck as, for example, a container to the client terminal Ck.

[0135] Furthermore, the federated learning system 100 according to the above-described embodiment can also be configured as follows [1] to

[11] .

[0136] [1] The federated learning system 100 includes a plurality of client terminals C1 to C3, each having a learning data set D1 to D3, and a server S that can communicate with the plurality of client terminals C1 to C3. The federated learning system 100 performs federated learning in which each of the plurality of client terminals C1 to C3 learns a model using the learning data set D1 to D3, and the server S uses the learning results to repeatedly integrate the models of the plurality of client terminals C1 to C3.

[0137] The learning data set Dk has one or more data samples i including a client ID 201 that identifies the client terminal Ck, a first explanatory variable 203, and a first objective variable 204.

[0138] The federated learning system 100 executes a calculation process (learning weight calculation 401-403) by each of the plurality of client terminals C1-C3, and also executes a first federated learning process (FL2) that repeats a first learning process (weighted model learning 411t-413t) by each of the plurality of client terminals C1-C3 and a first update process (integrated learning 330t) by the server S until a first termination condition is met.

[0139] In the calculation process (learning weight calculation 401 to 403), each of the plurality of client terminals C1 to C3 inputs the data sample i into a similarity calculation model M1 that calculates the similarity between the data sample i and the plurality of learning data sets D1 to D3, thereby calculating the similarity (prediction probability [p ji k ] t ) is calculated.

[0140] In the first learning process (weighted model learning 411t to 413t), each of the plurality of client terminals C1 to C3 calculates a predicted value of the first dependent variable 204 from the first explanatory variable 203 using an individual analysis model (learning target individual analysis models M21-1t, M22-1t, M23-1t), and calculates a specific similarity (prediction probability [p ji 1 ] t ), and the individual analytical models (learning target individual analytical models M21-1t, M22-1t, M23-1t) are trained based on the above.

[0141] In the first integration process (integrated learning 330t), the server S integrates a plurality of first learning results (421-1t, 422-1t, 423-1t) from the plurality of client terminals C1 to C3 by the first learning process (weighted model learning 411t to 413t), and generates first integration information (integrated individual analytical model M21(t+1) or integrated model parameter θ1 t+1 )

[0142] [2] In the federated learning system 100 described in [1] above, in the first integrated process (integrated learning 330t), the server S continues to process the first integrated information (integrated individual analysis model M21(t+1) or integrated model parameter θ1 t+1 ) to each of the plurality of client terminals C1 to C3.

[0143] In the first learning process (weighted model learning 411t to 413t), each of the plurality of client terminals C1 to C3 uses the first integrated information (integrated individual analysis model M21(t+1) or integrated model parameter θ1 t+1 ) to update the individual analysis models (learning target individual analysis models M21-1t, M22-1t, M23-1t), and to use the first explanatory variable 203, the first objective variable 204, and the specific similarity (prediction probability [p ji 1 ] t ), and the updated individual analytical models (learning target individual analytical models M21-1t, M22-1t, M23-1t) are learned based on the above.

[0144] [3] In the federated learning system 100 described in [1] above, in the first integrated process (integrated learning 330t), when the first termination condition is satisfied, the server S outputs the first integrated information (integrated individual analysis model M21(t+1) or integrated model parameter θ1 t+1 ) to a specific client terminal having the specific training data set.

[0145] [4] In the federated learning system 100 of [1] above, in the calculation process (learning weight calculation 401-403), each of the plurality of client terminals C1-C3 inputs a combination of the first explanatory variable 203 and the first objective variable 204 of the data sample i into the similarity calculation model (learning target similarity calculation models M11t-M13t) as a second explanatory variable 301, thereby calculating the similarity (predicted probability [p ji k ] t ) is calculated.

[0146] [5] In the federated learning system 100 of [1] above, in the calculation process (learning weight calculation 401 to 403), each of the plurality of client terminals C1 to C3 calculates the specific similarity (predicted probability [p ji 1 ] t ) according to the learning weight (w1 1 , w2 1 , w3 1 ) is calculated.

[0147] In the first learning process (weighted model learning 411t to 413t), each of the plurality of client terminals C1 to C3 uses the individual analysis model (learning target individual analysis models M21-1t, M22-1t, M23-1t), the first explanatory variable 203, the first objective variable 204, and a specific similarity (prediction probability [p ji 1 ] t ) according to the learning weight (w1 1 , w2 1 , w3 1 ), and the individual analytical models (learning target individual analytical models M21-1t, M22-1t, M23-1t) are trained based on the above.

[0148] [6] Prior to the first associative learning process (FL2), the associative learning system 100 of [1] above executes a second associative learning process (FL1) in which a second learning process (model learning 311t to 313t) by each of the plurality of client terminals C1 to C3 and a second integrated process (integrated learning 330t) by the server S are repeated until a second termination condition is met.

[0149] In the second learning process (model learning 311t to 313t), each of the multiple client terminals C1 to C3 learns the learning object similarity calculation models M11t to M13t using a combination of the first explanatory variable 203 and the first objective variable 204 as a second explanatory variable 301 and the client ID 201 as a second objective variable 302.

[0150] In the second integration process (integrated learning 330t), the server S integrates the second learning results 321t to 323t of the learning object similarity calculation models M11t to M13t from the plurality of client terminals C1 to C3 by the second learning process (model learning 311t to 313t), and generates second integration information (integrated similarity calculation model M1(t+1) or its integrated model parameter φ t+1 )

[0151] [7] In the federated learning system 100 described in [6] above, in the second integrated process (integrated learning 330t), the server S generates the second integrated information (integrated similarity calculation model M1(t+1) or its integrated model parameter φ t+1 ) to each of the plurality of client terminals C1 to C3.

[0152] In the second learning process (model learning 311t to 313t), each of the plurality of client terminals C1 to C3 acquires the second integrated information (an integrated similarity calculation model M1(t+1) or its integrated model parameters φ t+1) and trains the updated learning object similarity calculation models M11t to M13t based on the second explanatory variable 301 and the second objective variable 302.

[0153] [8] In the federated learning system 100 described in [6] above, in the second learning process (model learning 311t to 313t), if each of the client terminals C1 to C3 satisfies the second termination condition, it sets the updated learning object similarity calculation model M11t to M13t to the similarity calculation model M1.

[0154] [9] In the federated learning system 100 described in [3] above, the server S receives the first integrated information (the integrated model parameters θ j ) is output (model parameter list 1002).

[0155]

[10] In the federated learning system 100 described in [3] above, the server S calculates the specific similarity (predicted probability [p ji 1 ]) and the similarity (prediction probability [p ji k ]) and calculates a contribution 1030 indicating how much the specific learning data set D1 contributes to the learning of the integrated individual analysis model M21.

[0156]

[11] The federated learning system 100 includes a plurality of client terminals C1 to C3, each having a learning data set D1 to D3, and a server S that can communicate with the plurality of client terminals C1 to C3. The federated learning system 100 performs federated learning in which each of the plurality of client terminals C1 to C3 learns a model using the learning data sets D1 to D3, and the server S uses the learning results to repeatedly integrate the models of each of the plurality of client terminals C1 to C3.

[0157] The learning data set Dk has one or more data samples i including a client ID 201 that identifies the client terminal Ck, a first explanatory variable 203, and a first objective variable 204.

[0158] The federated learning system 100 executes a second federated learning process (FL1) that repeats a second learning process (model learning 311t to 313t) by each of the multiple client terminals C1 to C3 and a second integrated process (integrated learning 330t) by the server S until a second termination condition is met.

[0159] In the second learning process (model learning 311t to 313t), each of the multiple client terminals C1 to C3 learns a learning object similarity calculation model M11t to M13t using a combination of the first explanatory variable 203 and the first objective variable 204 as a second explanatory variable 301 and the client ID 201 as a second objective variable 302.

[0160] In the second integration process (integrated learning 330t), the server S integrates second learning results 321t to 323t of the learning object similarity calculation models M11t to M13t from the plurality of client terminals C1 to C3 by the second learning process (model learning 311t to 313t), and generates second integration information (integrated similarity calculation model M1(t+1) or its integrated model parameter φ t+1 )

[0161] The present invention is not limited to the above-described embodiments, and includes various modifications and equivalent configurations within the spirit and scope of the appended claims. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to configurations including all of the described configurations. Furthermore, part of the configuration of one embodiment may be replaced with the configuration of another embodiment. Furthermore, the configuration of another embodiment may be added to the configuration of one embodiment. Furthermore, part of the configuration of each embodiment may be added to, deleted from, or replaced with other configurations.

[0162] Furthermore, the aforementioned configurations, functions, processing units, processing means, etc. may be realized in part or in whole in hardware, for example by designing them as integrated circuits, or may be realized in software by having a processor interpret and execute a program that realizes each function.

[0163] Information such as programs, tables, files, etc. that realize each function can be stored in storage devices such as memory, hard disks, SSDs (Solid State Drives), or recording media such as IC (Integrated Circuit) cards, SD cards, and DVDs (Digital Versatile Discs).

[0164] In addition, the control lines and information lines shown are those that are considered necessary for explanation, and do not necessarily represent all the control lines and information lines that are necessary for implementation. In reality, it can be assumed that almost all components are interconnected. [Explanation of symbols]

[0165] 100 Federated Learning System 203 explanatory variables 204 Objective Variable 301 explanatory variables 302 Objective variable 330t Integrated Learning 411t~413t model update 430t Integrated Learning 1000 management screen 1100 Information display screen C1~C3 Client terminals D1~D3 training dataset M1 Integrated Similarity Computation Model M21~M23 Individual Analysis Model

Claims

1. A federated learning system comprising: a plurality of client terminals each having a learning data set; and a server capable of communicating with the plurality of client terminals, wherein each of the plurality of client terminals learns a model using the learning data set; and the server repeats a process of integrating the models of the plurality of client terminals using a learning result, The learning data set has one or more data samples including a client ID that identifies the client terminal, a first explanatory variable, and a first objective variable; executing a calculation process by each of the plurality of client terminals, and executing a first combined learning process in which a first learning process by each of the plurality of client terminals and a first integrated process by the server are repeated until a first end condition is satisfied; In the calculation process, each of the plurality of client terminals calculates a similarity between the data sample and the plurality of learning data sets by inputting the data sample into a similarity calculation model that calculates a similarity between the data sample and the plurality of learning data sets; In the first learning process, each of the plurality of client terminals learns an individual analysis model that calculates a predicted value of the first objective variable from the first explanatory variable, and learns the individual analysis model based on the first explanatory variable, the first objective variable, and a specific similarity between the first explanatory variable, the first objective variable, and a specific learning data set calculated by each of the plurality of client terminals by the calculation process; In the first integration process, the server integrates a plurality of first learning results obtained by the first learning process from the plurality of client terminals, and generates first integration information regarding an integrated individual analysis model obtained by integrating the individual analysis models of the plurality of client terminals. A federated learning system characterized by:

2. 2. The federated learning system of claim 1, In the first integration process, the server transmits the first integration information to each of the plurality of client terminals until the first end condition is satisfied; In the first learning process, each of the plurality of client terminals updates the individual analysis model with the first integrated information, and learns the updated individual analysis model based on the first explanatory variable, the first objective variable, and the specific similarity. A federated learning system characterized by:

3. 2. The federated learning system of claim 1, In the first integration process, when the first termination condition is satisfied, the server transmits the first integration information to a specific client terminal having the specific learning data set. A federated learning system characterized by:

4. 2. The federated learning system of claim 1, In the calculation process, each of the plurality of client terminals calculates the similarity by inputting a combination of the first objective variable and the first explanatory variable of the data sample as a second explanatory variable into the similarity calculation model. A federated learning system characterized by:

5. 2. The federated learning system of claim 1, In the calculation process, each of the plurality of client terminals calculates a learning weight according to the specific similarity; In the first learning process, each of the plurality of client terminals learns the individual analysis model based on the individual analysis model, the first explanatory variable, the first objective variable, and a learning weight according to a specific similarity calculated in each of the plurality of client terminals by the calculation process. A federated learning system characterized by:

6. 2. The federated learning system of claim 1, executing a second combined learning process in which, prior to the first combined learning process, a second learning process by each of the plurality of client terminals and a second integration process by the server are repeated until a second end condition is satisfied; In the second learning process, each of the plurality of client terminals learns a learning object similarity calculation model by using a combination of the first explanatory variable and the first objective variable as a second explanatory variable and the client ID as a second objective variable; In the second integration process, the server integrates second learning results of the learning target similarity calculation model from the plurality of client terminals by the second learning process, and generates second integration information regarding an integrated similarity calculation model obtained by integrating the similarity calculation models of the plurality of client terminals. A federated learning system characterized by:

7. The federated learning system according to claim 6, In the second integration process, the server transmits the second integration information to each of the plurality of client terminals; In the second learning process, each of the plurality of client terminals updates the learning object similarity calculation model with the second integrated information, and learns the updated learning object similarity calculation model based on the second explanatory variable and the second objective variable. A federated learning system characterized by:

8. The federated learning system according to claim 6, In the second learning process, when the second end condition is satisfied, each of the plurality of client terminals sets the updated learning object similarity calculation model as the similarity calculation model. A federated learning system characterized by:

9. The federated learning system according to claim 3, The server, outputting the first integrated information at the end of the first combined learning process; A federated learning system characterized by:

10. The federated learning system according to claim 3, The server calculates a contribution degree indicating how much the specific learning data set contributes to the learning of the integrated individual analysis model based on the specific similarity degree and the similarity degree. A federated learning system characterized by:

11. A federated learning system comprising: a plurality of client terminals each having a learning data set; and a server capable of communicating with the plurality of client terminals, wherein each of the plurality of client terminals learns a model using the learning data set; and the server repeats a process of integrating the models of the plurality of client terminals using a learning result, The learning data set has one or more data samples including a client ID that identifies the client terminal, a first explanatory variable, and a first objective variable; executing a second combined learning process in which a second learning process by each of the plurality of client terminals and a second integrated process by the server are repeated until a second end condition is satisfied; In the second learning process, each of the plurality of client terminals learns a learning object similarity calculation model by using a combination of the first explanatory variable and the first objective variable as a first explanatory variable and the client ID as a second objective variable; In the second integration process, the server integrates second learning results of the learning object similarity calculation model from the plurality of client terminals by the second learning process, and generates second integration information regarding an integrated similarity calculation model in which the learning object similarity calculation models of the plurality of client terminals are integrated. A federated learning system characterized by:

12. A federated learning method for performing federated learning in which a federated learning system includes a plurality of client terminals each having a learning data set, and a server capable of communicating with the plurality of client terminals, the federated learning system repeating a process in which each of the plurality of client terminals learns a model using the learning data set, and the server uses a learning result to integrate the models of the plurality of client terminals, the method comprising: The learning data set has one or more data samples including a client ID that identifies the client terminal, a first explanatory variable, and a first objective variable; executing a calculation process by each of the plurality of client terminals, and executing a first combined learning process in which a first learning process by each of the plurality of client terminals and a first integrated process by the server are repeated until a first end condition is satisfied; In the calculation process, each of the plurality of client terminals calculates a similarity between the data sample and the plurality of learning data sets by inputting the data sample into a similarity calculation model that calculates a similarity between the data sample and the plurality of learning data sets; In the first learning process, each of the plurality of client terminals learns an individual analysis model that calculates a predicted value of the first objective variable from the first explanatory variable, and learns the individual analysis model based on the first explanatory variable, the first objective variable, and a specific similarity between the first explanatory variable, the first objective variable, and a specific learning data set calculated by each of the plurality of client terminals by the calculation process; In the first integration process, the server integrates a plurality of first learning results obtained by the first learning process from the plurality of client terminals, and generates first integration information regarding an integrated individual analysis model obtained by integrating the individual analysis models of the plurality of client terminals. A federated learning method comprising:

13. A federated learning system includes a plurality of client terminals each having a learning data set, and a server capable of communicating with the plurality of client terminals, the federated learning system repeating a process of each of the plurality of client terminals learning a model using the learning data set, and the server using a learning result to integrate the models of the plurality of client terminals, the federated learning program being executed by a processor of the client terminal, the federated learning system comprising: The learning data set has one or more data samples including a client ID that identifies the client terminal, a first explanatory variable, and a first objective variable; The processor, A calculation process of calculating similarities between the data sample and the multiple learning data sets by inputting the data sample into a similarity calculation model that calculates similarities between the data sample and the multiple learning data sets; execute an individual analysis model that calculates a predicted value of the first dependent variable from the first explanatory variable, and a first learning process that learns the individual analysis model based on the first explanatory variable, the first dependent variable, and a specific similarity with a specific learning data set calculated in each of the multiple client terminals by the calculation process; the server integrates a plurality of first learning results from the plurality of client terminals through the first learning process, and when the server receives from the server first integration information regarding an integrated individual analytical model obtained by integrating the individual analytical models of the plurality of client terminals, the server updates the individual analytical model with the first integration information and causes the calculation process and the first learning process to be repeatedly executed; A federated learning program characterized by:

Images