Association learning system, association learning method, and association learning program

Abstract

To provide an association learning system that realizes model integration through association learning that takes into account inference target data.SOLUTION: In an association learning system, a server takes an input dataset, inputs the input data set to a similarity calculation model, and calculates a first similarity between the input data set and a plurality of learning data sets; each of a plurality of client terminals inputs a data sample into the similarity calculation model to calculate a second similarity between the data sample and the plurality of learning data sets, performs similarity calculation between the first similarity and the second similarity to output a learning weight; and learns a first analytical model on the basis of the first analytical model that calculates a predicted value of a first objective variable from a first explanatory variable for the input data set, the first explanatory variables and the first objective variable of a learning data set, and a learning weight, and transmits a first learning result; and the server integrates the first learning results to generate a second analysis model that calculates a predicted value of the first objective variable from the first explanatory variable for the input data set..SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a federated learning system, a federated learning method, and a federated learning program for performing federated learning. [Background technology]

[0002] As healthcare information becomes more electronic, secondary use of healthcare information managed by local governments, medical institutions, individuals, etc. (hereafter referred to as clients) is increasing. In particular, federated learning, which allows learning of models in a distributed environment without centralizing and managing information on a server, is attracting attention from the perspective of protecting personal information.

[0003] The following Patent Document 1 discloses a machine learning system and method in federated learning, an integrated server, an information processing device, a program, and a method for creating an inference model. In the following Patent Document 1, each of multiple client terminals classifies data stored in a medical institution based on the data acquisition conditions, and classifies learning data for each data group acquired under the same or similar acquisition conditions. Each client terminal executes machine learning of a learning model for each learning data group classified by condition category, and transmits each learning result and condition information to an integrated server. The integrated server integrates the received learning results by condition category to create multiple master model candidates, and evaluates the inference accuracy of each master model candidate. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2021 / 079792 Summary of the Invention [Problem to be solved by the invention]

[0005] The above-mentioned Patent Document 1 describes a method of performing learning for each learning data group in which the quality of captured images used for learning is roughly homogenized. Therefore, an integrated model corresponding to the inference target data is not generated, and differences in distribution of the learning data group are not taken into consideration.

[0006] The present invention aims to realize model integration through associative learning that takes into account the data to be inferred. [Means for solving the problem]

[0007] A federated learning system according to one aspect of the invention disclosed in the present application includes a plurality of client terminals each having a learning dataset, and a server capable of communicating with the plurality of client terminals, and executes federated learning in which each of the plurality of client terminals learns a model using the learning dataset, and the server repeats a process of integrating the models of each of the plurality of client terminals using a learning result, wherein the learning dataset includes one or more data samples including a client ID that identifies the client terminal, a first explanatory variable, and a first objective variable, and the system executes a first calculation process by the server and a second calculation process by each of the plurality of client terminals, and executes a first federated learning process in which the first learning process by each of the plurality of client terminals and the first integration process by the server are repeated until a first termination condition is satisfied, and in the first calculation process, the server acquires an input dataset including one or more input data including a client ID that identifies the client terminal, a first explanatory variable, and a first objective variable, and in the first calculation process, the server acquires an input dataset including one or more input data including a client ID that identifies the client terminal, a first explanatory variable, and a first objective variable, and in the first calculation process, the server acquires an input dataset including one or more input data including a client ID that identifies the client terminal, a first explanatory variable, and a first objective variable, and In the second calculation process, each of the plurality of client terminals calculates a second similarity between the data sample and the plurality of learning data sets by inputting the input data set into the similarity calculation model that calculates a similarity between the data sample and the plurality of learning data sets. In the second calculation process, each of the plurality of client terminals executes a similarity calculation between the first similarity and the second similarity to output a learning weight. In the first learning process, each of the plurality of client terminals learns the first analysis model based on a first analysis model that calculates a predicted value of the first objective variable from the first explanatory variable for the input data set, the learning data set, the first explanatory variable and the first objective variable of the learning data set, and the learning weight, and transmits a first learning result to the server. In the first integration process, the server integrates the first learning results from the first learning process from the plurality of client terminals,and generating a second analysis model that calculates a predicted value of the first objective variable from the first explanatory variable for the input data set. Effect of the Invention

[0008] According to a representative embodiment of the present invention, model integration can be realized by federated learning that takes into account the inference target data. Problems, configurations, and effects other than those described above will become clear from the following description of the embodiments. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating an example of the associative learning according to the first embodiment. [Diagram 2] FIG. 2 is a diagram illustrating an example of a learning data set. [Diagram 3] FIG. 3 is an explanatory diagram showing an example of associative learning of a similarity calculation model. [Figure 4] FIG. 4 is an explanatory diagram showing an example of associative learning of an analysis model for input variables. [Diagram 5] FIG. 5 is an explanatory diagram showing a specific example of the learning weight calculation shown in FIG. [Figure 6] FIG. 6 is an explanatory diagram showing a specific example of model learning using the weighting shown in FIG. [Figure 7] FIG. 7 is a sequence diagram of the federated learning in the federated learning system according to the first embodiment. [Figure 8] FIG. 8 is a diagram illustrating an example of the associative learning according to the second embodiment. [Figure 9] FIG. 9 is an explanatory diagram showing an example of the federated learning of individual analysis models. [Figure 10] FIG. 10 is an explanatory diagram showing a specific example of the learning weight calculation shown in FIG. [Figure 11] FIG. 11 is an explanatory diagram showing a specific example of the weighted model learning 911t shown in FIG. [Figure 12] FIG. 12 is an explanatory diagram showing a specific example of the weighted model learning 912t shown in FIG. [Figure 13] FIG. 13 is an explanatory diagram showing a specific example of the weighted model learning 913t shown in FIG. [Figure 14] FIG. 14 is an explanatory diagram showing an example of the integration of analytical models for input variables. [Figure 15] FIG. 15 is a sequence diagram of the federated learning in the federated learning system according to the second embodiment. [Figure 16] FIG. 16 is an explanatory diagram showing an example of a management screen according to the first to third embodiments. [Figure 17] FIG. 17 is a block diagram of an example of a hardware configuration of the server and the client terminal according to the first to third embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS EXAMPLES

[0010] <Example of Associative Learning> FIG. 1 is an explanatory diagram showing an example of federated learning according to the first embodiment. The federated learning system 100 includes a server S and a plurality of client terminals C1 to C3 (three in FIG. 1 as an example). When there is no need to distinguish between them, they are represented as client terminals Ck (k=1, 2, 3, ..., K). The number of client terminals Ck is not limited to three, and may be two or more. In this example, K=3. The server S and the client terminals Ck are connected to each other so as to be able to communicate with each other via a network such as the Internet, a LAN (Local Area Network), or a WAN (Wide Area Network).

[0011] The client terminals C1 to C3 have learning data sets D1 to D3. When there is no need to distinguish between them, they are referred to as a learning data set Dk. The learning data set Dk is a combination of learning data that is an explanatory variable and answer data that is an objective variable. It is assumed that the learning data set Dk is prohibited from being taken out of the client terminal Ck or the base where the client terminal Ck is installed.

[0012] The client terminal Ck is a computer that provides a learning data set Dk to a prediction model to learn it individually, and transmits the learning results, such as model parameters of the learned prediction model or their gradients, to the server S each time learning is performed.

[0013] The server S is a computer that uses the learning results from the client terminals Ck to integrate the prediction models for the client terminals Ck to generate an integrated prediction model and transmits the integrated prediction model to the client terminal Ck. The client terminal Ck provides the learning data set Dk to the integrated prediction model from the server S to learn the prediction model. By repeating such learning, the federated learning system 100 executes federated learning.

[0014] In the first embodiment, the associative learning system 100 executes two types of associative learning: one is associative learning FL1 of a similarity calculation model, and the other is associative learning FL2 of an input variable analysis model.

[0015] The similarity calculation model federated learning FL1 is federated learning that generates an integrated similarity calculation model M1 that integrates the similarity calculation models from each client terminal Ck by executing the above-mentioned federated learning using the similarity calculation model as a prediction model. The similarity calculation model is, when focusing on the learning data set Dk (hereinafter, written as learning data set Dj to distinguish from Dk) of the learning target client terminal Ck (hereinafter, written as client terminal Cj to distinguish from Ck) among the client terminals C1 to CK, the i-th (i is 1≦i≦N j An integer that satisfies N j is a prediction model that calculates the similarity between each data sample (hereinafter referred to as data sample i) of the learning data set Dj (the total number of data samples in the learning data set Dj) and the learning data set Dk.

[0016] Specifically, for example, the similarity calculation model is a model that calculates a propensity score with a client ID that uniquely identifies a client terminal Ck as an assigned variable.

[0017] Hereinafter, k will be used for any client terminal and its learning data set, but j will be used for the client terminal to be trained and its learning data set.

[0018] By generating the integrated similarity calculation model M1 prior to the associative learning FL2 of the analytical model for input variables, it becomes possible to adjust the influence on the associative learning for each data sample i in the associative learning FL2 of the analytical model for input variables.

[0019] The associative learning FL2 of the analytical model for input variables is associative learning that generates an analytical model for input variables MQ using the integrated similarity calculation model M1 generated in the associative learning FL1, the input variable Q, and the learning dataset Dk. The input variable Q is a matrix consisting of r rows of input data to be inferred (r is an integer satisfying 1≦r≦R, R is an integer equal to or greater than 1). The details of the associative learning FL2 of the analytical model for input variables will be described later. When the input variable Q is input to the analytical model for input variables MQ, an inference IN is executed and an inference result A is obtained.

[0020] <Learning dataset Dk> 2 is an explanatory diagram showing an example of a learning dataset Dk. The learning dataset Dk has, as fields, a client ID (sometimes referred to as "CID" in the following drawings) 201, a data ID 202, an explanatory variable 203 (sometimes referred to as explanatory variable X), and a response variable 204 (sometimes referred to as response variable y). A combination of values ​​of each field in the same row becomes an entry that specifies one data sample. Each of the learning datasets D1 to D3 is, for example, a collection of data samples of a patient group for each hospital.

[0021] The client ID 201 is identification information that uniquely identifies the client terminal Ck. The value of the client ID 201 is expressed as Ck. The data ID 202 is identification information that uniquely identifies a data sample. The value of the data ID 202 is expressed as Dki. i is a number unique to the data sample. The data ID 202, for example, identifies a patient. The explanatory variables 203 are learning data used in the associative learning FL2 of the individual analysis model, and include one or more features x1, x2, ... (when these are not distinguished, they are simply referred to as feature x). The feature x is, for example, the height, weight, blood pressure, etc. of the patient identified by the data ID 202.

[0022] The objective variable 204 is the correct answer data used in the associative learning FL2 of the individual analysis model. The objective variable 204 indicates, for example, whether or not the patient identified by the data ID 202 has a disease. y=1 indicates the presence of a disease, and y=0 indicates the absence of a disease.

[0023] 1 is a combination of the values ​​of x1, x2, ... in the explanatory variables 203 that are the inference targets of the inference IN and the value of y in the objective variable 204. The input variable Qr may be the value of the explanatory variables 203 and the objective variable 204 in any of the data IDs 202 in the learning dataset Dk, or may be the value of the explanatory variables 203 and the objective variable 204 that are not in the learning dataset Dk.

[0024] <Associative learning of similarity calculation model FL1> 3 is an explanatory diagram showing an example of the associative learning FL1 of the similarity calculation model. In the associative learning FL1 of the similarity calculation model, a combination of the explanatory variable 203 and the objective variable 204 of the learning data set Dk becomes the explanatory variable 301, and the client ID 201 becomes the objective variable 302.

[0025] The server S has a base similarity calculation model (hereinafter, referred to as a base similarity calculation model) M10. The base similarity calculation model M10 may be an untrained neural network, and has model parameters φ such as weights and biases. k tmay be a trained neural network in which the above has been set. t is an ascending natural number starting from 1 indicating the number of times the associative learning FL1 of the similarity calculation model has been executed. The server S transmits the base similarity calculation model M10 to the client terminals C1 to C3.

[0026] If the client terminal Ck has an untrained neural network, the server S calculates the model parameters φ k t The client terminal Ck receives the model parameters φ k t A base similarity calculation model M10 may be constructed by setting the above to an untrained neural network.

[0027] The base similarity calculation model M10 becomes the learning object similarity calculation model M11t of the first associative learning FL1 in the client terminal Ck.

[0028] The client terminal Ck executes model learning 31kt (311t to 313t) in the t-th associative learning FL1. Specifically, for example, the client terminal Ck provides the explanatory variables 301 and the objective variables 302 of the learning data set D1 to the learning object similarity calculation model M1kt (M11t to M13t) and learns them individually. The client terminal Ck updates the model parameter φ k t or its gradient gs k t The learning result 32kt (321t to 323t) is sent to the server S.

[0029] The server S executes integrated learning 330t in the t-th associative learning FL1 using the learning result 32kt, and generates the next integrated similarity calculation model M1(t+1). Specifically, for example, the server S generates the integrated similarity calculation model M1(t+1) using the integrated result obtained by integrating the learning result 32kt, and calculates the integrated similarity calculation model M1(t+1) or its integrated model parameter φ t+1to the client terminal Ck. As a result, the learning object similarity calculation model M1kt in the next associative learning FL1 is set in the client terminal Ck.

[0030] In this way, the associative learning FL1 is repeatedly executed. When the execution count t reaches a predetermined threshold T1 or the accuracy of the integrated similarity calculation model M1t reaches a target accuracy, the server S ends the associative learning FL1, outputs the latest integrated similarity calculation model M1(t+1) as the integrated similarity calculation model M1, and transmits it to the client terminals C1 to C3.

[0031] The formulas used in the associative learning FL1 of the similarity calculation model are defined below.

[0032] [p j ] t =h(X,y;φ j t ) · · · (1)

[0033] The above formula (1) is a calculation formula that specifies the similarity calculation model, and is executed by the model learning 31jt of the client terminal Cj. The function h is a function of the explanatory variables 301, which are a combination of the explanatory variables 203 (X) and the objective variables 204 (y), and the model parameters φ j t The learning object similarity calculation model M1jt (M11t to M13t) is defined as follows. [p j ] t is a predicted probability indicating which learning data set Dk each data sample i in the learning data set Dj of the client terminal Cj is similar to in the t-th associative learning FL1.

[0034] Let N be the total number of data samples i in the training data set Dj. j Then, the predicted probability [p j ] t is N j ×K matrix. In other words, the row vector [p ji ] tare the predicted probabilities indicating the similarity between data sample i of the learning data set Dj and the learning data set D1, the predicted probability indicating the similarity between data sample i of the learning data set Dj and the learning data set D2, the predicted probability indicating the similarity between data sample i of the learning data set Dj and the learning data set D3, ..., the predicted probability indicating the similarity between data sample i of the learning data set Dj and the learning data set DK.

[0035] In addition, the predicted probability [p j ] t The kth column of the matrix j A column vector [p j k ] t is the predicted probability indicating the similarity between the first (i=1) data sample i in the learning data set Dj and the learning data set Dk, the predicted probability indicating the similarity between the second (i=2) data sample i in the learning data set Dj and the learning data set Dk, the predicted probability indicating the similarity between the third (i=3) data sample i in the learning data set Dj and the learning data set Dk, ..., N j th (i=N j ) is the predicted probability indicating the similarity between data sample i and the training data set Dk.

[0036]

number

[0037] The above formula (2) expresses the loss function H(φ j t ). ji is the similarity between the data sample i of the learning data set Dj and the learning data sets D1 to D3. ji has a range of, for example, 0.0 to 1.0, and a larger value indicates greater similarity.

[0038] For j=1, p ji= (1.0, 0, 0), which indicates that data sample i is a data sample in the training data set D1. For j = 2, p ji = (0, 1.0, 0), which indicates that data sample i is a data sample in the training data set D2. For j = 3, p ji = (1.0, 0, 0), indicating that the data sample i is a data sample in the training data set D3.

[0039] [p ji ] t is the predicted probability [p j ] t is a row vector that is a prediction probability indicating the degree to which the data sample i is similar to the learning data sets D1 to D3 in the matrix indicated by .

[0040] N j is the total number of data samples i in the training data set Dj. The function loss is the error function of data sample i, and the average value of the error function loss of each data sample i is the loss function H(φ j t )

[0041]

number

[0042] The above equation (3) is the model parameter φ j t The gradient of gs j t This is a formula that specifies the gradient gs, which is executed by the model learning 31jt of the client terminal Cj. η is the learning rate. J is the total number of client terminals Cj to be learned, and J=K. When applying the following formula (4), the client terminal Cj j t is sent to server S as the learning result 32jt.

[0043]

number

[0044] The above equation (4) is the integrated model parameter φ j t The integrated model parameters φ j t+1 This is a formula for updating the gradient gs from the client terminal Cj. j t The integrated model parameter φ in the first term on the right side of the above equation (4) is received as the learning result 32jt and the above equation (4) is executed. t is the integrated model parameter calculated as a result of executing the above formula (4) in the previous associative learning FL1.

[0045]

number

[0046] The above equation (5) is based on the model parameter φ j t The model parameter φ j t+1 This is a formula for updating the model parameter φ after updating, which is executed by the integrated learning 330t of the server S. This is executed by the model learning 31jt of the client terminal Cj. When applying the above formula (5), the client terminal Cj updates the model parameter φ after updating. j t+1 is sent to server S as the learning result 32jt.

[0047] The above formula (6) is the model parameter φ after updating the above formula (5). j t+1 Using the integrated model parameter φ t+1 This is a formula for calculating the above, and is executed by the integrated learning 330t of the server S. N is the total number of data samples i of the client terminals C1 to CJ.

[0048] The associative learning system 100 updates the integrated model parameter φ by using either the update method according to the above formula (4) or the update method according to formulas (5) or (6). t+1The server S calculates the integrated similarity calculation model M1t (the base similarity calculation model M10 when t=0) with the integrated model parameter φ t+1 and generate the integrated similarity calculation model M1(t+1).

[0049] The server S calculates the integrated similarity calculation model M1(t+1) or its integrated model parameters φ t+1 to the client terminals C1 to C3, thereby updating the above formula (1).

[0050] [p j ] t+1 =h(X,y;φ t+1 ) · · · (7)

[0051] The above formula (7) is a calculation formula that specifies the integrated similarity calculation model M1(t+1), which is generated by the integrated learning 330 of the server S and transmitted to the client terminal Ck, and becomes the learning target similarity calculation model M1kt in the next associative learning FL1.

[0052] [p j ] = h(X,y;φ) (8)

[0053] The above formula (8) is a calculation formula that specifies the integrated similarity calculation model M1 when associative learning FL1 is completed. Since it is the same formula as the above formula (7) at the end of associative learning FL1, "t+1" has been deleted from the above formula (7).

[0054] <Associative learning of analytical models for input variables FL2> 4 is an explanatory diagram showing an example of federated learning FL2 of the analysis model for input variables. The client terminals C1 to C3 execute learning weight calculations 401 to 403. Specifically, for example, the client terminal Cj inputs the explanatory variables 301 of the learning data set Dj to the integrated similarity calculation model M1 (the above formula (8)) generated in FIG. 3, thereby calculating a prediction probability [p j The client terminal Cj calculates the calculated predicted probability [pj ] and the input variable similarity QSr, the learning weights w j Calculate.

[0055] The server S calculates the input variable similarity QS by inputting the input variable Q to the integrated similarity calculation model M1 in the input variable similarity calculation 404. The input variable similarity QS is used in the learning weight calculations 401 to 403.

[0056] The server S also has a base analysis model for input variables (hereinafter, base analysis model) M20. The base analysis model M20 may be an untrained neural network, and has model parameters θ such as weights and biases. t It is also possible to use a trained neural network with the model parameters θ t are the model parameters generated in the t-th federated learning FL2 when the input variable Q is given. t is an ascending natural number starting from 1 indicating the number of times the federated learning FL2 of the analytical model for the input variable is executed. The server S transmits the base analytical model M20 to the client terminal Ck.

[0057] The base analytical model M20 becomes the analytical model M2j-1 for the learning target input variables in the associative learning FL2 at t=1 in the client terminal Cj. In the associative learning FL2 at t=2 and thereafter, the analytical model M2j-t (t≧2) for the learning target input variables is transmitted to the client terminal Ck as the base analytical model.

[0058] If the client terminal Cj has an untrained neural network, the server S t The client terminal Ck receives the model parameters θ t A base individual analysis model M20 may be constructed by setting

[0059] When j=1, the client terminal C1 holds an analytical model M21-t for input variables to be learned. When j=2, the client terminal C2 holds an analytical model M22-t for input variables to be learned. When j=3, the client terminal C3 holds an analytical model M23-t for input variables to be learned.

[0060] The client terminal Cj executes model learning 41jt by weighting. Specifically, for example, the client terminal Cj inputs the explanatory variables 203 of the learning data set Dj to the analytical model M2j-t for the learning target input variables, thereby obtaining the predicted value [y j ]. The client terminal Cj calculates the learning weight w j and the response variable y and the predicted value [y j ] and model parameters θ j t Then, the loss function F(θ j t ) is calculated.

[0061] Then, the client terminal Cj sets the model parameters θ j t The gradient of g j t is calculated as the input variable analysis model learning result 42j-t, and the model parameters θ j t The model parameters θ j (t+1) The model parameters θ j (t+1) is also included in the input variable analysis model learning results 42j-tr.

[0062] In the case of j=1, the client terminal C1 generates an analytical model learning result 421-t for input variables. The analytical model learning result 421-t for input variables is a model parameter θ1 t or its gradient ga1 t It is.

[0063] In the case of j=2, the client terminal C2 generates an analytical model learning result 422-t for input variables. The analytical model learning result 422-t for input variables is a model parameter θ2 t or its gradient ga2 t It is.

[0064] In the case of j=3, the client terminal C3 generates an analytical model learning result 423-t for input variables. The analytical model learning result 423-t for input variables is a model parameter θ3 updated in the analytical model M23-t for input variables to be learned. t or its gradient ga3 t It is.

[0065] The server S receives the analytical model for input variables learning result 42j-kt from the client terminal Ck. Specifically, for example, the server S receives the analytical model for input variables learning result 421-1t from the client terminal C1, receives the analytical model for input variables learning result 422-1t from the client terminal C2, and receives the analytical model for input variables learning result 423-1t from the client terminal C3.

[0066] The server S executes integrated learning 430t in the t-th associative learning FL2 and generates an analytical model MQ(t+1) for input variables for the next time. Specifically, for example, the server S integrates the analytical model learning results 421-t, 422-t, and 423-t for input variables to generate an analytical model MQ(t+1) for input variables.

[0067] The server S calculates the analytical model MQ(t+1) for the input variables or its integrated model parameters θ (t+1) to the client terminal Ck. As a result, the analytical model M2j-t for input variables to be learned in the next round of federated learning FL2 is set in the client terminal Ck.

[0068] Specifically, for example, the server S may generate an analytical model MQ(t+1) for input variables or its integrated model parameters θ (t+1)to the client terminals C1 to C3. Using this received information, the client terminal C1 sets an analytical model M21-t for input variables to be learned in the next round of associative learning FL2, the client terminal C2 sets an analytical model M22-t for input variables to be learned in the next round of associative learning FL2, and the client terminal C3 sets an analytical model M23-t for input variables to be learned in the next round of associative learning FL2.

[0069] In this manner, the federated learning FL2 is repeatedly executed. When the number of executions t reaches a predetermined threshold T2, or when the accuracy of all the analytical models MQ for input variables reaches the target accuracy, the server S ends the federated learning FL2 and outputs the latest analytical model MQ for input variables(t+1) as the analytical model MQ for input variables.

[0070] <Learning weight calculation 40k> Fig. 5 is an explanatory diagram showing a specific example of the learning weight calculation shown in Fig. 4. In Fig. 5, the learning data set D1 has data samples D11 and D12 as data sample i, the learning data set D2 has data samples D21 and D22 as data sample i, and the learning data set D3 has data samples D31 and D32 as data sample i. In addition, the server S has input data Q1 and Q2 having the value of the explanatory variable 203 as the input variable Q.

[0071] In the input variable similarity calculation 404, the server S inputs the input variable Q to the integrated similarity calculation model M1 and calculates the predicted probability [pq] as the input variable similarity QS.

[0072] The client terminal Cj inputs the explanatory variables 301 of the learning data set Dj into the integrated similarity calculation model M1 and calculates the predicted probability [p j ] is calculated. j ] is the predicted probability for each data sample i [p ji ]. The predicted probability [p ji] is a row vector indicating the predicted probability indicating how similar a data sample i in the training data set Dj is to each training data set Dk.

[0073] Specifically, for example, when j=1, the client terminal C1 inputs the explanatory variables 301 of the learning data set D1 to the integrated similarity calculation model M1 to calculate the predicted probability [p1]. By inputting the explanatory variables 301 of the data sample D11 to the integrated similarity calculation model M1, the predicted probability [p 11 ] is calculated, and the explanatory variables 301 of the data sample D12 are input to the integrated similarity calculation model M1 to obtain the predicted probability [p 12 ] is calculated.

[0074] In the learning weight calculation 401, the client terminal C1 calculates the predicted probability [p1] and the predicted probability [pq r ]. Specifically, for example, the client terminal C1 performs a similarity calculation with the predicted probability [p 11 ] and the predicted probability [pq1] corresponding to the input data Q1. The calculated similarity is used as the learning weight w 11 1 Similarly, the client terminal C1 calculates the predicted probability [p 12 ] and the predicted probability [pq1] corresponding to the input data Q1. The calculated similarity is used as the learning weight w 12 1 Let the learning weights w 11 1 ,w 12 1 Putting these together, the learning weight w1 1 Let us assume that.

[0075] In addition, when j=2, the client terminal C2 inputs the explanatory variables 301 of the learning data set D2 into the integrated similarity calculation model M1 to calculate the predicted probability [p2]. By inputting the explanatory variables 301 of the data sample D21 into the integrated similarity calculation model M1, the predicted probability [p 21] is calculated, and the explanatory variables 301 of the data sample D22 are input to the integrated similarity calculation model M1 to obtain the predicted probability [p 22 ] is calculated.

[0076] In the learning weight calculation 402, the client terminal C2 calculates the predicted probability [p2] and the predicted probability [pq r ]. Specifically, for example, the client terminal C2 performs a similarity calculation with the predicted probability [p 21 ] and the predicted probability [pq1] corresponding to the input data Q1. The calculated similarity is used as the learning weight w 21 1 Similarly, the client terminal C2 calculates the predicted probability [p 22 ] and the predicted probability [pq1] corresponding to the input data Q1. The calculated similarity is used as the learning weight w 22 1 Let the learning weights w 21 1 ,w 22 1 Putting these together, the learning weight w2 1 Let us assume that.

[0077] In addition, when j=3, the client terminal C3 inputs the explanatory variables 301 of the learning data set D3 into the integrated similarity calculation model M1 to calculate the predicted probability [p3]. By inputting the explanatory variables 301 of the data sample D31 into the integrated similarity calculation model M1, the predicted probability [p 31 ] is calculated, and the explanatory variables 301 of the data sample D32 are input to the integrated similarity calculation model M1 to obtain the predicted probability [p 32 ] is calculated.

[0078] In the learning weight calculation 403, the client terminal C3 calculates the predicted probability [p3] and the predicted probability [pq r ]. Specifically, for example, the client terminal C3 performs a similarity calculation with the predicted probability [p 31] and the predicted probability [pq1] corresponding to the input data Q1. The calculated similarity is used as the learning weight w 31 1 Similarly, the client terminal C3 calculates the predicted probability [p 32 ] and the predicted probability [pq1] corresponding to the input data Q1. The calculated similarity is used as the learning weight w 32 1 Let the learning weights w 31 1 ,w 32 1 Putting these together, the learning weight w3 1 Let us assume that.

[0079] Below, we use the learning weights w in the federated learning FL2 of the input variable analysis model. ji r This specifies the formula (9) to be used in the calculation of

[0080]

number

[0081] W on the left ji r is the learning weight applied to the similarity relationship between the i-th data sample Dji of the learning data set Dj and the input data Qri of the input variable Qr. ji ] is the predicted probability corresponding to the data sample Dji. r ] is the predicted probability corresponding to the data sample Qr. The α on the right side is a parameter that adjusts the degree of distance. Thus, the learning weight w ji r is the predicted probability [p ji ] and the predicted probability [pq r ] is expressed as the inverse of the distance between the vector.

[0082] <Model Learning by Weighting 41jt> Next, a specific example of the weighted model learning 41jt (411t to 413t) shown in FIG. 4 will be described.

[0083] Fig. 6 is an explanatory diagram showing a specific example of the weighted model learning 411t shown in Fig. 4. The client terminal C1 updates the analytical model for input variables to be learned M21-t by using the explanatory variables 203 and the objective variables 204 of the learning data set D1 and the learning weight w1, and outputs the analytical model for input variables learning result 421-t.

[0084] The client terminal C2 updates the analytical model for input variables to be learned M22-t using the explanatory variables 203 and the objective variables 204 of the learning data set D2 and the learning weights w2, and outputs the analytical model for input variables learning results 422-t.

[0085] The client terminal C3 updates the analytical model for input variables to be learned M23-t using the explanatory variables 203 and the objective variables 204 of the learning data set D3 and the learning weights w3, and outputs the analytical model for input variables learning results 423-t.

[0086] The server S integrates the analytical model learning results 421-t, 422-t, and 423-t for input variables in the integrated learning 430t to generate the analytical model MQ(t+1) for input variables for the next round. The server S generates the analytical model MQ(t+1) for input variables or its integrated model parameters θ (t+1) to the client terminals C1 to C3. Using this received information, the client terminal C1 sets an analytical model M21-t for input variables to be learned in the next round of associative learning FL2, the client terminal C2 sets an analytical model M22-t for input variables to be learned in the next round of associative learning FL2, and the client terminal C3 sets an analytical model M23-t for input variables to be learned in the next round of associative learning FL2. The analytical model MQ for input variables is the analytical model established at the end of associative learning FL2.

[0087] Below, we define the formula (10) used in the federated learning FL2 of the analytical model for input variables.

[0088] [y]=f(x q ;θq ) · · · (10)

[0089] The above formula (10) defines the analytical model MQ for the input variables. x q is the explanatory variable 203 of the input variable Q, and θ q are the integrated model parameters of the individual analysis model MQ. [y] is a column vector of the predicted value of the objective variable 204 for each data sample i of the input variable Q, that is, the inference result Ar.

[0090]

number

[0091] The above formula (11) is the loss function F(θ j t y ji is the objective variable 204 of data sample i in the learning data set Dj, and [y ji ] is the predicted value. [y ji ] is [y j ]. ji r is the learning weight calculated by the above formula (9).

[0092]

number

[0093] The above formula (12) is the model parameter θ j t The gradient of g j t The gradient ga is j t is calculated on the client terminal Cj.

[0094]

number

[0095] The above formula (13) is the model parameter θ j t This is a formula for updating the model by weighting the client terminal Cj, and is executed in model learning 41jt. j r As shown in the above formula (14), the learning weight w of the client terminal Cj in the t-th federated learning FL2 is ji r The sum of W on the right side of the above formula (13) r As shown in the above formula (15), the learning weights W1 r ~W K r is the sum of.

[0096] The client terminal Cj updates the model parameters θ1 (t+1) ~θ3 (t+1) The server S transmits the input variable analysis model learning results 421-t, 422-t, and 423-t to the server S in the integrated learning 430t. (t+1) ), the analytical model learning result for input variables 422-t (updated model parameters θ2 (t+1) ), the analytical model learning result for input variables 423-t (updated model parameters θ3 (t+1) ) are integrated, for example, by averaging, to obtain the updated integrated model parameters θ (t+1) and generate an analytical model MQ(t+1) for the input variables.

[0097]

number

[0098] The above formula (16) is the gradient ga from the client terminal Cj as the input variable analysis model learning result 42j-t. j tWhen receiving the model parameter θ j t This is a formula for performing the update of , and is executed by the integrated learning 430t of the server S.

[0099] The above formula (17) is the gradient ga j t When receiving the tth federated learning FL2, the integrated model parameter θ t This is a formula for updating the integrated model parameters θ j t+1 and the learning weights W j r and the learning weights W r Using the above, the model parameters θ of the input variable analysis model MQ(t+1) are calculated. (t+1) This gives us the analytical model MQ(t+1) for the input variables.

[0100] <Associative learning sequence> 7 is a sequence diagram of federated learning in the federated learning system 100 according to the first embodiment. First, prior to the federated learning FL1 and FL2, the server S transmits a client ID 201 to each client terminal Ck (step S701). The client terminal Ck associates the client ID 201 with the learning data set Dk.

[0101] Next, the associative learning system 100 executes associative learning FL1 of the similarity calculation model (step S702), and executes associative learning FL2 of the input variable analysis model (step S703).

[0102] In the federated learning FL1 of the similarity calculation model (step S702), the server S transmits similarity calculation model information (integrated similarity calculation model M1(t+1) or its integrated model parameter φ t+1 ) is transmitted (step S721).

[0103] The client terminal Cj learns the learning target similarity calculation model M1jt by using the learning data set Dj and the similarity calculation model information (step S722).

[0104] The client terminal Cj transmits the learning result 32jt in step S722 to the server S (step S723).

[0105] The server S executes the integrated learning 330t of the learning object similarity calculation model M1jt using the learning result 32jt (step S724). The server S determines whether or not the end condition of the integrated learning 330t of the learning object similarity calculation model M1jt (step S724) is satisfied (step S725).

[0106] If the termination condition is not satisfied, the server S transmits the integrated similarity calculation model information of the updated integrated similarity calculation model M1(t+1) (the integrated similarity calculation model M1(t+1) or its model parameters) to the client terminal Cj (step S721).

[0107] When the end condition is satisfied, the server S registers the updated integrated similarity calculation model information (the integrated similarity calculation model M1 or its model parameters) (step S726). This ends the associative learning FL1 of the similarity calculation model (step S702).

[0108] A user terminal 700 capable of communicating with the server S accepts an input of an input variable Q (step S703) and transmits the input variable Q to the server S (step S704). The user terminal 700 may be a client terminal Cj. The input variable Q may also be input directly to the server S.

[0109] In the federated learning FL2 of the input variable analysis model (step S705), the server S acquires the input variable Q (step S731). Next, the server S executes the input variable similarity calculation 404 using the integrated similarity calculation model M1, and outputs the input variable similarity QS (step S732).

[0110] The server S transmits the input variable similarity QS to the client terminal Cj (step S733). The client terminal Cj calculates the learning weight wj using the input variable similarity QS as shown in FIG.

[0111] If t=1, the server S calculates the base analysis model M20 (or its model parameters θ 1 ) for t ≥ 2, the analytical model M2j-t for the input variables to be trained (or its model parameters θ j t ) is sent to the client terminal Cj as the analytical model information for input variables (step S735).

[0112] As shown in FIG. 6, the client terminal Cj performs model learning by weighting to generate the analytical model learning result 42j-t for input variables (step S736), and transmits the analytical model learning result 42j-t for input variables to the server S (step S737).

[0113] As shown in FIG. 6, the server S acquires the analytical model learning results 42j-t for input variables from the client terminal Cj, performs integrated learning 430t, and generates the t+1th analytical model MQ(t+1) for input variables (step S738).

[0114] The server S determines (step S739) whether or not the end condition of the integrated learning 430t (step S738) is satisfied.

[0115] If the termination condition is not satisfied, the server S transmits updated analytical model information for input variables to the client terminal Cj, whereby the client terminal Cj updates the analytical model M2j-t for input variables to be learned.

[0116] If the termination condition is satisfied, the server S registers the updated analytical model for input variables MQ(t+1) as the analytical model for input variables MQ (step S740). This ends the associative learning FL2 of the analytical model for input variables (step S705).

[0117] After that, the server S executes inference IN by inputting the input variables Q to the input variable analysis model MQ (step S706). Then, the server S transmits the inference result A to the user terminal 700 (step S707).

[0118] In this way, according to the first embodiment, the analytical model MQ for input variables that takes into account the value of the input variable Q can be generated as an integrated model that integrates the analytical models M21-t to M23-t for input variables to be learned of the client terminals C1 to C3. EXAMPLES

[0119] Next, a second embodiment will be described. In the second embodiment, an example in which associative learning of an individual analysis model is executed instead of associative learning FL2 of an analysis model for input variables will be described. In the second embodiment, differences from the first embodiment will be mainly described, and therefore common parts with the first embodiment will not be described.

[0120] 8 is an explanatory diagram showing an example of associative learning according to Example 2. The associative learning FL3 of the individual analysis model is associative learning that generates an individual analysis model M3j (M31 to M33) for the client terminal Cj obtained from each of the client terminals C1 to C3 by using the integrated similarity calculation model M1.

[0121] Specifically, for example, individual analytical model M31 is a predictive model that integrates individual analytical models for client terminal C1 (j=1) from client terminals C1 to C3, individual analytical model M32 is a predictive model that integrates individual analytical models for client terminal C2 (j=2) from client terminals C1 to C3, and individual analytical model M33 is a predictive model that integrates individual analytical models for client terminal C3 (j=3) from client terminals C1 to C3.

[0122] By executing the federated learning FL3 of the individual analytical models, an appropriate individual analytical model M3j (M31 to M33) is generated for each client terminal Cj.

[0123] The server S executes the input variable analysis model integration 800, which integrates the individual analysis models M31 to M33 using the input variable similarity QS obtained in the input variable similarity calculation 404, and generates an input variable analysis model MQ. When the input variable Q is input to the input variable analysis model MQ, an inference IN is executed and an inference result A is obtained.

[0124] <Associative learning of individual analysis models FL3> 9 is an explanatory diagram showing an example of federated learning FL3 of an individual analysis model. The client terminal C1 executes learning weight calculation 901. Specifically, for example, the client terminal Cj inputs the explanatory variables 301 of the learning data set Dj into the integrated similarity calculation model M1 (the above formula (8)) generated in FIG. 3, thereby calculating a predicted probability [p j The client terminal Cj calculates the calculated predicted probability [p j ] is used to calculate the learning weights w9j.

[0125] The server S has a base individual analysis model (hereinafter, base individual analysis model) M20. The base individual analysis model M20 may be an untrained neural network, and has model parameters θ t Alternatively, the server S may transmit the base individual analysis model M20 to the client terminal Ck.

[0126] If the client terminal Ck has an untrained neural network, the server S t The client terminal Ck receives the model parameters θ t A base individual analysis model M20 may be constructed by setting

[0127] The base individual analytical model M20 becomes the learning target individual analytical model M2j-kt of the first associative learning FL2 in the client terminal Cj, where t is an ascending natural number starting from 1 indicating the number of times the associative learning FL2 of the individual analytical model has been executed.

[0128] When j=1, the client terminal C1 holds the learning subject individual analytical models M31-1t to M31-3t. When j=2, the client terminal C2 holds the learning subject individual analytical models M32-1t to M32-3t. When j=3, the client terminal C3 holds the learning subject individual analytical models M33-1t to M33-3t.

[0129] The client terminal Cj executes model learning 91jt by weighting. Specifically, for example, the client terminal Cj inputs the explanatory variables 203 of the learning data set Dj into each of the learning target individual analysis models M3j-kt, thereby obtaining the predicted value [y j k The client terminal Cj calculates the learning weight w9j, the objective variable y, and the predicted value [y j k ] and model parameters θ j kt As a result, the loss function F(θ j kt ) is calculated.

[0130] Then, the client terminal Cj sets the model parameters θ j kt The gradient of g j kt Calculate the individual analysis model learning result 92j-kt, and the model parameters θ j kt The model parameters θ j k(t+1) The model parameters θ j k(t+1) is also included in the individual analysis model learning result 92j-kt.

[0131] In the case of j=1, the client terminal C1 generates individual analytical model learning results 921-1t to 921-3t. The individual analytical model learning results 921-1t are generated based on the model parameters θ1 1t or its gradient ga1 1t The individual analysis model learning result 921-2t is the model parameter θ1 updated in the learning target individual analysis model M31-2t. 2t or its gradient ga1 2t The individual analysis model learning result 921-3t is the model parameter θ1 updated in the learning target individual analysis model M31-3t. 3t or its gradient ga1 3t It is.

[0132] In the case of j=2, the client terminal C2 generates individual analytical model learning results 922-1t to 922-3t. The individual analytical model learning results 922-1t are the model parameters θ2 1t or its gradient ga2 1t The individual analysis model learning result 922-2t is the model parameter θ2 updated in the learning target individual analysis model M32-2t. 2t or its gradient ga2 2t The individual analysis model learning result 922-3t is the model parameter θ2 updated in the learning target individual analysis model M32-3t. 3t or its gradient ga2 3t It is.

[0133] In the case of j=3, the client terminal C3 generates individual analytical model learning results 923-1t to 923-3t. The individual analytical model learning results 923-1t are the model parameters θ3 1t or its gradient ga3 1t The individual analysis model learning result 923-2t is the model parameter θ3 updated in the learning target individual analysis model M33-2t. 2t or its gradient ga3 2tThe individual analysis model learning result 923-3t is the model parameter θ3 updated in the learning target individual analysis model M33-3t. 3t or its gradient ga3 3t It is.

[0134] The server S receives the individual analysis model learning results 92j-kt from the client terminal Ck. Specifically, for example, the server S receives the individual analysis model learning results 921-1t to 921-3t from the client terminal C1, receives the individual analysis model learning results 922-1t to 922-3t from the client terminal C2, and receives the individual analysis model learning results 923-1t to 923-3t from the client terminal C3.

[0135] The server S executes integrated learning 930t in the t-th associative learning FL2, and generates an individual analysis model M3k(t+1) for the next round. Specifically, for example, the server S integrates the individual analysis model learning results 921-1t, 922-1t, and 923-1t to generate an individual analysis model M31(t+1). The server S also integrates the individual analysis model learning results 921-2t, 922-2t, and 923-2t to generate an individual analysis model M32(t+1). The server S also integrates the individual analysis model learning results 921-3t, 922-3t, and 923-3t to generate an individual analysis model M33(t+1).

[0136] The server S calculates the individual analysis model M3j(t+1) or its integrated model parameters θ j t+1 to the client terminal Ck. As a result, the learning target individual analytical model M3j-kt in the next round of federated learning FL3 is set in the client terminal Ck.

[0137] Specifically, for example, the server S may generate an individual analysis model M31(t+1) or its integrated model parameter θ1 t+1to the client terminals C1 to C3. Using this received information, the client terminal C1 sets a learning subject individual analytical model M31-1t in the next round of associative learning FL3, the client terminal C2 sets a learning subject individual analytical model M32-1t in the next round of associative learning FL3, and the client terminal C3 sets a learning subject individual analytical model M33-1t in the next round of associative learning FL3.

[0138] In addition, the server S calculates the individual analysis model M32(t+1) or its integrated model parameter θ2 t+1 to the client terminals C1 to C3. Using this received information, the client terminal C1 sets a learning subject individual analytical model M31-2t in the next associative learning FL3, the client terminal C2 sets a learning subject individual analytical model M32-2t in the next associative learning FL3, and the client terminal C3 sets a learning subject individual analytical model M33-2t in the next associative learning FL3.

[0139] In addition, the server S calculates the individual analysis model M33(t+1) or its integrated model parameter θ3 t+1 to the client terminals C1 to C3. Using this received information, the client terminal C1 sets a learning subject individual analytical model M31-3t in the next associative learning FL3, the client terminal C2 sets a learning subject individual analytical model M32-3t in the next associative learning FL3, and the client terminal C3 sets a learning subject individual analytical model M33-3t in the next associative learning FL3.

[0140] In this way, the federated learning FL3 is repeatedly executed. When the number of executions t reaches a predetermined threshold T2, or when the accuracy of all of the individual analysis models M31(t+1) to M3K(t+1) reaches the target accuracy, the server S ends the federated learning FL3, outputs the latest individual analysis model M3k(t+1) as the individual analysis model M3k, and transmits it to the client terminal Ck.

[0141] Specifically, for example, the server S outputs the individual analytical model M31(t+1) as the individual analytical model M31 and transmits it to the client terminal C1. The server S also outputs the individual analytical model M32(t+1) as the individual analytical model M32 and transmits it to the client terminal C2. The server S also outputs the individual analytical model M33(t+1) as the individual analytical model M33 and transmits it to the client terminal C3.

[0142] <Learning weight calculation 40k> Fig. 10 is an explanatory diagram showing a specific example of the learning weight calculation 90k (901 to 903) shown in Fig. 9. In Fig. 10, the learning data set D1 has D11 and D12 as the data sample i, the learning data set D2 has D21 and D22 as the data sample i, and the learning data set D3 has D31 and D32 as the data sample i.

[0143] The client terminal Cj inputs the explanatory variables 301 of the learning data set Dj into the integrated similarity calculation model M1 and calculates the predicted probability [p j ] is calculated. j ] is the predicted probability [p j k ] is a column vector showing the predicted probability [p ji ] is a row vector indicating the predicted probability indicating how similar a data sample i in the training data set Dj is to each training data set Dk.

[0144] Specifically, for example, when j=1, the client terminal C1 inputs the explanatory variables 301 of the learning data set D1 to the integrated similarity calculation model M1 to calculate the predicted probability [p1]. By inputting the explanatory variables 301 of the data sample D11 to the integrated similarity calculation model M1, the predicted probability [p 11 ] is calculated, and the explanatory variables 301 of the data sample D12 are input to the integrated similarity calculation model M1 to obtain the predicted probability [p 12] is calculated. In addition, the columns of CID=1 to 3 of the predicted probability [p1] are the predicted probabilities [p1 1 ], [p1 2 ], [p1 3 ]. The client terminal C1 predicts the probability [p 1 ], [p1 2 ], [p1 3 ] for each of the learning weights w1 1 , w1 2 , w1 3 Calculate the learning weight w1 1 , w1 2 , w1 3 The sum of these information is the learning weight w91.

[0145] In addition, when j=2, the client terminal C2 inputs the explanatory variables 301 of the learning data set D2 into the integrated similarity calculation model M1 to calculate the predicted probability [p2]. By inputting the explanatory variables 301 of the data sample D21 into the integrated similarity calculation model M1, the predicted probability [p 21 ] is calculated, and the explanatory variables 301 of the data sample D22 are input to the integrated similarity calculation model M1 to obtain the predicted probability [p 22 ] is calculated. In addition, the columns of CID=1 to 3 of the predicted probability [p2] are the predicted probabilities [p2 1 ], [p2 2 ], [p2 3 ]. Client terminal C2 predicts the probability [p2 1 ], [p2 2 ], [p2 3 ] for each of the learning weights w2 1 , w2 2 , w2 3 Calculate the learning weight w2 1 , w2 2 , w2 3 The sum of these information is the learning weight w92.

[0146] In addition, when j=3, the client terminal C3 inputs the explanatory variables 301 of the learning data set D3 into the integrated similarity calculation model M1 to calculate the predicted probability [p3]. By inputting the explanatory variables 301 of the data sample D31 into the integrated similarity calculation model M1, the predicted probability [p 31 ] is calculated, and the explanatory variables 301 of the data sample D32 are input to the integrated similarity calculation model M1 to obtain the predicted probability [p 32 ] is calculated. In addition, the columns of CID=1 to 3 in the predicted probability [p3] indicate how similar each of the data samples i in the learning data set D3 is to the learning data sets D1 to D3. 1 ], [p3 2 ], [p3 3 ]. The client terminal C3 predicts the probability [p3 1 ], [p3 2 ], [p3 3 ] for each of the learning weights w3 1 , w3 2 , w3 3 Calculate the learning weight w3 1 , w3 2 , w3 3 The sum of these information is the learning weight w93.

[0147] Below, we use the learning weights w j k This specifies the formula to be used in the calculation of

[0148]

number

[0149] The left side of the above formula (18) ji k is the learning weight applied to the similarity relationship between data sample i of the learning data set Dj and the learning data set Dk. ji k] is the predicted probability that data sample i in the training data set Dj is similar to the training data set Dk. The learning weights w ji k The set of learning weights w j k It is.

[0150] <Model learning by weighting 41kt> Next, a specific example of the weighted model learning 91kt (911t to 913t) shown in FIG. 9 will be described.

[0151] Fig. 11 is an explanatory diagram showing a specific example of the weighted model learning 911t shown in Fig. 9. The client terminal C1 learns the explanatory variables 203 and the objective variables 204 of the learning data set D1, and the learning weights w1 1 The learning object individual analytical model M31-1t is updated using and the individual analytical model learning result 921-1t is output.

[0152] The client terminal C2 has explanatory variables 203 and objective variables 204 of the learning data set D2, and learning weights w2 1 The learning subject individual analytical model M32-1t is updated using and the individual analytical model learning result 922-1t is output.

[0153] The client terminal C3 has explanatory variables 203 and objective variables 204 of the learning data set D3, and learning weights w3 1 The learning subject individual analytical model M33-1t is updated using and the individual analytical model learning result 923-1t is output.

[0154] The server S integrates the individual analysis model learning results 921-1t, 922-1t, and 923-1t in the integrated learning 930t to generate the individual analysis model M31(t+1) for the next round. The server S generates the individual analysis model M31(t+1) or its integrated model parameters θ1 t+1to the client terminals C1 to C3. Using this received information, the client terminal C1 sets a learning target individual analytical model M21-1t in the next associative learning FL2, the client terminal C2 sets a learning target individual analytical model M32-1t in the next associative learning FL2, and the client terminal C3 sets a learning target individual analytical model M33-1t in the next associative learning FL2. The individual analytical model M21 is the individual analytical model of the client terminal C1 that was finalized at the end of associative learning FL3.

[0155] 12 is an explanatory diagram showing a specific example of the weighted model learning 912t shown in FIG. 9. The client terminal C1 learns the explanatory variables 203 and the objective variables 204 of the learning data set D1, and the learning weights w1 2 The learning object individual analytical model M31-2t is updated using and the individual analytical model learning result 921-2t is output.

[0156] The client terminal C2 has explanatory variables 203 and objective variables 204 of the learning data set D2, and learning weights w2 2 The learning object individual analytical model M32-2t is updated using and the individual analytical model learning result 922-2t is output.

[0157] The client terminal C3 has explanatory variables 203 and objective variables 204 of the learning data set D3, and learning weights w3 2 The learning subject individual analysis model M33-2t is updated using and the individual analysis model learning result 923-2t is output.

[0158] The server S integrates the individual analysis model learning results 921-2t, 922-2t, and 923-2t in the integrated learning 930t to generate the individual analysis model M32(t+1) for the next round. The server S generates the individual analysis model M32(t+1) or its integrated model parameters θ2 t+1to the client terminals C1 to C3. Using this received information, the client terminal C1 sets a learning target individual analytical model M31-2t in the next associative learning FL2, the client terminal C2 sets a learning target individual analytical model M32-2t in the next associative learning FL2, and the client terminal C3 sets a learning target individual analytical model M33-2t in the next associative learning FL2. The individual analytical model M32 is the individual analytical model of the client terminal C2 that was finalized at the end of associative learning FL3.

[0159] 13 is an explanatory diagram showing a specific example of the weighted model learning 913t shown in FIG. 9. The client terminal C1 learns the explanatory variables 203 and the objective variables 204 of the learning data set D1, and the learning weights w1 3 The learning subject individual analytical model M31-3t is updated using and the individual analytical model learning result 921-3t is output.

[0160] The client terminal C2 has explanatory variables 203 and objective variables 204 of the learning data set D2, and learning weights w2 3 The learning subject individual analytical model M32-3t is updated using and the individual analytical model learning result 922-3t is output.

[0161] The client terminal C3 has explanatory variables 203 and objective variables 204 of the learning data set D3, and learning weights w3 3 The learning subject individual analysis model M33-3t is updated using and the individual analysis model learning result 923-3t is output.

[0162] The server S integrates the individual analysis model learning results 921-3t, 922-3t, and 923-3t in the integrated learning 930t to generate the individual analysis model M33(t+1) for the next round. The server S generates the individual analysis model M33(t+1) or its integrated model parameters θ3 t+1to the client terminals C1 to C3. Using this received information, the client terminal C1 sets a learning target individual analytical model M31-3t in the next associative learning FL2, the client terminal C2 sets a learning target individual analytical model M32-3t in the next associative learning FL2, and the client terminal C3 sets a learning target individual analytical model M33-3t in the next associative learning FL2. The individual analytical model M33 is the individual analytical model of the client terminal C3 that was finalized at the end of associative learning FL3.

[0163] Below, we define the formulas used in the federated learning FL3 of the individual analysis model.

[0164] [y j ]=f(x j ;θ j ) · · · (19)

[0165] The above formula (19) defines the individual analysis model M2j. x j is the explanatory variable 203 of the training data set Dj, and θ j is the integrated model parameter of the individual analysis model M2j. [y j ] is the predicted value of the objective variable 204 of the training data set Dj.

[0166]

number

[0167] The above formula (20) is the loss function F(θ j kt y ji is the objective variable 204 of data sample i in the learning data set Dj, and [y ji ] is the predicted value. [y ji ] is [y j ]. ji k is the learning weight calculated by the above formula (18).

[0168]

number

[0169] The above formula (21) is the model parameter θ j kt The gradient of g j kt The gradient ga is j kt is calculated on the client terminal Cj.

[0170]

number

[0171] The above formula (22) is the model parameter θ j kt This is a formula for updating the weighted model learning 91jt of the client terminal Cj. P j t As shown in the above formula (23), P of the client terminal Ck in the t-th federated learning FL3 j kt It is the sum of P j kt As shown in the above formula (24), the predicted probability [p ji k ] t is the sum of.

[0172] The client terminal Cj updates the model parameters θ j 1(t+1) ~θ j 3(t+1) The server S transmits the individual analysis model learning results 921-1t (updated model parameters θ1 1(t+1) ), individual analysis model learning result 922-1t (updated model parameters θ21(t+1) ), individual analysis model learning result 923-1t (updated model parameters θ3 1(t+1) ) for example by averaging and integrating them to obtain the updated integrated model parameters θ1 for the client terminal C1. (t+1) is calculated to generate the individual analysis model M31(t+1).

[0173] In addition, in the integrated learning 930t, the server S obtains the individual analysis model learning result 921-2t (the updated model parameter θ2 1(t+1) ), individual analysis model learning result 922-2t (updated model parameters θ2 2(t+1) ), individual analysis model learning result 923-2t (updated model parameters θ3 2(t+1) ) for the client terminal C2, for example, by averaging and integrating them to obtain the updated integrated model parameters θ (t+1) is calculated and an individual analysis model M32(t+1) is generated.

[0174] In addition, in the integrated learning 930t, the server S obtains the individual analysis model learning result 921-3t (the updated model parameter θ3 1(t+1) ), Individual analysis model learning result 922-3t (updated model parameters θ2 3(t+1) ), individual analysis model learning result 923-3t (updated model parameters θ3 3(t+1) ) are integrated, for example, by averaging, to obtain the updated integrated model parameters θ3 for the client terminal C2. (t+1) is calculated to generate the individual analysis model M33(t+1).

[0175]

number

[0176] The above formula (25) is the gradient ga from the client terminal Cj as the individual analysis model learning result 92j-kt. j kt and predicted probability [p ji k ] tWhen receiving, each model parameter θ j kt This is a formula for performing the update of , and is executed by the integrated learning 430t of the server S.

[0177] The above formula (26) is the gradient ga from the client terminal Cj as the individual analysis model learning result 92j-kt. j kt and predicted probability [p ji k ] t When receiving the tth federated learning FL3, the integrated model parameter θ j t This is a formula for updating the integrated model parameter θ1 after updating, and is executed by the integrated learning 430t of the server S. Specifically, for example, the server S updates the integrated model parameter θ1 after updating. t+1 The individual analysis model M31(t+1) is generated using the updated integrated model parameters θ2 t+1 The individual analysis model M32(t+1) is generated using the updated integrated model parameters θ3 t+1 Generate the individual analysis model M33(t+1) using

[0178] <Analysis Model Integration for Input Variables 800> 14 is an explanatory diagram showing an example of the input variable analysis model integration 800. As in the first embodiment, the server S inputs the input variable Q to the integrated similarity calculation model M1, executes the input variable similarity calculation 404, and outputs the input variable similarity QS.

[0179] The server S executes an input variable analysis model integration 800 that integrates the individual analysis models M31 to M33 using the input variable similarity QS according to the following formula (27), and generates an input variable analysis model MQ.

[0180]

number

[0181] The [y] on the left side represents the inference result A, that is, a column vector consisting of inference values ​​for each piece of input data Q1, Q2, . . . , which is a row vector of the input variable Q.

[0182] p on the left side r j is the similarity vector p calculated for the input data Qr r θ indicates the j-th element (the degree of similarity with the j-th client terminal Cj). j are the model parameters of the individual analysis model M3j.

[0183] <Associative learning sequence> 15 is a sequence diagram of the federated learning in the federated learning system 100 according to the embodiment 2. First, the transmission of the client ID 201 from the server S to each client terminal Cj (step S701) and the execution of the federated learning FL1 of the similarity calculation model (step S702) are the same as those in the embodiment 1.

[0184] Next, the associative learning system 100 executes associative learning FL3 of the individual analytical model (step S1503).

[0185] In the associative learning FL3 of the individual analysis model (step S1503), the client terminal Cj executes the learning weight calculation 90j (901 to 903) using the integrated similarity calculation model M1, and calculates the learning weights w9j (w91 to w93) (step S1531).

[0186] In addition, the server S transmits the individual analysis model information (the individual analysis model M3j(t+1) or its integrated model parameters θ j t+1 ) is transmitted (step S1532).

[0187] The client terminal Cj executes the weighted model learning 91jt (step S1533) and transmits the individual analysis model learning result 92j-kt to the server S (step S1534).

[0188] The server S uses the individual analytical model learning results 92j-kt to execute integrated learning 930t of the individual analytical models (step S1535).

[0189] The server S determines whether or not a termination condition for the integrated learning 930t of the individual analytical model (step S1535) is satisfied (step S1536).

[0190] If the termination condition is not satisfied, the server S registers the updated individual analytical model information (step S1537) and transmits it to the client terminal Cj. If the termination condition is satisfied, the server S transmits the updated individual analytical model information to the client terminal Cj. As a result, the client terminal Cj generates an individual analytical model M3j using the updated individual analytical model information and terminates the individual analytical model associative learning FL3 (step S1503).

[0191] The user terminal 700 capable of communicating with the server S accepts the input of the input variable Qr (step S703), and transmits the input variable Qr to the server S (step S704).

[0192] After that, the server S executes input variable inference (step S1505). In the input variable inference (step S1505), the server S acquires an input variable Q (step S731). Next, the server S executes the input variable similarity calculation 404 using the integrated similarity calculation model M1, and outputs the input variable similarity QS (step S732).

[0193] The server S executes the input variable analysis model integration 800 for integrating the individual analysis models M31 to M33 registered in step S1537 using the input variable similarity QS, and generates an input variable analysis model MQ (step S1553).

[0194] After that, the server S executes inference IN by inputting the input variables Q to the input variable analysis model MQ (step S706). Then, the server transmits the inference result A to the user terminal 700 (step S707).

[0195] In this way, according to the second embodiment, the input variable analytical model MQ that takes into account the value of the input variable Q can be generated as an integrated model that integrates the individual analytical models M31 to M33 of the client terminals C1 to C3.

[0196] Moreover, according to the second embodiment, it is possible to provide an individual analysis model M2k suitable for each client terminal Ck participating in the federated learning. In addition, when generating the integrated similarity calculation model M1, the client terminal Ck can predict the similarity with other learning data sets Dk without passing the data sample i to other client terminals Ck or the server S, so that it is possible to prevent the data sample i itself from being leaked. EXAMPLES

[0197] Next, a third embodiment will be described. In the second embodiment, the federated learning FL3 of the individual analytical model is executed, but in the third embodiment, the federated learning system 100 does not execute the federated learning FL3 of the individual analytical model, and the client terminal Cj holds the individual analytical model M3j. The individual analytical model M3j may be a prediction model provided from the outside, or may be a prediction model created by the user of the client terminal Cj.

[0198] In the third embodiment, similarly to the second embodiment, the server S executes the input variable similarity calculation 404 and the input variable analysis model integration 800 to generate the input variable analysis model MQ. Similarly, inference IN is executed to obtain the inference result A, similarly to the second embodiment.

[0199] In this way, according to the third embodiment, the calculation load can be reduced.

[0200] <Display screen example> 16 is an explanatory diagram showing an example of a management screen according to Examples 1 to 3. The management screen 1600 is displayed on the server S. The management screen 1600 displays an input variable Q, an inference result A, and integrated model information for input variables 1601. The inference result A includes a predicted value 1602 for each data ID 202 included in the input variable Q. The integrated model information for input variables 1601 includes a learning contribution degree 1603 for each data ID 202 included in the input variable Q.

[0201] The integrated model information for input variables 1601 includes a learning contribution degree 1603 of the client terminal Cj for each data ID 202 included in the input variable Q. The learning contribution degree 1603 is calculated by the following formula. The learning contribution degree 1603 is an index value indicating how much the learning data set Dk of the client terminal Ck contributes to the learning of the individual analysis model.

[0202]

number

[0203] R on the left side of the above formula (28) j k is the learning contribution 1603 of the client terminal Cj. The denominator on the right side is W r The numerator on the right side is W shown in the above formula (14). j r The learning contribution degree 1603 of the client terminal Cj is calculated by the server S.

[0204] In this way, the learning contribution 1603 for each client terminal Cj makes it possible to confirm how much each data sample i of the client terminal Cj contributed to the inference result A. For example, for the input data Q1 (data ID 202 is "D01") in the input variable Q, the learning contribution 1603 of the client terminal C1 is 57%, the learning contribution 1603 of the client terminal C2 is 33%, and the learning contribution 1603 of the client terminal C3 is 10%. Therefore, it can be seen that the client terminal Cj that contributed to the predicted value 1602 of the input data Q1 (data ID 202 is "D01"), which is "0.9", is the client terminal C1 with the highest learning contribution 1603. Therefore, it can be seen that the input data Q1 is more similar to the data sample i of the learning dataset D1 of the client terminal C1 than to the data sample i of the learning dataset D2 of the client terminal C2 and the learning dataset D3 of the client terminal C3.

[0205] <Hardware configuration example of server S and client terminal Ck> FIG. 17 is a block diagram showing an example of a hardware configuration of the server S and the client terminal Ck (hereinafter, computer 1700) according to the first to third embodiments. The computer 1700 includes a processor 1701, a storage device 1702, an input device 1703, an output device 1704, and a communication interface (communication IF) 1705. The processor 1701, the storage device 1702, the input device 1703, the output device 1704, and the communication IF 1705 are connected by a bus 1706. The processor 1701 controls the computer 1700. The storage device 1702 is a working area for the processor 1701. The storage device 1702 is a non-transient or temporary recording medium that stores various programs and data. Examples of the storage device 1702 include a ROM (Read Only Memory), a RAM (Random Access Memory), a HDD (Hard Disk Drive), and a flash memory. The input device 1703 inputs data. The input device 1703 may be, for example, a keyboard, a mouse, a touch panel, a numeric keypad, a scanner, a microphone, or a sensor. The output device 1704 outputs data. The output device 1704 may be, for example, a display, a printer, or a speaker. The communication IF 1705 connects to a network and transmits and receives data.

[0206] The above-mentioned associative learning system 100 can also be configured as follows [1] to

[10] .

[0207] [1] A federated learning system 100 includes a plurality of client terminals C1 to C3, each having a learning data set D1 to D3, and a server S capable of communicating with the plurality of client terminals C1 to C3. The system performs federated learning in which a model is learned in each of the plurality of client terminals C1 to C3 using the learning data sets D1 to D3, and the server S repeats a process of integrating the models of the plurality of client terminals C1 to C3 using the learning results.

[0208] The learning data set Dk has one or more data samples i including a client ID 201 for identifying the client terminal Ck, a first explanatory variable 203, and a first objective variable 204.

[0209] The federated learning system 100 executes a first calculation process (input variable similarity calculation) by the server S and a second calculation process (learning weight calculation 401-403) by each of the multiple client terminals C1-C3, and also executes a first federated learning process (FL2) that repeats a first learning process (weighting model learning 411t-413t) by each of the multiple client terminals C1-C3 and a first integration process (integrated learning 330t) by the server S until a first termination condition is satisfied.

[0210] In the first calculation process (input variable similarity calculation), the server S acquires an input data set (input variable Q) including a client ID 201 that identifies the client terminal Ck, and one or more input data (Q1, Q2) including a first explanatory variable 203 and a first objective variable 204.

[0211] In the first calculation process (input variable similarity calculation), the server S inputs the input dataset (input variable Q) into a similarity calculation model M1 that calculates the similarity between the data sample i and the multiple learning datasets D1 to D3, thereby calculating a first similarity (predicted probability [pq]) between the input dataset and the multiple learning datasets D1 to D3.

[0212] In the second calculation process (learning weight calculation 401 to 403), each of the multiple client terminals C1 to C3 inputs the data sample i to the similarity calculation model M1 to calculate a second similarity (prediction probability [p j ]) is calculated.

[0213] In the second calculation process (learning weight calculation 401 to 403), each of the multiple client terminals C1 to C3 calculates the first similarity (predicted probability [pq]) and the second similarity (predicted probability [p j ]) to output the learning weights wj.

[0214] In the first learning process (model learning by weighting 411t to 413t), each of the multiple client terminals C1 to C3 learns the first analytical model (analysis models for input variables to be learned M21-1t, M22-1t, M23-1t) that calculates a predicted value of the first objective variable 204 from the first explanatory variable 203 for the input data set, based on the first explanatory variable 203 and the first objective variable 204 of the learning data set, and the learning weight, and transmits first learning results (421-t, 422-t, 423-t) to the server S.

[0215] In the first integration process (integrated learning 330t), the server S integrates the first learning results (421-t, 422-t, 423-t) from the multiple client terminals C1 to C3 through the first learning process (weighted model learning 411t to 413t) to generate a second analysis model (analysis model for input variables MQ) that calculates a predicted value of the first objective variable 204 from the first explanatory variable 203 for the input data set.

[0216] [2] In the federated learning system 100 described above in [1], each of the multiple client terminals C1 to C3 executes the first learning process (weighted model learning 411t to 413t) using the second analytical model generated by the first integrated process (integrated learning 330t) as the first analytical model.

[0217] [3] In the federated learning system 100 of [1] above, when the first termination condition is satisfied, the server S executes inference IN by inputting the input data set (input variables Q) into the second analytical model (analysis model MQ for input variables).

[0218] [4] The federated learning system 100 includes a plurality of client terminals C1 to C3, each having a learning data set D1 to D3, and a server S capable of communicating with the plurality of client terminals C1 to C3. The federated learning system 100 performs federated learning in which a model is learned in each of the plurality of client terminals C1 to C3 using the learning data sets D1 to D3, and the server S repeats a process of integrating the models of the plurality of client terminals C1 to C3 using the learning results.

[0219] The learning data set Dk has one or more data samples i including a client ID 201 for identifying the client terminal Ck, a first explanatory variable 203, and a first objective variable 204.

[0220] The federated learning system 100 executes a first calculation process (input variable similarity calculation) by the server S and a second calculation process (learning weight calculation 901-903) by each of the multiple client terminals C1-C3, and also executes a first federated learning process (FL2) that repeats a first learning process (weighting model learning 911t-913t) by each of the multiple client terminals C1-C3 and a first integration process (integrated learning 930t) by the server S until a first termination condition is satisfied.

[0221] In the first calculation process (input variable similarity calculation), the server S acquires an input data set (input variable Q) including a client ID 201 that identifies the client terminal Ck, and one or more input data (Q1, Q2) including a first explanatory variable 203 and a first objective variable 204.

[0222] In the first calculation process (input variable similarity calculation), the server S inputs the input dataset (input variable Q) into a similarity calculation model M1 that calculates the similarity between the data sample i and the multiple learning datasets D1 to D3, thereby calculating a first similarity (predicted probability [pq]) between the input dataset and the multiple learning datasets D1 to D3.

[0223] In the second calculation process (learning weight calculation 901 to 903), each of the multiple client terminals C1 to C3 inputs the data sample i to a similarity calculation model M1 that calculates the similarity between the data sample i and the multiple learning data sets D1 to D3, thereby calculating a second similarity (prediction probability [p ji k ] t ) is calculated.

[0224] In the first learning process (weighted model learning 911t to 913t), each of the multiple client terminals C1 to C3 calculates a predicted value of the first objective variable 204 from the first explanatory variable 203 by using an individual analysis model (learning target individual analysis models M21-1t, M22-1t, M23-1t), and calculates a specific second similarity (prediction probability [p ji 1 ] t ), and the individual analytical models (learning target individual analytical models M21-1t, M22-1t, M23-1t) are trained based on these.

[0225] In the first integration process (integrated learning 930t), the server S uses a first similarity (prediction probability [pq]) to integrate multiple first learning results (921-1t, 922-1t, 923-1t) from the multiple client terminals C1 to C3 through the first learning process (weighted model learning 911t to 913t) to generate an analytical model (analysis model for input variables MQ) that calculates a predicted value of the first objective variable 204 from the first explanatory variable 203 for the input data set.

[0226] [5] In the federated learning system 100 described in [4] above, each of the multiple client terminals C1 to C3 executes the first learning process (weighted model learning 911t to 913t) using the analytical model generated by the first integrated process (integrated learning 930t) as the individual analytical model.

[0227] [6] In the federated learning system 100 of [4] above, when the first termination condition is satisfied, the server S executes inference IN by inputting the input data set (input variables Q) into the analytical model (analysis model MQ for input variables).

[0228] [7] In the federated learning system 100 of the above [1], in the second calculation process (learning weight calculation 401-403), each of the multiple client terminals C1-C3 inputs a combination of the first explanatory variable 203 and the first objective variable 204 of the data sample i as a second explanatory variable 301 to the similarity calculation model M1, thereby calculating the second similarity (prediction probability [p j ]) is calculated.

[0229] [8] In the federated learning system 100 according to [4] above, in the second calculation process (learning weight calculations 401 to 403), each of the plurality of client terminals C1 to C3 calculates the specific second similarity (predicted probability [p ji 1 ] t ) according to the learning weight (w1 1 , w2 1 , w3 1 ) is calculated.

[0230] In the first learning process (weighted model learning 411t to 413t), each of the plurality of client terminals C1 to C3 learns the individual analysis model (learning target individual analysis models M21-1t, M22-1t, M23-1t), the first explanatory variable 203, the first objective variable 204, and a specific second similarity (prediction probability [p ji 1 ] t ) according to the learning weight (w1 1 , w2 1 , w3 1 ), and the individual analytical models (learning target individual analytical models M21-1t, M22-1t, M23-1t) are trained based on these.

[0231] [9] Prior to the first associative learning process (FL2), the associative learning system 100 of [1] above executes a second associative learning process (FL1) in which a second learning process (model learning 311t-313t) by each of the multiple client terminals C1-C3 and a second integrated process (integrated learning 330t) by the server S are repeated until a second termination condition is satisfied.

[0232] In the second learning process (model learning 311t to 313t), each of the multiple client terminals C1 to C3 learns learning target similarity calculation models M11t to M13t using a combination of the first explanatory variable 203 and the first objective variable 204 as a second explanatory variable 301 and the client ID 201 as a second objective variable 302.

[0233] In the second integration process (integrated learning 330t), the server S integrates the second learning results 321t to 323t of the learning object similarity calculation models M11t to M13t from the multiple client terminals C1 to C3 by the second learning process (model learning 311t to 313t), and generates an integrated similarity calculation model M1(t+1) by integrating the learning object similarity calculation models (M11t to M13t) of the multiple client terminals C1 to C3 as the similarity calculation model M1.

[0234]

[10] In the federated learning system 100 of [9] above, in the second learning process (model learning 311t to 313t), when the second termination condition is satisfied, each of the multiple client terminals C1 to C3 sets the updated learning target similarity calculation models M11t to M13t to the similarity calculation model M1.

[0235] The present invention is not limited to the above-described embodiments, and includes various modified examples and equivalent configurations within the spirit of the appended claims. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those having all of the configurations described. Also, a part of the configuration of one embodiment may be replaced with a configuration of another embodiment. Also, a configuration of another embodiment may be added to a configuration of one embodiment. Also, a part of the configuration of each embodiment may be added, deleted, or replaced with another configuration.

[0236] Furthermore, each of the aforementioned configurations, functions, processing units, processing means, etc. may be realized in hardware, for example by designing some or all of them as an integrated circuit, or may be realized in software by a processor interpreting and executing a program that realizes each function.

[0237] Information such as programs, tables, files, etc. that realize each function can be stored in a storage device such as a memory, a hard disk, or an SSD (Solid State Drive), or in a recording medium such as an IC (Integrated Circuit) card, an SD card, or a DVD (Digital Versatile Disc).

[0238] In addition, the control lines and information lines shown are those considered necessary for the explanation, and do not necessarily show all the control lines and information lines necessary for implementation. In reality, it can be considered that almost all components are connected to each other. [Explanation of symbols]

[0239] 100 Federated Learning System 203 Explanatory variables 204 Objective Variable 301 Explanatory variables 302 Objective variable 330t Integrated Learning 430t Integrated Learning C1~C3 Client terminals D1~D3 Training dataset M1 Integrated Similarity Computation Model Qr Input Variable QMr Analysis model for input variables

Claims

1. A federated learning system comprising: a plurality of client terminals each having a learning data set; and a server capable of communicating with the plurality of client terminals, the system executing federated learning in which a model is learned using the learning data set in each of the plurality of client terminals, and the server repeats a process of integrating the models of the plurality of client terminals using a learning result; The learning data set has one or more data samples including a client ID that identifies the client terminal, a first explanatory variable, and a first objective variable; executing a first calculation process by the server and a second calculation process by each of the plurality of client terminals, and executing a first combined learning process in which a first learning process by each of the plurality of client terminals and a first integrated process by the server are repeated until a first end condition is satisfied; In the first calculation process, the server Acquire an input data set including a client ID for identifying the client terminal, a first explanatory variable, and a first objective variable; Calculating a first similarity between the input data set and the multiple learning data sets by inputting the input data set into a similarity calculation model that calculates a similarity between the data sample and the multiple learning data sets; In the second calculation process, each of the plurality of client terminals: Calculating a second similarity between the data sample and the multiple learning data sets by inputting the data sample into the similarity calculation model; Executing a similarity calculation between the first similarity and the second similarity to output a learning weight; In the first learning process, each of the plurality of client terminals: learning the first analytical model based on a first analytical model that calculates a predicted value of the first dependent variable from the first explanatory variable for the input data set, the first explanatory variable and the first dependent variable of the learning data set, and the learning weight, and transmitting a first learning result to the server; In the first integration process, the server generating a second analysis model that calculates a predicted value of the first objective variable from the first explanatory variable for the input data set by integrating the first learning results from the first learning process from the plurality of client terminals; A federated learning system characterized by:

2. 2. The federated learning system of claim 1, each of the plurality of client terminals executes the first learning process using the second analytical model generated by the first integration process as the first analytical model; A federated learning system characterized by:

3. 2. The federated learning system of claim 1, the server performs inference by inputting the input data set into the second analytical model if the first termination condition is satisfied. A federated learning system characterized by:

4. A federated learning system comprising: a plurality of client terminals each having a learning data set; and a server capable of communicating with the plurality of client terminals, the system executing federated learning in which a model is learned using the learning data set in each of the plurality of client terminals, and the server repeats a process of integrating the models of the plurality of client terminals using a learning result; The learning data set has one or more data samples including a client ID that identifies the client terminal, a first explanatory variable, and a first objective variable; executing a first calculation process by the server and a second calculation process by each of the plurality of client terminals, and executing a first combined learning process in which a first learning process by each of the plurality of client terminals and a first integrated process by the server are repeated until a first end condition is satisfied; In the first calculation process, the server Acquire an input data set including a client ID for identifying the client terminal, a first explanatory variable, and a first objective variable; Calculating a first similarity between the input data set and the multiple learning data sets by inputting the input data set into a similarity calculation model that calculates a similarity between the data sample and the multiple learning data sets; In the second calculation process, each of the plurality of client terminals: Calculating a second similarity between the data sample and the multiple learning data sets by inputting the data sample into a similarity calculation model that calculates similarities between the data sample and the multiple learning data sets; In the first learning process, each of the plurality of client terminals: an individual analysis model that calculates a predicted value of the first dependent variable from the first explanatory variable, and learning the individual analysis model based on the first explanatory variable, the first dependent variable, and a specific second similarity with a specific learning data set calculated in each of the multiple client terminals by the calculation process; In the first integration process, the server generating an analytical model that calculates a predicted value of the first objective variable from the first explanatory variable for the input data set by integrating a plurality of first learning results by the first learning process from the plurality of client terminals using the first similarity; A federated learning system characterized by:

5. The federated learning system according to claim 4, each of the plurality of client terminals executes the first learning process using the analytical model generated by the first integration process as the individual analytical model; A federated learning system characterized by:

6. The federated learning system according to claim 4, the server performs inference by inputting the input data set into the analytical model if the first termination condition is satisfied. A federated learning system characterized by:

7. 2. The federated learning system of claim 1, In the second calculation process, each of the plurality of client terminals: calculating the second similarity by inputting a combination of the first explanatory variable and the first objective variable of the data sample as a second explanatory variable into the similarity calculation model; A federated learning system characterized by:

8. The federated learning system according to claim 4, In the second calculation process, each of the plurality of client terminals: Calculating a learning weight according to the specific second similarity; In the first learning process, each of the plurality of client terminals: learning the individual analysis model based on the individual analysis model, the first explanatory variable, the first objective variable, and a learning weight corresponding to a specific second similarity calculated in each of the plurality of client terminals by the second calculation process; A federated learning system characterized by:

9. 2. The federated learning system of claim 1, executing a second combined learning process in which, prior to the first combined learning process, a second learning process by each of the plurality of client terminals and a second integration process by the server are repeated until a second end condition is satisfied; In the second learning process, each of the plurality of client terminals: a combination of the first explanatory variable and the first objective variable is set as a second explanatory variable, and the client ID is set as a second objective variable, to learn a learning object similarity calculation model; In the second integration process, the server Integrating second learning results of the learning object similarity calculation models from the plurality of client terminals by the second learning process, and generating an integrated similarity calculation model as the similarity calculation model by integrating the learning object similarity calculation models of the plurality of client terminals. A federated learning system characterized by:

10. 10. The federated learning system of claim 9, In the second learning process, when the second end condition is satisfied, each of the plurality of client terminals sets the updated learning object similarity calculation model as the similarity calculation model. A federated learning system characterized by:

11. A federated learning method for performing federated learning in which a federated learning system including a plurality of client terminals each having a learning data set and a server capable of communicating with the plurality of client terminals repeats a process of learning a model using the learning data set in each of the plurality of client terminals, and the server uses a learning result to integrate the models of the plurality of client terminals, The learning data set has one or more data samples including a client ID that identifies the client terminal, a first explanatory variable, and a first objective variable; The federated learning system includes: executing a first calculation process by the server and a second calculation process by each of the plurality of client terminals, and executing a first combined learning process in which a first learning process by each of the plurality of client terminals and a first integrated process by the server are repeated until a first end condition is satisfied; In the first calculation process, the server Acquire an input data set including a client ID for identifying the client terminal, a first explanatory variable, and a first objective variable; Calculating a first similarity between the input data set and the multiple learning data sets by inputting the input data set into a similarity calculation model that calculates a similarity between the data sample and the multiple learning data sets; In the second calculation process, each of the plurality of client terminals: Calculating a second similarity between the data sample and the multiple learning data sets by inputting the data sample into the similarity calculation model; Executing a similarity calculation between the first similarity and the second similarity to output a learning weight; In the first learning process, each of the plurality of client terminals: learning the first analytical model based on a first analytical model that calculates a predicted value of the first dependent variable from the first explanatory variable for the input data set, the first explanatory variable and the first dependent variable of the learning data set, and the learning weight, and transmitting a first learning result to the server; In the first integration process, the server generating a second analysis model that calculates a predicted value of the first objective variable from the first explanatory variable for the input data set by integrating the first learning results from the first learning process from the plurality of client terminals; A federated learning method comprising:

Images