Associative learning systems, associative learning methods, and associative learning programs

Abstract

To provide an association learning system that realizes model integration through association learning that takes into account inference target data.SOLUTION: In an association learning system, a server takes an input dataset, inputs the input data set to a similarity calculation model, and calculates a first similarity between the input data set and a plurality of learning data sets; each of a plurality of client terminals inputs a data sample into the similarity calculation model to calculate a second similarity between the data sample and the plurality of learning data sets, performs similarity calculation between the first similarity and the second similarity to output a learning weight; and learns a first analytical model on the basis of the first analytical model that calculates a predicted value of a first objective variable from a first explanatory variable for the input data set, the first explanatory variables and the first objective variable of a learning data set, and a learning weight, and transmits a first learning result; and the server integrates the first learning results to generate a second analysis model that calculates a predicted value of the first objective variable from the first explanatory variable for the input data set..SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a federated learning system, a federated learning method, and a federated learning program for performing federated learning. [Background technology]

[0002] With the advancement of digitization of healthcare information, secondary utilization of healthcare information managed by local governments, medical institutions, individuals, etc. (hereinafter referred to as "clients") is progressing. In particular, federative learning, which enables model training in a distributed environment without centralized management of information on a server, is attracting attention from the perspective of protecting personal information.

[0003] Patent Document 1 below discloses a machine learning system and method in federated learning, an integrated server, an information processing device, a program, and a method for creating an inference model. In Patent Document 1 below, each of multiple client terminals classifies data stored in a medical institution based on data acquisition conditions, and classifies training data for each group of data acquired under the same or similar acquisition conditions. Each client terminal performs machine learning of a training model for each group of training data classified by condition category, and sends the respective training results and condition information to the integrated server. The integrated server integrates the received training results by condition category to create multiple master model candidates and evaluates the inference accuracy of each master model candidate. [Prior art documents] [Patent Documents]

[0004] [Patent Document 1] International Publication No. 2021 / 079792 [Overview of the project] [Problems that the invention aims to solve]

[0005] Patent Document 1, mentioned above, describes a method for performing training on each training data set in which the quality of the captured images used for training is generally homogenized. Therefore, it does not generate an integrated model corresponding to the data to be inferred, and does not take into account differences in the distribution of the training data sets.

[0006] The present invention aims to achieve model integration through federated learning that takes into account the data to be inferred. [Means for solving the problem]

[0007] A federated learning system comprising one aspect of the invention disclosed herein comprises a plurality of client terminals, each having a learning dataset, and a server capable of communicating with the plurality of client terminals, wherein each of the plurality of client terminals learns a model using the learning dataset, and the server integrates the models of each of the plurality of client terminals using the learning results, and the federated learning system performs federated learning, wherein the learning dataset has one or more data samples including a client ID that identifies the client terminal, a first explanatory variable, and a first objective variable, and the server performs a first calculation process, each of the plurality of client terminals performs a second calculation process, and the first federated learning process repeats the first learning process by each of the plurality of client terminals and the first integration process by the server until a first termination condition is met, and in the first calculation process, the server, The Input data containing one or more inputs, including one explanatory variable and one dependent variable. variable The input is obtained and used in a similarity calculation model that calculates the similarity between the data sample and the multiple training datasets. variable By entering the above input variable The first similarity between the above-mentioned multiple training datasets is For each of the aforementioned client terminalsIn the second calculation process, each of the multiple client terminals inputs the data sample into the similarity calculation model to calculate the second similarity between the data sample and the multiple training datasets, and outputs the training weights by performing a similarity calculation between the first similarity and the second similarity. In the first training process, each of the multiple client terminals inputs the data sample into the model to calculate the second similarity between the data sample and the multiple training datasets, and outputs the training weights. variable A first analysis model calculates a predicted value of the first target variable from the first explanatory variable, and based on the first explanatory variable and the first target variable of the training dataset and the training weights, the first analysis model is trained and the first training result is sent to the server. In the first integration process, the server integrates the first training results from the first training process from the multiple client terminals, thereby inputting variable This method is characterized by generating a second analytical model that calculates a predicted value of the first dependent variable from the first explanatory variable. [Effects of the Invention]

[0008] According to a typical embodiment of the present invention, model integration can be achieved through federative learning that takes into account the data to be inferred. Other issues, configurations, and effects not mentioned above will be clarified by the following description of the embodiments. [Brief explanation of the drawing]

[0009] [Figure 1] Figure 1 is an explanatory diagram showing an example of associative learning according to Example 1. [Figure 2] Figure 2 is an explanatory diagram showing an example of a training dataset. [Figure 3] Figure 3 is an explanatory diagram illustrating an example of federated learning in a similarity calculation model. [Figure 4] Figure 4 is an explanatory diagram illustrating an example of associative learning in an analysis model for input variables. [Figure 5] Figure 5 is an explanatory diagram illustrating a specific example of the learning weight calculation shown in Figure 4. [Figure 6] Figure 6 is an explanatory diagram illustrating a specific example of model learning using the weighting method shown in Figure 4. [Figure 7] Figure 7 is a sequence diagram of associative learning in the associative learning system according to Example 1. [Figure 8] Figure 8 is an explanatory diagram showing an example of associative learning according to Example 2. [Figure 9] Figure 9 is an explanatory diagram illustrating an example of associative learning in an individual analysis model. [Figure 10] Figure 10 is an explanatory diagram illustrating a specific example of the learning weight calculation shown in Figure 9. [Figure 11] Figure 11 is an explanatory diagram illustrating a specific example of the weighted model learning 911t shown in Figure 9. [Figure 12] Figure 12 is an explanatory diagram illustrating a specific example of the weighted model learning 912t shown in Figure 9. [Figure 13] Figure 13 is an explanatory diagram illustrating a specific example of the weighted model learning 913t shown in Figure 9. [Figure 14] Figure 14 is an explanatory diagram illustrating an example of integrating an analytical model for input variables. [Figure 15] Figure 15 is a sequence diagram of associative learning in the associative learning system according to Example 2. [Figure 16] Figure 16 is an explanatory diagram showing examples of management screens according to Examples 1 to 3. [Figure 17] Figure 17 is a block diagram showing examples of server and client terminal hardware configurations according to Examples 1 to 3. [Modes for carrying out the invention] [Examples]

[0010] <Examples of Associative Learning> Figure 1 is an explanatory diagram showing an example of federated learning according to Embodiment 1. The federated learning system 100 has a server S and multiple client terminals C1 to C3 (three in Figure 1 as an example). When these are not distinguished, they are referred to as client terminal Ck (k=1, 2, 3, ..., K). The number of client terminals Ck is not limited to three, but can be two or more. In this example, K=3. The server S and client terminals Ck are connected in a way that allows communication via a network such as the Internet, LAN (Local Area Network), or WAN (Wide Area Network).

[0011] Client terminals C1 to C3 each possess training datasets D1 to D3. If not distinguished, these will be referred to as training dataset Dk. Training dataset Dk is a combination of training data (explanatory variables) and ground truth data (target variables). The training dataset Dk is prohibited from being removed from client terminal Ck or the location where client terminal Ck is installed.

[0012] The client terminal Ck is a computer that provides the training dataset Dk to the prediction model, trains it individually, and sends the training results, such as the model parameters or gradients of the trained prediction model, to the server S each time training is performed.

[0013] Server S is a computer that uses the learning results from client terminals Ck to integrate the prediction models for each client terminal Ck to generate an integrated prediction model, which it then sends to client terminal Ck. Client terminal Ck then provides the training dataset Dk to the integrated prediction model from Server S to train the prediction model. Through this repeated training, the associative learning system 100 performs associative learning.

[0014] In Example 1, the associative learning system 100 performs two types of associative learning: one is associative learning FL1 for the similarity calculation model, and the other is associative learning FL2 for the analysis model for the input variables.

[0015] The federated learning FL1 of the similarity calculation model is a federated learning process that generates an integrated similarity calculation model M1 by integrating the similarity calculation models from each client terminal Ck by performing the above-described federated learning with the similarity calculation model as the prediction model. The similarity calculation model is defined as the i-th element of the training dataset Dk (hereinafter referred to as training dataset Dj to distinguish it from Dk) of the client terminal Ck (hereinafter referred to as client terminal Cj to distinguish it from Ck) among the client terminals C1 to CK. j An integer satisfying the condition N. j This is a predictive model that calculates the similarity between each data sample (hereinafter referred to as data sample i) of the training dataset Dk and the total number of data samples in the training dataset Dj.

[0016] Specifically, for example, the similarity calculation model is a model that calculates a propensity score using a client ID that uniquely identifies the client terminal Ck as the assigned variable.

[0017] From now on, we will use k for any client terminal and its training dataset, but we will use j for the client terminal being trained and its training dataset.

[0018] By generating the integrated similarity calculation model M1 prior to the federated learning FL2 of the input variable analysis model, the influence of each data sample i on the federated learning in the federated learning FL2 of the input variable analysis model can be adjusted.

[0019] The federated learning FL2 for input variable analysis models is a federated learning process that generates an input variable analysis model MQ using the integrated similarity calculation model M1 generated in federated learning FL1, the input variable Q, and the training dataset Dk. The input variable Q is a matrix consisting of r rows (where r is an integer satisfying 1 ≤ r ≤ R, and R is an integer greater than or equal to 1) of input data to be inferred. Details of the federated learning FL2 for input variable analysis models will be described later. When the input variable Q is input to the input variable analysis model MQ, inference IN is performed and the inference result A is obtained.

[0020] <Training dataset Dk> Figure 2 is an explanatory diagram showing an example of a training dataset Dk. The training dataset Dk has the following fields: Client ID (sometimes referred to as "CID" in subsequent figures) 201, Data ID 202, Explanatory variable 203 (sometimes referred to as Explanatory variable X), and Target variable 204 (sometimes referred to as Target variable y). The combination of values ​​for each field in the same row constitutes an entry that defines one data sample. Each of the training datasets D1 to D3 is, for example, a collection of data samples of patient groups for each hospital.

[0021] Client ID 201 is an identifier that uniquely identifies the client terminal Ck. The value of Client ID 201 is represented by Ck. Data ID 202 is an identifier that uniquely identifies the data sample. The value of Data ID 202 is represented by Dki, where i is a unique number for that data sample. Data ID 202 identifies, for example, a patient. Explanatory variable 203 is, For input variables This is the training data used in the associative learning FL2 of the analytical model, and it contains one or more features x1, x2, ... (if these are not distinguished, they are simply referred to as feature x). Feature x may be, for example, the height, weight, and blood pressure of a patient identified in data ID 202.

[0022] The dependent variable 204 is, For input variablesThis is the ground truth data used in the associative learning FL2 of the analysis model. The dependent variable 204 indicates, for example, whether a patient identified by data ID 202 has the disease. y=1 indicates the presence of the disease, and y=0 indicates the absence of the disease.

[0023] Note that the input variable Q shown in Figure 1 is a combination of the values ​​of x1, x2, ... in the explanatory variable 203 that is the subject of inference IN, and the value of y in the target variable 204. The input variable Qr may be the values ​​of explanatory variable 203 and target variable 204 in any data ID 202 within the training dataset Dk, or it may be the values ​​of explanatory variable 203 and target variable 204 that are not present in the training dataset Dk.

[0024] <Symbiotic learning FL1 for similarity calculation models> Figure 3 is an explanatory diagram showing an example of federated learning FL1 of the similarity calculation model. In federated learning FL1 of the similarity calculation model, the combination of explanatory variable 203 and target variable 204 of the training dataset Dk becomes explanatory variable 301, and client ID 201 becomes target variable 302.

[0025] Server S has a base similarity calculation model (hereinafter referred to as the base similarity calculation model) M10. The base similarity calculation model M10 may be an untrained neural network, and model parameters such as weights and biases φ k t A pre-trained neural network with the necessary settings may also be used. t is a natural number in ascending order, starting from 1, indicating the number of times the federated learning FL1 of the similarity calculation model is executed. Server S sends the base similarity calculation model M10 to client terminals C1 to C3.

[0026] Furthermore, if the client terminal Ck has an untrained neural network, the server S will use the model parameter φ k t The client terminal Ck receives the model parameter φ from the server S. k tBy setting this in an untrained neural network, the base similarity calculation model M10 may be constructed.

[0027] The base similarity calculation model M10 serves as the similarity calculation model M11t to be trained for the first federated learning FL1 at the client terminal Ck.

[0028] The client terminal Ck executes model learning 31kt (311t to 313t) in the t-th federated learning FL1. Specifically, for example, the client terminal Ck inputs the explanatory variable 301 and the objective variable 302 of the training data set D1 to the similarity calculation model M1kt (M11t to M13t) to be trained, and performs individual learning. The client terminal Ck transmits a learning result 32kt (321t to 323t), which is the model parameter φ k t or a gradient gs thereof k t , to the server S.

[0029] The server S uses the learning result 32kt to execute integrated learning 330t in the t-th federated learning FL1, and generates an integrated similarity calculation model M1(t+1) for the next round. Specifically, for example, the server S generates the integrated similarity calculation model M1(t+1) using an integrated result obtained by integrating the learning results 32kt, and transmits the integrated similarity calculation model M1(t+1) or the integrated model parameter φ t+1 to the client terminal Ck. Accordingly, the similarity calculation model M1kt to be trained for the next round of federated learning FL1 is set in the client terminal Ck.

[0030] In this way, federated learning FL1 is repeatedly executed. When the number of executions t reaches a predetermined threshold T1, or the accuracy of the integrated similarity calculation model M1t reaches the target accuracy, the server S terminates the federated learning FL1, outputs the latest integrated similarity calculation model M1(t+1) as the integrated similarity calculation model M1, and transmits the model to client terminals C1 to C3.

[0031] The following defines the formulas used in the federated learning FL1 of the similarity calculation model.

[0032] [p j ] t =h(X,y;φ j t )···(1)

[0033] The above equation (1) is a calculation formula that defines the similarity calculation model and is executed by the model training 31jt of the client terminal Cj. Function h is a combination of explanatory variables 301, which is the combination of explanatory variables 203(X) and the target variable 204(y), and the model parameter φ j t This is the learning target similarity calculation model M1jt (M11t~M13t) defined by [p j ] t This is the predicted probability indicating which training dataset Dk each data sample i of the training dataset Dj on client terminal Cj is similar to in the t-th associative learning FL1.

[0034] The total number of data samples i in the training dataset Dj is N. j Then, the predicted probability [p j ] t is N j It is a matrix of size ×K. That is, the row vector [p ji ] t These are the predicted probabilities showing the similarity between data sample i of training dataset Dj and training dataset D1, the predicted probabilities showing the similarity between data sample i of training dataset Dj and training dataset D2, the predicted probabilities showing the similarity between data sample i of training dataset Dj and training dataset D3, ..., and the predicted probabilities showing the similarity between data sample i of training dataset Dj and training dataset DK.

[0035] Also, the predicted probability [p j ] t N in the kth column of the matrix j A column vector [p j k ] tThis represents the predicted probability of similarity between the first data sample i (i=1) of the training dataset Dj and the training dataset Dk, the predicted probability of similarity between the second data sample i (i=2) of the training dataset Dj and the training dataset Dk, the predicted probability of similarity between the third data sample i (i=3) of the training dataset Dj and the training dataset Dk, ..., N of the training dataset Dj. j i=N j This is the predicted probability showing the similarity between the data sample i and the training dataset Dk.

[0036]

number

[0037] Equation (2) above is the loss function H(φ) calculated by the model training 31jt of client terminal Cj in the t-th associative learning FL1. j t ) is p ji This is the similarity of data sample i from the training dataset Dj to the training datasets D1-D3. ji For example, it has a range of 0.0 to 1.0, with larger values ​​indicating greater similarity.

[0038] If j=1, then p ji =( 0,0,1.0 ) indicates that data sample i is a data sample from the training dataset D1. If j=2, then p ji =(0,1.0,0), indicating that data sample i is a data sample from the training dataset D2. If j=3, then p ji =(1.0,0,0), indicating that data sample i is a data sample from the training dataset D3.

[0039] [p ji ] t This is the predicted probability [p] in the tth associative learning LF1. j ] tThe row vectors in the matrix shown represent the predicted probabilities indicating how similar data sample i is to the training datasets D1-D3.

[0040] N j is the total number of data samples i in the training dataset Dj. The function loss is the error function of data sample i, and the average of the error function loss for each data sample i is the loss function H(φ j t )

[0041]

number

[0042] Equation (3) above represents the model parameter φ j t gradient gs j t This is a formula that defines the gradient g j t The result 32jt is sent to server S.

[0043]

number

[0044] Equation (4) above represents the integrated model parameter φ j t Integrated model parameter φ j t+1 This is the calculation formula to update the gradient gs, which is executed by the integrated learning 330t of server S. Server S receives the gradient gs from the client terminal Cj. j t The learning result 32jt is received, and equation (4) above is executed. Note that the first term on the right-hand side of equation (4) above is the integrated model parameter φ tThese are the integrated model parameters calculated as a result of executing equation (4) above in the previous associative learning FL1.

[0045]

number

[0046] Equation (5) above represents the model parameter φ j t Model parameter φ j t+1 This is a calculation formula that updates to , client terminal Cj This is performed by model training 31jt. When applying the above equation (5), the client terminal Cj will have the updated model parameters φ j t+1 The result 32jt is sent to server S.

[0047] Equation (6) above is the updated model parameter φ from equation (5) above. j t+1 Using the integrated model parameter φ t+1 This is a calculation formula that is performed by the integrated learning 330t of server S. N is the number of client terminals C1 to C j This is the total number of data samples i.

[0048] The associative learning system 100 uses either the update method according to equation (4) or the update methods according to equations (5) and (6) to update the integrated model parameter φ t+1 The server S calculates the integrated similarity calculation model M1t (or the base similarity calculation model M10 when t=0) using the integrated model parameter φ. t+1 Update and generate the integrated similarity calculation model M1(t+1).

[0049] Server S is the integrated similarity calculation model M1(t+1) or its integrated model parameters φ t+1 By sending this to client terminals C1-C3, the above equation (1) is updated.

[0050] [p j ] t+1=h(X,y;φ t+1 )···(7)

[0051] Equation (7) above is a calculation formula that defines the integrated similarity calculation model M1(t+1), which is generated by the integrated learning 330 of server S, sent to client terminal Ck, and becomes the learning target similarity calculation model M1kt in the next federated learning FL1.

[0052] [p j ]=h(X,y;φ)···(8)

[0053] Equation (8) above is the formula that defines the integrated similarity calculation model M1 when associative learning FL1 is completed. Since it is the same formula as equation (7) above at the end of associative learning FL1, "t+1" has been removed from equation (7).

[0054] <Federation Learning FL2 for Analytical Models for Input Variables> Figure 4 is an explanatory diagram showing an example of federated learning FL2 of the analysis model for input variables. Client terminals C1 to C3 perform learning weight calculations 401 to 403. Specifically, for example, client terminal Cj inputs the explanatory variable 301 of the training dataset Dj into the integrated similarity calculation model M1 (equation (8) above) generated in Figure 3, thereby predicting the probability [p] that each data sample i of the training dataset Dj is similar to which training dataset Dk. j The client terminal Cj calculates the calculated predicted probability [p j Using the input variable similarity QSr, the learning weight w j Calculate.

[0055] In the input variable similarity calculation 404, server S calculates the input variable similarity QS by inputting the input variable Q into the integrated similarity calculation model M1. The input variable similarity QS is used in the learning weight calculations 401-403.

[0056] Furthermore, Server S has a base analysis model for input variables (hereinafter referred to as the base analysis model) M20. The base analysis model M20 may be an untrained neural network, and the model parameters θ such as weights and biases are... t A pre-trained neural network with the following settings may also be used. Model parameter θ t is the model parameters generated in the t-th associative learning FL2 given the input variable Q. t is a natural number in ascending order, starting from 1, indicating the number of runs of the associative learning FL2 of the analysis model for the input variable. Server S sends the base analysis model M20 to client terminal Ck.

[0057] The base analysis model M20 becomes the analysis model M2j-1 for the input variables to be learned in the associative learning FL2 at t=1 on the client terminal Cj. For associative learning FL2 from t=2 onwards, the analysis model M2j-t (t≧2) for the input variables to be learned is sent to the client terminal Ck as the base analysis model.

[0058] Furthermore, if client terminal Cj has an untrained neural network, server S will use the model parameter θ. t The client terminal Ck receives the model parameter θ from the server S. t The base individual analysis model M20 may be constructed by setting it to an untrained neural network.

[0059] If j=1, client terminal C1 holds the analysis model M21-t for the input variables to be trained. If j=2, client terminal C2 holds the analysis model M22-t for the input variables to be trained. If j=3, client terminal C3 holds the analysis model M23-t for the input variables to be trained.

[0060] The client terminal Cj executes weighted model learning 41jt. Specifically, for example, the client terminal Cj inputs the explanatory variable 203 of the learning data set Dj into the analysis model for learning target input variable M2j-t, thereby obtaining, for each analysis model for learning target input variable M2j-t, the predicted value [y of the objective variable 204 j . The client terminal Cj uses the learning weight w j , the objective variable y, the predicted value [y j and the model parameter θ j t to calculate the loss function F(θ j t ) for each analysis model for learning target input variable M2j-t.

[0061] Then, the client terminal Cj calculates the gradient ga of the model parameter θ j t as the input variable analysis model learning result 42j-t, updates the model parameter θ j t to the model parameter θ j t to the model parameter θ j (t+1) . The model parameter θ j (t+1) is also included in the input variable analysis model learning result 42j- t .

[0062] When j=1, the client terminal C1 generates the input variable analysis model learning result 421-t. The input variable analysis model learning result 421-t is the analysis model for learning target input variable M21- t the updated model parameter θ1 t or the gradient ga1 thereof t .

[0063] When j=2, the client terminal C2 generates the input variable analysis model learning result 422-t. The input variable analysis model learning result 422-t is the analysis model for learning target input variable M22- t the updated model parameter θ2 t or the gradient ga2 thereof tThat is the case.

[0064] When j=3, client terminal C3 generates the input variable analysis model training result 423-t. The input variable analysis model training result 423-t contains the model parameters θ3 updated in the training target input variable analysis model M23-t. t or its gradient ga3 t That is the case.

[0065] Server S receives the analysis model training results 42j- for input variables from client terminal Ck. t It receives. Specifically, for example, server S receives the learning results 421- of the analysis model for input variables from client terminal C1. t The client terminal C2 receives the learning results of the analysis model for input variables 422- t The client terminal C3 received the learning results 423- for the analysis model for the input variables. t Receive.

[0066] Server S performs integrated learning 430t in the t-th associative learning FL2 and generates the next input variable analysis model MQ(t+1). Specifically, for example, Server S integrates the input variable analysis model learning results 421-t, 422-t, and 423-t to generate the input variable analysis model MQ(t+1).

[0067] Server S uses the analysis model MQ(t+1) for the input variables or its integrated model parameters θ. (t+1) This is sent to the client terminal Ck. As a result, the client terminal Ck is configured with the analysis model M2j-t for the input variables to be learned in the next associative learning FL2.

[0068] Specifically, for example, server S uses the analysis model MQ(t+1) for the input variables or its integrated model parameter θ. (t+1)This information is sent to client terminals C1 to C3. Using this received information, client terminal C1 sets up the analysis model M21-t for the input variables to be learned in the next associative learning FL2 session, client terminal C2 sets up the analysis model M22-t for the input variables to be learned in the next associative learning FL2 session, and client terminal C3 sets up the analysis model M23-t for the input variables to be learned in the next associative learning FL2 session.

[0069] In this way, the associative learning FL2 is repeatedly executed. When the number of executions t reaches a predetermined threshold T2, or when the accuracy of all input variable analysis models MQ reaches the target accuracy, the server S terminates the associative learning FL2 and outputs the latest input variable analysis model MQ(t+1) as the input variable analysis model MQ.

[0070] <Calculation of learning weights: 40k> Figure 5 is an explanatory diagram illustrating a specific example of the learning weight calculation shown in Figure 4. In Figure 5, the learning dataset D1 has data samples D11 and D12 as data sample i, the learning dataset D2 has data samples D21 and D22 as data sample i, and the learning dataset D3 has data samples D31 and D32 as data sample i. The server S has input data Q1 and Q2, which have values ​​for the explanatory variable 203, as input variable Q.

[0071] In the input variable similarity calculation 404, server S inputs the input variable Q into the integrated similarity calculation model M1 and calculates the predicted probability [pq] as the input variable similarity QS.

[0072] The client terminal Cj inputs the explanatory variable 301 of the training dataset Dj into the integrated similarity calculation model M1 and predicts the probability [p j Calculate the predicted probability [p j ] represents the predicted probability for each data sample i [p ji It is composed of ]. Predicted probability [p ji] is a row vector that shows the predicted probability of how similar a given data sample i in the training dataset Dj is to each training dataset Dk.

[0073] Specifically, for example, when j=1, client terminal C1 inputs explanatory variable 301 from training dataset D1 into integrated similarity calculation model M1 to calculate the predicted probability [p1]. By inputting explanatory variable 301 from data sample D11 into integrated similarity calculation model M1, the predicted probability [p 11 The following is calculated, and by inputting the explanatory variable 301 of data sample D12 into the integrated similarity calculation model M1, the predicted probability [p 12 The result is calculated.

[0074] In the learning weight calculation 401, client terminal C1 uses the predicted probability [p1] and the predicted probability [pq r The similarity calculation is performed with ]. Specifically, for example, client terminal C1 calculates the predicted probability [p 11 The similarity between [ ] and the predicted probability [pq1] corresponding to the input data Q1 is calculated. The calculated similarity is used as the learning weight w 11 1 Similarly, client terminal C1 predicts the probability [p] corresponding to data sample D12. 12 The similarity between [ ] and the predicted probability [pq1] corresponding to the input data Q1 is calculated. The calculated similarity is used as the learning weight w 12 1 Let's assume the learning weight is w. 11 1 ,w 12 1 Combine these, and learn weight w1 1 Let's assume that.

[0075] Furthermore, when j=2, client terminal C2 inputs explanatory variable 301 from the training dataset D2 into the integrated similarity calculation model M1 to calculate the predicted probability [p2]. By inputting explanatory variable 301 from data sample D21 into the integrated similarity calculation model M1, the predicted probability [p 21The following is calculated, and by inputting the explanatory variable 301 of data sample D22 into the integrated similarity calculation model M1, the predicted probability [p 22 The result is calculated.

[0076] In the learning weight calculation 402, client terminal C2 uses the predicted probability [p2] and the predicted probability [pq]. r The similarity calculation is performed with ]. Specifically, for example, client terminal C2 calculates the predicted probability [p 21 The similarity between [ ] and the predicted probability [pq1] corresponding to the input data Q1 is calculated. The calculated similarity is used as the learning weight w 21 1 Similarly, client terminal C2 predicts the probability [p] corresponding to data sample D22. 22 The similarity between [ ] and the predicted probability [pq1] corresponding to the input data Q1 is calculated. The calculated similarity is used as the learning weight w 22 1 Let's assume the learning weight is w. 21 1 ,w 22 1 Combine these, and learn weight w2 1 Let's assume that.

[0077] Furthermore, when j=3, client terminal C3 inputs explanatory variable 301 from training dataset D33 into integrated similarity calculation model M1 to calculate the predicted probability [p3]. By inputting explanatory variable 301 from data sample D31 into integrated similarity calculation model M1, the predicted probability [p 31 The following is calculated, and by inputting the explanatory variable 301 of data sample D32 into the integrated similarity calculation model M1, the predicted probability [p 32 The result is calculated.

[0078] In the learning weight calculation 403, client terminal C3 uses the predicted probability [p3] and the predicted probability [pq]. r The similarity calculation is performed with ]. Specifically, for example, client terminal C3 calculates the predicted probability [p 31The similarity between [ ] and the predicted probability [pq1] corresponding to the input data Q1 is calculated. The calculated similarity is used as the learning weight w 31 1 Similarly, client terminal C3 predicts the probability [p] corresponding to data sample D32. 32 The similarity between [ ] and the predicted probability [pq1] corresponding to the input data Q1 is calculated. The calculated similarity is used as the learning weight w 32 1 Let's assume the learning weight is w. 31 1 ,w 32 1 Combine these and learn weights w3 1 Let's assume that.

[0079] Below are the training weights w in the associative learning FL2 of the analysis model for input variables. ji r Formula (9) used in the calculation of is defined.

[0080]

number

[0081] w on the left side ji r [p] is the training weight applied to the similarity relationship between the i-th data sample Dji of the training dataset Dj and the input data Qri of the input variable Qr. ji [pq] is the predicted probability corresponding to the data sample Dji. r ]teeth, Input Variables This is the predicted probability corresponding to Qr. α on the right-hand side is a parameter that adjusts the degree of distance. Thus, the learning weight w ji r is the predicted probability [p ji [pq] is a vector representing the predicted probability. r It is represented by the reciprocal of the distance between the vectors that indicate [ ] and [ ].

[0082] <Weighted Model Learning 41jt> Next, we will explain a specific example of the weighted model training 41jt (411t~413t) shown in Figure 4.

[0083] Figure 6 is an explanatory diagram showing a specific example of the weighted model training 411t shown in Figure 4. Client terminal C1 updates the analysis model M21-t for the input variables to be trained using the explanatory variables 203 and the target variable 204 of the training dataset D1, and the training weights w1, and outputs the analysis model training result 421-t for the input variables.

[0084] Client terminal C2 updates the analysis model M22-t for the input variables to be trained using the explanatory variables 203 and the target variable 204 from the training dataset D2, along with the training weights w2, and outputs the training result 422-t for the analysis model for the input variables.

[0085] Client terminal C3 updates the analysis model M23-t for the input variables to be trained using the explanatory variables 203 and the target variable 204 from the training dataset D3, along with the training weights w3, and outputs the training result 423-t for the analysis model for the input variables.

[0086] Server S integrates the input variable analysis model training results 421-t, 422-t, and 423-t in integrated learning 430t to generate the next input variable analysis model MQ(t+1). Server S then generates the input variable analysis model MQ(t+1) or its integrated model parameters θ. (t+1) This information is sent to client terminals C1 to C3. Using this received information, client terminal C1 sets up the analysis model M21-t for the input variables to be learned in the next associative learning FL2 session, client terminal C2 sets up the analysis model M22-t for the input variables to be learned in the next associative learning FL2 session, and client terminal C3 sets up the analysis model M23-t for the input variables to be learned in the next associative learning FL2 session. The input variable analysis model MQ is the analysis model that was finalized at the end of associative learning FL2.

[0087] The following defines equation (10) used in the associative learning FL2 of the analysis model for input variables.

[0088] [y]=f(x q ;θq )···(10)

[0089] Equation (10) above defines the analysis model MQ for the input variables. q θ is an explanatory variable for the input variable Q, and q teeth For input variables These are the integrated model parameters of the analysis model MQ. [y] is the column vector of predicted values ​​of the target variable 204 for each data sample i of the input variable Q, i.e., the inference result Ar.

[0090]

number

[0091] Equation (11) above represents the loss function F(θ) in the analysis model M2j-t for the input variable to be learned at client terminal Cj during the t-th associative learning FL2. j t ) defines y ji The target variable is 204 in a data sample i of the training dataset Dj, and [y ji ] is its predicted value. [y ji ] is [y j ] is. w ji r This is the learning weight calculated using equation (9) above.

[0092]

number

[0093] Equation (12) above represents the model parameter θ in the tth associative learning FL2. j t The gradient ga j t This defines the gradient ga j t This is calculated on the client terminal Cj.

[0094]

number

[0095] Equation (13) above represents each model parameter θ in the tth associative learning FL2. j t This is the calculation formula that performs the update and is executed by model training 41jt with weighting of client terminal Cj. W on the right side of equation (13) above j r As shown in equation (14) above, the learning weight w of the client terminal Cj in the t-th associative learning FL2 is ji r This is the sum of the values ​​of W on the right side of equation (13) above. r As shown in equation (15) above, the learning weights W1 of client terminals C1 to CK r ~W K r It is the sum of [the numbers].

[0096] Client terminal Cj updates model parameter θ1 (t+1) ~θ3 (t+1) The results of the analysis model training for the input variables, 421-t, 422-t, and 423-t, are sent to the server S. In integrated training 430t, the server S uses the analysis model training results for the input variables 421-t (updated model parameters θ1 (t+1) ), Analysis model learning results for input variables 422-t (updated model parameters θ2 (t+1) ), Analysis model learning results for input variables 423-t (updated model parameters θ3 (t+1) By, for example, averaging and integrating the data, the updated integrated model parameters θ for client terminals C1-C3 can be obtained. (t+1) The following is calculated and the analysis model MQ(t+1) for the input variables is generated.

[0097]

number

[0098] Equation (16) above calculates the gradient ga from the client terminal Cj as the learning result 42j-t of the analysis model for the input variable. j tWhen a signal is received, each model parameter θ in the tth associative learning FL2 is considered. j t This is the calculation formula that performs the update, and it is executed on server S's integrated learning 430t.

[0099] Equation (17) above calculates the gradient ga from the client terminal Cj as the learning result 42j-t of the analysis model for the input variable. j t When a signal is received, the integrated model parameter θ is used in the tth associative learning FL2. t This is the calculation formula that performs the update, and is executed in the integrated learning 430t of server S. Specifically, for example, server S will perform the update of the integrated model parameters θ j t+1 and the learning weight W j r and the learning weight W r Using this, the model parameter θ of the analysis model MQ(t+1) for the input variable (t+1) This generates the analysis model MQ(t+1) for the input variables.

[0100] <Associative Learning Sequence> Figure 7 is a sequence diagram of federated learning in the federated learning system 100 according to Embodiment 1. First, prior to federated learning FL1 and FL2, the server S sends client ID 201 to each client terminal Ck (step S701). The client terminal Ck associates client ID 201 with the training dataset Dk.

[0101] Next, the associative learning system 100 performs associative learning FL1 of the similarity calculation model (step S702) and associative learning FL2 of the analysis model for the input variables (step S703).

[0102] In the federated learning FL1 (step S702) of the similarity calculation model, the server S sends similarity calculation model information (integrated similarity calculation model M1(t+1) or its integrated model parameters φ) to the client terminal Ck. t+1 Send (step S721).

[0103] The client terminal Cj trains the target similarity calculation model M1jt using the training dataset Dj and similarity calculation model information (step S722).

[0104] The client terminal Cj sends the learning result 32jt from step S722 to the server S (step S723).

[0105] Server S performs integrated training 330t of the training target similarity calculation model M1jt using the training result 32jt (step S724). Server S determines whether the termination conditions for the integrated training 330t of the training target similarity calculation model M1jt (step S724) are met (step S725).

[0106] If the termination conditions are not met, the server S sends the updated integrated similarity calculation model information (integrated similarity calculation model M1(t+1) or its model parameters) to the client terminal Cj (step S721).

[0107] If the termination conditions are met, server S registers the updated integrated similarity calculation model information (integrated similarity calculation model M1 or its model parameters) (step S726). This completes the federated learning FL1 of the similarity calculation model (step S702).

[0108] A user terminal 700 capable of communicating with server S receives input of input variable Q (step S703) and sends input variable Q to server S (step S704). User terminal 700 may also be a client terminal Cj. Alternatively, input variable Q may be input directly to server S.

[0109] In the federated learning FL2 of the analysis model for input variables (step S705), server S obtains the input variable Q (step S731). Next, server S uses the integrated similarity calculation model M1 to perform the input variable similarity calculation 404 and outputs the input variable similarity QS (step S732).

[0110] Server S sends the input variable similarity QS to client terminal Cj (step S733). Client terminal Cj calculates the learning weight wj using the input variable similarity QS, as shown in Figure 5 (step S734).

[0111] If server S is t=1, then the base analysis model M20 (or its model parameters θ) 1 ) If t≧2, then the analysis model M2j-t for the input variable to be learned (or its model parameters θ) j t The following information is sent to the client terminal Cj as analysis model information for input variables (step S735).

[0112] As shown in Figure 6, the client terminal Cj performs weighted model learning to generate the analysis model learning result 42j-t for the input variables (step S736), and sends the analysis model learning result 42j-t for the input variables to the server S (step S737).

[0113] As shown in Figure 6, Server S obtains the input variable analysis model training result 42j-t from client terminal Cj, performs integrated training 430t, and generates the t+1th input variable analysis model MQ(t+1) (step S738).

[0114] Server S determines whether the termination conditions for integrated learning 430t (step S738) are met (step S739).

[0115] If the termination conditions are not met, server S sends the updated analysis model information for the input variables to client terminal Cj. As a result, client terminal Cj updates the analysis model M2j-t for the input variables to be trained.

[0116] If the termination conditions are met, the server S registers the updated input variable analysis model MQ(t+1) as the input variable analysis model MQ (step S740). This completes the associative learning FL2 of the input variable analysis model (step S705).

[0117] Thereafter, the server S executes inference IN by inputting the input variable Q into the input variable analysis model MQ (step S706). Then, the server S transmits the inference result A to the user terminal 700 (step S707).

[0118] As described above, according to the first embodiment, the input variable analysis model MQ that considers the value of the input variable Q can be generated as an integrated model obtained by integrating the learning target input variable analysis models M21-t to M23-t of the client terminals C1 to C3.

Examples

[0119] Next, a second embodiment will be described. In the second embodiment, an example will be described in which federated learning of individual analysis models is executed instead of the federated learning FL2 for input variable analysis models. Note that, in the second embodiment, description will be given focusing on differences from the first embodiment, and thus description of common parts with the first embodiment is omitted.

[0120] FIG. 8 is an explanatory diagram showing an example of federated learning according to the second embodiment. Federated learning FL3 of individual analysis models is federated learning that uses the integrated similarity calculation model M1 to generate individual analysis models M3j (M31 to M33) for the client terminal Cj obtained by each of the client terminals C1 to C3.

[0121] Specifically, for example, the individual analysis model M31 is a prediction model obtained by integrating individual analysis models for the client terminal C1 (j=1) from the client terminals C1 to C3, the individual analysis model M32 is a prediction model obtained by integrating individual analysis models for the client terminal C2 (j=2) from the client terminals C1 to C3, and the individual analysis model M33 is a prediction model obtained by integrating individual analysis models for the client terminal C3 (j=3) from the client terminals C1 to C3.

[0122] By executing the federated learning FL3 of individual analysis models, an appropriate individual analysis model M3j (M31 to M33) is generated for each client terminal Cj.

[0123] Server S executes an input variable analysis model integration 800, which integrates the individual analysis models M31 to M33 using the input variable similarity QS obtained in the input variable similarity calculation 404, and generates the input variable analysis model MQ. When the input variable Q is input to the input variable analysis model MQ, inference IN is executed and the inference result A is obtained.

[0124] <Associative Learning FL3 for Individual Analysis Models> Figure 9 is an explanatory diagram showing an example of federated learning FL3 of individual analysis models. Client terminal C1 performs the learning weight calculation 901. Specifically, for example, client terminal Cj inputs the explanatory variables 301 of the learning dataset Dj into the integrated similarity calculation model M1 (equation (8) above) generated in Figure 3, thereby predicting the probability [p] that each data sample i of the learning dataset Dj is similar to which learning dataset Dk. j The client terminal Cj calculates the calculated predicted probability [p j The learning weights w9j are calculated using [ ].

[0125] Server S has a base individual analysis model (hereinafter referred to as the base individual analysis model) M20. The base individual analysis model M20 may be an untrained neural network, and model parameters such as weights and biases θ t A pre-trained neural network with the necessary settings may also be used. Server S sends the base individual analysis model M20 to client terminal Ck.

[0126] Furthermore, if the client terminal Ck has an untrained neural network, the server S will use the model parameter θ. t The client terminal Ck receives the model parameter θ from the server S. t The base individual analysis model M20 can be constructed by setting it to an untrained neural network.

[0127] The base individual analysis model M20 undergoes first federative learning FL on client terminal Cj. 3 The individual analysis model M2j-kt is the target of learning. t is the federated learning FL of the individual analysis model. 3 This is a natural number in ascending order, starting from 1, indicating the number of times the operation has been performed.

[0128] If j=1, client terminal C1 holds the individual analysis models M31-1t to M31-3t. If j=2, client terminal C2 holds the individual analysis models M32-1t to M32-3t. If j=3, client terminal C3 holds the individual analysis models M33-1t to M33-3t.

[0129] The client terminal Cj performs weighted model training 91jt. Specifically, for example, the client terminal Cj inputs the explanatory variables 203 of the training dataset Dj into each of the individual analysis models M3j-kt, thereby predicting the value of the target variable 204 [y j k The client terminal Cj calculates the learning weights w9j, the target variable y, and the predicted value [y j k ] and the model parameter θ j kt Therefore, for each individual analysis model M3j-kt that is being trained, the loss function F(θ j kt Calculate ).

[0130] Then, the client terminal Cj has the model parameter θ. j kt The gradient ga j kt The individual analysis model learning result 92j-kt was calculated, and the model parameter θ was calculated. j kt Model parameter θ j k(t+1) Update to the model parameter θ. j k(t+1) This is also included in the individual analysis model learning results 92j-kt.

[0131] When j=1, client terminal C1 generates the individual analysis model training results 921-1t to 921-3t. The individual analysis model training result 921-1t is the model parameter θ1 updated in the training individual analysis model M31-1t. 1t or its gradient ga1 1t The individual analysis model learning result 921-2t is obtained by updating the model parameter θ1 in the individual analysis model M31-2t that was the target of the learning. 2t or its gradient ga1 2t The individual analysis model learning result 921-3t is obtained by updating the model parameter θ1 in the individual analysis model M31-3t that was the target of the learning. 3t or its gradient ga1 3t That is the case.

[0132] When j=2, client terminal C2 generates the individual analysis model training results 922-1t to 922-3t. The individual analysis model training result 922-1t is the model parameter θ2 updated in the training individual analysis model M32-1t. 1t or its gradient ga2 1t The individual analysis model learning result 922-2t is obtained by updating the model parameter θ2 in the individual analysis model M32-2t that was the target of the learning. 2t or its gradient ga2 2t The individual analysis model learning result 922-3t is obtained by updating the model parameter θ2 in the individual analysis model M32-3t that was the target of the learning. 3t or its gradient ga2 3t That is the case.

[0133] When j=3, client terminal C3 generates the individual analysis model training results 923-1t to 923-3t. The individual analysis model training result 923-1t is the model parameter θ3 updated in the individual analysis model M33-1t being trained. 1t or its gradient ga3 1t The individual analysis model learning result 923-2t is obtained by updating the model parameter θ3 in the individual analysis model M33-2t that was the target of the learning. 2t or its gradient ga3 2tThe individual analysis model learning result 923-3t is obtained by updating the model parameter θ3 in the individual analysis model M33-3t that was the target of the learning. 3t or its gradient ga3 3t That is the case.

[0134] Server S receives individual analysis model training results 92j-kt from client terminal Ck. Specifically, for example, Server S receives individual analysis model training results 921-1t~921-3t from client terminal C1, individual analysis model training results 922-1t~922-3t from client terminal C2, and individual analysis model training results 923-1t~923-3t from client terminal C3.

[0135] Server S performs its tth associative learning FL 3 The integrated learning process 930t is performed to generate the individual analysis model M3k(t+1) for the next iteration. Specifically, for example, Server S integrates the individual analysis model learning results 921-1t, 922-1t, and 923-1t to generate individual analysis model M31(t+1). Server S also integrates the individual analysis model learning results 921-2t, 922-2t, and 923-2t to generate individual analysis model M32(t+1). Server S also integrates the individual analysis model learning results 921-3t, 922-3t, and 923-3t to generate individual analysis model M33(t+1).

[0136] Server S is the individual analysis model M3j(t+1) or its integrated model parameter θ j t+1 This is sent to the client terminal Ck. As a result, the client terminal Ck is configured with the individual analysis model M3j-kt, which will be the target of learning in the next associative learning FL3.

[0137] Specifically, for example, server S is the individual analysis model M31(t+1) or its integrated model parameter θ1 t+1is transmitted to client terminals C1 to C3. Accordingly, using this received information, the client terminal C1 sets the individual analysis model M31-1t to be learned in the next round of federated learning FL3, the client terminal C2 sets the individual analysis model M32-1t to be learned in the next round of federated learning FL3, and the client terminal C3 sets the individual analysis model M33-1t to be learned in the next round of federated learning FL3.

[0138] Further, the server S transmits the individual analysis model M32(t+1) or the integrated model parameter θ2 thereof t+1 to the client terminals C1 to C3. Accordingly, using this received information, the client terminal C1 sets the individual analysis model M31-2t to be learned in the next round of federated learning FL3, the client terminal C2 sets the individual analysis model M32-2t to be learned in the next round of federated learning FL3, and the client terminal C3 sets the individual analysis model M33-2t to be learned in the next round of federated learning FL3.

[0139] Further, the server S transmits the individual analysis model M33(t+1) or the integrated model parameter θ3 thereof t+1 to the client terminals C1 to C3. Accordingly, using this received information, the client terminal C1 sets the individual analysis model M31-3t to be learned in the next round of federated learning FL3, the client terminal C2 sets the individual analysis model M32-3t to be learned in the next round of federated learning FL3, and the client terminal C3 sets the individual analysis model M33-3t to be learned in the next round of federated learning FL3.

[0140] In this way, federated learning FL3 is repeatedly executed. When the number of executions t reaches a predetermined threshold T2, or all the accuracies of the individual analysis models M31(t+1) to M3K(t+1) reach the target accuracy, the server S terminates the federated learning FL3, outputs the latest individual analysis model M3k(t+1) as the individual analysis model M3k, and transmits the same to the client terminal Ck.

[0141] Specifically, for example, server S outputs individual analysis model M31(t+1) as individual analysis model M31 and sends it to client terminal C1. Server S also outputs individual analysis model M32(t+1) as individual analysis model M32 and sends it to client terminal C2. Server S also outputs individual analysis model M33(t+1) as individual analysis model M33 and sends it to client terminal C3.

[0142] <Calculation of Learning Weights> 9 0k> Figure 10 is an explanatory diagram illustrating a specific example of the learning weight calculation 90k (901-903) shown in Figure 9. In Figure 10, the learning dataset D1 has data samples D11 and D12, the learning dataset D2 has data samples D21 and D22, and the learning dataset D3 has data samples D31 and D32.

[0143] The client terminal Cj inputs the explanatory variable 301 of the training dataset Dj into the integrated similarity calculation model M1 and predicts the probability [p j Calculate the predicted probability [p j The column with CID=k in ] represents the predicted probability [p], which indicates how similar all data samples i in the training dataset Dj are to the training dataset Dk. j k This is a column vector representing ]. Also, the predicted probability [p ji ] is a row vector that shows the predicted probability of how similar a given data sample i in the training dataset Dj is to each training dataset Dk.

[0144] Specifically, for example, when j=1, client terminal C1 inputs explanatory variable 301 from training dataset D1 into integrated similarity calculation model M1 to calculate the predicted probability [p1]. By inputting explanatory variable 301 from data sample D11 into integrated similarity calculation model M1, the predicted probability [p 11 The following is calculated, and by inputting the explanatory variable 301 of data sample D12 into the integrated similarity calculation model M1, the predicted probability [p12 The following is calculated: The column CID=1~3 in the predicted probability [p1] shows how similar each data sample i of the training dataset D1 is to the training datasets D1~D3. 1 ], [p1 2 ], [p1 3 ]. Client terminal C1 predicts probability [p1 1 ], [p1 2 ], [p1 3 For each of ], the learning weight w1 1 w1 2 w1 3 Calculate the learning weight w1. 1 w1 2 w1 3 The information summarizing this is the learning weight w91.

[0145] Furthermore, when j=2, client terminal C2 inputs explanatory variable 301 from the training dataset D2 into the integrated similarity calculation model M1 to calculate the predicted probability [p2]. By inputting explanatory variable 301 from data sample D21 into the integrated similarity calculation model M1, the predicted probability [p 21 The following is calculated, and by inputting the explanatory variable 301 of data sample D22 into the integrated similarity calculation model M1, the predicted probability [p 22 The following is calculated: The column CID=1~3 in the predicted probability [p2] shows how similar each data sample i of the training dataset D2 is to the training datasets D1~D3. 1 ], [p2 2 ], [p2 3 ]. Client terminal C2 predicts the probability [p2 1 ], [p2 2 ], [p2 3 For each of ], the learning weight w2 1 w2 2 w2 3 Calculate the learning weight w2. 1 w2 2 w2 3 The information summarizing this is the learning weight w92.

[0146] Furthermore, when j=3, client terminal C3 inputs explanatory variable 301 from training dataset D33 into integrated similarity calculation model M1 to calculate the predicted probability [p3]. By inputting explanatory variable 301 from data sample D31 into integrated similarity calculation model M1, the predicted probability [p 31 The following is calculated, and by inputting the explanatory variable 301 of data sample D32 into the integrated similarity calculation model M1, the predicted probability [p 32 The following is calculated: The column CID=1~3 in the predicted probability [p3] shows how similar each data sample i of the training dataset D3 is to the training datasets D1~D3. 1 ], [p3 2 ], [p3 3 ]. Client terminal C3 predicts the probability [p3 1 ], [p3 2 ], [p3 3 For each of ], the learning weight w3 1 w3 2 w3 3 Calculate the learning weight w3. 1 w3 2 w3 3 The information summarizing this is the learning weight w93.

[0147] Below is the federated learning FL of the individual analysis model. 3 In this case, the learning weight w j k This defines the formula used in the calculation of [the value].

[0148]

number

[0149] w on the left side of equation (18) above ji k [p] is the training weight applied to the similarity relationship between data sample i of the training dataset Dj and the training dataset Dk. ji k] is the predicted probability indicating how similar data sample i of the training dataset Dj is to the training dataset Dk. The training weight w of all data sample i of the training dataset Dj ji k The set of these is the learning weights w j k That is the case.

[0150] Model learning using weights 9 1kt> Next, we will explain a specific example of the weighted model training 91kt (911t~913t) shown in Figure 9.

[0151] Figure 11 is an explanatory diagram showing a specific example of model training 911t using the weighting method shown in Figure 9. Client terminal C1 has explanatory variables 203 and target variable 204 of the training dataset D1, and training weights w1 1 Using this, the individual analysis model M31-1t is updated, and the individual analysis model learning result 921-1t is output.

[0152] Client terminal C2 uses the explanatory variables 203 and the target variable 204 of the training dataset D2, and the training weights w2 1 Using this, the individual analysis model M32-1t is updated, and the individual analysis model learning result 922-1t is output.

[0153] Client terminal C3 uses the explanatory variables 203 and the target variable 204 of the training dataset D3, and the training weights w3 1 Using this, the individual analysis model M33-1t is updated, and the individual analysis model learning result 923-1t is output.

[0154] Server S integrates the individual analysis model learning results 921-1t, 922-1t, and 923-1t in integrated learning 930t to generate the next individual analysis model M31(t+1). Server S then generates the individual analysis model M31(t+1) or its integrated model parameter θ1 t+1This information is sent to client terminals C1-C3. Using this received information, client terminal C1 then performs the next federated learning FL. 3 Individual analysis model M for learning targets 3 Set up 1-1t, and client terminal C2 will perform the next federated learning FL. 3 The individual analysis model M32-1t is set up for learning targets, and client terminal C3 performs the following federated learning FL. 3 The individual analysis model M33-1t is set up for learning targets. Individual analysis model M 3 1 is the individual analysis model of client terminal C1 determined at the end of associative learning FL3.

[0155] Figure 12 is an explanatory diagram showing a specific example of the weighted model training 912t shown in Figure 9. Client terminal C1 has explanatory variables 203 and target variable 204 of the training dataset D1, and training weights w1 2 Using this, the individual analysis model M31-2t is updated, and the individual analysis model learning result 921-2t is output.

[0156] Client terminal C2 uses the explanatory variables 203 and the target variable 204 of the training dataset D2, and the training weights w2 2 Using this, the individual analysis model M32-2t is updated, and the individual analysis model learning result 922-2t is output.

[0157] Client terminal C3 uses the explanatory variables 203 and the target variable 204 of the training dataset D3, and the training weights w3 2 Using this, the individual analysis model M33-2t is updated, and the individual analysis model learning result 923-2t is output.

[0158] Server S integrates the individual analysis model learning results 921-2t, 922-2t, and 923-2t in integrated learning 930t to generate the next individual analysis model M32(t+1). Server S then generates the individual analysis model M32(t+1) or its integrated model parameters θ2 t+1This information is sent to client terminals C1-C3. Using this received information, client terminal C1 then performs the next federated learning FL. 3 The individual analysis model M31-2t is set up for learning targets, and client terminal C2 performs the following federated learning FL. 3 The individual analysis model M32-2t is set up for learning targets, and client terminal C3 performs the following federated learning FL. 3 The individual analysis model M33-2t is set for the learning target. Individual analysis model M32 is the individual analysis model of client terminal C2 that was determined at the end of associative learning FL3.

[0159] Figure 13 is an explanatory diagram showing a specific example of model training 913t using the weighting method shown in Figure 9. Client terminal C1 has explanatory variables 203 and target variable 204 of the training dataset D1, and training weights w1 3 Using this, the individual analysis model M31-3t is updated, and the individual analysis model learning result 921-3t is output.

[0160] Client terminal C2 uses the explanatory variables 203 and the target variable 204 of the training dataset D2, and the training weights w2 3 Using this, the individual analysis model M32-3t is updated, and the individual analysis model learning result 922-3t is output.

[0161] Client terminal C3 uses the explanatory variables 203 and the target variable 204 of the training dataset D3, and the training weights w3 3 Using this, the individual analysis model M33-3t is updated, and the individual analysis model learning result 923-3t is output.

[0162] Server S integrates the individual analysis model learning results 921-3t, 922-3t, and 923-3t in integrated learning 930t to generate the next individual analysis model M33(t+1). Server S then generates the individual analysis model M33(t+1) or its integrated model parameter θ3 t+1 This information is sent to client terminals C1-C3. Using this received information, client terminal C1 then performs the next federated learning FL.3 The individual analysis model M31-3t is set up for learning targets, and client terminal C2 performs the following federated learning FL. 3 The individual analysis model M32-3t is set up for learning targets, and client terminal C3 performs the following federated learning FL. 3 The individual analysis model M33-3t is set up for the learning target. Individual analysis model M33 is the individual analysis model of client terminal C3 that was determined at the end of associative learning FL3.

[0163] The following formulas are defined for use in the associative learning FL3 of the individual analysis model.

[0164] [y j ]=f(x j ;θ j )···(19)

[0165] Equation (19) above defines the individual analysis model M2j. x j These are the 203 explanatory variables of the training dataset Dj, and θ j [y] is the integrated model parameter of the individual analysis model M2j. j ] represents the predicted values ​​of the target variable 204 in the training dataset Dj.

[0166]

number

[0167] Equation (20) above represents the t-th associative learning FL 3 Loss function F(θ) in the individual analysis model M2j-kt for learning targets j kt ) defines y ji The target variable is 204 in a data sample i of the training dataset Dj, and [y ji ] is its predicted value. [y ji ] is [y j ] is. w ji k This is the learning weight calculated using the above formula (18).

[0168]

number

[0169] Equation (21) above represents the model parameter θ in the tth associative learning FL3. j kt The gradient ga j kt This defines the gradient ga j kt This is calculated on the client terminal Cj.

[0170]

number

[0171] Equation (22) above represents each model parameter θ in the tth associative learning FL3. j kt This is the calculation formula that performs the update, and is executed in the model training 91jt with weighting of client terminal Cj. j t As shown in equation (23) above, in the t-th associative learning FL3, the P of client terminal Ck j kt It is the sum. P j kt As shown in equation (24) above, in the t-th associative learning FL3, the predicted probability at client terminal Cj [p ji k ] t It is the sum of [the numbers].

[0172] The client terminal Cj has updated model parameters θ. j 1(t+1) ~θ j 3(t+1) Individual analysis model learning results 92j-1t, 92j-2t, 92 j -3t is sent to server S. Server S, in integrated learning 930t, receives the individual analysis model learning result 921-1t (updated model parameters θ1). 1(t+1) ), Individual analysis model learning results 922-1t (updated model parameters θ21(t+1) ), Individual analysis model learning results 923-1t (updated model parameters θ3 1(t+1) ) can be averaged and integrated, for example, to update the integrated model parameter θ1 for client terminal C1. (t+1) This is calculated, and the individual analysis model M31(t+1) is generated.

[0173] Furthermore, in integrated learning 930t, server S obtained the individual analysis model learning results 921-2t (updated model parameters θ1) 2(t+1) ), Individual analysis model learning results 922-2t (updated model parameters θ2 2(t+1) ), Individual analysis model learning results 923-2t (updated model parameters θ3 2(t+1) ) can be averaged and integrated, for example, to update the integrated model parameter θ2 for client terminal C2. (t+1) This is calculated, and the individual analysis model M32(t+1) is generated.

[0174] Furthermore, in integrated learning 930t, server S obtained the individual analysis model learning results 921-3t (updated model parameters θ1) 3(t+1) ), Individual analysis model learning results 922-3t (updated model parameters θ2 3(t+1) ), Individual analysis model learning results 923-3t (updated model parameters θ3 3(t+1) ) for example, by averaging and integrating, client terminal C 3 Updated integrated model parameter θ3 (t+1) This is calculated, and the individual analysis model M33(t+1) is generated.

[0175]

number

[0176] Equation (25) above calculates the gradient ga from the client terminal Cj as the individual analysis model learning result 92j-kt. j kt and predicted probability [p ji k ] tWhen a signal is received, each model parameter θ in the tth associative learning FL3 j kt This is the calculation formula that performs the update, and it is executed on server S's integrated learning 430t.

[0177] Equation (26) above calculates the gradient ga from the client terminal Cj as the individual analysis model learning result 92j-kt. j kt and predicted probability [p ji k ] t When this is received, the integrated model parameter θ is used in the tth associative learning FL3. j t This is the calculation formula that performs the update of the integrated model parameter θ1 on server S, and is executed in integrated learning 430t on server S. Specifically, for example, server S will update the integrated model parameter θ1 t+1 The individual analysis model M31(t+1) is generated using this. Server S is the updated integrated model parameter θ2 t+1 The individual analysis model M32(t+1) is generated using this. Server S then updates the integrated model parameters θ3 t+1 The individual analysis model M33(t+1) is generated using this method.

[0178] <Integration of analytical models for input variables 800> Figure 14 is an explanatory diagram showing an example of the input variable analysis model integration 800. Server S, as in Example 1, inputs the input variable Q into the integrated similarity calculation model M1, performs the input variable similarity calculation 404, and outputs the input variable similarity QS.

[0179] Server S uses the input variable similarity QS to perform input variable analysis model integration 800, which integrates individual analysis models M31 to M33 according to equation (27) below, and generates input variable analysis model MQ.

[0180]

number

[0181] The left-hand side [y] represents the column vector consisting of the inferred values ​​for each input data Q1, Q2, ... which are row vectors of the input variable Q, i.e., the inference result A.

[0182] p on the left side r j is, input variable Similarity vector p calculated for Qr r This represents the j-th element (the degree of similarity with the j-th client terminal Cj). θ j These are the model parameters of the individual analysis model M3j.

[0183] <Associative Learning Sequence> Figure 15 is a sequence diagram of federated learning in the federated learning system 100 according to Example 2. First, the transmission of client ID 201 from the server S to each client terminal Cj (step S701) and the execution of federated learning FL1 of the similarity calculation model (step S702) are the same as in Example 1.

[0184] Next, the associative learning system 100 performs associative learning FL3 of the individual analysis model (step S1503).

[0185] In the federated learning FL3 of the individual analysis model (step S1503), the client terminal Cj uses the integrated similarity calculation model M1 to perform learning weight calculation 90j (901~903) and calculates the learning weight w9j (w91~w93) (step S1531).

[0186] Furthermore, server S sends individual analysis model information (individual analysis model M3j(t+1) or its integrated model parameter θ) to client terminal Cj. j t+1 Send (step S1532).

[0187] The client terminal Cj performs weighted model training 91jt (step S1533) and sends the individual analysis model training results 92j-kt to the server S (step S1534).

[0188] Server S uses the individual analysis model training results 92j-kt to perform integrated training 930t on the individual analysis models (step S1535).

[0189] Server S determines whether the termination conditions for the integrated learning 930t of the individual analysis model (step S1535) are met (step S1536).

[0190] If the termination conditions are not met, server S registers the updated individual analysis model information (step S1537) and sends it to client terminal Cj. If the termination conditions are met, server S sends the updated individual analysis model information to client terminal Cj. As a result, client terminal Cj generates the individual analysis model M3j using the updated individual analysis model information and terminates the federated learning FL3 of the individual analysis model (step S1503).

[0191] The user terminal 700, which can communicate with the server S, accepts input of the input variable Qr (step S703) and sends the input variable Qr to the server S (step S704).

[0192] Next, Server S performs input variable inference (step S1505). In input variable inference (step S1505), Server S obtains the input variable Q (step S731). Then, Server S uses the integrated similarity calculation model M1 to perform input variable similarity calculation 404 and outputs the input variable similarity QS (step S732).

[0193] Server S uses the input variable similarity QS to perform an input variable analysis model integration 800, which integrates the individual analysis models M31 to M33 registered in step S1537, and generates an input variable analysis model MQ (step S1553).

[0194] Next, server S performs inference IN by inputting the input variable Q into the analysis model MQ for the input variable (step S706). Then, server sends the inference result A to user terminal 700 (step S707).

[0195] Thus, according to Example 2, an analysis model MQ for input variables that takes into account the value of the input variable Q can be generated as an integrated model by combining the individual analysis models M31 to M33 for client terminals C1 to C3.

[0196] Furthermore, according to Example 2, it is possible to provide an individual analysis model M2k suitable for each client terminal Ck participating in federative learning. In addition, when generating the integrated similarity calculation model M1, the client terminal Ck can predict its similarity with other training datasets Dk without passing its data sample i to other client terminals Ck or the server S, thus preventing the leakage of the data sample i itself. [Examples]

[0197] Next, we will describe Example 3. In Example 2, federated learning FL3 of the individual analysis model was performed, but in Example 3, the federated learning FL3 of the individual analysis model is not performed in the federated learning system 100, and the client terminal Cj holds the individual analysis model M3j. The individual analysis model M3j may be a prediction model provided externally, or it may be a prediction model created by the user of the client terminal Cj.

[0198] In Example 3, as in Example 2, Server S performs input variable similarity calculation 404 and input variable analysis model integration 800 to generate the input variable analysis model MQ. .fruit Similar to Example 2, inference IN is performed and inference result A is obtained.

[0199] Thus, according to Example 3, the computational load can be reduced.

[0200] <Display screen example> Figure 16 is an explanatory diagram showing examples of management screens for Examples 1 to 3. Management screen 1600 is displayed on server S. Management screen 1600 displays the input variable Q, the inference result A, and the integrated model information for the input variable 1601. The inference result A includes the predicted value 1602 for each data ID 202 included in the input variable Q. The integrated model information for the input variable 1601 includes the learning contribution 1603 for each data ID 202 included in the input variable Q.

[0201] The integrated model information 1601 for input variables includes the learning contribution 1603 of client terminal Cj for each data ID 202 included in the input variable Q. The learning contribution 1603 is calculated by the following formula. The learning contribution 1603 is an index value that indicates how much the learning dataset Dk of client terminal Ck contributes to the learning of the individual analysis model.

[0202]

number

[0203] R on the left side of equation (28) above j k This represents the learning contribution of client terminal Cj, which is 1603. The denominator on the right side is W as shown in equation (15) above. r And the numerator on the right side is W as shown in equation (14) above. j r The learning contribution of client terminal Cj, 1603, is calculated by server S.

[0204] Thus, the learning contribution of 1603 for each client terminal Cj allows us to see how much each data sample i of client terminal Cj contributed to the inference result A. For example, for input data Q1 (data ID 202 is "D01") within the input variable Q, the learning contribution of client terminal C1 is 57%, client terminal C2 is 33%, and client terminal C3 is 10%. Therefore, it can be seen that the client terminal Cj that contributed the most to the predicted value of 1602, which is "0.9", for input data Q1 (data ID 202 is "D01") is client terminal C1, which has the highest learning contribution of 1603. Thus, it can be seen that input data Q1 is more similar to the data sample i of client terminal C1's training dataset D1 than to the data sample i of client terminal C2's training dataset D2 and client terminal C3's training dataset D3.

[0205] <Example hardware configuration for server S and client terminal Ck> Figure 17 is a block diagram showing an example of the hardware configuration of a server S and a client terminal Ck (hereinafter referred to as computer 1700) according to Examples 1 to 3. Computer 1700 includes a processor 1701, a storage device 1702, an input device 1703, an output device 1704, and a communication interface (communication IF) 1705. The processor 1701, storage device 1702, input device 1703, output device 1704, and communication IF 1705 are connected by a bus 1706. The processor 1701 controls computer 1700. The storage device 1702 serves as the work area for the processor 1701. The storage device 1702 is also a non-temporary or temporary recording medium that stores various programs and data. Examples of storage devices 1702 include ROM (Read Only Memory), RAM (Random Access Memory), HDD (Hard Disk Drive), and flash memory. The input device 1703 receives data. Input devices 1703 include, for example, a keyboard, mouse, touch panel, numeric keypad, scanner, microphone, and sensor. Output devices 1704 output data. Output devices 1704 include, for example, a display, printer, and speaker. Communication IF 1705 connects to the network and sends and receives data.

[0206] Furthermore, the associative learning system 100 described above can also be configured as shown in [1] to

[10] below.

[0207] [1] The federated learning system 100 comprises a plurality of client terminals C1 to C3, each having a learning dataset D1 to D3, and a server S capable of communicating with the plurality of client terminals C1 to C3. The system performs federated learning in which each of the plurality of client terminals C1 to C3 learns a model using the learning dataset D1 to D3, and the server S uses the learning results to integrate the models of each of the plurality of client terminals C1 to C3, repeating this process.

[0208] The training dataset Dk has one or more data samples i, which include a client ID 201 that identifies the client terminal Ck, a first explanatory variable 203, and a first objective variable 204.

[0209] The federated learning system 100 performs a first calculation process (input variable similarity calculation) by the server S and a second calculation process (learning weight calculation 401-403) by each of the multiple client terminals C1-C3, and also performs a first federated learning process (FL2) in which the first learning process (weighted model learning 411t-413t) by each of the multiple client terminals C1-C3 and the first integration process (integrated learning 330t) by the server S are repeated until the first termination condition is met.

[0210] In the first calculation process (input variable similarity calculation), the server S obtains an input dataset (input variable Q) which includes one or more input data (Q1, Q2) that include a client ID 201 that identifies the client terminal Ck, a first explanatory variable 203, and a first objective variable 204.

[0211] In the first calculation process (input variable similarity calculation), the server S inputs the input dataset (input variable Q) into a similarity calculation model M1 that calculates the similarity between the data sample i and the multiple training datasets D1 to D3, thereby calculating a first similarity (prediction probability [pq]) between the input dataset and the multiple training datasets D1 to D3.

[0212] In the second calculation process (learning weight calculation 401-403), each of the multiple client terminals C1-C3 inputs the data sample i into the similarity calculation model M1, thereby calculating the second similarity (predicted probability [p]) between the data sample i and the multiple learning datasets D1-D3. j Calculate ]).

[0213] In the second calculation process (learning weight calculation 401-403), each of the multiple client terminals C1-C3 calculates the first similarity (prediction probability [pq]) and the second similarity (prediction probability [p j The learning weight wj is output by performing a similarity calculation with ]).

[0214] In the first learning process (weighted model learning 411t~413t), each of the multiple client terminals C1~C3 learns the first analysis model (analysis model for input variables to be learned M21-1t, M22-1t, M23-1t) based on the input dataset, the first explanation variable 203 and the first target variable 204 of the learning dataset, and the learning weights, and transmits the first learning results (421-t, 422-t, 423-t) to the server S.

[0215] In the first integration process (integrated learning 330t), the server S integrates the first learning results (421-t, 422-t, 423-t) from the first learning process (weighted model learning 411t-413t) from the multiple client terminals C1-C3 to generate a second analysis model (input variable analysis model MQ) that calculates predicted values ​​of the first target variable 204 from the first explanatory variable 203 for the input dataset.

[0216] [2] In the federated learning system 100 described in [1] above, each of the multiple client terminals C1 to C3 uses the second analysis model generated by the first integration process (integrated learning 330t) as the first analysis model and executes the first learning process (weighted model learning 411t to 413t).

[0217] [3] In the federated learning system 100 described in [1] above, the server S performs inference IN by inputting the input dataset (input variable Q) into the second analysis model (analysis model MQ for input variables) when the first termination condition is met.

[0218] [4] The federated learning system 100 comprises a plurality of client terminals C1 to C3, each having a learning dataset D1 to D3, and a server S capable of communicating with the plurality of client terminals C1 to C3. The system performs federated learning in which each of the plurality of client terminals C1 to C3 learns a model using the learning dataset D1 to D3, and the server S uses the learning results to integrate the models of each of the plurality of client terminals C1 to C3, repeating this process.

[0219] The training dataset Dk has one or more data samples i, which include a client ID 201 that identifies the client terminal Ck, a first explanatory variable 203, and a first objective variable 204.

[0220] The federated learning system 100 performs a first calculation process (input variable similarity calculation) by the server S and a second calculation process (learning weight calculation 901-903) by each of the multiple client terminals C1-C3, and also performs a first federated learning process (FL2) in which the first learning process (model learning by weighting 911t-913t) by each of the multiple client terminals C1-C3 and the first integration process (integrated learning 930t) by the server S are repeated until the first termination condition is met.

[0221] In the first calculation process (input variable similarity calculation), the server S obtains an input dataset (input variable Q) which includes one or more input data (Q1, Q2) that include a client ID 201 that identifies the client terminal Ck, a first explanatory variable 203, and a first objective variable 204.

[0222] In the first calculation process (input variable similarity calculation), the server S inputs the input dataset (input variable Q) into a similarity calculation model M1 that calculates the similarity between the data sample i and the multiple training datasets D1 to D3, thereby calculating a first similarity (prediction probability [pq]) between the input dataset and the multiple training datasets D1 to D3.

[0223] In the second calculation process (learning weight calculation 901-903), each of the multiple client terminals C1-C3 inputs the data sample i into a similarity calculation model M1 that calculates the similarity between the data sample i and the multiple learning datasets D1-D3, thereby calculating the second similarity (predicted probability [p ji k ] t Calculate ).

[0224] In the first learning process (weighted model learning 911t-913t), each of the multiple client terminals C1-C3 calculates a predicted value of the first target variable 204 from the first explanatory variable 203 using an individual analysis model (learning target individual analysis model M) 3 1-1t, M 3 2-1t, M 3 3-1t) and the first explanatory variable 203, the first objective variable 204, and a specific second similarity (predicted probability [p ji 1 ] t Based on ) and , the individual analysis model (learning target individual analysis model M 3 1-1t, M 3 2-1t, M 3 Learn 3-1t).

[0225] In the first integration process (integrated learning 930t), the server S uses the first similarity (prediction probability [pq]) to integrate multiple first learning results (921-1t, 922-1t, 923-1t) from the first learning processes (weighted model learning 911t-913t) from the multiple client terminals C1-C3 to generate an analysis model (input variable analysis model MQ) that calculates the predicted value of the first target variable 204 from the first explanatory variable 203 for the input dataset.

[0226] [5] In the federated learning system 100 described in [4] above, each of the multiple client terminals C1 to C3 uses the analysis model generated by the first integration process (integrated learning 930t) as the individual analysis model and executes the first learning process (weighted model learning 911t to 913t).

[0227] [6] In the federated learning system 100 described in [4] above, the server S performs inference IN by inputting the input dataset (input variable Q) into the analysis model (analysis model MQ for input variables) when the first termination condition is met.

[0228] [7] In the federated learning system 100 described in [1] above, in the second calculation process (learning weight calculation 401-403), each of the multiple client terminals C1-C3 inputs the combination of the first explanatory variable 203 and the first target variable 204 of the data sample i as the second explanatory variable 301 to the similarity calculation model M1, thereby calculating the second similarity (predicted probability [p j Calculate ]).

[0229] [8] In the associative learning system 100 described in [4] above, the second calculation process (learning weight calculation) 9 01~ 9 03) In this case, each of the multiple client terminals C1 to C3 determines the specific second similarity (prediction probability [p ji 1 ] t Learning weights (w1) according to ) 1 w2 1w3 1 Calculate ).

[0230] The first learning process (model learning using weighting) 9 11t~ 9 In 13t), each of the multiple client terminals C1 to C3 is the individual analysis model (learning target individual analysis model M 3 1-1t, M 3 2-1t, M 3 3-1t) and the first explanatory variable 203, the first objective variable 204, and a specific second similarity (predicted probability [p) calculated in each of the multiple client terminals C1 to C3 by the second calculation process (learning weight calculation 401 to 403) ji 1 ] t Learning weights (w1) according to ) 1 w2 1 w3 1 Based on ) and , the individual analysis model (learning target individual analysis model M 3 1-1t, M 3 2-1t, M 3 Learn 3-1t).

[0231] [9] Prior to the first federated learning process (FL2), the federated learning system 100 described in [1] executes a second federated learning process (FL1) in which each of the plurality of client terminals C1 to C3 performs a second learning process (model learning 311t to 313t) and the server S performs a second integration process (integrated learning 330t), and this process is repeated until the second termination condition is met.

[0232] In the second learning process (model learning 311t to 313t), each of the multiple client terminals C1 to C3 learns the learning target similarity calculation model M11t to M13t, using the combination of the first explanatory variable 203 and the first target variable 204 as the second explanatory variable 301, and the client ID 201 as the second target variable 302.

[0233] In the second integration process (integrated learning 330t), the server S integrates the second learning results 321t to 323t of the learning target similarity calculation models M11t to M13t from the multiple client terminals C1 to C3 obtained in the second learning process (model learning 311t to 313t), and generates an integrated similarity calculation model M1(t+1) as the similarity calculation model M1, which is an integrated similarity calculation model M1 obtained by integrating the learning target similarity calculation models (M11t to M13t) from the multiple client terminals C1 to C3.

[0234]

[10] In the federated learning system 100 described in [9] above, in the second learning process (model learning 311t to 313t), each of the multiple client terminals C1 to C3 sets the updated learning target similarity calculation models M11t to M13t to the similarity calculation model M1 if the second termination condition is met.

[0235] It should be noted that the present invention is not limited to the embodiments described above, but includes various modifications and equivalent configurations within the spirit of the attached claims. For example, the embodiments described above are described in detail to make the present invention easier to understand, and the present invention is not necessarily limited to having all of the described configurations. Furthermore, some of the configurations of one embodiment may be replaced with those of another embodiment. Furthermore, some of the configurations of one embodiment may be added to those of another embodiment. Furthermore, some of the configurations of each embodiment may be added, deleted, or replaced with other configurations.

[0236] Furthermore, each of the aforementioned configurations, functions, processing units, and processing means may be implemented in hardware, for example, by designing them as integrated circuits, or they may be implemented in software by having a processor interpret and execute programs that realize each function.

[0237] Information such as programs, tables, and files that implement each function can be stored in memory, hard disks, SSDs (Solid State Drives), or on recording media such as IC (Integrated Circuit) cards, SD cards, and DVDs (Digital Versatile Discs).

[0238] Furthermore, the control lines and information lines shown are those deemed necessary for explanation purposes and do not necessarily represent all control lines and information lines required for implementation. In reality, it can be assumed that almost all components are interconnected. [Explanation of Symbols]

[0239] 100 Associative Learning Systems 203 explanatory variables 204 Dependent variable 301 explanatory variables 302 Target Variable 330t Integrated Learning 430t Integrated Learning C1-C3 Client Terminals D1-D3 Training Datasets M1 Integrated Similarity Calculation Model Qr input variable QMr Analysis Model for Input Variables

Claims

1. A federated learning system comprising: a plurality of client terminals, each having a training dataset; and a server capable of communicating with the plurality of client terminals, wherein each of the plurality of client terminals trains a model using the training dataset, and the server repeatedly integrates the models of each of the plurality of client terminals using the training results, wherein federated learning is performed, The aforementioned training dataset has one or more data samples including a client ID that identifies the client terminal, a first explanatory variable, and a first objective variable. The server performs a first calculation process and each of the multiple client terminals performs a second calculation process, and the first federated learning process is executed, which repeats the first learning process by each of the multiple client terminals and the first integration process by the server until the first termination condition is met. In the first calculation process, the server, Obtain an input variable that includes one or more input data points, including the first explanatory variable and the first dependent variable. By inputting the input variables into a similarity calculation model that calculates the similarity between the data sample and the multiple training datasets, a first similarity between the input variables and the multiple training datasets is calculated for each client terminal. In the second calculation process described above, each of the multiple client terminals, By inputting the data sample into the similarity calculation model, the second similarity between the data sample and the multiple training datasets is calculated. By performing a similarity calculation between the first similarity and the second similarity, the learning weights are output. In the first learning process, each of the multiple client terminals, A first analysis model calculates a predicted value of the first target variable from the first explanatory variable for the input variable, and the first analysis model is trained based on the first explanatory variable and the first target variable of the training dataset and the training weights, and the first training result is transmitted to the server. In Series 1 integrated processing, the server, By integrating the first learning results from the first learning process from the multiple client terminals, a second analysis model is generated that calculates a predicted value of the first target variable from the first explanatory variable for the input variable. An associative learning system characterized by the following:

2. The associative learning system according to claim 1, Each of the aforementioned client terminals executes the first learning process using the second analysis model generated by the first integration process as the first analysis model. An associative learning system characterized by the following:

3. The associative learning system according to claim 1, The server, when the first termination condition is met, performs inference by inputting the input variables into the second analysis model. An associative learning system characterized by the following:

4. A federated learning system comprising: a plurality of client terminals, each having a training dataset; and a server capable of communicating with the plurality of client terminals, wherein each of the plurality of client terminals trains a model using the training dataset, and the server repeatedly integrates the models of each of the plurality of client terminals using the training results, wherein federated learning is performed, The aforementioned training dataset has one or more data samples including a client ID that identifies the client terminal, a first explanatory variable, and a first objective variable. The server performs a first calculation process and each of the multiple client terminals performs a second calculation process, and the first combined learning process is executed, which is repeated by each of the multiple client terminals performing a first learning process and the server performing a first integration process until the first termination condition is met. In the first calculation process, the server, Obtain an input variable that includes one or more input data points, including the first explanatory variable and the first dependent variable. By inputting the input variables into a similarity calculation model that calculates the similarity between the data sample and the multiple training datasets, a first similarity between the input variables and the multiple training datasets is calculated for each client terminal. In the second calculation process described above, each of the multiple client terminals, By inputting the data sample into a similarity calculation model that calculates the similarity between the data sample and the multiple training datasets, a second similarity between the data sample and the multiple training datasets is calculated. In the first learning process, each of the multiple client terminals, An individual analysis model that calculates a predicted value of the first dependent variable from the first independent variable, and a specific second similarity between the first independent variable, the first dependent variable, and a specific training dataset calculated on each of the multiple client terminals by the second calculation process, is used to train the individual analysis model. In Series 1 integrated processing, the server, Using the first similarity, an analytical model is generated that calculates the predicted value of the first target variable from the first explanatory variable for the input variable by integrating the multiple first learning results from the multiple client terminals through the first learning process. An associative learning system characterized by the following:

5. The associative learning system according to claim 4, Each of the plurality of client terminals executes the first learning process using the analysis model generated by the first integration process as the individual analysis model. An associative learning system characterized by the following:

6. The associative learning system according to claim 4, The server, when the first termination condition is met, performs inference by inputting the input variables into the analysis model. An associative learning system characterized by the following:

7. The associative learning system according to claim 1, In the second calculation process described above, each of the multiple client terminals, The second similarity is calculated by inputting the combination of the first explanatory variable and the first dependent variable of the data sample as the second explanatory variable into the similarity calculation model. An associative learning system characterized by the following:

8. The associative learning system according to claim 4, In the second calculation process described above, each of the multiple client terminals, The learning weights corresponding to the aforementioned specific second similarity are calculated, In the first learning process, each of the multiple client terminals, The individual analysis model is trained based on the individual analysis model, the first explanatory variable, the first objective variable, and the learning weights corresponding to a specific second similarity calculated on each of the multiple client terminals by the second calculation process. An associative learning system characterized by the following:

9. The associative learning system according to claim 1, Prior to the first federated learning process, a second federated learning process is executed, in which the second learning process by each of the plurality of client terminals and the second integration process by the server are repeated until the second termination condition is met. In the second learning process described above, each of the multiple client terminals, The combination of the first explanatory variable and the first objective variable is used as the second explanatory variable, and the client ID is used as the second objective variable to train the model for calculating similarity to the target object. In the second integrated process described above, the server, The second learning results of the learning target similarity calculation model from the multiple client terminals obtained by the second learning process are integrated, and a unified similarity calculation model is generated as the similarity calculation model by integrating the learning target similarity calculation models of the multiple client terminals. An associative learning system characterized by the following:

10. The associative learning system according to claim 9, In the second learning process, if each of the multiple client terminals satisfies the second termination condition, it sets the updated learning target similarity calculation model to the similarity calculation model. An associative learning system characterized by the following:

11. A federated learning method comprising a federated learning system having a plurality of client terminals, each having a training dataset, and a server capable of communicating with the plurality of client terminals, wherein the federated learning system performs federated learning in which each of the plurality of client terminals trains a model using the training dataset, and the server integrates the models of each of the plurality of client terminals using the training results, and this process is repeated, The aforementioned training dataset has one or more data samples including a client ID that identifies the client terminal, a first explanatory variable, and a first objective variable. The aforementioned associative learning system is The server performs a first calculation process and each of the multiple client terminals performs a second calculation process, and the first federated learning process is executed, which repeats the first learning process by each of the multiple client terminals and the first integration process by the server until the first termination condition is met. In the first calculation process, the server, Obtain an input variable that includes one or more input data points, including the first explanatory variable and the first dependent variable. By inputting the input variables into a similarity calculation model that calculates the similarity between the data sample and the multiple training datasets, a first similarity between the input variables and the multiple training datasets is calculated. In the second calculation process described above, each of the multiple client terminals, By inputting the data sample into the similarity calculation model, the second similarity between the data sample and the multiple training datasets is calculated. By performing a similarity calculation between the first similarity and the second similarity, the learning weights are output. In the first learning process, each of the multiple client terminals, A first analysis model calculates a predicted value of the first target variable from the first explanatory variable for the input variable, and the first analysis model is trained based on the first explanatory variable and the first target variable of the training dataset and the training weights, and the first training result is transmitted to the server. In Series 1 integrated processing, the server, By integrating the first learning results from the first learning process from the multiple client terminals, a second analysis model is generated that calculates a predicted value of the first target variable from the first explanatory variable for the input variable. A characteristic associative learning method.

Images