A training method of a longitudinal federated neural network model

Abstract

This invention proposes a training method for a vertical federated neural network model. The method includes: S1, acquiring multiple datasets and their corresponding label sets in a target domain, each dataset including multiple feature vectors of the target domain data, distributing a dataset to each local network, and distributing the label set to the prediction network; S2, setting weight parameters for each neuron in each local network and initializing them; S3, performing multiple rounds of vertical federated iterative training on the prediction network and multiple local networks based on the acquired data until each local network has completed neuron reduction, so as to obtain a trained vertical federated neural network model.

Description

Technical Field

[0001] This invention relates to the field of medical and health care, specifically to vertical federated learning technology in the field of machine learning, and more specifically to a model training method for human health assessment in the field of medical and health care, namely, a training method for a vertical federated neural network model. Background Technology

[0002] With the development of machine learning, it has been widely applied in various fields, such as healthcare, finance, and e-commerce. In the healthcare field, machine learning models can be used for health assessments, such as evaluating blood sugar, blood pressure, eye diseases, and diabetes by acquiring human medical data. Federated learning is a commonly used model building method in the healthcare field.

[0003] Federated learning is a privacy-preserving machine learning method that allows multiple participants to collaboratively train a model without disclosing the original data. It is particularly suitable for fields with high data privacy requirements, such as healthcare assessment. Federated learning is divided into horizontal federated learning, vertical federated learning, and federated transfer learning. In horizontal federated learning, different participants may possess different features from different samples; in vertical federated learning, different participants may possess different features from the same samples; and in federated transfer learning, different participants possess both different samples and different features.

[0004] In vertical federated learning, typically only one participant possesses a label, known as the active party, while the others possess only features and are called passive parties. When building a model based on vertical federated learning, each passive party has a local network with multiple neurons, used to transform the features it possesses into a data representation vector; the active party has a prediction network used to transform the outputs of the local networks into the final prediction result; thus, a vertical federated neural network model consisting of multiple local networks and one prediction network is obtained. (See attached diagram) Figure 1 As shown, it illustrates the communication process of a longitudinal federated neural network model. During forward propagation, each passive party passes the output of its corresponding local network to the prediction network (active party); during backward propagation, the prediction network (active party) passes the gradient to each local network.

[0005] While federated learning allows multiple participants to collaboratively train a model without disclosing raw data, the training process of vertical federated neural network models involves significant communication overhead due to the need to exchange intermediate results such as the output of the local network and the gradient of the prediction network at each iteration. This hinders the training of vertical federated neural network models and their rapid promotion and application in the healthcare field. To reduce the communication overhead during training, some researchers have proposed using multi-step local iterations, where the prediction network iterates using the output passed from the local network and the gradient passed from the prediction network. Other researchers have proposed compression methods, using quantization or sparsification to compress the output of the local network and the gradient of the prediction network to reduce the amount of data transmitted. Although these methods can reduce the communication overhead during training, they neglect the structural redundancy of vertical federated neural network models, resulting in significant communication overhead during the training of vertical federated neural network models used in the healthcare field. Summary of the Invention

[0006] Therefore, the purpose of this invention is to overcome the shortcomings of the prior art and provide a training method for a longitudinal federated neural network model, a training method for a diabetes assessment model, a diabetes assessment method, and a diabetes assessment system.

[0007] The objective of this invention is achieved through the following technical solutions.

[0008] According to a first aspect of the present invention, a training method for a longitudinal federated neural network model is provided. The longitudinal federated neural network model includes a prediction network and multiple local networks, each local network including multiple neurons. The method includes: S1, acquiring multiple datasets of a target domain and a corresponding label set, each dataset including multiple feature vectors of target domain data, distributing a dataset to each local network, and distributing the label set to the prediction network; S2, setting weight parameters for each neuron of each local network and initializing them; S3, performing multiple rounds of longitudinal federated iterative training on the prediction network and the multiple local networks based on the acquired data until each local network has completed neuron reduction to obtain a trained longitudinal federated neural network model. During the iterative training process, the parameters of each local network and the prediction network are updated according to a preset loss function, the weight parameters of the neurons of each local network are updated according to a preset weight update function, and neurons of each local network that do not conform to the preset rules are removed.

[0009] In some embodiments of the present invention, in step S3, multiple rounds of longitudinal federated iterative training are performed as follows: the prediction network and multiple local networks are subjected to a first preset round of longitudinal federated iterative training based on the data they have obtained; in each round of the first preset round of iterative training, the neuron weight parameters of each local network are updated according to a preset weight update function; neurons in each local network that do not conform to the rules after the first preset round of longitudinal federated iterative training are removed according to preset rules; the prediction network after the first preset round of training and the multiple local networks after neuron removal are subjected to a second preset round of longitudinal federated iterative training based on the data they have obtained to obtain a trained longitudinal federated neural network model; wherein, in each round of iterative training in the first preset round and the second preset round, the parameters of each local network and the prediction network are updated by backpropagation according to a preset loss function.

[0010] In some embodiments of the present invention, step S3 includes: S31, performing multi-stage iterative neuron pruning training on the prediction network and multiple local networks based on the data obtained therefrom, wherein each stage of iterative neuron pruning training is performed in the following manner: the prediction network and multiple local networks after the previous stage of iterative training are subjected to a third preset round of vertical federated iterative training based on the data obtained therefrom, and in each round of iterative training in the current stage, the neuron weight parameters of each local network are updated according to a preset weight update function, and neurons that do not conform to the rules in each local network after the current stage of training are removed according to preset rules; S32, performing a fourth round of vertical federated iterative training on the vertical federated neural network model after the multi-stage iterative neuron pruning training is completed to obtain a trained vertical federated neural network model; wherein in each round of iterative training in the third preset round and the fourth preset round, the parameters of each local network and the prediction network are updated by backpropagation according to a preset loss function.

[0011] In some embodiments of the present invention, the preset loss function is:

[0012]

[0013] Where L(·) represents the loss function, l(·,·) represents the prediction loss for a single data point, R(·) represents the regularization term, λ represents the regularization weight, and θ1, ..., θ K Let ξ represent the parameters of the 1st to the Kth local networks, ξ represent the parameters of the prediction network, and n represent the amount of data.

[0014] Preferably, the preset weight update function is:

[0015]

[0016]

[0017] Where s represents the importance parameter, Indicates neuron weights, The loss function L represents the loss function applied to the neuron weights. gradient, Represents neuron weights The gradient with respect to the importance parameter s, μ represents the learning rate, and h(·) represents a monotonically nonnegative function.

[0018] In some embodiments of the present invention, the preset rule is: to sort the weight parameters of neurons in each local network in descending order, and to delete a preset number of neurons at the end of the sort.

[0019] According to a second aspect of the present invention, a method for training a diabetes assessment model is provided, the method comprising: T1, acquiring medical data of multiple target objects and corresponding diabetes results, and constructing target domain data using medical data as samples and diabetes results as labels; T2, training a longitudinal federated neural network model based on the target domain data obtained in step T1 using the method described in the first aspect of the present invention to obtain a diabetes assessment model.

[0020] According to a third aspect of the present invention, a method for assessing diabetes is provided, the method comprising: Y1, acquiring medical data of a target object; Y2, processing the medical data of the target object using a diabetes assessment model trained as described in the second aspect of the present invention to obtain a diabetes assessment result.

[0021] According to a fourth aspect of the present invention, a diabetes assessment system is provided, the system comprising: a data acquisition module for acquiring medical data of a target object; and a diabetes assessment model trained by the method described in the second aspect of the present invention for processing the medical data of the target object obtained by the data acquisition module to obtain a diabetes assessment result.

[0022] Compared with the prior art, the advantages of the present invention are: by reducing neurons, the amount of data exchanged between the local network and the prediction network during the training of the longitudinal federated neural network model is reduced, thereby reducing communication overhead and enabling better construction of models for medical and health assessment. Attached Figure Description

[0023] The embodiments of the present invention will be further described below with reference to the accompanying drawings, wherein:

[0024] Figure 1 This is a schematic diagram of the communication process in a vertical federated neural network model in the prior art;

[0025] Figure 2 This is a schematic diagram of the training method for the vertical federated neural network model according to an embodiment of the present invention;

[0026] Figure 3 A schematic diagram of the communication process after deleting each local network neuron in the vertical federated neural network model described in this embodiment of the invention. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this invention clearer, the invention is further described in detail below through specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0028] As mentioned in the background section, although the methods proposed in the prior art can reduce the communication overhead of the longitudinal federated neural network model during the training process, these methods ignore the structural redundancy of the longitudinal federated neural network model. As a result, the longitudinal federated neural network model used in the medical and health field still has a large communication overhead during the training process, which is not conducive to its rapid promotion and application in the medical and health field.

[0029] To address the issue of high communication overhead during the training of vertical federated neural network models, this invention proposes a novel training scheme. By removing unimportant neurons from local networks, the amount of data transmitted during training is reduced, thereby decreasing communication overhead and enabling better construction of models for healthcare assessment. In summary, the scheme involves first acquiring multiple datasets and their corresponding label sets from the target domain, distributing the datasets to local networks and the label sets to prediction networks. Then, weight parameters are set and initialized for each neuron in the local network. Finally, multiple rounds of vertical federated iterative training are performed on both the local and prediction networks based on the acquired data. During the iterative training, the neuron weight parameters of each local network are updated, and neurons are removed based on the updated neuron weight parameters of each local network.

[0030] According to one embodiment of the present invention, as shown in the appendix Figure 2As shown, this invention provides a training method for a vertical federated neural network model. The vertical federated neural network model includes a prediction network and multiple local networks, each of which includes multiple neurons. The method includes: setting weight parameters for each neuron of each local network and initializing them, and then performing the following steps: S1, obtaining multiple datasets of the target domain and their corresponding label sets, each dataset including multiple feature vectors of the target domain data, distributing a dataset to each local network, and distributing the label set to the prediction network; S2, setting weight parameters for each neuron of each local network and initializing them; S3, performing multiple rounds of vertical federated iterative training on the prediction network and the multiple local networks based on the obtained data until each local network has completed neuron reduction to obtain a trained vertical federated neural network model. During the iterative training process, the parameters of each local network and the prediction network are updated according to a preset loss function, the weight parameters of the neurons of each local network are updated according to a preset weight update function, and neurons of each local network that do not conform to the preset rules are removed.

[0031] To better understand this invention, the following detailed description, in conjunction with the accompanying drawings and embodiments, covers aspects such as initial weight parameter setting, data acquisition, iterative training, and verification experiments.

[0032] 1. Obtain data

[0033] Before iterative training begins, training data is required, which involves acquiring multiple datasets and their corresponding label sets for the target domain. Each dataset includes multiple feature vectors from the target domain data. One dataset is distributed to each local network, and the label set is distributed to the prediction network. It's important to note that the multiple datasets and label sets required for training come from different data providers. To maintain data privacy, these providers do not interact with each other; datasets from different providers are directly transmitted to different local networks, and the label set is directly transmitted to the prediction network. In the actual data acquisition process, there is no data aggregation or distribution; data from different providers is directly passed to the local network or the prediction network. However, to ensure that the obtained dataset contains different feature vectors with the same sample ID, and that the label set contains labels with the same sample ID, privacy set intersection processing is required for the data from different data providers. For example, the first data provider includes the family medical history of users A, B, C, D, E, and F (sample IDs), the second data provider includes the consultation records of users C, D, E, and F (sample IDs), and the third data provider includes the diabetes risk (label value) of users A, B, C, D, and E (sample IDs). By performing privacy set intersection processing on the first, second, and third data providers, we obtain the first dataset containing the family medical history of users C, D, and E, the second dataset containing the consultation records of users C, D, and E, and the third dataset containing the diabetes risk of users C, D, and E. It should also be noted that different types of longitudinal federated neural network models require different data for training different functions. For example, a longitudinal federated neural network model used for medical and health assessment needs to acquire medical data of several target objects in different institutions or local devices and their corresponding labels. The medical data includes the target object's health records, consultation records, test reports and imaging data, etc. The label values ​​are determined according to the training task of the actual application. For example, if the training task is a diabetes prediction task, the label values ​​can be low-risk diabetes, high-risk diabetes, ordinary diabetes, intermediate-risk diabetes, high-risk diabetes, etc. In addition, the label values ​​can be labeled by professionals, identified based on diagnostic data in medical data, or obtained by other means. This application does not limit this.

[0034] 2. Initial weight parameter settings

[0035] After obtaining the training data, initial weight parameters are set for the neurons of each local network in the vertical federated neural network model. According to one embodiment of the invention, the initial weight parameter of each local network neuron is set to 1. It should be noted that before iterative training begins, the weight parameter of each local network neuron is set to 1, indicating that all neurons are equally important at this time. After iterative training begins, the weight parameters of each local network neuron will be updated along with the parameters of the vertical federated neural network model.

[0036] 3. Iterative training

[0037] After acquiring target domain data and allocating it to local and prediction networks, and setting initial weight parameters for neurons in each local network, the prediction network and multiple local networks undergo multiple rounds of longitudinal federated iterative training based on the acquired data until each local network completes neuron pruning, resulting in a trained longitudinal federated neural network model. It should be noted that different neuron pruning training strategies exist during the multiple rounds of longitudinal federated iterative training; however, regardless of the strategy, the neuron pruning operation is the same. The iterative training process of this invention is explained below from two dimensions: neuron pruning training strategy and neuron pruning. The neuron pruning training strategy is illustrated using single-round iterative pruning training, two-stage iterative pruning training, and multi-stage iterative pruning training as examples. The principles of other neuron pruning training strategies are similar and will not be elaborated upon here.

[0038] 3.1 Neuron Reduction Training Strategy

[0039] Before explaining the strategies for single-round iterative pruning training, two-stage iterative pruning training, and multi-stage iterative pruning training, let's first introduce the process of a single round of vertical federated iterative training for a vertical federated neural network model (this process does not involve neuron pruning): First, each local network transforms its acquired data into a local output and passes it to the prediction network; then, the prediction network aggregates its received local outputs and makes predictions on the aggregated data to obtain the prediction results; finally, it calculates the loss according to a preset loss function and updates the parameters of the vertical federated neural network model based on backpropagation of the loss. To more intuitively understand the process of a single round of vertical federated iterative training, this process can be represented as:

[0040] z i，k =f k (x i，k ;θ k ), k = 1, ..., K (1)

[0041] z i =agg(z i，1 , zi，2 , ..., z i，K (2)

[0042]

[0043]

[0044] Equation (1) represents the process by which the local network transforms the acquired data into local output, f k (·) represents the transformation function, θ k The parameter x represents the local network k. i，k This represents the data obtained from local network k, z i，k It is data x i，k The local output of the local network k. Equation (2) represents the aggregation of the received local outputs by the prediction network, z i This represents the local output z of the prediction network for the 1st to Kth local networks. i，1 , ..., z i，K The result after aggregation, agg represents the aggregation function, which can be concatenation, summation, etc. Equation (3) indicates that the prediction network makes predictions based on the aggregation result, f p (·) represents the prediction function. Let ξ represent the prediction result of the longitudinal federated neural network model, and let ξ represent the parameters of the prediction network. Equation (4) represents the preset loss function, l(·,·) represents the prediction loss for a single data point, R(·) represents the regularization term, λ represents the weight of the regularization term, and θ1,...,θ K This represents the parameters of the 1st to the Kth local networks, where n represents the amount of data.

[0045] 3.11 Single-round iterative pruning training

[0046] Single-round iterative pruning training means that after completing one round of longitudinal federated iterative training, a pruning operation is performed on the neurons in the local network.

[0047] According to one embodiment of the present invention, the prediction network and multiple local networks are subjected to multiple rounds of longitudinal federated iterative training based on the data obtained therefrom. During each round of iterative training, the parameters of each local network and the prediction network are updated via backpropagation according to a preset loss function. The neuron weight parameters of each local network are updated according to a preset weight update function. Furthermore, neurons in each local network that do not conform to the rules after one round of longitudinal federated iterative training are removed according to preset rules. It should be noted that the number of training rounds is determined according to actual needs, and the present invention does not impose any special limitations.

[0048] 3.12 Two-stage iterative pruning training

[0049] Two-stage iterative pruning training means that after completing one stage of longitudinal federated iterative training, a pruning operation is performed on the neurons in the local network, and then another stage of longitudinal federated iterative training is performed.

[0050] According to one embodiment of the present invention, a two-stage iterative pruning training is performed as follows: The prediction network and multiple local networks undergo a first preset round of longitudinal federated iterative training based on the data they have obtained. During each round of the first preset round of training, the neuron weight parameters of each local network are updated according to a preset weight update function. Neurons in each local network that do not conform to the rules after the first preset round of longitudinal federated iterative training are removed according to preset rules. The prediction network after the first preset round of training and the multiple local networks after neuron pruning undergo a second preset round of longitudinal federated iterative training based on the data they have obtained to obtain a trained longitudinal federated neural network model. In each round of the first and second preset rounds of training, the parameters of each local network and the prediction network are updated via backpropagation according to a preset loss function. It should be noted that in this two-stage iterative pruning training method, neuron selection is performed after a specific number of longitudinal federated iterative training rounds to reduce the number of neurons to a specified level. Furthermore, the number of neurons in the local networks is reduced in an earlier training phase (the first preset round) to minimize communication overhead. It should be noted that the number of the first and second preset rounds is determined according to actual needs, and the present invention does not impose any special restrictions.

[0051] 3.13 Multi-stage iterative pruning training

[0052] Multi-stage iterative pruning training means that after each stage of longitudinal federated iterative training is completed, a pruning operation needs to be performed on the neurons in the local network until the neurons are pruned, and finally another stage of longitudinal federated iterative training is performed.

[0053] According to one embodiment of the present invention, the multi-stage iterative pruning training includes: F1, performing multi-stage iterative neuron pruning training on the prediction network and multiple local networks based on the data obtained therefrom, wherein each stage of iterative neuron pruning training is performed as follows: a third preset round of longitudinal federated iterative training is performed on the prediction network and multiple local networks after the previous stage of iterative training based on the data obtained therefrom, wherein in each round of iterative training in the current stage, the neuron weight parameters of each local network are updated according to a preset weight update function, and neurons in each local network after the current stage of training that do not conform to the rules are removed according to preset rules; F2, performing a fourth round of longitudinal federated iterative training on the longitudinal federated neural network model after the multi-stage iterative neuron pruning training is completed to obtain a trained longitudinal federated neural network model; wherein in each round of iterative training in the third and fourth preset rounds, the parameters of each local network and the prediction network are updated by backpropagation according to a preset loss function. It should be noted that the number of the third and fourth preset rounds is determined according to actual needs, and the present invention does not impose any special limitations.

[0054] Although different neuron reduction training strategies are used in single-round iterative reduction training, two-stage iterative reduction training, and multi-stage iterative reduction training, the same preset loss function, preset weight update function, and preset rules are used in the training process. It should be noted that the preset loss function is consistent with the content of the aforementioned equation (4), and will not be repeated here.

[0055] According to one embodiment of the present invention, the preset weight update function is:

[0056]

[0057]

[0058] Where s represents the importance parameter, Indicates neuron weights, The loss function L represents the weights of the neurons. gradient, Represents neuron weights The gradient of the importance parameter s is given by μ, which represents the learning rate, and h(·) which represents a monotonically nonnegative function. According to one embodiment of the present invention, the monotonically nonnegative function can be represented by h(x) = e^(-ε / ε). x .

[0059] According to one embodiment of the present invention, the preset rule is as follows: the weight parameters of neurons in each local network are sorted in descending order, and a preset number of neurons at the end of the sorted list are deleted. It should be noted that the preset number is determined according to actual needs, and the present invention does not impose any special limitations.

[0060] 3.2 Neuron Reduction

[0061] Neuron pruning is required in single-round iterative pruning training, two-stage iterative pruning training, and multi-stage iterative pruning training. The principle of neuron pruning is explained in detail below.

[0062] To reduce the amount of data exchanged between the local network and the prediction network during model training, this invention proposes deleting neurons to reduce data exchange. However, deleting neurons usually weakens the network's expressive power and interferes with its output, leading to a certain loss of accuracy. Therefore, to reduce the accuracy loss caused by neuron selection, neurons with low importance are selected for deletion. Furthermore, to prevent the deletion of too many neurons, only some unimportant output neurons can be deleted, or only a certain number of neurons can be deleted. The principle of neuron deletion is illustrated by taking the deletion of only some unimportant output neurons as an example; other methods are similar and will not be explained in detail.

[0063] To implement neuron selection, each local network's output neuron is assigned a weight; for example, let the weight of the j-th neuron in local network k be... At this point, the output of the neuron can be expressed as:

[0064]

[0065] Where ⊙ represents multiplying the vector elements one by one, z i，k，j f represents the neuron output. k (·) represents the transformation function, θ k The parameter representing the local network k. When When the value is 1, equation (5) is equivalent to equation (1) above. When At that time, the neuron outputs z i，k，j A value of 0 indicates that the neuron's output is invalid, meaning that z can be omitted during the transmission of the local output. i，k，j This reduces data transmission and communication overhead. Therefore, the process of deleting neurons is equivalent to reducing a portion of the weights. Setting the value to 0 invalidates the neuron's output. It's important to note that the purpose of setting neuron weight parameters is to assist in neuron selection by filtering out unimportant neurons.

[0066] Deleting neurons typically weakens the network's expressive power and interferes with its output, leading to a loss of accuracy. Therefore, to reduce the accuracy loss caused by neuron selection, it is necessary to identify neurons with low importance. Since the loss function changes with the weight of a particular neuron, the loss from deleting a neuron, when other variables are ignored, can be expressed as:

[0067]

[0068] Where L(0) represents the loss value of setting the weight of the j-th neuron of the local network k to 0; Let ΔL represent the loss value if the weight of the j-th neuron in local network k is not reset to 0; ΔL represents the loss for deleting the j-th neuron in local network k. This value represents the penalty incurred after deleting the neuron and also reflects the importance of that neuron. However, directly calculating this value for each neuron would be cumbersome, so a first-order Taylor expansion is used to estimate this value, yielding...

[0069]

[0070] Where, ΔL k，j This reflects the importance of the j-th neuron in the local network k. This represents the weight of the j-th neuron in the local network k. The loss of all data is expressed with respect to the weight of the j-th neuron in the local network k. The gradient is obtained directly during longitudinal federated iterative training. Instead, it obtains the gradient of the loss function over all data. To calculate the gradient... This would require separate backpropagation calculations on all the data, resulting in additional computational and communication overhead. To avoid this computation... To reduce the additional overhead, this invention estimates ΔL by introducing additional parameters into the training process. k，j First, assign weights. Parameterization s k，j These are additional parameters involved in training; h is the transformation function, and because h(s) k，j Let represent the weights of the neuron. Since the weights should be non-negative, the function h(·) should be a non-negative function, for example, h(·) = e^(-1 / 2). s In addition, give s k，j A regular term r(s) k，j At this point during training, parameter s k，j Forward propagation is performed according to the above equation (5), and training is performed through backpropagation and gradient descent. When s k，j When convergence occurs, the gradient should be zero, as shown below:

[0071] g k，j h′(s k，j )+r′(s k，j )=0 (8)

[0072] Where h′(·) and r′(·) are the derivatives of the functions h(·) and r(·), respectively. And if h′(s k，j If )≠0, then we can obtain the result for ΔL. k，j The estimator

[0073]

[0074] If the function h(s)r′(s) / h′(s) is an increasing function, then according to equation (9), when the parameter s k，j When convergence, s k，j Size order and ΔL k，j Consistent. Due to parameter s k，j This reflects the importance of neurons, and will s k，j This is called the importance parameter. It should be noted that in actual calculations, h(s) = e s r(s) = λ1s + λ2s 2 λ1 and λ2 are both hyperparameters.

[0075] To better understand the neuron pruning process, we will use a two-stage iterative pruning training method as an example to illustrate the specific process of pruning output neurons. During the two-stage iterative pruning training process, if the local network initially has d... i If there are 10 neurons, then after T rounds, sort them according to the weight parameters of the output neurons of each local network, and keep the d with the largest weight parameter value. t The most important d neurons t One neuron is removed, and the remaining neurons are deleted, where T and d are... t These are all hyperparameters. It's important to note that a binary mask `m` can be used to indicate whether a neuron is deleted. When `m = 0`, the neuron corresponding to that mask is deleted; when `m = 1`, the neuron corresponding to that mask is not deleted. It's also important to note that after neuron deletion, each local network sends its own binary mask `m` to the prediction network. In subsequent iterative training, when each local network passes its local output to the prediction network, the output of the neuron with `m = 0` is omitted. The prediction network then determines the input neuron corresponding to each local network based on the received local output and the binary mask. Furthermore, when the prediction network transmits gradients to the local networks, the gradients of the neurons with `m = 0` are omitted. (See attached diagram.) Figure 3As shown, it illustrates the communication process between each local network and the prediction network after a neuron is deleted. The cross in the figure indicates that the neuron is deleted, that is, the weights of these neurons are set to 0. During the forward propagation, the outputs of these neurons are omitted (arrows in the figure point from bottom to top), and the gradients of these neurons are also omitted during the backpropagation (arrows in the figure point from top to bottom).

[0076] The training scheme for the longitudinal federated neural network model proposed in this invention can be applied to the medical and health field to achieve health assessment of target subjects.

[0077] According to an embodiment of the present invention, the present invention provides a method for training a diabetes assessment model, the method comprising: T1, acquiring medical data of multiple target objects and corresponding diabetes results, and constructing target domain data using medical data as samples and diabetes results as labels; T2, training a longitudinal federated neural network model based on the target domain data obtained in step T1 using the method described in the foregoing embodiment to obtain a diabetes assessment model.

[0078] According to an embodiment of the present invention, a diabetes assessment method includes: Y1, acquiring medical data of a target object; Y2, processing the medical data of the target object using a diabetes assessment model trained as described in the foregoing embodiments to obtain a diabetes assessment result.

[0079] According to one embodiment of the present invention, a diabetes assessment system includes: a data acquisition module for acquiring medical data of a target object; and a diabetes assessment model trained by the method described in the foregoing embodiments for processing the medical data of the target object obtained by the data acquisition module to obtain a diabetes assessment result.

[0080] Other medical and health fields employ model training methods, assessment methods, and assessment systems for human health assessment based on the same principles as the schemes proposed in the foregoing embodiments, and will not be elaborated upon here.

[0081] 4. Verification Experiment

[0082] To verify that the proposed solution has better performance than existing solutions, validation experiments were conducted on the public datasets MNIST and CIFAR10, using two-stage iterative training with reduction of output neurons in the local network as an example.

[0083] On the MNIST dataset, the longitudinal federated neural network model uses a multilayer perceptron as the local network and a multilayer perceptron with one hidden layer as the prediction network. After 60 training epochs, the validation results are shown in Table 1. Here, Base represents the result without deleting neurons, which includes the output neuron d. i=16 and d i =12 results; Ours represents the results of deleting neurons, which includes the results of deleting output neurons from 16 to 6, 7 and 8, all of which are deleted when the number of training rounds is 10; Acc. represents the highest test set accuracy achieved during training; #Comm.(M)) represents the communication overhead, which is measured by the number of values ​​transmitted, including the number of local results and the number of values ​​in the gradient, in one million; Acc.Limited represents the highest accuracy achieved when the communication overhead is limited to 110. Table 1 shows that when the number of output neurons is 16, the highest test set accuracy achieved during iterative training is 98.73%, with a communication overhead of 230.4. Under limited communication overhead, the highest accuracy achieved is 98.47%. When the number of output neurons is 12, the highest test set accuracy achieved during iterative training is 98.65%, with a communication overhead of 172.8. Under limited communication overhead, the highest accuracy achieved is 98.55%. When the number of output neurons is reduced from 12 to 8, the highest test set accuracy achieved during iterative training is 98.62%. The communication overhead was 134.4, and the highest accuracy achieved under limited communication overhead was 98.55%. When the number of output neurons was reduced from 12 to 7, the highest test set accuracy achieved during iterative training was 98.54%, with a communication overhead of 122.4, and the highest accuracy achieved under limited communication overhead was 98.51%. When the number of output neurons was reduced from 12 to 6, the highest test set accuracy achieved during iterative training was 98.47%, with a communication overhead of 110.4, and the highest accuracy achieved under limited communication overhead was 98.47%. As shown in Table 1, compared with the case of 12 neurons without neuron reduction, the accuracy of the model was similar after reducing neurons from 16 to 8, but the communication overhead after neuron reduction was 134.4, only 77.78% of that without reduction. However, when the number of neurons was reduced from 16 to 8, compared with the case where no neurons were removed and the number of neurons remained at 16, the accuracy of the model after the reduction of neurons was significantly lower. However, under the condition of limiting communication overhead, it was slightly better than not selecting neurons at all.

[0084] Table 1

[0085] Model Acc. #Comm.(M)) Acc.Limited Base 98.73％ 230.4 98.47％ Base 98.65％ 172.8(75％) 98.55％ Ours 98.62％ 134.4(57.33％) 98.55％ Ours 98.54％ 122.4(53.12％) 98.51％ Ours 98.47％ 110.4(47.92％) 98.47％

[0086] On the CIFAR10 dataset, the longitudinal federated neural network model uses a convolutional neural network as the local network and a multilayer perceptron with one hidden layer for the prediction network. After 75 training epochs, the validation results are shown in Table 2. The meanings in the table are the same as in Table 1, and will not be repeated here. Table 2 shows that when the number of output neurons is 64, the highest test set accuracy achieved during iterative training is 84.76%, with a communication overhead of 960. Under limited communication overhead, the highest accuracy achieved is 70.52%. When the number of output neurons is 32, the highest test set accuracy achieved during iterative training is 84.57%, with a communication overhead of 480. Under limited communication overhead, the highest accuracy achieved is 78.12%. When the number of output neurons is reduced from 64 to 16, the highest test set accuracy achieved during iterative training is 84.78%. The communication overhead at this point is 336, and the highest accuracy achieved with limited communication overhead is 77.54%. When the number of output neurons is reduced from 64 to 14, the highest test set accuracy achieved during iterative training is 84.73%, with a communication overhead of 310, and the highest accuracy achieved with limited communication overhead is 78.83%. When the number of output neurons is reduced from 64 to 12, the highest test set accuracy achieved during iterative training is 84.56%, with a communication overhead of 284, and the highest accuracy achieved with limited communication overhead is 80.54%. As shown in Table 2, reducing the number of neurons from 64 to 14 achieves an accuracy similar to that of 64 neurons, while requiring only 32.29% of the original communication overhead, which is 310.

[0087] Table 2

[0088] Model Acc. #Comm.(M)) Acc.Limited Base 84.76％ 960 70.52％ Base 84.57％ 480(50％) 78.12％ Ours 84.78％ 336(35％) 77.54％ Ours 84.73％ 310(32.29％) 78.83％ Ours 84.56％ 284(29.58％) 80.54％

[0089] The beneficial effects of this invention are as follows: by reducing neurons, the amount of data exchanged between the local network and the prediction network during the training of the longitudinal federated neural network model is reduced, thereby reducing communication overhead and enabling the construction of a better model for medical and health assessment.

[0090] It should be noted that although the steps are described in a specific order above, it does not mean that the steps must be executed in the above specific order. In fact, some of these steps can be executed concurrently, or even in a different order, as long as the required function can be achieved.

[0091] This invention can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of the invention.

[0092] Computer-readable storage media can be tangible devices that hold and store instructions for use by an instruction execution device. Computer-readable storage media can be, for example, including but not limited to, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof.

[0093] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A training method for a longitudinal federated neural network model for constructing a diabetes assessment model, wherein the longitudinal federated neural network model includes a prediction network and multiple local networks, each local network including multiple neurons, characterized in that, The method includes: S1. Obtain multiple datasets in the target domain and their corresponding label set. Each dataset includes multiple feature vectors of the target domain data. Distribute a dataset to each local network and distribute the label set to the prediction network. The target domain data is constructed using medical data as samples and diabetes results as labels. S2. Set weight parameters for each neuron in each local network and initialize them; S3. The prediction network and multiple local networks undergo multiple rounds of longitudinal federated iterative training based on the data they obtain until each local network has completed neuron reduction, resulting in a trained longitudinal federated neural network model as a diabetes assessment model. During the iterative training process, the parameters of each local network and the prediction network are updated according to a preset loss function, the weight parameters of the neurons in each local network are updated according to a preset weight update function, and neurons in each local network that do not conform to the preset rules are removed. The preset loss function is: in, Represents the loss function. This represents the prediction loss for a single data point. Represents a regular term, Indicates the weight of the regularization term. This represents the parameter of the first local network. Indicates the first to the second Parameters of a local network, This represents the parameters of the prediction network. Indicates the amount of data; The preset weight update function is: in, Indicates the importance parameter, Indicates neuron weights, Represents the loss function Neuron weights gradient, Represents neuron weights For importance parameters gradient, Indicates the learning rate. Represents a monotonically nonnegative function; The preset rule is as follows: sort the weight parameters of neurons in each local network in descending order, and delete a preset number of neurons at the end of the sort.

2. The method according to claim 1, characterized in that, In step S3, multiple rounds of longitudinal federated iterative training are performed in the following manner: The prediction network and multiple local networks are subjected to a first preset round of longitudinal federated iterative training based on the data they obtain. In each round of iterative training in the first preset round, the neuron weight parameters of each local network are updated according to a preset weight update function. Neurons that do not conform to the rules in each local network after the first preset round of vertical federated iterative training are removed according to the preset rules. The prediction network after the first preset round of training and the multiple local networks after neuron reduction are subjected to a second preset round of longitudinal federated iterative training based on the data obtained to obtain a trained longitudinal federated neural network model. In each iteration of training in the first and second preset rounds, the parameters of each local network and the prediction network are updated by backpropagation according to a preset loss function.

3. The method according to claim 1, characterized in that, Step S3 includes: S31. Perform multi-stage iterative neuron pruning training on the prediction network and multiple local networks based on the data they obtain, wherein each stage of iterative neuron pruning training is performed in the following manner: The prediction network and multiple local networks, after the previous stage of iterative training, are subjected to a third preset round of longitudinal federated iterative training based on the data they have obtained. In each round of iterative training in the current stage, the neuron weight parameters of each local network are updated according to a preset weight update function, and neurons that do not conform to the rules in each local network after the current stage of training are removed according to preset rules. S32. Perform a fourth round of longitudinal federated iterative training on the longitudinal federated neural network model after the multi-stage iterative neuron reduction training to obtain a well-trained longitudinal federated neural network model. In each iteration of training in the third and fourth preset rounds, the parameters of each local network and the prediction network are updated by backpropagation according to a preset loss function.

4. A method for training a diabetes assessment model, characterized in that, The method includes: T1. Obtain medical data and corresponding diabetes outcomes for multiple target objects, and construct target domain data using medical data as samples and diabetes outcomes as labels; T2. Based on the target domain data obtained in step T1, a longitudinal federated neural network model is trained using the method described in any one of claims 1-3 to obtain a diabetes assessment model.

5. A method for assessing diabetes, characterized in that, The method includes: Y1. Obtain the medical data of the target object; Y2. The diabetes assessment model trained using the method described in claim 4 is used to process the medical data of the target object to obtain diabetes assessment results.

6. A diabetes assessment system, characterized in that, The system includes: The data acquisition module is used to acquire the medical data of the target object; The diabetes assessment model trained by the method described in claim 4 is used to process the medical data of the target object obtained by the data acquisition module to obtain diabetes assessment results.

7. A computer-readable storage medium, characterized in that, It stores a computer program that can be executed by a processor to implement the steps of the method according to any one of claims 1 to 5.

8. An electronic device, characterized in that, include: One or more processors; A storage device for storing one or more programs that, when executed by one or more processors, cause the electronic device to perform the steps of the method as described in any one of claims 1 to 5.

Images