Safe interaction method and device for federal training medical model, equipment and medium
By using blockchain registration server and client in a federated learning environment to verify and update the accuracy of model weights, the problem of model correction value evaluation in federated learning is solved, reducing the risk of attacks and improving the training effect.
Patent Information
- Application Number
- CN202411708876.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-26
- Publication Date
- 2025-03-18
AI Technical Summary
In a federated learning environment, the central server cannot evaluate the correctness and quality of the model correction values provided by the client, resulting in the possibility of an attacker's attack on training data, training process, or model updates, causing the global model to deviate from its expected direction.
By registering the server and client in the blockchain, the server broadcasts the global model weight to the blockchain, the client updates the local model and trains it, and then returns the updated model weight to the blockchain, the server verifies the accuracy of the update model weight and updates the global model, and the client calculates the reward allocation based on the accuracy.
It realizes improving interaction capabilities in federated learning, reducing the possibility of attacks, thereby improving training effect, limiting clients through the blockchain's reward mechanism to prevent the use of forged offensive data.
Smart Images
Figure CN119646807A_ABST
Abstract
Description
Technical Field
[0001] The present application is applicable to the field of digital medical technology, and in particular, relates to a secure interaction method, device, equipment and medium for a federated training medical model. Background Art
[0002] With the development of digital medical technology, the use of advanced computing models for data computing in the medical field has become an important topic. However, such computing models generally require a large amount of data support and need to perform a relatively large amount of data operations. In this regard, it is necessary to rely on multi-terminal federated training to deal with the training of models in the case of large data volumes and large models. Federated training relies on Federated Learning (FL), a subfield of machine learning that allows participants to collaborate in model training without requiring them to move data to a central location. This alone can significantly improve privacy because the data remains on the user's device. Considering the excessive amount of data generated at the terminal device (for example, smartphones), it is usually preferred to perform training at the edge. In the simplest and most popular form of FL, the data sets belonging to the participating entities have similar feature sets. This occurs when organizations have related operations (such as customer data sets belonging to Bank A and Bank B, respectively). In this example, it is expected that the sample space does not overlap between the data sets. This type of FL is called horizontal FL. Vertical FL can be performed when the data sets do not share the same feature set but at least share some common sample space.
[0003] In the FL environment, the global model becomes more accurate through the continuous contribution of the clients. However, the central server cannot access the actual training data and therefore cannot evaluate the correctness and quality of the model corrections provided by the clients. Therefore, it is possible for an attacker to make the global model deviate as much as possible from its expected direction by attacking the training data, the training process, or the resulting model updates. Two popular attacks are label flipping and backdoor attacks.
[0004] Label flipping attacks involve manipulating the training dataset. That is, the training labels are intentionally changed while the data remains unchanged. In the FL setting, the resulting model updates will hamper the performance of the global model and, depending on the scale of the attack, the configured learning rate, and the averaging algorithm used, may even render the model unavailable and cause a denial of service attack. Backdoor attacks involve installing hidden features in machine learning models. The machine learning model initially appears to be functioning normally until the attacker triggers the backdoor using a specific pattern, at which point the model's behavior changes. In order for this attack to be successful, the attacker trains the model to make a certain prediction when a specific pattern appears as input. Therefore, how to improve the interactive capabilities in federated learning to reduce the occurrence of attacks and thus improve training results has become an urgent problem to be solved. Summary of the invention
[0005] In view of this, the embodiments of the present application provide a secure interaction method, device, equipment and medium for federated training of medical models to solve the problem of how to improve the interaction capability in federated learning to reduce the occurrence of attacks and thus improve the training effect.
[0006] In a first aspect, an embodiment of the present application provides a secure interaction method for a federated training medical model, wherein the secure interaction method is run in a federated system, wherein the federated system includes a server and at least one client, wherein the server and all clients are registered in the same blockchain, and wherein the secure interaction method includes: The server broadcasts the current global model weight of the current global model obtained from the previous round of training to the blockchain; For any client, receive the current global model weight broadcasted in the blockchain, update the current local model of the client according to the current global model weight to obtain an updated local model, train the updated local model using the local data of the client to obtain a trained local model, and broadcast the updated model weight of the trained local model to the blockchain; The server receives all the updated model weights broadcasted in the blockchain, calculates the accuracy of each updated model weight using the verification data, obtains the first accuracy of the corresponding updated model weight, updates the current global model according to all the updated model weights, obtains the current global model of this round of training, and broadcasts the first accuracy of all the updated model weights to the blockchain; For any client, the first accuracy of all updated model weights is received, and the reward allocation amount of the client is calculated, and the reward allocation amount is used to apply for rewards from the server after all rounds of training are completed, wherein the first accuracy is proportional to the reward allocation amount.
[0007] In a second aspect, an embodiment of the present application provides a secure interaction device for a federated training medical model, wherein the secure interaction device runs in a federated system, wherein the federated system includes a server and at least one client, wherein the server and all clients are registered in the same blockchain, and wherein the secure interaction device includes: A server broadcast module, used for the server to broadcast the current global model weight of the current global model obtained from the previous round of training to the blockchain; A client training module is used to receive the current global model weight broadcasted in the blockchain for any client, update the current local model of the client according to the current global model weight to obtain an updated local model, train the updated local model using the local data of the client to obtain a trained local model, and broadcast the updated model weight of the trained local model to the blockchain; A server training module is used for receiving all the updated model weights broadcasted in the blockchain, using the verification data to calculate the accuracy of each updated model weight, obtaining the first accuracy of the corresponding updated model weight, updating the current global model according to all the updated model weights, obtaining the current global model of this round of training, and broadcasting the first accuracy of all the updated model weights to the blockchain; The client reward application module is used to receive the first accuracy of all updated model weights for any client and calculate the reward allocation amount of the client, and the reward allocation amount is used to apply for rewards from the server after all rounds of training are completed, wherein the first accuracy is proportional to the reward allocation amount.
[0008] In a third aspect, an embodiment of the present application provides a computer device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the secure interaction method as described in the first aspect when executing the computer program.
[0009] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the secure interaction method as described in the first aspect is implemented.
[0010] Compared with the prior art, the embodiments of the present application have the following beneficial effects: the method of the present application is run in a federated system, the federated system includes a server and at least one client, the server and all the clients are registered in the same blockchain, the server broadcasts the current global model weight of the current global model obtained in the previous round of training to the blockchain, any client receives the current global model weight broadcasted in the blockchain, updates the current local model of the client according to the current global model weight, obtains an updated local model, trains the updated local model using the local data of the client, obtains a trained local model, broadcasts the updated model weight of the trained local model to the blockchain, and the server receives all updated models broadcasted in the blockchain. New model weights, use verification data to calculate the accuracy of each updated model weight respectively, obtain the first accuracy of the corresponding updated model weight, update the current global model according to all updated model weights, obtain the current global model of this round of training, broadcast the first accuracy of all updated model weights to the blockchain, the client receives the first accuracy of all updated model weights, and calculates the client's reward allocation. The reward allocation is used to apply for rewards from the server after all rounds of training are completed, realize the federated learning of the model, and limit the client based on the reward method in the blockchain to reduce the impact of the client's use of forged offensive data and other means on federated learning during the interaction process, thereby improving the effect of federated learning. BRIEF DESCRIPTION OF THE DRAWINGS
[0011] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0012] Figure 1 This is a schematic diagram of an application environment of a secure interactive method for a federated training medical model provided in Example 1 of the present application; Figure 2 It is a flowchart of a secure interaction method for a federated training medical model provided in Example 2 of the present application; Figure 3 It is a flowchart of a secure interaction method for a federated training medical model provided in Example 3 of the present application; Figure 4 It is a schematic diagram of the structure of a secure interactive device for a federated training medical model provided in the fourth embodiment of the present application; Figure 5 It is a structural diagram of a computer device provided in Example 5 of the present application. DETAILED DESCRIPTION
[0013] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.
[0014] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.
[0015] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.
[0016] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.
[0017] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.
[0018] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.
[0019] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.
[0020] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.
[0021] It should be understood that the size of the serial numbers of the steps in the following embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.
[0022] In order to illustrate the technical solution of the present application, a specific embodiment is provided below for illustration.
[0023] The secure interaction method of a federated training medical model provided in the first embodiment of the present application can be applied in the following aspects: Figure 1 In the application environment, the server communicates with the client. The client includes but is not limited to PDA, desktop computer, notebook computer, ultra-mobile personal computer (UMPC), netbook, cloud computing device, personal digital assistant (PDA) and other computer devices. The server can be implemented by an independent server or a server cluster composed of multiple servers.
[0024] See also Figure 2, is a flowchart of a safe interactive method for a federated training medical model provided in Example 2 of the present application. The medical model includes some classification models, prediction models, image segmentation models, etc., all of which are used to realize functions such as image segmentation, data classification, case analysis, etc. in the medical field. For example, for images of internal human tissues obtained in a non-invasive manner, such as images of the stomach, abdomen, heart, knees, and brain, such as computed tomography (CT) imaging, magnetic resonance imaging (MRI), ultrasound (US) imaging, X-ray imaging, electroencephalogram, and optical photography lights generated by medical instruments, image segmentation of the above images can locate the lesion location or identify the lesion type, etc., which helps to improve the efficiency of image analysis.
[0025] The secure interaction method of the above-mentioned federated training medical model is applied to Figure 1 In the federated system consisting of the interaction between the server and the client, the client and the server are connected through a blockchain, that is, the client and the server are both registered in a blockchain to execute the smart contract of the blockchain, which is used to constrain the client and the server to perform necessary operations and not to breach the contract. Figure 2 As shown, the secure interaction method of the federated training medical model may include the following steps: Step S201: The server broadcasts the current global model weight of the current global model obtained from the previous round of training to the blockchain.
[0026] In this application, a global model is running in the server, and a local model is running in each client, where the global model and the local model are models with the same architecture. Federated training is that the client receives information such as parameters of the global model, updates the local model, and then uses local data for training, and sends the parameters and other information obtained from the training to the server. The server retrains the global model based on the received information to obtain the global model of this training.
[0027] After a round of training, the global model is obtained as the current global model. The weight of the current global model is obtained as the current global model weight, and it is broadcasted to the blockchain (that is, the information is put on the chain). The information broadcasted in the blockchain can be received by other devices in the blockchain. Among them, the current global model weight can refer to the weight of each convolutional layer, fully connected layer, activation layer, etc. in the global model.
[0028] Step S202: For any client, receive the current global model weight broadcasted in the blockchain, update the current local model of the client according to the current global model weight, obtain an updated local model, train the updated local model using the local data of the client to obtain a trained local model, and broadcast the updated model weight of the trained local model to the blockchain.
[0029] In this application, the current global model weight is broadcast in the blockchain, so each client can obtain the current global model weight. Since the global model and the local model have the same model architecture, the current local model of the client can be updated according to the current global model weight. The update process is to replace the weight at the corresponding parameter position in the current local model with the current global model weight, thereby obtaining an updated local model.
[0030] The weights in the updated local model are the current global model weights. The client obtains local data to train the updated local model. After the training, a trained local model is obtained. The trained local model is only the model trained in this round of training. When entering the next round of training, the trained model corresponding to the next round of training will still be obtained.
[0031] The global model, local model, etc. of this application are models for use in medical treatment. The specific training process is performed in the client, that is, the model is trained using the client's local data. The local data can be an image data training set, a text data training set, an audio data training set, or other data training sets.
[0032] For example, if the medical model is an image segmentation model, an image data training set can be used. In this case, the image data training set can include at least one MRI image and the segmentation results corresponding to each image. The image data training set is input into the model, and the output prediction result is compared with the segmentation result to adjust the parameter weights of the model, and finally the iteration is completed to obtain a trained model.
[0033] For example, if the medical model is a case analysis model, a text data training set can be used. In this case, the text data training set can include at least one medical record text and the analysis results corresponding to each medical record text. The text data training set is input into the model, and the output prediction result is compared with the analysis result to adjust the parameter weights of the model, and finally the iteration is completed to obtain a trained model.
[0034] For example, if the medical model is an audio classification model, an audio data training set can be used. In this case, the audio data training set can include at least one patient voiceprint data and the classification result corresponding to each voiceprint data. The audio data training set is input into the model, and the output prediction result is compared with the classification result to adjust the parameter weights of the model, and finally the iteration is completed to obtain a trained model.
[0035] During the training process, each iteration will adjust the weights in the local model so that the local model reaches the target. Therefore, the weights of the trained local model may change compared to the weights of the updated local model. The weights of the trained local model are collected and sent to the server for use in training the server's global model. The updated model weights of the collected trained local model are broadcasted to the chain, so that the server and even other clients can obtain the updated model weights.
[0036] Optionally, the current local model of the client is updated according to the current global model weight, and the updated local model includes: Get the current local model weight corresponding to the current local model of the client; Fuse the current global model weight with the current local model weight to obtain the local fusion weight; The client's current local model is updated using the local fusion weights to obtain an updated local model.
[0037] Among them, after the client obtains the current global model weight, it no longer directly replaces the local model weight, but fuses the current global model weight with the current local model weight to obtain the local fusion weight, which is then used to update the local model, so that the updated local model can better participate in the training of the local model, which helps to improve the efficiency of training convergence.
[0038] In step S203, the server receives all updated model weights broadcasted in the blockchain, calculates the accuracy of each updated model weight using the verification data, obtains the first accuracy of the corresponding updated model weight, updates the current global model according to all updated model weights, obtains the current global model of this round of training, and broadcasts the first accuracy of all updated model weights to the blockchain.
[0039] In the present application, verification data is stored in the server, which may refer to a set of accurately labeled data. Such data can be input into the model to obtain the predicted label predicted by the model. The predicted label and the label itself can calculate the degree of difference or accuracy, so as to evaluate the accuracy of the model.
[0040] The verification data needs to calculate the accuracy of each updated model weight separately to obtain the first accuracy corresponding to each updated model weight, which is used to implement the reward distribution of the client later. The higher the first accuracy corresponding to the updated model weight, the higher the value of the updated model weight and the higher the contribution to the global model. Accordingly, more rewards should be allocated when the rewards are finally distributed. To improve the first accuracy of the updated model weight, it is necessary to use accurate local data when training the local model, and avoid using illegal data or offensive data.
[0041] To update the current global model according to all the update model weights, it is necessary to use all the update model weights to obtain the update weights used to update the global model, so as to use the update weights to replace the weights at the corresponding positions in the current global model.
[0042] The first accuracy is broadcasted to the blockchain to be received by the client for subsequent reward application and collection. Furthermore, if the global model after this round of training meets the preset conditions and is determined to be the final global model, the weight of the final global model can also be broadcasted to the blockchain for the client to obtain and update the local model, thus realizing the model update at the client, so that the model on the client meets the conditions for use.
[0043] Optionally, the current global model is updated according to all updated model weights, and the current global model for this round of training includes: Detecting whether the first accuracy of all updated model weights is greater than a first accuracy threshold, and determining the updated model weights greater than the first accuracy threshold as target weights; The current global model is updated according to all target weights to obtain the current global model for this round of training.
[0044] Among them, after the server receives all the updated model weights broadcasted, it needs to use all the updated model weights to update the current global model of this round of training. This application has added a process of verifying the accuracy of the updated model weights before this, so as to improve the accuracy of the update when updating the current global model.
[0045] The first accuracy threshold is used to judge whether the updated model weight can contribute to the update of the current global model. For example, the first accuracy threshold is 0.5, that is, the updated model weight with a first accuracy less than or equal to 0.5 cannot contribute to improving the accuracy of the update of the current global model and can be eliminated without reference. The updated model weight with a first accuracy greater than 0.5 can contribute to improving the accuracy of the update of the current global model and can be retained for use.
[0046] Optionally, the current global model is updated according to all target weights, and the current global model for this round of training includes: Perform weighted summation on all target weights to obtain a weighted summation result; The weighted sum result is used to update the current global model to obtain the current global model for this round of training.
[0047] Among them, for the retained target weights, when updating the current global model, a weighted summation method is adopted to obtain an update weight, which is used to update the current global model to obtain the current global model of this round of training.
[0048] The sum of all target weights is 1. Furthermore, under certain weights, all target weights can be averaged.
[0049] Optionally, all target weights are weighted and summed, and the weighted sum result includes: Perform normalization calculation on the first accuracy of each target weight to obtain a normalized value of the corresponding target weight; Each target weight is multiplied by its corresponding normalized value and then summed up to obtain the weighted summation result.
[0050] Among them, the weight of the weight during weighting can be associated with the above-mentioned first accuracy. The higher the accuracy, the higher the corresponding weight, and the lower the accuracy, the lower the corresponding weight. The above-mentioned normalized calculation can obtain the corresponding weight that meets the conditions.
[0051] Step S204: for any client, receiving the first accuracy of all updated model weights, and calculating the reward allocation amount of the client.
[0052] In this application, the reward allocation amount is used to apply for rewards from the server after all rounds of training are completed, and the first accuracy is proportional to the reward allocation amount. After receiving the first accuracy, the client can apply for rewards from the server according to its own first accuracy to obtain corresponding rewards. It can be seen that the reward allocation amount is proportional to the first accuracy. For malicious or non-contributory clients, the weights they provide do not contribute to the global model, so they cannot obtain rewards. Furthermore, if no reward is obtained within a certain period of time, or the amount of reward obtained is low, the client can be regarded as a malicious attack client and thus be eliminated to reduce the impact of attacks on the federal system.
[0053] The method of the embodiment of the present application runs in a federated system, which includes a server and at least one client. The server and all the clients are registered in the same blockchain. The server broadcasts the current global model weight of the current global model obtained in the previous round of training to the blockchain. Any client receives the current global model weight broadcasted in the blockchain, updates the current local model of the client according to the current global model weight, obtains an updated local model, trains the updated local model using the local data of the client, obtains a trained local model, broadcasts the updated model weight of the trained local model to the blockchain, and receives all the updated model weights broadcasted in the blockchain. The server verifies the updated model weights using the local data of the client. The data calculates the accuracy of each updated model weight respectively, obtains the first accuracy of the corresponding updated model weight, updates the current global model according to all the updated model weights, obtains the current global model of this round of training, and broadcasts the first accuracy of all updated model weights to the blockchain. The client receives the first accuracy of all updated model weights and calculates the client's reward allocation. The reward allocation is used to apply for rewards from the server after all rounds of training are completed to realize the federated learning of the model. The client is limited by the reward method in the blockchain to reduce the impact of the client's use of forged offensive data and other means on federated learning during the interaction process, thereby improving the effect of federated learning.
[0054] See also Figure 3 , is a flowchart of a secure interactive method for a federated training medical model provided in Example 3 of the present application. Figure 3 As shown, the secure interaction method of the federated training medical model may include the following steps: Step S301: The server broadcasts the current global model weight of the current global model obtained from the previous round of training to the blockchain.
[0055] Step S302: For any client, receive the current global model weight broadcasted in the blockchain, update the current local model of the client according to the current global model weight, obtain an updated local model, train the updated local model using the local data of the client, obtain a trained local model, and broadcast the updated model weight of the trained local model to the blockchain.
[0056] Among them, the contents of step S301 to step S302 are the same as part of the contents of the above-mentioned steps S201 to step S202. Please refer to the description of steps S201 to step S202, and will not be repeated here.
[0057] Step S303: The server receives all updated model weights broadcasted in the blockchain, and for any updated model weight, uses the updated model weight to update a verification model to obtain an updated verification model corresponding to the updated model weight.
[0058] In this application, each updated model weight can update the weight of a verification model, making the verification model become an updated verification model, and the input of all updated verification models is verification data. The server can use only one verification model. After one updated model weight completes the verification model update and subsequent use, the next updated model weight updates and subsequently uses the verification model, forming a serial verification calculation process, which helps to reduce resource usage.
[0059] Furthermore, the server can copy the verification model to obtain the same number of verification models as the update model weights, and then form an updated verification model corresponding to one update model weight for subsequent use. This process is a parallel verification calculation process, which helps to improve efficiency.
[0060] Step S304, obtaining verification data, and using the updated verification model to perform prediction calculations on the verification data to obtain prediction calculation results.
[0061] In the present application, the verification model is updated to perform predictive calculations on the verification data, and the prediction includes classification, segmentation, etc., wherein, if applied to medical classification, the verification model is updated to be a classification model, and the verification data is the data object and the true label (that is, the classification result of the data object), and the classification result is obtained after classifying the data object; if applied to medical image segmentation, the verification model is updated to be a segmentation model, and the verification data is the image data and the true label (that is, the segmentation result of the image data), and the segmentation result is obtained after segmenting the image data, and so on.
[0062] Step S305, obtaining the first accuracy of the prediction calculation according to the prediction calculation result and the true label of the verification data, traversing all the update model weights, and obtaining the first accuracy corresponding to each update model weight.
[0063] In this application, the verification data can be a data set, and the first accuracy of the prediction calculation is determined by calculating the two ratios of consistency and inconsistency between the prediction calculation result and the true label. Of course, the verification data can be a data object, and the final prediction calculation result is compared with the true label for similarity, and the similarity result is the first accuracy.
[0064] Optionally, after receiving the first accuracy of all updated model weights for any client, further comprising Receive updated model weights broadcasted by other clients from the blockchain; According to the verification data and verification model stored in the client and the same as those in the server, the accuracy of the received updated model weight is calculated to obtain a second accuracy corresponding to the updated model weight; For any updated model weight, compare the first accuracy and the second accuracy of the updated model weight to obtain a comparison result. If the comparison result is a large difference, an error prompt is generated and broadcast. The error prompt is used to be received by the server after being broadcast to all clients and stop this round of training.
[0065] Among them, the client can obtain the updated model weights of other clients, and use the verification data and verification model stored in itself that are the same as those of the server for calculation, to prevent the server from making calculation errors or maliciously giving false information.
[0066] The above verification data and verification model can be broadcasted by the server on the blockchain so that the client can obtain them.
[0067] After self-verification by all clients and consensus in the blockchain, that is, most of the self-verification results in the clients are significantly different from the results given by the server, it can be considered that there is a certain fault on the server, and training needs to be stopped and the operator needs to make certain corrections.
[0068] Step S306: Update the current global model according to all updated model weights to obtain the current global model of this round of training, and broadcast the first accuracy of all updated model weights to the blockchain.
[0069] Step S307: for any client, receiving the first accuracy of all updated model weights, and calculating the reward allocation amount of the client.
[0070] Among them, the contents of step S306 to step S307 are the same as part of the contents of the above-mentioned steps S203 to step S204. Please refer to the description of steps S203 to step S204, and will not be repeated here.
[0071] The method of the embodiment of the present application runs in a federated system, which includes a server and at least one client. The server and all the clients are registered in the same blockchain. The server broadcasts the current global model weight of the current global model obtained in the previous round of training to the blockchain. Any client receives the current global model weight broadcasted in the blockchain, updates the current local model of the client according to the current global model weight, obtains an updated local model, trains the updated local model using the local data of the client to obtain a trained local model, broadcasts the updated model weight of the trained local model to the blockchain, and the server receives all the updated model weights broadcasted in the blockchain. For any updated model weight, uses the updated model weight to update a verification model to obtain an updated verification model corresponding to the updated model weight, obtains verification data, and uses The verification model is updated to perform prediction calculations on the verification data to obtain the prediction calculation results. According to the prediction calculation results and the true labels of the verification data, the first accuracy of the prediction calculation is obtained. All update model weights are traversed to obtain the first accuracy corresponding to each update model weight. The current global model is updated according to all update model weights to obtain the current global model for this round of training. The first accuracy of all update model weights is broadcast to the blockchain. The client receives the first accuracy of all update model weights and calculates the client's reward allocation. The reward allocation is used to apply for rewards from the server after all rounds of training are completed to realize the federated learning of the model. The client is limited by the reward method in the blockchain to reduce the impact of the client's use of forged offensive data and other means on federated learning during the interaction process, thereby improving the effect of federated learning.
[0072] Corresponding to the secure interaction method of the federated training medical model in the above embodiment, Figure 4 A structural block diagram of a secure interactive device for a federated training medical model provided in Example 4 of the present application is shown. The medical model includes some classification models, prediction models, image segmentation models, etc., all of which are used to implement functions in the medical field such as image segmentation, data classification, and case analysis.
[0073] The secure interaction method of the above-mentioned federated training medical model is applied to Figure 1 In the federated system consisting of the interaction between the server and the client, the client and the server are connected through a blockchain, that is, the client and the server are both registered in a blockchain to execute the smart contract of the blockchain, which is used to constrain the client and the server to perform necessary operations and not to breach the contract. For ease of explanation, only the parts related to the embodiments of the present application are shown.
[0074] See also Figure 4 , the safety interaction device comprises: The server broadcast module 41 is used for the server to broadcast the current global model weight of the current global model obtained from the previous round of training to the blockchain; The client training module 42 is used for receiving the current global model weight broadcasted in the blockchain for any client, updating the current local model of the client according to the current global model weight to obtain an updated local model, training the updated local model using the local data of the client to obtain a trained local model, and broadcasting the updated model weight of the trained local model to the blockchain; The server training module 43 is used for receiving all the updated model weights broadcasted in the blockchain, using the verification data to calculate the accuracy of each updated model weight, obtaining the first accuracy of the corresponding updated model weight, updating the current global model according to all the updated model weights, obtaining the current global model of this round of training, and broadcasting the first accuracy of all the updated model weights to the blockchain; The client reward application module 44 is used to receive the first accuracy of all updated model weights for any client and calculate the reward allocation amount of the client. The reward allocation amount is used to apply for a reward from the server after all rounds of training are completed, wherein the first accuracy is proportional to the reward allocation amount.
[0075] Optionally, the client training module 42 includes: A local weight acquisition unit, used to acquire a current local model weight corresponding to a current local model of the client; A local weight fusion unit, used to fuse the current global model weight with the current local model weight to obtain a local fusion weight; The local model updating unit is used to update the current local model of the client using the local fusion weight to obtain an updated local model.
[0076] Optionally, the server training module 43 includes: An update verification unit, for updating a verification model using the update model weight for any update model weight, to obtain an update verification model corresponding to the update model weight; A prediction calculation unit is used to obtain verification data, perform prediction calculation on the verification data using an updated verification model, and obtain a prediction calculation result; The first accuracy calculation unit is used to obtain the first accuracy of the prediction calculation according to the prediction calculation result and the real label of the verification data, and traverse all the update model weights to obtain the first accuracy corresponding to each update model weight.
[0077] Optionally, the safety interaction device further includes A weight receiving module, which is used to receive the updated model weights broadcasted by other clients from the blockchain after receiving the first accuracy of all updated model weights for any client; A second accuracy calculation module, configured to perform accuracy calculation on the received updated model weights according to the same verification data and verification model stored in the client as those in the server, to obtain a second accuracy corresponding to the updated model weights; The comparison module is used to compare the first accuracy and the second accuracy of the updated model weight for any updated model weight to obtain a comparison result. If the comparison result is a large difference, an error prompt is generated and broadcast. The error prompt is used to be received by the server after being broadcast to all clients and stop the current round of training.
[0078] Optionally, the server training module 43 includes: an accuracy detection unit, configured to detect whether the first accuracy of all updated model weights is greater than a first accuracy threshold, and determine the updated model weights greater than the first accuracy threshold as target weights; The global model updating unit is used to update the current global model according to all target weights to obtain the current global model of this round of training.
[0079] Optionally, the global model updating unit includes: The weighted summation subunit is used to perform weighted summation on all target weights to obtain a weighted summation result; The global model updating subunit is used to update the current global model using the weighted summation result to obtain the current global model of this round of training.
[0080] Optionally, the weighted summation subunit is specifically used for: Perform normalization calculation on the first accuracy of each target weight to obtain a normalized value of the corresponding target weight; Each target weight is multiplied by its corresponding normalized value and then summed up to obtain the weighted summation result.
[0081] It should be noted that the information interaction, execution process and other contents between the above-mentioned modules are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.
[0082] Figure 5 This is a schematic diagram of the structure of a computer device provided in Example 5 of this application. Figure 5 As shown, the computer device of this embodiment includes: at least one processor ( Figure 5Only one is shown), a memory, and a computer program stored in the memory and executable on at least one processor, which implements the steps of any of the above-mentioned embodiments of the method for secure interaction of federated training medical models when executing the computer program.
[0083] The computer device may include, but is not limited to, a processor and a memory. Those skilled in the art will appreciate that Figure 5 This is merely an example of a computer device and does not constitute a limitation on the computer device. The computer device may include more or fewer components than those shown in the figure, or a combination of certain components, or different components. For example, it may also include a network interface, a display screen, and an input device.
[0084] The processor may be a CPU, or other general-purpose processors, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.
[0085] The memory includes a readable storage medium, an internal memory, etc., wherein the internal memory may be the memory of a computer device, and the internal memory provides an environment for the operation of the operating system and computer-readable instructions in the readable storage medium. The readable storage medium may be a hard disk of a computer device, and in other embodiments, it may also be an external storage device of the computer device, for example, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device. Further, the memory may also include both an internal storage unit of the computer device and an external storage device. The memory is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of a computer program, etc. The memory may also be used to temporarily store data that has been output or is to be output.
[0086] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned device can refer to the corresponding process in the above-mentioned method embodiment, which will not be repeated here. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned method embodiment can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device capable of carrying computer program code, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.
[0087] The present application implements all or part of the processes in the above-mentioned embodiment method, and may also be completed through a computer program product. When the computer program product runs on a computer device, the computer device can implement the steps in the above-mentioned method embodiment when executing the computer program product.
[0088] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0089] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0090] In the embodiments provided in the present application, it should be understood that the disclosed devices / computer equipment and methods can be implemented in other ways. For example, the device / computer equipment embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0091] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0092] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.
Claims
1. A secure interactive method for a federated training medical model, characterized in that: The secure interaction method is run in a federated system, the federated system includes a server and at least one client, the server and all the clients are registered in the same blockchain, and the secure interaction method includes: The server broadcasts the current global model weight of the current global model obtained from the previous round of training to the blockchain; For any client, receive the current global model weight broadcasted in the blockchain, update the current local model of the client according to the current global model weight to obtain an updated local model, train the updated local model using the local data of the client to obtain a trained local model, and broadcast the updated model weight of the trained local model to the blockchain; The server receives all the updated model weights broadcasted in the blockchain, calculates the accuracy of each updated model weight using the verification data, obtains the first accuracy of the corresponding updated model weight, updates the current global model according to all the updated model weights, obtains the current global model of this round of training, and broadcasts the first accuracy of all the updated model weights to the blockchain; For any client, the first accuracy of all updated model weights is received, and the reward allocation amount of the client is calculated, and the reward allocation amount is used to apply for rewards from the server after all rounds of training are completed, wherein the first accuracy is proportional to the reward allocation amount.
2. The secure interaction method according to claim 1, characterized in that: Updating the current local model of the client according to the current global model weight to obtain the updated local model includes: Obtaining a current local model weight corresponding to the current local model of the client; Fusing the current global model weight with the current local model weight to obtain a local fusion weight; The local fusion weight is used to update the current local model of the client to obtain an updated local model.
3. The secure interaction method according to claim 1, characterized in that: Use the validation data to calculate the accuracy of each updated model weight, and obtain the first accuracy of the corresponding updated model weight, including: For any updated model weight, use the updated model weight to update a verification model to obtain an updated verification model corresponding to the updated model weight; Acquire verification data, and use the updated verification model to perform prediction calculation on the verification data to obtain a prediction calculation result; According to the prediction calculation result and the true label of the verification data, a first accuracy of the prediction calculation is obtained, and all update model weights are traversed to obtain a first accuracy corresponding to each update model weight.
4. The secure interaction method according to claim 3, characterized in that: After receiving the first accuracy of all updated model weights for any client, further comprising Receiving updated model weights broadcasted by other clients from the blockchain; According to the verification data and the verification model stored in the client and the same as those in the server, the accuracy of the received updated model weight is calculated to obtain a second accuracy corresponding to the updated model weight; For any updated model weight, compare the first accuracy and the second accuracy of the updated model weight to obtain a comparison result. If the comparison result is a large difference, an error prompt is generated and broadcast. The error prompt is used to be received by the server after being broadcast to all clients and stop the current round of training.
5. The secure interaction method according to claim 1, characterized in that: The current global model is updated according to all updated model weights, and the current global model obtained in this round of training includes: Detecting whether the first accuracy of all updated model weights is greater than a first accuracy threshold, and determining the updated model weight greater than the first accuracy threshold as the target weight; The current global model is updated according to all target weights to obtain the current global model of this round of training.
6. The secure interaction method according to claim 5, characterized in that: The current global model is updated according to all target weights, and the current global model of this round of training is obtained, including: Perform weighted summation on all target weights to obtain a weighted summation result; The current global model is updated using the weighted sum result to obtain the current global model of this round of training.
7. The secure interaction method according to claim 6, characterized in that: All target weights are weighted and summed to obtain the weighted sum results including: Perform normalization calculation on the first accuracy of each target weight to obtain a normalized value of the corresponding target weight; Each target weight is multiplied by its corresponding normalized value and then summed up to obtain the weighted summation result.
8. A secure interactive device for a federated training medical model, characterized in that: The secure interaction device runs in a federated system, the federated system includes a server and at least one client, the server and all the clients are registered in the same blockchain, and the secure interaction device includes: A server broadcast module, used for the server to broadcast the current global model weight of the current global model obtained from the previous round of training to the blockchain; A client training module is used to receive the current global model weight broadcasted in the blockchain for any client, update the current local model of the client according to the current global model weight to obtain an updated local model, train the updated local model using the local data of the client to obtain a trained local model, and broadcast the updated model weight of the trained local model to the blockchain; A server training module is used for receiving all the updated model weights broadcasted in the blockchain, using the verification data to calculate the accuracy of each updated model weight, obtaining the first accuracy of the corresponding updated model weight, updating the current global model according to all the updated model weights, obtaining the current global model of this round of training, and broadcasting the first accuracy of all the updated model weights to the blockchain; The client reward application module is used to receive the first accuracy of all updated model weights for any client and calculate the reward allocation amount of the client, and the reward allocation amount is used to apply for rewards from the server after all rounds of training are completed, wherein the first accuracy is proportional to the reward allocation amount.
9. A computer device, characterized in that: The computer device includes a processor, a memory, and a computer program stored in the memory and executable on the processor, and the processor implements the secure interaction method according to any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the secure interaction method according to any one of claims 1 to 7 is implemented.