Medical system federated learning method based on watermark embedding and medical system

Abstract

The invention provides a medical system federal learning method based on watermark embedding. According to the technical scheme of the invention, the first-stage training of the preset round is carried out on each client in the medical system to obtain the watermark depth of each client, and the adaptation capability of each client to the watermark task is evaluated based on the watermark depth of each client. And reconfiguring the proportion of the main task and the watermark task of the client based on the watermark depth of each client, adjusting the preset loss function of the client according to the proportion, and finally performing multiple rounds of federal training based on the adjusted loss function of each client so as to enable each client to obtain a watermark-embedded medical task model. Compared with an existing watermark embedding technology, watermark embedding is achieved through cooperation of the central server and the clients in the medical system model, meanwhile, the proportion of the watermark task and the main task of each client is adjusted based on the watermark task adaptation capacity of each client, the watermark task performance is improved, and meanwhile the precision of the main task is ensured.

Description

Technical Field

[0001] The present invention relates to the field of federated learning, specifically to the field of watermark embedding of a federated learning model, and more specifically to a medical system federated learning method and a medical system based on watermark embedding. Background Art

[0002] With the continuous development of artificial intelligence technology, data privacy and security issues have become particularly important in sensitive areas such as the medical system. Federated learning in the medical field effectively protects data privacy by training models on devices or nodes in multiple hospitals or medical institutions without sharing original medical data, and aggregating local updates into a global model. However, the distributed nature of federated learning also brings new security and trust challenges, such as model theft, data tampering, and attacks by malicious participants. In the medical system, the challenges brought by these issues are particularly prominent, and they are directly related to patient privacy and the security of medical data. In order to solve these problems, model watermarking technology, as an effective model protection method, has begun to attract the attention of researchers. Model watermarking embeds a secret identifier in the model without affecting the performance of the model itself as much as possible, allowing the owner of the model to prove its ownership by identifying the watermark when the model is stolen or abused. This is particularly important for copyright protection of deep learning in the medical field. In the federated learning scenario of the medical system, large hospitals have a large amount of evenly distributed patient data, while specialized hospitals focus on specific departments, resulting in significant label offsets in the data. Township hospitals face the problem of small data volume and uneven distribution. These factors will affect the effect of watermark injection, because the existing global watermark injection method does not fully consider the heterogeneity of each client. In order to improve the robustness and effectiveness of watermarks, it is necessary to develop new watermark injection methods that can adapt to the data heterogeneity of different hospital clients and ensure model security and copyright protection in the medical federated learning environment.

[0003] When applying federated learning in medical systems, the use of model watermarking technology can effectively protect the intellectual property rights of the model and enhance the security of the system. The mainstream medical model watermarking methods can be divided into black-box watermarking and white-box watermarking. Among them, black-box watermarking allows the watermark in the model to be verified by inputting specific triggers in the medical federated learning environment without accessing the internal structure of the model. This method adds specific trigger data and target labels during the training process of the medical task model so that the model outputs specific results when the trigger data is input. For example, when the model is suspected of being stolen, the hospital can verify the watermark by inputting these trigger data to check whether the model outputs the expected results. The advantage of black-box watermarking is that it does not require access to the internal parameters of the model, so it can adapt to different model structures and is more friendly to the privacy protection of the model. White-box watermarking requires access to the internal structure of the medical task model, such as model parameters, gradients, etc., to embed and detect watermarks. In medical federated learning, white-box watermarking embeds identifiable watermark information by modifying the internal parameters or structure of the model. This involves adding special structural noise or identification to specific layers of the model to embed watermark information into the model parameters. The embedding process of white-box watermark is relatively complicated and needs to take into account the structure of the model. In addition, the internal parameters of the model need to be accessed during the verification process, which poses certain risks in terms of medical data protection.

[0004] In summary, in the federated learning method of the medical system, some existing model watermark embedding methods attempt to evenly distribute watermark tasks to different hospital clients to achieve watermark embedding, but this method does not fully consider the heterogeneity between clients caused by non-independent and identically distributed data, which will lead to inconsistency between watermarks aggregated by the central server between different clients, affecting the reliability of the watermark. Other methods only complete the watermark task on the central server side. Although such methods are centralized, they fail to effectively utilize the computing resources of each hospital client, increase the burden on the server side, and prolong the training time of the overall model. At the same time, due to the coupling between the watermark task and the main task of the medical model, such centralized watermark embedding methods will have a negative impact on the accuracy of the model in the main tasks such as disease diagnosis.

[0005] It should be noted that this background technology is only used to introduce the relevant information of the present invention to help understand the technical solution of the present invention, but it does not mean that the relevant information is necessarily the prior art. If there is no evidence that the relevant information has been disclosed before the application date of the present invention, the relevant information shall not be regarded as the prior art. Summary of the invention

[0006] Therefore, the purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide a medical system federated learning method based on watermark embedding and a medical system

[0007] The objective of the present invention is achieved through the following technical solutions:

[0008] According to a first aspect of the present invention, a medical system federated learning method based on watermark embedding is proposed, wherein the medical system includes multiple medical units and a medical center, wherein each medical unit is a client, and the medical center is configured with a central server, and the method includes: step S1, performing watermark design according to the medical data type of the client to obtain a watermark task data set, the watermark task data set includes multiple watermark samples, and all watermark samples are configured with the same watermark label; step S2, obtaining an initial medical task model and a main task medical data set for each client, wherein the main task medical data set includes multiple main task data samples configured with disease labels, and combining the main task medical data set with the watermark task data set. The sets are combined to obtain the training set of each client; step S3, performing the first stage of training, including allowing each client to train its own initial model for a preset number of times and calculate its own watermark depth according to its own training set and the preset loss function and the preset initial proportion of the main task and the watermark task; step S4, allowing each client to reconfigure the proportion of its main task and watermark task based on its own watermark depth and adjust its preset loss function based on the configured proportion, wherein the deeper the watermark depth, the higher the proportion of the watermark task in the client; step S5, performing the second stage of training, performing multiple rounds of federated training based on the adjusted loss function of each client so that each client can obtain a medical task model.

[0009] Preferably, in step S2, the watermark depth of each client is calculated in the following manner; step S31, the intermediate layer output of each client model after the first stage training is obtained, wherein the intermediate layer output of each client model includes the predicted score of each disease label in the main task medical data set of the client; step S32, the watermark depth of each client is determined based on the intermediate layer output of each client model in the following manner:

[0010]

[0011] Among them, WDS represents the watermark depth of the client, z1 represents the maximum prediction score in the intermediate layer output of the client model, z2 represents the second largest prediction score in the intermediate layer output of the client model, and H(z) represents the entropy of the intermediate layer output of the client model.

[0012] Preferably, the entropy of the intermediate layer output of each client model is determined as follows:

[0013]

[0014]

[0015] Among them, H(z) represents the entropy of the client's intermediate layer output, C represents the total number of disease label types in the client's main task medical dataset, and p a represents the probability corresponding to the a-th disease label in the client's training set, z a 、z b They represent the prediction scores corresponding to the a-th disease label and the b-th disease label in the client's training set, respectively.

[0016] Preferably, the preset loss function of each client is:

[0017]

[0018] in, Indicates the client's main task loss, Indicates the watermark task loss of the client, represents the cross entropy loss function, α represents the main task proportion, β represents the watermark task proportion, N represents the number of samples in the client's main task medical dataset, M represents the number of watermark samples in the client's watermark training set, and y (i)′ represents the client's predicted disease label for the i-th main task medical sample in its main task medical dataset, y (i) represents the true disease label of the i-th main task medical sample in the main task medical dataset, y t (j)′ represents the client's predicted watermark label for the jth watermark sample in its watermark training set, y t Represents the true watermark label of the watermark sample.

[0019] Preferably, the initial weight of the main task is configured to be 0.5, and the initial weight of the watermark task is configured to be 0.5.

[0020] Preferably, the preset loss function in each client is adjusted in the following manner: the main task proportion in the preset loss function of the client is configured to be 0.5; based on the watermark depth of the client, the watermark task proportion in the preset loss function of the client is configured in the following manner:

[0021]

[0022] Among them, β k Indicates the watermark task ratio of the current client, WDS k represents the watermark depth of the current client, λ rp Indicates the preset reward and punishment coefficient, WDS max Indicates the maximum watermark depth of all clients, WDS min Indicates the minimum watermark depth of all clients.

[0023] Preferably, in step S5, the FedAvg algorithm is used to perform multiple rounds of federated learning.

[0024] According to a second aspect of the present invention, a medical system is proposed, which includes multiple medical units and a medical center. The medical system is configured to adopt any method as described in the first aspect of the present invention to perform federated learning to obtain a medical task model.

[0025] According to a third aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored. The computer program can be executed by a processor to implement the steps of any one of the methods described in the first aspect of the present invention.

[0026] According to a fourth aspect of the present invention, an electronic device is proposed, comprising: one or more processors; and a memory, wherein the memory is used to store executable instructions; and the one or more processors are configured to implement the steps of any one of the methods described in the first aspect of the present invention by executing the executable instructions.

[0027] Compared with the prior art, the advantages of the present invention are:

[0028] The present invention adopts the method of embedding watermarks by the client executing the watermark task in the medical system and then aggregating them through the central server. The complete watermark is embedded through the collaboration of the central server and the client in the medical system, so as to achieve the concealment and accuracy of the watermark. At the same time, based on the watermark task adaptation capability of each client in the medical system, the proportion of the watermark task to the main task of each client is adjusted, and the resources of each client are fully utilized to realize the decoupling of the main task and the watermark task, thereby ensuring the accuracy of the final medical task model and the reliability of the embedded watermark. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The embodiments of the present invention are further described below with reference to the accompanying drawings, in which:

[0030] Figure 1 A schematic diagram of the steps of a medical system federated learning method based on watermark embedding according to an embodiment of the present invention;

[0031] Figure 2 A schematic diagram of the first stage training of a medical system federated learning method based on watermark embedding according to an embodiment of the present invention;

[0032] Figure 3 Schematic diagram of the second stage training of a medical system federated learning method based on watermark embedding according to an embodiment of the present invention. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail by specific embodiments below. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0034] As mentioned in the background technology section, in the federated learning method of the medical system, some existing model watermark embedding methods attempt to evenly distribute watermark tasks to different hospital clients to achieve watermark embedding, but this method does not fully consider the heterogeneity between clients caused by non-independent and identically distributed data, which will cause the watermarks aggregated by the central server to be inconsistent between different clients, affecting the reliability of the watermark. Other methods only complete the watermark task on the central server side. Although such methods are centralized, they fail to effectively utilize the computing resources of each hospital client, increase the burden on the server side, and prolong the training time of the overall model. At the same time, due to the coupling between the watermark task and the main task of the medical model, such centralized watermark embedding methods will have a negative impact on the accuracy of the model in main tasks such as disease diagnosis.

[0035] Before specifically introducing the embodiments of the present invention, some of the terms used therein are explained as follows:

[0036] Watermarking tasks: Watermarking tasks are related to the intellectual property protection of models. In federated learning, watermarks can be embedded into models to prove the ownership or origin of the model. Watermarking tasks involve embedding specific patterns or information during model training that can also be detected when the model is illegally copied or used. The purpose of the watermarking task is to improve the security and traceability of the model and prevent the model from being illegally copied or abused.

[0037] Main task: The main task refers to the core task in federated learning, which involves the main goal or problem. In the field of healthcare, the main task is generally disease diagnosis, and the model needs to learn from the existing patient medical data how to accurately identify and predict diseases based on the data. The sample data of the main task is used to train the model to solve specific problems such as disease classification and disease prediction.

[0038] Intermediate layer output (logit output): The intermediate layer output refers to the output of the last layer before applying the activation function in the neural network. This output is the original prediction value of the model. In the binary classification problem, the intermediate layer output is a real value, expressed as the linear score of the positive class. In the multi-classification problem, the intermediate layer output is a vector, each element corresponds to the prediction score of a category.

[0039] In order to better understand the present invention, each step is described in detail below in conjunction with specific embodiments.

[0040] In order to better understand the present invention, the inventive concept of the present invention is briefly introduced first. The present invention proposes a medical system federated learning method based on watermark embedding. The method includes five steps, such as Figure 1 As shown, the five steps shown in the figure include: step S1, designing a watermark according to the type of medical data of the client to obtain a watermark task data set, the watermark task data set includes multiple watermark samples, and all watermark samples are configured with the same watermark label; step S2, obtaining the initial medical task model and the main task medical data set of each client, the main task medical data set includes multiple main task data samples configured with disease labels, and combining the main task medical data set with the watermark task data set to obtain a training set for each client; step S3, performing the first stage of training, including allowing each client to train its own initial model for a preset number of times according to its own training set and a preset loss function and the preset initial proportion of the main task and the watermark task and calculate its own watermark depth; step S4, allowing each client to reconfigure the proportion of its main task and watermark task based on its own watermark depth and adjust its preset loss function based on the configured proportion, wherein the deeper the watermark depth, the higher the proportion of the watermark task in the client; step S5, performing the second stage of training, performing multiple rounds of federated training based on the adjusted loss function of each client so that each client obtains a medical task model.

[0041] Each step is described in detail below.

[0042] In the step S1, a watermark is designed according to the type of medical data of the client to obtain a watermark task data set, and the watermark task data set includes multiple watermark samples, and all watermark samples are configured with the same watermark label. Specifically, the watermark sample is designed based on the data set type in the client, and the watermark sample can be a text, an image, or other recognizable pattern. The data type of the watermark sample is determined based on the data set type used by the client model to ensure that the client model can recognize the watermark data set. In the medical system, the main task data samples of some clients are text data, such as blood pressure data, genomic data, drug data, etc., and the main task data samples of some clients are medical image data, such as X-rays, magnetic resonance imaging, ultrasound images, etc. Therefore, the watermark data set on each client needs to be the same as the data type of its main task data sample to ensure that each client can correctly identify the watermark sample.

[0043] In step S2, the initial medical task model and the main task medical data set of each client are obtained, wherein the main task medical data set includes a plurality of main task data samples configured with disease labels, and the main task medical data set is combined with the watermark task data set to obtain a training set for each client. According to one embodiment of the present invention, when obtaining the initial medical task model of each client, it is also necessary to initialize the training environment of each client and load the pre-trained weights of each client. Pre-loading the pre-trained weights of each client is beneficial to improving the rate of model training.

[0044] In step S3, the first stage of training is performed, including enabling each client to train its own initial model for a preset number of times and calculate its own watermark depth according to its own training set, the preset loss function, and the preset initial weights of the main task and the watermark task. Figure 2 As shown, it shows the first stage training process of each client, which includes: training the initial medical task model of each client for a preset round, and evaluating the watermark depth of each client based on the intermediate layer output of the trained medical task model, wherein the watermark depth indicates the degree of adaptation of the client to the watermark task, and the deeper the watermark depth, the higher the degree of adaptation of the client to the watermark task. It should be noted that the first stage training only conducts a few rounds of training. In the medical system, the first stage training is that each medical institution performs preset rounds of training on the initial medical task model based on the training set of the main task medical data set and the watermark data set. Based on the medical task model after the first stage training of each client (medical institution), the intermediate layer output of the model for each disease label in the main task medical data set on the client can be obtained to calculate the watermark depth of the client. The intermediate layer output refers to the output of the last layer before the activation function is applied, which represents the original prediction score of the model for each category (label). In the medical system, the intermediate layer output of each client includes the prediction score of each disease label in the main task medical data set of the client. The intermediate layer output of each client is represented by z = (z1, z2, ..., z C ), C represents the total number of disease label types in the main task medical dataset on each client.

[0045] It should be noted that there is a deviation in the amount of data owned by each client in the medical system. The medical task model trained on the client with a large amount of evenly distributed medical data contributes greatly to the global model and can play a key role in embedding watermarks in the global model. The medical task model trained on the client with a small amount of data or uneven data distribution can improve the generalization ability of the global model, but the watermarks embedded by these clients will be covered or distorted by the watermarks embedded by other clients during the aggregation process. Therefore, it is very important to divide the proportion of the main task and the watermark task according to the degree of adaptation of each client of the medical system to the watermark task. Therefore, the present invention calculates the watermark depth of each client based on the predicted score of each disease label in the main task medical data set of the client based on the medical task model on each client to evaluate the degree of adaptation of each client to the watermark task, and configures the proportion of the main task and the watermark task of each client according to the watermark depth of each client. According to one embodiment of the present invention, the method of obtaining the watermark depth of each client is as follows:

[0046]

[0047] Wherein, WDS represents the watermark depth of the client, z1 represents the maximum prediction score in the intermediate layer output of the client model, z2 represents the second largest prediction score in the intermediate layer output of the client model, z1-z2 represents the confidence of the intermediate layer output of the client model, H(z) represents the entropy of the intermediate layer output of the client model, where the entropy of the intermediate layer output of the client model is used to measure the degree of confusion of the prediction output of the medical task model on the client, and entropy is an indicator of uncertainty or information in information theory. The lower the entropy of the intermediate layer output, the higher the confidence of the model in a certain category when making predictions. That is to say, the medical task model with low entropy of the intermediate layer output is more confident in the classification of the input medical data sample, and at the same time, the generalization ability of the model is also improved. According to one embodiment of the present invention, the entropy of the intermediate layer output is calculated as follows:

[0048]

[0049]

[0050] Among them, H(z) represents the entropy of the client's intermediate layer output, C represents the total number of disease label types in the client's main task medical dataset, and p a represents the probability of the a-th disease label in the client's training set, z a , z b They represent the prediction scores corresponding to the a-th disease label and the b-th disease label in the client's training set, respectively.

[0051] According to one embodiment of the present invention, in step S3, the preset loss function of each client consists of two parts: main task loss and watermark task loss. The preset loss function is as follows:

[0052]

[0053] in, Indicates the client's main task loss, Indicates the watermark task loss of the client, represents the cross entropy loss function, α represents the main task proportion, β represents the watermark task proportion, N represents the number of samples in the client's main task medical dataset, M represents the number of watermark samples in the client's watermark training set, and y (i)′ represents the client's predicted disease label for the i-th main task medical sample in its main task medical dataset, y (i) represents the true disease label of the i-th main task medical sample in the main task medical dataset, y t (j)′ represents the client's predicted watermark label for the jth watermark sample in its watermark training set, y t Represents the true watermark label of the watermark sample.

[0054] In step S4, each client is made to reconfigure the proportion of its main task and watermark task based on its own watermark depth and adjust its preset loss function based on the configured proportion, wherein the deeper the watermark depth is, the higher the proportion of the watermark task is in the client. It should be noted that the client with a larger watermark depth has a stronger adaptability to the watermark task and a better effect of model watermark embedding, while the client with a smaller watermark depth has a weaker adaptability to the watermark task and a weaker effect of model watermark embedding. In the medical system, large hospitals usually have a large amount of evenly distributed data, and their watermark accommodating capacity is relatively strong, so more watermark learning tasks can be assigned to them. The data of specialized hospitals has significant label offsets and is concentrated on the patient groups of specific departments. For township hospitals with a lack of patient cases, the data volume is small and unevenly distributed. Adding a watermark task will have a greater impact on the performance of its main task, which can allow it to focus on the learning of the main task to enhance the main task generalization ability of the medical task model, thereby achieving the decoupling of the main task and the watermark task, while ensuring the accuracy of the main task and the accuracy of the watermark task. Compared with the existing method of measuring the watermark learning ability of the client by using the watermark accuracy of each client, the watermark depth measurement method of the present invention is more scientific, and the existing method of measuring the adaptation degree of the watermark according to the watermark accuracy is insufficient and unreasonable, and cannot reflect the watermark adaptation degree of each client. According to one embodiment of the present invention, in order to improve the performance of the watermark task while ensuring the accuracy of the main task, the present invention adjusts the proportion of the watermark task to the main task of each client based on the watermark depth of each client, so that the client with a shallower watermark focuses on the learning of the main task, and the client with a deeper watermark focuses on the deepening of the watermark task. The present invention adjusts the proportion of the watermark task to the main task by adjusting the ratio of the main task loss to the watermark task loss in the preset loss function (the task proportion is reflected in the proportion coefficient of the loss function). According to one embodiment of the present invention, the main task proportion of the loss function of each client is configured to 0.5, and the watermark task proportion of the loss function of each client is configured in the following manner based on the watermark depth of the client:

[0055]

[0056] Among them, β k Indicates the watermark task proportion of the kth client, WDS k represents the watermark depth of the kth client, λ rp Indicates the preset reward and punishment coefficient, which is generally set to 20% of the total global injection. max Indicates the maximum watermark depth of all clients, WDS min Indicates the minimum watermark depth of all clients.

[0057] In step S5, the second stage of training is performed, and multiple rounds of federated training are performed based on the adjusted loss function of each client so that each client obtains a medical task model. Figure 3 As shown, the figure shows the process of the second stage training, which includes training rounds, and each training round includes: training the medical task model on each client to convergence, and then sending the parameters of the medical task model on each client to the central server, and finally the central server aggregates these parameters to build a global model and distributes the global model to each client. Among them, the present invention uses the FedAvg algorithm to aggregate the parameters or updates of the medical task model on each client, so that the watermark task in the deep watermark-adapted client affects other clients through global aggregation, and allows the shallow watermark client to focus on the main task learning to improve the accuracy of the main task, so as to give play to the collaborative effect of each client. The FedAvg algorithm is a well-known technology for those skilled in the art and will not be described in detail here.

[0058] It should be noted that due to the differences in medical equipment and diagnostic methods of various medical institutions in the medical system, and the differences in patient groups and medical technologies of each medical institution, the distribution of main task medical data in each medical institution is also heterogeneous. Taking medical imaging as an example, large medical institutions have a complete medical system with a large number of medical images with uniform disease types; the data of specialized hospitals have significant deviations and are concentrated on medical images of specific departments; there is a shortage of patient cases in township hospitals, and the amount of medical imaging data is small and unevenly distributed. In order to more clearly understand the scheme of the present invention, the following takes the case where the type of medical data is medical imaging as an example to illustrate the steps of a medical system federated learning method based on watermark embedding proposed by the present invention. The first step is to design a watermark to obtain a watermark task dataset. Since the type of medical data is medical images, the watermark task dataset obtained is multiple image watermark samples with the same watermark label to ensure that the model of each client in the medical system can recognize the watermark samples; the second step is to obtain the initial medical image classification model and the main task medical image dataset of each client, and combine the main task medical image dataset with the watermark task dataset to obtain the training set of each client; the third step is to perform the first stage of training, including allowing each client to train its own initial medical image classification model for a preset number of times and calculate its own watermark depth according to its own training set and a preset loss function with a main task weight of 0.5 and a watermark task weight of 0.5. Since the distribution of medical images in medical institutions in the medical system is heterogeneous, the first stage of training is performed with fewer rounds of training to obtain the watermark depth of each medical institution (client). The adaptation degree of the watermark task; the fourth step, after obtaining the watermark depth of each client, each client reconfigures the proportion of its main task and watermark task based on its own watermark depth. The deeper the watermark depth, the higher the proportion of the watermark task in the client, and adjusts the preset loss function on each client based on the configured proportion; the fifth step, conducts the second stage of training, and conducts multiple rounds of federated training based on the adjusted loss function of each client so that each client obtains a medical image classification model. In the second stage of training, each client is trained according to the proportion of the main task and watermark task configured according to its own watermark depth. The training tasks are allocated according to the characteristics of large medical institutions, specialized hospitals, and township hospitals to make full use of the resources of each medical institution, realizing the decoupling of the medical image classification model training task (main task) and the watermark task of the medical system, thereby ensuring the accuracy of the medical image classification model and the reliability of the embedded watermark.

[0059] It should be understood that the medical system federated learning method based on watermark embedding proposed in the present invention is not only applicable to medical systems, but also to other federated systems similar to medical systems that have heterogeneity in client data sets.

[0060] According to one embodiment of the present invention, the present invention also proposes a medical system, which includes multiple medical units and a medical center, wherein each medical unit acts as a client in a federated learning framework, and the medical center server acts as a central server in the federated learning framework, and the medical system is configured to adopt the medical system federated learning method based on watermark embedding to perform federated learning so that each client obtains a medical image classification model.

[0061] In the above medical system, each medical institution allocates the proportion of its main task and watermark task based on its own watermark depth, and fully utilizes the resources of each medical institution in the medical system during the training process, realizing the decoupling of the main task and the watermark task, thereby improving the accuracy of the main task and the reliability of the embedded watermark.

[0062] It should be noted that although the above describes the various steps in a specific order, it does not mean that the various steps must be executed in the above specific order. In fact, some of these steps can be executed concurrently or even in a different order as long as the required functions can be achieved.

[0063] The present invention may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present invention.

[0064] Computer readable storage medium can be a tangible device that holds and stores instructions used by an instruction execution device. Computer readable storage medium can include, for example, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a protruding structure in a groove on which instructions are stored, and any suitable combination thereof.

[0065] The embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A medical system federated learning method based on watermark embedding, wherein the medical system includes multiple medical units and a medical center, wherein: Each medical unit is a client, and the medical center is equipped with a central server. The method includes: Step S1, designing a watermark according to the medical data type of the client to obtain a watermark task data set, the watermark task data set includes multiple watermark samples, and all watermark samples are configured with the same watermark label; Step S2, obtaining an initial medical task model and a main task medical data set of each client, wherein the main task medical data set includes a plurality of main task data samples configured with disease labels, and combining the main task medical data set with the watermark task data set to obtain a training set for each client; Step S3, performing the first stage of training, including enabling each client to train its own initial model for a preset number of times and calculate its own watermark depth according to its own training set and the preset loss function and the preset initial weights of the main task and the watermark task; Step S4, enabling each client to reconfigure the weights of its main task and watermark task based on its own watermark depth and adjust its preset loss function based on the configured weights, wherein the deeper the watermark depth, the higher the weight of the watermark task in the client; Step S5: perform the second stage of training, and perform multiple rounds of federated training based on the adjusted loss function of each client so that each client obtains a medical task model.

2. The method according to claim 1, characterized in that In step S2, the watermark depth of each client is calculated as follows; Step S31, obtaining the intermediate layer output of each client model after the first stage of training, wherein the intermediate layer output of each client model includes the predicted score of each disease label in the main task medical data set of the client; Step S32: Determine the watermark depth of each client based on the intermediate layer output of each client model in the following manner: Among them, WDS represents the watermark depth of the client, z1 represents the maximum prediction score in the intermediate layer output of the client model, z2 represents the second largest prediction score in the intermediate layer output of the client model, and H(z) represents the entropy of the intermediate layer output of the client model.

3. The method according to claim 2, characterized in that The entropy of the intermediate layer output of each client model is determined as follows: Among them, H(z) represents the entropy of the client's intermediate layer output, C represents the total number of disease label types in the client's main task medical dataset, and p a represents the probability corresponding to the a-th disease label in the client's training set, z a 、z b They represent the prediction scores corresponding to the a-th disease label and the b-th disease label in the client's training set, respectively.

4. The method according to claim 1, characterized in that: The default loss function for each client is: in, Indicates the client's main task loss, Indicates the watermark task loss of the client, represents the cross entropy loss function, α represents the main task proportion, β represents the watermark task proportion, N represents the number of samples in the client's main task medical dataset, M represents the number of watermark samples in the client's watermark training set, and y (i)′ represents the client's predicted disease label for the i-th main task medical sample in its main task medical dataset, y (i) represents the true disease label of the i-th main task medical sample in the main task medical dataset, y t (j)′ represents the client's predicted watermark label for the jth watermark sample in its watermark training set, y t Represents the true watermark label of the watermark sample.

5. The method according to claim 4, characterized in that The initial weight of the main task is configured as 0.5, and the initial weight of the watermark task is configured as 0.

5.

6. The method according to claim 4, characterized in that The default loss function in each client is adjusted as follows: Set the main task ratio in the preset loss function of the client to 0.5; Based on the watermark depth of the client, the watermark task weight in the preset loss function of the client is configured as follows: Among them, β k Indicates the watermark task ratio of the current client, WDS k represents the watermark depth of the current client, λ rp Indicates the preset reward and punishment coefficient, WDS max Indicates the maximum watermark depth of all clients, WDS min Indicates the minimum watermark depth of all clients.

7. The method according to claim 1, characterized in that In step S5, the FedAvg algorithm is used to perform multiple rounds of federated learning.

8. A medical system, comprising a plurality of medical units and a medical center, characterized in that: The medical system is configured to perform federated learning using the method described in any one of claims 1 to 7 to obtain a medical task model.

9. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program can be executed by a processor to implement the steps of the method according to any one of claims 1 to 7.

10. An electronic device, characterized in that: include: one or more processors; as well as A memory, wherein the memory is used to store executable instructions; The one or more processors are configured to implement the steps of the method of any one of claims 1 to 7 by executing the executable instructions.

Images