A method for constructing a federated medical system with a model watermark and a federated medical system

Abstract

The application provides a federal medical system construction method with a model watermark, comprising the following steps: S1, constructing a benchmark medical task model for each client; S2, constructing a difficult sample set on each client; S3, constructing a watermark generator for each client; S4, performing multiple rounds of second-stage federal learning until convergence and deploying the final converged medical task model to each client, wherein each round of second-stage federal learning comprises the following steps: generating a watermark sample set based on the difficult sample set by using the watermark generator on each client, optimizing the parameters of the generator, uploading the shared backbone network parameters in the optimized generator to the server; training the medical task model after the last round of federal learning based on the current round of data set on each client until convergence, optimizing the parameters of the medical task model, and uploading the converged medical task model parameters to the server; and the server aggregates the medical model parameters uploaded by each client.

Description

Technical Field

[0001] This invention relates to the field of medical artificial intelligence technology, specifically to the application of federated learning and model watermarking in the field of medical artificial intelligence, and more specifically, to a method for constructing a federated healthcare system with personalized and harmless model watermarking and a federated healthcare system. Background Technology

[0002] With the rapid development of artificial intelligence and machine learning technologies, data-driven intelligent models are playing a vital role in all areas of social production and daily life. Whether in medical diagnosis, financial risk control, intelligent manufacturing, or intelligent services such as speech recognition and image analysis, machine learning models have greatly improved decision-making efficiency and prediction accuracy, driving the construction of the digital economy and intelligent society. However, as the scale of models continues to expand and application scenarios become increasingly complex, people are gradually realizing the need for reasonable management of model development, use, and distribution, ensuring that while creating value, they still adhere to legal and ethical principles such as privacy protection, fairness, and security.

[0003] Currently, machine learning systems applied across various fields face three core requirements. First, machine learning systems are high-value knowledge collections trained using application-domain data and algorithms, containing the developer's core technologies and experience. They must be protected from illegal copying, dissemination, or redistribution to safeguard their intellectual property and commercial interests. Second, the application-domain data upon which machine learning systems rely for training is highly private and sensitive. As a crucial production factor, data requires strict privacy protection and access control during use, especially in the federal healthcare field. Third, the outputs of machine learning systems in various application domains often influence actual decision-making in critical scenarios. Therefore, the interpretability, reliability, and security of these systems must be audited and managed to ensure the fairness and credibility of their results.

[0004] In protecting data privacy and sensitivity in application domains, federated systems are an effective means. As a distributed machine learning framework, they aim to protect data privacy by training on local client data and uploading model parameters or gradients to the server. The core concept is "the model moves as the data doesn't move." However, during federated training, client data and computing power are abstracted into the model. As a collection of data with higher value, the lack of effective means to verify ownership when the model is stolen has become a major challenge limiting the development of federated systems in various fields.

[0005] Against this backdrop, model copyright protection in various application domains has gradually become an important research direction in machine learning, especially federated learning governance. Model copyright protection aims to verify whether a machine learning model in a specific domain belongs to the party claiming the right to own the model. To address this issue, academia and industry have proposed various model watermarking methods, aiming to embed verifiable identifier information into the model, enabling the rights holder to verify in a controlled manner whether the model contains its embedded identifier when needed. Based on different access and verification permissions, common watermarking embedding methods can be broadly divided into white-box watermarking and black-box watermarking: white-box watermarking typically embeds a hidden signature in the model weights or internal features; verification requires access to the model's internal parameters and extraction of the specified signature identifier using matrix multiplication and transformations. Black-box watermarking, on the other hand, constructs specific trigger samples and corresponding labels, causing the model to give specific outputs to these trigger samples, thus completing verification without accessing internal parameters. Within the federated learning framework, existing watermarking technologies can be broadly divided into server-side watermark injection and client-side watermark injection. Server-side methods, such as WAFFLE, involve the server independently using the watermarked data to perform additional training after the model is uploaded to each application client, achieving model identification without accessing the original data. However, it does not include samples from the main task during the watermark training phase, which can easily lead to "catastrophic forgetting" of the main task. While the FedTracker method mitigates this problem through continuous learning, it is still limited by the distribution bias of the server-side watermarked data, affecting the performance of the main task. Client-side methods, such as FedIPR, use decodable transformations to embed encoded information in the model layer to achieve client-level verification, but verification requires access to the complete model parameters, limiting its practicality. Some methods attempt to construct trigger samples to directly inject watermarks on the client side, but these watermark samples are still out-of-distribution noise, which may compromise the decision boundaries of high-reliability tasks.

[0006] In general, both types of methods have their advantages and disadvantages: white-box methods typically perform better in verifying reliability, but often lack access to the model's internal mechanisms in real-world scenarios; black-box methods are more adaptable and can be verified externally, but their implementation often relies on label tampering or the insertion of anomalous samples (i.e., "label tampering / triggering" watermarks). These practices are essentially out-of-distribution noise, easily introducing spurious correlations or disrupting the model's original discrimination boundaries during training, thus causing significant damage to the performance of the main task. This problem is further amplified in federal healthcare scenarios. First, medical datasets often exhibit uneven class distribution, limited sample size, and cross-institutional distribution differences, making any form of out-of-distribution perturbation more likely to impair the model's generalization performance on real clinical samples. Second, the need for independence and accountability among healthcare institutions requires embedded watermarks to support independent client-level verification (i.e., each hospital can individually prove its contribution to the model or ownership of the watermark), rather than merely providing a coarse-grained identification of the overall model.

[0007] In existing federated watermarking research, from the perspective of protecting model privacy, many methods still follow the black-box approach of label tampering or rely on out-of-distribution trigger samples. This exhibits two prominent shortcomings in a federated healthcare environment: First, it interferes with the main task. Label tampering or strong perturbations can change the decision boundaries learned by the healthcare model, leading to a decline in the performance of the entire or some healthcare client systems. Second, there is mutual interference between watermarks. When multiple clients concurrently embed different watermarks, there is a lack of mechanisms to ensure that each client's watermark can still be individually identified in the aggregated global model, resulting in weakened or conflicting watermark signals. In summary, current technology cannot simultaneously meet the requirements of harmless main task performance and independent client verification in federated healthcare scenarios.

[0008] Therefore, it is necessary to propose a harmless, personalized, and independently verifiable model watermark embedding method and system suitable for federal healthcare environments. This method should not only maintain the original performance of the main task on each healthcare institution's client side, but also provide quantifiable and auditable attribution proof for each participant in the event of model leakage or misuse.

[0009] It should be noted that the background information presented here is only for illustrating relevant information about the present invention to aid in understanding the technical solution of the present invention, and does not imply that the relevant information is necessarily prior art. The relevant information was submitted and disclosed together with the present invention, and should not be considered prior art unless there is evidence that the relevant information was disclosed before the filing date of the present invention. Summary of the Invention

[0010] Therefore, the purpose of this invention is to overcome the shortcomings of the prior art and provide a method for constructing a federal healthcare system with model watermarking, as well as a federal healthcare system.

[0011] The objective of this invention is achieved through the following technical solution:

[0012] According to a first aspect of the present invention, a method for constructing a federal healthcare system with model watermarking is provided. The federal healthcare system includes a server and multiple clients, each client being deployed in a healthcare institution. Each healthcare institution contains its own set of original medical data samples, and each client is configured with an initial healthcare task model. The method includes: S1, performing multiple rounds of first-order federated learning on the original medical data sample set as input and predicted category as output until convergence, thereby constructing a baseline healthcare task model for each client; S2, selecting multiple difficult samples from their respective original medical data sample sets in a preset manner to construct a difficult sample set, wherein the difficult sample is a medical data sample whose inference difficulty for the client's baseline healthcare task model meets a preset requirement; S3, constructing a watermark generator for each client, the watermark generator including a shared backbone network and a client-specific perturbation module, wherein the client-specific perturbation modules in each client are different, wherein: the shared backbone network is used to extract the basic features of the difficult samples; the client-specific perturbation module is used to add perturbations to the basic features of the difficult samples and generate a watermark model that matches the difficult sample's characteristics. S4. Perform multiple rounds of second-stage federated learning until convergence and deploy the finally converged medical task model to each client. Each round of second-stage federated learning includes: on each client, using the watermark generator from the previous round of federated learning, generating the current round's watermark sample set based on the hard sample set; optimizing the generator parameters based on the hard sample set and the current round's watermark sample set using a preset generator loss function; and uploading the optimized shared backbone network parameters in the generator to the server; on each client, constructing the current round's dataset based on the current round's watermark sample set and the original medical data sample set, and using it to train the medical task model from the previous round of federated learning until convergence; optimizing the medical task model parameters based on the classification loss of the main task; and uploading the converged medical task model parameters to the server; the server aggregates the medical model parameters uploaded by each client to obtain a fused medical task model and distributes the fused medical task model parameters to each client; and aggregates the shared backbone network parameters uploaded by each client to obtain a fused shared backbone network and distributes the fused shared backbone network parameters to each client.

[0013] Preferably, the first stage of each round of federated learning includes: training the medical task model after the first stage of federated learning in the previous round to convergence on each client based on the original medical data sample set of its medical institution and uploading the medical task model parameters to the server; the server aggregates the medical model parameters uploaded by each client to obtain a fused medical task model and distributes the fused medical task model parameters to each client.

[0014] Preferably, in S2, a set of difficult samples is constructed for each client as follows: The client's current medical task model is used to infer the predicted category for each medical data sample in the original medical data sample set on the client, and the probability distribution of each category prediction corresponding to each medical data sample is predicted, and the highest category prediction probability is selected as the confidence level of that medical data sample; the gradient of the loss function with respect to each medical data sample is calculated; and the difficulty level that each medical data sample should be able to achieve is calculated as follows:

[0015]

[0016] in, Indicates medical data sample The level of difficulty Indicates medical data sample gradient, Indicates medical data sample The confidence level is determined by grouping all medical data samples in the original medical data sample set on each client according to the true category. Within each category, the medical data samples are selected in descending order of difficulty, and the top number of samples are selected to form the difficult sample set.

[0017] Preferably, on each client, a watermark generator generates the watermark sample set for the current round based on the hard sample set in the following manner: The generator uses a shared backbone network to extract the basic features of each hard sample in the client and normalizes these features; the generator's style code modulation module adds perturbation coding to each normalized basic feature to complete modulation.

[0018]

[0019]

[0020] in, Representing the basic features of hard samples, Indicates the first Perturbation coding for each client, and They are respectively calculating the mean and standard deviation for each feature channel. It is a numerical stability constant. and These represent the fully connected transforms that map client perturbation coding to channel scaling and translation parameters, respectively; and the transformation of the modulated features of each hard sample to the frequency domain:

[0021]

[0022] in, This represents a two-dimensional Fast Fourier Transform; a frequency-domain gating matrix is ​​introduced for the features transformed to the frequency domain for each difficult sample:

[0023]

[0024] in, Indicates the first A client-private or shared low-frequency gating mask is used; the hard sample features with the frequency domain gating matrix are inversely transformed back to the spatial domain, and a convolutional encoder is used to obtain the corresponding pixel-level perturbation based on the hard sample features in the inverse transformation back to the spatial domain; the pixel-level perturbation corresponding to the hard sample is added to the hard sample to obtain the watermark sample corresponding to the hard sample, and all the watermark samples constitute the watermark sample set.

[0025] Preferably, the preset generator loss function is:

[0026]

[0027] in, This represents the reconstruction loss between watermarked samples and difficult samples. This represents the mutual information loss between hard samples and watermarked samples. This represents the classification consistency loss between hard samples and watermarked samples. , , They are respectively , , The weighting coefficients, where:

[0028]

[0029]

[0030]

[0031] in, Indicates the number of difficult samples. Indicates the first A difficult sample, Indicates the relationship with the first The first difficult sample corresponds to the first... One watermark sample, Indicates the first The feature vectors of a difficult sample in the shared backbone feature space. Indicates the first The feature vectors of the watermarked samples in the shared backbone feature space, where cos represents the cosine similarity. This represents a generator composed of a shared backbone network and client-side private perturbation modules. This represents the generator's output for the watermark sample. This represents the generator's output for difficult samples. It is logistic regression. It is divergence.

[0032] Preferably, the classification loss is:

[0033]

[0034] in, Indicates the first The main task classification cross-entropy loss on each client, Indicates the first Data sets on each client, Represents the dataset The Middle One sample, It is the first The category label of each sample, Indicates the first Medical task model parameters on each client, Indicates the first Medical task models on individual clients for samples The predicted output, Indicates sample The predicted loss.

[0035] Preferably, during the second stage of federated training, the training convergence is determined when it is verified that a watermark is embedded in the medical task model. Specifically, the watermark is determined to be embedded in the medical task model as follows: when the difference between the prediction success rate of the medical task model on the watermarked sample set and the prediction success rate of the basic medical task model on the watermarked sample set is greater than or equal to a preset threshold, the medical task model is determined to have embedded a watermark.

[0036] According to a second aspect of the present invention, a model-watermarked federal healthcare system based on the method described in the first aspect of the present invention is provided. The system includes a server and multiple clients, wherein: each client is deployed in a healthcare institution, each healthcare institution contains its own set of original medical data samples, each client is configured with a medical task model constructed using the method described in the first aspect of the present invention, wherein each client incrementally trains its own medical task model to update the medical task model when new medical data samples are added, and uploads the updated medical task model parameters to the server; the server is used to aggregate the updated medical task model parameters uploaded by each client to obtain aggregated medical tasks in parallel and distribute the aggregated medical task model parameters to each client.

[0037] Compared with existing technologies, this invention provides a label-consistent and harmless watermark generation mechanism for federal healthcare scenarios. The method extracts a unified semantic representation through a shared feature backbone network, and then filters difficult samples near the task boundary from local client data to generate watermark samples that are consistent with the original label and contain only subtle perturbations. This mechanism achieves watermark embedding without introducing additional noise or changing the task label, allowing the model to learn watermark features without deviating from the optimization direction of the main task, thus significantly reducing the performance loss of the main task and meeting the "harmless embedding" requirements of high-precision scenarios such as healthcare. The watermark generator structure of this invention, consisting of a "shared backbone network + client-specific perturbation module," achieves distinguishable and stable personalized watermark embedding. The shared backbone provides a unified semantic foundation for watermark generation across all clients, while the client-specific perturbation module differentiates the shared features, ensuring that each client-generated watermark sample has a unique and stable feature expression. By combining the frequency domain amplitude constraint mechanism, the impact of perturbation on the feature space of the main task can be further limited, so that the watermarks of different clients remain independent and do not interfere with each other in the multi-round federated aggregation process, avoiding confusion in the federated aggregation, thereby realizing the client-distinguished and verifiable watermark tracking capability, and meeting the actual needs of model source identification and copyright protection in multi-center medical federated learning.

[0038] Compared with existing technologies, this invention not only effectively avoids the decline in accuracy of the main task in medical scenarios, but also supports independent verification of the contributions and sources of different client models, providing a more reliable and scalable technical approach for multi-center collaboration, especially for model copyright protection in medical scenarios. Furthermore, this invention significantly reduces the computational and communication overhead of the verification process while ensuring the accuracy of watermark verification. Attached Figure Description

[0039] The embodiments of the present invention will be further described below with reference to the accompanying drawings, wherein:

[0040] Figure 1 This is a schematic diagram of the construction process of a federal healthcare system according to an embodiment of the present invention. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative and are not intended to limit the invention.

[0042] As mentioned in the background section, existing federated watermarking research, from the perspective of protecting model privacy, still largely relies on black-box label tampering or out-of-distribution trigger samples. This exhibits two significant shortcomings in a federated healthcare environment: first, it interferes with the main task; label tampering or strong perturbations alter the decision boundaries learned by the healthcare model, leading to a decline in the performance of the entire or some healthcare client systems; second, watermarks interfere with each other. When multiple clients concurrently embed different watermarks, there is a lack of mechanisms to ensure that each client's watermark can still be individually identified in the aggregated global model, resulting in weakened or conflicting watermark signals. In summary, current technology cannot simultaneously meet the requirements of harmless main task performance and independent client verification in federated healthcare scenarios.

[0043] To better understand the present invention, before describing the solution of the present invention, we will first analyze in detail the defects of the solutions in the prior art.

[0044] In existing federated learning model watermarking and copyright protection techniques, the relevant methods can be broadly classified into three categories based on the watermark injection location and the executing entity: one category injects watermarks into the aggregated model on the server side; another category embeds independently verifiable parameter-level watermarks into the local model on the client side; and the third category improves the watermark injection strategy on the client side to alleviate the performance degradation of the main task. Although the above methods have achieved watermark embedding and detection in federated models to varying degrees, they still have significant shortcomings in maintaining the performance of the main task, watermark stability, and multi-client collaborative training. Among them, the first type of approach has a drawback: server-side watermark injection methods (such as WAFFLE) typically have the server independently perform watermark fine-tuning after the model is uploaded by the client. Although this approach avoids direct access to the client's original data, because the server only uses the watermark data for fine-tuning, the distribution of the watermark samples differs significantly from the original task data. This causes the model to deviate from the original task's feature space after fine-tuning, resulting in a decrease in the accuracy of the main task. Furthermore, the watermark learning process in this method is independent of the federated training process, lacking awareness of the feature distribution of the client model, which can easily lead to the watermark being weakened or invalidated during the aggregation process. The second type of approach suffers from the following drawbacks: Client-side watermark injection methods (such as FedIPR) embed specific binary encoded information into the local model, enabling the client to independently complete watermark verification. However, these methods require accessing or decoding all model parameters during verification, increasing the computational and communication burden. Simultaneously, the embedded structure is unrelated to the main task's learning objective, failing to consider the differences in data distribution and inconsistent model update directions among different clients, resulting in a cumulative interference effect after global aggregation, affecting the performance of the main task and the stability of the model. The third type of approach suffers from the following drawbacks: This type of approach is an improvement on client-side watermark injection, attempting to alleviate the problem of decreased accuracy in the main task by optimizing the watermark injection strategy. However, the watermarked samples generated by this type of approach are essentially still out-of-distribution noise data, significantly deviating from the feature space of the main task. This type of watermark can guide the model gradient update direction away from the optimization objective of the main task, leading to a decrease in training convergence speed and further exacerbating the model performance degradation problem during multiple rounds of federated aggregation.

[0045] Therefore, none of the three methods mentioned above can meet the requirements of harmlessness of the main task and independent verification by the client in the context of federal healthcare. Research on copyright protection of existing federated learning models reveals that most existing watermarking methods rely on "incorrect labels" to construct watermarked samples, which is essentially noise interference detached from the semantic distribution of the main task. This type of watermarking induces the model to generate specific outputs on non-real labels to embed identification information. While it has high detectability, it significantly deviates from the normal feature distribution of the model in the real task, disrupting the model's stable convergence process. Especially in applications such as medical imaging where the reliability and accuracy of the model are extremely important, this interference can easily lead to the underutilization of limited labeled data, resulting in a decline in the performance of the main task and failing to meet the safety and stability requirements of actual clinical applications.

[0046] The purpose of this invention is to address the following issues: Existing federated watermarking methods generally rely on samples with tampered or abnormal labels, leading to inconsistencies between the watermarking task and the main task labels. This causes the watermarked samples to behave as noise signals during training, resulting in global model gradient shifts during federated aggregation and thus reducing the accuracy of the main task. This problem is particularly pronounced in applications such as medical imaging, where diagnostic accuracy and model stability are extremely critical, further amplifying the risk of model performance degradation and making it difficult to meet practical application needs. Furthermore, in federated scenarios, watermarks from multiple clients can overlap or interfere with each other in the global model, easily reducing the reliability of copyright verification and increasing the impact on the model's main task. Therefore, this invention proposes a new technical approach: watermarked samples should originate from the actual data distribution within the medical institution's client, rather than relying on erroneous labels or abnormal noise. This ensures semantic rationality while achieving harmless embedding into the main task. The inventors discovered in their research that each medical institution's client's local data contains a portion of "marginal samples where the model is prone to errors." These samples are usually located near the original decision boundary. The predictions of medical task models for them are the most unstable and most susceptible to perturbations, thus best reflecting the model's fine-grained recognition capabilities.

[0047] Based on this idea, this invention proposes a personalized, harmless model watermarking mechanism for federal healthcare scenarios. By automatically screening such "misjudgment-prone samples" on the client side of the medical institution, and constructing watermarks based on these samples, special samples that are consistent with the real labels but have a certain difficulty in distinguishing them for the medical task model are generated. This allows the medical task model to naturally form verifiable watermark behavior during the learning process of the main task, thereby achieving stable and detectable watermark embedding without damaging the performance of the main task. In order to make the watermark both distinguishable and not harmful to the main task, the system introduces a watermark generator network to convert these "misjudgment-prone samples" into a set of "harmless watermark samples". These watermark samples have the following core characteristics: (1) For the original medical task model without embedded watermark, it still has a high probability of misjudgment; (2) For the medical task model trained with watermark, it will be stably and accurately identified, thereby naturally forming a verifiable watermark behavior.

[0048] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0049] According to an embodiment of the present invention, the present invention provides a method for constructing a federal healthcare system with model watermarking. The federal healthcare system includes a server and multiple clients, each client being deployed in a medical institution. Each medical institution contains its own set of original medical data samples, and each client is configured with an initial medical task model. The method includes: S1, using medical data samples as input and predicted categories as output, performing multiple rounds of first-order federated learning on the original medical data sample set until convergence to construct a baseline medical task model for each client; S2, for each client, selecting multiple difficult samples from its own set of original medical data samples according to a preset method to construct a difficult sample set, wherein the difficult sample is a medical data sample whose inference difficulty for the client's baseline medical task model meets a preset requirement; S3, constructing a watermark generator for each client, the watermark generator including a shared backbone network and a client-specific perturbation module, wherein the client-specific perturbation modules in each client are different, wherein: the shared backbone network is used to extract the basic features of the difficult samples; the client-specific perturbation module is used to add perturbations to the basic features of the difficult samples and generate a watermark. S4. Perform multiple rounds of second-stage federated learning until convergence and deploy the finally converged medical task model to each client. Each round of second-stage federated learning includes: on each client, using the watermark generator from the previous round of federated learning, generating the current round's watermark sample set based on the hard sample set; optimizing the generator parameters based on the hard sample set and the current round's watermark sample set using a preset generator loss function; uploading the optimized shared backbone network parameters in the generator to the server; on each client, constructing the current round's dataset based on the current round's watermark sample set and the original medical data sample set, and using it to train the medical task model from the previous round of federated learning until convergence; optimizing the medical task model parameters based on the classification loss of the main task; and uploading the converged medical task model parameters to the server; the server aggregates the medical model parameters uploaded by each client to obtain a fused medical task model and distributes the fused medical task model parameters to each client; and aggregates the shared backbone network parameters uploaded by each client to obtain a fused shared backbone network and distributes the fused shared backbone network parameters to each client.

[0050] In summary, the present invention mainly addresses two core technical problems faced by existing federated watermarking technology in practical applications:

[0051] First, there is the issue of watermark injection damaging the performance of the main task. Existing federated watermarking methods generally rely on mislabeled or out-of-distribution samples to construct watermarks. During training, this is equivalent to injecting abnormal noise into the medical task model, causing the gradient direction to deviate from the original convergence direction. In the context of federated medical aggregation, this cumulative deviation will be further amplified, ultimately leading to a decrease in the accuracy of the main task of the global medical task model. The federated harmless watermarking mechanism proposed in this application selects samples with difficult class discrimination from within the data distribution of medical institution clients and generates watermarked samples with consistent labels on them. This ensures that the watermarking task maintains the same semantic direction as the main task, fundamentally eliminating the impact of mislabeled watermarks on the model's discrimination boundary, and achieving harmless and controllable watermark injection.

[0052] Second, there is the problem of mutual interference and indistinguishability of watermark features among multiple clients. In federated learning, the data distribution, training conditions, and generation capabilities of different clients vary significantly, making it difficult for traditional watermarking methods to guarantee the stability and distinguishability of watermarks. When multiple clients inject watermarking tasks, watermark features are easily diluted or conflicted during federated aggregation, making it impossible for the server to accurately distinguish or verify watermarks from specific clients. This application proposes a watermark generation system of "shared backbone + client-specific perturbation module." On the one hand, the shared backbone ensures semantic consistency and generation stability across clients; on the other hand, client-specific lightweight perturbation encoding generates differentiated watermarks, ensuring that watermarks from different clients still retain recognizability and traceability in federated aggregation.

[0053] As can be seen, in the solution of this invention, in order to achieve personalized support for medical institution clients, the watermark generator of each client is constructed as a structure of "shared backbone network + client private perturbation module".

[0054] The shared backbone network is used to uniformly process medical data from various healthcare clients within a federal healthcare environment. Trained through parameter aggregation, the shared backbone network can stably extract consistent high-level semantic features across clients, such as the shape, texture, contour, and key diagnostic attributes of lesions in medical images. By fixing the parameter update path of the shared backbone during training, it ensures that different clients generate harmless watermark samples based on the same semantic foundation, avoiding fluctuations in watermark sample usability and quality due to inconsistent feature extraction capabilities. This significantly improves the overall reliability of watermark embedding and verification within the federal healthcare system.

[0055] Building upon shared semantics, this invention establishes a private perturbation module for each healthcare institution client. This module generates independent perturbation codes and performs lightweight modulation on features extracted from the shared backbone network. Each client's private perturbation module utilizes a feature normalization modulation mechanism, combined with spatial and frequency domain gating strategies, to apply weak, client-unique perturbations to difficult samples. This results in watermarked samples exhibiting distinguishable, personalized, and subtle shifts while maintaining the category label and key medical semantics. Through this design, the system can achieve client-level watermark differentiation and independent verification without compromising diagnostic performance in medical tasks, providing a robust and harmless technical path for large-scale copyright protection in a federal healthcare environment.

[0056] This invention addresses two key issues by transforming traditional label-tampering watermarks into harmless watermarks with consistent labels. It combines a federated shared backbone network with a client-side private perturbation coding module: (1) the impact of watermark injection on the main task performance; and (2) the decrease in distinguishability caused by mutual interference between watermarks from multiple clients. The shared backbone provides a consistent semantic feature foundation across clients, ensuring that the watermark generation process does not compromise the main discriminative capabilities of the medical task model. The private perturbation module applies a controllable and discriminative lightweight perturbation to each medical institution client, ensuring that the final watermark expression maintains client-side uniqueness without altering the category semantics. Through this design, while ensuring the stability of the main task's accuracy, it achieves independent watermark verification capabilities at the client level, making it suitable for data-sensitive scenarios such as federated healthcare where model reliability is paramount.

[0057] The federated training process of the present invention will be described in detail below with reference to an accompanying drawing. It should be noted that because there are many formulas, variable symbols may be used repeatedly; please refer to the specific textual explanation in each formula for accurate interpretation. According to one embodiment of the present invention, as... Figure 1 As shown, steps S101, S102, S103, S104, and S105 are included. Each step is described in detail below.

[0058] In step S101, representative hard samples are selected from the local raw medical data dataset of each medical institution client to construct the subsequent domain watermark sample set. This step aims to ensure the class balance and representativeness of the samples, thereby providing high-quality raw candidate data for watermark injection.

[0059] Specifically, each medical institution's client Through the current local model Its local raw medical dataset ( Indicates the first The original medical dataset on the client side of each medical institution Indicates the first one One sample, Indicates the first Inference is performed on the labels of each sample, and the model's output is recorded for each sample. Calculate the probability distribution of its prediction:

[0060]

[0061] And take the highest class probability as the confidence level:

[0062]

[0063] in, This indicates a category index.

[0064] At the same time, calculate the loss function for difficult sample inputs. For input gradient:

[0065]

[0066] The gradient magnitude reflects the model's sensitivity to the sample, i.e., the model's response to sample perturbations. Considering both gradient strength and confidence level, the sample difficulty score is defined as:

[0067]

[0068] in, The larger the value of k, the closer the sample is to the model's decision boundary, indicating a greater challenge to the model. The samples are grouped according to their true class, and for each class k, the following steps are performed: Sort the data from highest to lowest, select the top K medical data samples to form a class-balanced difficult sample set, i.e., the [number]th [item / group]. Hard sample set of local data from individual medical institution clients :

[0069]

[0070] This set ensures balance and representativeness under multi-class data conditions, avoiding excessive concentration of high-frequency class samples, with each class containing an equal number of samples. These samples will be used in subsequent steps to generate domain watermark samples and perform model watermark embedding. By introducing a joint difficulty score based on input gradient and confidence, this invention can select samples that best reflect the characteristics of the model's decision boundary while maintaining class balance. This method avoids the bias caused by random sampling or relying solely on confidence for selection, and can effectively improve the representativeness and stability of subsequent watermark generation samples. At the same time, the class balance of difficult samples ensures the generalization ability and robustness of domain watermarks on various types of data, providing a solid foundation for the construction of harmless watermarks.

[0071] In step S102, each client, without introducing watermarked samples, uses its local real dataset to perform preliminary training on the main task model to obtain a medical task model that stably converges to the main task. Each client... Use its local dataset For model parameters Perform gradient descent and minimize the empirical loss function of the main task:

[0072]

[0073] in, Indicates the first Each client uses its own main task classification cross-entropy loss and performs gradient updates and optimization iterations based on that loss. Indicates the first Data sets on each client, Represents the dataset The Middle One sample, It is the first The category label of each sample, Indicates the first Medical task model parameters on each client, Indicates the first Medical task models on individual clients for samples The predicted output, Indicates sample The prediction loss. After multiple rounds of local training, each medical institution's client uploads the model parameters to the server, which then updates the global model using an aggregation algorithm (e.g., FedAvg).

[0074]

[0075] in, and These are all indexes from medical institution clients and have no specific meaning. express Global model parameters at time t. express Time of the first Medical task model parameters on each client.

[0076] After multiple rounds of communication, the global model achieves stable convergence on the main task, ensuring that the performance of each medical institution's client model on the original main task is fully optimized, serving as the benchmark model for subsequent watermark injection. The convergence state obtained during this training phase not only guarantees that the model has strong task representation capabilities but also provides a foundation for the next step of client adaptability evaluation, ensuring that subsequent watermark injection will not affect the performance of the main task.

[0077] In step S103, the client-side harmless watermark training phase is performed. Each client performs harmless watermark training locally to achieve personalized generation of watermark samples while ensuring that the accuracy of the main task is not significantly affected. This step includes feature extraction using a shared backbone network, regulation using a client-side private perturbation module, and watermark training, and is conducted through a two-stage optimization strategy.

[0078] Each medical institution client builds a shared backbone network for difficult samples. Extracting basic features:

[0079]

[0080] Only the backbone network parameters are shared to participate in federated aggregation, ensuring consistency of all clients in the feature space. The training of the shared backbone network is mainly used to provide stable feature representations for watermark generation.

[0081] After sharing the basic feature F of the backbone output, each client is configured with a private perturbation module, providing independent perturbation coding for each client:

[0082]

[0083] in, For the first Independent perturbation coding on each client, No. Style vectors for each client, It is the style vector dimension, and the generated perturbation code is used to generate a client-specific watermark.

[0084] Basic features After normalization, modulation is then performed using perturbation coding:

[0085]

[0086]

[0087] in, Representing the basic features of hard samples, Indicates the first Perturbation coding for each client, and They are respectively calculating the mean and standard deviation for each feature channel. It is a numerical stability constant. and These represent fully connected transformations that map client perturbation codes to channel scaling and translation parameters, respectively.

[0088] To avoid perturbations from damaging the main task features, the modulated features of each difficult sample are transformed into the frequency domain:

[0089]

[0090] in, This represents a two-dimensional Fast Fourier Transform;

[0091] To control perturbation and energy distribution, a frequency domain gating matrix is ​​introduced for the features transformed into the frequency domain for each hard sample:

[0092]

[0093] in, Indicates the first A low-frequency gating mask, private or shared by each client, is used to preserve stable low-frequency characteristics, suppress high-frequency noise, and ensure that the watermark is harmless.

[0094] The difficult sample features, which incorporate a frequency domain gating matrix, are inversely transformed back to the spatial domain:

[0095]

[0096] A convolutional encoder is used to obtain the corresponding pixel-level perturbation based on the hard sample features obtained by inverse transformation back to the spatial domain:

[0097]

[0098] Add pixel-level perturbations corresponding to the hard samples to obtain watermarked samples corresponding to the hard samples:

[0099]

[0100] All watermark samples constitute a watermark sample set.

[0101] During training, this invention adopts a two-stage training process. External optimization is applied to normal samples, while the client first optimizes the local real dataset (including the original medical dataset and the watermarked sample set, still using...) (Representation) on the main model Perform training to minimize the loss of the main task:

[0102]

[0103] Internal optimization focuses on optimizing the watermarked samples. Locally on the client, the optimization goal of the watermark generator is primarily to ensure that the generated watermarked samples maintain the classification consistency of the original samples, embed unique domain features of the client, and do not compromise the stability of shared backbone features. The optimization process is based on the following weighted loss function:

[0104]

[0105] in, This represents the reconstruction loss between watermarked samples and difficult samples. This represents the mutual information loss between hard samples and watermarked samples. This represents the classification consistency loss between hard samples and watermarked samples. , , They are respectively , , The weighting coefficients, where:

[0106]

[0107]

[0108]

[0109] in, Indicates the number of difficult samples. Indicates the first A difficult sample, Indicates the relationship with the first The first difficult sample corresponds to the first... One watermark sample, Indicates the first The feature vectors of a difficult sample in the shared backbone feature space. Indicates the first The feature vectors of the watermarked samples in the shared backbone feature space, where cos represents the cosine similarity. This represents a generator composed of a shared backbone network and client-side private perturbation modules. This represents the generator's output for the watermark sample. This represents the generator's output for difficult samples. It is logistic regression. It is divergence.

[0110] Through the above optimizations, the watermark generator aligns information in three different dimensions: the reconstruction loss constrains the generator output to remain consistent with the original sample in the input space, ensuring that the watermark sample is visually and classifiably similar to the original sample; the mutual information loss constrains the generator to share core features, enabling the watermark sample to maintain semantic relevance with the original sample in the feature space, thus ensuring the stability of the shared features; and the classification loss constrains the output of the private classification head, ensuring that the watermark sample remains consistent with the original sample in the main task classification. These three losses work synergistically to constrain the generator at different levels, ensuring that the generated watermark sample embeds unique domain features of the medical institution's client while maximizing the retention of the main task performance, achieving independent verification capabilities for watermarks from each client, and guaranteeing efficient and stable training.

[0111] In step S104, after completing the main task training and domain watermark embedding on each client, the server enters the aggregation and distribution phase. The core objective of this phase is to achieve joint updates of the main task model and the domain watermark module, thereby maintaining the independence and verifiability of client-side watermark features while ensuring global performance consistency.

[0112] The model uploaded by the client mainly includes two sets of parameters: the main task model parameters and the watermark generator shared backbone network (used to capture general features and extract stable watermark representations). Let the first... The medical task model parameters for each medical institution client are: The watermark generator shares backbone network parameters as follows: The corresponding data volume is Number of medical institution clients The server then uses a weighted average method for aggregation:

[0113]

[0114]

[0115] This aggregation process is equivalent to the FedAvg algorithm, enabling the global model to generalize well across different client data distributions. By aggregating the main task parameters, a stable improvement in classification accuracy and convergence speed of the global model can be ensured. Meanwhile, It is a cross-client shared watermark feature extractor. Its aggregation process can reduce the watermark feature drift problem caused by data distribution heterogeneity and improve the consistency of watermark embedding and recognition. The private classification head in the domain watermark module is responsible for introducing client-specific style features and frequency domain perturbations, which do not participate in the aggregation, so as to maintain the uniqueness and distinguishability of watermarks from each client.

[0116] After aggregation, the server distributes the updated main task model parameters and watermark generation shared backbone network parameters to each medical institution client. Through this step, the present invention ensures that the global main task continues to improve in classification performance through this synchronization mechanism; the watermark feature extraction capabilities of each client remain consistent; and the private watermark perturbations of each client remain independent and traceable.

[0117] In step S105, after completing the client-side medical task model training and server-side aggregation, the system enters the domain watermark verification stage to determine whether the target model has embedded a specific domain watermark. Training ends once the watermark is verified, and each medical institution's client obtains the watermarked medical task model. To improve the reliability of verification and sample utilization efficiency, according to an embodiment of the present invention, an adaptive watermark verification method based on statistical significance testing is proposed. Specifically, the present invention assumes that the watermarked medical task model (hereinafter referred to as the watermarked model) is trained with the watermark. Compared with the basic medical task model (referred to as the original model). In the same watermark sample set (the watermark sample also uses ( The accuracy rates on () are as follows:

[0118]

[0119]

[0120] in, For indicator functions, Used to indicate whether the watermarking model has successfully and accurately predicted the sample. This is used to indicate whether the base model has successfully and accurately predicted the sample. To verify whether a watermarked sample has been embedded, this invention aims to verify the following hypothesis:

[0121]

[0122] in, To verify the interval, if the medical task model to be verified slowly follows the above assumptions, the model is considered to have successfully embedded the watermark and the rights are confirmed successfully; otherwise, it does not contain a watermark.

[0123] To ensure the credibility of watermark verification, a confidence assessment is also required when querying. After 100 samples, the empirical success rate of watermarking is:

[0124]

[0125] in, Let be a Bernoulli random variable, used to indicate the first... The validation success rate of a sample is obtained based on Hoeffding's inequality; the empirical success rate of watermarking should satisfy:

[0126]

[0127] in, For the error radius, For a preset error rate (e.g.) A value of 0.01 means that the success rate of watermarking needs to exceed 99%, and

[0128]

[0129] Assume the lower bound of the empirical confidence rate for watermarking is:

[0130] ;

[0131] if, If the hypothesis is true, then the watermark is considered to have been successfully embedded in the watermark model.

[0132] According to one embodiment of the present invention, in order to balance the validation confidence and sample cost, this embodiment designs an adaptive sampling mechanism, assuming the initial sample size is... Each round increases the step size by step, and the maximum sample size is [value missing]. The significance level (lower limit of success rate) is .

[0133] Initial use For each sample, calculate the lower limit of the empirical confidence rate and the empirical success rate of watermarking. Value; if significance is not reached Then gradually increase the sample size:

[0134]

[0135] And recalculate the significance until the test passes. The sample size has reached the upper limit. Or, terminate the process early if there is no significant improvement after several consecutive rounds.

[0136] As can be seen from the above embodiments, compared with the prior art, this invention provides a label-consistent and harmless watermark generation mechanism for federal healthcare scenarios. The method extracts a unified semantic representation through a shared feature backbone network, and on this basis, filters difficult samples near the task boundary from local client data to generate watermark samples that are consistent with the original label and contain only minor perturbations. This mechanism achieves watermark embedding without introducing additional noise or changing the task label, allowing the model to learn watermark features without deviating from the optimization direction of the main task, thereby significantly reducing the performance loss of the main task and meeting the requirements of "harmless embedding" in high-precision scenarios such as healthcare. The watermark generator structure of this invention, consisting of a "shared backbone network + client-specific perturbation module," achieves distinguishable and stable personalized watermark embedding. The shared backbone provides a unified semantic foundation for watermark generation across all clients, while the client-specific perturbation module differentiates the shared features, ensuring that each client-generated watermark sample has a unique and stable feature expression. By combining the frequency domain amplitude constraint mechanism, the impact of perturbation on the feature space of the main task can be further limited, so that the watermarks of different clients remain independent and do not interfere with each other in the multi-round federated aggregation process, avoiding confusion in the federated aggregation, thereby realizing the client-distinguished and verifiable watermark tracking capability, and meeting the actual needs of model source identification and copyright protection in multi-center medical federated learning.

[0137] Compared with existing technologies, this invention not only effectively avoids the decline in accuracy of the main task in medical scenarios, but also supports independent verification of the contributions and sources of different client models, providing a more reliable and scalable technical approach for multi-center collaboration, especially for model copyright protection in medical scenarios. Furthermore, this invention significantly reduces the computational and communication overhead of the verification process while ensuring the accuracy of watermark verification.

[0138] It should be noted that although the steps are described in a specific order above, it does not mean that the steps must be executed in the above specific order. In fact, some of these steps can be executed concurrently, or even in a different order, as long as the required function can be achieved.

[0139] This invention can be a system, method, electronic device, computing device, computer-readable medium, and / or computer program product. A computer program product primarily refers to a software product that implements this solution through a computer program.

[0140] A computer-readable storage medium can be a tangible device that holds and stores instructions for use by an instruction execution device. Computer-readable storage media can include, for example, but not limited to, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof.

[0141] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for constructing a federal healthcare system with model watermarking, wherein the federal healthcare system includes a server and multiple clients, each client is deployed in a healthcare institution, each healthcare institution contains its own set of original medical data samples, and each client is configured with an initial medical task model, characterized in that, The method includes: S1. Using medical data samples as input and predicted categories as output, multiple rounds of first-order federated learning are performed on the original medical data sample set until convergence, so as to build a benchmark medical task model for each client. S2. Each client selects multiple difficult samples from its own original medical data sample set according to a preset method to construct a difficult sample set. The difficult sample is a medical data sample whose inference difficulty of the client's benchmark medical task model meets the preset requirements. S3. Build a watermark generator for each client. The watermark generator includes a shared backbone network and a client-private perturbation module. The client-private perturbation modules in each client are different. The shared backbone network is used to extract the basic features of hard samples. The client-private perturbation module is used to add perturbation to the basic features of hard samples and generate watermark samples that are consistent with the labels of hard samples. S4. Perform multiple rounds of second-stage federated learning until convergence, and then deploy the finally converged medical task model to various clients. Each round of second-stage federated learning includes: On each client, the watermark generator from the previous round of federated learning is used to generate the watermark sample set for the current round based on the set of difficult samples. The parameters of the generator are optimized based on the set of difficult samples and the set of watermark samples for the current round using a preset generator loss function. The shared backbone network parameters in the optimized generator are then uploaded to the server. Each client constructs the current round dataset based on the watermark sample set and the original medical data sample set, and uses it to train the medical task model after the previous round of federated learning until convergence. The parameters of the medical task model are optimized based on the classification loss of the main task, and the converged medical task model parameters are uploaded to the server. The server aggregates the medical model parameters uploaded by each client to obtain a fused medical task model and distributes the fused medical task model parameters to each client. It also aggregates the shared backbone network parameters uploaded by each client to obtain a fused shared backbone network and distributes the fused shared backbone network parameters to each client.

2. The method for constructing a federal healthcare system with model watermarking according to claim 1, characterized in that, Each round of Phase 1 federal learning includes: Each client trains the medical task model from the previous first-stage federated learning to convergence based on the original medical data sample set of its medical institution and uploads the medical task model parameters to the server. The server aggregates the medical model parameters uploaded by each client to obtain a fusion medical task model and then distributes the fusion medical task model parameters to each client.

3. The method for constructing a federal healthcare system with model watermarking according to claim 2, characterized in that, In S2, a set of difficult samples is constructed for each client in the following manner: The current medical task model on the client is used to infer the predicted category for each medical data sample in the original medical data sample set on the client, and the probability distribution of each category prediction for each medical data sample is predicted, and the highest category prediction probability is selected as the confidence level of the medical data sample. Calculate the gradient of the loss function with respect to each medical data sample; The difficulty level of each piece of medical data should be calculated as follows: in, Indicates medical data sample The level of difficulty Indicates medical data sample gradient, Indicates medical data sample Confidence level; All medical data samples in the original medical data sample set on each client are grouped according to their actual categories. Within each category, the medical data samples are selected in descending order of difficulty, with the top number of samples ranked. The samples selected from all categories constitute the difficult sample set.

4. The method for constructing a federal healthcare system with model watermarking according to claim 3, characterized in that, Each client uses a watermark generator to generate the current round's watermark sample set based on the hard sample set in the following way: The shared backbone network in the generator is used to extract the basic features of each hard sample in the client and normalize the basic features; The style code modulation module in the generator adds perturbation codes to each normalized basic feature to complete the modulation: in, Representing the basic features of hard samples, Indicates the first Perturbation coding for each client, and They are respectively calculating the mean and standard deviation for each feature channel. It is a numerical stability constant. and These represent fully connected transformations that map client perturbation codes to channel scaling and translation parameters, respectively. Transform the modulated features of each difficult sample into the frequency domain: in, This represents a two-dimensional Fast Fourier Transform; For each hard sample, a frequency-domain gating matrix is ​​introduced to transform the features into the frequency domain: in, Indicates the first Low-frequency gating masks that are private or shared by individual clients; The features of difficult samples with frequency domain gating matrix are inversely transformed back to the spatial domain, and a convolutional encoder is used to obtain the corresponding pixel-level perturbation based on the features of difficult samples in the inversely transformed back to the spatial domain. The pixel-level perturbation corresponding to the difficult sample is added to the difficult sample to obtain the watermark sample corresponding to the difficult sample. All the watermark samples constitute the watermark sample set.

5. A method for constructing a federal healthcare system with model watermarking according to claim 4, characterized in that, The default generator loss function is: in, This represents the reconstruction loss between watermarked samples and difficult samples. This represents the mutual information loss between hard samples and watermarked samples. This represents the classification consistency loss between hard samples and watermarked samples. , , They are respectively , , The weighting coefficients, where: in, Indicates the number of difficult samples. Indicates the first A difficult sample, Indicates the relationship with the first The first difficult sample corresponds to the first... One watermark sample, Indicates the first The feature vectors of a difficult sample in the shared backbone feature space. Indicates the first The feature vectors of the watermarked samples in the shared backbone feature space, where cos represents the cosine similarity. This represents a generator composed of a shared backbone network and client-side private perturbation modules. This represents the generator's output for the watermark sample. This represents the generator's output for difficult samples. It is logistic regression. It is divergence.

6. A method for constructing a federal healthcare system with model watermarking according to claim 5, characterized in that, The classification loss is: in, Indicates the first The main task classification cross-entropy loss on each client, Indicates the first Data sets on each client, Represents the dataset The Middle One sample, It is the first The category label of each sample, Indicates the first Medical task model parameters on each client, Indicates the first Medical task models on individual clients for samples The predicted output, Indicates sample The predicted loss.

7. A method for constructing a federal healthcare system with model watermarking according to claim 6, characterized in that, During the second phase of federated training, the convergence of training was determined when a watermark was embedded in the medical task model. Specifically, the watermark was determined as follows: if the difference between the prediction success rate of the medical task model on the watermarked sample set and the prediction success rate of the basic medical task model on the watermarked sample set is greater than or equal to a preset threshold, the medical task model is considered to have embedded a watermark.

8. A federal healthcare system with model watermarking based on the method of any one of claims 1-7, characterized in that, The system includes a server and multiple clients, wherein: Each client is deployed in a medical institution, each medical institution contains its own set of original medical data samples, and each client is configured with a medical task model constructed using the method described in any one of claims 1-7. Each client performs incremental training on its own medical task model to update the medical task model when there are new medical data samples, and uploads the updated medical task model parameters to the server. The server is used to aggregate the updated medical task model parameters uploaded by each client to obtain aggregated medical tasks in parallel, and then distribute the aggregated medical task model parameters to each client.

9. A computer program product comprising a computer program / instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1-7.

10. A computer device / equipment / system, comprising a memory, a processor, and computer programs / instructions stored in the memory, characterized in that, The processor executes the computer program / instructions to implement the steps of the method according to any one of claims 1-7.

Images