Federal learning method and device for personalized image segmentation model selection
By calculating the cosine similarity of image feature vectors and selecting appropriate batch normalization layer parameters, a personalized image segmentation model is constructed. This solves the problem that FedBN cannot handle data with unknown numbers and improves the application efficiency of federated learning models in the target domain.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- UNIV OF SCI & TECH BEIJING
- Filing Date
- 2026-01-05
- Publication Date
- 2026-04-21
AI Technical Summary
Existing federated learning methods such as FedBN cannot effectively handle data with unknown numbers or unlabeled data, resulting in limited model generalization ability.
By calculating the cosine similarity between the feature vector of the image to be segmented and the source domain modal feature vector library, the representative vector with the highest similarity is selected, thereby obtaining the corresponding batch normalization layer parameters and global model parameters, constructing a personalized image segmentation model, and performing image segmentation.
In the case of unlabeled data in the target domain, appropriate model parameters are selected based on the similarity of data modalities, which improves the application efficiency of federated learning models in the target domain and realizes efficient transfer and application of source domain models in the target domain.
Smart Images
Figure CN121904061A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image segmentation technology, and in particular to a federated learning method and apparatus for selecting personalized image segmentation models. Background Technology
[0002] Federated learning, as an emerging distributed machine learning paradigm, allows participants to collaboratively train a global model on their local data without sharing the original data, through the core mechanism of "model parameter flow replacing data flow." This approach avoids the risk of data leakage during transmission and fully utilizes the value of each party's data, enabling cross-institutional and cross-device collaborative data utilization and improving model performance and generalization ability. Therefore, federated learning has become an effective paradigm for breaking down data silos and achieving distributed collaborative model training.
[0003] In federated learning, the Non-IID (Non-Independent and Non-Identically Distributed) problem of data is a critical and pervasive challenge. Non-IID primarily manifests in three types: label distribution shift, quantity distribution shift, and feature distribution shift. The main causes of non-IID data include differences in data sources (data is collected from different environments, devices, or groups by different participants); and different business scenarios (participants operate in different business domains, have different data priorities, and use different methods to generate data). Non-IID data significantly impacts model performance and generalization ability. When participants have a limited number of label types, model accuracy drops sharply; as the number of participants increases, the smaller local data volume slows convergence, increases training costs, and leads to overfitting, severely affecting generalization ability.
[0004] Currently, federated learning primarily addresses the Non-IID problem in two ways: centralized federated learning and personalized federated learning. Centralized federated learning (FedAvg) aggregates model parameters through weighted averaging, offering some handling capability against imbalanced data. However, as the number of participants increases, the parameters of the local model gradually deviate from the global objective during training, making it difficult for the global model to adequately consider data information from different clients. Personalized federated learning (FedBN) provides an effective solution to the non-independent and identically distributed (IID) problem. It trains each client's batch normalization parameters individually while using federated aggregation to train the remaining parameters, achieving personalized federated learning. Specifically, for each client's data, only the local model constructed using the batch normalization parameters corresponding to that client is used for inference. However, this method requires explicit knowledge of the client ID of the image being analyzed during inference, making it ineffective at handling data with unknown IDs and limiting the generalization ability of FedBN. Summary of the Invention
[0005] To address the technical problem that existing FedBN methods cannot effectively handle data with unknown identifiers, this invention provides a federated learning method and apparatus for personalized image segmentation model selection. The technical solution is as follows:
[0006] On the one hand, a federated learning method for personalized image segmentation model selection is provided, which is implemented by a federated learning device for personalized image segmentation model selection. The method includes: S1. Obtain the image data to be segmented. Download the aggregated global model parameters, batch normalization layer parameter library and source domain modality feature vector library from the server. Extract features from the image data to be segmented using a pre-trained feature extraction model to obtain the feature vector of the image to be segmented.
[0007] S2. Calculate the cosine similarity between the feature vector of the image to be segmented and each representative vector in the source domain modal feature vector library. Based on the cosine similarity calculation results, select the representative vector with the highest similarity to the feature vector of the image to be segmented.
[0008] S3. Obtain the batch normalization layer parameters of the corresponding source domain client node from the batch normalization layer parameter library based on the representative vector with the highest similarity.
[0009] S4. Construct a personalized image segmentation model based on the obtained batch normalization layer parameters and the aggregated global model parameters. Perform image segmentation on the image data to be segmented according to the personalized image segmentation model to obtain the segmentation result.
[0010] Optionally, the construction process of the aggregated global model parameters, the batch normalized layer parameter library, and the source domain modal feature vector library in S1 includes: S11. Obtain an image dataset with non-independent and identically distributed characteristics, divide the image dataset into multiple image data subsets, and send the multiple image data subsets to multiple source domain client nodes respectively; wherein, each image data subset includes a training set and a test set.
[0011] S12. Each of the multiple source domain client nodes extracts features from the image data in the training set using a pre-trained feature extraction model, obtains a representative vector for each source domain client node, and uploads the representative vector to the server.
[0012] S13. The server aggregates the representative vectors of multiple source domain client nodes to obtain the source domain modal feature vector library.
[0013] S14. The server sends global model parameters (excluding batch normalization layers) to multiple source domain client nodes. Each source domain client node trains an image segmentation model based on the global model parameters, updates the global model parameters using backpropagation, and uploads the updated global model parameters to the server.
[0014] S15. The server aggregates the updated global model parameters (excluding batch normalization layer) from multiple source domain client nodes to obtain the aggregated global model parameters, and constructs a batch normalization layer parameter library based on the batch normalization layer parameters.
[0015] S16. Determine whether the preset conditions are met. If they are met, then perform the image segmentation model detection based on the test set. If they are not met, then execute step S14.
[0016] Optionally, each source domain client node in S12 performs feature extraction on the image data in the training set using a pre-trained feature extraction model, including: Each source domain client node extracts features from the image data in the training set using a model based on a contrastive language-image pre-trained architecture.
[0017] Optionally, each source domain client node in S14 trains an image segmentation model based on global model parameters, including: Each source domain client node trains a U-Net image segmentation model based on global model parameters and Dice coefficients.
[0018] Optionally, in S14, the updated global model parameters are uploaded to the server, including: For the parameters in the updated global model parameters, excluding those in the batch normalization layer, upload them to the server after each update.
[0019] The parameters for the batch normalization layer are uploaded to the server after the image segmentation model training is completed.
[0020] On the other hand, a federated learning apparatus for personalized image segmentation model selection is provided, which is applied to a federated learning method for personalized image segmentation model selection. The apparatus includes: The image feature extraction module is used to acquire the image data to be segmented. It downloads the aggregated global model parameters, batch normalization layer parameter library and source domain modality feature vector library from the server, and extracts features from the image data to be segmented using a pre-trained feature extraction model to obtain the feature vector of the image to be segmented.
[0021] The similarity calculation module is used to calculate the cosine similarity between the feature vector of the image to be segmented and each representative vector in the source domain modality feature vector library. Based on the cosine similarity calculation result, the representative vector with the highest similarity to the feature vector of the image to be segmented is selected.
[0022] The parameter determination module is used to obtain the batch normalization layer parameters of the corresponding source domain client node from the batch normalization layer parameter library based on the representative vector with the highest similarity.
[0023] The output module is used to construct a personalized image segmentation model based on the acquired batch normalization layer parameters and the aggregated global model parameters, and to perform image segmentation on the image data to be segmented according to the personalized image segmentation model to obtain the segmentation result.
[0024] Optionally, the image feature extraction module is further used for: S11. Obtain an image dataset with non-independent and identically distributed characteristics, divide the image dataset into multiple image data subsets, and send the multiple image data subsets to multiple source domain client nodes respectively; wherein, each image data subset includes a training set and a test set.
[0025] S12. Each of the multiple source domain client nodes extracts features from the image data in the training set using a pre-trained feature extraction model, obtains a representative vector for each source domain client node, and uploads the representative vector to the server.
[0026] S13. The server aggregates the representative vectors of multiple source domain client nodes to obtain the source domain modal feature vector library.
[0027] S14. The server sends global model parameters (excluding batch normalization layers) to multiple source domain client nodes. Each source domain client node trains an image segmentation model based on the global model parameters, updates the global model parameters using backpropagation, and uploads the updated global model parameters to the server.
[0028] S15. The server aggregates the updated global model parameters (excluding batch normalization layer) from multiple source domain client nodes to obtain the aggregated global model parameters, and constructs a batch normalization layer parameter library based on the batch normalization layer parameters.
[0029] S16. Determine whether the preset conditions are met. If they are met, then perform the image segmentation model detection based on the test set. If they are not met, then execute step S14.
[0030] Optionally, the image feature extraction module is further used for: Each source domain client node extracts features from the image data in the training set using a model based on a contrastive language-image pre-trained architecture.
[0031] Optionally, the image feature extraction module is further used for: Each source domain client node trains a U-Net image segmentation model based on global model parameters and Dice coefficients.
[0032] Optionally, the image feature extraction module is further used for: For the parameters in the updated global model parameters, excluding those in the batch normalization layer, upload them to the server after each update.
[0033] The parameters for the batch normalization layer are uploaded to the server after the image segmentation model training is completed.
[0034] On the other hand, a federated learning device for personalized image segmentation model selection is provided, the federated learning device for personalized image segmentation model selection comprising: a processor; and a memory storing computer-readable instructions, which, when executed by the processor, implement any of the methods described above for federated learning of personalized image segmentation model selection.
[0035] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored therein, the at least one instruction being loaded and executed by a processor to implement any of the above-described federated learning methods for personalized image segmentation model selection.
[0036] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following: This invention proposes a federated learning method—FedBNStyle—for personalized image segmentation model selection. This method is a feature similarity-based model selection approach. Its core involves calculating the feature vectors of source domain data, uploading them to the server to construct a source domain modality feature vector library, and then calculating the cosine similarity between the feature vectors of the target domain image and each vector in the source domain modality feature vector library. The client-side BN layer parameters corresponding to the vector with the highest cosine similarity are selected and combined with the global model parameters to form a personalized model for inference on the target domain data. This feature similarity-based model selection method, even when the target domain data is unlabeled, selects suitable model parameters based on the similarity of data modalities, providing a practical strategy for generating models suitable for the target domain. It effectively solves the problem that FedBN cannot effectively handle data with unknown numbers or unlabeled data, significantly improving the application efficiency of federated learning models in the target domain and achieving efficient transfer and application of source domain models to the target domain. Attached Figure Description
[0037] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0038] Figure 1 This is a flowchart of a federated learning method for personalized image segmentation model selection provided by an embodiment of the present invention; Figure 2 This is a model training architecture diagram of the image segmentation model selection method based on federated learning and feature similarity model selection provided in the embodiments of the present invention; Figure 3 This is a schematic diagram of the data distribution of the five clients in the experiment involved in this embodiment of the invention; Figure 4 This is a dataset of microscopic images of five types of modal materials involved in the experiments described in this embodiment of the invention; Figure 5 This is a block diagram of a federated learning device for personalized image segmentation model selection provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of the structure of a federated learning device for personalized image segmentation model selection provided in an embodiment of the present invention. Detailed Implementation
[0039] The technical solution of the present invention will now be described with reference to the accompanying drawings.
[0040] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.
[0041] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning.
[0042] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.
[0043] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.
[0044] This invention provides a federated learning method for personalized image segmentation model selection. This method can be implemented by a federated learning device for personalized image segmentation model selection, which can be a terminal or a server. Figure 1 The flowchart shown is a federated learning method for selecting a personalized image segmentation model. The processing flow of this method may include the following steps:
[0045] S1. Obtain the image data to be segmented. Download the aggregated global model parameters, batch normalization layer parameter library and source domain modality feature vector library from the server. Extract features from the image data to be segmented using a pre-trained feature extraction model to obtain the feature vector of the image to be segmented.
[0046] Optionally, such as Figure 2 As shown, the construction process of the aggregated global model parameters, batch normalized layer parameter library, and source domain modal feature vector library in S1 can include the following steps S11-S16: S11. Obtain an image dataset with non-independent and identically distributed characteristics, divide the image dataset into multiple image data subsets, and send the multiple image data subsets to multiple source domain client nodes respectively; wherein, each image data subset includes a training set and a test set.
[0047] In one feasible implementation, the Dirichlet method is used to simulate the non-independent identical distribution of data in real-life scenarios, and the partitioned data is distributed to each client node.
[0048] S12. Each of the multiple source domain client nodes extracts features from the image data in the training set using a pre-trained feature extraction model, obtains a representative vector for each source domain client node, and uploads the representative vector to the server.
[0049] Optionally, each source domain client node in S12 performs feature extraction on the image data in the training set using a pre-trained feature extraction model, including: Each source domain client node extracts features from the image data in the training set using a model based on a contrastive language-image pre-trained architecture.
[0050] In one feasible implementation, each client node uses a Vit-B / 32 model with a Clip (Contrastive Language-Image Pre-Training) architecture to extract features from the training set data to obtain a representative vector of the client's data modality. ,Will Upload to the server.
[0051] S13. The server aggregates the representative vectors of multiple source domain client nodes to obtain the source domain modal feature vector library.
[0052] In one feasible implementation, the server aggregates the feature matrices uploaded by each client. Construct a source domain modal feature vector library }
[0053] S14. The server sends global model parameters (excluding batch normalization layers) to multiple source domain client nodes. Each source domain client node trains an image segmentation model based on the global model parameters, updates the global model parameters using backpropagation, and uploads the updated global model parameters to the server.
[0054] Optionally, each source domain client node in S14 trains an image segmentation model based on global model parameters, including: Each source domain client node trains a U-Net image segmentation model based on global model parameters and Dice coefficients.
[0055] Optionally, in S14, the updated global model parameters are uploaded to the server, including: For the parameters in the updated global model parameters, excluding those in the batch normalization layer, upload them to the server after each update.
[0056] The parameters for the batch normalization layer are uploaded to the server after the image segmentation model training is completed.
[0057] In one feasible implementation, the server sends global model parameters (without a Batch Normalization (BN) layer) to each client node. Each client node loads global model parameters. The U-Net image segmentation network is trained using Dice coefficients. Each client updates its parameters using backpropagation. After updating, each client records its own BN layer parameters and then uploads the local model parameters (without BN layers) to the server.
[0058] S15. The server aggregates the updated global model parameters (excluding batch normalization layer) from multiple source domain client nodes to obtain the aggregated global model parameters, and constructs a batch normalization layer parameter library based on the batch normalization layer parameters.
[0059] In one feasible implementation, the server receives local model parameters uploaded by each client and aggregates them using the mean method.
[0060] S16. Determine whether the preset conditions are met. If they are met, then perform the image segmentation model detection based on the test set. If they are not met, then execute step S14.
[0061] In one feasible implementation, after each parameter update, the server repeats steps S14-S15. Each repetition of S14-S15 constitutes one round of communication between the client and the server. Model detection is performed every 25 rounds of communication, with all clients calculating the feature vector corresponding to each image in the test set in real time. ( , (This refers to the target domain dataset).
[0062] Furthermore, the target domain user downloads the required model parameters from the server, including the shared global model without BN parameters. ), BN parameter library ( ) and source domain modal feature vector library ( Each client node calculates the feature vector of the target domain image. With each vector in the source domain modal feature vector library ( The cosine similarity of () is calculated using the following formula:
[0063] (1) In the formula, express and The cosine similarity is calculated, with a value ranging from [-1, 1]. The closer the cosine similarity value is to 1, the higher the similarity between the two vectors, meaning that the modal features of the target domain image are more similar to the modal features of the corresponding source domain client data.
[0064] Furthermore, by calculating the cosine similarity, the vector with the highest similarity to the feature vector of the target domain image is selected from the source domain modal feature vector library. The formula is as follows: (2) The corresponding BN of the source domain client is denoted as .
[0065] Use a shared global model without BN parameters. Combined with the choice That is, from " The constructed model performs inference on the target domain data. During the inference process, the model calculates based on the input target domain image data and the selected model parameters to obtain the predicted segmentation result.
[0066] The specific experimental setup includes: (1) Dataset: This experiment focuses on image segmentation tasks, such as... Figure 3 As shown in Table 1, the experimental dataset originated from real data from universities and steel product manufacturing plants, such as... Figure 4 As shown, the dataset contains microscopic images of materials in five modalities: lanthanum dendrites in aluminum, DP590 dual-phase steel (cold-rolled), high-temperature alloy 1, high-temperature alloy 2, and nickel-based high-temperature alloys. While both high-temperature alloys belong to the same category, high-temperature alloy 1 and high-temperature alloy 2 exhibit significant differences in texture, shape, and grayscale visual features due to their different material compositions. Each modality contains 220 images, divided into a training set (176 images) and a test set (44 images) at an 8:2 ratio, with a uniform image size of 256×256 pixels. The segmentation objective of this dataset is to accurately distinguish precipitates in the microscopic images of metallic materials from the matrix and other microstructures, identifying them as foreground, while other areas are identified as background. Due to differences in material preparation processes and composition control among steel companies, these data exhibit a clear non-independent and identically distributed characteristic. This dataset provides a realistic and diverse data scenario for verifying the algorithm's performance in handling complex data distributions.
[0067] Table 1. Data distribution of the 5 clients in the experiment. (2) Comparison method: To comprehensively evaluate the method proposed in this invention, it is compared with the centralized federated learning algorithm FedAvg and the personalized federated learning algorithm FedBN, and finally the average value of the evaluation results of each model is calculated.
[0068] In terms of evaluation metrics, to comprehensively and objectively quantitatively evaluate the segmentation performance of different methods, multiple metrics are used for comprehensive evaluation from multiple perspectives such as pixels and regions. These include Dice Similarity Coefficient, VI (Variation of Information), PA (Pixel Accuracy), MAP (Mean Average Precision), and Mean Recall. These metrics are first calculated for a single image, and the evaluation result for image segmentation of the entire test set is the average of the calculated results for each image.
[0069] (3) Experimental environment and parameter settings: All experiments in this invention were conducted on an NVIDIA TITAN RTX 24GB computing card. The computing platform's system version was Ubuntu 20.04.4 x86_64, the programming language was Python 3.8.10, and the deep learning tool was PyTorch 1.8.1+cu111.
[0070] Regarding the model training parameter settings, the batch size was set to 16, the initial learning rate was 0.005, each algorithm underwent 200 rounds of communication, and each node trained locally for 1 epoch in each round of communication. DiceLoss was selected as the loss function, the cross-validation folds (num_folds) were set to 5, and the model was not merged during inference. The Adam optimizer was used to update the model parameters.
[0071] (4) Experimental comparison and analysis: For the task of segmenting material microscopic images in the target domain, Table 2 compares the quantitative evaluation results of the proposed FedBNStyle algorithm with FedAvg and FedBN on the target domain dataset. The table shows the average evaluation metrics of the algorithm on each target domain dataset.
[0072] Table 2 Comparison of Experimental Results When the target domain dataset contains images of multiple modalities and no reference samples are available, the results of the feature similarity-based model selection method are shown in the last row of Table 2. The FedBNStyle algorithm achieves the best results compared to existing federated learning algorithms in terms of Dice, VI, PA, MAP, and Mean_Recall metrics. Specifically, compared to the FedAvg algorithm, it improves the Dice, VI, PA, MAP, and Mean_Recall metrics by 0.97%, 4.72%, 0.61%, 1.54%, and 0.91%, respectively; and compared to the well-performing personalized federated learning algorithm FedBN, it improves the Dice, VI, PA, MAP, and Mean_Recall metrics by 0.15%, 1.50%, 0.04%, 3.12%, and 0.05%, respectively. Overall, the FedBNStyle algorithm does not rely on reference samples. This feature similarity-based model selection method selects suitable model parameters based on the similarity of data modalities when the target domain data is unlabeled. It provides a practical strategy for generating models suitable for the target domain, effectively improving the application efficiency of federated learning models in the target domain and effectively realizing the efficient transfer and application of source domain models in the target domain.
[0073] S2. Calculate the cosine similarity between the feature vector of the image to be segmented and each representative vector in the source domain modal feature vector library. Based on the cosine similarity calculation results, select the representative vector with the highest similarity to the feature vector of the image to be segmented.
[0074] S3. Obtain the batch normalization layer parameters of the corresponding source domain client node from the batch normalization layer parameter library based on the representative vector with the highest similarity.
[0075] S4. Construct a personalized image segmentation model based on the obtained batch normalization layer parameters and the aggregated global model parameters. Perform image segmentation on the image data to be segmented according to the personalized image segmentation model to obtain the segmentation result.
[0076] This invention proposes a federated learning method—FedBNStyle—for personalized image segmentation model selection. This method is a feature similarity-based model selection approach. Its core involves calculating the feature vectors of source domain data, uploading them to the server to construct a source domain modality feature vector library, and then calculating the cosine similarity between the feature vectors of the target domain image and each vector in the source domain modality feature vector library. The client-side BN layer parameters corresponding to the vector with the highest cosine similarity are selected and combined with the global model parameters to form a personalized model for inference on the target domain data. This feature similarity-based model selection method, even when the target domain data is unlabeled, selects suitable model parameters based on the similarity of data modalities, providing a practical strategy for generating models suitable for the target domain. It effectively solves the problem that FedBN cannot effectively handle data with unknown numbers or unlabeled data, significantly improving the application efficiency of federated learning models in the target domain and achieving efficient transfer and application of source domain models to the target domain.
[0077] Figure 5 This is a block diagram of a federated learning apparatus for personalized image segmentation model selection, illustrated according to an exemplary embodiment. The apparatus is used in a federated learning method for personalized image segmentation model selection. (Refer to...) Figure 5 The device includes an image feature extraction module 310, a similarity calculation module 320, a parameter determination module 330, and an output module 340. Wherein:
[0078] The image feature extraction module 310 is used to acquire the image data to be segmented, download the aggregated global model parameters, batch normalization layer parameter library and source domain modality feature vector library from the server, and extract features from the image data to be segmented through the pre-trained feature extraction model to obtain the feature vector of the image to be segmented.
[0079] The similarity calculation module 320 is used to calculate the cosine similarity between the feature vector of the image to be segmented and each representative vector in the source domain modal feature vector library, and select the representative vector with the highest similarity to the feature vector of the image to be segmented based on the cosine similarity calculation result.
[0080] The parameter determination module 330 is used to obtain the batch normalization layer parameters of the corresponding source domain client node from the batch normalization layer parameter library based on the representative vector with the highest similarity.
[0081] The output module 340 is used to construct a personalized image segmentation model based on the acquired batch normalization layer parameters and the aggregated global model parameters, and to perform image segmentation on the image data to be segmented based on the personalized image segmentation model to obtain the segmentation result.
[0082] This invention proposes a federated learning method—FedBNStyle—for personalized image segmentation model selection. This method is a feature similarity-based model selection approach. Its core involves calculating the feature vectors of source domain data, uploading them to the server to construct a source domain modality feature vector library, and then calculating the cosine similarity between the feature vectors of the target domain image and each vector in the source domain modality feature vector library. The client-side BN layer parameters corresponding to the vector with the highest cosine similarity are selected and combined with the global model parameters to form a personalized model for inference on the target domain data. This feature similarity-based model selection method, even when the target domain data is unlabeled, selects suitable model parameters based on the similarity of data modalities, providing a practical strategy for generating models suitable for the target domain. It effectively solves the problem that FedBN cannot effectively handle data with unknown numbers or unlabeled data, significantly improving the application efficiency of federated learning models in the target domain and achieving efficient transfer and application of source domain models to the target domain.
[0083] Figure 6 This is a schematic diagram of the structure of a federated learning device for personalized image segmentation model selection provided in an embodiment of the present invention, as shown below. Figure 6 As shown, federated learning devices for selecting personalized image segmentation models can include the above-mentioned Figure 5 The illustrated federated learning apparatus for selecting a personalized image segmentation model. Optionally, the federated learning apparatus 410 for selecting a personalized image segmentation model may include a first processor 2001.
[0084] Optionally, the federated learning device 410 for selecting a personalized image segmentation model may also include a memory 2002 and a transceiver 2003.
[0085] The first processor 2001, memory 2002, and transceiver 2003 can be connected via a communication bus.
[0086] The following is combined Figure 6 The components of the federated learning device 410 for personalized image segmentation model selection are described in detail below: The first processor 2001 is the control center of the federated learning device 410 for selecting personalized image segmentation models. It can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 can be one or more central processing units (CPUs), application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement embodiments of the present invention, such as one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs).
[0087] Optionally, the first processor 2001 can perform various functions of the federated learning device 410 for personalized image segmentation model selection by running or executing software programs stored in the memory 2002 and calling data stored in the memory 2002.
[0088] In a specific implementation, as one example, the first processor 2001 may include one or more CPUs, for example... Figure 6 CPU0 and CPU1 are shown in the diagram.
[0089] In a specific implementation, as one example, the federated learning device 410 for selecting a personalized image segmentation model may also include multiple processors, for example... Figure 6 The first processor 2001 and the second processor 2004 are shown in the diagram. Each of these processors can be a single-core processor or a multi-core processor. Here, a processor can refer to one or more devices, circuits, and / or processing cores used to process data (such as computer program instructions).
[0090] The memory 2002 is used to store the software program that executes the present invention, and is controlled by the first processor 2001 to execute it. The specific implementation method can be referred to the above method embodiment, and will not be repeated here.
[0091] Optionally, the memory 2002 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 2002 may be integrated with the first processor 2001 or may exist independently, and may be accessed via the interface circuitry of the federated learning device 410 for selecting personalized image segmentation models. Figure 6 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.
[0092] The transceiver 2003 is used to communicate with network devices or with terminal devices.
[0093] Alternatively, transceiver 2003 may include a receiver and a transmitter. Figure 6 (Not shown separately). The receiver is used to implement the receiving function, and the transmitter is used to implement the transmitting function.
[0094] Alternatively, the transceiver 2003 can be integrated with the first processor 2001 or exist independently, and can be interfaced through the federated learning device 410 for selecting personalized image segmentation models. Figure 6 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.
[0095] It should be noted that, Figure 6 The structure of the federated learning device 410 for personalized image segmentation model selection shown in the figure does not constitute a limitation on the router. Actual knowledge structure recognition devices may include more or fewer components than shown, or combine certain components, or have different component arrangements.
[0096] Furthermore, the technical effectiveness of the federated learning device 410 for personalized image segmentation model selection can be referenced from the technical effectiveness of the federated learning method for personalized image segmentation model selection described in the above method embodiments, and will not be repeated here.
[0097] It should be understood that the first processor 2001 in the embodiments of the present invention may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.
[0098] It should also be understood that the memory in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).
[0099] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.
[0100] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.
[0101] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.
[0102] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.
[0103] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.
[0104] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.
[0105] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.
[0106] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0107] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.
[0108] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0109] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.
Claims
1. A federated learning method for personalized image segmentation model selection, characterized in that, The method includes: S1. Obtain the image data to be segmented. Download the aggregated global model parameters, batch normalization layer parameter library and source domain modal feature vector library from the server. Extract features from the image data to be segmented using the pre-trained feature extraction model to obtain the feature vector of the image to be segmented. S2. Calculate the cosine similarity between the feature vector of the image to be segmented and each representative vector in the source domain modality feature vector library, and select the representative vector with the highest similarity to the feature vector of the image to be segmented based on the cosine similarity calculation result. S3. Obtain the batch normalization layer parameters of the corresponding source domain client node from the batch normalization layer parameter library based on the representative vector with the highest similarity. S4. Construct a personalized image segmentation model based on the obtained batch normalization layer parameters and the aggregated global model parameters. Perform image segmentation on the image data to be segmented according to the personalized image segmentation model to obtain the segmentation result.
2. The federated learning method for personalized image segmentation model selection according to claim 1, characterized in that, The construction process of the aggregated global model parameters, batch normalized layer parameter library, and source domain modal feature vector library in S1 includes: S11. Obtain an image dataset with non-independent and identically distributed characteristics, divide the image dataset into multiple image data subsets, and send the multiple image data subsets to multiple source domain client nodes respectively; wherein, each image data subset includes a training set and a test set; S12. Each of the multiple source domain client nodes extracts features from the image data in the training set using a pre-trained feature extraction model to obtain a representative vector for each source domain client node, and then uploads the representative vector to the server. S13. The server aggregates the representative vectors of multiple source domain client nodes to obtain the source domain modal feature vector library. S14. The server sends global model parameters (excluding batch normalization layer) to multiple source domain client nodes. Each source domain client node trains an image segmentation model based on the global model parameters and updates the global model parameters using backpropagation. The updated global model parameters are then uploaded to the server. S15. The server aggregates the updated global model parameters (excluding the batch normalization layer) from multiple source domain client nodes to obtain the aggregated global model parameters, and constructs a batch normalization layer parameter library based on the batch normalization layer parameters. S16. Determine whether the preset conditions are met. If they are met, then perform the image segmentation model detection based on the test set. If they are not met, then execute step S14.
3. The federated learning method for personalized image segmentation model selection according to claim 2, characterized in that, Each source domain client node in S12 extracts features from the image data in the training set using a pre-trained feature extraction model, including: Each source domain client node extracts features from the image data in the training set using a model based on a contrastive language-image pre-trained architecture.
4. The federated learning method for personalized image segmentation model selection according to claim 2, characterized in that, Each source domain client node in S14 trains an image segmentation model based on global model parameters, including: Each source domain client node trains a U-Net image segmentation model based on global model parameters and Dice coefficients.
5. The federated learning method for personalized image segmentation model selection according to claim 2, characterized in that, Uploading the updated global model parameters to the server in S14 includes: For the parameters in the updated global model parameters, excluding those in the batch normalization layer, upload them to the server after each update; The parameters for the batch normalization layer are uploaded to the server after the image segmentation model training is completed.
6. A federated learning apparatus for personalized image segmentation model selection, wherein the federated learning apparatus for personalized image segmentation model selection is used to implement the federated learning method for personalized image segmentation model selection as described in any one of claims 1-5, characterized in that, The device includes: The image feature extraction module is used to acquire the image data to be segmented. It downloads the aggregated global model parameters, batch normalization layer parameter library and source domain modality feature vector library from the server, and extracts features from the image data to be segmented using a pre-trained feature extraction model to obtain the feature vector of the image to be segmented. The similarity calculation module is used to calculate the cosine similarity between the feature vector of the image to be segmented and each representative vector in the source domain modality feature vector library, and select the representative vector with the highest similarity to the feature vector of the image to be segmented based on the cosine similarity calculation result. The parameter determination module is used to obtain the batch normalization layer parameters of the corresponding source domain client node from the batch normalization layer parameter library based on the representative vector with the highest similarity. The output module is used to construct a personalized image segmentation model based on the acquired batch normalization layer parameters and the aggregated global model parameters, and to perform image segmentation on the image data to be segmented according to the personalized image segmentation model to obtain the segmentation result.
7. The federated learning device for personalized image segmentation model selection according to claim 6, characterized in that, The construction process of the aggregated global model parameters, batch normalized layer parameter library, and source domain modal feature vector library includes: S11. Obtain an image dataset with non-independent and identically distributed characteristics, divide the image dataset into multiple image data subsets, and send the multiple image data subsets to multiple source domain client nodes respectively; wherein, each image data subset includes a training set and a test set; S12. Each of the multiple source domain client nodes extracts features from the image data in the training set using a pre-trained feature extraction model to obtain a representative vector for each source domain client node, and then uploads the representative vector to the server. S13. The server aggregates the representative vectors of multiple source domain client nodes to obtain the source domain modal feature vector library. S14. The server sends global model parameters (excluding batch normalization layer) to multiple source domain client nodes. Each source domain client node trains an image segmentation model based on the global model parameters and updates the global model parameters using backpropagation. The updated global model parameters are then uploaded to the server. S15. The server aggregates the updated global model parameters (excluding the batch normalization layer) from multiple source domain client nodes to obtain the aggregated global model parameters, and constructs a batch normalization layer parameter library based on the batch normalization layer parameters. S16. Determine whether the preset conditions are met. If they are met, then perform the image segmentation model detection based on the test set. If they are not met, then execute step S14.
8. The federated learning device for personalized image segmentation model selection according to claim 6, characterized in that, Uploading the updated global model parameters to the server includes: For the parameters in the updated global model parameters, excluding those in the batch normalization layer, upload them to the server after each update; The parameters for the batch normalization layer are uploaded to the server after the image segmentation model training is completed.
9. A federated learning device for personalized image segmentation model selection, characterized in that, The federated learning device for selecting personalized image segmentation models includes: processor; A memory storing computer-readable instructions that, when executed by the processor, implement the method as described in any one of claims 1 to 5.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains program code that can be invoked by a processor to execute the method as described in any one of claims 1 to 5.