Federal learning-based echocardiogram segmentation method and device
Through federated learning methods, global and local models are built for anti-noise training and updates, solving the problems of limited data and privacy leakage in echocardiography segmentation, and realizing privacy protection and efficient segmentation of multi-center collaborative segmentation.
Patent Information
- Application Number
- CN202510366463.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-25
- Publication Date
- 2025-07-11
AI Technical Summary
In the prior art, the echocardiography segmentation method has problems such as limited training data, easy privacy leakage, high computing resources consumption in data concentration, and noise propagation affects the segmentation effect.
Using a federated learning method, the global initial model deployed on the server side and the local initial model of the client is carried out, and the federated learning framework is built to realize multi-center collaborative segmentation, avoid data exchange, and use joint loss assessment to improve noise immunity.
While protecting privacy, it improves the accuracy and reliability of echocardiography segmentation, reduces computing resource consumption, and is suitable for multi-center collaborative segmentation.
Smart Images
Figure CN120298693A_ABST
Abstract
Description
Technical Field
[0001] The present invention mainly relates to the field of image processing, and particularly to an echocardiogram segmentation method based on federated learning. Background Art
[0002] Echocardiogram segmentation is one of the key tasks in the field of medical image processing and is of great significance for cardiac health assessment and disease diagnosis. Echocardiograms are widely used in clinical practice due to their low cost and radiation-free acquisition method. Accurate echocardiogram segmentation can greatly improve the diagnostic efficiency and accuracy of clinicians. Traditional methods based on manual feature extraction and classical image processing have problems such as time-consuming, laborious, relying on expert experience, and poor generalization. With the development of computer vision and deep learning technologies, automated segmentation methods based on deep neural networks have gradually become a research hotspot.
[0003] However, echocardiogram segmentation algorithms based on deep learning need to rely on a large amount of labeled data for training. The number of echocardiograms in a single medical center is often limited, and the manual annotation process is laborious and time-consuming. This defect limits the clinical application of echocardiogram segmentation algorithms based on deep learning. In recent years, echocardiogram segmentation algorithms based on multi-medical center collaboration have received extensive attention.
[0004] However, due to factors such as patient privacy protection, data security, and data management regulations between different medical institutions, it is difficult to centrally store and uniformly manage echocardiogram data from multiple medical centers. In addition, there are significant differences in the imaging device models, parameter settings, and operation specifications of different medical centers, resulting in obvious heterogeneity in data distribution. If the data from multiple medical centers are directly put together for centralized training, it not only faces great challenges at the technical level but also easily causes the risk of patient privacy leakage.
[0005] To perform multi-medical center collaborative segmentation, one method is to establish a unified data sharing platform where each medical center uploads its data to the platform for other medical centers to access and use according to authorization. For example, some regional medical alliances will build such platforms to integrate patient medical records, examination and test results, etc. from different hospitals in the region to support scientific research, clinical diagnosis, and other work. However, it is difficult to protect the data security and privacy of such platforms. Once the platform is attacked by hackers, all data is at risk; in addition, the data formats and standards of different medical centers may be different, requiring a large amount of work for data cleaning and standardization processing, and the construction and maintenance costs of the platform are high, requiring a large amount of manpower, material resources, and financial resources.
[0006] Data encryption and sharing is also a multi-medical center collaboration method. After encrypting medical data, it is shared among medical centers. Only the receiving party can decrypt and use the data through a specific key. For example, in some multi-center clinical trials, encryption technology is used to protect the privacy of patient data while enabling secure sharing of data among different centers. However, this method increases the time and computational resource consumption of data processing during the encryption and decryption processes; and key management is complex. Once the key is lost or leaked, data security will be seriously threatened.
[0007] Blockchain-based data collaboration is a distributed multi-medical center collaboration method. Using the distributed ledger technology of the blockchain, each medical center participates in the recording and verification of data as a node. The data is stored on the blockchain in an encrypted form to ensure the immutability and traceability of the data. However, the performance of the blockchain is limited, and efficiency problems may occur when processing large-scale data, such as long transaction confirmation times; the writing and maintenance of smart contracts require professional technical knowledge, increasing the implementation difficulty; the storage cost is relatively high, and as the amount of data increases, the amount of data that blockchain nodes need to store will also continue to increase. Summary of the Invention
[0008] In view of the above problems, the present application is proposed to provide a method and device for echocardiogram segmentation based on federated learning that overcomes or at least partially solves the above problems, including:
[0009] A method for echocardiogram segmentation based on federated learning, the echocardiogram segmentation method is implemented through at least 2 initial models, the initial models include a global initial model deployed on the server side and the local initial model deployed on the client side, including the steps of:
[0010] Obtain an initial data set, and perform noise-resistant training on the initial local model according to the initial data set to obtain a local noise-resistant model and determine the weight of the local noise-resistant model;
[0011] Generate a global noise-resistant model according to the local noise-resistant model and the weight;
[0012] Perform iterative noise-resistant updates on the initial global model and the initial local model according to the global noise-resistant model and the local noise-resistant model until a target global model and a target local model are obtained;
[0013] Perform echocardiogram segmentation through the target local model.
[0014] Further, the step of obtaining the initial data set includes:
[0015] Obtain an initial data set, and filter the initial data set according to preset conditions to obtain a filtered data set;
[0016] Perform standardized preprocessing on the filtered data set to obtain a grayscale image set of a unified size;
[0017] Segment and store the grayscale image set on a number of clients, where the data distributions among the number of clients are different.
[0018] Further, the step of performing standardized preprocessing on the filtered data set to obtain a grayscale image set of a unified size includes:
[0019] Perform standardized preprocessing on all images in the filtered data set to obtain preprocessed images;
[0020] Annotate pixel-level heart tissue semantic labels for the preprocessed images; wherein, the heart tissue semantic labels include an accurate label sample set and a noisy label sample set.
[0021] Further, the step of performing noise-resistant training on the initial local model according to the initial data set to obtain a local noise-resistant model and determining the weights of the local noise-resistant model includes:
[0022] Train the local model according to the initial data set and calculate the loss function to obtain a loss function set;
[0023] Sort the loss functions in the loss function set, and take a preset number of loss functions to generate a pseudo-clean sample set;
[0024] Fine-tune the local model according to the pseudo-clean sample set to obtain a local noise-resistant model;
[0025] Determine the weights of the local noise-resistant model according to the number of samples in the local noise-resistant model.
[0026] Further, the step of generating a global noise-resistant model according to the local noise-resistant model and the weights includes:
[0027] Send the local noise-resistant model to the server side;
[0028] The server side performs weighted aggregation on the local noise-resistant models of all clients to obtain a global noise-resistant model.
[0029] Further, the step of performing noise-resistant updates on the initial global model and the initial local model according to the global noise-resistant model and the local noise-resistant model to obtain a target global model and a target local model includes:
[0030] Update the initial global model according to the global anti-noise model to obtain an intermediate global model;
[0031] Transfer the intermediate global model to the client, and perform anti-noise training on the initial local model according to the intermediate global model to obtain an intermediate local model;
[0032] Set the intermediate local model as the initial local model and perform anti-noise training;
[0033] Repeat the anti-noise training and model update steps until the target global model and target local model are obtained.
[0034] Further, in the step of transferring the target global model to the client and performing anti-noise training on the initial local model according to the target global model to obtain the target local model, the anti-noise training is weighted training on the local model based on joint loss evaluation.
[0035] An apparatus for ultrasonic echocardiogram segmentation based on federated learning, the apparatus for ultrasonic echocardiogram segmentation based on federated learning implements the steps of the method for ultrasonic echocardiogram segmentation based on federated learning described in any one of the above, including:
[0036] A model acquisition module, which acquires an initial data set, and performs anti-noise training on the initial local model according to the initial data set to obtain a local anti-noise model and determine the weight of the local anti-noise model;
[0037] A global anti-noise module, which is used to generate a global anti-noise model according to the local anti-noise model and the weight;
[0038] A target model module, which is used to perform iterative anti-noise updates on the initial global model and the initial local model according to the global anti-noise model and the local anti-noise model until the target global model and target local model are obtained;
[0039] A segmentation module, which is used to perform ultrasonic echocardiogram segmentation through the target local model.
[0040] An electronic device, including a processor, a memory, and a computer program stored on the memory and capable of running on the processor, where when the computer program is executed by the processor, it implements the steps of the method for ultrasonic echocardiogram segmentation based on federated learning described in any one of the above.
[0041] A computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the method for ultrasonic echocardiogram segmentation based on federated learning described in any one of the above.
[0042] This application has the following advantages:
[0043] In an embodiment of the present application, aiming at the disadvantages of limited training data for echocardiogram segmentation and easy leakage of centralized training privacy in the prior art, the present application provides an echocardiogram segmentation method based on federated learning. The echocardiogram segmentation method is implemented by at least two initial models. The initial models include a global initial model deployed on the server side and the local initial models deployed on the client side, and the method includes the steps of: obtaining an initial data set, and performing noise-resistant training on the initial local models according to the initial data set to obtain local noise-resistant models and determining the weights of the local noise-resistant models; generating a global noise-resistant model according to the local noise-resistant models and the weights; performing iterative noise-resistant updates on the initial global model and the initial local models according to the global noise-resistant model and the local noise-resistant models until a target global model and a target local model are obtained; and performing echocardiogram segmentation through the target local model. Through the noise-resistant multi-center collaborative echocardiogram segmentation method based on federated learning, multi-center collaborative segmentation can be performed while protecting privacy, and the noise resistance of federated learning can be improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] In order to more clearly illustrate the technical solutions of the present application, the drawings required for the description of the present application will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.
[0045] Figure 1 is a flowchart of the steps of an echocardiogram segmentation method based on federated learning provided by an embodiment of the present application;
[0046] Figure 2 is a schematic diagram of the federated learning framework and its working process of an echocardiogram segmentation method based on federated learning provided by an embodiment of the present application;
[0047] Figure 3 is a schematic diagram of the training process based on joint loss evaluation of an echocardiogram segmentation method based on federated learning provided by an embodiment of the present application;
[0048] Figure 4 is a schematic diagram of the module structure of an apparatus for echocardiogram segmentation based on federated learning provided by an embodiment of the present application;
[0049] Figure 5 is a schematic diagram of the structure of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0050] To make the objectives, features, and advantages of this application more apparent and understandable, the following provides a more detailed description of this application in conjunction with the accompanying drawings and specific implementation manners. Obviously, the described embodiments are some, but not all, of the embodiments of this application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in this application without creative efforts belong to the scope of protection of this application.
[0051] The inventors found through analysis of the prior art that: in the method of a unified data sharing platform, it is difficult to ensure data privacy. The data security and privacy protection of such a platform are difficult. Once the platform is attacked by hackers, all data is at risk; the methods of data encrypted sharing and blockchain collaboration consume huge amounts of computing resources and storage resources respectively, and the data processing efficiency is low; the method based on federated learning faces the problem of unknown data distribution. Although this unknowability can effectively protect data privacy, it also makes it difficult to discover the introduced noise. This noise is introduced into each medical center along with the federated learning process, affecting the effect of echocardiogram segmentation: especially when there are annotation noises in the data of some medical centers that affect the data of other clients, these noises may spread from a single client to the central server through the process of federated learning, resulting in a deviation of the global model. Such noise propagation may not only reduce the overall performance of the model, but also mislead clinical diagnosis and reduce the accuracy and reliability of diagnosis.
[0052] It should be noted that in any embodiment of the present invention, the federated learning algorithm is a multi-medical center data collaboration algorithm that has been widely studied at present. The data of each medical center remains local, without sharing data, only sharing the deep learning model trained by local data. The global model aggregates the local models to update the global parameters, and then distributes the global parameters to each local model to update the local parameters. In addition, the global model is stored on the central server, and the local model is stored on the client. This method can efficiently integrate the data resources of multiple medical centers while ensuring data privacy.
[0053] Refer to Figure 1 , which shows a method for echocardiogram segmentation based on federated learning provided by an embodiment of this application;
[0054] The method is implemented through at least two initial models. The initial models include a global initial model deployed on the server side and the local initial model deployed on the client side, and the method includes the steps:
[0055] S110. Obtain an initial data set, and perform noise-resistant training on the initial local model according to the initial data set to obtain a local noise-resistant model and determine the weight of the local noise-resistant model;
[0056] S120. Generate a global noise-resistant model based on the local noise-resistant model and the weights.
[0057] S130. Iteratively update the initial global model and the initial local model based on the global noise-resistant model and the local noise-resistant model until a target global model and a target local model are obtained.
[0058] S140. Perform echocardiogram segmentation using the target local model.
[0059] In an embodiment of the present application, aiming at the disadvantages of limited training data for echocardiogram segmentation and easy leakage of centralized training privacy in the prior art, the present application provides an echocardiogram segmentation method based on federated learning. The echocardiogram segmentation method is implemented through at least two initial models. The initial models include a global initial model deployed on the server side and the local initial model deployed on the client side, and the method includes the steps of: obtaining an initial data set, and performing noise-resistant training on the initial local model based on the initial data set to obtain a local noise-resistant model and determining the weights of the local noise-resistant model; generating a global noise-resistant model based on the local noise-resistant model and the weights; iteratively updating the initial global model and the initial local model based on the global noise-resistant model and the local noise-resistant model until a target global model and a target local model are obtained; performing echocardiogram segmentation using the target local model. Through the noise-resistant multi-center collaborative echocardiogram segmentation method of federated learning, multi-center collaborative segmentation can be performed while protecting privacy, and the noise resistance of federated learning can be improved.
[0060] The method constructs a federated learning framework, deploys a client model and a server-side deep learning model for model initialization; trains the model on the client side, and trains a noise-resistant local model based on a local loss evaluation algorithm; sends the client model parameters to the server, and realizes privacy protection without exchanging data; updates the global model on the server side; the global model sends the parameters to the local, and performs weighted training on the local samples based on the joint loss evaluation, so as to update the model in a noise-resistant manner. While realizing privacy protection, there is enough data to train the model, and the image can be accurately segmented.
[0061] Next, a method for echocardiogram segmentation based on federated learning in this exemplary embodiment will be further described.
[0062] As described in step S110 above, obtain an initial data set, and perform noise-resistant training on the initial local model based on the initial data set to obtain a local noise-resistant model and determine the weights of the local noise-resistant model.
[0063] It should be noted that to build a multi - data - center environment, echocardiogram sequences need to be collected from different devices, and representative image frames are extracted from the echocardiogram sequences. Since the heart shapes between adjacent frames are highly similar, this solution adopts a fixed - interval sampling strategy to ensure the diversity of samples.
[0064] As Figure 2 shown, to build a federated learning framework, a deep echocardiogram segmentation network is pre - trained on the initial dataset and used as the initial model, which is deployed on the server and each client respectively to obtain the global model on the server and the local models on the clients. In this framework, the central server and the clients all use homogeneous models.
[0065] In an embodiment of the present invention, the specific process of "obtaining an initial dataset, performing noise - resistant training on the initial local model according to the initial dataset, obtaining a local noise - resistant model, and determining the weights of the local noise - resistant model" in step S110 can be further described in combination with the following description.
[0066] As described in the following steps, an initial dataset is obtained, and the initial dataset is screened through preset conditions to obtain a screened dataset;
[0067] As described in the following steps, the screened dataset is pre - processed by standardization to obtain a grayscale image set with a unified size;
[0068] As described in the following steps, the grayscale image set is segmented and stored in several clients, where the data distributions among the several clients are different.
[0069] It should be noted that the quality of the acquired images is evaluated, and samples with poor imaging quality, low contrast, or blurred boundaries are removed. The screened images are pre - processed by standardization and converted into grayscale images with a unified size to construct an initial dataset. The initial dataset is stored in K clients respectively, and the data of each client needs to meet the following conditions: the data distributions among the clients are different; each client contains an accurate label sample set and a noisy label sample set to simulate a multi - data - center environment in reality.
[0070] In an embodiment of the present invention, the specific process of "pre - processing the screened dataset by standardization to obtain a grayscale image set with a unified size" can be further described in combination with the following description.
[0071] As described in the following steps, all the images in the screened dataset are pre - processed by standardization to obtain pre - processed images;
[0072] As described in the following steps, pixel - level semantic labels of heart tissue are annotated for the pre - processed image; wherein, the semantic labels of heart tissue include an accurate label sample set and a noisy label sample set.
[0073] It should be noted that in the annotation stage, pixel - level semantic labels of heart tissue are annotated for the images in the initial data set, including the accurate label sample set {D p |(x,y)} and the noisy label sample {D n |(x,y)}.
[0074] In an embodiment of the present invention, the specific process of the step "performing noise - resistant training on the initial local model according to the initial data set to obtain a local noise - resistant model and determining the weights of the local noise - resistant model" can be further described in combination with the following description.
[0075] As described in the following steps, the local model is trained according to the initial data set and the loss function is calculated to obtain a loss function set;
[0076] As described in the following steps, the loss functions in the loss function set are sorted, and a preset number of loss functions are taken to generate a pseudo - clean sample set;
[0077] As described in the following steps, the local model is fine - tuned according to the pseudo - clean sample set to obtain a local noise - resistant model;
[0078] As described in the following steps, the weights of the local noise - resistant model are determined according to the number of samples in the local noise - resistant model.
[0079] It should be noted that during the initial training of each client, there is no global model to guide the model update. To reduce the impact of annotation noise on the segmentation result, the initial training is divided into two stages, and the training is carried out separately on all clients, and the data used is all single - center data;
[0080] After training the local model using all the data, parameters Use this model to calculate the loss function for all the data and sort them, and take the samples with the smallest loss value to construct a pseudo - clean sample set {D c |(x,y)}, where N is the total number of labeled samples;
[0081] Use the pseudo - clean sample D c to fine - tune the local model:
[0082] Its training process can be expressed as:
[0083]
[0084] As described in the above step S120, a global anti-noise model is generated based on the local anti-noise model and the weights.
[0085] In an embodiment of the present invention, the specific process of "generating a global anti-noise model based on the local anti-noise model and the weights" described in step S120 can be further described in combination with the following description.
[0086] As described in the following steps, the local anti-noise model is sent to the server side;
[0087] As described in the following steps, the server side performs weighted aggregation on the local anti-noise models of all clients to obtain a global anti-noise model.
[0088] It should be noted that after obtaining the anti-noise local model it is uploaded to the central server, and a global anti-noise model is obtained based on weighted aggregation.
[0089] As described in the above step S130, the initial global model and the initial local model are subjected to anti-noise update based on the global anti-noise model and the local anti-noise model to obtain a target global model and a target local model.
[0090] In an embodiment of the present invention, the specific process of "iteratively performing anti-noise update on the initial global model and the initial local model based on the global anti-noise model and the local anti-noise model until a target global model and a target local model are obtained" described in step S130 can be further described in combination with the following description.
[0091] As described in the following steps, the initial global model is updated based on the global anti-noise model to obtain an intermediate global model;
[0092] As described in the following steps, the intermediate global model is transmitted to the client, and the initial local model is subjected to anti-noise training based on the intermediate global model to obtain the intermediate local model;
[0093] As described in the following steps, the intermediate local model is set as the initial local model and anti-noise training is performed;
[0094] As described in the following steps, the anti-noise training and model update steps are repeated until the target global model and the target local model are obtained.
[0095] It should be noted that the following processes are alternately performed in a loop. Only the model parameters need to be exchanged between the server and the client, and no data needs to be uploaded, ensuring data privacy.
[0096] In each round, denotes the global model parameters of this round, represents the local model parameters for this round. The following steps are the execution process for the r-th round. Assume that has been uploaded to the server.
[0097] Global model noise-resistant update:
[0098] After obtaining the noise-resistant local model it is uploaded to the central server, and a global noise-resistant model is obtained based on weighted aggregation.
[0099] Perform weighted aggregation of the model globally, adjust the weights of each local model according to the number of samples in the client, and obtain a new global model:
[0100]
[0101] where represents the number of clients participating in the update in the r-th round, and N k represents the number of data samples contained in the k-th client;
[0102] Local model noise-resistant update: Based on joint loss evaluation, update the local model in a noise-resistant manner;
[0103] Transfer the global model to the local. At this time, the local model can screen noise labels based on the global model. To perform local noise-resistant training according to the global model without introducing noise from other centers, a noise-resistant local model training process based on loss evaluation is designed.
[0104] The core idea is: During the training process, for each local sample, re-weighting is performed based on the loss gap between the global model and the local model in this round. The basis is: For data trained on clean samples, its loss value remains constant during multiple model trainings. Through this calculation method based on loss evaluation, the local model not only utilizes the capabilities of the global model but also effectively reduces the interference of global noise on training, improving the noise-resistant performance and generalization ability of the model.
[0105] This process is as Figure 3 shown and is expressed as:
[0106]
[0107] where η L represents the learning rate, λ represents the gradient weight, L s and L m represent the total loss and the weighted training loss respectively, and α represents the weight scaling factor.
[0108] In an embodiment of the present invention, the step of transmitting the target global model to the client and performing noise-resistant training on the initial local model according to the target global model to obtain the target local model, the noise-resistant training is weighted training on the local model based on joint loss evaluation.
[0109] The method for ultrasonic echocardiogram segmentation based on federated learning proposed by the present invention can perform multi-center collaborative segmentation while protecting privacy. Introducing a noise-resistant mechanism of joint loss evaluation can improve the noise resistance of federated learning, resist the intrusion of label noise, improve the performance of the ultrasonic echocardiogram segmentation model, assist clinical diagnosis, without introducing expert samples or redundant annotation costs, is easy to implement, has good generality and practical value, and can be extended and applied to other medical image segmentation tasks.
[0110] For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and for the relevant parts, refer to the partial description of the method embodiment.
[0111] Refer to Figure 4 , which shows a device for ultrasonic echocardiogram segmentation based on federated learning provided by an embodiment of the present application;
[0112] Specifically, it includes:
[0113] A model acquisition module 410, which acquires an initial data set, performs noise-resistant training on the initial local model according to the initial data set to obtain a local noise-resistant model and determines the weight of the local noise-resistant model;
[0114] A global noise-resistant module 420, which is used to generate a global noise-resistant model according to the local noise-resistant model and the weight;
[0115] A target model module 430, which is used to perform iterative noise-resistant update on the initial global model and the initial local model according to the global noise-resistant model and the local noise-resistant model until a target global model and a target local model are obtained;
[0116] A segmentation module 440, which is used to perform ultrasonic echocardiogram segmentation through the target local model.
[0117] In an embodiment of the present invention, the model acquisition module 410 includes:
[0118] A data set screening sub-module, which includes acquiring an initial data set and screening the initial data set through preset conditions to obtain a screened data set;
[0119] A preprocessing sub-module, which is used to perform standardized preprocessing on the screened data set to obtain a grayscale image set with a unified size;
[0120] A split storage sub-module for splitting and storing the grayscale image set in a number of clients, where the data distribution among the clients is different.
[0121] In an embodiment of the present invention, the preprocessing sub-module includes:
[0122] A normalization unit for performing normalization preprocessing on all images in the screened data set to obtain preprocessed images;
[0123] An annotation unit for annotating pixel-level semantic labels of heart tissue for the preprocessed images; wherein the semantic labels of heart tissue include an accurate label sample set and a noisy label sample set.
[0124] In an embodiment of the present invention, the model acquisition module 410 further includes:
[0125] A damage function sub-module for training the local model based on the initial data set and calculating the loss function to obtain a loss function set;
[0126] A sample set sub-module for sorting the loss functions in the loss function set and taking a preset number of loss functions to generate a pseudo-clean sample set;
[0127] A local anti-noise sub-module for fine-tuning the local model based on the pseudo-clean sample set to obtain a local anti-noise model;
[0128] A weight sub-module for determining the weight of the local anti-noise model according to the number of samples in the local anti-noise model.
[0129] In an embodiment of the present invention, the global anti-noise module 420 includes:
[0130] An anti-noise model generation sub-module for sending the local anti-noise model to the server side;
[0131] An aggregation sub-module for the server side to perform weighted aggregation on the local anti-noise models of all clients to obtain a global anti-noise model.
[0132] In an embodiment of the present invention, the target model module 430 includes:
[0133] A global model update sub-module for updating the initial global model based on the global anti-noise model to obtain a target global model;
[0134] A local model update sub-module for transmitting the target global model to the client and performing anti-noise training on the initial local model based on the target global model to obtain the target local model.
[0135] In an embodiment of the present invention, the global model update sub-module further includes:
[0136] A joint damage assessment unit for weighted training of the local model based on joint loss assessment.
[0137] Refer to Figure 5 , which shows a computer device for an echocardiogram segmentation method based on federated learning according to the present invention. Specifically, it may include the following:
[0138] The above computer device 12 is presented in the form of a general computing device. The components of the computer device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 connecting different system components (including the system memory 28 and the processing unit 16).
[0139] The bus 18 represents one or more of several types of bus 18 structures, including a memory bus 18 or a memory controller, a peripheral bus 18, a graphics acceleration port, a processor, or a local bus 18 using any bus 18 structure in a variety of bus 18 structures. For example, these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus 18, a Micro Channel Architecture (MAC) bus 18, an Enhanced ISA bus 18, a Video Electronics Standards Association (VESA) local bus 18, and a Peripheral Component Interconnect (PCI) bus 18.
[0140] The computer device 12 typically includes a variety of computer system-readable media. These media can be any available media accessible by the computer device 12, including volatile and non-volatile media, removable and non-removable media.
[0141] The system memory 28 may include computer system-readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The computer device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, a storage system 34 may be used for reading and writing non-removable, non-volatile magnetic media (commonly referred to as a "hard disk drive"). Although Figure 5 not shown in the figure, a disk drive for reading and writing a removable non-volatile disk (such as a "floppy disk") and an optical disk drive for reading and writing a removable non-volatile optical disk (such as a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 through one or more data media interfaces. The memory may include at least one program product having a set (e.g., at least one) of program modules 42 configured to perform the functions of the embodiments of the present invention.
[0142] A program / utilities 40 having a set (at least one) of program modules 42 can be stored, for example, in a memory. Such program modules 42 include - but are not limited to - an operating system, one or more application programs, other program modules 42, and program data. Each or some combination of these examples may include the implementation of a network environment. The program modules 42 generally execute the functions and / or methods in the embodiments described in the present invention.
[0143] The computer device 12 can also communicate with one or more external devices 14 (such as a keyboard, a pointing device, a display 24, a camera, etc.), can also communicate with one or more devices that enable medical staff to interact with the computer device 12, and / or can communicate with any device that enables the computer device 12 to communicate with one or more other computing devices (such as a network card, a modem, etc.). Such communication can be carried out through an input / output (I / O) interface 22. Moreover, the computer device 12 can also communicate with one or more networks (such as a local area network (LAN)), a wide area network (WAN), and / or a public network (such as the Internet) through a network adapter 20. As shown in the figure, the network adapter 20 communicates with other modules of the computer device 12 through a bus 18. It should be understood that although Figure 5 not shown in the figure, other hardware and / or software modules can be used in combination with the computer device 12, including but not limited to: microcode, device drivers, a redundant processing unit 16, an external disk drive array, a RAID system, a tape drive, and a data backup storage system 34, etc.
[0144] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing a method for ultrasonic echocardiogram segmentation based on federated learning provided by the embodiments of the present invention.
[0145] That is, when the above-mentioned processing unit 16 executes the above-mentioned program, it realizes: obtaining an initial data set, and performing noise-resistant training on the initial local model according to the initial data set to obtain a local noise-resistant model and determining the weights of the local noise-resistant model;
[0146] generating a global noise-resistant model according to the local noise-resistant model and the weights;
[0147] performing iterative noise-resistant updates on the initial global model and the initial local model according to the global noise-resistant model and the local noise-resistant model until a target global model and a target local model are obtained;
[0148] performing ultrasonic echocardiogram segmentation through the target local model.
[0149] In an embodiment of the present invention, the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements a method for ultrasonic echocardiogram segmentation based on federated learning provided in all embodiments of the present application:
[0150] That is, when the program is executed by a processor, it implements: obtaining an initial data set, and performing noise-resistant training on the initial local model according to the initial data set to obtain a local noise-resistant model and determining the weight of the local noise-resistant model;
[0151] Generating a global noise-resistant model according to the local noise-resistant model and the weight;
[0152] Performing iterative noise-resistant update on the initial global model and the initial local model according to the global noise-resistant model and the local noise-resistant model until a target global model and a target local model are obtained;
[0153] Performing ultrasonic echocardiogram segmentation through the target local model.
[0154] Any combination of one or more computer-readable media may be used. The computer-readable medium may be a computer signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive list) of the computer-readable storage medium include: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPOM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.
[0155] The computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries the computer-readable program code. Such a propagated data signal may take various forms, including - but not limited to - an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer-readable signal medium may also be any computer-readable medium other than the computer-readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.
[0156] Computer program code for performing the operations of the present invention may be written in one or more programming languages or combinations thereof. The above-mentioned programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on the healthcare provider's computer, partially on the healthcare provider's computer, executed as a stand-alone software package, partially on the healthcare provider's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the healthcare provider's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., connected through the Internet using an Internet service provider). Each embodiment in this specification is described in a progressive manner, and the key point of each embodiment is to illustrate the differences from other embodiments. For the same or similar parts among the embodiments, reference may be made to each other.
[0157] Although the preferred embodiments of the embodiments of the present application have been described, those skilled in the art can make additional changes and modifications once they learn the basic creative concepts. Therefore, the appended claims are intended to be construed as including the preferred embodiments and all changes and modifications falling within the scope of the embodiments of the present application.
[0158] Finally, it should also be noted that in this text, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or terminal device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or terminal device. Without further limitation, an element defined by the statement "comprising one..." does not exclude the existence of additional identical elements in the process, method, article or terminal device comprising the said element.
[0159] The above has introduced in detail a method and device for echocardiogram segmentation based on federated learning provided by the present application. Specific examples are used in this text to elaborate on the principle and implementation manner of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application; at the same time, for those of ordinary skill in the art, according to the idea of the present application, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present application.
Claims
1. An echocardiogram segmentation method based on federated learning, characterized in that, The echocardiogram segmentation method is implemented through at least two initial models. The initial models include a global initial model deployed on the server side and the local initial model deployed on the client side, and the method includes the steps of: Obtain an initial data set, and perform noise-resistant training on the initial local model according to the initial data set to obtain a local noise-resistant model and determine the weight of the local noise-resistant model; Generate a global noise-resistant model according to the local noise-resistant model and the weight; Perform iterative noise-resistant updates on the initial global model and the initial local model according to the global noise-resistant model and the local noise-resistant model until a target global model and a target local model are obtained; Perform echocardiogram segmentation through the target local model.
2. The method according to claim 1, wherein The step of obtaining the initial data set includes: Obtain an initial data set, and screen the initial data set through preset conditions to obtain a screened data set; Perform standardized preprocessing on the screened data set to obtain a grayscale image set with a unified size; Segment and store the grayscale image set in a number of clients, where the data distributions among the number of clients are different.
3. The method according to claim 2, characterized in that, The step of performing standardized preprocessing on the screened data set to obtain a grayscale image set with a unified size includes: Perform standardized preprocessing on all images in the screened data set to obtain preprocessed images; Annotate pixel-level heart tissue semantic labels for the preprocessed images; among them, the heart tissue semantic labels include an accurate label sample set and a noise label sample set.
4. The method according to claim 1, wherein The step of performing noise-resistant training on the initial local model according to the initial data set to obtain a local noise-resistant model and determine the weight of the local noise-resistant model includes: Train the local model according to the initial data set and calculate the loss function to obtain a loss function set; Sort the loss functions in the loss function set, and take a preset number of loss functions to generate a pseudo-clean sample set; Fine-tune the local model according to the pseudo-clean sample set to obtain a local noise-resistant model; Determine the weight of the local noise-resistant model according to the number of samples in the local noise-resistant model.
5. The method according to claim 1, wherein The step of generating a global noise-resistant model according to the local noise-resistant model and the weight includes: Send the local noise-resistant model to the server side; The server side performs weighted aggregation on the local noise-resistant models of all clients to obtain a global noise-resistant model.
6. The method according to claim 1, wherein The step of performing iterative noise-resistant updates on the initial global model and the initial local model according to the global noise-resistant model and the local noise-resistant model until a target global model and a target local model are obtained includes: Update the initial global model according to the global noise-resistant model to obtain an intermediate global model; Transfer the intermediate global model to the client, and perform noise-resistant training on the initial local model according to the intermediate global model to obtain an intermediate local model; Set the intermediate local model as the initial local model and perform noise-resistant training; Repeat the noise-resistant training and model update steps until the target global model and the target local model are obtained.
7. The method according to claim 6, wherein The step of transmitting the target global model to the client and performing noise-resistant training on the initial local model based on the target global model to obtain the target local model, where the noise-resistant training is weighted training on the local model based on joint loss evaluation.
8. An apparatus for echocardiogram segmentation based on federated learning, characterized in that, The steps for the device for ultrasonic echocardiogram segmentation based on federated learning to implement the method for ultrasonic echocardiogram segmentation based on federated learning according to any one of claims 1 to 7 include: A model acquisition module, which acquires an initial data set and performs noise-resistant training on the initial local model based on the initial data set to obtain a local noise-resistant model and determine the weight of the local noise-resistant model; A global noise-resistant module, which is used to generate a global noise-resistant model based on the local noise-resistant model and the weight; A target model module, which is used to perform iterative noise-resistant updates on the initial global model and the initial local model based on the global noise-resistant model and the local noise-resistant model until a target global model and a target local model are obtained; A segmentation module, which is used to perform ultrasonic echocardiogram segmentation through the target local model.
9. An electronic device, characterized in that, It includes a processor, a memory, and a computer program stored on the memory and capable of running on the processor. When the computer program is executed by the processor, the steps of the method for ultrasonic echocardiogram segmentation based on federated learning according to any one of claims 1-7 are implemented.
10. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium. When the computer program is executed by a processor, the steps of the method for ultrasonic echocardiogram segmentation based on federated learning according to any one of claims 1 to 7 are implemented.