Systems and methods for continuous joint learning based on deep learning

By working collaboratively with global and local sites and selectively tuning specific layers of the local model, the catastrophic forgetting problem of deep learning models when data distribution changes is solved, enabling efficient incremental learning in data privacy protection scenarios.

CN115879540BActive Publication Date: 2026-11-17GE PRECISION HEALTHCARE LLC
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202211145643.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2021-09-27
Filing Date
2022-09-20
Publication Date
2026-11-17
Estimated Expiration
2042-09-20

AI Technical Summary

Technical Problem

Existing deep learning models are prone to catastrophic forgetting when faced with changes in data distribution, and they struggle to perform effective incremental learning in data privacy protection scenarios.

Method used

A continuous joint learning system is adopted, which uses the model tuning module to selectively tune specific layers of the local model through the collaborative work of global and local sites, and generates a retrained model.

Benefits of technology

It enables rapid adaptation to new data and maintenance of model performance without sharing training data, avoiding catastrophic forgetting and improving the model's adaptability and accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115879540B_ABST
    Figure CN115879540B_ABST
Patent Text Reader

Abstract

This invention provides a deep learning-based continuous joint learning network system. The system includes a global station comprising a global model and multiple local stations having corresponding local models derived from the global model. Multiple model tuning modules with processing systems are positioned at the multiple local stations to tune the respective local models. The processing systems are programmed to receive incremental data and select one or more layers of the local models for tuning based on the incremental data. Finally, the selected layers are tuned to generate a retrained model.
Need to check novelty before this filing date? Find Prior Art

Description

Background Technology

[0001] The topics disclosed in this paper relate to deep learning techniques, and more specifically, to systems and methods for using deep learning techniques that leverage continuous joint learning with a distributed selective local retuning process.

[0002] Deep learning models have proven successful in solving problems involving sufficiently large, balanced, and labeled datasets in computer vision, speech processing, image processing, and other fields. Ideally, these models would continuously learn and adapt to new data, but this remains a challenge for neural network models because most of them are trained on large, static datasets. Retraining with incremental data often leads to catastrophic forgetting (i.e., training the model with new information interferes with previously learned knowledge).

[0003] Ideally, artificial intelligence (AI) learning systems should continuously adapt and learn new knowledge while improving upon existing knowledge. Current AI learning schemes assume that all samples are available during the training phase, thus requiring retraining of network parameters across the entire dataset to adapt to changes in data distribution. While retraining from scratch effectively addresses catastrophic forgetting, data privacy concerns in many real-world scenarios prevent the sharing of training data. In these cases, retraining with incrementally new data can lead to a significant loss of accuracy (catastrophic forgetting). Summary of the Invention

[0004] According to an implementation scheme of this technology, a deep learning-based continuous joint learning network system is provided. The system includes a global station comprising a global model; and multiple local stations having corresponding local models derived from the global model and multiple model tuning modules. Each of the multiple models includes a processing system programmed to receive incremental data and select one or more layers of the local model for tuning based on the incremental data. Finally, the selected layers in the local models are tuned to generate a retrained model.

[0005] According to another embodiment of the present technology, a method is provided. The method includes receiving a global model from a global site at multiple local sites, and deriving a local model from the global model at each of the multiple local sites. The method further includes tuning the corresponding local models at the multiple local sites. To tune the corresponding local models, incremental data is received from the local sites, and one or more layers of the local model are selected for tuning based on the incremental data. A retrained model is generated based on the tuning of the selected layers in the local model.

[0006] According to another embodiment of the present invention, a non-transitory computer-readable medium is provided, which stores instructions to be executed by a processor to perform a method. The method includes receiving a global model from a global site at multiple local sites, and deriving a local model from the global model at each of the multiple local sites. The method further includes tuning corresponding local models at the multiple local sites by receiving incremental data and selecting one or more layers of the local model for tuning based on the incremental data. A retrained model is generated based on the tuning of the selected layers in the local model. Attached Figure Description

[0007] These and other features, aspects, and advantages of the invention will be better understood when the following detailed description is read with reference to the accompanying drawings, in which the same reference numerals denote the same parts throughout the drawings, wherein:

[0008] Figure 1 It is an implementation scheme based on the schematic diagram of the continuous joint learning scheme or scenario of various aspects of this disclosure;

[0009] Figure 2 This is an implementation scheme of a flowchart for retraining a local model and a global model according to various aspects of this disclosure;

[0010] Figure 3 It is an implementation scheme of a system for generating a global model based on various aspects of this disclosure;

[0011] Figure 4 It is an embodiment of a graphical curve comparing the simulated feature distributions of the first and second outputs of the local model and the global model, respectively, according to various aspects of this disclosure;

[0012] Figure 5 This is a schematic diagram depicting the simulated scatter plot output of a DL model for two different datasets, based on various aspects of this disclosure; and

[0013] Figures 6A to 6C A schematic diagram of a segmented knee joint model and its results according to various aspects of this disclosure is shown. Detailed Implementation

[0014] One or more specific implementations will be described below. To provide a concise description of these implementations, not all features of an actual implementation will be described in this specification. It should be understood that in the development of any such actual implementation, as in any engineering or design project, many implementation-specific decisions must be made to achieve the developer's specific objectives, such as complying with system-related and business-related constraints that may differ from implementation to implementation. Furthermore, it should be understood that such development efforts may be complex and time-consuming, but remain routine tasks of design, fabrication, and manufacturing for those skilled in the art who benefit from this disclosure.

[0015] When describing elements of various embodiments of the invention, the articles “a,” “an,” “the,” and “the” are intended to indicate the presence of one or more of the elements. The terms “comprising,” “including,” and “having” are intended to be inclusive and mean that additional elements may be present in addition to the listed elements. Furthermore, any numerical examples in the following discussion are intended to be non-limiting, and therefore the additional values, ranges, and percentages are within the scope of the disclosed embodiments.

[0016] Some general information is provided to offer general background to the various aspects of this disclosure and to facilitate understanding and interpretation of certain technical concepts described herein.

[0017] In deep learning (DL), computer models learn to perform classification tasks directly from images, text, or sound. Deep neural networks combine feature representation learning and classifiers within a unified framework. It should be noted that the term "depth" typically refers to the number of hidden layers in a neural network. Traditional neural networks contain only 2-3 hidden layers, while deep networks can have up to 150. Models are trained using a large set of labeled data and a neural network architecture containing multiple layers, where the model learns features directly from the data without manual feature extraction. Neural networks are organized into layers consisting of a set of interconnected nodes. The output from a layer represents features that may have associated data values. As an incomplete example, features can be combinations of shape, color, appearance, texture, aspect ratio, etc.

[0018] Convolutional Neural Networks (CNNs) are processes used in deep learning where CNNs find patterns in data. They learn directly from the data, using patterns to classify items, thus eliminating the need for manual feature extraction. For example, a CNN can have dozens or hundreds of layers that learn to detect different features in images, text, sound, etc. Like other neural networks, a CNN consists of input layers and output layers, with multiple hidden layers in between. These hidden layers perform operations that modify the data to learn data-specific features. An example of a layer is a convolutional layer, which feeds input data through a set of convolutional filters, each of which activates certain features that form the image. For example, filters are applied to each training data at different resolutions, and the output of each convolutional data is used as the input to the next layer. These operations are repeated across dozens or hundreds of layers, with each layer learning to recognize different features. After learning features across multiple layers, the CNN moves on to classification and can provide a classification output.

[0019] The expectation is that models will continuously learn and adapt to new data, but this is a challenge for standard neural network models. This is especially true for healthcare or flight surveillance, where there is limited data, diversity in sample distributions, and limited or no access to training data. Transfer learning is a common framework for retraining models given new incoming data, but these models suffer from catastrophic forgetting. As those skilled in the art will know, catastrophic forgetting occurs when a model is trained with new information, and this interferes with previously learned knowledge. With catastrophic forgetting, the model “forgets” what it has learned and is only retuned to the incoming data. Therefore, the model is only trained on new information, so it learns on a smaller scale. Catastrophic loss of previously learned responses is particularly undesirable whenever an attempt is made to train the network with a new (additional) response.

[0020] Standard models are typically trained using static, large datasets. Conventional models assume all samples are available during training, thus requiring retraining of the network parameters across the entire dataset to adapt to changes in data distribution. While retraining from scratch effectively addresses catastrophic forgetting, this process is inefficient and hinders real-time learning on new data. Furthermore, data privacy concerns in many real-world scenarios prevent the sharing of training data. In these cases, retraining with incrementally new data can lead to a significant loss of accuracy (catastrophic forgetting).

[0021] Alternatively, standard deep learning models can be trained on centralized training data. The performance of deep learning models can be adversely affected by site-specific variables such as machine manufacturing, software version, patient baseline information, and site-specific clinical preferences. Continuous joint learning enables incremental site-specific tuning of the global model to create local versions. In continuous joint learning scenarios, the global model is deployed across multiple sites where data cannot be exported. Site-specific ground truths are generated using an automated synthesis processing model, which can utilize segmentation, registration machine learning, and / or deep learning models. Such ground truths may need to be refined based on the local preferences of experts.

[0022] Typically, a model can be retrained based on its last layer. The decision of which layer to retrain is usually made iteratively, which is time-consuming and may not lead to a single solution.

[0023] To address these issues, one or more implementations provide a data generation framework with a model tuning module that trains a local model. New incoming incremental data received by the model tuning module may affect some aspects of the local model but not others. Therefore, the model tuning module can retrain layers of the local model affected by the new incremental data, rather than retraining the entire model. Continuing with the citrus example above, for shape features, if the new model produces a shape similar to what you would expect to see (e.g., a circle), then that layer does not need retraining. However, if the image of the citrus shows an ellipse due to distortion from the new camera, then that layer may need retraining (i.e., the weights associated with the shape in that layer can be adjusted) to be able to recognize the shape of the citrus.

[0024] The model tuning module determines which nodes are available for retraining and which should be retained and no longer trained. It can partially retrain the model for newly input incremental data, while simultaneously analyzing model features to determine which layers need retraining or "tuning," and then inferring which layer needs tuning to maintain performance on previously trained tasks / data. Layer determination can be based on feature values ​​informing the layer weights. One or more implementations can provide local learning and faster adaptation to new data without catastrophic forgetting, while also enabling retraining in scenarios where training data cannot be shared.

[0025] Considering the foregoing, and by providing helpful context, Figure 1A schematic diagram of a continuous joint learning scheme 10 is shown. A standard deep learning model is trained on centralized training data. The performance of the DL model can be adversely affected by site-specific variability such as machine making, software version, patient basic information, and site-specific clinical preferences. As shown, the continuous joint learning scheme 10 includes a global site 12 (e.g., a central site or master site) and multiple local sites or nodes 14 (e.g., far from the global site 12). The global site 12 includes a global model 16 (e.g., a global neural network or machine learning model) trained on a master dataset 17 (e.g., a global dataset). Joint learning enables site-specific incremental tuning of the global model 16 (through local incremental learning on local data) to create a local version / model 18. In one implementation, a model tuning module 22 located at each local site in the local sites 14 tunes the global model based on local site data, as will be discussed later regarding... Figure 2 As explained, such local models are therefore more robust to site-specific variability. The local model 18 (e.g., a local neural network or machine learning model) from local site 14 is then further sent to the cloud using encrypted communication to fine-tune the global model 16. During this process, performance standards must be maintained across both the global and local test datasets.

[0026] In continuous joint learning scenario 10, a global model 16 is deployed across multiple sites 14 where data cannot be exported. Site-specific ground truths are generated using an automated synthesis processing model, which may employ segmentation, registration, machine learning, and / or deep learning models. The site-specific ground truths may need to be refined based on local expert preferences. The automatically generated and refined ground truths are then further used for local training of the model. Selective local updates to the weights of the global model 16 create local mutants 18 of the global model 16. The weights of the local model 18 are then encrypted and sent to a central server for selective updates to the global model 16, as shown in box 20. These local updates or site-specific preferences (e.g., weights) from the local sites 14 are combined when the global model 16 is updated at global site 12. Global model updates will be strategic and will depend on domain and industry-specific requirements.

[0027] Figure 2Flowcharts are provided for a process 100 for retraining local and global models according to some embodiments. Process 100 and any other processes described herein can be performed using any suitable combination of hardware (e.g., circuitry), software, or manual means. For example, a computer-readable storage medium may store instructions thereon that, when executed by a machine, produce the performance according to any embodiment described herein. In one or more embodiments, system 10 is regulated to perform process 100 such that the system is a dedicated element configured to perform operations that cannot be performed by a general-purpose computer or apparatus. The software embodying these processes can be stored by any non-transitory tangible medium including fixed disks, floppy disks, CDs, DVDs, flash drives, or magnetic tapes. Examples of these processes are described below with respect to embodiments of the system, but the embodiments are not limited thereto. The flowcharts described herein do not imply a fixed order of steps, and embodiments of the invention can be practiced in any feasible order.

[0028] Initially, at step 110, a global model is received at multiple local sites. In one implementation, the global model is a gold-tested training model, as will be discussed below. Figure 3 As explained. At step 112, a local model is derived at each of the multiple local sites based on the global model. In one implementation, deriving the local model involves making an exact copy of the global model (layers, nodes, and corresponding weights) at the local site. The distribution of features for each layer based on the global model can be referred to as the trained / expected feature distribution. The distribution of features can be visualized using any dimensionality reduction method (PCA, TSNE, etc.) in a scatter plot or any other suitable display. It should also be noted that the distribution of features may or may not be shown to the user, and is shown herein for ease of description.

[0029] Next, at step 114, incremental data is received at the model tuning module located at the local site. This incremental data is received from the corresponding local site and is different from the gold test data. The incremental data may include, for example, more pediatric brain scans compared to the global data. At step 116, one or more layers of the local model are selected for tuning based on the incremental data. Typically, a few cycles are run on the local model based on the incremental data. The model tuning module then compares the first output of a layer in the local model with the second output of the corresponding layer in the global model. Based on the variance between the first and second outputs, the model tuning module selects the local model layers that need to be tuned. For example, if the variance between the first and second outputs of a particular layer exceeds a threshold, the model tuning module will select that layer for tuning; otherwise, the model tuning module will freeze that layer, meaning that the layer will not be changed or retuned to different values.

[0030] In other words, to determine which layer should be tuned, the model tuning module can compare the trained feature distribution of each layer in the global model (i.e., the second output) with the feature distribution of the corresponding layer output from the local model (i.e., the first output). Layers output from the local model that have a feature distribution close to the trained feature distribution of the corresponding layer (determined by feature statistics such as mean, variance, etc.) can be frozen or not trained, while layers output from the local model that do not have a feature distribution close to the trained feature distribution can be selected by the model tuning module for training. In one or more implementations, to determine whether the feature distribution in a layer output from the local model is close to the trained feature distribution of the corresponding layer, the difference between the distributions can be compared with a user-defined threshold.

[0031] Finally, at step 118, the selected layers of the local model are tuned to generate a retrained model. In one or more embodiments, tuning the layers includes adjusting the weights associated with the nodes in the layer. The adjusted weights replace their corresponding weights in the global model to become the new retrained model. The retrained model is then tested on incremental data to ensure model accuracy. In one or more embodiments, the application of incremental data to the retrained model can confirm that the retraining is accurate at a given value or within a range of values ​​(e.g., the model is expected to operate with 95% accuracy). However, if the retrained model is not accurate enough, the model's weights are further modified to update the retrained model to meet accuracy requirements. Finally, the weights of the retrained models from all local sites are combined, encrypted, and sent to a central server for further training of the global model, as per [reference needed]. Figure 1 The explanation given.

[0032] Figure 3 A schematic diagram of a system 200 for generating a global model is shown. System 200 first generates a trained model 202. To generate the trained model 202, an untrained convolutional neural network model 202 is first received by system 200. The untrained model can be trained using a main dataset 204 from one or more data sources 206. The result of this training assigns weights to nodes in neural network model layers 208 through 216. The initial input layer 208 may include one or more nodes 210. Each node 210 may represent a variable assigned to a value during training, such that each node 210 has meaning depending on the task being learned. Depending on the input data from the main dataset 204, the layer can have any number of nodes greater than zero.

[0033] Typically, a convolutional neural network 202 consists of an input layer 208, hidden layers 212 and 214, and an output layer 216. In one or more implementations, nodes in the layers are weighted based on the importance of variables / nodes to the task. During training, the weights of the nodes are optimized in the hidden layers (212, 214). The training process ensures the optimization of the weights. In one or more implementations, the convolutional neural network is independent of the model architecture and can generalize to more or fewer layers. It should be noted that while the implementations and incomplete examples included herein can be described based on multiple layers, deep learning is typically described using architectures with multiple layers.

[0034] In one or more embodiments, once the convolutional neural network has been trained, the weights of the nodes in the initial input layer 208, any intermediate layers (e.g., second layer 212, third layer 214, etc.), and the output layer 216 are optimized to output a trained model 202. In one or more embodiments, the trained model 202 can then be executed with a set of gold test data 218 to confirm the accuracy of the trained model 202. The gold test data 218 is data that has been validated by a suitable party. When the output of the trained model 202 executed with the gold test data 218 matches the expected output within a given threshold, the trained model 202 can be classified as a global model 220.

[0035] Figure 4A graphical curve 300 is shown comparing the simulated feature distributions of the first and second outputs of the local and global models, respectively. As previously explained, the comparison between the first and second outputs is used for the purpose of selecting one or more layers of the local model for tuning. In the graph 300, there is a trained feature distribution of the global model (i.e., the second output), including positive samples 302 and negative samples 304. The trained feature distribution positive sample 302 can be distinguished from the trained feature distribution negative sample 304 by the trained classification line 306. Now, when the local model is initialized with new incremental data, the output of the local model is a new data distribution (i.e., the first output) including both the new data positive sample 308 and the new data negative sample 310. The new data positive sample 308 can be distinguished from the new data negative sample 310 by the new feature distribution line 312. If the difference between either the positive (308) or negative (310) new data sample distribution and the corresponding trained feature distribution (i.e., 302 and 304, respectively) is sufficiently large based on a comparison of the corresponding difference with a user-defined threshold, then the layer will be selected for tuning. However, if the difference between the trained feature distribution (302 and 304) and the new data samples (308 and 310) is not significant compared to the user-defined threshold, then the corresponding layer may remain untrained. When a layer is selected for tuning, in one or more embodiments, the trained feature distribution line 306 of that layer can be moved by adjusting the weights assigned to the feature of that node in the global model to better distinguish the feature, thereby matching the distribution of the new line 312.

[0036] Figure 5 A schematic diagram 400 shows the simulated scatter plot output of a DL model depicting two different datasets. Typically, schematic diagram 400 shows two random data distributions: a first data distribution 402 and a second data distribution 404, each with three class labels 406, 408, and 410. The horizontal axis 412 and the vertical axis 414 in plots 402 and 404 illustrate two distinct features of the data. For example, one feature could be color (on the horizontal axis 412), and the other could be aspect ratio (on the vertical axis 414). Figure 5 It can be seen that, compared with the second data distribution 404, the first data distribution 402 has relatively better class separation.

[0037] It should be noted that the DL model simulated in this paper includes a hidden layer and an output layer with three nodes (each node for one class). The DL model was trained for 100 epochs using a first data distribution with 50% of the data to assign labels. The class label accuracy of this DL model with validation data was 91%, and the class label accuracy of the DL model with 50% of the hold-out test data used for the first data distribution was 92%. As those skilled in the art will understand, during training, the data was divided into training and validation datasets. The performance of the trained model was validated using the validation dataset, and another dataset was retained entirely during training and validation. This retained dataset is called the hold-out test dataset and is retained to test the generality of the validated model on a new dataset. When validated with 50% of the samples on the hold-out test dataset, the performance of the DL model dropped to 80% of that of the second data distribution. Therefore, it can be seen that the accuracy of the original DL model decreases when tested on new data. Therefore, the technique presented in this paper selects certain layers of the original DL model for tuning, updates their weights according to a second data distribution, and thus improves the performance of the DL model for data classification.

[0038] Figures 6A to 6C A schematic diagram of a knee joint segmentation model according to an embodiment of this disclosure and its results is shown. Specifically, Figure 6A A knee segmentation model 500 trained on knee locator images from data acquired from five different local sites is shown. Model 500 consists of two parts, a U-net, followed by a shape autoencoder (AE) that outputs a segmentation mask. As those skilled in the art will understand, the U-net processes the intensity of the object, and the shape autoencoder learns the shape of the object. The underlying trained model 500 was retrained with new incoming data from the local sites in three different ways: (1) retraining the entire model, allowing all layers to retune their weights; (2) retuning only the weights of the shape AE layers while freezing the weights of the U-net layers; and (3) retuning only the weights of the U-net layers while freezing the weights of the shape AE layers.

[0039] Figure 6BGraph 502 shows the test module results for all three different models described above. As can be seen from Graph 502, the test module values ​​are lower for the fully retrained model compared to models that only tune the AE layer or only tune the U-net layer. In other words, partial training of selected layers achieves better overall test accuracy compared to retraining all layers. Furthermore, it can be seen that tuning the U-net layer shows the best accuracy for this model. This is because the shape AE layer handles the shape of the object, and the shape of the object does not change from one local site to another. Since the U-net layer is the only layer that handles the object's intensity, tuning the U-net layer shows better accuracy for this model.

[0040] Figure 6C A schematic diagram 504 depicts the outputs of all three different models described above. In these schematic diagrams, the ground reality is shown via a gray mask 506, and the mask predicted by the model is shown by a dark gray mask 508 overlaid on the gray mask 506. Optimal performance will occur when the first and second shadow regions completely overlap each other. As can be seen from schematic diagram 504, the retrained model exhibits optimal performance when only the U-net layer is retrained while keeping the weights of the AE layer frozen; specifically, U-net layer retraining shows the best overlap between the ground reality and the predicted shape mask compared to retraining the entire model or only the AE layer.

[0041] One advantage of this technique is that retraining only specific layers of the DL model based on features ensures that the model can quickly adapt to local data. This is particularly useful in scenarios where 1) site-specific customization is required, and 2) local data may not be shared with the global source to allow for global model retraining with all new and old data, as described further below.

[0042] Referring to the technology presented herein and protected by the claims, and applying it to physical objects and concrete examples of practical nature, which explicitly improves the present art, it is therefore not abstract, intangible, or purely theoretical. Furthermore, if any claim appended to the end of this specification contains one or more elements designated as “means for [performing]...the function” or “steps for [performing]...the function,” such elements are intended to be interpreted pursuant to Section 35, Section 112(f) of the USC. However, for any claim containing elements designated in any other manner, such elements are not intended to be interpreted pursuant to Section 35, Section 112(f) of the USC.

[0043] This written description uses examples to disclose the subject matter, including best practices, and also enables those skilled in the art to practice the subject matter, including making and using any device or system and performing any included methods. The patent scope of this subject matter is defined by the claims and may include other examples that would occur to those skilled in the art. Such other examples are intended to fall within the scope of the claims if they have structural elements that are not indistinguishable from the literal language of the claims, or if they include equivalent structural elements that have minor differences from the literal language of the claims.

Claims

1. A deep learning-based continuous joint learning network system for knee joint segmentation, the continuous joint learning network system comprising: A global site, which includes a global model, wherein the global model is a global knee joint segmentation model; Multiple local sites, wherein each of the multiple local sites includes a corresponding local model derived from the global model, wherein the local model is a local knee joint segmentation model; Multiple model tuning modules, located at multiple local sites, are provided for tuning corresponding local models. Each of the multiple local models includes a processing system programmed to: Incremental knee joint image data are received at the model tuning module; Based on the incremental knee joint image data, one or more layers of the local model are selected for tuning by comparing the first output of the local model's layer with the second output of the corresponding layer of the global model. The selected layers in the local model are tuned to generate a retrained model.

2. The continuous joint learning network system of claim 1, wherein the processing system is programmed to select one or more layers in the local model in such a way as: Whether to select the layer of the local model for tuning is determined based on the variance between the first output and the second output.

3. The continuous joint learning network system of claim 1, wherein the processing system is programmed to select one or more layers in the local model in such a way as: The first output is compared with the second output to generate a comparison value, and when the comparison value exceeds a threshold, the layer of the local model is selected for tuning.

4. The continuous joint learning network system of claim 1, wherein tuning the selected layers in the local model includes adjusting the weights of the nodes of one or more selected layers.

5. The continuous joint learning network system of claim 4, wherein the updated weights of the nodes of the retrained model at each local site in the local sites are combined to further retrain the global model.

6. The continuous joint learning network system according to claim 1, wherein the processing system is further programmed to: Apply the gold test dataset to the retrained model; and The application based on the gold test dataset determines the accuracy of the retrained model.

7. The continuous joint learning network system of claim 6, wherein when the accuracy of the retrained model does not exceed an accuracy threshold, the processing system is programmed to further tune the selected layer to satisfy the accuracy threshold.

8. The continuous joint learning network system according to claim 1, wherein the global model is generated by a trained model, the trained model being tested on a set of gold test data.

9. The continuous joint learning network system of claim 8, wherein the trained model comprises an initial input layer, a plurality of intermediate layers and an output layer, and wherein the output of each of the initial input layer, the plurality of intermediate layers and the output layer represents an image feature having an associated data value therewith.

10. A method for segmenting a knee joint, the method comprising: Receive a global model from a global site at multiple local sites, wherein the global model is a global knee joint segmentation model; A local model is derived from the global model at each of the plurality of local sites, wherein the local model is a local knee joint segmentation model; Tuning the corresponding local models at the plurality of local sites, wherein tuning the corresponding local models includes: Receive incremental knee joint image data; Based on the incremental knee joint image data, one or more layers of the local model are selected for tuning by comparing the first output of the local model's layer with the second output of the corresponding layer of the global model. The selected layers in the local model are tuned to generate a retrained model.

11. The method of claim 10, wherein selecting one or more layers in the local model comprises: Whether to select the layer of the local model for tuning is determined based on the variance between the first output and the second output.

12. The method of claim 10, wherein selecting one or more layers in the local model comprises: The first output is compared with the second output to generate a comparison value, and when the comparison value exceeds a threshold, the layer of the local model is selected for tuning.

13. The method of claim 10, wherein tuning the selected layer in the local model includes adjusting the weights of nodes in one or more of the selected layers.

14. The method of claim 13, wherein the updated weights of the nodes of the retrained model at each local site in the local site are combined to further train the global model.

15. The method of claim 10, further comprising: The gold test dataset is applied to the retrained model; as well as The application based on the gold test dataset determines the accuracy of the retrained model.

16. The method of claim 15, wherein if the accuracy of the retrained model does not exceed an accuracy threshold, the selected layer is further tuned to satisfy the accuracy threshold.

17. The method of claim 10, wherein the global model is generated by a trained model, the trained model being tested against a set of gold test data.

18. The method of claim 17, wherein the trained model comprises an initial input layer, a plurality of intermediate layers and an output layer, and wherein each of the initial input layer, the plurality of intermediate layers and the output layer comprises one or more nodes having weights with associated data values.

19. The method of claim 18, wherein the features of the knee joint image include at least one of the image's shape, color, appearance, texture, aspect ratio, or combinations thereof.

20. A non-transitory computer-readable medium for knee joint segmentation, the non-transitory computer-readable medium storing instructions to be executed by a processor to perform a method for knee joint segmentation, the method comprising: Receive a global model from a global site at multiple local sites, wherein the global model is a global knee joint segmentation model; A local model is derived from the global model at each of the plurality of local sites, wherein the local model is a local knee joint segmentation model; Tuning the corresponding local models at the plurality of local sites, wherein tuning the corresponding local models includes: Receive incremental knee joint image data; Based on the incremental knee joint image data, one or more layers of the local model are selected for tuning by comparing the first output of the local model's layer with the second output of the corresponding layer of the global model. The selected layers in the local model are tuned to generate a retrained model.

Citation Information

Patent Citations

  • Federated modeling method based on selective gradient updating and related equipment

    CN112446040A