Medical image classification method, device and system based on federated learning

Abstract

The application relates to the technical field of medical image classification, and specifically discloses a medical image classification method, device and system based on federated learning, which comprises the following steps: acquiring medical image information; pre-processing the medical image information to obtain a medical image multi-resolution graph structure; inputting the medical image multi-resolution graph structure and the medical image information into a multi-resolution hybrid encoder based on federated learning for feature extraction; the multi-resolution hybrid encoder based on federated learning is obtained after the multi-resolution hybrid encoder receives the aggregation result of the federated learning of a server and is iteratively updated; the multi-resolution hybrid encoder is obtained after the multi-resolution hybrid encoder is trained based on a multi-view weight loss function according to a stacked sparse autoencoder and a sparse graph autoencoder; and classification processing is performed according to the multi-view feature of the medical image. The medical image classification method based on federated learning can effectively capture the details of medical images to improve the classification performance.

Description

Technical Field

[0001] This invention relates to the field of medical image classification technology, and in particular to a medical image classification method, a medical image classification device, and a medical image classification system based on federated learning. Background Technology

[0002] Medical image classification, as an important research direction at the intersection of medical image processing and artificial intelligence, has undergone rapid evolution in its research methods, from traditional image processing techniques to intelligent assisted diagnostic systems based on deep learning, thanks to advancements in imaging equipment technology and breakthroughs in computer vision algorithms. Early medical image classification methods primarily relied on manually extracting image features and combining them with traditional machine learning algorithms for classification. However, these methods had significant limitations in feature representation capabilities, robustness, and generalization performance, making them ill-suited for complex and diverse medical image data.

[0003] In recent years, deep learning has garnered widespread attention and profoundly impacted the development of machine learning due to its powerful ability to automatically learn data features. As an important unsupervised algorithm in deep learning, the autoencoder (AE) plays a crucial role in extracting effective features from unlabeled data. The goal of an autoencoder is to compress input data into a low-dimensional representation (encoder) and then reconstruct output data (decoder) that is as similar as possible to the original input data using this low-dimensional representation. This encoding and decoding process allows the autoencoder to learn the key features of the data. Due to its simple structure, convenient training, and excellent generalization ability, AE has been widely used in medical image analysis. Currently, AE and its various improved algorithms are widely applied in multiple fields such as data classification.

[0004] In medical image classification tasks, autoencoders have been widely used due to their superior feature extraction capabilities, especially in the context of massive datasets and high annotation costs. However, when dealing with multi-resolution medical images and distributed data, numerous factors can affect the accuracy and stability of classification, leading to fluctuations in model performance. Although traditional autoencoder (AE) models can effectively extract the features required for classification, they often require a trade-off between stability and accuracy, particularly when processing multi-resolution images (such as CT and MRI) and large-scale distributed data. Furthermore, medical images are often distributed across different hospitals or institutions, involving sensitive and private information, making it difficult to centralize data for unified modeling.

[0005] To address these issues, Federated Learning (FL), an emerging distributed learning paradigm, enables model training without direct access to raw data, significantly enhancing data privacy protection. As a decentralized distributed machine learning framework, Federated Learning aims to allow multiple clients (such as hospitals and medical devices) to independently train models locally, then aggregate updated model parameters to a central server for integration, without sharing the original data. This allows for global model training while protecting data privacy and security. In image classification tasks, Federated Learning allows devices to train models on local data, avoiding the sharing of image data and effectively protecting user privacy. By aggregating model updates from different devices, Federated Learning not only improves the performance of the global model but also enhances its robustness and personalization. However, current medical image classification still faces multiple challenges. On the one hand, frequent transmission of model parameters between hospitals and the central server incurs significant communication overhead, especially with a large number of devices or high model complexity. On the other hand, the inconsistent distribution of medical data makes it difficult for models to effectively capture global features. Furthermore, single-resolution models struggle to fully extract rich information from medical images (such as details of lesion areas), limiting the improvement of classification performance.

[0006] Therefore, how to provide a medical image classification method that can effectively capture medical image details to improve classification performance has become a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0007] This invention provides a medical image classification technology that solves the problem that medical image classification in related technologies cannot effectively capture the details of medical images.

[0008] As a first aspect of the present invention, a medical image classification method based on federated learning is provided, comprising:

[0009] Acquire medical imaging information, including images and / or videos;

[0010] The medical image information is preprocessed to obtain a multi-resolution image structure of the medical image;

[0011] The medical image multi-resolution map structure and the medical image information are both input into a federated learning-based multi-resolution hybrid encoder for feature extraction to obtain multi-view features of the medical image. The federated learning-based multi-resolution hybrid encoder is obtained by updating and iterating after receiving the aggregation results of federated learning from the server. The multi-resolution hybrid encoder is obtained by training based on a stacked sparse autoencoder and a sparse graph autoencoder based on a multi-view weight loss function. The sparse graph autoencoder is obtained by training based on a graph autoencoder with added sparsity constraints.

[0012] The medical images are classified based on their multi-view features to obtain classification results.

[0013] Furthermore, the federated learning-based multi-resolution hybrid encoder is obtained by updating and iterating the aggregated results of federated learning from the server, including:

[0014] A multi-resolution hybrid encoder is obtained by training a stacked sparse autoencoder and a sparse graph autoencoder based on a multi-view weight loss function. The multi-resolution hybrid encoder can extract features from the input medical image multi-resolution graph structure and the medical image information to obtain multi-view features.

[0015] Receive the initial global model parameters from the server, and train the multi-resolution hybrid encoder according to the initial global model parameters to obtain local model parameters;

[0016] The local model parameters are sent to the server, and the server can perform federated learning based on multiple local model parameters to obtain updated global model parameters.

[0017] The system receives the updated global model parameters returned by the server and iteratively trains the multi-resolution hybrid encoder based on the updated global model parameters to obtain a federated learning-based multi-resolution hybrid encoder. The federated learning-based multi-resolution hybrid encoder can obtain the multi-view features of the medical image after iterative training of the multi-view features based on the updated global model parameters.

[0018] Furthermore, a multi-resolution hybrid encoder is obtained by training the stacked sparse autoencoder and the sparse graph autoencoder based on the multi-view weight loss function, including:

[0019] By adding sparsity constraints to the weighted graph reconstructed from the encoded features of the graph autoencoder, a sparse graph autoencoder is obtained.

[0020] Determine the loss function of the stacked sparse autoencoder;

[0021] The multi-view weight loss function is determined based on the loss function of the stacked sparse autoencoder and the loss function of the sparse graph autoencoder.

[0022] The multi-view weight loss function is iteratively trained to obtain a multi-resolution hybrid encoder. Further, the expression for the loss function of the sparse graph autoencoder is:

[0023] J SGAE =J GAE +λJ GAE-sp ,

[0024] Among them, J SGAE J represents the loss function of a sparse graph autoencoder. GAE This represents the loss function of a graph autoencoder. N represents the number of nodes in the graph, and A represents the original adjacency matrix of the graph. J represents the reconstructed adjacency matrix of the graph. GAE-sp This represents the sparse constraint loss function of a graph autoencoder. λ represents the hyperparameter used to balance the reconstruction loss term and the sparse constraint term;

[0025] The expression for the loss function of the stacked sparse autoencoder is:

[0026]

[0027] Among them, J SSAE Let J(W,b) represent the loss function of the stacked sparse autoencoder, and J(W,b) represent the reconstruction error. sparse Let represent the sparsity penalty term, β represent the sparsity penalty factor, S represent the dimension of the hidden layer, and ρ represent the target sparsity. denoted as the average activation value, m as the number of training samples, x as the input image sample, and z as the output reconstruction result.

[0028] Furthermore, the expression for the multi-view weight loss function is:

[0029]

[0030] in, Let W1 represent the weights of the input x, W2 represent the biases of the input x, and σ represent the multi-view weight loss function. SSAE σ represents the noise parameter of a stacked sparse autoencoder. SGAE y represents the noise parameter of the sparse graph autoencoder. SSAE y represents the target value of the stacked sparse autoencoder. SGAE This represents the target value of the sparse graph autoencoder. This represents the output of the sparse graph autoencoder. J represents the output of the stacked sparse autoencoder. SGAE J represents the loss function of a sparse graph autoencoder. SSAE This represents the loss function of a stacked sparse autoencoder.

[0031] Furthermore, the server can perform federated learning based on multiple local model parameters to obtain updated global model parameters, including:

[0032] The weighted average result is obtained by weighting multiple local model parameters according to the federated index weighted average algorithm.

[0033] The initial global model parameters are updated based on the weighted average result to obtain the updated global model parameters, the expression of which is:

[0034]

[0035] Among them, w t w represents the updated global model parameters. t-1 Indicates local model parameters. represents the global model parameters, and μ represents the momentum coefficient used to characterize the impact of the current local model parameter update on the global model parameters.

[0036] Further, the medical image information is preprocessed to obtain a multi-resolution image structure of the medical image, including:

[0037] The relationship between image pixels and image pixel regions in the medical image information is converted into a medical image graph structure;

[0038] The medical image structure is constructed using a multi-resolution map to obtain a multi-resolution medical image structure.

[0039] Furthermore, classification processing is performed based on the multi-view features of the medical images to obtain medical image classification results, including:

[0040] The medical image multi-view features are sent to the server. The server can perform unified feature extraction on the medical image multi-view features according to the updated global model, and input the extracted unified features into the classifier for classification to obtain the medical image classification result.

[0041] Receive the medical image classification results returned by the server.

[0042] As another aspect of the present invention, a federated learning-based medical image classification device is provided for implementing the federated learning-based medical image classification method described above, wherein the device includes:

[0043] The acquisition module is used to acquire medical imaging information, including images and / or videos;

[0044] The preprocessing module is used to preprocess the medical image information to obtain the multi-resolution image structure of the medical image;

[0045] The feature extraction module is used to input the multi-resolution map structure of the medical image and the medical image information into a federated learning-based multi-resolution hybrid encoder for feature extraction, thereby obtaining multi-view features of the medical image. The federated learning-based multi-resolution hybrid encoder is obtained by updating and iterating after receiving the aggregation results of federated learning from the server. The multi-resolution hybrid encoder is obtained by training based on a stacked sparse autoencoder and a sparse graph autoencoder based on a multi-view weight loss function. The sparse graph autoencoder is obtained by training based on a graph autoencoder with added sparsity constraints.

[0046] The classification module is used to perform classification processing based on the multi-view features of the medical images to obtain medical image classification results.

[0047] In another aspect, a federated learning-based medical image classification system is provided, comprising a client and a server, wherein the client is communicatively connected to the server, the client including the aforementioned federated learning-based medical image classification device and a federated learning-based multi-resolution hybrid encoder, and the server including a federated exponential weighted average algorithm model, wherein the federated learning-based multi-resolution hybrid encoder is obtained by training the multi-resolution hybrid encoder and the federated exponential weighted average algorithm model.

[0048] The federated learning-based medical image classification device can call a federated learning-based multi-resolution hybrid encoder to classify the input medical image information and obtain the medical image classification result.

[0049] The medical image classification method based on federated learning provided by this invention acquires medical image information and preprocesses it to obtain a multi-resolution image structure. Both the medical image information and the multi-resolution image structure are input into a pre-obtained multi-resolution hybrid encoder based on federated learning. The multi-resolution hybrid encoder is obtained by updating and iterating the multi-resolution hybrid encoder after receiving the aggregation results of federated learning from the server. The multi-resolution hybrid encoder is obtained by training a stacked sparse autoencoder and a sparse graph autoencoder based on a multi-view weight loss function. The sparse graph autoencoder is obtained by training a graph autoencoder with added sparsity constraints. Because the multi-resolution hybrid encoder obtained by stacking sparse autoencoders and sparse graph autoencoders can simultaneously capture the detailed features of multi-resolution images and the global relationships of graph structure data, the federated learning-based multi-resolution hybrid encoder can significantly improve feature extraction capabilities, providing richer feature representations for classification tasks. Furthermore, the multi-view weight loss function can deeply fuse graph structure information with image feature information, optimizing the consistency of feature representations. Additionally, the dynamic adjustment of model update weights through server-side aggregation results based on federated learning effectively alleviates performance fluctuations caused by uneven data distribution in a distributed environment. Therefore, the federated learning-based medical image classification method provided in this invention effectively captures medical image details to improve classification performance using the multi-resolution hybrid encoder, enhances the model's discriminative ability through the multi-view weight loss function to improve the robustness of classification performance, and finally, dynamically adjusts model update weights through server-side aggregation results based on federated learning, effectively mitigating performance fluctuations caused by uneven data distribution in a distributed environment, improving model convergence speed and training stability, thus enabling the classification method to obtain more effective and accurate classification results. Attached Figure Description

[0050] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the following detailed description to explain the invention, but do not constitute a limitation thereof.

[0051] Figure 1 A flowchart of the federated learning-based medical image classification method provided by this invention.

[0052] Figure 2 This invention provides a flowchart for preprocessing medical image information.

[0053] Figure 3 This is an example diagram of the three-layer multi-resolution image structure provided by the present invention.

[0054] Figure 4The flowchart illustrates the process of obtaining a multi-resolution hybrid encoder based on federated learning, as provided in this invention.

[0055] Figure 5 The flowchart for training to obtain a multi-resolution hybrid encoder provided by the present invention.

[0056] Figure 6 This is a schematic diagram of the structure of the SGAE provided by the present invention.

[0057] Figure 7 This is a structural diagram of the multi-resolution hybrid automatic encoder provided by the present invention.

[0058] Figure 8 This is a flowchart for classifying medical images based on multi-view features, provided by the present invention.

[0059] Figure 9 The experimental results provided by this invention are shown in the figure.

[0060] Figure 10 The structural block diagram of the medical image classification device based on federated learning provided by the present invention. Detailed Implementation

[0061] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0062] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0063] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of the invention described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0064] This embodiment provides a medical image classification method based on federated learning. Figure 1 This is a flowchart of a federated learning-based medical image classification method provided according to an embodiment of the present invention, such as... Figure 1 As shown, it includes:

[0065] S100. Acquire medical imaging information, wherein the medical imaging information includes images and / or videos;

[0066] In this embodiment of the invention, the medical imaging information can be an image or a video, such as an X-ray of the lungs, a CT image, an ultrasound image, or a dynamic ultrasound video, etc.

[0067] S200: Preprocess the medical image information to obtain a multi-resolution image structure of the medical image;

[0068] In this embodiment of the invention, the medical image information is preprocessed, specifically by processing the medical image information into a graph structure to obtain a multi-resolution graph structure of the medical image.

[0069] S300. The medical image multi-resolution graph structure and the medical image information are both input into a federated learning-based multi-resolution hybrid encoder for feature extraction to obtain multi-view features of the medical image. The federated learning-based multi-resolution hybrid encoder is obtained by updating and iterating after receiving the aggregation results of federated learning from the server. The multi-resolution hybrid encoder is obtained by training based on a stacked sparse autoencoder and a sparse graph autoencoder based on a multi-view weight loss function. The sparse graph autoencoder is obtained by training based on a graph autoencoder with added sparsity constraints.

[0070] In this embodiment of the invention, the medical image multi-resolution map structure and medical image information are both input into a pre-obtained federated learning-based multi-resolution hybrid encoder. This federated learning-based multi-resolution hybrid encoder can automatically extract image features based on the medical image multi-resolution map structure and medical image information to obtain multi-view features of medical images.

[0071] Specifically, the federated learning-based multi-resolution hybrid encoder is obtained by updating and iterating the multi-resolution hybrid encoder after receiving the aggregation results of federated learning from the server. The multi-resolution hybrid encoder is obtained by training a stacked sparse autoencoder and a sparse graph autoencoder based on a multi-view weight loss function. The sparse graph autoencoder is obtained by training a graph autoencoder with added sparsity constraints. It should be understood that the multi-resolution hybrid encoder obtained by combining stacked sparse autoencoders (SSAE) and sparse graph autoencoders (SGAE) can simultaneously capture the detailed features of multi-resolution images and the global relationships of graph structure data. This hybrid structure significantly improves feature extraction capabilities, providing richer feature representations for classification tasks. Furthermore, the multi-view weight loss function can deeply fuse graph structure information with image feature information, optimizing the consistency of feature representations. This mechanism not only enhances the model's discriminative ability but also improves the robustness of classification performance. Finally, the model update weights are dynamically adjusted based on the server-side federated learning aggregation results, effectively mitigating the performance fluctuation problem caused by uneven data distribution in a distributed environment and improving the model's convergence speed and training stability.

[0072] S400. Classify the medical images according to the multi-view features to obtain the medical image classification results.

[0073] The extracted multi-view features of medical images are input into a classifier for classification processing to obtain the medical image classification results.

[0074] In summary, the federated learning-based medical image classification method provided by this invention acquires medical image information and preprocesses it to obtain a multi-resolution graph structure of the medical image. Both the medical image information and the multi-resolution graph structure are input into a pre-obtained federated learning-based multi-resolution hybrid encoder. The federated learning-based multi-resolution hybrid encoder is obtained by updating and iterating the multi-resolution hybrid encoder after receiving the aggregation results of federated learning from the server. The multi-resolution hybrid encoder is obtained by training based on a stacked sparse autoencoder and a sparse graph autoencoder using a multi-view weight loss function. The sparse graph autoencoder is obtained by training based on a graph autoencoder with added sparsity constraints. Because the multi-resolution hybrid encoder obtained by stacking sparse autoencoders and sparse graph autoencoders can simultaneously capture the detailed features of multi-resolution images and the global relationships of graph structure data, the federated learning-based multi-resolution hybrid encoder can significantly improve feature extraction capabilities, providing richer feature representations for classification tasks. Furthermore, the multi-view weight loss function can deeply fuse graph structure information with image feature information, optimizing the consistency of feature representations. Additionally, the dynamic adjustment of model update weights through server-side aggregation results based on federated learning effectively alleviates performance fluctuations caused by uneven data distribution in a distributed environment. Therefore, the federated learning-based medical image classification method provided in this invention effectively captures medical image details to improve classification performance using the multi-resolution hybrid encoder, enhances the model's discriminative ability through the multi-view weight loss function to improve the robustness of classification performance, and finally, dynamically adjusts model update weights through server-side aggregation results based on federated learning, effectively mitigating performance fluctuations caused by uneven data distribution in a distributed environment, improving model convergence speed and training stability, thus enabling the classification method to obtain more effective and accurate classification results.

[0075] In embodiments of the present invention, such as Figure 2 As shown, the medical image information is preprocessed to obtain a multi-resolution image structure of the medical image, including:

[0076] S210. Convert the relationship between image pixels and image pixel regions in the medical image information into a medical image graph structure;

[0077] Specifically, the graph representation of an image transforms the relationships between pixels or regions of an image into a graph form. Suppose we have a medical image of size H×W×3, which can be divided into N blocks, each block being considered a node in the graph. The feature matrix x of each node... i It can be the average RGB value of all pixels within the block, thus obtaining Where d represents the feature dimension, the nodes in the image form an unordered set, represented as... Then each node v i Its K nearest neighbors By connecting nodes with edges, a globally connected graph is formed. Node v j and its neighbor node v i The edge e between ji It can be constructed based on distance, similarity, or other metrics, and Finally, an undirected graph is obtained. It contains a set of nodes Sum of edges

[0078] S220. Perform multi-resolution image construction on the medical image structure to obtain a multi-resolution image structure of the medical image.

[0079] Specifically, given an undirected weighted graph in and The node set and edge set of a graph can be represented by an adjacency matrix. To represent the structure of the graph, This represents the number of nodes in the graph. The element A of the adjacency matrix... ij Represents node v i to node v j Edge weights between nodes. For an undirected graph, if node v i to node v j If there is an edge between them, then A ij =A ji =1 otherwise 0. To construct a multi-resolution graph representation, the complexity of the graph is gradually reduced through a coarsening process until it is simplified to a single supernode, thus obtaining graphs at different resolution levels. For example... Figure 3 As shown, multi-resolution representation of an image can be achieved by dividing the original image into multiple layers. Each layer is a coarsened version of the original image, with the number of nodes gradually decreasing until the most simplified image is obtained. This results in an L-layer image of the image.

[0080] In multi-resolution maps, K-means clustering is used to partition the map. After coarsening, the image is obtained. K-cluster partitioning will divide the node set Divide into K mutually exclusive clusters, denoted as Each cluster resembles an induced subgraph. The image after coarsening There are K nodes, each node Representative from the original image The resulting inducible graph During the coarsening process, each cluster is treated as a new supernode, and the adjacency matrix of the new graph can be obtained by calculating the edge weights between each cluster. Assuming a multi-resolution graph has three layers, the graph is gradually coarsened from the bottom layer (highest resolution) to the top layer (lowest resolution), ultimately resulting in a single-node graph. Specifically, It is a picture The adjacency matrix of each of the K clusters, A 2 It is a picture The adjacency matrix. Coarsening graph at each layer. adjacency matrix It can be calculated using the following formula:

[0081]

[0082] Specifically, the above formula can be understood as: The diagonal elements represent the number of edges within each cluster, reflecting the cluster's... Internal "internal connections", that is, those belonging to clusters in the original graph The connection strength between all nodes. It is calculated by considering the cluster. The edge weights between all nodes are summed. To avoid double counting (since each edge is considered twice in symmetric computation), the summation is multiplied by a factor of 1 / 2. Furthermore, The off-diagonal elements represent the number of connecting edges between two clusters, which is calculated by assigning clusters to each cluster in the original graph. and cluster The adjacency matrix is ​​derived from the sum of edge weights between nodes. No additional coefficients are needed because each cross-cluster edge is calculated only between one cluster pair. This calculation method preserves the local structure of the graph and the connectivity between clusters. By merging the adjacency matrices, the complexity of the graph is reduced while maintaining the relative relationships between clusters, which facilitates more efficient multi-scale analysis.

[0083] In this way, a multi-resolution graph structure for the image is constructed, where the graph... The L-layer coarsening represents a series of images at different resolutions. Specifically as follows:

[0084] Bottom layer diagram: That is, the original image It itself contains all nodes and edges;

[0085] Middle layer diagram: When 1≤l≤L-1, Representation diagram The rough version, then... Each node corresponds to A cluster in the graph. That is, the graph... The number of nodes in the graph is equal to the number of nodes in the graph. The number of clusters;

[0086] Top-level diagram: Top-level diagram This is the last layer of the coarsened graph, containing only one node, which serves as a "supernode" to capture high-level image features and integrate the global graph structure. It's important to note that the adjacency matrix of the top-level graph is not calculated; instead, the aggregated features of this node are used as the global input to the model to extract more effective features.

[0087] In this embodiment of the invention, the federated learning-based multi-resolution hybrid encoder is obtained by receiving the aggregation results of federated learning from the server and then updating and iterating accordingly. Figure 4 As shown, it includes:

[0088] S310. A multi-resolution hybrid encoder is obtained by training the stacked sparse autoencoder and the sparse graph autoencoder based on the multi-view weight loss function. The multi-resolution hybrid encoder can extract features from the input medical image multi-resolution graph structure and the medical image information to obtain multi-view features.

[0089] It should be understood that the multi-resolution hybrid encoder is specifically trained by stacked sparse autoencoders and sparse graph autoencoders based on the multi-view weight loss function. The multi-resolution hybrid encoder obtained by combining stacked sparse autoencoders (SSAE) and sparse graph autoencoders (SGAE) can simultaneously capture the detailed features of multi-resolution images and the global relationships of graph structure data. This hybrid structure significantly improves the feature extraction capability and provides richer feature representations for classification tasks.

[0090] Specifically, a multi-resolution hybrid encoder is obtained by training stacked sparse autoencoders and sparse graph autoencoders based on a multi-view weight loss function, such as... Figure 5 As shown, it includes:

[0091] S311. Add sparsity constraints to the weighted graph reconstructed from the coding features of the graph autoencoder to obtain a sparse graph autoencoder.

[0092] Specifically, the expression for the loss function of the sparse graph autoencoder is:

[0093] J SGAE =J GAE +λJ GAE-sp ,

[0094] Among them, J SGAE J represents the loss function of a sparse graph autoencoder. GAE This represents the loss function of a graph autoencoder. N represents the number of nodes in the graph, and A represents the original adjacency matrix of the graph. J represents the reconstructed adjacency matrix of the graph. GAE-sp This represents the sparse constraint loss function of a graph autoencoder. This represents the hyperparameter used to balance the reconstruction loss term and the sparse constraint term.

[0095] In this embodiment of the invention, the graph autoencoder will be described first.

[0096] Specifically, given a graph with a set of nodes and a set of edges... Graph Convolutional Networks (GCNs) will convert graphs into ... The adjacency matrix A and the node feature matrix X are taken as input. Then, the output of the graph convolutional layer can be expressed as:

[0097]

[0098] Where W represents the learnable parameters of the graph convolutional layer, and σ represents the activation function (such as ReLU, Sigmoid, etc.). and Here express The degree matrix can be represented as Here, N represents the number of nodes in the graph, and I represents the identity matrix. It should be noted that... This represents the normalized version of the degree matrix of the graph, and This represents a self-looping graph, where each node is connected to itself. Therefore, the outputs of multiple graph convolutional layers can be obtained:

[0099]

[0100] Among them, H (l) Let H represent the node feature matrix of the l-th layer, and H (0) =X. Furthermore, by stacking multiple graph convolutional layers, graph convolutional networks can gradually expand the receptive domain of each node, that is, the range of neighboring nodes that each node can perceive, thereby capturing more information about the neighbors. This enables GCN to effectively extract features from graph data.

[0101] Specifically, a Graph Autoencoder (GAE) is a deep learning model that extends the concept of autoencoders to graph-structured data. A GAE consists of an encoder and a decoder. The encoder uses a Graph Convolutional Network (GCN) to map node features to a low-dimensional space, and the decoder reconstructs the adjacency matrix of the graph based on this low-dimensional representation. In the implementation of this invention, the encoder consists of two layers of GCN, while the decoder reconstructs the adjacency matrix of the graph through inner product operations.

[0102] Through the two-layer information aggregation operation of GCN, nodes can acquire structural information and features within a two-hop range. This information typically contains sufficient context to effectively characterize the node's features. While increasing the number of layers can improve the model's expressive power, it also significantly increases training time and computational resource consumption, and may even lead to overfitting. Therefore, based on comprehensive considerations, this embodiment of the invention employs a two-layer GCN to significantly improve model performance.

[0103] Therefore, the encoder output can be defined as:

[0104]

[0105] Here, nonlinear ReLU and linear activation functions are used as the activation functions for the first and second layers of the GCN, respectively. Considering the simplicity, effectiveness, and practicality of the inner product decoder in various tasks, it is chosen to use the inner product decoder to reconstruct the graph structure matrix. The inner product decoder can be expressed as:

[0106]

[0107] in, Let A represent the reconstructed adjacency matrix, and σ represent the sigmoid activation function. Next, GAE's loss function uses binary cross-entropy to measure the difference between the original adjacency matrix A and the reconstructed adjacency matrix A. The difference between them can be expressed by the loss function as:

[0108]

[0109] From the perspective of probability in a weighted graph, for node v i The probability that a node is connected to another node v can be expressed as p(v|v). i This represents node v. i Potential connection distribution. Here r represents the number of edges in the reconstructed weighted graph. From this perspective, the edge (or connection) between two nodes can be viewed as a conditional probability distribution p(v|v) i The results of sampling.

[0110] In the hidden layer feature matrix Z output by the encoder, many node pairs typically lack direct connections. Therefore, a weighted graph can be constructed to reflect the connection probabilities between nodes, where the edge weights represent the connection strength between nodes. Because p(v j |v i The probability p(v) ≥ 0 indicates that the values ​​are calculated based on the distance between node representations, thus these probabilities effectively represent the connection strength as the weights of edges in the graph. It should be noted that p(v) ≥ 0 indicates that the values ​​are calculated based on the distance between node representations. j |vi )≠p(v i |v j That is, the constructed graph is a directed graph, and each directed edge (v) i ,v j The graphs have different weights based on their connectivity probability distributions. Therefore, a weighted graph can be constructed by calculating the connectivity distribution of the graph. To approximate the connectivity probability distribution, these probabilities can be derived using Assumption 1.

[0111] Assumption 1: In an ideal state, when p(v) i |v j When the value is large enough, node v i Indicates that it will be with v j The expression is similar.

[0112] Based on this assumption, Euclidean distance can be used to reconstruct the underlying connectivity distribution p(v|v). i For any two nodes v in the hidden layer feature Z). i and v j The distance between them can be expressed as:

[0113]

[0114] To ensure distance d ij The smaller the value, the higher the probability of connection between nodes p(v). j |v i The larger the value, the greater the distance d. ij The input is fed into the softmax layer. It should be noted that the input value is -d. ij Then, through a normalization step, the connectivity distribution of the graph can be reconstructed:

[0115]

[0116] To better reflect the global structure of the graph and capture local features, while preserving more graph structure and important connectivity information, sparsity constraints are introduced into the weighted graph reconstructed from the features Z encoded by GAE. The network architecture of the Sparse Graph Autoencoder (SGAE) is as follows: Figure 6 As shown.

[0117] Assumption 2: Ideally, if two nodes have the same representation, one of the nodes can be pruned.

[0118] This assumption helps in obtaining sparse constraints. Based on assumption 2, the goal is to maximize the connectivity distribution of the reconstructed graph, pruning redundant nodes in the process. To achieve this goal, the sparse constraint loss function of GAE can be expressed as:

[0119]

[0120] Therefore, the total loss function of the SGAE method can be expressed as:

[0121] J SGAE =J GAE +λJ GAE-sp ,

[0122] Here, λ represents a hyperparameter used to effectively balance the reconstruction loss term and the sparsity constraint term, ensuring a reasonable trade-off between the two. Through such optimization, SGAE can reduce redundant nodes, improve graph sparsity, and further enhance graph representation capabilities while preserving the global features and important connectivity information of the graph structure.

[0123] S312. Determine the loss function of the stacked sparse autoencoder;

[0124] Specifically, the expression for the loss function of the stacked sparse autoencoder is:

[0125]

[0126] Among them, J SSAE Let J(W,b) represent the loss function of the stacked sparse autoencoder, and J(W,b) represent the reconstruction error. sparse Let represent the sparsity penalty term, β represent the sparsity penalty factor, S represent the dimension of the hidden layer, and ρ represent the target sparsity. denoted as the average activation value, m as the number of training samples, x as the input image sample, and z as the output reconstruction result.

[0127] In this embodiment of the invention, the autoencoder (AE) is a single-hidden-layer neural network with encoding and decoding functions. This structure includes an input layer, a hidden layer, and an output layer. The input image sample X = [x1, x2, ..., x...] m In the diagram, m represents the number of training samples for the network. The encoder maps the input data to a hidden layer representation h, and then the decoder maps the hidden layer representation h to the reconstructed output z. These two steps can be represented as:

[0128] h = f(W1X + b1),

[0129] z = g(W²h + b²),

[0130] Where W1 and b1 represent the weights and biases of the encoding layer, respectively, and W2 and b2 represent the weights and biases of the decoding layer, respectively. f(·) and g(·) represent the activation functions, typically the sigmoid function is chosen as the non-linear activation function. During training, traditional autoencoders minimize the reconstruction error. The network parameters θ = {W1, b1, W2, b2} are optimized by considering the differences between the input and output data.

[0131] Sparse autoencoders (SAEs) introduce sparsity constraints into traditional autoencoders, extracting more meaningful features by limiting the number of activation units in the hidden layers. The sparsity penalty term makes the activation of most hidden layers sparser, meaning that most hidden units have activation values ​​close to zero, with only a few units activated. If sigmoid is chosen as the activation function, an output close to 0 indicates a node is inactive, and close to 1 indicates a node is active. Assume a... j (x i ) represents the activation value of hidden neuron j, and a j (x i )=f(W j x i +b j Therefore, the average activation of the hidden layer can be defined as:

[0132]

[0133] To satisfy the sparsity constraint, a sparsity penalty term is added to limit the average activation value of the hidden layer. It is close to the target sparsity ρ value. This penalty term can be expressed as:

[0134]

[0135] Where ρ represents the target sparsity, typically taking a value close to 0.1. β represents the sparsity penalty factor, usually a value between 0 and 1. s represents the dimension of the hidden layer. The sparsity penalty term is typically measured using Kullback-Leibler (KL) divergence to measure the difference between the actual activation distribution and the expected sparse distribution. By introducing the sparsity penalty term, autoencoders can extract sparse and interpretable features more effectively.

[0136] Stacked Sparse Autoencoders (SSAEs) combine stacked autoencoders (SAEs) with sparsity constraints. By stacking multiple sparse autoencoders layer by layer, SSAEs can learn higher-level features while preserving the sparsity constraints of each layer, thus extracting more interpretable features. For each hidden layer of an SSAE, l∈{1,2,…,D}, where D is the total number of layers, h... l =f(W l h l-1 +b l Let represent the output of the l-th layer. To maintain consistency with the SGAE structure in the previous section, a two-layer stacked sparse autoencoder is considered here. Therefore, the final loss function of the SSAE is defined as:

[0137]

[0138] Here, β represents the hyperparameter balancing the reconstruction error term and the sparsity penalty term. In this way, SSAE can not only learn hierarchical features but also ensure that the representation of each layer is sparsity, thereby improving the efficiency and representational power of feature extraction. By introducing sparsity into deep learning models, SSAE can effectively learn high-level features that are sparse and interpretable, which has significant advantages for tasks such as image classification and feature learning.

[0139] S313. Determine the multi-view weight loss function based on the loss function of the stacked sparse autoencoder and the loss function of the sparse graph autoencoder.

[0140] Specifically, the expression for the multi-view weight loss function is:

[0141]

[0142] in, Let W1 represent the weights of the input x, W2 represent the biases of the input x, and σ represent the multi-view weight loss function. SSAE σ represents the noise parameter of a stacked sparse autoencoder. SGAE y represents the noise parameter of the sparse graph autoencoder. SSAE y represents the target value of the stacked sparse autoencoder. SGAE This represents the target value of the sparse graph autoencoder. This represents the output of the sparse graph autoencoder. J represents the output of the stacked sparse autoencoder. SGAE J represents the loss function of a sparse graph autoencoder. SSAE This represents the loss function of a stacked sparse autoencoder.

[0143] It should be noted that in federated learning, the local model employs a multi-resolution hybrid autoencoder (MHAE) to extract data features at different resolutions. By fusing these multi-view features from different resolutions, a richer and more diverse representation is generated. This MHAE combines multi-resolution graph structure with multi-view learning to form a composite model. Specifically, the sparse graph autoencoder (SGAE) captures the structural and relational information of the data through graph view learning, while the stacked sparse autoencoder (SSAE) acquires the feature and attribute information of the data through feature view learning.

[0144] The goal of multi-view learning is to extract complementary information from different perspectives to achieve a more comprehensive and accurate understanding of the data. The loss functions J for SGAE and SSAE were obtained earlier. SGAEand J SSAE Next, we will derive the multi-view loss function based on maximizing Gaussian likelihood, which is similar to the likelihood function in multi-task learning. Specifically, the loss functions for the graph view and the feature view are expressed as J... SGAE (W1) and J SSAE (W2). Definition and For the output, the weights and biases of the input x are W1 and W2, respectively, and y SGAE and y SSAE This represents the target value of the SGAE and SSAE models proposed in this embodiment of the invention. Assuming that the loss function of each view conforms to a Gaussian distribution and that the loss functions between views are independent, the joint likelihood function of the graph view and the feature view can be expressed as:

[0145]

[0146] Where, σ SGAE and σ SSAE These represent the noise parameters of the network's SGAE and SSAE, respectively. This is represented by a normal distribution (Gaussian distribution). In maximum likelihood inference, it is necessary to maximize the joint log-likelihood function of the model. The joint log-likelihood function can be defined as:

[0147]

[0148] The log-likelihood function of a Gaussian distribution can be expressed as:

[0149]

[0150] The constant term in the log-likelihood here It can be ignored; the joint log-likelihood function can be simplified to:

[0151]

[0152] Maximizing the log-likelihood function is equivalent to minimizing its negative value. Through this process, the proposed multi-view learning loss function can be obtained:

[0153]

[0154] Introducing noise parameter σ SGAE and σ SSAE Afterwards, the loss weights for different views can be dynamically adjusted during training, thereby better integrating multi-view information and improving the overall performance of the model. When the noise parameter σ SGAE As the value increases, the loss term J associated with this noise parameter... SGAEThe weight of (W1) decreases; conversely, when the noise parameter decreases, the weight of the corresponding loss term increases. The last term, logσ... SGAE σ SSAE This can be viewed as a regularizer for the noise term, designed to prevent noise values ​​from becoming too large or too small, ensuring numerical stability. This optimization strategy effectively fuses information from the graph view and the feature view within the federated learning framework, thereby improving model performance. Subsequently, gradient descent can be used to solve for the optimal model parameters.

[0155] S314. Iteratively train the multi-view weight loss function to obtain a multi-resolution hybrid encoder.

[0156] By iteratively training the multi-view weight loss function described above, a multi-resolution hybrid encoder is finally obtained.

[0157] S320. Receive the initial global model parameters from the server, and train the multi-resolution hybrid encoder according to the initial global model parameters to obtain local model parameters;

[0158] S330. The local model parameters are sent to the server, and the server can perform federated learning based on multiple local model parameters to obtain the updated global model parameters.

[0159] S340. Receive the updated global model parameters returned by the server, and iteratively train the multi-resolution hybrid encoder according to the updated global model parameters to obtain a federated learning-based multi-resolution hybrid encoder. The federated learning-based multi-resolution hybrid encoder can obtain the multi-view features of the medical image after iterative training of the multi-view features based on the updated global model parameters.

[0160] It should be noted that the multi-resolution hybrid autoencoder (MHAE) obtained above can effectively extract features from image data of different resolutions and generate rich representations. In order to further protect data privacy, this embodiment of the invention extends the multi-resolution hybrid autoencoder to the federated learning framework, that is, combines federated learning with the multi-resolution hybrid autoencoder (FL-MHAE) to achieve distributed training and effectively protect data privacy.

[0161] Specifically, the server can perform federated learning based on multiple local model parameters to obtain updated global model parameters, including:

[0162] The weighted average result is obtained by weighting multiple local model parameters according to the federated index weighted average algorithm.

[0163] The initial global model parameters are updated based on the weighted average result to obtain the updated global model parameters, the expression of which is:

[0164]

[0165] Among them, w t w represents the updated global model parameters. t-1 Indicates local model parameters. represents the global model parameters, and μ represents the momentum coefficient used to characterize the impact of the current local model parameter update on the global model parameters.

[0166] In this embodiment of the invention, during the aggregation process of federated learning, an exponential moving average (EMA) method is used instead of the traditional weighted average. This aggregation method is called Fed-EMA. EMA is a decreasing weighted moving average method where the weight of each value decreases exponentially over time. Newer data receives higher weights, while older data retains some influence. By introducing a momentum coefficient μ, the parameters of the global model can be updated incrementally. This is achieved after collecting local model parameters from all clients. Then, the server aggregates these parameters to obtain a new global model parameter. Then the server updates the global model parameters, and the updated formula is as follows:

[0167]

[0168] The momentum coefficient μ ranges from 0 to 1, representing the degree to which the global model depends on historical parameter updates. In this embodiment of the invention, μ is typically set to 0.9 to smooth out the influence of historical models. and w t-1 These represent the parameters of the global and local models, respectively. The momentum coefficient μ controls the degree to which the current local model update affects the global model. A higher μ value means greater emphasis is placed on historical parameter updates, while a lower μ value means greater emphasis is placed on the current local update. By employing the EMA method, Fed-EMA can more effectively and robustly aggregate model parameters, thereby improving model performance in federated learning. The FL-MHAE algorithm can specifically include the following steps:

[0169] Step 1: Initialize the global model. The server sends the initialized global model parameters to all clients (hospitals, research institutes, and other medical data holders).

[0170] Step 2: Client-side local training. Each client trains a multi-resolution fusion autoencoder (MHAE) on its local dataset to obtain optimal local model parameters. The architecture of the MHAE is as follows: Figure 7As shown.

[0171] Step 3: Parameter update and upload. Each client updates the model parameters based on the local training results and sends the updated parameters back to the server.

[0172] Step 4: Global model update. The server uses the Fed-EMA algorithm (Federal Exponential Weighted Average) to perform a weighted average of the model updates from all clients, thereby updating the global model parameters.

[0173] Step 5: Iterative training. Repeat steps 2 to 4 to perform multiple rounds of local training and global model updates until the model converges or reaches the set number of iterations.

[0174] In this embodiment of the invention, classification processing is performed based on the multi-view features of the medical images to obtain medical image classification results, such as... Figure 8 As shown, it includes:

[0175] S410. The medical image multi-view features are sent to the server. The server can perform unified feature extraction on the medical image multi-view features according to the updated global model, and input the extracted unified features into the classifier for classification to obtain the medical image classification result.

[0176] S420. Receive the medical image classification results returned by the server.

[0177] It should be understood that after the algorithm training is complete, the server uses the updated global model to extract a unified feature representation Z and inputs it into the Softmax classifier for classification. The Softmax classifier predicts the probability of each disease based on the extracted feature vectors, and the specific calculation formula is as follows:

[0178]

[0179] Among them, w k b k Let p represent the weight and bias of the k-th disease class, respectively, where K represents the total number of disease classes. k This represents the probability distribution of a sample belonging to the k-th class. Therefore, the final classification result is the class with the highest probability.

[0180]

[0181] The following section provides a detailed explanation of the effectiveness of the federated learning-based medical image classification method of this invention, using experimental data.

[0182] ChestX-ray14 was used as the image dataset. ChestX-ray14 is a large-scale chest X-ray image dataset released by the National Institutes of Health (NIH) in 2017, containing 112,120 frontal chest X-ray images from approximately 30,805 patients, with image sizes typically 1024×1024 pixels. This dataset provides multi-label annotations for 14 common chest diseases for each image, including atelectasis, cardiomegaly, effusion, exudation, mass, nodule, pneumonia, pneumothorax, consolidation, edema, emphysema, fibrosis, pleural thickening, and hernia. It also includes a "No Finding" label to represent normal samples. In the experiments, the Adam optimization function was selected to train the model with a learning rate of 0.01, 100 epochs, and a batch size of 4. To comprehensively evaluate the model's performance in multi-label medical image classification tasks, this embodiment of the invention uses AUC (Area Under the Receiver Operating Characteristic Curve) as the primary evaluation metric. AUC measures a model's ability to distinguish between positive and negative samples; the closer the value is to 1.0, the better the model's performance. Experimental results are as follows: Figure 9 As shown in the analysis, the proposed FL-MHAE outperforms the traditional SAE and VAE models in most disease categories of 14 chest diseases, further validating its robustness and generalization ability in identifying complex lesions in chest images.

[0183] Extensive experiments demonstrate that FL-MHAE significantly outperforms existing methods on multiple standard datasets, exhibiting faster convergence, higher classification accuracy, and stronger generalization ability. This model not only provides an efficient and privacy-preserving solution for image classification tasks in distributed environments but also lays a solid foundation for future expansion and improvement in complex tasks, while offering new research directions for further optimizing knowledge transfer efficiency.

[0184] In summary, the federated learning-based medical image classification method provided by this invention significantly enhances feature extraction capabilities through its multi-resolution hybrid encoder, offering richer feature representations for the classification task. Furthermore, the multi-view weight loss function deeply integrates graph structure information with image feature information, optimizing the consistency of feature representations. Additionally, the dynamic adjustment of model update weights based on server-side federated learning aggregation results effectively mitigates performance fluctuations caused by uneven data distribution in a distributed environment. Therefore, the federated learning-based medical image classification method provided by this invention effectively captures medical image details to improve classification performance through its multi-resolution hybrid encoder. The multi-view weight loss function enhances the model's discriminative ability, improving robustness. Finally, the dynamic adjustment of model update weights based on server-side federated learning aggregation results effectively alleviates performance fluctuations caused by uneven data distribution in a distributed environment, improving model convergence speed and training stability, thus enabling the classification method to achieve more effective and accurate classification results.

[0185] As another embodiment of the present invention, a federated learning-based medical image classification device 100 is provided to implement the federated learning-based medical image classification method described above, wherein, as Figure 10 As shown, it includes:

[0186] The acquisition module 110 is used to acquire medical imaging information, which includes images and / or videos;

[0187] Preprocessing module 120 is used to preprocess the medical image information to obtain a multi-resolution image structure of the medical image;

[0188] The feature extraction module 130 is used to input the multi-resolution map structure of the medical image and the medical image information into a federated learning-based multi-resolution hybrid encoder for feature extraction to obtain multi-view features of the medical image. The federated learning-based multi-resolution hybrid encoder is obtained by updating and iterating after receiving the aggregation results of federated learning from the server. The multi-resolution hybrid encoder is obtained by training based on a stacked sparse autoencoder and a sparse graph autoencoder based on a multi-view weight loss function. The sparse graph autoencoder is obtained by training based on a graph autoencoder with added sparsity constraints.

[0189] The classification module 140 is used to perform classification processing based on the multi-view features of the medical images to obtain medical image classification results.

[0190] The medical image classification device based on federated learning provided by this invention acquires medical image information and preprocesses it to obtain a multi-resolution image structure. Both the medical image information and the multi-resolution image structure are input into a pre-obtained multi-resolution hybrid encoder based on federated learning. The multi-resolution hybrid encoder is obtained by updating and iterating after receiving the aggregation results of federated learning from the server. The multi-resolution hybrid encoder is obtained by training based on a stacked sparse autoencoder and a sparse graph autoencoder using a multi-view weight loss function. The sparse graph autoencoder is obtained by training based on a graph autoencoder with added sparsity constraints. Because the multi-resolution hybrid encoder obtained by stacking sparse autoencoders and sparse graph autoencoders can simultaneously capture the detailed features of multi-resolution images and the global relationships of graph structure data, the federated learning-based multi-resolution hybrid encoder can significantly improve feature extraction capabilities, providing richer feature representations for classification tasks. Furthermore, the multi-view weight loss function can deeply fuse graph structure information with image feature information, optimizing the consistency of feature representations. Additionally, the dynamic adjustment of model update weights through server-side federated learning aggregation results effectively alleviates performance fluctuations caused by uneven data distribution in a distributed environment. Therefore, the federated learning-based medical image classification device provided by this invention effectively captures medical image details to improve classification performance based on the multi-resolution hybrid encoder. It also enhances the model's discriminative ability through the multi-view weight loss function, improving the robustness of classification performance. Finally, the dynamic adjustment of model update weights through server-side federated learning aggregation results effectively alleviates performance fluctuations caused by uneven data distribution in a distributed environment, improving the model's convergence speed and training stability, thereby enabling the classification device to obtain more effective and accurate classification results.

[0191] The specific working principle and process of the federated learning-based medical image classification device of the present invention can be referred to the description of the federated learning-based medical image classification method above, and will not be repeated here.

[0192] As another embodiment of the present invention, a medical image classification system based on federated learning is provided, comprising a client and a server, wherein the client is communicatively connected to the server, the client including the aforementioned medical image classification device based on federated learning and a multi-resolution hybrid encoder based on federated learning, and the server including a federated exponential weighted average algorithm model, wherein the multi-resolution hybrid encoder based on federated learning is obtained by training the multi-resolution hybrid encoder and the federated exponential weighted average algorithm model.

[0193] The federated learning-based medical image classification device can call a federated learning-based multi-resolution hybrid encoder to classify the input medical image information and obtain the medical image classification result.

[0194] The federated learning-based medical image classification system provided by this invention employs the aforementioned federated learning-based medical image classification device. Its multi-resolution hybrid encoder effectively captures medical image details to improve classification performance. Furthermore, the multi-view weight loss function enhances the model's discriminative ability, improving the robustness of classification performance. Finally, the system dynamically adjusts model update weights based on the server-side federated learning aggregation results, effectively mitigating performance fluctuations caused by uneven data distribution in a distributed environment. This improves the model's convergence speed and training stability, resulting in more effective and accurate classification results.

[0195] The specific working principle and process of the federated learning-based medical image classification system of the present invention can be referred to the description of the federated learning-based medical image classification method above, and will not be repeated here.

[0196] It is understood that the above embodiments are merely exemplary implementations used to illustrate the principles of the present invention, and the present invention is not limited thereto. For those skilled in the art, various modifications and improvements can be made without departing from the spirit and essence of the present invention, and these modifications and improvements are also considered to be within the scope of protection of the present invention.

Claims

1. A method for medical image classification based on federated learning, characterized in that, include: Acquire medical imaging information, including images and / or videos; The medical image information is preprocessed to obtain a multi-resolution image structure of the medical image; The medical image multi-resolution map structure and the medical image information are both input into a federated learning-based multi-resolution hybrid encoder for feature extraction to obtain multi-view features of the medical image. The federated learning-based multi-resolution hybrid encoder is obtained by updating and iterating after receiving the aggregation results of federated learning from the server. The multi-resolution hybrid encoder is obtained by training based on a stacked sparse autoencoder and a sparse graph autoencoder based on a multi-view weight loss function. The sparse graph autoencoder is obtained by training based on a graph autoencoder with added sparsity constraints. The medical images are classified based on their multi-view features to obtain classification results. The expression for the loss function of the sparse graph autoencoder is: ， wherein, denotes a loss function of the sparse graph autoencoder, denotes a loss function of the graph autoencoder, , N denotes the number of nodes of the graph, A denotes the original adjacency matrix of the graph, denotes the reconstructed adjacency matrix of the graph, denotes a sparse constraint loss function of the graph autoencoder, denotes a hyperparameter for balancing the reconstruction loss term and the sparse constraint term; wherein, , denotes a conditional connection probability of any two nodes and in the latent feature, denotes the Euclidean distance between any two nodes and in the latent feature.

2. The medical image classification method based on federated learning according to claim 1, characterized in that, The federated learning-based multi-resolution hybrid encoder is obtained by receiving the aggregation results of federated learning from the server and updating and iterating accordingly, including: A multi-resolution hybrid encoder is obtained by training a stacked sparse autoencoder and a sparse graph autoencoder based on a multi-view weight loss function. The multi-resolution hybrid encoder can extract features from the input medical image multi-resolution graph structure and the medical image information to obtain multi-view features. Receive the initial global model parameters from the server, and train the multi-resolution hybrid encoder according to the initial global model parameters to obtain local model parameters; The local model parameters are sent to the server, and the server can perform federated learning based on multiple local model parameters to obtain updated global model parameters. The system receives the updated global model parameters returned by the server and iteratively trains the multi-resolution hybrid encoder based on the updated global model parameters to obtain a federated learning-based multi-resolution hybrid encoder. The federated learning-based multi-resolution hybrid encoder can obtain the medical image multi-view features by iteratively training the multi-view features based on the updated global model parameters.

3. The medical image classification method based on federated learning according to claim 2, characterized in that, A multi-resolution hybrid encoder is obtained by training stacked sparse autoencoders and sparse graph autoencoders based on a multi-view weight loss function, including: By adding sparsity constraints to the weighted graph reconstructed from the encoded features of the graph autoencoder, a sparse graph autoencoder is obtained. Determine the loss function of the stacked sparse autoencoder; The multi-view weight loss function is determined based on the loss function of the stacked sparse autoencoder and the loss function of the sparse graph autoencoder. The multi-resolution hybrid encoder is obtained by iteratively training the multi-view weight loss function.

4. The medical image classification method based on federated learning according to claim 3, characterized in that, The expression for the loss function of the stacked sparse autoencoder is: ， in, The loss function of a stacked sparse autoencoder is represented by... Indicates reconstruction error. Indicates a sparse penalty term. Let S denote the sparsity penalty factor, and S denote the dimension of the hidden layer. Indicates the sparsity of the target. Indicates the average activation value. Indicates the number of training samples. This represents the input image sample. This indicates the output of the reconstruction result.

5. The medical image classification method based on federated learning according to claim 3, characterized in that, The expression for the multi-view weight loss function is: ， in, This represents the multi-view weight loss function. This represents the weight of the input x. This represents the deviation from the input x. This represents the noise parameter of a stacked sparse autoencoder. This represents the noise parameters of a sparse graph autoencoder. This represents the target value of the stacked sparse autoencoder. This represents the target value of the sparse graph autoencoder. This represents the output of the sparse graph autoencoder. This represents the output of a stacked sparse autoencoder. This represents the loss function of a sparse graph autoencoder. This represents the loss function of a stacked sparse autoencoder.

6. The medical image classification method based on federated learning according to claim 2, characterized in that, The server can perform federated learning based on multiple local model parameters to obtain updated global model parameters, including: The weighted average result is obtained by weighting multiple local model parameters according to the federated index weighted average algorithm. The initial global model parameters are updated based on the weighted average result to obtain the updated global model parameters, the expression of which is: ， in, This represents the updated global model parameters. Indicates local model parameters. Represents global model parameters. The momentum coefficient represents the degree to which the current local model parameter update affects the global model parameters.

7. The medical image classification method based on federated learning according to claim 1, characterized in that, The medical image information is preprocessed to obtain a multi-resolution image structure of the medical image, including: The relationship between image pixels and image pixel regions in the medical image information is converted into a medical image graph structure; The medical image structure is constructed using a multi-resolution map to obtain a multi-resolution medical image structure.

8. The medical image classification method based on federated learning according to claim 1, characterized in that, Based on the multi-view features of the medical images, classification processing is performed to obtain medical image classification results, including: The medical image multi-view features are sent to the server. The server can perform unified feature extraction on the medical image multi-view features according to the updated global model, and input the extracted unified features into the classifier for classification to obtain the medical image classification result. Receive the medical image classification results returned by the server.

9. A medical image classification device based on federated learning, used to implement the medical image classification method based on federated learning as described in any one of claims 1 to 8, characterized in that, include: The acquisition module is used to acquire medical imaging information, including images and / or videos; The preprocessing module is used to preprocess the medical image information to obtain the multi-resolution image structure of the medical image; The feature extraction module is used to input the multi-resolution map structure of the medical image and the medical image information into a federated learning-based multi-resolution hybrid encoder for feature extraction to obtain multi-view features of the medical image. The federated learning-based multi-resolution hybrid encoder is obtained by updating and iterating after receiving the aggregation results of federated learning from the server. The multi-resolution hybrid encoder is obtained by training based on a stacked sparse autoencoder and a sparse graph autoencoder based on a multi-view weight loss function. The sparse graph autoencoder is obtained by training based on a graph autoencoder with added sparsity constraints. The classification module is used to perform classification processing based on the multi-view features of the medical images to obtain the medical image classification results.

10. A medical image classification system based on federated learning, characterized in that, The system includes a client and a server, with the client communicating with the server. The client includes the federated learning-based medical image classification device and the federated learning-based multi-resolution hybrid encoder as described in claim 9. The server includes a federated exponentially weighted average algorithm model, and the federated learning-based multi-resolution hybrid encoder is obtained by training the multi-resolution hybrid encoder and the federated exponentially weighted average algorithm model. The federated learning-based medical image classification device can call a federated learning-based multi-resolution hybrid encoder to classify the input medical image information and obtain the medical image classification result.

Images