A Hypergraph-Based Federated Learning Survival Prediction Method

Abstract

This invention discloses a hypergraph-based federated learning survival prediction method. First, for r centers, n pathological images are acquired for each center to obtain the feature information of the dataset. Then, the nearest neighbor algorithm is used to calculate the nearest neighbors of p patch images for each pathological image, and the neighboring patches are connected to form a hypergraph, obtaining the association matrix. Next, the features of the entire pathological image are learned and obtained through a hypergraph convolutional neural network. Furthermore, hypergraph distillation is used to update the local network. Finally, based on the local network of each center, the global network is updated, and survival prediction is performed. This invention utilizes hypergraph distillation to narrow the gap between HGSurvNet and MLP, achieving fast inference while retaining the ability to capture structural information; it improves the accuracy and robustness issues in the field of federated learning for survival prediction, increasing the accuracy of survival prediction and its robustness to predictions for data from different centers.

Description

Technical Field

[0001] This invention belongs to the fields of distillation learning, distributed machine learning, and hypergraph representation learning, and specifically relates to a hypergraph-based federated learning survival prediction method. Background Technology

[0002] Survival prediction is a key objective of precision medicine, as accurate survival assessments help physicians choose appropriate treatments for individual patients. To achieve this, large amounts of data are essential for training predictive models and preventing overfitting. However, collecting patient data for disease prediction is challenging due to potential differences in data sources across institutions and concerns about privacy and ownership issues in data sharing. To facilitate the integration of cancer data from different institutions without violating privacy laws, federated learning-based survival prediction tasks, such as AdFed, are emerging. However, existing methods do not incorporate structural information about the data into federated learning, resulting in shortcomings in enhancing the robustness and inference efficiency of multi-center data processing. Summary of the Invention

[0003] To address the shortcomings of existing technologies, this invention provides a hypergraph-based federated learning survival prediction method. A novel Hypergraph Federated Learning (HGFL) approach is proposed for survival prediction of multi-center pathology images. At local centers, HGFL first models high-order correlations within gigapixel pathology images using a hypergraph structure. Furthermore, a hypergraph knowledge distillation technique is introduced to transfer high-order structural information to shallow neural networks, preserving the ability to infer high-order correlations while significantly improving the model's inference efficiency. At global centers, local centers only need to share the parameters of their shallow neural networks with the central server. Through parameter aggregation, the shallow networks of each center can learn the high-order structural information of other centers. Experiments on multi-center datasets demonstrate that the proposed method not only effectively improves the robustness and consistency of multi-center survival prediction tasks but also significantly reduces the model's inference time, thus achieving efficient and accurate survival prediction while protecting privacy.

[0004] The proposed model utilizes feature and structural information from multimodal data for self-supervised clustering. First, for r centers, n pathological images are obtained for each center. Each pathological image is uniformly divided into p patches. A pre-trained feature extraction model is used to obtain the image embedding of each patch, and the coordinate information of each patch is recorded. Finally, the two are fused. Then, the nearest neighbor algorithm is used to calculate the nearest neighbors of the p patches for each pathological image. Patient patches are connected to form a hypergraph, resulting in an association matrix. Each pathological image forms an independent hypergraph and association matrix. Next, the features of the entire pathological image are learned and obtained through a hypergraph convolutional neural network (HGSurvNet). Furthermore, hypergraph distillation is used to update the local network. Finally, based on the local network of each center, the global network is updated, and survival prediction is performed.

[0005] A hypergraph-based federated learning survival prediction method includes the following steps:

[0006] Step 1: For r centers, obtain n pathological images for each center, and evenly divide each pathological image into p patch images, where p is an optional hyperparameter. Use a pre-trained feature extraction model to obtain the image embedding of each patch image and record the coordinate information of each patch image. Finally, fuse the two to obtain the feature information of the dataset.

[0007] Step 2: Use the nearest neighbor algorithm to calculate the nearest neighbors of the p patch images for each pathological image, connect the neighboring patches to form a hypergraph, and obtain the correlation matrix.

[0008] Step 3: Learn and obtain the features of the entire pathological image through a hypergraph convolutional neural network (HGSurvNet).

[0009] Step 4: Update the local network using hypergraph distillation.

[0010] Step 5: Update the global network based on the local network of each center and perform survival prediction.

[0011] Furthermore, the specific steps of step one are as follows:

[0012] For r centers, independently obtain their respective pathology image-related datasets. For a given pathology image x i The dataset D = {x i} i∈{1,…,N} Where N is the total number of samples. For each pathological image x... i The image is uniformly divided into p patches, and a pre-trained feature extraction model with frozen parameters is used to extract the image embedding of each pathological image as F. i ={f1,f2,…f j ,…fp} ∈{1,…,N} , where f is the image embedding extracted from the patch graph by the pre-trained feature extraction model.

[0013] Therefore, the image embedding of dataset D is obtained as F = {F i} i∈{1,…,N} .

[0014] Then, record the coordinates of the top-left corner of each patch image on the pathological image as c = (a, b). 0≤a≤h,0≤b≤w Where h is the height of the pathological image and w is the width of the pathological image, the coordinate information of each pathological image is further obtained as C. i ={c1,c2,…c j ,…c p} i∈{1,…,N} .

[0015] Thus, the coordinate information of dataset D, C = {C}, is obtained. i} i∈{1,…,N} .

[0016] Finally, the image embeddings F and coordinate information C of the dataset are fused to form the feature information of the dataset:

[0017] X = {X i} i∈{1,…,N}

[0018] Among them, the characteristic information X of a single pathological image i Defined as follows:

[0019] X i =F i +C i ={f1+c1,f2+c2,…f j +c j ,…f p +c p} i∈{1,…,N}

[0020] Where f j +c j Concatenate the two vectors.

[0021] In one embodiment, the pre-trained feature extraction model uses ResNet-34.

[0022] Furthermore, the specific steps of step two are as follows:

[0023] x for each pathological image i The p patch graphs in the graph are mapped to the vertex set of the hypergraph. Where vertex v j For x i Mapping in the hypergraph and defining Vertex features, where By calculating vertex features The Euclidean distance is used to measure the distance between vertices, where the Euclidean distance is defined as follows:

[0024]

[0025] Where d(v) a ,v b ) represents vertex v a and v b The Euclidean distance between them. and Representing vertex v a and v b The feature is C, which represents the dimension of the feature. and Representing vertex v a and v b The value of the feature in the c-th dimension, and the pathological image data X in dataset D. i All are mapped to the vertices of their respective hypergraphs using the method described above.

[0026] Next, for each vertex, the Euclidean distance between that vertex and other vertices is scaled to [0,1], as shown in the following formula:

[0027]

[0028] d norm (v a ,v b )=d(v a ,v b ) / d(v a ) max

[0029] Where d(v) a ) max Indicates from vertex v a The maximum Euclidean distance to other vertices; d norm (v a ,v b ) represents the scaled vertex v a Euclidean distances to other vertices.

[0030] Then, the nearest neighbors of the modal data in the modal dataset are obtained by simultaneously using the ∈-ball (∈-neighborhood) and K-NN (k-nearest neighbor) graph construction methods. K-NN is a graph structure built based on the relationship between each point and its k nearest neighbors; ∈-ball is a graph structure built based on the relationship between each vertex and its neighbors whose distance is less than a certain radius ∈. The specific steps are as follows:

[0031] First, vertex v is computed using the ∈-ball graph construction method (∈-neighborhood). a The neighbor subset is calculated as follows:

[0032] neigh th (v a )={v b ∣d norm (v a ,v b ) <th}

[0033] Among them, neigh th (v a Let ) be the vertex v obtained by constructing the ∈-ball graph. a The neighbor subset of , where th is a pre-set distance threshold.

[0034] At the same time, the K-NN graph construction method (K-nearest neighbors) is used to compute vertex v. a The neighbor subset is calculated as follows:

[0035] neigh ne (v a )={v b ∣(d(v a ,v b ) <sortd(v a ,v b ))[K]}

[0036] Where K is the pre-set number of neighboring vertices, and neighbor ne (v a ) represents the vertex v obtained using the K-NN graph construction method. a The neighbor subset, sort(·) denotes the distance sorting algorithm, |neigh ne (v a )|=K.

[0037] In summary, by using two different graph construction methods, for each vertex v a It can generate a set of neighboring vertices. th (v a ) and neighbor ne (v a The final set of neighboring vertices is as follows:

[0038] neigh(v a )={neigh th (v a ),neigh ne (v a )}

[0039] Based on the set of neighboring vertices, the hyperedge set is defined as follows:

[0040] ε={neigh(v a )} a∈{1,…,N}

[0041] The correlation matrix can be obtained from the neighbor vertex set. The correlation matrix is ​​defined as follows:

[0042]

[0043] Where H(a,b) is the element value in the a-th row and b-th column of the correlation matrix H.

[0044] Furthermore, step three is detailed below:

[0045] The features Z of the entire pathological image were learned and obtained through a hypergraph convolutional neural network (HGSurvNet). i First, the result of l-fold convolution of the feature matrix of each pathological image is obtained through the hypergraph convolution formula. The formula is as follows:

[0046]

[0047] Where l is the number of convolutions in the hypergraph convolutional neural network (a hyperparameter that can be selected during training), H i Let i be the correlation matrix corresponding to the i-th pathological image. The result of the l-th convolution of the feature matrix of the i-th pathological image is given by Θ, where Mask(·) is the mask operator for the hyperedge features. l The parameters that need to be learned during the training process, and the pathological image feature set {X} i} i∈{1,…,N} Shared Θ l ,in Where c l c l+1 This is an optional hyperparameter.

[0048] After l layers of convolution, vertex features that fuse topological and semantic information are obtained. The convolution result of the feature matrix of the i-th pathological image is then further processed by a pooling operator. The feature vector of the pathological image is obtained by summing the elements and passing them through a linear layer. Where c o This is an optional hyperparameter.

[0049] Furthermore, step four is detailed below:

[0050] Define the global network of federated learning as f(w), and the local networks as {f j (w)} j∈{1,…,r} The model architecture for the global network and the local network is the same, and the appropriate model architecture is selected based on the device's memory and the requirements for running speed.

[0051] Local networks {f} are constructed using hypergraph distillation. j (w)} j∈{1,…,r} Updated. To enhance federated learning, HGSurvNet is used for soft label guidance, directly distilling knowledge from HGSurvNet to federated learning. j (w). Therefore, HGSurvNet is used as the teacher network, f j (w) is the student network, using the following objective function to extract knowledge:

[0052]

[0053] in, Number of pathological images and h θ (x i ) indicates that they are respectively composed of HGSurvNet and f j (w) Outputs the probability of survival risk.

[0054] Furthermore, step five is detailed below:

[0055] The global network update is defined by the following objectives:

[0056]

[0057] For machine learning problems, we usually take f j (w)=l(x j ,y j ;w), that is, using model parameters w on the example (x) j ,y j The loss is used for prediction. Since there are r centers, the data is distributed among these centers, where... It is the set of indices of data points at center j, where Therefore, the above objectives can be rewritten as:

[0058]

[0059] For data from r centers, process the pathological image data of each center into X using the method in step one. 1 ,…X j ,…X r The feature information X of r datasets j The inputs are respectively fed into its own HGSurvNet and local network f j In step (w), the local network is updated end-to-end according to the loss function in step four. Finally, the global network f(w) is updated based on the above objectives.

[0060] Finally, for any pathological image x i The feature information X is processed using the method in step 1. i Then, the feature vector Z is further obtained through the trained HGSurvNet. i We will Z i As input to the global network f(w), a scalar representing the number of days a patient survives after diagnosis is obtained as output.

[0061] Features and beneficial effects of the present invention:

[0062] This invention addresses the shortcomings of survival prediction in distributed training and the limitations of traditional federated learning in terms of robustness and performance. It proposes Hypergraph Federated Learning, leveraging the high-order correlations between WSI data and the superior modeling capabilities of hypergraph neural networks to enhance federated learning. Experiments were conducted on survival prediction tasks, retaining the Model-Level Processing (MLP) as a local and global model required for deployment. Furthermore, this invention utilizes hypergraph distillation to bridge the gap between HGSurvNet and MLP, achieving fast inference while preserving the ability to capture structural information. Experimental results demonstrate that, through hypergraph convolutional neural networks, this method improves the accuracy and robustness of federated learning in survival prediction, enhancing both the accuracy of survival prediction and its robustness to predictions on data with different centers. Attached Figure Description

[0063] Figure 1 This is a flowchart of the method of the present invention.

[0064] Figure 2 To compare the effectiveness of this method with other methods

[0065] Figure 3 This represents the results of training the method at one center and testing it at other centers.

[0066] Figure 4 The results of testing this method and other methods using a global model at other centers are presented.

[0067] Figure 5 This study presents an ablation experiment of the supergraph distillation portion using this method. Detailed Implementation

[0068] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0069] like Figure 1 As shown, a hypergraph-based federated learning survival prediction method includes the following steps:

[0070] Step 1: For r centers, obtain n pathological images for each center. Divide each pathological image evenly into p patches (p = 2000 in this example), where p is an optional hyperparameter. Use a pre-trained feature extraction model to obtain the image embedding for each patch and record the coordinate information of each patch. Finally, fuse the two to obtain the feature information of the dataset. The specific steps are as follows:

[0071] For r centers, independently obtain their respective pathology image-related datasets. For a given pathology image x i The dataset D = {x i} i∈{1,…,N} Where N is the total number of samples. For each pathological image x... i The image is uniformly divided into p patches, and a pre-trained feature extraction model with frozen parameters is used to extract the image embedding of each pathological image as F. i ={f1,f2,…f j ,…f p} i∈{1,…,N} , where f is the image embedding extracted from the patch graph by the pre-trained feature extraction model.

[0072] Therefore, the image embedding of dataset D is obtained as F = {F i} i∈{1,…,N} .

[0073] Then, record the coordinates of the top-left corner of each patch image on the pathological image as c = (a, b). 0≤a≤h,0≤b≤w Where h is the height of the pathological image and w is the width of the pathological image, the coordinate information of each pathological image is further obtained as C. i ={c1,c2,…c j ,…c p} i∈{1,…,N} .

[0074] Thus, the coordinate information of dataset D, C = {C}, is obtained. i} i∈{1,…,N} .

[0075] Finally, the image embeddings F and coordinate information C of the dataset are fused to form the feature information of the dataset:

[0076] X = {X i}i∈{1,…,N}

[0077] Among them, the characteristic information X of a single pathological image i Defined as follows:

[0078] X i =F i +C i ={f1+c1,f2+c2,…f j +c j ,…f p +c p} i∈{1,…,N}

[0079] Where f j +c j Concatenate the two vectors.

[0080] The pre-trained feature extraction model used is ResNet-34.

[0081] Step 2: Use the nearest neighbor algorithm to calculate the nearest neighbors of the p patch images for each pathological image, connect the neighboring patches to form a hypergraph, and obtain the correlation matrix.

[0082] x for each pathological image i The p patch graphs in the graph are mapped to the vertex set of the hypergraph. Where vertex v j For x i Mapping in the hypergraph and defining Vertex features, where By calculating vertex features The Euclidean distance is used to measure the distance between vertices, where the Euclidean distance is defined as follows:

[0083]

[0084] Where d(v) a ,v b ) represents vertex v a and v b The Euclidean distance between them. and Representing vertex v a and v b The feature is C, which represents the dimension of the feature. and Representing vertex v a and v b The value of the feature in the c-th dimension, and the pathological image data X in dataset D. i All are mapped to the vertices of their respective hypergraphs using the method described above.

[0085] Next, for each vertex, the Euclidean distance between that vertex and other vertices is scaled to [0,1], as shown in the following formula:

[0086]

[0087] d norm (v a ,v b )=d(v a ,v b ) / d(v a ) max

[0088] Where d(v) a ) max Indicates from vertex v a The maximum Euclidean distance to other vertices; d norm (v a ,v b ) represents the scaled vertex v a Euclidean distances to other vertices.

[0089] Then, the nearest neighbors of the modal data in the modal dataset are obtained by simultaneously using the ∈-ball (∈-neighborhood) and K-NN (k-nearest neighbor) graph construction methods. K-NN is a graph structure built based on the relationship between each point and its k nearest neighbors; ∈-ball is a graph structure built based on the relationship between each vertex and its neighbors whose distance is less than a certain radius ∈. The specific steps are as follows:

[0090] First, vertex v is computed using the ∈-ball graph construction method (∈-neighborhood). a The neighbor subset is calculated as follows:

[0091] neigh th (v a )={v b ∣d norm (v a ,v b ) <th}

[0092] Among them, neigh th (v a Let ) be the vertex v obtained by constructing the ∈-ball graph. a The neighbor subset of , where th is a pre-set distance threshold.

[0093] At the same time, the K-NN graph construction method (K-nearest neighbors) is used to compute vertex v. a The neighbor subset is calculated as follows:

[0094] neighne (v a )={v b ∣(d(v a ,v b ) <sortd(v a ,v b ))[K]}

[0095] Where K is the pre-set number of neighboring vertices, and neighbor ne (v a ) represents the vertex v obtained using the K-NN graph construction method. a The neighbor subset, sort(·) denotes the distance sorting algorithm, |neigh ne (v a )|=K.

[0096] In summary, by using two different graph construction methods, for each vertex v a It can generate a set of neighboring vertices. th (v a ) and neighbor ne (v a The final set of neighboring vertices is as follows:

[0097] neigh(v a )={neigh th (v a ),neigh ne (v a )}

[0098] Based on the set of neighboring vertices, the hyperedge set is defined as follows:

[0099] ε={neigh(v a )} a∈{1,…,N}

[0100] The correlation matrix can be obtained from the neighbor vertex set. The correlation matrix is ​​defined as follows:

[0101]

[0102] Where H(a,b) is the element value in the a-th row and b-th column of the correlation matrix H.

[0103] Step 3: Learn and obtain the features of the entire pathological image through a hypergraph convolutional neural network (HGSurvNet).

[0104] The features Z of the entire pathological image were learned and obtained through a hypergraph convolutional neural network (HGSurvNet). i First, the result of l-fold convolution of the feature matrix of each pathological image is obtained through the hypergraph convolution formula. The formula is as follows:

[0105]

[0106] Where l is the number of convolutions in the hypergraph convolutional neural network (a hyperparameter that can be selected during training), H i Let i be the correlation matrix corresponding to the i-th pathological image. The result of the l-th convolution of the feature matrix of the i-th pathological image is given by Θ, where Mask(·) is the mask operator for the hyperedge features. l The parameters that need to be learned during the training process, and the pathological image feature set {X} i} i∈{1,…,N} Shared Θ l ,in Where c l c l+1 This is an optional hyperparameter.

[0107] After l layers of convolution, vertex features that fuse topological and semantic information are obtained. The convolution result of the feature matrix of the i-th pathological image is then further processed by a pooling operator. The feature vector of the pathological image is obtained by summing the elements and passing them through a linear layer. Where c o This is an optional hyperparameter.

[0108] Step 4: Update the local network using hypergraph distillation.

[0109] Define the global network of federated learning as f(w), and the local networks as {f j (w)} j∈{1,…,r} The global network and local network have the same model architecture. The appropriate model architecture can be selected according to the device's memory and the requirements for running speed. Theoretically, any model architecture can be selected that does not exceed the maximum memory during training. For the sake of low memory consumption and high speed, this example uses MLP.

[0110] Local networks {f} are constructed using hypergraph distillation. j (w)} j∈{1,…,r} Updated. To enhance federated learning, HGSurvNet is used for soft label guidance, directly distilling knowledge from HGSurvNet to federated learning. j (w). Therefore, HGSurvNet is used as the teacher network, f j (w) is the student network, using the following objective function to extract knowledge:

[0111]

[0112] in, Number of pathological images and h θ (x i ) indicates that they are respectively composed of HGSurvNet and f j (w) Outputs the probability of survival risk.

[0113] Step 5: Update the global network based on the local network of each center and perform survival prediction.

[0114] The global network update is defined by the following objectives:

[0115]

[0116] For machine learning problems, we usually take f j (w)=l(x j ,y j ;w), that is, using model parameters w on the example (x) j ,y j The loss is used for prediction. Since there are r centers, the data is distributed among these centers, where... It is the set of indices of data points at center j, where Therefore, the above objectives can be rewritten as:

[0117]

[0118] For data from r centers, process the pathological image data of each center into X using the method in step one. 1 ,…X j ,…X r The feature information X of r datasets j The inputs are respectively fed into its own HGSurvNet and local network f j (w) (This embodiment uses MLP). The local network is updated end-to-end according to the loss function in step four. Finally, the global network f(w) is updated according to the above objectives.

[0119] Finally, for any pathological image x i We process the pathological images into feature information X using the method in step 1. i Then, the feature vector Z is further obtained through the trained HGSurvNet. i We will Z i As input to the global network f(w), a scalar representing the number of days a patient survives after being diagnosed with cancer is output.

[0120] For the pathology image datasets collected from three medical centers, GY, ZY, and SY, we first used ResNet to extract visual semantic information and obtain topological information based on the spatial location of the images. Then, we used HGSurvNet as the teacher network and MLP as the student network for distillation learning, with a batch size of 256, 30 epochs, and the ADAM optimizer with an initial learning rate of 10. -3 The hypergraph has 2 convolutional layers. Furthermore, we update the global MLP network by averaging the model weights of the three local MLP networks (GY, ZY, and SY).

[0121] Figure 2 For comparative experiments, the method of this invention is about six points higher than the traditional method. Figure 3 The results of testing the local model with a single center on other center data using this method are on average 15 points lower than those tested with a global model. Figure 4 To use this method, tests were conducted on multiple centers using a global model, achieving an average improvement of 10 points over traditional methods. Figure 5 For the ablation experiment of the supergraph distillation part of this method, the ablation results using supergraph distillation were on average 5 points higher than those without supergraph distillation, which proves the effectiveness of the supergraph distillation network.

[0122] The above description, in conjunction with specific / preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. Those skilled in the art can make various substitutions or modifications to these described embodiments without departing from the inventive concept, and all such substitutions or modifications should be considered within the scope of protection of the present invention.

[0123] The parts of this invention not described in detail are well-known to those skilled in the art.

Claims

1. A federated learning survival prediction method based on hypergraphs, characterized in that, Includes the following steps: Step 1: For r centers, obtain n pathological images for each center, and evenly divide each pathological image into p patch images, where p is an optional hyperparameter. Use a pre-trained feature extraction model to obtain the image embedding of each patch image and record the coordinate information of each patch image. Finally, fuse the two to obtain the feature information of the dataset. Step 2: Use the nearest neighbor algorithm to calculate the nearest neighbors of the p patch images for each pathology image, connect the neighboring patches to form a hypergraph, and obtain the association matrix; Step 3: Learn and obtain the features of the entire pathological image using a hypergraph convolutional neural network; details are as follows: Features of the entire pathological image were learned and obtained using a hypergraph convolutional neural network (HGSurvNet). First, the feature matrix of each pathological image is obtained using the hypergraph convolution formula. The result of the convolution The formula is as follows: in The number of convolutions in the hypergraph convolutional neural network. Let i be the correlation matrix corresponding to the i-th pathological image. The first feature matrix of the i-th pathological image The result of the convolution. For mask operators of hyperedge features, These are the parameters that need to be learned during the training process, and the pathological image feature set. Shared ,in ,in , These are optional hyperparameters; After After convolution, vertex features that fuse topological and semantic information are obtained. The convolution result of the feature matrix of the i-th pathological image is then further processed by a pooling operator. The feature vector of the pathological image is obtained by summing the elements and passing them through a linear layer. ,in These are optional hyperparameters; Step 4: Update the local network using hypergraph distillation; details are as follows: Define the global network for federated learning as follows: Local networks are The model architecture for the global network and the local network is the same. The appropriate model architecture is selected based on the device's memory and the requirements for running speed. Local networks using supergraph distillation Update; to enhance federated learning, HGSurvNet is used for soft label guidance, directly distilling knowledge from HGSurvNet to... Therefore, HGSurvNet is used as the teacher network. For the student network, the following objective function is used to extract knowledge: in, Number of pathological images and Indicates that they are respectively composed of HGSurvNet and Output survival risk probability; Step 5: Update the global network based on the local network of each center and perform survival prediction.

2. The hypergraph-based federated learning survival prediction method according to claim 1, characterized in that, The specific steps for Step One are as follows: For r centers, independently obtain their respective pathology image-related datasets. For a given pathology image... The dataset consists of Where N is the total number of samples; each pathological image The image is uniformly divided into p patches, and a pre-trained feature extraction model with frozen parameters is used to extract the image of each pathological image and embed it into... , where f is the image embedding extracted from the patch map by the pre-trained feature extraction model; Therefore, the image embeddings of dataset D are obtained as follows: ; Then record the coordinates of the top left corner of each patch image on the pathology image. Where h is the height of the pathological image and w is the width of the pathological image, the coordinate information of each pathological image is further obtained as follows: ; This yields the coordinate information of dataset D. ; Finally, the image embeddings F and coordinate information C of the dataset are fused to form the feature information of the dataset: Among them, the characteristic information of a single pathological image Defined as follows: in Concatenate the two vectors.

3. A hypergraph-based federated learning survival prediction method according to claim 1 or 2, characterized in that, The pre-trained feature extraction model used is ResNet-34.

4. The hypergraph-based federated learning survival prediction method according to claim 3, characterized in that, The specific steps for step two are as follows: Each pathology image The p patch graphs in the graph are mapped to the vertex set of the hypergraph. , where the vertex for Mapping in the hypergraph and defining Vertex features, where By calculating vertex features The Euclidean distance is used to measure the distance between vertices, where the Euclidean distance is defined as follows: in Represents vertices and The Euclidean distance between them; and Representing vertices respectively and The feature is C, which represents the dimension of the feature. and Representing vertices respectively and The value of the feature in the c-th dimension, for each pathological image in dataset D. All are mapped to the vertices of their respective hypergraphs using the method described above; Next, for each vertex, the Euclidean distance between that vertex and other vertices is scaled to [0, 1], as shown in the following formula: ； in Indicates from vertex The maximum Euclidean distance to other vertices; For scaled vertices Euclidean distances to other vertices; Then, the ϵ-ball and K-NN graph construction methods are used simultaneously to obtain the nearest neighbors of the modal data in the modal dataset. K-NN is a graph structure built based on the relationship between each point and its k nearest neighbors; ϵ-ball is a graph structure built based on the relationship between each vertex and its neighbors whose distance is less than a certain radius ϵ. The specific steps are as follows: First, the vertices are calculated using the ϵ-ball graph construction method. The neighbor subset is calculated as follows: in Vertices obtained using the ϵ-ball construction method The neighbor subset, where th is a pre-set distance threshold; At the same time, the K-NN graph construction method is used to calculate vertices. The neighbor subset is calculated as follows: Where K is the pre-set number of neighboring vertices. Represents the vertices obtained using the K-NN graph construction method. The neighbor subset, sort(·) denotes the distance sorting algorithm, | | = K; In summary, by using two different graph construction methods, for each vertex It can generate a set of neighboring vertices. and The final set of neighboring vertices is as follows: Based on the set of neighboring vertices, the hyperedge set is defined as follows: The correlation matrix can be obtained from the neighbor vertex set. The correlation matrix is ​​defined as follows: in Let be the element value in the a-th row and b-th column of the correlation matrix H.

5. The hypergraph-based federated learning survival prediction method according to claim 4, characterized in that, Step five is as follows: The global network update is defined by the following objectives: For machine learning problems, take That is, using model parameters For example The loss for prediction; since there are r centers, the data is divided among these centers, where It is the center The index set of the data points, where Therefore, the above objectives can be rewritten as: For data from r centers, the pathological image data of each center are processed using the method in step one. ; The feature information of r datasets The inputs are respectively fed into its own HGSurvNet and local network. In the middle, the local network is updated end-to-end according to the loss function in step four; finally, the global network is updated according to the above objectives. ; Finally, for any pathological image The information is processed into feature information using the method in step 1. Then, the feature vectors are further obtained through the trained HGSurvNet. ,Will As a global network The input is a scalar representing the number of days a patient survives after diagnosis, which is then output.

Images