Ovarian cancer subtype classification method based on prototype learning and multi-view deep embedding clustering

Abstract

The invention relates to the technical field of pathological image analysis and mining, and particularly discloses an ovarian cancer subtype classification method based on prototype learning and multi-view deep embedding clustering, and the method comprises the following steps: S1, collecting a tissue pathological image of an ovarian cancer patient and a corresponding full-view digital pathological image; and S2, generating a multi-view data set. According to the ovarian cancer subtype classification method based on prototype learning and multi-view deep embedding clustering, the problem that in the prior art, patch-level labels are generally lacked in the field of multi-instance pathological images, so that many natural image processing methods cannot be applied to the field of pathological images is solved. A ResNet backbone network is used for extracting features of pathological images under the maximum magnification, a small number of pathology prototypes are introduced to guide deep embedded clustering through pathology expert priori knowledge, a pathology image spectrogram is introduced to serve as a reference view, and the accuracy and stability of clustering are enhanced.

Description

Technical Field

[0001] This invention relates to the field of pathological image analysis and mining technology, specifically to a method for classifying ovarian cancer subtypes using prototype learning and multi-view deep embedding clustering. Background Technology

[0002] Ovarian cancer is the eighth most common malignant tumor among women worldwide. Detection and diagnosis of ovarian cancer are very difficult, screening is ineffective, and nonspecific symptoms similar to those of menopause appear. The disease includes primary malignant tumors of the ovaries, fallopian tubes, and peritoneum. By the time of diagnosis, it has usually already begun to spread in the abdomen. This typical late-stage diagnosis makes ovarian cancer a particularly deadly disease. Most ovarian cancers are epithelial cancers, mainly classified into four histological subtypes: high-grade serous carcinoma (HGSC), clear cell carcinoma (CCOC), endometrioid carcinoma (ENOC), and mucinous carcinoma (MUC).

[0003] In traditional pathological diagnosis, pathologists directly examine pathological slides under a microscope to make a diagnosis. The pathologist's professional knowledge and subjectivity can affect the consistency of the diagnostic results. However, the breakthrough progress of deep learning methods has made automated pathological image analysis possible. Deep learning research in the field of pathological images has lagged behind that in the field of natural images. This is mainly because the size of WSI (Wide Image Sensor) is large (tens of thousands of pixels), which makes it impossible to directly use conventional deep learning methods in the field of pathological images. Currently, researchers have proposed a deep learning method specifically for the field of digital pathology—Multiple Instance Learning (MIL). However, this method lacks patch-level labels, which means that only weak supervision can be used to train the model, which greatly limits the accuracy of the classification model and the ability to locate cancerous tissue. Summary of the Invention

[0004] The purpose of this invention is to provide a subtype classification method for ovarian cancer using prototype learning and multi-view deep embedding clustering, in order to solve the problem mentioned in the background art that the existing multi-instance learning methods applied to the field of digital pathology of ovarian cancer lack patch-level labels, which limits the accuracy of ovarian cancer pathological diagnosis.

[0005] To achieve the above objectives, the present invention provides the following technical solution: an ovarian cancer subtype classification method based on prototype learning and multi-view deep embedding clustering, comprising the following steps: S1. Acquire tissue pathology images and corresponding full-view digital pathology images of ovarian cancer patients; S2. Generate a multi-view dataset, including: (a) Spatial domain view: Image patch of tissue region extracted from the pathological image; (b) Frequency domain view: Convert the patch into a spectral representation; S3. Use a ResNet backbone network pre-trained on the ImageNet dataset to extract deep feature embeddings for spatial domain views or frequency domain views. S4. Construct a multi-view benign and malignant prototype set: Based on the annotations of pathologists, select the feature embeddings of typical patches and their spectrograms of benign regions as benign prototypes, and select the feature embeddings of typical patches and their spectrograms of malignant regions as malignant prototypes. S5. Construct a multi-view deep embedding clustering model, including prototype pairwise constraint K-means clustering, prototype-guided multi-view deep embedding clustering, and bidirectional asymmetric consistency constraint. S6. Establish a pseudo-label inheritance mechanism to construct a pseudo-label sample set; S7. Train the classifier and classify the patches in the test set to obtain patient-level predictions. Preferably, the processing of the original full-view digital slice image WSI in S2 includes the following steps: Remove mismatched data, use a sliding window to cut out 224×224 pixel patches from the histopathological images at a magnification of 40, and filter out sub-patterns with blank backgrounds exceeding the first preset threshold. Obtain the corresponding spectrograms of all sub-plots using FFT; Save the cropped spatial domain patch and its corresponding spectrogram separately according to the corresponding histopathological image.

[0006] By adopting the above technical solution, invalid information can be removed, and effective organizational region features can be preserved, laying the foundation for subsequent feature extraction and clustering.

[0007] Preferably, S3 includes the following parts: ResNet pre-training part: Uses ResNet pre-trained on the ImageNet dataset as the backbone network for feature extraction; Feature extraction: The pre-trained ResNet network is input with patches from different views, and its corresponding feature vectors are extracted; details are as follows: For a WSI slice , in Indicates the first The original feature vectors of each view; Each instance via encoder Extract the embedding features of each view , , The sample feature set constructed by WSI is defined as follows:

[0008] in, This represents the feature set of the i-th sample. Indicates the first Zhang WSI slice number The first view Feature representation of each patch; By adopting the above technical solution, the deep features of multi-view patches can be efficiently extracted using pre-trained networks, which can provide a high-quality feature foundation for subsequent clustering and classification.

[0009] Preferably, the specific implementation process of constructing the multi-view benign / evil prototype set in S4 is as follows: The tissue types in the WSI are divided into cancerous tissues that are useful for subtype diagnosis and benign tissues that are not useful for diagnosis. Based on prior knowledge, n most representative prototype patches are selected for each of these two types of tissues in the area delineated by the pathologist, and a multi-view prototype vector set is constructed. :

[0010] in, Representing the A benign prototype set of views, This is a set of malignant prototypes.

[0011] By adopting the above technical solution and introducing prior pathological knowledge, a set of benign and malignant prototypes can be constructed to guide subsequent clustering.

[0012] Preferably, the specific process of constructing the multi-view deep embedding clustering model in S5 is as follows: S5.1 The prototype pairwise constraint improves the clustering effect of the traditional K-means algorithm by introducing pairwise constraints (must-link and cannot-link). Pairwise constraints are applied to prototype vectors. Instances within a benign prototype are connected, instances within a malignant prototype are connected, there is no connection between benign and malignant prototypes, all instances belonging to benign prototypes belong to the same cluster, all instances belonging to malignant prototypes belong to the same cluster, and instances of benign and malignant prototypes belong to different clusters. S5.2 The prototype-guided multi-view deep embedding clustering consists of two autoencoders. In the embedding space, the samples of each view first approach the most similar prototype, and then the samples form cluster centers and clear cluster structures by guiding the movement of different prototypes. S5.3 The bidirectional asymmetric consistency constraint uses the frequency domain as an auxiliary view to optimize the clustering results by bidirectionally aligning the distribution of prototypes and samples in different views, so that there are samples with the same distribution in both the spatial domain and the frequency domain that are close to the cluster center.

[0013] By adopting the above technical solution and optimizing the clustering effect, the clustering can be made more accurate and stable.

[0014] Preferably, the specific implementation process of the prototype pairwise constraint in S5.1 is as follows: Will Defined as a must-link index set; Defined as a set of cannot-link indexes, the and The calculation formula is as follows:

[0015] in Represents a view benign prototype set The One prototype vector, Represents a view Malignant Prototype Set The One prototype vector; For any two View Prototype and Cluster assignment in clustering The following formula must be satisfied:

[0016] in, express The clusters that were assigned express The assigned cluster, The prototype set is used. The objective function for k-means clustering based on prototype pairwise constraints is defined as:

[0017] in, Represents cosine similarity calculation. For the maximum number of views, The total number of clusters. Indicates the first The first view Each cluster represents a cluster center. Optimization is achieved by maximizing the similarity between samples and cluster centers, with two constraints: similar prototypes must be grouped into the same cluster, while dissimilar prototypes cannot be grouped into the same cluster. The sample set for clustering is composed of samples from each bag. Plus The clustering is composed of prototype vectors, which ensures that prototype vectors of the same type are all assigned to the same cluster in different views, while prototype vectors of different types are not assigned to the same cluster. In subsequent steps, it is necessary to ensure that the allocation relationship between prototype vectors is absolutely stable in order to guide the clustering.

[0018] By adopting the above technical solution and constraining prototype clustering allocation, it is possible to ensure that prototypes of the same type are in the same cluster and prototypes of different types are in different clusters.

[0019] The specific implementation process of prototype-guided deep embedding clustering in S5.2 is as follows: The method performs clustering operations within a single WSI; for the first WSI... Zhang slice , its first The original feature vector set of each view ,in This represents the maximum number of patches. It is the feature dimension; constructing a multi-view autoencoder and decoder ,in and These represent the parameters of the encoder and decoder. and accomplish: and ; in, For input The reconstructed output. The original feature vector. An encoder network consisting of a fully connected network and nonlinear activations. After performing low-dimensional nonlinear mapping, we obtain ,in It is a feature embedded in space. The feature dimension of the embedding space is represented by formulas (6) and (7). The multi-view reconstruction loss is defined as follows:

[0020] By constraining the distribution matching of sample features with that of the medical prototype using KL divergence, we first calculate the soft allocation matrix q based on the Student's t-distribution, then obtain the target distribution p through frequency normalization, and finally minimize the KL divergence between the two:

[0021] in, It is a sample With prototype vector The cosine similarity between students is given by the formula based on the student distribution:

[0022] The model uses multiple prototype vectors scattered in the latent space to replace the role of cluster centers, causing samples to move closer to their respective prototypes. The interactions between these prototypes force them to move, cluster, or move away, a phenomenon known as bipolar clustering. The clustering results are then sharpened using a soft distribution. Construct the target distribution This makes the probability assignment more significant and improves the confidence of the clustering results. The formula is as follows:

[0023] in, Assigned to the first The prototype's soft frequency is used to balance the scale of different allocations; squaring operation Increase the weight of high-confidence assignments; Normalization ensures that it is a probability distribution. While guiding samples to converge towards the prototype vector, the prototype vector itself guides the movement of prototypes through intra-class compactness and inter-class segregation, thus proposing a prototype loss. Regularization constraints:

[0024] in, It is an intra-class compactness loss based on a benign prototype of cosine similarity. It is a malignant prototype. This represents the similarity loss between the two types of prototype centers. and To balance the weights, the class centers of the two prototypes are calculated separately. :

[0025] in Representative view The first of the c-th prototypes A prototype vector, through the prototype vector With the corresponding class center The similarity matrix is ​​used to describe its compactness loss. and inter-class separability loss :

[0026] in, The vector dot product is used to calculate the intra-class compactness of each type of prototype, while the inter-class separation is described as the similarity of the class centers. The intra-class compactness loss brings similar prototype samples closer together and makes them move towards similar prototypes, while the inter-class separation loss makes benign and malignant clusters have clear clustering boundaries.

[0027] By adopting the above technical solution, the inter-class separation loss can be used to make benign clusters and malignant clusters have clear clustering boundaries.

[0028] Preferably, the specific implementation process of the bidirectional asymmetric consistency constraint in S5.3 is as follows: The soft distribution between the obtained different view samples and the prototype vector Split into real-domain soft label distribution Frequency domain soft tag distribution We use KL divergence to align the soft labels of the two domains, as shown in the following formula:

[0029] Where 'a' is the weight coefficient of the main view, and the frequency domain view is used as an auxiliary view. In the experiment, 'a' is set to 0.8 to prevent excessive interference with the main view.

[0030] By adopting the above technical solution, the distribution of real-domain and frequency-domain soft labels can be balanced through bidirectional alignment, thereby optimizing the clustering results.

[0031] Preferably, the pseudo-label inheritance mechanism established in S6 to construct the pseudo-label sample set is specifically described as follows: K clusters are obtained through the multi-view deep embedding clustering model in S5. and cluster centers :

[0032] in, Indicates the first The package is a benign prototype in the embedded space. The assigned cluster is denoted as a benign cluster; The cluster centers of benign clusters and malignant clusters are represented by the cluster centers of the clusters of benign clusters and Similarly.

[0033] For each cluster , Calculate its sample To the cluster center The similarity. Based on the similarity... All samples are sorted in descending order to obtain an ordered sequence. According to a certain proportion Select the top-ranked samples to form a sampling subset:

[0034] Ensure that the most representative samples in each cluster are selected first by controlling the sampling range. The size of the pseudo-label is used to control its precision. For samples sampled from malignant clusters, the precision will be inherited. Subtype tags Samples sampled from benign clusters will be labeled as subtype 0. That is, the instance pseudo-tag assignment rules:

[0035] in, Indicates the first Pseudo-label samples selected from WSI. For the corresponding pseudo-tags. When all packages After all processing is complete, a labeled sample set is obtained. and pseudo tag sets :

[0036] By employing the above technical solution, a high-quality pseudo-label sample set can be constructed to provide data for classifier training.

[0037] Preferably, the pseudo-label inheritance mechanism established in S7 to construct the pseudo-label sample set is specifically described as follows: The pseudo-label sample set is obtained according to the method in S6. The instance classifier consists of 4 fully connected layers and the output node is the number of predicted subtypes N+1. Benign tissue is defined as the 0th subtype. The learning rate, optimizer, training batch size and training rounds of the network are selected. The obtained pseudo-label sample set is input into the set instance classifier for training to obtain the best instance classifier model. For the samples in the test set, each WSI is classified by an instance classifier and then the patient level prediction is obtained by majority voting.

[0038] By employing the above technical solution and training an efficient classifier, reliable patient-level predictions can be achieved.

[0039] Compared with existing technologies, the beneficial effects of this invention are: the ovarian cancer subtype classification method based on prototype learning and multi-view deep embedding clustering: 1. In this invention, the lack of patch-level labels in the field of multi-instance pathological images in the existing technology makes it impossible to apply many natural image processing methods to the field of pathological images. The invention uses the ResNet backbone network to extract the features of pathological images at the maximum magnification, and introduces a small number of pathological prototypes to guide deep embedding clustering by using the prior knowledge of pathology experts. The pathological image spectrogram is introduced as a reference view to enhance the accuracy and stability of clustering. 2. In this invention, a pseudo-label sample set is constructed through a pseudo-label inheritance mechanism to train the classifier. Compared with existing methods, only a very small number of labeled samples are needed to achieve millions of sample annotations, enabling accurate classification of ovarian cancer subtypes and localization of cancerous areas, which can provide an important reference for the diagnosis of ovarian cancer. Attached Figure Description

[0040] Figure 1 This is a schematic diagram of the frame structure of the present invention; Figure 2 This is a schematic diagram of ovarian data samples from Yunnan Cancer Hospital. Figure 3 This is a schematic diagram of the confusion matrix of classification results between the method of the present invention and existing advanced methods on gastric cancer patient data from Yunnan Cancer Hospital. Figure 4 This is a clustering heatmap and classification visualization diagram based on gastric cancer patient data from Yunnan Cancer Hospital, according to the present invention. Detailed Implementation

[0041] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0042] Please see Figures 1-4 This invention provides a technical solution: a method for classifying ovarian cancer subtypes using prototype learning and multi-view deep embedding clustering.

[0043] Flowchart as follows Figure 1 As shown, the specific steps include: S1. Acquire tissue pathology images and corresponding full-view digital pathology images of ovarian cancer patients; S2. Generate a multi-view dataset, including: (a) Spatial domain view: Image patch that extracts tissue regions from pathological images; (b) Frequency Domain View: Converts the patch into a spectral representation; S3. Use a ResNet backbone network pre-trained on the ImageNet dataset to extract deep feature embeddings for spatial domain views or frequency domain views. S4. Construct a multi-view benign and malignant prototype set: Based on the annotations of pathologists, select the feature embeddings of typical patches and their spectrograms of benign regions as benign prototypes, and select the feature embeddings of typical patches and their spectrograms of malignant regions as malignant prototypes. S5. Construct a multi-view deep embedding clustering model, including prototype pairwise constraint K-means clustering, prototype-guided multi-view deep embedding clustering, and bidirectional asymmetric consistency constraint. S6. Establish a pseudo-label inheritance mechanism to construct a pseudo-label sample set; S7. Train the classifier and classify the patches in the test set to obtain patient-level predictions.

[0044] The processing of the original full-view digital slice image WSI in S2 includes the following steps: After removing mismatched data, the histopathological images at a magnification of 40 were cropped into 224×224 pixel patches using a sliding window, and sub-patterns with blank backgrounds exceeding a first preset threshold were filtered out. All sub-patterns were then processed using FFT to obtain their corresponding spectrograms. After these operations, the cropped spatial domain patches and their corresponding spectrograms were saved separately for each histopathological image. The specific datasets are shown in Table 1. Table 1

[0045] In S3, a ResNet50 backbone network pre-trained on the ImageNet dataset is used to extract deep feature embeddings for spatial / frequency domain views. The specific operations are as follows: ResNet pre-training part: Uses ResNet pre-trained on the ImageNet dataset as the backbone network for feature extraction; Feature extraction: The pre-trained ResNet network is input with patches from different views, and its corresponding feature vectors are extracted; details are as follows: For a WSI slice ,in Indicates the first The original feature vectors of each view; each instance via encoder Extract the embedding features of each view , , The sample feature set constructed by WSI is defined as follows:

[0046] in, Indicates the first Zhang WSI slice number The first view The feature representation of each patch, these feature vectors can capture the cell, texture and structural information of the patch; finally, the patch features of each WSI are saved as two .pt files in the spatial domain and frequency domain respectively, for convenient subsequent use.

[0047] The specific implementation process of constructing a multi-view benign / evil prototype set in S4 is as follows: The tissue types in WSI are quite complex, but they can be broadly categorized into cancerous tissues useful for subtype diagnosis and benign tissues not useful for diagnosis. Based on prior knowledge, n representative prototype patches are selected for each of these two types of tissues within the area delineated by the pathologist, constructing a multi-view prototype vector set. :

[0048] in, Representing the A benign prototype set of views, This is a set of malignant prototypes.

[0049] The specific process of constructing a multi-view deep embedding clustering model in S5 is as follows: S5.1 Prototype Pair Constraints: By introducing pair constraints (must-link and cannot-link), the clustering effect of the traditional K-means algorithm is improved. Pair constraints are applied to prototype vectors. Instances within a benign prototype are connected, instances within a malignant prototype are connected, and there is no connection between benign and malignant prototypes. All instances belonging to benign prototypes belong to the same cluster, all instances belonging to malignant prototypes belong to the same cluster, and instances of benign and malignant prototypes belong to different clusters. S5.2 Prototype-guided multi-view deep embedding clustering consists of two autoencoders. In the embedding space, the samples of each view first approach the most similar prototype, and then the samples are driven to form cluster centers and clear cluster structures by guiding the movement of different prototypes. S5.3, Bidirectional Asymmetric Consistency Constraint uses the frequency domain as an auxiliary view to optimize the clustering results by bidirectionally aligning the distribution of prototypes and samples in different views, so that the same distributed samples in both the spatial and frequency domains are close to the cluster center.

[0050] Furthermore, the specific implementation process of prototype pairwise constraints in S5.1 is as follows: Will Defined as a must-link index set; Defined as a set of cannot-link indexes. and The calculation formula is as follows:

[0051] in, Represents a view benign prototype set The One prototype vector, Represents a view Malignant Prototype Set The One prototype vector; For any two View Prototype and Cluster assignment in clustering The following formula must be satisfied:

[0052] in, express The clusters that were assigned express The assigned cluster, The prototype set is used. The objective function for k-means clustering based on prototype pairwise constraints is defined as:

[0053] in, Represents cosine similarity calculation. For the maximum number of views, The total number of clusters. Indicates the first The first view Each cluster represents a cluster center. Optimization is achieved by maximizing the similarity between samples and cluster centers, with two constraints: similar prototypes must be grouped into the same cluster, while dissimilar prototypes cannot be grouped into the same cluster. The sample set for clustering is composed of samples from each bag. Plus The clustering is composed of prototype vectors, which ensures that prototype vectors of the same type are all assigned to the same cluster in different views, while prototype vectors of different types are not assigned to the same cluster. In subsequent steps, it is necessary to ensure that the allocation relationship between prototype vectors is absolutely stable in order to guide the clustering.

[0054] In section 5.2, the multi-view encoder in the prototype-guided deep embedding clustering consists of two autoencoders. The number of hidden layer neurons in each autoencoder is [1024, 521, 256, 128, 32, 128, 256, 512, 1024], where 32 is the feature dimension of the embedding space. For the ... Zhang slice , its first The original feature vector set of each view ,in This represents the maximum number of patches. It is the feature dimension; constructing a multi-view autoencoder and decoder ,in and These represent the parameters of the encoder and decoder. and accomplish: and ; Original feature vector An encoder network consisting of a fully connected network and nonlinear activations. After performing low-dimensional nonlinear mapping, we obtain ,in It is a feature embedded in space. The feature dimension of the embedding space is represented by formulas (6) and (7). The multi-view reconstruction loss is defined as follows:

[0055] By constraining the distribution matching of sample features with that of the medical prototype using KL divergence, we first calculate the soft allocation matrix q based on the Student's t-distribution, then obtain the target distribution p through frequency normalization, and finally minimize the KL divergence between the two:

[0056] in, It is a sample With prototype vector The cosine similarity between students is given by the formula based on the student distribution:

[0057] Compared to the classic DEC paradigm that calculates the distribution relationship between samples and cluster centers in the latent space, this model uses multiple prototype vectors scattered in the latent space to replace the role of cluster centers. This causes samples to move closer to their respective prototypes, and the interactions between prototypes force them to move, cluster, or move away – a process known as bipolar clustering. The clustering results are then sharpened using soft distributions. Construct the target distribution This makes the probability assignment more significant and improves the confidence of the clustering results. The formula is as follows:

[0058] in, Assigned to the first The prototype's soft frequency is used to balance the scale of different allocations; squaring operation Increase the weight of high-confidence assignments; Normalization ensures that it is a probability distribution. While guiding samples to converge towards the prototype vector, the prototype vector itself guides the movement of prototypes through intra-class compactness and inter-class separability, thus proposing a prototype loss. Regularization constraints:

[0059] in, It is an intra-class compactness loss based on a benign prototype of cosine similarity. It is a malignant prototype. This represents the similarity loss between the two types of prototype centers. and To balance the weights, the class centers of the two prototypes are calculated separately. :

[0060]

[0061] in Representative view The first of the c-th prototypes A prototype vector, through the prototype vector With the corresponding class center The similarity matrix is ​​used to describe its compactness loss. and inter-class separability loss :

[0062] in, The vector dot product is used to calculate the intra-class compactness of each type of prototype, while the inter-class separation is described as the similarity of the class centers. The intra-class compactness loss brings similar prototype samples closer together and makes them move towards similar prototypes, while the inter-class separation loss makes benign and malignant clusters have clear clustering boundaries.

[0063] The specific implementation process of the bidirectional asymmetric consistency constraint in S5.3 is as follows:

[0064] The soft distribution between the obtained different view samples and the prototype vector Split into real-domain soft label distribution Frequency domain soft tag distribution We use KL divergence to align the soft labels of the two domains, as shown in the following formula:

[0065] Where 'a' is the weight coefficient of the main view, and the frequency domain view is used as an auxiliary view. In the experiment, 'a' is set to 0.8 to prevent excessive interference with the main view.

[0066] The pseudo-label inheritance mechanism established in S6 to construct a pseudo-label sample set is specifically described as follows: K clusters are obtained by using the multi-view deep embedding clustering model in S5. and cluster centers :

[0067] in, Indicates the first The package is a benign prototype in the embedded space. The assigned cluster is denoted as a benign cluster; The cluster centers of benign clusters and malignant clusters are represented by the cluster centers of the clusters of benign clusters and Similarly.

[0068] For each cluster , Calculate its sample To the cluster center The similarity. Based on the similarity... All samples are sorted in descending order to obtain an ordered sequence. According to a certain proportion Select the top-ranked samples to form a sampling subset:

[0069] Ensure that the most representative samples in each cluster are selected first by controlling the sampling range. The size of the pseudo-label is used to control its precision. For samples sampled from malignant clusters, the precision will be inherited. Subtype tags Samples sampled from benign clusters will be labeled as subtype 0. That is, the instance pseudo-tag assignment rules:

[0070] in, Indicates the first Pseudo-label samples selected from WSI. For the corresponding pseudo-tags. When all packages After all processing is complete, a labeled sample set is obtained. and pseudo tag sets :

[0071] Furthermore, S7 trains a classifier and classifies the patches in the test set: definition Fully connected layer network:

[0072] in, for The weight matrix of the layer, for The number of neurons in a layer ; for Layer bias vector, For ReUL activation function, For batch normalization operations, the output header is defined as follows:

[0073] in, For the predicted one-hot vector, on the pseudo-label sample set Based on formulas (21) and (22), the multi-class cross-entropy loss during the classification stage is defined as follows:

[0074] in, This represents all trainable parameters in the network. Indicates the first Are the pseudo-labels of each sample categories? , Indicates the first Each sample belongs to category The probability is that, for samples in the test set, each WSI is classified by an instance classifier and then the patient level prediction is obtained by majority voting.

[0075] During the overall model training process, after calculating the loss, the backpropagation algorithm is used for optimization. The Adam optimizer is selected for optimization, with MLP layers, batch size=4096, and pseudo-label samples are re-divided into training and validation sets in a 4:1 ratio. The maximum epoch is set to 200 or the early stopping mechanism is triggered by the validation set loss to end the training. Packet-level prediction is obtained by majority voting.

[0076] To demonstrate the effectiveness of the proposed method, numerous experiments were conducted on ovarian cancer subtype data from Yunnan Cancer Hospital and publicly available datasets. The ablation experiment results are shown in Table 2. The comparison results with existing advanced methods are shown in Table 3. The visualization of the results of this invention on ovarian cancer subtype data from Yunnan Cancer Hospital is shown in Figure 1. Table 2. Results of the model ablation experiment (%)

[0077] The experimental results are shown in Table 2. The accuracy (ACC) of the base model IDEC was 84.91%. After adding the pairwise constraint of benign and malignant prototypes, the ACC increased to 88.68%, an increase of 3.77%, which is a significant improvement. This indicates that after introducing prior knowledge of pathology, clustering does not simply consider the distribution characteristics of the data itself; the model knows how to cluster. However, adding intra-cluster compactness loss alone is still effective. and inter-class separability loss The effect will diminish, especially when added alone. The significant drop in accuracy may be due to the model, while searching for the optimal solution, having prototype vectors far from the class centers pull samples from another cluster closer, resulting in an unclear boundary between the two clusters. However, by combining intra-class compactness and inter-class separability losses, the model finds a clear optimization direction, and the role of prototype vectors is fully utilized, achieving an ACC of 91.51%. Adding a frequency domain view, which references data distributions from both the original image and the spectrogram, and uses the original image as the primary data source with the frequency domain image as a secondary source for clustering, significantly enhances cluster robustness, achieving an ACC of 92.45%, with improvements across all metrics compared to a single view.

[0078] Table 3. Classification performance comparison with existing methods (%)

[0079] This invention was compared with several classic multi-instance learning algorithms, and the results are shown in Table 3. Specifically, our method achieved the best performance in all comparison methods, with an average accuracy 3% higher than the second-best model. It is significantly higher than the bag classification methods ABMIL, CLAM, DSMIL, and TransMIL. The effectiveness of the proposed method is demonstrated through examples.

[0080] Working principle: First, histopathological images of ovarian cancer patients and corresponding full-view digital pathological images are acquired to generate a multi-view dataset. Deep feature embeddings are extracted through a ResNet backbone network pre-trained on ImageNet. Based on the pathologist's annotations, the feature embeddings of typical patches and their spectrograms of benign regions are selected as benign prototypes, and the feature embeddings of typical patches and their spectrograms of malignant regions are selected as malignant prototypes. A multi-view deep embedding clustering model is constructed, and K-means is improved using prototype pairwise constraints to ensure that prototypes of the same type are in the same cluster and prototypes of different types are in different clusters. Prototype-guided multi-view deep embedding clustering is achieved through dual autoencoders, making samples converge towards prototypes and form a clear cluster structure. The clustering results are optimized with the help of bidirectional asymmetric consistency constraints. Then, through a pseudo-label inheritance mechanism, representative samples are selected from the clustering results to construct a pseudo-label sample set. A classifier consisting of 4 fully connected layers is trained. After classifying the test set patches, patient-level predictions are obtained through majority voting, realizing the classification of ovarian cancer subtypes and the localization of cancerous areas.

[0081] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention.

Claims

1. A method for classifying ovarian cancer subtypes using prototype learning and multi-view deep embedding clustering, characterized by: Includes the following steps: S1. Acquire tissue pathology images and corresponding full-view digital pathology images of ovarian cancer patients; S2. Generate a multi-view dataset, including: (a) Spatial domain view: Image patch of tissue region extracted from the pathological image; (b) Frequency domain view: Convert the patch into a spectral representation; S3. Use a ResNet backbone network pre-trained on the ImageNet dataset to extract deep feature embeddings for spatial domain views or frequency domain views. S4. Construct a multi-view benign and malignant prototype set: Based on the annotations of pathologists, select the feature embeddings of typical patches and their spectrograms of benign regions as benign prototypes, and select the feature embeddings of typical patches and their spectrograms of malignant regions as malignant prototypes. S5. Construct a multi-view deep embedding clustering model, including prototype pairwise constraint K-means clustering, prototype-guided multi-view deep embedding clustering, and bidirectional asymmetric consistency constraint. S6. Establish a pseudo-label inheritance mechanism to construct a pseudo-label sample set; S7. Train the classifier and classify the patches in the test set to obtain patient-level predictions.

2. The ovarian cancer subtype classification method based on prototype learning and multi-view deep embedding clustering according to claim 1, characterized in that: The processing of the original full-view digital slice image WSI in S2 includes the following steps: Remove mismatched data, use a sliding window to cut out 224×224 pixel patches from the histopathological images at a magnification of 40, and filter out sub-patterns with blank backgrounds exceeding the first preset threshold. Obtain the corresponding spectrograms of all sub-plots using FFT; Save the cropped spatial domain patch and its corresponding spectrogram separately according to the corresponding histopathological image.

3. The ovarian cancer subtype classification method based on prototype learning and multi-view deep embedding clustering according to claim 1, characterized in that: S3 includes the following parts: ResNet pre-training part: Uses ResNet pre-trained on the ImageNet dataset as the backbone network for feature extraction; Feature extraction: The pre-trained ResNet network is input with patches from different views, and its corresponding feature vectors are extracted; details are as follows: For a WSI slice ,in Indicates the first The original feature vectors of each view; Each instance via encoder Extract the embedding features of each view , , , The feature dimension of the embedded vector is D, and the sample feature set constructed by WSI is defined as follows: ； in, This represents the feature set of the i-th sample. Indicates the first Zhang WSI slice number The first view Feature representation of each patch.

4. The ovarian cancer subtype classification method based on prototype learning and multi-view deep embedding clustering according to claim 1, characterized in that: The specific implementation process of constructing the multi-view benign / evil prototype set in S4 is as follows: The tissue types in the WSI are divided into cancerous tissues that are useful for subtype diagnosis and benign tissues that are not useful for diagnosis. Based on prior knowledge, n most representative prototype patches are selected for each of these two types of tissues in the area delineated by the pathologist, and a multi-view prototype vector set is constructed. : ； in, Representing the A benign prototype set of views, This is a set of malignant prototypes.

5. The ovarian cancer subtype classification method based on prototype learning and multi-view deep embedding clustering according to claim 1, characterized in that: The specific process of constructing the multi-view deep embedding clustering model in S5 is as follows: S5.1 The prototype pairwise constraints improve the clustering effect of the traditional K-means algorithm by introducing pairwise constraints must-link and cannot-link. Pairwise constraints are applied to prototype vectors. Instances within a benign prototype are connected, instances within a malignant prototype are connected, there is no connection between benign and malignant prototypes, all instances belonging to benign prototypes belong to the same cluster, all instances belonging to malignant prototypes belong to the same cluster, and instances of benign and malignant prototypes belong to different clusters. S5.2 The prototype-guided multi-view deep embedding clustering consists of two autoencoders. In the embedding space, the samples of each view first approach the most similar prototype, and then the samples are driven to form cluster centers and clear cluster structures by guiding the movement of different prototypes. S5.3 The bidirectional asymmetric consistency constraint uses the frequency domain as an auxiliary view to optimize the clustering results by bidirectionally aligning the distribution of prototypes and samples in different views, so that there are samples with the same distribution in both the spatial domain and the frequency domain that are close to the cluster center.

6. The ovarian cancer subtype classification method based on prototype learning and multi-view deep embedding clustering according to claim 5, characterized in that: The specific implementation process of the prototype pairwise constraint in S5.1 is as follows: Will Defined as a must-link index set; Defined as a set of cannot-link indexes, the and The calculation formula is as follows: ； in, Represents a view benign prototype set The One prototype vector, Represents a view Malignant Prototype Set The One prototype vector; For any two View Prototype and Cluster assignment in clustering The following formula must be satisfied: ； in, express The clusters that were assigned express The assigned cluster, The prototype set is used; the objective function for k-means clustering based on prototype pair constraints is defined as: ； in, Represents cosine similarity calculation. For the maximum number of views, The total number of clusters; Indicates the first The first view Each cluster represents a cluster center; optimization is achieved by maximizing the similarity between samples and cluster centers, including two constraints: similar prototypes must be assigned to the same cluster, while dissimilar prototypes cannot be assigned to the same cluster; the sample set for clustering is composed of samples from each bag. Plus The clustering is composed of prototype vectors, which ensures that prototype vectors of the same type are all assigned to the same cluster in different views, while prototype vectors of different types are not assigned to the same cluster. In subsequent steps, it is necessary to ensure that the allocation relationship between prototype vectors is absolutely stable in order to guide the clustering.

7. The ovarian cancer subtype classification method based on prototype learning and multi-view deep embedding clustering according to claim 5, characterized in that: The specific implementation process of prototype-guided deep embedding clustering in S5.2 is as follows: The method performs clustering operations within a single WSI; for the first WSI... Zhang slice , its first The original feature vector set of each view ,in This represents the maximum number of patches. It is the feature dimension; Building a multi-view autoencoder and decoder ,in and These represent the parameters of the encoder and decoder. and accomplish: ； and ； in, For input The reconstructed output; the original feature vector An encoder network consisting of a fully connected network and nonlinear activations. After performing low-dimensional nonlinear mapping, we obtain ,in It is a feature embedded in space. The feature dimension of the embedding space is represented by formulas (6) and (7). The multi-view reconstruction loss is defined as follows: ； By constraining the sample features with the distribution of the medical prototype through KL divergence, we first calculate the soft assignment matrix q based on the student distribution, then obtain the target distribution p through frequency normalization, and finally minimize the KL divergence between the two: ； in, It is sample embedding With prototype vector The cosine similarity between students is given by the formula based on the student distribution: ； The model uses multiple prototype vectors scattered in the latent space to replace the role of cluster centers, causing samples to move towards their respective prototypes. The interactions between these prototypes force them to move, cluster, or move away. The clustering results are then sharpened using a soft distribution. Construct the target distribution This makes the probability assignment more significant and improves the confidence of the clustering results. The formula is as follows: ； in, Indicates allocation to the first The prototype's soft frequency is used to balance the scale of different allocations; squaring operation Increase the weight of high-confidence assignments; Normalization ensures that it is a probability distribution. While guiding samples to converge towards the prototype vector, the prototype vector itself guides the movement of prototypes through intra-class compactness and inter-class segregation, thus proposing a prototype loss. Regularization constraints: ； in, It is an intra-class compactness loss based on a benign prototype of cosine similarity. It is a malignant prototype. This represents the similarity loss between the two types of prototype centers. and To balance the weights, the class centers of the two prototypes are calculated separately. : ； in Representative view The first of the c-th prototypes A prototype vector, through the prototype vector With the corresponding class center The similarity matrix is ​​used to describe its compactness loss. and inter-class separability loss : ； in, The vector dot product is used to calculate the intra-class compactness of each type of prototype, while the inter-class separation is described as the similarity of the class centers. The intra-class compactness loss brings similar prototype samples closer together and makes them move towards similar prototypes, while the inter-class separation loss makes benign and malignant clusters have clear clustering boundaries.

8. The ovarian cancer subtype classification method based on prototype learning and multi-view deep embedding clustering according to claim 5, characterized in that: The specific implementation process of the bidirectional asymmetric consistency constraint in S5.3 is as follows: The soft distribution between the obtained different view samples and the prototype vector Split into real-domain soft label distribution Frequency domain soft tag distribution We use KL divergence to align the soft labels of the two domains, as shown in the following formula: ； in, The weighting coefficients are used as the main view, and the frequency domain view is used as an auxiliary view. In the experiment, it was set to 0.8 to prevent excessive interference with the main view.

9. The ovarian cancer subtype classification method based on prototype learning and multi-view deep embedding clustering according to claim 1, characterized in that: The pseudo-label inheritance mechanism established in S6 to construct the pseudo-label sample set is specifically described as follows: K clusters are obtained through the multi-view deep embedding clustering model in S5. and cluster centers : ； in, Indicates the first The package is a benign prototype in the embedded space. The assigned cluster is denoted as a benign cluster; The cluster centers of benign clusters and malignant clusters are represented by the cluster centers of the clusters of benign clusters. and Similarly; For each cluster , Calculate its sample To the cluster center Similarity; based on similarity All samples are sorted in descending order to obtain an ordered sequence. According to a certain proportion Select the top-ranked samples to form a sampling subset: ； Ensure that the most representative samples in each cluster are selected first by controlling the sampling range. The size of the pseudo-label is used to control its precision. For samples sampled from malignant clusters, the precision will be inherited. Subtype tags Samples sampled from benign clusters will be labeled as subtype 0. That is, the instance pseudo-tag assignment rules: ； in, Indicates the first Pseudo-label samples selected from WSI. For the corresponding pseudo-tags; when all packages After all processing is complete, a labeled sample set is obtained. and pseudo tag set : 。 10. The ovarian cancer subtype classification method based on prototype learning and multi-view deep embedding clustering according to claim 1, characterized in that: The pseudo-label inheritance mechanism established in S7 to construct the pseudo-label sample set is specifically described as follows: The pseudo-label sample set is obtained according to the method in S6. The instance classifier consists of 4 fully connected layers and the output node is the number of predicted subtypes N+1. Benign tissue is defined as the 0th subtype. The learning rate, optimizer, training batch size and training rounds of the network are selected. The obtained pseudo-label sample set is input into the set instance classifier for training to obtain the best instance classifier model. For the samples in the test set, each WSI is classified by an instance classifier and then the patient level prediction is obtained by majority voting.

Images