Ovarian cancer slice image classification modeling method and device based on Laplace rank constraint and semi-supervised learning

Abstract

The invention discloses an ovarian cancer slice image classification modeling method based on Laplace rank constraint and semi-supervised learning, and the method comprises the steps: carrying out the preprocessing of a WSI data set, and generating a first view training data set and a second view training data set; performing feature extraction on the first view training data set and the second view training data set according to a pre-training feature extraction model on the interactive supervision network; constructing a training strategy for the classifier model to obtain a first image classification model and a second image classification model; inputting unlabeled samples in the first view training data set and the second view training data set into corresponding image classification models to obtain a prediction result; making a pseudo-label sample generation strategy, determining a pseudo-label sample according to a prediction result, and constructing a pseudo-label data set; clustering the pseudo label data set in the embedding space to obtain a plurality of clusters; and according to the clustered clusters, selecting a third preset number of de-duplicated samples closest to the clustering center, endowing the samples with cluster labels, and filling the samples endowing the cluster labels into the labeled data set, so that iteration is performed after updating of the first view training data set and the second view training data set until a preset termination condition is met, and an optimal image classification model is obtained. According to the method, the problem of patch-level labeling scarcity of the pathological image is effectively relieved, and the classification precision of the model is improved.

Description

Technical Field

[0001] This invention relates to a method and apparatus for classifying and modeling ovarian cancer slice images based on Laplacian rank constraints and semi-supervised learning, belonging to the field of pathological image analysis and mining. Background Technology

[0002] Ovarian cancer is one of the deadliest malignant tumors of the female reproductive system, with its high mortality rate primarily attributed to the insidious nature of early symptoms and the prevalence of advanced diagnosis. Clinical practice and research have clearly demonstrated that ovarian cancer is not a single disease entity, but rather a highly heterogeneous collection of tumors, encompassing multiple subtypes that differ significantly in histopathology, molecular characteristics, clinical behavior, and treatment response. Epithelial ovarian cancer accounts for approximately 90% of all ovarian cancer cases, and according to the World Health Organization's classification standards, it mainly includes four major histological subtypes: high-grade serous carcinoma, endometrioid carcinoma, clear cell carcinoma, and mucinous carcinoma. Therefore, accurate subtyping of ovarian cancer is both a cornerstone and a core challenge for achieving personalized treatment and improving patient prognosis. Data shows that the misdiagnosis rate of manual interpretation in complex cases can reach 10%-30%, while early-stage cancer, if diagnosed and treated promptly, can significantly improve the 5-year survival rate to over 80%. Therefore, developing efficient and accurate automated pathological image classification methods has become an urgent need in the field of medical image analysis.

[0003] With the development of artificial intelligence and deep learning, computational pathology, especially AI technologies represented by deep learning, is rapidly becoming a promising solution to address the subjectivity and complexity challenges in histological diagnosis. Considering the significant resources and time required for precise and detailed annotation of gigapixel-level WSI (Weighted Injection Sequences), semi-supervised learning, a strategy that can simultaneously utilize large amounts of unlabeled data and small amounts of labeled data for model training, is increasingly becoming a highly attractive research direction in this field.

[0004] Modern semi-supervised learning (SSL) methods, such as Unsupervised Data Augmentation (UDA) based on consistency regularization and pseudo-labels, perform well in processing single-instance images. Some studies employ self-supervised learning as a pre-training stage, first learning general features on a large number of unlabeled slices and then fine-tuning on a small number of labeled WSIs. However, the core assumption of these methods is "one sample, one label." In the multi-instance learning (MIL) paradigm, WSI labels are at the "bag" level, while the model needs to learn from tens of thousands of unlabeled instances. If a complete WSI is directly treated as a sample and assigned a pseudo-label, the inherent high heterogeneity within tumor tissue is ignored. This "bag-level" supervision information is too coarse, masking the discriminative information of key local regions and hindering the model's effective learning of fine pathological features. Introducing patches results in an extremely large number of patches, and label acquisition is cumbersome. Therefore, achieving accurate prediction using limited labeled data is an urgent need in current pathological image processing. Summary of the Invention

[0005] This invention provides a method and apparatus for classifying and modeling ovarian cancer slice images based on Laplacian rank constraints and semi-supervised learning. By cleverly integrating a small amount of prior knowledge (i.e., image patch labels), it effectively alleviates the problem of scarce patch-level annotations in pathological images. Furthermore, by training an interactive supervised network according to a training strategy, it achieves dynamic updates of the labeled dataset, thereby improving the classification accuracy of the image classification model of this invention and its performance in precise localization of cancer tissue.

[0006] The technical solution of this invention is:

[0007] According to a first aspect of the present invention, a method for classifying and modeling ovarian cancer slice images based on Laplacian rank constraints and semi-supervised learning is provided, comprising:

[0008] S1. Collect full-view digital tile images, treat each full-view digital tile image as a package, and construct the WSI dataset;

[0009] S2. Preprocess the WSI dataset to generate a dual-view training dataset: a first-view training dataset and a second-view training dataset.

[0010] S3. Construct an interactive supervision network, which includes two parallel branches with identical structures. Each branch includes a feature extraction model and a classifier model.

[0011] S4. Extract features from the first view training set and the second view training dataset using the pre-trained feature extraction model on the interactive supervised network.

[0012] S5. Construct a training strategy for the classifier model to obtain the first and second image classification models;

[0013] S6. Input the unlabeled samples in the first view training dataset and the corresponding augmented unlabeled samples in the second view training dataset into the corresponding image classification model to obtain the prediction results; formulate a pseudo-label sample generation strategy to determine pseudo-label samples based on the prediction results, and then construct a pseudo-label dataset.

[0014] S7. Cluster the pseudo-label dataset in the embedding space to obtain multiple clusters; based on the clusters, select the third preset number of deduplicated samples closest to the cluster center, assign cluster labels to the deduplicated samples as real labels, and fill the labeled dataset with the samples with cluster labels to update the first view training dataset and the second view training dataset; iterate the updated first view training dataset and second view training dataset according to steps S4-S6 until the preset termination condition is reached to obtain the optimal image classification model.

[0015] Further, S2 includes:

[0016] Using a sliding window, image patches of a preset size are cut out from the tissue region in a non-overlapping manner from the full-view digital slice image at a preset magnification, and used as an unlabeled dataset.

[0017] A first preset number of image patches are randomly extracted from the unlabeled dataset to create image patch labels, which are then used as the labeled dataset. The unlabeled dataset is divided into a first training set and a first test set according to the packets. The first training set and the labeled dataset are used as the first view training dataset. The first view training dataset is subjected to random data augmentation to obtain the augmented dataset, which is then used as the second view training dataset.

[0018] Further, S4 includes:

[0019] ResNet pre-training: ResNet-50 was pre-trained based on a publicly available dataset of pathological images to obtain a pre-trained ResNet model;

[0020] Feature extraction: The first view training set and the second view training dataset are respectively input into the pre-trained ResNet model of each branch of the interactive supervision network for feature extraction.

[0021] Further, S5 includes:

[0022] The training strategy for classifier models is as follows: In the first training phase, prototype loss is used to train the labeled samples in the first and second view training datasets for the first two preset number of cycles to obtain the first and second preheating models; In the second training phase, for the remaining cycles, a preheating model constraint mechanism is constructed, and prototype loss and pseudo-label loss are used to perform weighted summation to train the first and second view training datasets to update the corresponding preheating models and obtain the first and second image classification models.

[0023] Furthermore, the process of using prototype loss to train labeled samples for the first second preset number of cycles to obtain a preheating model specifically involves: for the first second preset number of cycles, the features extracted from samples with image patch labels in the first view training dataset and the second view training dataset based on the pre-trained ResNet model are respectively fed into the first MLP model and the second MLP model for training to obtain the first and second preheating models.

[0024] Furthermore, for the remaining periods, a preheating model constraint mechanism is constructed. The prototype loss and pseudo-label loss are weighted and summed to train the first view training dataset and the second view training dataset to update the preheating model. Specifically, for the remaining periods, the features extracted from all samples in the first view training dataset and the second view training dataset based on the pre-trained ResNet model are respectively fed into the first and second preheating models. During the training of the first and second preheating models, the parameters of the corresponding preheating models are updated by backpropagation after weighted summation of cross-entropy loss and pseudo-label loss.

[0025] Furthermore, the strategy for generating pseudo-labeled samples, which determines pseudo-labeled samples based on prediction results, specifically involves: when the prediction results of the first image classification model for unlabeled samples in the first view training dataset are consistent with the prediction results of the second image classification model for the corresponding enhanced unlabeled samples in the second view training dataset; and the confidence of the prediction results is greater than a preset confidence threshold, then pseudo-labeled samples are constructed from the unlabeled samples in the first view training dataset and the labels in the corresponding prediction results.

[0026] Furthermore, the step of clustering the pseudo-label dataset in the embedding space to obtain multiple clusters specifically involves performing Laplacian low-dimensional embedding on the pseudo-label dataset and then performing K-means clustering in the embedding space to obtain multiple clusters.

[0027] According to a second aspect of the present invention, an apparatus for classifying and modeling ovarian cancer slice images based on Laplacian rank constraints and semi-supervised learning is provided, comprising modules of any of the methods described above.

[0028] According to a third aspect of the present invention, a terminal device is provided, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of any of the methods described above.

[0029] The beneficial effects of this invention are as follows: Considering the excessively large pixel count (tens of thousands of pixels) and complex tissue structure of full-slice images, leading to high annotation costs and a scarcity of labeled data, this invention uses a sliding window to cut out image blocks of a preset size from the tissue region of a full-view digital slice image at a preset magnification in a non-overlapping manner to create a small number of labels. Furthermore, it designs a pseudo-label generation strategy based on interactive supervision consensus and high confidence, achieving the construction of a high-quality pseudo-label dataset under mutual supervision. Finally, it uses the Laplacian embedding space of high-quality pseudo-label samples for clustering and dynamically updates labeled samples, improving the generalization of the classification model. Compared to traditional methods, this invention requires only a very small number of labeled samples to complete the annotation task of millions of samples, achieving accurate classification of ovarian cancer slice images and accurate localization of cancerous areas, thus providing crucial evidence for assisting in the diagnosis of ovarian cancer. Attached Figure Description

[0030] Figure 1 This is a flowchart of the present invention.

[0031] Figure 2 The following are examples of ovarian data samples from a tumor hospital provided according to embodiments of the present invention: (a) HGSC serous carcinoma, (b) MUC mucinous carcinoma, (c) CCOC clear cell carcinoma, (d) ENOC endometrial carcinoma, and (e) Benign benign tissue.

[0032] Figure 3 This is a confusion matrix diagram showing the classification results of the method of this invention and existing advanced methods on gastric cancer patient data from a cancer hospital. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this application can be arbitrarily combined with each other.

[0034] Example 1: As Figures 1-3 As shown, according to a first aspect of the present invention, a method for classifying and modeling ovarian cancer slice images based on Laplacian rank constraints and semi-supervised learning is provided, comprising the following steps:

[0035] S1. Collect full-view digital tile images, treat each full-view digital tile image as a package, and construct the WSI dataset.

[0036] S2. Preprocess the WSI dataset to generate a dual-view training dataset: a first-view training dataset and a second-view training dataset.

[0037] Further, S2 includes:

[0038] A sliding window is used to cut out image patches of a preset size from the tissue region in a non-overlapping manner from the full-view digital slice image at a preset magnification, which are used as an unlabeled dataset. Pure background regions with no tissue regions are filtered out through this step.

[0039] A first preset number of image patches are randomly extracted from the unlabeled dataset to create image patch labels, which are then used as the labeled dataset. The unlabeled dataset is divided into a first training set and a first test set according to the packets. The first training set and the labeled dataset are used as the first view training dataset. The first view training dataset is subjected to random data augmentation to obtain the augmented dataset, which is then used as the second view training dataset.

[0040] For example, the data augmentation methods include flipping, rotating, Gaussian noise, and contrast enhancement; random data augmentation is to randomly select one of "flipping, rotating, Gaussian noise, and contrast enhancement". As can be seen from the above, the first view training dataset includes both labeled samples and unlabeled samples.

[0041] S3. Construct an interactive supervision network, which includes two parallel branches with identical structures. Each branch includes a ResNet model and a classifier model.

[0042] S4. Extract features from the first view training set and the second view training dataset using a pre-trained ResNet model on the interactive supervised network.

[0043] Further, S4 includes:

[0044] ResNet pre-training: ResNet-50 is pre-trained based on publicly available pathological image datasets to obtain a pre-trained ResNet model (the publicly available pathological image datasets use various types of publicly available pathological image datasets); for example, ResNet-50 uses an encoder based on a deep convolutional neural network as the backbone network for feature extraction. This invention pre-trains ResNet-50 based on the ImageNet dataset to obtain a pre-trained ResNet model, and using a pre-trained model helps the network learn rich general image features;

[0045] Feature extraction: The first-view training set and the second-view training dataset are input into the pre-trained ResNet models of each branch of the interactive supervised network for feature extraction; the details are as follows:

[0046] For the portion of the first view training set / second view training dataset corresponding to the first training set, each part comes from N full-view digital slice images WSI. Zhang WSI, the WSI is... The sample set consists of several image patches and can be represented as follows:

[0047] ;

[0048] in, It is the first Zhang WSI's Image blocks.

[0049] Will Each image patch in Encoder of an independent input pre-trained ResNet model Extract its corresponding 3D feature vector :

[0050] ;

[0051] After completing the first All of Zhang WSI After feature extraction of each image patch, all eigenvectors By combining these features, a complete set of sample features for the WSI can be constructed. :

[0052] ;

[0053] In specific implementation, this feature set It is organized into a data structure containing a feature matrix and corresponding paths, and saved as a single file through serialization for use by subsequent models.

[0054] The same principle applies to the labeled portions of the first-view training set / second-view training set.

[0055] S5. Classifier model construction and training strategy: In the first training phase, prototype loss is used to train the labeled samples in the first and second view training datasets for the first two preset number of cycles to obtain the first and second preheating models; In the second training phase, for the remaining cycles, a preheating model constraint mechanism is constructed, and prototype loss and pseudo-label loss are used to perform weighted summation to train the first and second view training datasets to update the corresponding preheating models and obtain the first and second image classification models; The prediction results include classification labels and confidence scores.

[0056] Furthermore, the process of training the labeled samples using prototype loss for the first second preset number of cycles to obtain a preheating model specifically involves:

[0057] For the first two preset number of cycles, features extracted from samples with image patch labels in the first view training dataset and the second view training dataset based on the pre-trained ResNet model will be fed into the first MLP model and the second MLP model for training, respectively, to obtain the first and second warm-up models. During the training of the first MLP model and the second MLP model, the parameters of the corresponding MLP model will be updated by backpropagation using cross-entropy loss. This achieves the provision of initial and reliable gradients to the model in a standard supervised learning manner, which is the cornerstone of MLP model learning and ensures that the model first learns basic classification ability from the gold standard data.

[0058] Specifically, for a containing A batch of labeled samples ,in It is the first Feature information extracted from each pathological image patch using ResNet, and This is the corresponding true category label. The data is then fed into two MLP models (M1 and M2), each using standard cross-entropy loss. As the loss function, it is calculated as follows:

[0059] ;

[0060] in, Representative model index; It is the total number of categories (in this embodiment of the invention, it is taken as 5 (4 types of cancer + benign)). It is a model For input Predict its category as The probability of. It is an indicator function, when the actual label equals category When the value is 1, its value is 1; otherwise, its value is 0.

[0061] For the remaining periods, a preheating model constraint mechanism is constructed. The prototype loss and pseudo-label loss are weighted and summed to train the first view training dataset and the second view training dataset to update the preheating model. Specifically, for the remaining periods, the features extracted from all samples in the first view training dataset and the second view training dataset based on the pre-trained ResNet model are respectively fed into the first and second preheating models. During the training of the first and second preheating models, the parameters of the corresponding preheating models are updated by backpropagation after weighted summation of cross-entropy loss and pseudo-label loss.

[0062] Specifically, in the aforementioned warm-up phase, loss calculation is performed only on labeled samples. To utilize the large amount of unlabeled data, this invention introduces the concepts of pseudo-labels and consistency regularization. Within the interactive supervision network framework, this manifests as a cross-pseudo-supervision mechanism: the confident prediction of one model is used as a supervision signal for another model. Specifically:

[0063] 5.1 For the remaining periods, the standard cross-entropy loss is calculated using the S5 method for labeled samples; for samples containing... A batch of unlabeled samples That is, the unlabeled samples in the first view training dataset and the second view training dataset correspond to respectively and The first and second preheating models were used to make predictions, and the probability distributions were obtained. and When the model is applied to the corresponding samples and The prediction confidence exceeds a preset threshold. At that time, this prediction was considered reliable. And use its prediction results as pseudo-labels (such as the pseudo-labels obtained by the second preheating model). Then we use the pseudo-labels of one model to supervise the predictions of another model on the corresponding samples. Conversely, we use the pseudo-labels of one model to supervise the predictions of the other model on the corresponding samples. The formula for calculating the pseudo-label loss is as follows:

[0064] ;

[0065] ;

[0066] in, The pseudo-label loss of the first preheating model, The pseudo-label loss for the second preheating model; Pseudo-labels representing the second preheating model Supervised first warm-up model on corresponding samples The prediction above. It is an indicator function, which only applies when the condition within the parentheses (prediction confidence level is greater than a threshold) is met. The loss is only 1 when the condition is met, thus calculating the loss only for high-confidence samples. This loss prompts the two networks to reach a prediction consensus on unlabeled data, greatly expanding the amount of information available for training.

[0067] 5.2 The final model training objective is to minimize the weighted sum of the two loss functions mentioned above (prototype loss (i.e., cross-entropy loss) and pseudo-label loss). The overall loss function for each preheating model is... Defined as:

[0068] ;

[0069] in, and It is a function that changes over training time. These are the weight coefficients of the pseudo-label loss, used in the early stages of training. Set to 0 (for example, the total number of cycles is 120, the prototype loss is used to train the labeled samples for the first 70 cycles, that is, the second preset number is 70, and the loss is introduced starting from the 71st cycle). It increases linearly to 0.4 at 120 cycles. These are the weighting coefficients of the prototype loss (in the early stages of training). Set to 1 (i.e., take 1 for the first 70 cycles), gradually decrease from the 71st cycle onwards, and linearly decrease to 0.6 at 120 cycles. Represents the model index.

[0070] S6. Input the unlabeled samples from the first view training dataset and the corresponding augmented unlabeled samples from the second view training dataset into the corresponding image classification model to obtain the prediction results; formulate a pseudo-label sample generation strategy to determine pseudo-label samples based on the prediction results, and then construct a pseudo-label dataset.

[0071] Furthermore, the strategy for generating pseudo-labeled samples, which determines pseudo-labeled samples based on prediction results, specifically involves: when the prediction results of the first image classification model for unlabeled samples in the first view training dataset are consistent with the prediction results of the second image classification model for the corresponding enhanced unlabeled samples in the second view training dataset; and the confidence of the prediction results is greater than a preset confidence threshold, then pseudo-labeled samples are constructed from the unlabeled samples in the first view training dataset and the labels in the corresponding prediction results, and high-quality pseudo-labeled samples are stored in the pseudo-labeled dataset.

[0072] S7. Perform Laplacian low-dimensional embedding on the pseudo-label dataset. Reduce the 1024-dimensional features to 5 dimensions through rank constraints. Perform K-means clustering (5 clusters) in the embedding space to obtain multiple clusters. Based on the clusters, select the A samples closest to the cluster centers. Deduplicate the selected samples based on the labeled dataset (if the selected sample number is the same as a sample in the current labeled dataset, delete the selected sample) until the number of deduplicated samples reaches a third preset number (in this embodiment, the third preset number is 10). Assign cluster labels to the deduplicated samples as real labels. Fill the labeled dataset in step S2 with the samples with cluster labels to update the first view training dataset and the second view training dataset. Iterate the updated first view training dataset and second view training dataset again according to steps S4-S6 until the preset termination condition is reached (e.g., the maximum number of iterations / the number of times the classification accuracy of the image classification model decreases reaches a preset number) to obtain the optimal image classification model. For example, the preset number of iterations is 5. That is, if the classification accuracy of the image classification model decreases 5 times during the multiple iterations according to the present invention, the process is terminated. And from the multiple iterations, the model with the best classification accuracy is selected as the optimal image classification model.

[0073] Specifically, samples from the obtained high-quality pseudo-label dataset are input into the Laplacian network, which is specifically:

[0074] 7.1 Map the high-dimensional original input data to a low-dimensional embedding space to generate an embedding representation. To avoid the network learning trivial solutions and to ensure good spectral characteristics, for Perform orthogonalization constraints:

[0075] First calculate Cholesky decomposition:

[0076] ;

[0077] in, It is a lower triangular matrix obtained through Cholesky decomposition; Represents the identity matrix. It is a small positive number used to ensure the positive definiteness and numerical stability of the matrix. The orthogonalized embedding matrix is ​​then calculated using the following formula. :

[0078] ;

[0079] in, The scaling factor (set to the number of samples in the current batch) is the scaling factor in the orthogonalization constraint. The matrix after this transformation... satisfy Their column vectors are mutually orthogonal, thus avoiding the model from collapsing to a low-rank trivial solution.

[0080] 7.2 Constructing an Adaptive Affinity Matrix Based on Orthogonal Embedding: Using the orthogonal embedding moments E obtained in step 7.1, an adaptive affinity matrix is ​​constructed. For any two data points and Define a mixing distance :

[0081] ;

[0082] in, It is a primitive feature. It is the corresponding orthogonal embedding. It is a hyperparameter used to balance the importance of the original space and the embedded space.

[0083] For each point Find its distance-based of Let the nearest neighbors be denoted as set. Elements in the affinity matrix Only When the time is non-zero, its value is given by the following formula:

[0084] ;

[0085] in, It is a point To its first The distance to the nearest neighbor. This method can adaptively scale the affinity value. It is not sensitive to data scale. Finally, through... This is to ensure the symmetry of the affinity matrix.

[0086] 7.3 Using the adaptive affinity matrix constructed in step 7.2 Define the degree matrix (its diagonal elements are) (The sum of the corresponding rows), and construct a normalized symmetric Laplace matrix. :

[0087] ;

[0088] Construct a Laplace loss function based on this symmetric Laplace matrix. And minimize this loss to encourage similar samples to cluster in the embedding space:

[0089] ;

[0090] Simultaneously, a rank constraint mechanism is introduced: the Laplacian matrix is ​​calculated at fixed training intervals. The eigenvalues, counting the number of eigenvalues ​​close to zero. .like ( If the preset number of clusters is used, then increase the hyperparameter in step 7.2. To enhance the influence of the embedding space on the graph structure, thus promoting cluster separation; if Then Restore the initial values. Finally, obtain the optimized embedding space E.

[0091] Furthermore, K-means clustering is used to divide the samples into k clusters in the optimized embedding space. The cluster center of each cluster is calculated, and the Euclidean distance from the samples within the cluster to the cluster center is also calculated. From each cluster, the 10 deduplicated samples closest to the cluster center are selected, and their corresponding cluster labels are assigned as the true labels.

[0092] S8. Use the optimal image classification model to classify the patches in the first test set, and finally obtain the WSI-level packet prediction by majority voting (that is, for all patches under the current packet, the category with the largest proportion of types is taken as the current WSI packet prediction result).

[0093] According to a second aspect of the present invention, an ovarian cancer slice image classification and modeling apparatus based on Laplacian rank constraint and semi-supervised learning is provided, comprising modules of any one of the methods described above; specifically comprising: a first construction module, used to collect full-view digital slice images, treating each full-view digital slice image as a package to construct a WSI dataset; a generation module, used to preprocess the WSI dataset to generate a dual-view training dataset: a first-view training dataset and a second-view training dataset; a second construction module, used to construct an interactive supervision network, the interactive supervision network comprising two parallel branches with identical structures, each branch comprising a feature extraction model and a classifier model; a feature extraction module, used to extract features from the first-view training set and the second-view training dataset according to the pre-trained feature extraction model on the interactive supervision network; and a first acquisition module, used to construct a training strategy for the classifier model to obtain the first and second views. The system consists of two modules: a second image classification model and a third construction module. The second module takes unlabeled samples from the first view training dataset and corresponding augmented unlabeled samples from the second view training dataset as inputs to the corresponding image classification model to obtain prediction results. It then formulates a pseudo-label sample generation strategy to determine pseudo-label samples based on the prediction results, thereby constructing a pseudo-label dataset. The third module obtains clusters of the pseudo-label dataset in the embedding space. Based on the clusters, it selects a third preset number of deduplicated samples closest to the cluster center, assigns cluster labels as real labels to these samples, and fills the labeled dataset with these cluster-labeled samples, thus updating the first and second view training datasets. The updated first and second view training datasets are iterated through the feature extraction module and the third feature extraction module until a preset termination condition is reached, obtaining the optimal image classification model. For details not described in the above modules, please refer to the relevant descriptions in this embodiment.

[0094] According to a third aspect of the present invention, a terminal device is provided, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of any of the methods described above.

[0095] Example 2: The following describes an optional embodiment of the present invention in conjunction with experimental data:

[0096] I. Full-field digital slice images obtained from ovarian cancer subtype data scans at the Cancer Hospital (e.g.) Figure 2As shown in the example, construct the WSI dataset (i.e., the YNC_OC dataset, which includes 400 labeled full-view digital slide images, with 100 full-view digital slide images for each cancer type. The labels of the full-view digital slide images are treated as package-level labels, and the labels of subsequent patches are treated as block-level labels). For the full-view digital slide images at 20x magnification, use a sliding window to cut out 256×256 pixels from the tissue region in a non-overlapping manner. The process of generating image patch data at the pixel level resulted in 400 packets, with each WSI slice yielding one patch, resulting in a total of 4,487,334 image patch data of size 256×256 pixels. This was used as an unlabeled dataset (i.e., an unlabeled dataset without block-level labels). Then, a very small number of patches were labeled by pathology experts (in this embodiment, 500 patches were used to create labels, categorized into four types: high-grade serous carcinoma (HGSC), endometrioid carcinoma (ENOC), clear cell carcinoma (CCOC), and mucinous carcinoma (MUC)). This was used as a labeled dataset (i.e.,...). The dataset consists of a labeled dataset with block-level labels; the unlabeled dataset is divided into a first training set and a first test set in a 7:3 ratio (resulting in 280 packets in the first training set and 120 packets in the first test set; since this invention derives packet predictions from multi-instance patch-level predictions, the packets in the test set are balanced, with 30 packets for each of the four categories); the first training set and the labeled dataset are used as the first view training dataset; random data augmentation is performed on the first view training dataset to obtain the augmented dataset, which is used as the second view training dataset. The WSI dataset is shown in Table 1.

[0097] Table 1

[0098]

[0099] 2. Preprocess the WSI dataset to generate a dual-view training dataset: a first-view training dataset and a second-view training dataset.

[0100] 3. Construct an interactive supervision network, which includes two parallel branches with identical structures, each branch including a ResNet model and a classifier model;

[0101] IV. Feature extraction is performed on the first view training set and the second view training dataset based on the pre-trained ResNet model on the interactive supervised network.

[0102] V. Classifier Model Construction and Training Strategy: In the first training phase, prototype loss is used to train the labeled samples in the first view training dataset and the second view training dataset for the first 70 cycles to obtain the first and second pre-warm-up models; In the second training phase, for cycles 71-120, a pre-warm-up model constraint mechanism is constructed, and prototype loss and pseudo-label loss are used to perform weighted summation to train the first view training dataset and the second view training dataset to update the corresponding pre-warm-up models and obtain the first and second image classification models; The prediction results include classification labels and confidence scores.

[0103] 6. Input the unlabeled samples from the first view training dataset and the corresponding augmented unlabeled samples from the second view training dataset into the corresponding image classification model to obtain the prediction results; formulate a pseudo-label sample generation strategy to determine pseudo-label samples based on the prediction results, and then construct a pseudo-label dataset; in this embodiment of the invention, a preset confidence threshold of 0.63 is set.

[0104] It should be noted that unlabeled samples are used to calculate the pseudo-label loss. An optimization mechanism is implemented during loss calculation: when two networks predict the same unlabeled sample, and the corresponding prediction probability (confidence) of the softmax algorithm is greater than 0.6, the pseudo-label is defined as a high-quality pseudo-label and participates in the pseudo-label loss calculation. If the confidence is less than 0.6 or the predicted labels of the two models are inconsistent, the calculation of the pseudo-label loss is abandoned, as ambiguous predictions negatively optimize the model, as shown in Table 2. When the confidence threshold parameter of this invention is set from 0.3 to 0.6, the accuracy improves; however, when it continues to increase, the accuracy begins to decrease. This indicates that the model is overfitting. Therefore, multiple experiments were conducted in the range of 0.6 to 0.7, and finally, the desired result was selected. The model achieved its optimal accuracy when the parameter value was 0.63, which also indicates that the parameter... The settings are crucial to model performance.

[0105] Table 2

[0106]

[0107] 7. Perform Laplacian low-dimensional embedding on the pseudo-label dataset, reducing the 1024-dimensional features to 5 dimensions through rank constraints. Perform K-means clustering in the embedding space to obtain 5 clusters. Based on the clusters, select the A samples closest to the cluster center. From each cluster, select the 10 deduplicated samples closest to the cluster center and assign their corresponding cluster labels as the true labels. Fill the labeled dataset with the samples with cluster labels to update the first-view training dataset and the second-view training dataset. Iterate the updated first-view training dataset and second-view training dataset again according to steps 4-6 until the preset termination condition is reached to obtain the optimal image classification model.

[0108] 8. In the final testing phase, the optimal image classification model is used to classify the patches in the first test set. Through package-level aggregation, all patches within a package are subject to majority voting to determine the WSI label of the package.

[0109] To demonstrate the effectiveness of the proposed method, its performance is compared with that of several classic cutting-edge methods (ABMIL, CLAM, FixMatch, Giga-SSL, ICMIL) on the ovarian cancer dataset from a cancer hospital (it should be noted that this invention can also be used on publicly available ovarian cancer datasets, etc.), as shown in Table 3. Figure 3 As shown in the figure. Experimental results show that the present invention outperforms other comparative methods in all evaluation metrics, achieving the best performance. Compared with CLAM and FixMatch, the present invention improves the accuracy, precision, recall, and F1 scores by 5-6%, and is significantly better than the first classic ABMIL method. The effectiveness of the proposed method is demonstrated through examples.

[0110] Table 3

[0111]

[0112] Furthermore, to demonstrate the impact of the mutual constraints of the interactive supervision network and the embedding space constraints of the Laplace module on the model, a triple ablation method was set up: a single-branch network model (Model 1), a single-branch network + Laplace constraint model (Model 2), a single-branch network + Laplace constraint + pseudo-label constraint model (Model 3), and finally, the method of our invention (Double, i.e., parallel branch). As shown in Table 4, it can be seen that the training accuracy of the single-branch network is much lower than that of our invention. This indicates that the interactive supervision network architecture of our invention, through mutual supervision and utilization of the relationships between networks, can obtain richer results than the single-branch network and can more effectively select information-rich samples, thus making good use of semi-supervised learning. In Model 2, it can also be seen that after introducing the Laplace rank constraint module on top of Model 1, the accuracy improved by about 4%. Better constraints on positive and negative sample pairs in the embedding space guide the model to better distinguish between positive and negative samples, thereby improving classification accuracy. When introducing pseudo-label constraints, the model improvement is relatively small because the high-quality pseudo-labels trained by the single-branch network cannot be mutually constrained, making them prone to low-probability misprediction, which actually affects the model's self-optimization.

[0113] Table 4

[0114]

[0115] The specific embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.

Claims

1. A method for classifying and modeling ovarian cancer slice images based on Laplacian rank constraints and semi-supervised learning, characterized in that, include: S1. Collect full-view digital tile images, treat each full-view digital tile image as a package, and construct the WSI dataset; S2. Preprocess the WSI dataset to generate a dual-view training dataset: a first-view training dataset and a second-view training dataset. S3. Construct an interactive supervision network, which includes two parallel branches with identical structures. Each branch includes a feature extraction model and a classifier model. S4. Extract features from the first view training set and the second view training dataset using the pre-trained feature extraction model on the interactive supervised network. S5. Construct a training strategy for the classifier model to obtain the first and second image classification models; S6. Input the unlabeled samples in the first view training dataset and the corresponding augmented unlabeled samples in the second view training dataset into the corresponding image classification model to obtain the prediction results; A pseudo-label sample generation strategy is formulated to determine pseudo-label samples based on the prediction results, and then a pseudo-label dataset is constructed. S7. Cluster the pseudo-label dataset in the embedding space to obtain multiple clusters; Based on the clusters, select the third preset number of deduplicated samples that are closest to the cluster center, assign cluster labels to the deduplicated samples as real labels, and fill the labeled dataset with the samples with cluster labels, thereby updating the first view training dataset and the second view training dataset. The updated first-view training dataset and second-view training dataset are iterated according to steps S4-S6 until the preset termination condition is reached to obtain the optimal image classification model.

2. The ovarian cancer slice image classification and modeling method based on Laplacian rank constraint and semi-supervised learning according to claim 1, characterized in that, The S2 includes: Using a sliding window, image patches of a preset size are cut out from the tissue region in a non-overlapping manner from the full-view digital slice image at a preset magnification, and used as an unlabeled dataset. A first preset number of image patches are randomly extracted from the unlabeled dataset to create image patch labels, which are then used as the labeled dataset. The unlabeled dataset is divided into a first training set and a first test set according to the packets. The first training set and the labeled dataset are used as the first view training dataset. The first view training dataset is subjected to random data augmentation to obtain the augmented dataset, which is then used as the second view training dataset.

3. The ovarian cancer slice image classification and modeling method based on Laplacian rank constraint and semi-supervised learning according to claim 1, characterized in that, The S4 includes: ResNet pre-training: ResNet-50 was pre-trained based on a publicly available dataset of pathological images to obtain a pre-trained ResNet model; Feature extraction: The first view training set and the second view training dataset are respectively input into the pre-trained ResNet model of each branch of the interactive supervision network for feature extraction.

4. The ovarian cancer slice image classification and modeling method based on Laplacian rank constraint and semi-supervised learning according to claim 1, characterized in that, The S5 includes: The training strategy for classifier models is as follows: In the first training phase, prototype loss is used to train the labeled samples in the first and second view training datasets for the first two preset number of cycles to obtain the first and second preheating models; In the second training phase, for the remaining cycles, a preheating model constraint mechanism is constructed, and prototype loss and pseudo-label loss are used to perform weighted summation to train the first and second view training datasets to update the corresponding preheating models and obtain the first and second image classification models.

5. The ovarian cancer slice image classification and modeling method based on Laplacian rank constraint and semi-supervised learning according to claim 4, characterized in that, The process of training labeled samples with prototype loss to obtain a preheating model for the first and second preset number of cycles is as follows: For the first and second preset number of cycles, the features extracted from the samples with image patch labels in the first view training dataset and the second view training dataset based on the pre-trained ResNet model are respectively fed into the first MLP model and the second MLP model for training to obtain the first and second preheating models.

6. The ovarian cancer slice image classification and modeling method based on Laplacian rank constraint and semi-supervised learning according to claim 4, characterized in that, For the remaining periods, a preheating model constraint mechanism is constructed. The prototype loss and pseudo-label loss are weighted and summed to train the first view training dataset and the second view training dataset to update the preheating model. Specifically, for the remaining periods, the features extracted from all samples in the first view training dataset and the second view training dataset based on the pre-trained ResNet model are respectively fed into the first and second preheating models. During the training of the first and second preheating models, the parameters of the corresponding preheating models are updated by backpropagation after weighted summation of cross-entropy loss and pseudo-label loss.

7. The ovarian cancer slice image classification and modeling method based on Laplacian rank constraint and semi-supervised learning according to claim 1, characterized in that, The strategy for generating pseudo-labeled samples, which determines pseudo-labeled samples based on prediction results, specifically involves: when the prediction results of the first image classification model for unlabeled samples in the first view training dataset are consistent with the prediction results of the second image classification model for the corresponding augmented unlabeled samples in the second view training dataset; and the confidence of the prediction results is greater than a preset confidence threshold, then pseudo-labeled samples are constructed from the unlabeled samples in the first view training dataset and the labels in the corresponding prediction results.

8. The ovarian cancer slice image classification and modeling method based on Laplacian rank constraint and semi-supervised learning according to claim 1, characterized in that, The step of clustering the pseudo-label dataset in the embedding space to obtain multiple clusters specifically involves performing Laplacian low-dimensional embedding on the pseudo-label dataset and then performing K-means clustering in the embedding space to obtain multiple clusters.

9. A device for classifying and modeling ovarian cancer slice images based on Laplacian rank constraints and semi-supervised learning, characterized in that, The module includes the method described in any one of claims 1-8.

10. A terminal device, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1-8.

Images