Uterine smooth muscle sarcoma image classification system based on mixed view self-adaption

Abstract

The invention provides a uterine smooth muscle sarcoma image classification system based on mixed view self-adaption, and relates to the technical field of image classification, and the system comprises a data sampling module which is used for sampling patch features; the mixed view self-adaptive learning module is used for performing phenotype-driven patch self-optimization on the patch features, acquiring patch-level feature measurement and phenotype measurement, and generating phenotype-driven features; according to the method, unsupervised inter-patch adaptive learning and compensatory in-patch adaptive learning are introduced into patch self-optimization of phenotype driving, an optimization reward is constructed, and a strategy gradient method is utilized to iterate self-optimization phenotype driving features; and the uncertainty judgment module is used for processing the self-optimized phenotypic driving features through a full connection layer, performing forward parameterization through an improved Softplus activation function, inputting the phenotypic driving features after forward parameterization into a Dirichlet distribution formula, calculating uncertainty scores and judging the probability of image classification so as to obtain an image classification result.

Description

Technical Field

[0001] The present disclosure relates to the technical field of image classification, and particularly to a uterine leiomyosarcoma image classification system based on hybrid view adaptation. Background Art

[0002] The statements in this section merely provide background technical information related to the present disclosure and do not necessarily constitute prior art.

[0003] Uterine leiomyosarcoma (ULMS) is a rare malignant gynecological tumor that originates from the smooth muscle of the uterine wall. It presents as an aggressive cancer with a significant risk of recurrence and death, and the five-year survival rate of patients with metastatic ULMS at the time of initial diagnosis is only 10% to 15%. Whole slide images (WSIs) are widely regarded as the gold standard for diagnosing ULMS. Accurately diagnosing ULMS on WSIs using a high-resolution microscope is crucial for promoting its early detection, formulating personalized treatment plans, and ultimately improving the survival rate of patients. The differentiation between benign and malignant ULMS tumors is mainly based on cell abnormalities, cell division rate, and the presence of tumor cell necrosis. However, due to the high concealment of this tumor and its similarity to uterine benign leiomyomas, the diagnostic process is full of challenges. In addition, the phenotypic diversity of ULMS makes the clinical diagnostic procedure very time-consuming and highly subjective. Therefore, it is particularly important to develop a fully automated algorithm that can accurately diagnose ULMS in different WSIs.

[0004] So far, there has been a lack of research on the automatic diagnosis of ULMS on WSIs using an advanced deep convolutional neural network framework. This is mainly due to the sharp contrast between the limited availability of WSIs and the high demand for data in the framework training process. At the same time, the phenotypic diversity among the patches divided from WSIs also poses certain challenges to the recognition ability of the framework.

[0005] Some existing studies have successfully used different deep convolutional neural networks to automatically classify various tumors in WSIs images. However, the existing methods fail to consider the relationship between patches well, thus introducing redundant information, which in turn interferes with the decision-making process of the network to a certain extent. In addition, a WSI image only contains a side-level label. Under the weak supervision of the side-level label, some patch-level features with insufficient modeling may play a decisive role in tumor recognition. The existing solutions are to reorder or discard the patch features with high scores in the WSI image to enhance other discriminative features, but they ignore the potential diagnostic credibility of these features, thus hindering the network framework from obtaining reliable classification results. Summary of the Invention

[0006] To solve the above problems, the present disclosure proposes a uterine leiomyosarcoma image classification system based on hybrid view adaptation. In the classification system, an uncertainty-driven hybrid view adaptation learning framework is designed. By adopting the hybrid view adaptation learning method, unsupervised inter-patch adaptation learning ensures feature consistency. Combining WSI diagnosis with a quantitative confidence coefficient, discriminative features are corrected based on uncertainty modeling under the Dirichlet distribution, thereby improving the accuracy of image classification.

[0007] According to some embodiments, the present disclosure adopts the following technical solutions: A uterine leiomyosarcoma image classification system based on hybrid view adaptation, comprising: A data sampling module, configured to obtain whole-slide images of uterine leiomyosarcoma, divide the whole-slide images into multiple patches, and sample the patches into patch features; A hybrid view adaptation learning module, configured to perform phenotype-driven patch self-optimization on the patch features. First, feature metric extraction and phenotype metric extraction are respectively performed on the patch features to obtain patch-level feature metrics and phenotype metrics, and the patch-level feature metrics and phenotype metrics are concatenated to generate phenotype-driven features; wherein, unsupervised inter-patch adaptation learning and compensatory intra-patch adaptation learning are introduced in the phenotype-driven patch self-optimization to construct an optimization reward, and the policy gradient method is used to iteratively self-optimize the phenotype-driven features; An uncertainty discrimination module, configured to perform forward parameterization on the self-optimized phenotype-driven features after processing by a fully connected layer, input the forward parameterized phenotype-driven features into the Dirichlet distribution formula, calculate an uncertainty score, and discriminate the probability of image classification, thereby obtaining an image classification result.

[0008] According to some embodiments, the present disclosure adopts the following technical solutions: A computer program product, comprising a computer program, which when executed by a processor implements the following method steps: Obtain whole-slide images of uterine leiomyosarcoma, divide the whole-slide images into multiple patches, and sample the patches into patch features; Perform phenotype-driven patch self-optimization on the patch features. First, extract feature metrics and phenotype metrics from the patch features respectively to obtain patch-level feature metrics and phenotype metrics, and concatenate the patch-level feature metrics and phenotype metrics to generate phenotype-driven features. Among them, unsupervised inter-patch adaptive learning and compensatory intra-patch adaptive learning are introduced in the phenotype-driven patch self-optimization to construct an optimization reward, and the policy gradient method is used to iteratively self-optimize the phenotype-driven features. After processing the self-optimized phenotype-driven features through a fully connected layer, perform forward parameterization. Input the forward parameterized phenotype-driven features into the Dirichlet distribution formula to calculate the uncertainty score and determine the probability of image classification, thereby obtaining the image classification result.

[0009] According to some embodiments, the present disclosure adopts the following technical solutions: A non-transitory computer-readable storage medium for storing computer instructions, which when executed by a processor, implement the following method steps: Obtain a whole slide image of uterine leiomyosarcoma, divide the whole slide image into multiple patches, and sample the patches into patch features; Perform phenotype-driven patch self-optimization on the patch features. First, extract feature metrics and phenotype metrics from the patch features respectively to obtain patch-level feature metrics and phenotype metrics, and concatenate the patch-level feature metrics and phenotype metrics to generate phenotype-driven features. Among them, unsupervised inter-patch adaptive learning and compensatory intra-patch adaptive learning are introduced in the phenotype-driven patch self-optimization to construct an optimization reward, and the policy gradient method is used to iteratively self-optimize the phenotype-driven features. After processing the self-optimized phenotype-driven features through a fully connected layer, perform forward parameterization. Input the forward parameterized phenotype-driven features into the Dirichlet distribution formula to calculate the uncertainty score and determine the probability of image classification, thereby obtaining the image classification result.

[0010] According to some embodiments, the present disclosure adopts the following technical solutions: An electronic device includes: a processor, a memory, and a computer program; wherein, the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device runs, the processor executes the computer program stored in the memory so that the electronic device executes the following method steps: Obtain a whole slide image of uterine leiomyosarcoma, divide the whole slide image into multiple patches, and sample the patches into patch features; Perform phenotype-driven patch self-optimization on patch features. First, extract feature metrics and phenotype metrics from patch features respectively to obtain patch-level feature metrics and phenotype metrics, and then concatenate the patch-level feature metrics and phenotype metrics to generate phenotype-driven features. Among them, unsupervised inter-patch adaptive learning and compensatory intra-patch adaptive learning are introduced in the phenotype-driven patch self-optimization to construct an optimization reward, and the policy gradient method is used to iteratively self-optimize the phenotype-driven features. After processing the self-optimized phenotype-driven features through a fully connected layer, perform forward parameterization, and input the forward parameterized phenotype-driven features into the Dirichlet distribution formula to calculate the uncertainty score and determine the probability of image classification, thereby obtaining the image classification result.

[0011] Compared with the prior art, the beneficial effects of the present disclosure are as follows: The uterine leiomyosarcoma image classification system based on hybrid view adaptation of the present disclosure proposes an uncertainty-driven hybrid view adaptation learning (UHAL) framework, and proposes a hybrid view adaptation learning method to fully explore and utilize the key information in tumors to improve the accuracy and efficiency of tumor recognition. Among them, the unsupervised inter-patch adaptive learning module and the compensatory intra-patch adaptive learning module emphasize the significant cell features between patches and side-level labels, and at the same time eliminate unnecessary redundant information. In addition, the phenotype-driven patch self-optimization module further supplements and optimizes tumor features to accelerate the ULMS image classification process and assist clinical diagnosis.

[0012] The uterine leiomyosarcoma image classification system based on hybrid view adaptation of the present disclosure proposes a hybrid view adaptation learning method. Among them, the unsupervised inter-patch adaptive learning module ensures feature consistency and emphasizes the significant cell features between patches. The compensatory intra-patch adaptive learning module captures valuable information in limited side-level labels while minimizing redundant information. The phenotype-driven patch self-optimization module further supplements and optimizes tumor features for accurate diagnosis of ULMS. An uncertainty discrimination mechanism is proposed to provide a quantitative confidence coefficient for each diagnosis, and the uncertainty score is used to correct the features, ultimately achieving a reliable WSI-level diagnosis result.

[0013] The uterine leiomyosarcoma image classification system based on hybrid view adaptation of the present disclosure applies the uncertainty-driven hybrid view adaptation learning (UHAL) framework for the first time to the automatic diagnosis of ULMS in WSIs and achieves superior classification performance: the accuracy rate reaches 94.17%, the AUC is 97.92%, the precision is 92.31%, and the F1 score is 94.40%, providing auxiliary support for clinical ULMS management. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The accompanying drawings forming a part of this disclosure are used to provide a further understanding of the disclosure. The illustrative embodiments and descriptions thereof of the disclosure are used to explain the disclosure and do not constitute an improper limitation of the disclosure.

[0015] Figure 1 It is the phenotype-driven patch self-optimization process of the embodiment of the present disclosure; Figure 2 It is the self-optimization workflow of the controller of the embodiment of the present disclosure; Figure 3 It is the unsupervised inter-patch adaptation learning process of the embodiment of the present disclosure; Figure 4 It is the structural diagram of the uncertainty discrimination mechanism of the embodiment of the present disclosure; Figure 5 It is the overall framework diagram of the uncertainty-driven hybrid view adaptation learning framework (UHAL) of the embodiment of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0016] The present disclosure will be further described below in conjunction with the accompanying drawings and embodiments.

[0017] It should be noted that the following detailed descriptions are all illustrative and are intended to provide a further description of the present disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present disclosure belongs.

[0018] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present disclosure. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0019] Embodiment 1 In an embodiment of the present disclosure, a uterine leiomyosarcoma image classification system based on hybrid view adaptation is provided, and an uncertainty-driven hybrid view adaptation learning (UHAL) network framework is proposed, including a data sampling module, a hybrid view adaptation learning module, and an uncertainty discrimination module. The functions and methods executed in each module are as follows: The data sampling module is used to obtain whole-slide images of uterine leiomyosarcoma, divide the whole-slide images into multiple patches, and sample the patches into patch features by using the residual network ResNet and the K-Means clustering algorithm; The hybrid view adaptation learning module is used to perform phenotype-driven patch self-optimization on the patch features in the UHAL network framework. First, the patch features are respectively subjected to feature metric extraction and phenotype metric extraction to obtain patch-level feature metrics and phenotype metrics, and the patch-level feature metrics and phenotype metrics are concatenated to generate phenotype-driven features; among them, unsupervised inter-patch adaptation learning and compensatory intra-patch adaptation learning are introduced in the phenotype-driven patch self-optimization to construct an optimization reward, and the policy gradient method is used to iteratively self-optimize the phenotype-driven features; The uncertainty discrimination module is used to process the self-optimized phenotype-driven features through a fully connected layer, then forward parameterize them through an improved Softplus activation function, input the forward parameterized phenotype-driven features into the Dirichlet distribution formula, calculate the uncertainty score, discriminate the probability of image classification, and thus obtain the image classification result.

[0020] As an embodiment, in the uterine leiomyosarcoma image classification system based on hybrid view adaptation, an uncertainty-driven hybrid view adaptation learning framework is proposed, which includes a hybrid view adaptation learning method and an uncertainty discrimination mechanism for the automatic classification of ULMS in WSIs pathological images to achieve auxiliary diagnosis. Among them, the hybrid view adaptation learning method includes: selecting significant features from three aspects and removing the redundancy of patches (i.e., phenotype-driven patch self-optimization, unsupervised inter-patch adaptive learning, and compensatory intra-patch adaptive learning), aiming to improve the recognition ability of the network framework and alleviate the overfitting problem encountered in the training process. Specifically, the preprocessed patch features are concatenated with the phenotype feature matrix to obtain phenotype-driven features, and the significant information between patches within the WSI is maximized through unsupervised adaptive learning. Subsequently, the selected phenotype-driven patch features reduce the influence of irrelevant information in the patch through compensatory intra-patch adaptive learning. The above two-step adaptive learning strategy is optimized in the phenotype-driven patch self-optimization process. In addition, the uncertainty discrimination mechanism based on the Dirichlet distribution models the uncertainty of WSIs, and uses the identified uncertainty to prioritize high-confidence patch features, while correcting the phenotype-driven patch features related to out-of-distribution (OOD) classification with low confidence values. The specific details of the specific implementation of the system disclosed in this application are as follows: 1. In the data sampling module, it is used to obtain the whole-slide image of uterine leiomyosarcoma, divide the whole-slide image into multiple patches, and sample the patches into patch features by using the residual network ResNet and the K-Means clustering algorithm; Specifically, sampling the patches into patch features by using the residual network ResNet and the K-Means clustering algorithm includes: First, the image is segmented into multiple non-overlapping patches of a fixed size, and each patch is fed into the ResNet network to obtain the feature vector of the patch. The feature vectors of all patches are stacked together to form a feature matrix. Subsequently, the K-Means algorithm is applied to cluster the feature matrix, and all patch feature vectors are assigned to K clusters according to the Euclidean distance between patch features.

[0021] Furthermore, perform positional embedding processing on the patch features, including: associating the patch features in each cluster with their spatial positions in the original image. Specifically, it is necessary to sort the patch features in each cluster so that the patch features within each cluster are arranged in the order of their spatial coordinates in the original image. Positional embedding is used to represent the spatial positions of the patch features within each cluster. For each cluster, positional embedding defines a position index for each patch feature within it, and this index represents the sequential position of the feature within the cluster, facilitating the quick retrieval of the corresponding position sequence in the image based on the patch blocks selected from the cluster.

[0022] 2. In the hybrid view adaptive learning module, for the UHAL network framework, perform phenotype-driven patch self-optimization on the patch features. First, extract feature metrics and phenotype metrics from the patch features respectively to obtain patch-level feature metrics and phenotype metrics, and then connect the patch-level feature metrics and phenotype metrics to generate phenotype-driven features; among them, introduce unsupervised inter-patch adaptive learning and compensatory intra-patch adaptive learning in the phenotype-driven patch self-optimization, construct an optimization reward, and use the policy gradient method to iteratively self-optimize the phenotype-driven features; specifically, it includes: Step 1: Phenotype-driven patch self-optimization In a clinical environment, phenotype features are key biological information for identifying tumors, including nucleus size and shape, cell density, mitotic activity, etc. However, existing methods usually uniformly select patch features from the clusters obtained by the K-Means algorithm, but ignore the overlapping patch information in each cluster. To solve this problem, the present disclosure proposes a patch self-optimization method guided by cell phenotypes, as Figure 1 shown, the patch features preprocessed by positional embedding ( ∈ [1, ], where represents the total number of patches) are input into the UHAL network framework. First, extract feature metrics and phenotype metrics to generate patch-level feature metrics and phenotype metrics . Then, the phenotype-driven feature formed by connecting and is then self-optimized by the controller to further eliminate the redundancy caused by repeatedly processing all input features.

[0023] Step 11: Feature metric extraction, including: The patch features are first projected into a two-layer low-dimensional space through two convolutional operations. Subsequently, they are processed by different non-linear activation functions and attention metrics are generated. The attention metrics and the patch features are multiplied matrix-wise to obtain the patch-level feature metrics.

[0024] Specifically, the patch features are first projected into a two-layer low-dimensional space through two convolutional operations (with parameters and ), denoted as and . Subsequently, they are processed by different non-linear activation functions sigmoid and the tanh function and and , and attention metrics are generated, which are obtained by multiplying , a convolutional operation with parameter , and the Softmax function. Finally, the patch-level feature metrics are obtained by multiplying matrix-wise the attention metrics and the patch features , as: (1) (2) Step 12: Phenotype metric extraction, including: Each patch feature is regarded as a data point. Based on the Euclidean distance between each data point and the cluster center, the patch features are divided into different clusters. The patch features in different clusters are averaged, the average phenotype features of each cluster are calculated, and the average phenotype features are normalized using weighted metrics. Finally, the average phenotype features and the weighted metrics are integrated to obtain the phenotype feature metrics.

[0025] Specifically, each patch feature can be regarded as a data point, containing attributes such as pixel values, textures, and colors. First, based on the Euclidean distance ( ∈ [1, ), all the patch features are divided into different clusters ( ∈ [1, ). Then, by averaging the different clusters Mean processing of all patch features in [[]], calculating the average phenotypic features of each cluster . Apply weighted metrics to the average phenotypic features for normalization to quantify the expected weights under the patch-level feature metric interaction. Finally, integrate the obtained phenotypic feature metrics , with a shape of × , where is consistent with the dimension of the patch-level feature metric , and represents the number of patch features included in each cluster , (3) (4) (5) Step 13: Controller self-optimization, including: concatenating the patch-level feature metric and the phenotypic feature metric to generate phenotype-driven features, and then the controller performs self-optimization. Under the guidance of rewards from unsupervised inter-patch adaptive learning and compensatory intra-patch adaptive learning, the controller generates new position-embedded patch features for the next step to rearrange the patch features, replaces the initial patch features with the rearranged patch features, the length of the rearranged patch features is equal to that of the initial patch features, and using the policy gradient method, the controller takes the given position-embedded patch features at each step as guidance to iteratively self-optimize the phenotype-driven features, and the self-optimization process continues until the set number of steps is completed.

[0026] Specifically, the initial position embeddings obtained from each cluster identify patch features , and then is used as the input for the extraction process. After obtaining the patch-level feature metric and the phenotypic feature metric, further, the patch-level feature metric and the phenotypic feature metric are concatenated to generate phenotype-driven features, generating phenotype-driven features . After that, unsupervised inter-patch adaptive learning and compensatory intra-patch adaptive learning are introduced to construct a reward mechanism, and the controller generates new position-embedded patch features for the next step to rearrange the patch features , and uses the rearranged patch features ( ∈ [1, to replace the initial patch features . The rearranged has the same length as the initial , which is achieved by multiplying the sampling ratio by the number of features in each cluster. Using the policy gradient method, the controller embeds the given new position scanned at each step into the patch features as a guide to iteratively self-optimize the phenotype-driven features . The self-optimization process continues until the set steps are completed, focusing on the most discriminative information representation in the WSI and ultimately improving the performance of downstream analysis tasks.

[0027] Furthermore, Step 1.3.1: The unsupervised inter-patch adaptive learning process includes: To alleviate the constraints brought by slide-level weak labels, the UHAL framework introduces an unsupervised inter-patch adaptive learning module to strengthen the relevant and effective information contained in the inter-patch features. This method includes inter-patch contrast learning and inter-patch adaptive optimization, aiming to maintain the consistency between different phenotype-driven features , while learning the aggregated knowledge between different patch feature sets , as follows: a. Inter-patch contrast learning Two patch feature sets , are obtained from uniform sampling. The patch features and are processed through the phenotype-driven patch self-optimization process to generate phenotype-driven features and respectively. The similarity between and is used to construct the reward to guide the unsupervised inter-patch adaptive learning process for dynamic retrieval and localization of patch features.

[0028] (6) where ∈ (0, is the current step, ([[]] , ) and ([[]] , ) respectively constitute step and ​Phenotype-driven features. cos() is the cosine similarity function. By maximizing the cosine distance, the controller directly selects features with larger difference values, thus prompting the model to focus on effective patch features.

[0029] In addition, to maximize the consistency between different phenotype-driven features, the normalized cross-entropy function is used to learn the aggregated knowledge, (7) where N is the number of patients in the network input, is the indicator function, which is 1 when and 0 otherwise. It is used to maximize the consistency between different phenotype-driven features in unsupervised adaptive learning.

[0030] b. Adaptive optimization between patches Specifically, by maximizing the mutual information , between two patch feature sets (, the aggregated knowledge between two patch feature sets and is learned, thereby further adaptively promoting the learning of relevant effective information, (8) (9) (10) where, is the Hadamard product; (·) (Formula (9)) is the -order entropy of Rényi, which is used to calculate the information entropy in the high-dimensional space without variational approximation and distribution estimation; is -order parameter. is the normalization operation, represents the th eigenvalue, is the number of eigenvalues. G(·) is the Gram matrix, which is used to capture and the correlation between different channels within; γ is the random deviation introduced during the training process.

[0031] Unsupervised adaptive learning between patches enables the framework to comprehensively integrate relevant inter-patch information by achieving supervised inter-patch contrast learning and supervised inter-patch adaptive optimization under the constraint of .

[0032] (11) Further, step 1.3.2: Adaptive learning within compensatory patches An important challenge that the UHAL framework faces when attempting to effectively perform ULMS recognition lies in the redundancy generated by the numerous phenotype-driven patch features extracted from gigapixel WSIs, which may affect the final classification performance. Encoding all selected patch features indiscriminately during the adaptive learning process, including those that are uninformative for classification, not only impairs the achieved classification accuracy but also reduces efficiency. The present disclosure proposes adaptive learning within compensatory patches. The specific process is to obtain discriminative information features, obtain the mutual information between the classification label and the phenotype-driven features as an embodiment of the redundancy within the patch, and regard the prediction probability gradient as a reward to guide the self-optimization of the phenotype-driven features in the adaptive learning within compensatory patches.

[0033] Specifically, by to capture features that embody discriminative information related to ULMS while reducing redundant information that, although related to the disease diagnosis process, lacks discriminative ability. is the label and the phenotype-driven features The mutual information between them utilizes the effective information embedded in the slide-level label to improve the final classification performance. In the actual backpropagation, the cross-entropy function is used instead of , is, (12) where ∈ (0, 1) is the label, represents the prediction probability. Here, the prediction probability gradient and between = - , ([[]] ∈ (0, [[[]] ) is regarded as a reward to guide the self-optimization of the phenotype-driven features in the adaptive learning within compensatory patches. The cross-entropy function is used instead of , (13) is the mutual information between the patch feature and the phenotype-driven patch feature as an embodiment of the redundancy within the patch. The calculation formula of is the same as formulas (8)-(10).

[0034] Step 3: In the uncertainty discrimination module, after the self-optimized phenotype-driven features are processed by the fully connected layer, they are forward parameterized through an improved Softplus activation function. The forward parameterized phenotype-driven features are input into the Dirichlet distribution formula to calculate the uncertainty score, discriminate the probability of image classification, and thus obtain the image classification result.

[0035] Specifically, since the patch image blocks may not comprehensively represent the overall pathological features, the pathological heterogeneity within WSIs may lead to unreliable classification results. Some existing methods can use uncertainty assessment to demonstrate the reliability of their models and provide diagnostic guidance for clinicians, but they are limited to end-point evaluation and lack the ability to further improve the network based on uncertainty estimation. To solve this problem, the present disclosure designs an uncertainty discrimination mechanism based on the Dirichlet distribution for modeling the uncertainty of WSIs, and uses the identified uncertainty to prioritize high-confidence patch features while correcting the phenotype-driven features related to out-of-distribution (OOD) classifications with low confidence values.

[0036] First, the phenotype-driven features processed by the fully connected layer are forward parameterized through an improved Softplus activation function to , ensuring applicability to the Dirichlet distribution. Since the Dirichlet distribution has the ability to describe the probability distribution of multi-dimensional random variables, the Dirichlet distribution is used to encapsulate the discriminative features related to the presence of ULMS (for example, features containing moderate to severe nuclear atypia and high mitotic counts). Subsequently, the corresponding uncertainty score is calculated for ULMS diagnosis.

[0037] The forward parameterization process is (14) The Dirichlet distribution is (15) Calculate the corresponding uncertainty score , (16) where represents the probability density function of the Dirichlet distribution, represents the predicted probability, is the total number of categories, Γ is the gamma function, represents the Dirichlet strength.

[0038] Finally, the discriminator evaluates the credibility of the diagnostic result by comparing with to help update the features . Among them equals the average distribution value of the uncertainty scores associated with incorrect predictions during training, and B is the number of attention metrics selected from { | 1 ≤ ≤ }.

[0039] (17) (18) It is generally believed that predictions with lower uncertainty scores are more likely to be reliable, indicating a high confidence level in the diagnostic results. Therefore, when > , the discriminator assumes that the classification falls into the OOD situation, selects the top B attention metric values { | 1 ≤ ≤ B} through the aggregation process of multi-instance learning representation, and randomly sets some of them to zero using the network dropout probability dropout . The remaining non-zero attention metrics are updated through Equation (18) and fed back into the phenotype-driven patch features , shifting the focus of the UHAL framework from the highest attention metrics and further reallocating the focus to other discriminative features.

[0040] In special cases, if remains greater than after the repeated correction operations of Equation (18), the discriminator considers the final classification of the specific WSI to be unreliable. In this case, the framework requires a doctor's intervention and provision of a diagnostic result during the test phase, while remaining unchanged and not requiring intervention during the training phase.

[0041] Otherwise, when ≤ , the discriminator evaluates the current classification result as reliable, and then performs further supervision steps under the guidance of slide-level labels during training, or outputs classification probabilities during testing.

[0042] Simulation experiments The present disclosure evaluated the classification performance of the proposed UHAL framework on a private ULMS dataset (containing 120 WSIs images) identified and diagnosed by three medical experts. After preprocessing, the ULMS dataset contains 547,896 patch images related to malignant leiomyosarcoma and 893,168 patch images related to benign leiomyosarcoma. In addition, the present disclosure used 156 WSIs from the publicly available TCGA-Esca dataset (obtained from https: / / portal.gdc.cancer.gov / ) to verify the robustness of the UHAL framework.

[0043] In the experiment, the present disclosure adopted WSIs obtained at a 20x magnification level and cropped them into patch images of size 256×256. Additionally, the present disclosure conducted a five-fold cross-validation experiment to ensure the robustness of performance evaluation. In each fold, the WSIs were randomly divided into 80% for training and 20% for testing.

[0044] Based on the open-source PyTorch framework and the hardware environment of a 16 GB NVIDIA Tesla P100 GPU, the present disclosure implemented model training and test evaluation. Each WSI patch image was converted into a 512-dimensional feature vector through a ResNet18 encoder pre-trained on the ImageNet dataset. The patch feature set was selected by averaging the samples of each cluster after K-means clustering grouping. The Batchsize was set to 128, K was set to 10, and B was set to 8. dropout It was set to 0.25. The UHAL framework was trained using the Adam optimizer with an initial learning rate of 1e-4 and a weight decay rate of 1e-5.

[0045] Evaluation process The evaluation metrics in the experiment were: accuracy (ACC), area under the curve fraction (AUC), precision, and F1 score. In all experiments, 0.5 was used as the threshold for calculating the accuracy.

[0046] After five-fold cross-validation, the average values of the ACC, AUC, precision, and F1 score metrics on the ULMS dataset were 94.17%, 97.92%, 92.31%, and 94.40% respectively, showing relatively accurate automatic diagnosis ability. Table 1 shows the comparison of the classification performance obtained by the UHAL framework and the existing state-of-the-art methods on the ULMS dataset using five-fold cross-validation. The best scores are shown in bold.

[0047] Table 1. Comparison of classification performance obtained by the UHAL framework and existing methods on the ULMS dataset

[0048] To verify the robustness of the UHAL framework, the present disclosure conducted extensive experiments on the TCGA-Esca dataset and achieved excellent automatic classification results on four evaluation metrics. The ACC, AUC, precision, and F1 score reached 96.77%, 96.84%, 96.92%, and 96.12% respectively.

[0049] Table 2. Comparison of classification performance obtained by the UHAL framework and existing methods on the TCGA-Esca dataset

[0050] Example 2 In one embodiment of the present disclosure, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the following method steps are implemented: Obtain a whole-slide image of uterine leiomyosarcoma, divide the whole-slide image into multiple patches, and sample the patches into patch features using the residual network ResNet and the K-Means clustering algorithm; Input the patch features into the UHAL network framework, and perform phenotype-driven patch self-optimization on the patch features. First, perform feature metric extraction and phenotype metric extraction on the patch features respectively to obtain patch-level feature metrics and phenotype metrics, and connect the patch-level feature metrics and phenotype metrics to generate phenotype-driven features; wherein, unsupervised inter-patch adaptive learning and compensatory intra-patch adaptive learning are introduced in the phenotype-driven patch self-optimization to construct an optimization reward, and the policy gradient method is used to iteratively self-optimize the phenotype-driven features; After the self-optimized phenotype-driven features are processed by a fully connected layer, they are forward parameterized through an improved Softplus activation function, and the forward parameterized phenotype-driven features are input into the Dirichlet distribution formula to calculate the uncertainty score and discriminate the probability of image classification, thereby obtaining the image classification result.

[0051] Example 3 In one embodiment of the present disclosure, a non-transitory computer-readable storage medium is provided, and the non-transitory computer-readable storage medium is used to store computer instructions, and when the computer instructions are executed by a processor, the following method steps are implemented: Obtain a whole-slide image of uterine leiomyosarcoma, divide the whole-slide image into multiple patches, and sample the patches into patch features using the residual network ResNet and the K-Means clustering algorithm; Input the patch features into the UHAL network framework, and perform phenotype-driven patch self-optimization on the patch features. First, extract feature metrics and phenotype metrics from the patch features respectively to obtain patch-level feature metrics and phenotype metrics, and connect the patch-level feature metrics and phenotype metrics to generate phenotype-driven features. Among them, unsupervised inter-patch adaptive learning and compensatory intra-patch adaptive learning are introduced in the phenotype-driven patch self-optimization to construct an optimization reward, and the policy gradient method is used to iteratively self-optimize the phenotype-driven features. After processing the self-optimized phenotype-driven features through a fully connected layer, parameterize them forward through an improved Softplus activation function, input the forward-parameterized phenotype-driven features into the Dirichlet distribution formula, calculate the uncertainty score, and discriminate the probability of image classification, so as to obtain the image classification result.

[0052] Example 4 In an embodiment of the present disclosure, an electronic device is provided, including: a processor, a memory, and a computer program. Among them, the processor is connected to the memory, and the computer program is stored in the memory. When the electronic device runs, the processor executes the computer program stored in the memory, so that the electronic device executes the following method steps: Obtain a whole-slide image of uterine leiomyosarcoma, divide the whole-slide image into multiple patches, and sample the patches into patch features using the residual network ResNet and the K-Means clustering algorithm. Input the patch features into the UHAL network framework, and perform phenotype-driven patch self-optimization on the patch features. First, extract feature metrics and phenotype metrics from the patch features respectively to obtain patch-level feature metrics and phenotype metrics, and connect the patch-level feature metrics and phenotype metrics to generate phenotype-driven features. Among them, unsupervised inter-patch adaptive learning and compensatory intra-patch adaptive learning are introduced in the phenotype-driven patch self-optimization to construct an optimization reward, and the policy gradient method is used to iteratively self-optimize the phenotype-driven features. After processing the self-optimized phenotype-driven features through a fully connected layer, parameterize them forward through an improved Softplus activation function, input the forward-parameterized phenotype-driven features into the Dirichlet distribution formula, calculate the uncertainty score, and discriminate the probability of image classification, so as to obtain the image classification result.

[0053] The present disclosure is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, as well as the combination of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or means for implementing the functions specified in multiple blocks.

[0054] These computer program instructions can also be loaded onto a computer or other programmable data processing devices, such that a series of operation steps are executed on the computer or other programmable devices to generate a computer-implemented process, so that the instructions executed on the computer or other programmable devices provide steps for implementing the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or steps for implementing the functions specified in multiple blocks.

[0055] Although the specific implementation manners of the present disclosure have been described above in conjunction with the accompanying drawings, they are not limitations on the protection scope of the present disclosure. Those skilled in the art should understand that, based on the technical solutions of the present disclosure, various modifications or deformations that can be made by those skilled in the art without creative efforts are still within the protection scope of the present disclosure.

Claims

1. A hybrid view adaptive uterine leiomyosarcoma image classification system, characterized by: include: A data sampling module is used to obtain a full-slice image of uterine leiomyosarcoma, divide the full-slice image into multiple patches, and sample the patches as patch features; The hybrid view adaptive learning module is used to perform phenotype-driven patch self-optimization on patch features. First, the patch features are subjected to feature metric extraction and phenotype metric extraction respectively to obtain patch-level feature metric and phenotype metric, and the patch-level feature metric and phenotype metric are connected to generate phenotype-driven features. In the phenotype-driven patch self-optimization, unsupervised inter-patch adaptive learning and compensatory intra-patch adaptive learning are introduced to construct optimization rewards, and the policy gradient method is used to iteratively self-optimize the phenotype-driven features. The uncertainty discrimination module is used to process the self-optimized phenotypic driving features through the fully connected layer and then perform forward parameterization, input the forward parameterized phenotypic driving features into the Dirichlet distribution formula, calculate the uncertainty score, and discriminate the probability of image classification, thereby obtaining the image classification result.

2. The hybrid view adaptive uterine leiomyosarcoma image classification system according to claim 1, characterized in that: The patch features are positionally embedded and then input into the UHAL network framework. The feature metric extraction process includes: the patch features are first projected into a two-layer low-dimensional space through two convolution operations, and then processed by different nonlinear activation functions to generate attention metrics. The attention metrics and patch features are matrix multiplied to obtain the patch-level feature metrics.

3. The hybrid view adaptive uterine leiomyosarcoma image classification system according to claim 1, characterized in that: The phenotypic metric extraction process includes: treating each patch feature as a data point, dividing the patch features into different clusters based on the Euclidean distance between each data point and the cluster center, averaging the patch features in different clusters, calculating the average phenotypic feature of each cluster, applying a weighted metric to normalize the average phenotypic feature, and finally, integrating the average phenotypic feature and applying the weighted metric to obtain the phenotypic feature metric.

4. The hybrid view adaptive uterine leiomyosarcoma image classification system according to claim 1, characterized in that: The patch-level feature metric and the phenotypic feature metric are connected to generate the phenotypic driving feature, which is then self-optimized by the controller. Under the reward guidance of unsupervised inter-patch adaptive learning and compensatory intra-patch adaptive learning, the controller generates a new position embedding patch feature for the next step to rearrange the patch features and replace the initial patch features with the rearranged patch features. The length of the rearranged patch features is equal to the initial patch features. Using the policy gradient method, the controller takes the given position embedding patch features at each step as a guide to iteratively self-optimize the phenotypic driving features. The self-optimization process continues until the set number of steps is completed.

5. The hybrid view adaptive uterine leiomyosarcoma image classification system according to claim 1, characterized in that: Unsupervised patch-to-patch adaptive learning includes patch-to-patch contrast learning and patch-to-patch adaptive optimization to maintain the consistency between different phenotype-driven features. The two patch feature sets obtained from uniform sampling are processed by phenotype-driven patch self-optimization to generate phenotype-driven features. and phenotypic driver traits , the phenotype-driven features and phenotypic driver traits The similarity between them is used to construct rewards to guide the unsupervised patch adaptation learning process to dynamically retrieve and locate patch features.

6. The hybrid view adaptive uterine leiomyosarcoma image classification system according to claim 1, characterized in that: The compensatory intra-patch adaptive learning process is to obtain discriminative information features and the mutual information between classification labels and phenotypic driving features as a manifestation of intra-patch redundancy. The predicted probability gradient is regarded as a reward to guide the self-optimization of phenotypic driving features in compensatory intra-patch adaptive learning.

7. The hybrid view adaptive uterine leiomyosarcoma image classification system according to claim 1, characterized in that: The Dirichlet distribution formula is the uncertainty discrimination mechanism of the Dirichlet distribution, which is used to model the uncertainty of WSIs. The identified uncertainty is used to consider high-confidence patch features, while correcting phenotypic driving features related to out-of-distribution classifications with low confidence values. The discriminator evaluates the credibility of the classification results by calculating scores to update the features.

8. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the following method steps are implemented: Obtain a full-slice image of uterine leiomyosarcoma, divide the full-slice image into multiple patches, and sample the patches as patch features; The patch features are subjected to phenotype-driven patch self-optimization. First, the patch features are subjected to feature metric extraction and phenotype metric extraction respectively to obtain patch-level feature metric and phenotype metric, and the patch-level feature metric and phenotype metric are connected to generate phenotype-driven features. Among them, unsupervised inter-patch adaptive learning and compensatory intra-patch adaptive learning are introduced into the phenotype-driven patch self-optimization, optimization rewards are constructed, and the policy gradient method is used to iteratively self-optimize the phenotype-driven features. The self-optimized phenotypic driving features are processed by the fully connected layer and then forward parameterized. The forward parameterized phenotypic driving features are input into the Dirichlet distribution formula to calculate the uncertainty score and determine the probability of image classification, thereby obtaining the image classification result.

9. A non-transitory computer-readable storage medium, characterized in that: The non-transitory computer-readable storage medium is used to store computer instructions, and when the computer instructions are executed by the processor, the following method steps are implemented: Obtain a full-slice image of uterine leiomyosarcoma, divide the full-slice image into multiple patches, and sample the patches as patch features; The patch features are subjected to phenotype-driven patch self-optimization. First, the patch features are subjected to feature metric extraction and phenotype metric extraction respectively to obtain patch-level feature metric and phenotype metric, and the patch-level feature metric and phenotype metric are connected to generate phenotype-driven features. Among them, unsupervised inter-patch adaptive learning and compensatory intra-patch adaptive learning are introduced into the phenotype-driven patch self-optimization, optimization rewards are constructed, and the policy gradient method is used to iteratively self-optimize the phenotype-driven features. The self-optimized phenotypic driving features are processed by the fully connected layer and then forward parameterized. The forward parameterized phenotypic driving features are input into the Dirichlet distribution formula to calculate the uncertainty score and determine the probability of image classification, thereby obtaining the image classification result.

10. An electronic device, characterized in that: include: A processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory to enable the electronic device to perform the following method steps: Obtain a full-slice image of uterine leiomyosarcoma, divide the full-slice image into multiple patches, and sample the patches as patch features; The patch features are subjected to phenotype-driven patch self-optimization. First, the patch features are subjected to feature metric extraction and phenotype metric extraction respectively to obtain patch-level feature metric and phenotype metric, and the patch-level feature metric and phenotype metric are connected to generate phenotype-driven features. Among them, unsupervised inter-patch adaptive learning and compensatory intra-patch adaptive learning are introduced into the phenotype-driven patch self-optimization, optimization rewards are constructed, and the policy gradient method is used to iteratively self-optimize the phenotype-driven features. The self-optimized phenotypic driving features are processed by the fully connected layer and then forward parameterized. The forward parameterized phenotypic driving features are input into the Dirichlet distribution formula to calculate the uncertainty score and determine the probability of image classification, thereby obtaining the image classification result.

Images