A method, system, and device for multi-classification of whole-slide images in cervical tissue pathology

By employing dynamic slice selection and self-supervised learning, the problems of identifying key slices and utilizing useless slices in whole-slide images of cervical tissue pathology were solved, achieving more efficient and accurate cervical tissue pathology diagnosis.

CN118196516BActive Publication Date: 2026-11-17WUHAN LANDING INTELLIGENCE MEDICAL CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202410351760.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-03-26
Publication Date
2026-11-17
Estimated Expiration
2044-03-26

AI Technical Summary

Technical Problem

Existing technologies in the diagnosis of cervical tissue pathology using whole-slide images are insufficient to accurately identify key sections and effectively utilize the diagnostic value of seemingly useless sections, resulting in limitations in diagnostic accuracy and efficiency.

Method used

A dynamic slice selection-based approach is adopted, which transforms slices into feature vectors through a self-supervised pre-trained feature extractor. The key slice selection, useless slice encapsulation, and slice interaction modules in the aggregation model are used to refine and extract key features, which are then fed into a multi-classification network for classification.

Benefits of technology

It improves the accuracy and efficiency of cervical tissue pathology image diagnosis, enables fine-grained diagnosis of different pathological types, reduces interference from useless sections, and provides more accurate diagnostic information.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118196516B_ABST
    Figure CN118196516B_ABST
Patent Text Reader

Abstract

This invention provides a method, system, and apparatus for multi-classification of cervical histopathology whole slide images, comprising the following steps: segmenting the cervical histopathology whole slide image into non-overlapping slices; extracting features from all slices using a self-supervised pre-trained feature extractor and converting them into a series of feature vectors; feeding the feature vectors into an aggregation model for slice feature aggregation, further refining and extracting key features to obtain the classification result of the cervical histopathology whole slide image; and feeding the features of the whole slide image into a multi-classification network to obtain the classification result of the whole slide image. This invention proposes a multi-classification method for cervical histopathology whole slide images based on dynamic slice selection. By constructing a novel aggregation model, it adaptively selects key slices of the whole slide image, thereby achieving better feature aggregation, reducing the negative impact of a large number of useless slices on classification, and making full use of the information in the whole slide image to improve the accuracy and efficiency of classification.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the interdisciplinary field of deep learning and medicine, and in particular to a method, system and device for multi-classification of whole slide images of cervical tissue pathology. Background Technology

[0002] Cervical cancer is the fourth leading cause of cancer death among women worldwide, causing more than 300,000 deaths annually. Early detection and diagnosis of cervical-related diseases can provide better treatment opportunities, help reduce patient suffering, and prevent further disease progression. Cervical histopathology is the "gold standard" for diagnosing cervical cancer. Cervical histopathology includes normal or inflammatory reactions, cervical intraepithelial neoplasia (classified into three grades: mild, moderate, and severe: CIN I, CIN II, and CIN III), carcinoma in situ, and invasive carcinoma. Clinically, doctors observe the morphology and structure of cells in tissue sections under a microscope to determine the presence of abnormalities and to make a pathological diagnosis. However, this manual analysis method has some problems. First, the interpretation and diagnosis of cervical pathology requires experienced pathologists, which may be difficult to find in some regions. Second, due to human factors and subjective judgment, the interpretation of sections may have a certain degree of variability.

[0003] In recent years, with the rapid development of computer vision and deep learning, it has become possible to classify and diagnose cervical tissue pathology slide images using computer algorithms. By using deep learning algorithms, large numbers of cervical tissue images can be analyzed and classified automatically. These algorithms can learn to extract features from input images and perform classification or diagnosis based on these features.

[0004] Over the past decade, an increasing number of scholars have studied the application of image analysis-based CAD systems in cervical cancer diagnosis. However, these studies also face several challenges and problems. Taking whole-slide images as an example, firstly, due to their high resolution and large size, most researchers focus on small images segmented from the whole slide, neglecting the overall diagnosis of the whole slide image. Secondly, the diagnosis of cervical histopathology whole-slide images relies heavily on the labels of numerous slide images. For multi-classification tasks involving cervical histopathology whole-slide images, a large number of multi-classification labels for slide images are required. Obtaining these fine-grained labels requires significant human and material resources. Furthermore, cervical histopathology whole-slide images typically contain thousands of slides. For positive samples, most slides are negative, with only a small percentage being positive. These positive slides play a crucial role in the diagnosis of positive samples, while negative slides are often useless. In addition, a large number of negative slides can interfere with the model's recognition of positive slides. Existing patent CN117274699A primarily focuses on a multi-classification method, system, and device for cervical cytology slide images. Its significant feature lies in utilizing self-supervised and weakly supervised techniques to achieve multi-classification diagnosis at the slide level, thereby significantly reducing reliance on physician annotation. However, this patent differs from the current field of cervical histopathology image processing. In the field of cervical histopathology image processing, we still face two core challenges: first, how to accurately identify key slides from a massive amount of histopathology image slides; and second, how to focus on useful slides while also fully exploring and utilizing the potential diagnostic value hidden in those initially seemingly unimportant slides. Solving these two problems will help improve the overall accuracy and reliability of diagnosis, thereby promoting the further development of cervical histopathology image analysis technology. Summary of the Invention

[0005] The main objective of this invention is to provide a method, system, and apparatus for multi-classification of cervical tissue pathology whole slide images, which solves the problem of identifying key slides from a large number of cervical tissue pathology whole slide image sections and reduces the negative impact of a large number of useless slides on the classification of whole slide images.

[0006] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is: a multi-classification method for cervical tissue pathology whole slide images based on dynamic slice selection, comprising the following steps:

[0007] S1. Divide the whole slide image of cervical tissue pathology into non-overlapping slices to obtain the patch dataset;

[0008] S2. Use a self-supervised pre-trained feature extractor to extract features from all slices in the patch dataset and transform the slices into a series of feature vectors X.

[0009] S3. The feature vector X is fed into the aggregation model for slice feature aggregation. The aggregation model includes the key slice selection module Patch Selector, the useless slice encapsulation module Patch Packaging, and the key slice interaction module PatchFormer. By alternately looping these three modules, the key features are further refined and extracted to obtain the features of the whole slide image of cervical tissue pathology.

[0010] S4. Input the features of the whole slide image into a multi-classification network to obtain the classification result of the whole slide image;

[0011] The above steps enable multi-classification of cervical tissue pathology slide images.

[0012] In the preferred embodiment, step S2 involves using a feature extractor to transform the slices into a series of feature vectors X. Specifically, this includes using the feature extractor to transform the N slices of the whole slide image patch dataset obtained in step S1 into a series of feature vectors X, where X is represented as X = {f1, f2, ..., f...} N}, where f N ∈R C It is the vector of the Nth slice, X∈R N×C It is obtained by stacking all the slice vectors in the whole glass slide image, and then adding a learnable class label to the slice feature vector sequence.

[0013] The feature extractor is ResNet50, and the self-supervised framework used to train the feature extractor is DINO.

[0014] In the preferred scheme, step S3, the key slice selection module Patch Selector, is used to determine whether a specified slice is a useful slice or a useless slice, specifically including:

[0015] The Patch Selector module will calculate a This is used to indicate whether each slice is useful, with 1 indicating useful and 0 indicating useless, and N representing the total number of slices; in the initial stage, Initialized to all 1s to indicate that all slices are useful; in subsequent stages, the current slice features X∈R are used... N×C Update The update process is as follows: First, X is mapped using a multilayer perceptron (MLP) to obtain the local features z of the slice. local :

[0016]

[0017] Then according to Calculate the global feature z with X glocal :

[0018]

[0019] The Agg function represents the aggregation of useful slice features, and it is implemented through global average pooling.

[0020]

[0021] z local and z glocal The local-global features z of the full-wave plate image are obtained by stitching together:

[0022] z i =[z i local ,z i glocal ], 1≤i≤N (4)

[0023] Feed z into another MLP network to obtain the probability π of each slice being useful / useless:

[0024]

[0025] π i,0 π represents the probability that the slice is useless. i,1 This represents the probability that the slice is useful. We obtain D for the current stage using π, and then update it with D.

[0026] D = Gumbel - Soft max(π) *,1 ∈{0,1} N (6)

[0027]

[0028] Gumbel-Softmax performs reparameter sampling on the discrete distribution described above, and ⊙ represents element-level multiplication.

[0029] In the preferred embodiment, in step S3, the useless slice packaging module Patch Packaging is used for packaging. Useless slices with a value of 0 include:

[0030] Calculated using the Patch Selector module Slices with a value of 0 are considered useless slices. These slices are then packaged using the Patch Packaging module for useless slices. Assume there are Q useless slices: And their corresponding useful / useless probabilities: The encapsulation process involves calculating the representative feature P of all useless slices. The formula for calculating P is as follows:

[0031]

[0032] Where, n i Let m represent the feature vector of the i-th useless slice. i [0] represents the probability that the slice is useless.

[0033] In the preferred embodiment, in step S3, the key slice interaction module is used to extract interaction information between slices, learn the dependencies between slices, and use the Transformer architecture to process key slices, allowing interaction between slices to capture the dependencies between slices and generate a full-slide image feature z containing global information, specifically including:

[0034] Useful slices are selected through the Patch Selector module, and the features of the obtained useful slices are... The encapsulation feature P of the useless slices obtained by the PatchPackaging module will be... P and X are concatenated along the feature dimension, and the concatenated feature matrix X is used as the input to the PatchFormer module. The specific concatenation formula is as follows:

[0035]

[0036] Each PatchFormer contains two sub-modules: a Multi-Head Self-Attention (MHSA) sub-module and a Feedforward Neural Network (FFN). Layer normalization and residual connections are applied before and after each sub-layer, respectively. In the output of the last PatchFormer, the class-labeled state is used as the feature z of the whole slide image. The computation process of each PatchFormer can be represented as follows:

[0037] X l =X l-1 +MHSA(LN(X l-1 ))(10)

[0038] X l =X l '+FFN(LN(X l '))(11) where X l-1 X represents the output of the previous layer. l The final output of the current PatchFormer layer, if not the last iteration of the aggregation model, is X. l It can be used as the input feature representation for the next loop. If it is the last loop, X l This is the feature representation of the final output.

[0039] In the preferred embodiment, in step S4, the multi-classification network is a five-classification network, which classifies a cervical tissue pathology slide image into one of the following categories: Normal, CIN1, CIN2, CIN3, and Cancer.

[0040] In the preferred embodiment, the aggregation model described in step S3 and the multi-classification network described in step S4 are trained synchronously, and the loss function used for training the aggregation model and the multi-classification network is the cross-entropy loss between the predicted label and the true label.

[0041] In addition, the present invention also provides a multi-classification system for cervical tissue pathology whole slide images based on dynamic slice selection. The system includes an image acquisition unit, an image labeling unit, a feature extraction training unit, an aggregation training unit, a multi-classification model training unit, and a testing unit.

[0042] The image acquisition unit is used to acquire whole slide images of cervical tissue pathology and cervical section images;

[0043] Image labeling unit, for labeling the classification tags of the cervical tissue pathology slide image;

[0044] Feature extraction training unit, used for self-supervised training of cervical slice feature extractor;

[0045] An aggregated training unit is used to supervise the training of an aggregated model of cervical tissue pathology slide images.

[0046] A multi-classification model training unit is used for supervised training of a multi-classification model of cervical tissue pathology whole slide images;

[0047] The test unit is used to test the classification model of the multi-classification model of the cervical tissue pathology whole slide image.

[0048] Finally, the present invention also provides a multi-classification device for cervical tissue pathology slide images, comprising one or more processors and a machine-readable storage medium storing machine-executable instructions that can be executed by the processor, which reads and executes these instructions to implement the method described in any one of claims 1 to 7.

[0049] This invention provides a multi-classification method, system, and apparatus for cervical histopathology whole-slide images based on dynamic slice selection. First, the image is segmented into slices, and a pre-trained feature extractor converts the slices into feature vectors. Next, key features are further refined and extracted using key slice selection, useless slice encapsulation, and slice interaction modules in the aggregation model. Finally, the aggregated features are fed into a multi-classification network for classification. This invention fully utilizes the information in the whole-slide image, improving the accuracy and efficiency of classification through refined feature extraction and aggregation. It is of great significance for the research and clinical application of cervical histopathology, contributing to more accurate disease diagnosis and assessment.

[0050] The beneficial effects of this invention are as follows:

[0051] (1) Solving the labeling difficulty problem: By using self-supervised learning, the slide features are extracted in an unsupervised manner, which alleviates the sample labeling difficulty in the traditional whole slide image classification method of cervical tissue pathology, and can efficiently extract the key features of the slide.

[0052] (2) Improve feature extraction efficiency: The self-supervised learning mechanism enables the model to efficiently extract key features from the whole slide image, which not only shortens the processing time, but also improves the accuracy and reliability of feature extraction, laying the foundation for subsequent accurate diagnosis.

[0053] (3) Consider the interaction information between slices: Through the key slice interaction module, the interaction information between slices can be extracted, and the dependencies between slices can be learned. This helps to capture the correlation and contextual information between slices, further improving the accuracy of diagnosis.

[0054] (4) Adaptive Slice Selection: A novel aggregation model is proposed that can adaptively select key slices for each full slide image and fully utilize the information of all slices, including those that initially seem unimportant, thereby improving the accuracy and reliability of the overall diagnosis. This dynamic screening process can better capture key information in slide images and reduce the interference of useless information on diagnostic results.

[0055] (5) Achieving fine-grained diagnosis: The application of multi-classification networks enables this method to perform more fine-grained diagnosis of cervical tissue pathology, distinguish the subtle differences between different pathological types, provide doctors with more accurate diagnostic information, and help to develop more personalized treatment plans. Attached Figure Description

[0056] The present invention will be further described below with reference to the accompanying drawings and embodiments:

[0057] Figure 1 This is a flowchart of the method of the present invention;

[0058] Figure 2 These are whole-slide images of cervical tissue pathology used in embodiments of the present invention;

[0059] Figure 3 This is an overall structural diagram of an embodiment of the present invention;

[0060] Figure 4 This is a diagram of the aggregation network structure in an embodiment of the present invention;

[0061] Figure 5 This is a structural diagram of the PatchFormer module of the aggregation network in an embodiment of the present invention. Detailed Implementation

[0062] The inventors discovered that in the field of cervical histopathology whole-slide image diagnosis, current methods largely rely on labels for a large number of slide images, especially for multi-classification tasks involving cervical histopathology whole-slide images, which require numerous multi-classification labels for slide images. However, obtaining these fine-grained labels involves a huge investment of human and material resources. Furthermore, whole-slide images contain numerous slides, of which a relatively small percentage are useful, while a large number of useless slides may interfere with the model's identification of key information. Therefore, how to accurately identify key slides from a massive number of slides has become a core research problem. This invention addresses this challenge by proposing an innovative multi-classification method for cervical histopathology whole-slide images. Through feature extraction and aggregation, it achieves automatic identification and utilization of key slides, providing a more accurate and efficient solution for cervical histopathology diagnosis.

[0063] Example 1

[0064] like Figures 1-5 As shown, a multi-classification method for whole-slide images of cervical tissue pathology includes the following steps:

[0065] S1. The cervical histopathology slide image is segmented into non-overlapping slices to obtain a patch dataset; each patch in the patch dataset is 256×256 pixels. This facilitates subsequent feature extraction and processing, as processing small images is generally more efficient than processing large images.

[0066] S2. Use the self-supervised pre-trained feature extractor to extract features from all slices in the patch dataset and convert the slices into a series of feature vectors X. The dataset used for self-supervised pre-training is obtained by summarizing the slice images retained after step S1 from all whole-slide images in the training set. The total number of slices used for self-supervised pre-training is 500,000.

[0067] S3. The feature vector X is fed into the aggregation model for slice feature aggregation. The aggregation model includes the key slice selection module Patch Selector, the useless slice encapsulation module Patch Packaging, and the key slice interaction module PatchFormer. By alternately looping these three modules, the key features are further refined and extracted to obtain the features of the whole slide image of cervical tissue pathology; the feature dimension C is 2048.

[0068] S4. Input the features of the whole slide image into a multi-classification network to obtain the classification result of the whole slide image;

[0069] The above steps enable multi-classification of cervical tissue pathology slide images.

[0070] With the incidence of cervical cancer showing a significant upward trend, and addressing the lack of computational aids for the diagnostic prediction of cervical histopathology slide images, this invention proposes a multi-classification method for cervical histopathology slide images. By utilizing self-supervised learning, it extracts slide features in an unsupervised manner, alleviating the difficulty of sample labeling in traditional cervical histopathology slide image classification methods. Furthermore, it efficiently extracts key slide features and proposes a novel aggregation model that adaptively selects key slides for each slide image, achieving better feature aggregation. This adaptive dynamic slide selection method not only improves computational efficiency but also effectively captures the most representative information from the slide image, providing more targeted features for subsequent classification tasks. Finally, this invention employs a multi-classification network to classify slide images, enabling more fine-grained diagnosis of cervical histopathology, which has significant implications for clinical practice.

[0071] In the preferred embodiment, step S2 involves using a feature extractor to transform the slices into a series of feature vectors X. Specifically, this includes using the feature extractor to transform the N slices of the whole slide image patch dataset obtained in step S1 into a series of feature vectors X, where X is represented as X = {f1, f2, ..., f...} N}, where f N ∈R C It is the vector of the Nth slice, X∈R N×C It is obtained by stacking all the slice vectors in the whole glass slide image, and then adding a learnable class label to the slice feature vector sequence;

[0072] The feature extractor is ResNet50, and the self-supervised framework used to train the feature extractor is DINO.

[0073] ResNet50 is a widely validated deep convolutional neural network with powerful feature extraction capabilities. It can extract rich and representative features from image slices, which are crucial for subsequent pathological image analysis. Furthermore, as a general-purpose feature extractor, ResNet50 can be transferred to other related tasks.

[0074] By adding a learnable category label to the feature vector sequence, the model is able to learn important representations of global information about the image, which helps to capture the correlation and contextual information between slices and improve the understanding of the entire whole slide image.

[0075] Training a feature extractor using a self-supervised framework like DINO means that the model can learn meaningful feature representations without labels. This not only reduces the reliance on large amounts of labeled data, but also enables the model to learn more general and robust features applicable to a variety of different tasks.

[0076] In the preferred scheme, step S3, the key slice selection module Patch Selector, is used to determine whether a specified slice is a useful slice or a useless slice, specifically including:

[0077] The Patch Selector module will calculate a This is used to indicate whether each slice is useful, with 1 indicating useful and 0 indicating useless, and N representing the total number of slices; in the initial stage, Initialized to all 1s to indicate that all slices are useful; in subsequent stages, the current slice features X∈R are used... N×C Update The update process is as follows: First, X is mapped using a multilayer perceptron (MLP) to obtain the local features z of the slice. local :

[0078]

[0079] Then according to Calculate the global feature z with X glocal :

[0080]

[0081] The Agg function represents the aggregation of useful slice features, and it is implemented through global average pooling.

[0082]

[0083] z local and z glocal The local-global features z of the full-wave plate image are obtained by stitching together:

[0084] z i =[z i local ,z i glocal ], 1≤i≤N (4)

[0085] Feed z into another MLP network to obtain the probability π of each slice being useful / useless:

[0086]

[0087] π i,0 π represents the probability that the slice is useless. i,1 This represents the probability that the slice is useful. We obtain D for the current stage using π, and then update it with D.

[0088] D = Gumbel - Softmax(π) *,1 ∈{0,1} N (6)

[0089]

[0090] Gumbel-Softmax performs reparameter sampling on the discrete distribution described above, and ⊙ represents element-level multiplication.

[0091] When processing whole-slide images, not all patches contain information useful for the final classification. Some patches may contain redundant information, noise, or irrelevant background. By selecting key patches, those patches that do not contribute much to the classification can be excluded, thereby improving the efficiency and accuracy of the model.

[0092] The Patch Selector module computes a mask to indicate whether each slice is useful. Initially, all slices are considered useful. In subsequent stages, this mask is updated based on the features of the current slice. The useful / useless probability of each slice is calculated using a Multilayer Perceptron (MLP) and global average pooling operations, and the mask is updated accordingly.

[0093] In the preferred embodiment, in step S3, the useless slice packaging module Patch Packaging is used for packaging. Useless slices with a value of 0 include:

[0094] Calculated using the Patch Selector module Slices with a value of 0 are considered useless slices. These slices are then packaged using the Patch Packaging module for useless slices. Assume there are Q useless slices: And their corresponding useful / useless probabilities: The encapsulation process involves calculating the representative feature P of all useless slices. The formula for calculating P is as follows:

[0095]

[0096] Where, n i Let m represent the feature vector of the i-th useless slice. i [0] represents the probability that the slice is useless.

[0097] Although some slices may be considered useless, they may still contain valuable information that could help in understanding the whole slide image. Encapsulating these useless slices can preserve some of their information and may provide additional context or clues when interacting with useful slices later.

[0098] The Patch Packaging module calculates representative features for all useless slices. By aggregating the features of these useless slices, a wrapper feature is obtained, which can be used as a holistic representation of the useless slices.

[0099] In the preferred embodiment, in step S3, the key slice interaction module is used to extract interaction information between slices, learn the dependencies between slices, and use the Transformer architecture to process key slices, allowing interaction between slices to capture the dependencies between slices and generate a full-slide image feature z containing global information, specifically including:

[0100] Useful slices are selected through the Patch Selector module, and the features of the obtained useful slices are... The encapsulation feature P of the useless slices obtained by the PatchPackaging module will be... P and X are concatenated along the feature dimension, and the concatenated feature matrix X is used as the input to the PatchFormer module. The specific concatenation formula is as follows:

[0101]

[0102] Each PatchFormer contains two sub-modules: a Multi-Head Self-Attention (MHSA) sub-module and a Feedforward Neural Network (FFN). Layer normalization and residual connections are applied before and after each sub-layer, respectively. In the output of the last PatchFormer, the class-labeled state is used as the feature z of the whole slide image. The computation process of each PatchFormer can be represented as follows:

[0103] X′l=X l-1 +MHSA(LN(X l-1 (10)

[0104] X l =X′ l +FFN(LN(X′ l (11)

[0105] Among them, X l-1 X represents the output of the previous layer. l The final output of the current PatchFormer layer, if not the last iteration of the aggregation model, is X. l It can be used as the input feature representation for the next loop. If it is the last loop, X l This is the feature representation of the final output.

[0106] The three modules (Patch Selector for key slice selection, Patch Packaging for useless slices, and PatchFormer for key slice interaction) are arranged alternately and cyclically to form a complete aggregation model. In this embodiment, the input whole-slide image X = {f1, f2, ... f...} N First, a key slice selection module initially distinguishes useful / useless slices. Then, the useless slices are processed by a useless slice encapsulation module to obtain P1. Subsequently, the slices and P1 are sent to the key slice interaction module; this is the first major stage. After this, the useful slices are again sent to the key slice selection module for further useful / useless segmentation. The resulting useless slices are then sent back to the useless slice encapsulation module to obtain P2. Finally, the useful slices, along with P1 and P2, are sent to the key slice interaction module; this is the second major stage. The third and fourth major stages are similar to the first two. At the end of the fourth stage, the output of the key slice interaction module is: useful slices, P1, P2, P3, and P4. The category labels in the useful slices serve as the aggregation features of the whole slide image. These features are then fed into a five-class classification model to obtain the final classification result.

[0107] By taking into account the interactions between slices, the model can learn more complex and in-depth feature representations, thereby improving classification accuracy.

[0108] The PatchFormer module takes features from useful slices and encapsulated features from useless slices as input. It uses a multi-head self-attention submodule and a feedforward neural network to model the interactions between slices. Through the self-attention mechanism, the model learns the dependencies between slices and updates the feature representation of each slice accordingly. Finally, in the output of PatchFormer, the class-labeled state is used as a feature of the whole slide image for subsequent classification tasks.

[0109] In the preferred embodiment, in step S4, the multi-classification network is a five-classification network, which classifies a cervical tissue pathology slide image into one of the following categories: Normal, CIN1, CIN2, CIN3, and Cancer. Normal, CIN1, CIN2, CIN3, and Cancer represent normal, mild dysplasia, moderate dysplasia, severe dysplasia, and cancer, respectively.

[0110] In the preferred embodiment, the aggregation model described in step S3 and the multi-classification network described in step S4 are trained synchronously, and the loss function used for training the aggregation model and the multi-classification network is the cross-entropy loss between the predicted label and the true label.

[0111] By simultaneously training the aggregation model and the multi-classification network, joint optimization of both can be achieved. The aggregation model is responsible for extracting key features and encapsulating useless features from the whole slide image, while the multi-classification network is responsible for classification based on these features. Simultaneous training means that in each iteration, the parameters of both models are updated according to the gradient of the loss function, ensuring that they can work together to improve classification performance. Furthermore, cross-entropy loss is a commonly used loss function in classification tasks, measuring the difference between the model's predicted label and the true label. Using cross-entropy loss as the training objective ensures that the model continuously optimizes its predictive ability during training, making the prediction results closer to the true label. Therefore, the above steps help improve the model's classification performance, generalization ability, and training efficiency.

[0112] In addition, the present invention also provides a multi-classification system for cervical tissue pathology whole slide images based on dynamic slice selection. The system includes an image acquisition unit, an image labeling unit, a feature extraction training unit, an aggregation training unit, a multi-classification model training unit, and a testing unit.

[0113] The image acquisition unit is used to acquire whole slide images of cervical tissue pathology and cervical section images;

[0114] Image labeling unit, for labeling the classification tags of the cervical tissue pathology slide image;

[0115] Feature extraction training unit, used for self-supervised training of cervical slice feature extractor;

[0116] An aggregated training unit is used to supervise the training of an aggregated model of cervical tissue pathology slide images.

[0117] A multi-classification model training unit is used for supervised training of a multi-classification model of cervical tissue pathology whole slide images;

[0118] The test unit is used to test the classification model of the multi-classification model of the cervical tissue pathology whole slide image.

[0119] Finally, the present invention also provides a multi-classification device for cervical tissue pathology whole slide images, comprising one or more processors and a machine-readable storage medium storing machine-executable instructions that can be executed by the processor. The processor reads and executes these instructions to realize the multi-classification method for cervical tissue pathology whole slide images.

[0120] The above embodiments are merely preferred technical solutions of the present invention and should not be considered as limitations on the present invention. The scope of protection of the present invention should be limited to the technical solutions described in the claims, including equivalent substitutions of the technical features described in the claims. That is, equivalent substitutions and improvements within this scope are also within the scope of protection of the present invention.

Claims

1. A multi-classification method for whole-slide images of cervical tissue pathology, characterized by: Includes the following steps: S1. Divide the whole slide image of cervical tissue pathology into non-overlapping slices to obtain the patch dataset; S2. Use a self-supervised pre-trained feature extractor to extract features from all slices in the patch dataset and transform the slices into a series of feature vectors X. S3. The feature vector X is fed into the aggregation model for slice feature aggregation. The aggregation model includes the key slice selection module Patch Selector, the useless slice encapsulation module Patch Packaging, and the key slice interaction module PatchFormer. By alternately looping these three modules, the key features are further refined and extracted to obtain the features of the whole slide image of cervical tissue pathology. S4. Input the features of the whole slide image into a multi-classification network to obtain the classification result of the whole slide image; The above steps enable multi-classification of cervical tissue pathology slide images.

2. The multi-classification method for cervical tissue pathology slide images according to claim 1, characterized in that: In step S2, a feature extractor is used to convert the slices into a series of feature vectors X. Specifically, this includes using the feature extractor to convert the N slices of the whole slide image patch dataset obtained in step S1 into a series of feature vectors X, where X is represented as X = {f1, f2, ... f...}. N }, where f N ∈R C It is the vector of the Nth slice, X∈R N×C It is obtained by stacking all the slice vectors in the whole glass slide image, and then adding a learnable class label to the slice feature vector sequence; The feature extractor is ResNet50, and the self-supervised framework used to train the feature extractor is DINO.

3. The multi-classification method for cervical tissue pathology slide images according to claim 1, characterized in that: In step S3, the key slice selection module Patch Selector is used to determine whether a specified slice is a useful slice or a useless slice, specifically including: The Patch Selector module will calculate a This is used to indicate whether each slice is useful, with 1 indicating useful and 0 indicating useless, and N representing the total number of slices; in the initial stage, Initialized to all 1s to indicate that all slices are useful. In subsequent stages, the current slice features X∈R are used to... N×C Update The update process is as follows: First, X is mapped using a multilayer perceptron (MLP) to obtain the local features z of the slice. local : Then according to Calculate the global feature z with X glocal : The Agg function represents the aggregation of useful slice features, and it is implemented through global average pooling. z local and z glocal The local-global features z of the full-wave plate image are obtained by stitching together: With i =[z i local ,With i glocal ],1≤i≤N (4) Feed z into another MLP network to obtain the probability π of each slice being useful / useless: π i,0 π represents the probability that the slice is useless. i,1 The probability that a slice is useful is represented by π, which is used to obtain D for the current stage, and then D is used to update the value. D=Gumbel-Softmax(π) *,1 ∈{0,1} N (6) Gumbel-Softmax performs reparameter sampling on the discrete distribution described above, and ⊙ represents element-level multiplication.

4. The multi-classification method for whole-slide images of cervical tissue pathology according to claim 1, characterized in that: In step S3, the useless slice packaging module Patch Packaging is used for packaging. Useless slices with a value of 0 include: Calculated using the Patch Selector module Slices with a value of 0 are considered useless slices. These slices are then packaged using the Patch Packaging module for useless slices. Assume there are Q useless slices: And their corresponding useful / useless probabilities: The encapsulation process involves calculating the representative feature P of all useless slices. The formula for calculating P is as follows: Where, n i Let m represent the feature vector of the i-th useless slice. i [0] represents the probability that the slice is useless.

5. The multi-classification method for cervical tissue pathology slide images according to claim 1, characterized in that: In step S3, the key slice interaction module is used to extract interaction information between slices, learn the dependencies between slices, and use the Transformer architecture to process key slices, allowing interaction between slices, thereby capturing the dependencies between slices and generating full-slide image features z containing global information, specifically including: Useful slices are selected through the Patch Selector module, and the features of the useful slices are obtained. The encapsulation feature P of the useless slices obtained by the PatchPackaging module will be... P and X are concatenated along the feature dimension, and the concatenated feature matrix X is used as the input to the PatchFormer module. The specific concatenation formula is as follows: Each PatchFormer contains two sub-modules: a Multi-Head Self-Attention (MHSA) sub-module and a Feedforward Neural Network (FFN). Layer normalization and residual connections are applied before and after each sub-layer, respectively. In the output of the last PatchFormer, the class-labeled state is used as the feature z of the whole slide image. The computation process of each PatchFormer can be represented as follows: X l =X l-1 +MHSA(LN(X l-1 ))(10)X l =X l '+FFN(LN(X l '))(11) where X l-1 X represents the output of the previous layer. l The final output of the current PatchFormer layer, if not the last iteration of the aggregation model, is X. l It can be used as the input feature representation for the next loop. If it is the last loop, X l This is the feature representation of the final output.

6. The multi-classification method for whole-slide images of cervical tissue pathology according to claim 1, characterized in that: In step S4, the multi-classification network is a five-classification network, which classifies a cervical tissue pathology slide image into one of the following categories: Normal, CIN1, CIN2, CIN3, and Cancer.

7. The multi-classification method for cervical tissue pathology whole slide images based on dynamic slice selection according to claim 1, characterized in that: The aggregation model described in step S3 and the multi-classification network described in step S4 are trained synchronously. The loss function used for training the aggregation model and the multi-classification network is the cross-entropy loss between the predicted label and the true label.

8. The cervical tissue pathology whole slide image multi-classification system according to claim 1, characterized in that: The system includes an image acquisition unit, an image labeling unit, a feature extraction training unit, an aggregation training unit, a multi-classification model training unit, and a testing unit; The image acquisition unit is used to acquire whole slide images of cervical tissue pathology and cervical section images; Image labeling unit, for labeling the classification tags of the whole slide image of cervical tissue pathology; Feature extraction training unit, used for self-supervised training of cervical slice feature extractor; An aggregated training unit is used to supervise the training of an aggregated model of cervical tissue pathology slide images. A multi-classification model training unit is used for supervised training of a multi-classification model of cervical tissue pathology whole slide images; The test unit is used to test the classification model of the multi-classification model of the cervical tissue pathology whole slide image.

9. A multi-classification device for whole-slide images of cervical tissue pathology, characterized in that: It includes one or more processors and a machine-readable storage medium; The storage medium stores machine-executable instructions that can be executed by the processor; The processor reads and executes these instructions to implement the method described in any one of claims 1 to 7.

Citation Information

Patent Citations

  • Convolutional neural network model training method and device for medical image recognition

    CN110909756A

  • Method for analyzing pathology image based on weakly supervised learning using self-supervised model

    KR102640426B1