Whole slide image automated analysis method, apparatus, computer device, storage medium and computer program product

Abstract

The application relates to a full slice image automatic analysis method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: preprocessing a full slice image to be analyzed to obtain a preprocessed full slice image; performing feature extraction processing on an image block in the preprocessed full slice image through a pre-trained feature extractor to obtain local pathological features of the image block; performing feature aggregation processing on the local pathological features to obtain global pathological features of the preprocessed full slice image; determining a typing result and a grading result of the preprocessed full slice image according to the global pathological features; inputting the image block and the local pathological features corresponding to the image block into a trained segmentation model to obtain a segmentation result of the preprocessed full slice image; and obtaining an image analysis result of the full slice image based on the typing result, the grading result and the segmentation result. The method can improve the analysis accuracy of the full slice image.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, apparatus, computer device, computer-readable storage medium, and computer program product for automated analysis of full-slice images. Background Technology

[0002] Currently, with the increasing demand for intelligent analysis of whole-slice images, the ability to accurately perform image analysis on whole-slice images has become crucial.

[0003] In traditional techniques, a single classification model is generally used to analyze whole-slice images; however, the image analysis results obtained by this method are relatively one-sided, resulting in low accuracy of whole-slice image analysis. Summary of the Invention

[0004] Therefore, it is necessary to provide an automated analysis method, apparatus, computer equipment, computer-readable storage medium, and computer program product for whole-slice images that can improve the analysis accuracy of whole-slice images, addressing the aforementioned technical problems.

[0005] Firstly, this application provides an automated analysis method for whole-slice images, including:

[0006] Obtain the full slice image to be analyzed, and preprocess the full slice image to obtain the preprocessed full slice image;

[0007] By using a pre-trained feature extractor, feature extraction is performed on image patches in the preprocessed whole-slice image to obtain the local pathological features of the image patches;

[0008] The local pathological features are subjected to feature aggregation processing to obtain the global pathological features of the preprocessed whole slice image;

[0009] Based on the global pathological features, the typing and grading results of the preprocessed whole slice image are determined. The image block and the corresponding local pathological features are input into the trained segmentation model to obtain the segmentation result of the preprocessed whole slice image.

[0010] Based on the classification results, the grading results, and the segmentation results, the image analysis results of the whole slice image are obtained.

[0011] In one embodiment, the step of performing feature extraction processing on image patches in the preprocessed whole-slice image using a pre-trained feature extractor to obtain local pathological features of the image patches includes:

[0012] By using a pre-trained feature extractor, feature extraction is performed on image patches in the preprocessed whole-slice image to obtain keratinized bead features, intercellular bridge features, and nuclear atypia features of the image patches;

[0013] Based on the keratinized pearl features, the intercellular bridge features, and the nuclear atypia features, the local pathological features of the image patch are obtained.

[0014] In one embodiment, the pre-trained feature extractor is trained in the following manner:

[0015] Obtain sample image blocks from the full slice image of the sample, and perform partial masking processing on the sample image blocks to obtain masked image blocks;

[0016] The masked image patch is input into the feature extractor to be trained to obtain the sample local pathological features of the masked image patch;

[0017] The local pathological features of the sample are reconstructed using a decoder to obtain reconstructed image blocks.

[0018] Based on the difference between the sample image patch and the reconstructed image patch, the feature extractor to be trained is iteratively trained to obtain the pre-trained feature extractor.

[0019] In one embodiment, the step of performing feature aggregation processing on the local pathological features to obtain the global pathological features of the preprocessed whole-slice image includes:

[0020] The local pathological features are processed using an attention mechanism to obtain the feature weights corresponding to the local pathological features.

[0021] According to the aforementioned feature weights, the local pathological features are subjected to feature aggregation processing to obtain the global pathological features of the preprocessed whole-slice image.

[0022] In one embodiment, the step of inputting the image patch and the corresponding local pathological features into the trained segmentation model to obtain the segmentation result of the preprocessed whole-slice image includes:

[0023] The image patch is subjected to feature extraction processing to obtain the texture features of the image patch;

[0024] The texture features and the local pathological features corresponding to the image patch are fused to obtain the fused features of the image patch;

[0025] The fused features are input into the trained segmentation model to obtain the sub-segmentation results of the image patch; the sub-segmentation results are a probability map representing the probability that each pixel in the image patch belongs to an abnormal pixel;

[0026] The sub-segmentation results are stitched together to obtain the segmentation result of the preprocessed full-slice image.

[0027] In one embodiment, the preprocessing of the whole-slice image to obtain a preprocessed whole-slice image includes:

[0028] The full-slice image is downsampled to obtain the processed full-slice image;

[0029] The processed full-slice image is subjected to threshold filtering to obtain a filtered full-slice image;

[0030] The filtered full-slice image is subjected to color normalization and pixel value normalization to obtain a normalized full-slice image.

[0031] Based on the normalized full slice image, the preprocessed full slice image is obtained.

[0032] Secondly, this application also provides an automated whole-slice image analysis device, comprising:

[0033] The image processing module is used to acquire the full slice image to be analyzed, and to preprocess the full slice image to obtain the preprocessed full slice image.

[0034] The feature extraction module is used to perform feature extraction processing on image patches in the preprocessed whole slice image using a pre-trained feature extractor to obtain the local pathological features of the image patches;

[0035] The feature aggregation module is used to perform feature aggregation processing on the local pathological features to obtain the global pathological features of the preprocessed whole slice image;

[0036] The result determination module is used to determine the typing and grading results of the preprocessed whole slice image based on the global pathological features, and input the image block and the corresponding local pathological features into the trained segmentation model to obtain the segmentation result of the preprocessed whole slice image.

[0037] The results analysis module is used to obtain the image analysis results of the whole slice image based on the classification results, the grading results, and the segmentation results.

[0038] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0039] Obtain the full slice image to be analyzed, and preprocess the full slice image to obtain the preprocessed full slice image;

[0040] By using a pre-trained feature extractor, feature extraction is performed on image patches in the preprocessed whole-slice image to obtain the local pathological features of the image patches;

[0041] The local pathological features are subjected to feature aggregation processing to obtain the global pathological features of the preprocessed whole slice image;

[0042] Based on the global pathological features, the typing and grading results of the preprocessed whole slice image are determined. The image block and the corresponding local pathological features are input into the trained segmentation model to obtain the segmentation result of the preprocessed whole slice image.

[0043] Based on the classification results, the grading results, and the segmentation results, the image analysis results of the whole slice image are obtained.

[0044] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0045] Obtain the full slice image to be analyzed, and preprocess the full slice image to obtain the preprocessed full slice image;

[0046] By using a pre-trained feature extractor, feature extraction is performed on image patches in the preprocessed whole-slice image to obtain the local pathological features of the image patches;

[0047] The local pathological features are subjected to feature aggregation processing to obtain the global pathological features of the preprocessed whole slice image;

[0048] Based on the global pathological features, the typing and grading results of the preprocessed whole slice image are determined. The image block and the corresponding local pathological features are input into the trained segmentation model to obtain the segmentation result of the preprocessed whole slice image.

[0049] Based on the classification results, the grading results, and the segmentation results, the image analysis results of the whole slice image are obtained.

[0050] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0051] Obtain the full slice image to be analyzed, and preprocess the full slice image to obtain the preprocessed full slice image;

[0052] By using a pre-trained feature extractor, feature extraction is performed on image patches in the preprocessed whole-slice image to obtain the local pathological features of the image patches;

[0053] The local pathological features are subjected to feature aggregation processing to obtain the global pathological features of the preprocessed whole slice image;

[0054] Based on the global pathological features, the typing and grading results of the preprocessed whole slice image are determined. The image block and the corresponding local pathological features are input into the trained segmentation model to obtain the segmentation result of the preprocessed whole slice image.

[0055] Based on the classification results, the grading results, and the segmentation results, the image analysis results of the whole slice image are obtained.

[0056] The aforementioned automated whole-slice image analysis method, apparatus, computer equipment, storage medium, and computer program product first acquire the whole-slice image to be analyzed, preprocess the whole-slice image to obtain a preprocessed whole-slice image, then use a pre-trained feature extractor to extract features from image blocks in the preprocessed whole-slice image to obtain local pathological features of the image blocks, then perform feature aggregation processing on the local pathological features to obtain global pathological features of the preprocessed whole-slice image, then determine the classification and grading results of the preprocessed whole-slice image based on the global pathological features, input the image blocks and their corresponding local pathological features into the trained segmentation model to obtain the segmentation results of the preprocessed whole-slice image, and finally obtain the image analysis results of the whole-slice image based on the classification, grading, and segmentation results. In this way, when performing image analysis on whole-slice images, the local pathological features of each image block are extracted after the whole-slice image is preprocessed, and then global pathological features are generated through feature aggregation. Based on the global features, the classification and grading results are output simultaneously. At the same time, the image block and local pathological features are combined to obtain fine segmentation results through a segmentation model. The three types of results are integrated to make a comprehensive judgment on the final image analysis result. This avoids the shortcomings of traditional techniques that use a single classification model for analysis, which result in a relatively one-sided image analysis result and low accuracy of whole-slice image analysis. Thus, the accuracy of whole-slice image analysis is improved. Attached Figure Description

[0057] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0058] Figure 1 This is a flowchart illustrating an automated analysis method for full-slice images in one embodiment;

[0059] Figure 2 This is a flowchart illustrating an automated analysis method for full-slice images in another embodiment;

[0060] Figure 3 This is a schematic diagram illustrating the overall workflow of the CSCC-PAG (Cervical Squamous Cell Carcinoma - Pre-training, Adaptation, Generalization) framework in one embodiment.

[0061] Figure 4 This is a schematic diagram comparing the performance of various MIL (Multiple Instance Learning) models based on baseline UNI (Universal Neural Image) features and SPC (Spatial Pathological Context) features in one embodiment.

[0062] Figure 5 This is a schematic diagram comparing the performance of various MIL models based on baseline UNI features and SPC features in another embodiment;

[0063] Figure 6 This is a schematic diagram comparing the attention heatmaps of the same WSI (Whole Slide Image) under different feature extractors in one embodiment;

[0064] Figure 7 This is a schematic diagram comparing the attention heatmaps of the same WSI under different feature extractors in another embodiment;

[0065] Figure 8 This is a schematic diagram comparing the model performance of SPC features, UNI features, and ImageNet (Image Network) features on TransUNet (Transformer-based UNet, a segmentation network) in one embodiment.

[0066] Figure 9 This is a schematic diagram of the visualization of tumor region segmentation results in one embodiment;

[0067] Figure 10This is a performance evaluation comparison chart of the domain migration framework in multiple test scenarios in one embodiment, including a schematic diagram of the performance comparison of the complete domain migration framework, the baseline model, and the ablation experiment;

[0068] Figure 11 This is a schematic diagram illustrating the technical details of a domain migration self-supervised framework in one embodiment;

[0069] Figure 12 This is a structural block diagram of a full-slice image automated analysis device in one embodiment;

[0070] Figure 13 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0071] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0072] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0073] In one exemplary embodiment, such as Figure 1 As shown, an automated method for analyzing whole-slice images is provided. This embodiment illustrates the application of this method to a server; it is understood that this method can also be applied to a terminal, or to a system including a terminal and a server, and is implemented through interaction between the terminal and the server. The terminal can be, but is not limited to, various personal computers, laptops, smartphones, and tablets; the server can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server providing cloud computing services. In this embodiment, the method includes the following steps:

[0074] Step S101: Obtain the full slice image to be analyzed, and preprocess the full slice image to obtain the preprocessed full slice image.

[0075] Among them, the whole slide image, also known as the whole slide image of cervical squamous cell carcinoma, specifically refers to the whole slide image of cervical tissue cells, which can be obtained by scanning the entire cervical tissue pathology slide using a digital pathology slide scanner.

[0076] Among them, the preprocessed full slice image refers to the full slice image after preprocessing.

[0077] For example, in response to an image analysis request corresponding to the whole slide image to be analyzed, the server receives the whole slide image output by the digital pathology slide scanner; then, it performs downsampling processing, threshold filtering processing, staining normalization processing, and pixel value normalization processing on the whole slide image to obtain the preprocessed whole slide image.

[0078] Step S102: Using a pre-trained feature extractor, feature extraction is performed on image patches in the pre-processed whole-slice image to obtain local pathological features of the image patches.

[0079] Among them, the feature extractor refers to a dedicated deep convolutional feature network pre-trained based on a cervical squamous cell carcinoma dataset.

[0080] Here, an image block refers to a local sub-image obtained by sliding cropping and splitting the preprocessed full slice image according to a fixed pixel size.

[0081] Among them, local pathological features are feature vectors used to represent the pathological structural information of image patches.

[0082] For example, the server performs region-by-region sliding cropping on the preprocessed full-slice image according to a preset fixed window size and a preset sliding step size to obtain image blocks in the preprocessed full-slice image; then, the image blocks in the preprocessed full-slice image are input into a pre-trained feature extractor, and the pre-trained feature extractor performs feature extraction processing on the image blocks in the preprocessed full-slice image to obtain the local pathological features of the image blocks.

[0083] Step S103: Perform feature aggregation processing on local pathological features to obtain the global pathological features of the preprocessed whole slice image.

[0084] Among them, global pathological features are used to represent feature vectors of pathological structural information of the preprocessed whole slice image.

[0085] For example, the server performs feature aggregation processing on local pathological features through a preset attention-weighted aggregation mechanism to obtain the global pathological features of the preprocessed whole slice image.

[0086] Step S104: Based on the global pathological features, determine the typing and grading results of the preprocessed whole slice image, and input the image blocks and the corresponding local pathological features into the trained segmentation model to obtain the segmentation results of the preprocessed whole slice image.

[0087] The typing results are used to characterize the pathological category of cervical tissue cells in the preprocessed whole-slice images, such as normal cervical tissue, cervical intraepithelial neoplasia, squamous cell carcinoma, etc.

[0088] Among them, the grading results are used to characterize the cancer grade of cervical tissue cells in the preprocessed whole slice image, such as high differentiation, moderate differentiation, and low differentiation cancer grades.

[0089] Among them, the segmentation model refers to a semantic segmentation network based on TransUNet architecture and trained specifically using SPC pathological features.

[0090] The segmentation results are used to characterize the boundary contours and spatial distribution range of cancerous lesion areas, normal tissue areas, and inflammatory infiltration areas of cervical tissue cells in the preprocessed whole-slice image.

[0091] For example, the server determines the number of feature extractions corresponding to the global pathological features based on the feature sparsity of the global pathological features, and performs multiple feature extractions on the global pathological features according to the number of feature extractions corresponding to the global pathological features to obtain the processed global pathological features. Then, the processed global pathological features are input into the trained classification model to obtain the predicted probability of the preprocessed whole slice image under each preset classification result, and the preset classification result with the highest predicted probability is determined from each preset classification result as the classification result of the preprocessed whole slice image. At the same time, the processed global pathological features are input into the trained grading model to obtain the predicted probability of the preprocessed whole slice image under each preset grading result, and the preset grading result with the highest predicted probability is determined from each preset grading result as the grading result of the preprocessed whole slice image. Then, the image patch and the local pathological features corresponding to the image patch are input into the trained segmentation model to obtain the segmentation result of the preprocessed whole slice image.

[0092] Step S105: Based on the classification results, grading results, and segmentation results, obtain the image analysis results of the whole slice image.

[0093] Among them, the image analysis result refers to the comprehensive analysis result obtained by integrating the classification result, the grading result, and the segmentation result.

[0094] For example, the server performs structured fusion processing on the classification results, grading results, and segmentation results to obtain the image analysis results of the whole slice image.

[0095] In the above-mentioned automated whole-slice image analysis method, the whole-slice image to be analyzed is first acquired, and then preprocessed to obtain a preprocessed whole-slice image. Next, a pre-trained feature extractor is used to extract features from image patches in the preprocessed whole-slice image to obtain local pathological features of the image patches. Then, feature aggregation is performed on the local pathological features to obtain global pathological features of the preprocessed whole-slice image. Then, based on the global pathological features, the classification and grading results of the preprocessed whole-slice image are determined. The image patches and their corresponding local pathological features are input into the trained segmentation model to obtain the segmentation results of the preprocessed whole-slice image. Finally, based on the classification, grading, and segmentation results, the image analysis results of the whole-slice image are obtained. In this way, when performing image analysis on whole-slice images, the local pathological features of each image block are extracted after the whole-slice image is preprocessed, and then global pathological features are generated through feature aggregation. Based on the global features, the classification and grading results are output simultaneously. At the same time, the image block and local pathological features are combined to obtain fine segmentation results through a segmentation model. The three types of results are integrated to make a comprehensive judgment on the final image analysis result. This avoids the shortcomings of traditional techniques that use a single classification model for analysis, which result in a relatively one-sided image analysis result and low accuracy of whole-slice image analysis. Thus, the accuracy of whole-slice image analysis is improved.

[0096] In an exemplary embodiment, step S102 above, which uses a pre-trained feature extractor to perform feature extraction processing on image blocks in the pre-processed whole-slice image to obtain local pathological features of the image blocks, specifically includes the following: using a pre-trained feature extractor to perform feature extraction processing on image blocks in the pre-processed whole-slice image to obtain keratin pearl features, intercellular bridge features, and nuclear atypia features of the image blocks; and obtaining local pathological features of the image blocks based on keratin pearl features, intercellular bridge features, and nuclear atypia features.

[0097] Among them, the keratin pearl feature is used to characterize the morphological features, distribution density and spatial distribution scale of the keratin pearl structure corresponding to cervical tissue cells in the image block.

[0098] Among them, the intercellular bridge feature is used to characterize the connection morphology and integrity between cervical tissue cells in the image patch.

[0099] Among them, nuclear atypia features are used to characterize the degree of morphological abnormality of the cell nuclei of cervical tissue cells in the image patch.

[0100] For example, the server inputs image patches from the preprocessed whole-slice image into a pre-trained feature extractor. The pre-trained feature extractor performs feature extraction on the image patches in the preprocessed whole-slice image to obtain keratinized pearl features, intercellular bridge features, and nuclear atypia features of the image patches. Then, the keratinized pearl features, intercellular bridge features, and nuclear atypia features are normalized to obtain normalized keratinized pearl features, normalized intercellular bridge features, and normalized nuclear atypia features. Then, based on the feature weights corresponding to the normalized keratinized pearl features, the normalized intercellular bridge features, and the normalized nuclear atypia features, feature fusion processing is performed on the normalized keratinized pearl features, normalized intercellular bridge features, and normalized nuclear atypia features to obtain the local pathological features of the image patches.

[0101] In this embodiment, when extracting local pathological features of image blocks, three core pathological-specific micro-indicators—keratin pearl features, intercellular bridge features, and nuclear atypia features—are specifically decomposed and extracted separately. These three types of key fine-grained features are then fused to form local pathological features, providing underlying features with stronger pathological correlation for subsequent feature aggregation, classification and grading determination, and lesion segmentation.

[0102] In an exemplary embodiment, in step S102 above, the pre-trained feature extractor is trained in the following manner: acquiring sample image blocks from a full slice image of a sample, performing partial masking on the sample image blocks to obtain masked image blocks; inputting the masked image blocks into the feature extractor to be trained to obtain the sample local pathological features of the masked image blocks; performing image reconstruction processing on the sample local pathological features through a decoder to obtain reconstructed image blocks; and iteratively training the feature extractor to be trained based on the differences between the sample image blocks and the reconstructed image blocks to obtain the pre-trained feature extractor.

[0103] Here, the full-slice image of the sample refers to the full-slice image used to train the feature extractor.

[0104] Among them, a sample image block refers to a local sub-image obtained by sliding cropping and splitting a full slice of sample image according to a fixed pixel size.

[0105] Among them, a masked image block refers to a fragmented image block obtained by randomly occluding, masking, or filling blanks in a portion of the pixel region of a sample image block.

[0106] Among them, the local pathological features of the sample are used as feature vectors to represent the pathological structural information of the sample image block.

[0107] Among them, the reconstructed image block refers to the restored image block generated by reverse decoding and restoration based on the local pathological features of the sample.

[0108] For example, the server acquires a full-slice image of a sample, and performs region-by-region sliding cropping on the full-slice image according to a preset fixed window size and a preset sliding step size to obtain sample image blocks in the full-slice image. Then, a local region within the sample image block is randomly selected and a pixel masking operation is performed, setting the selected local region pixels to zero or filling them with a solid color to obtain a mask image block. Then, the mask image block is input into the feature extractor to be trained to obtain the keratinized bead features, intercellular bridge features, and nuclear atypia features of the mask image block, which are used as the sample local pathological features of the mask image block. Next, the sample local pathological features are input into the decoder, and the decoder performs image reconstruction processing on the sample local pathological features to obtain a reconstructed image block. Then, the loss value is obtained based on the pixel difference between the sample image block and the reconstructed image block. Next, the extractor parameters of the feature extractor to be trained are adjusted according to the loss value, and the feature extractor with adjusted extractor parameters is retrained until the loss value obtained by the trained feature extractor is less than the loss value threshold. Then, the trained feature extractor is used as the pre-trained feature extractor.

[0109] In this embodiment, by masking the local area of ​​the sample image block, and then using the decoder to restore the complete reconstructed image block based on the extracted local pathological features of the sample, the network parameters are optimized by iteratively optimizing the pixel loss of the original sample image block and the reconstructed image block. This helps to improve the feature extractor's ability to perceive subtle pathological structures and its feature robustness.

[0110] In an exemplary embodiment, step S103 above, which involves performing feature aggregation processing on local pathological features to obtain global pathological features of the preprocessed whole-slice image, specifically includes the following: performing attention mechanism processing on local pathological features to obtain feature weights corresponding to the local pathological features; and performing feature aggregation processing on the local pathological features according to the feature weights to obtain global pathological features of the preprocessed whole-slice image.

[0111] Among them, feature weight refers to the normalized weight coefficient obtained by scoring local pathological features through the attention mechanism.

[0112] For example, the server decomposes the local pathological features to obtain the keratinized pearl features, intercellular bridge features, and nuclear atypia features of the image patch. Then, the keratinized pearl features of the image patch are input into the attention mechanism model to obtain the first feature weight corresponding to the local pathological features. The intercellular bridge features of the image patch are input into the attention mechanism model to obtain the second feature weight corresponding to the local pathological features. The nuclear atypia features of the image patch are input into the attention mechanism model to obtain the third feature weight corresponding to the local pathological features. Then, the first feature weight, the second feature weight, and the third feature weight are fused to obtain the feature weight corresponding to the local pathological features. According to the feature weight, the local pathological features are weighted and summed to obtain the global pathological features of the preprocessed whole-slice image.

[0113] In this embodiment, by using an attention mechanism to adaptively assign differentiated feature weights to the local pathological features corresponding to each image block, the accuracy of global pathological feature representation is improved, providing more targeted and recognizable global feature basis for subsequent pathological classification and grading, and improving the overall accuracy of pathological diagnosis and judgment.

[0114] In an exemplary embodiment, step S104 above, which inputs the image patch and the corresponding local pathological features into the trained segmentation model to obtain the segmentation result of the preprocessed whole slice image, specifically includes the following: performing feature extraction processing on the image patch to obtain the texture features of the image patch; performing fusion processing on the texture features and the corresponding local pathological features to obtain the fused features of the image patch; inputting the fused features into the trained segmentation model to obtain the sub-segmentation result of the image patch; the sub-segmentation result is a probability map used to represent each pixel in the image patch as an abnormal pixel; and performing splicing processing on the sub-segmentation results to obtain the segmentation result of the preprocessed whole slice image.

[0115] Texture features are used to represent feature vectors that represent the surface structure information of image patches.

[0116] Among them, fusion features refer to the feature vector obtained by splicing or weighted fusion of texture features and corresponding local pathological features.

[0117] Abnormal pixels refer to pixels within an image block that belong to cancerous lesions or inflammatory infiltration areas.

[0118] Among them, the probability map refers to the pixel-level probability heatmap output by the segmentation model. Each pixel position in the map outputs a probability value in the range of 0 to 1. The higher the value, the higher the confidence that the pixel belongs to an abnormal lesion pixel.

[0119] For example, the server inputs an image patch into a texture feature extraction model, which performs feature extraction on the image patch to obtain its texture features. Then, according to the first weight corresponding to the texture features and the second weight corresponding to the local pathological features, the texture features and the corresponding local pathological features are fused to obtain the fused features of the image patch. Next, the fused features are input into a trained segmentation model, which outputs the sub-segmentation results of the image patch. The sub-segmentation results are probability maps used to represent each pixel in the image patch as an abnormal pixel. Then, according to the spatial coordinates of the image patch in the preprocessed full-slice image, the sub-segmentation results are smoothly stitched together to obtain the segmentation results of the preprocessed full-slice image.

[0120] In this embodiment, by fusing texture features with corresponding local pathological features during the segmentation stage, the shortcomings of single pathological features in easily losing subtle lesion boundaries and single texture features in lacking pathological discrimination basis are compensated for. This improves the segmentation model's ability to identify blurred areas at the boundary between lesion areas and normal tissues, providing accurate spatial positioning basis for subsequent comprehensive pathological analysis.

[0121] In an exemplary embodiment, step S101 above, which preprocesses the full slice image to obtain a preprocessed full slice image, specifically includes the following: downsampling the full slice image to obtain a processed full slice image; threshold filtering the processed full slice image to obtain a filtered full slice image; color normalization and pixel value normalization the filtered full slice image to obtain a normalized full slice image; and obtaining a preprocessed full slice image based on the normalized full slice image.

[0122] Downsampling refers to the downsampling operation that compresses the resolution and reduces the scale of an image.

[0123] Threshold filtering refers to the processing operation of filtering images based on grayscale thresholds.

[0124] Among them, staining normalization processing refers to the unified processing operation of standard staining color distribution of cell nuclei, cytoplasm and interstitial regions in the image through color space transformation and reference template mapping.

[0125] Pixel value normalization refers to the process of linearly mapping the grayscale values ​​of pixels in each channel of an image to a fixed range of values.

[0126] For example, the server uses a bilinear interpolation algorithm to downsample the full-slice image to obtain a processed full-slice image; then, according to a grayscale threshold, the processed full-slice image is threshold-filtered to obtain a filtered full-slice image; then, the filtered full-slice image is converted to a preset color space, and a preset coloring reference template is retrieved to complete coloring normalization correction to obtain a coloring normalized full-slice image; next, the grayscale values ​​of each channel pixel in the coloring normalized full-slice image are linearly scaled to obtain a normalized full-slice image; finally, the normalized full-slice image is subjected to data cleaning to obtain a preprocessed full-slice image.

[0127] In this embodiment, the whole slice image is downsampled, threshold filtered, stained and normalized, and pixel value normalized. After multi-step preprocessing, a standardized preprocessed whole slice image is obtained. This eliminates external interference from imaging, slide preparation and numerical scale at the data level, ensuring the accuracy and consistency of subsequent feature extraction, classification and grading and lesion segmentation.

[0128] In one exemplary embodiment, such as Figure 2 As shown, another method for automated analysis of whole-slice images is provided. Taking the application of this method to a server as an example, the specific steps include:

[0129] Step S201: Obtain the full slice image to be analyzed.

[0130] Step S202: Downsample the full slice image to obtain the processed full slice image; perform threshold filtering on the processed full slice image to obtain the filtered full slice image.

[0131] Step S203: Perform color normalization and pixel value normalization on the filtered full slice image to obtain a normalized full slice image; based on the normalized full slice image, obtain a preprocessed full slice image.

[0132] Step S204: Using a pre-trained feature extractor, feature extraction is performed on the image patches in the pre-processed whole-slice image to obtain the keratinized bead features, intercellular bridge features, and nuclear atypia features of the image patches.

[0133] Step S205: Based on the features of keratinized pearls, intercellular bridges, and nuclear atypia, the local pathological features of the image patch are obtained.

[0134] Step S206: Apply attention mechanism processing to the local pathological features to obtain the feature weights corresponding to the local pathological features.

[0135] Step S207: According to the feature weight, perform feature aggregation processing on the local pathological features to obtain the global pathological features of the preprocessed whole slice image.

[0136] Step S208: Based on the global pathological features, determine the typing and grading results of the preprocessed whole-slice images.

[0137] Step S209: Perform feature extraction processing on the image patch to obtain the texture features of the image patch.

[0138] Step S210: The texture features and the local pathological features corresponding to the image patch are fused to obtain the fused features of the image patch.

[0139] Step S211: Input the fused features into the trained segmentation model to obtain the sub-segmentation results of the image patch; the sub-segmentation results are a probability map used to represent each pixel in the image patch as an abnormal pixel.

[0140] Step S212: The sub-segmentation results are stitched together to obtain the segmentation result of the preprocessed full-slice image.

[0141] Step S213: Based on the classification results, grading results, and segmentation results, obtain the image analysis results of the whole slice image.

[0142] In the aforementioned automated whole-slice image analysis method, when performing image analysis on whole-slice images, the local pathological features of each image block are extracted after preprocessing the whole-slice image. Then, global pathological features are generated through feature aggregation. Based on the global features, the classification and grading results are output simultaneously. At the same time, the image blocks and local pathological features are combined to obtain fine segmentation results through a segmentation model. The three types of results are then integrated to comprehensively determine the final image analysis result. This avoids the shortcomings of traditional techniques that use a single classification model for analysis, which result in relatively one-sided image analysis results and low accuracy of whole-slice image analysis. Thus, the accuracy of whole-slice image analysis is improved.

[0143] In an exemplary embodiment, to more clearly illustrate the automated whole-slice image analysis method provided by the embodiments of this application, the following specific embodiment will be used to describe the automated whole-slice image analysis method. In one embodiment, this application also provides an automated whole-slice image analysis method and apparatus for cervical squamous cell carcinoma based on domain-specific pre-training and self-supervised domain transfer. Specifically, it includes the following:

[0144] In this embodiment, all WSIs were stained with standard hematoxylin and eosin and digitized at 40× magnification (0.25 μm / pixel resolution). The system first downsampled all WSIs to 20× magnification (approximately 0.5 μm / pixel) using the OpenSlide (a database) library.

[0145] Specifically as follows:

[0146] (1) Construction of a large-scale pre-training dataset: About 8,000 CSCC WSI images were collected as the upstream self-supervised pre-training dataset (CESC (Cervical Squamous Cell Carcinoma)-normal) for the domain-specific feature extractor SPC. This dataset does not require manual annotation and makes full use of large-scale pathological image information through self-supervised learning.

[0147] (2) Downstream task dataset construction: 1,000 WSI images (mainly from Ningbo Clinical Pathology Diagnostic Center) were selected from the above dataset and jointly annotated by multiple senior pathologists, including CSCC typing labels, grading labels, and fine segmentation masks for tumor regions. If multiple pathology experts disagreed, a third expert was invited to review and reach a consensus, which was used for supervised training and validation of downstream typing, grading, and segmentation tasks.

[0148] (3) Domain transfer dataset construction: 1,000 WSIs were collected for training the self-supervised domain transfer framework. Among them, 235 WSIs with complete annotations were independently selected as the target domain test set for evaluating the model’s cross-center domain transfer performance.

[0149] (4) Public validation dataset: The TCGA (The Cancer Genome Atlas) cervical cancer dataset (TCGA-CESC) was used in addition, and 189 WSIs diagnosed as squamous cell carcinoma were selected as an independent out-of-domain validation set to evaluate the framework’s generalization ability on completely unseen third-party data.

[0150] (5) Patch Extraction: Foreground regions of the tissue were extracted using a thresholding method, and all foreground regions were divided into 256×256 pixel non-overlapping patch images. These patches were then used to cover the entire tissue region in a non-overlapping tiling manner, ensuring that the complete set of full-slice images was used for both classification and segmentation tasks. To address the staining differences between different centers, the Staintools library (a database) was used for staining normalization. During training, the RGB (Red, Green, Blue) pixel values ​​were normalized to the range of [-1, 1] using the ImageNet mean and standard deviation.

[0151] To overcome the "domain gap" of general-purpose basic models, this embodiment constructs a domain-specific feature extractor, SPC, based on the ViT (Vision Transformer, feature extraction model) architecture, referring to... Figure 3 (A). SPC is based on the UNI model architecture and performs self-supervised pre-training on the aforementioned large-scale unlabeled CESC-normal dataset.

[0152] (1) Model construction: The SPC backbone adopts the Vision Transformer (ViT) architecture, which divides the input image into non-overlapping 16×16 patches, and then inputs them into a multi-layer Transformer encoder after linear mapping.

[0153] Let the input image be:

[0154] Equation (1)

[0155] After dividing into N patches, the sequence is obtained. Each patch is mapped to a vector. The core self-attention mechanism in Transformer is calculated as follows:

[0156] Equation (2)

[0157] in , , , For vector dimensions, , , These are learnable parameters. A deep representation is constructed by stacking ViT layers, which are then combined with residual connections and a feedforward network (FFN) to ultimately extract global contextual features of the image.

[0158] (2) Self-supervised pre-training strategy: The training process adopts a self-supervised learning method, which forces the model to learn the deep pathological characteristics specific to CSCC, including the formation pattern of keratin pearls, obvious intercellular bridges, and nuclear atypia differences between different differentiation levels, providing a high-quality feature initialization basis for all downstream tasks.

[0159] Specifically, SPC pre-training employs a self-supervised strategy of Masked Image Modeling (MIM): 75% of the token positions are randomly masked for each input image patch, the ViT encoder encodes the uncovered portion, and a lightweight decoder reconstructs the pixel values ​​of the masked region, using mean squared error (MSE) as the reconstruction loss.

[0160] Equation (3)

[0161] in These are the original pixel values. The values ​​represent the reconstructed values, where |masked| represents the total number of masked tokens. High masking rates force the model to infer local cervical tissue morphology under conditions of significant contextual loss, thereby learning fine-grained pathological characterization specific to CSCC. Pre-training uses the Adam (Adaptive Moment Estimation) optimizer with an initial learning rate of [value missing]. The training process lasts for 10 epochs, and the resulting SPC weights are used as the basis for initializing high-performance features for all downstream tasks.

[0162] After obtaining the SPC feature extractor, this embodiment constructs a parallel multi-task analysis framework, using SPC as the shared backbone network for all tasks, referencing... Figure 3 (B).

[0163] (1) Full-slice classification and grading module (MIL framework): The full-slice-level classification and grading task is based on the Multiple Instance Learning (MIL) paradigm. For each WSI, the system extracts all foreground patches under it. Features are obtained through the SPC encoder. Then, a slide-level representation is generated using the CLAM (CLustering-constrained Attention Multiple-instance Learning) model.

[0164] Equation (4)

[0165] in As an attention weight, CLAM automatically focuses on the most diagnostically valuable regions through its attention mechanism and aggregates features to output WSI-level typing and grading predictions.

[0166] CLAM uses a gated attention mechanism to calculate the contribution weight of each image patch to the slice-level prediction. The complete formula is as follows:

[0167] Equation (5)

[0168] in and Here, ⊙ represents element-wise multiplication, and σ is the Sigmoid function (an activation function). The parallel design of the tanh (an activation function) path and the Sigmoid dual-gated path effectively filters out regions with no diagnostic value, such as the matrix and artifacts, focusing on the core regions of malignant lesions. Simultaneously, CLAM introduces an auxiliary clustering constraint loss. By minimizing the cross-entropy of cluster assignments between high-weight patches (potential positive examples) and low-weight patches (potential negative examples), it explicitly narrows the distance between positive examples and widens the boundaries between positive and negative examples in the feature space, thereby significantly enhancing discriminative ability under limited slide-level annotation conditions.

[0169] (2) Tumor Region Segmentation Module (TransUNet Architecture): TransUNet was chosen as the segmentation model architecture because it combines the global modeling capabilities of Transformer with the fine-grained localization capabilities of UNet (U-shaped network). The system initializes the TransUNet encoder with the pre-trained weights of SPC, freezes the encoder parameters during training, and trains only the decoder part. The training loss function for the segmentation task uses a combination of Dice (overlap coefficient) loss and cross-entropy loss:

[0170] Equation (6)

[0171] The TransUNet encoder employs a hybrid design: a front-end CNN (Convolutional Neural Network) module extracts low-level spatial features, which are then reshaped into sequence input Transformer layers to capture long-range contextual dependencies while preserving fine texture and global organizational structure information. The decoder uses a progressive upsampling strategy, fusing high-resolution feature maps from each encoder layer with corresponding features from the decoder via skip connections.

[0172] Equation (7)

[0173] in This is the output of the layer l decoding. For upsampling operation, and These are the feature maps of the encoder and decoder at layer l, respectively. This is a channel-dimensional concatenation mechanism. This skip connection mechanism allows the model to simultaneously preserve fine local information and global semantics, ensuring that the generated tumor boundaries closely match the pathologist's annotations.

[0174] Specifically, such as Figure 3 As shown, the system is divided into three progressive modules: A, B, and C, which fully demonstrate the entire diagnostic architecture for cervical cancer (CESC) pathological images, from unsupervised pre-training and downstream multi-task diagnosis to domain adaptive generalization validation. Details of each part are as follows:

[0175] Module A is the upstream feature extractor (Unsupervised Pre-training on Large-Scale Unlabeled Data): It takes unlabeled CESC-normal (cervical squamous cell carcinoma normal control) pathological slide images as input and feeds them into the SPC (Feature Extractor) for self-supervised pre-training. Finally, the training converges to obtain a solidified SPC model (SPC pre-trained model), which completes the unsupervised learning of the underlying pathological features based on large-scale unlabeled data.

[0176] Module B is the Downstream Multi-Task Framework: End-to-End Automated Diagnosis. It reuses the SPC feature extractor trained in Module A, extracts features from labeled CESC-normal pathological images, and simultaneously feeds the features into three parallel diagnostic branches: Segmentation Model, which outputs a lesion segmentation visualization; Typing Aggregator, which completes the keratinized / non-keratinized pathological classification; and Grading Aggregator, which realizes the three-level malignancy classification of High, Medium, and Low, forming an end-to-end automated multi-task diagnosis.

[0177] Module C is the Domain Adaptation & Validation module (Ensuring Real-World Generalization and Trustworthiness): It targets the real-world target domain CESC-Target (cervical pathology data) and introduces the RSDA module to complete domain adaptation. After RSDA (Feature Extractor) extracts robust cross-domain features, it connects to the Grading Aggregator module to complete cross-domain validation, ensuring the model's generalization ability and clinical reliability across different data sources. The overall process first mines inherent pathological features through self-supervised pre-training, then transfers them to segmentation, typing, and grading combined multi-task diagnosis, and finally adapts the domain to the real clinical target dataset to solve the problem of pathological data distribution offset, achieving robust, fully automated intelligent diagnosis of cervical cancer.

[0178] in, Figure 4 and Figure 5 This is a performance comparison chart of various MIL models based on baseline UNI features and SPC features according to an embodiment of the present invention.

[0179] in Figure 4This section compares the performance of various models on the CSCC classification task. Specifically, it compares the performance of six multi-instance learning (MIL) models: AB (Attention-based MIL), AMD (Attention Max-Drop MIL), CLAM-SB (Clustering-constrained Attention MultipleInstance Learning - Single Branch), DGR (Diverse Global Representation MIL), DS (Deep Set MIL), and GATE_AB (Gated Attention Aggregation based Attention Bag MIL). The performance of these models is evaluated using two types of feature inputs: Feature Uni (light gray bars) and Feature SPC (dark gray bars). The vertical axes of the four sub-plots represent the following metrics: Accuracy (ACC), Balanced Accuracy (BACC), Macro... The AUC (Area Under ROC Curve) and the fourth chart show the Macro F1-Score. Error bars at the top of all bars represent result fluctuations and standard deviations. The horizontal axis is uniformly labeled with MIL Models, namely six mainstream multi-instance learning algorithms. The legend distinguishes between basic general features (Feature Uni) and SPC-specific pathological features. In terms of overall data trends, the dark gray bars using Feature SPC features have higher values ​​than the light gray bars using Feature Uni in most models and all four indicators. The Macro AUC indicator has the highest overall baseline value, with scores for each model generally close to 0.9, showing minimal performance differences. In the ACC, BACC, and Macro F1-Score indicators, the DGR model shows a relatively significant performance drop under the two types of features. The other models all achieve stable performance improvements after using SPC features. This intuitively verifies that the SPC feature extractor, compared to the general UNI feature, can bring better discrimination accuracy, class balance, global discrimination ability, and comprehensive classification robustness to the whole-slice pathological multi-instance classification task.

[0180] Figure 5This section compares the performance of various models in the CSCC classification task. Specifically, it compares the four classification performance metrics of six multi-instance learning models (MIL Models)—CLAM-SB, AMD, AB, DS, DGR, and GATE_AB—using the general basic feature Feature Uni (light gray bars) and the self-developed pathology-specific feature Feature SPC (dark gray bars). The error bars at the top of the bars represent the standard deviation and data stability of the experimental results. The vertical axis metrics of the four sub-plots, from left to right, are Accuracy (ACC), Balanced Accuracy (BACC), Macro AUC (area under the macro average ROC curve), and Macro F1-Score (macro average F1 score). All metric values ​​are limited to the range of 0 to 1. The overall experimental results clearly show that, across all six models and four evaluation metrics, the dark bars using Feature SPC are consistently taller than the light bars using Feature Uni, and the Macro... The AUC metric has the highest overall performance ceiling, with all model scores approaching 0.85 or higher. The GATE_AB model shows a significant performance improvement after using SPC features across all metrics. The small range of error bars for all groups indicates stable experimental reproducibility. This directly verifies that, compared to general Uni features, SPC features can bring higher overall recognition accuracy, balanced discrimination ability in class imbalance scenarios, global positive and negative sample differentiation ability, and multi-class comprehensive classification accuracy for multi-instance classification tasks of whole pathological slides. This fully demonstrates the effectiveness and superiority of this specialized pathological feature extraction scheme.

[0181] Figure 6 and Figure 7 This example uses a comparison of attention heatmaps from the same WSI image under different feature extractors to demonstrate how SPC-guided attention can more accurately focus on key tumor regions. Specifically, as shown... Figure 6 and Figure 7As shown, this visually demonstrates the differences in feature thermal response and local slice representation between the SPC feature extractor and the UNI universal feature extractor on cervical whole-slice pathological images. The left side shows a large image of a complete cervical pathological slice, with the selected region of interest used as a comparison benchmark. Leading lines connect upwards to the SPC branch and downwards to the UNI branch. The SPC column at the top contains three parts: vertical SPC markers, a feature thermal map of the corresponding lesion area (clear contours, precise focus on key cancerous structures, clear light and dark boundaries, distinguishing lesions from normal tissue areas), and a magnified image of the four-slice local pathological patch on the right, clearly showing core lesions such as keratinized pearls and atypical cell nuclei. The microstructure is analyzed; the UNI column below includes vertical UNI markers, a heatmap of the corresponding UNI region features (overall generalization is blurry, lesion differentiation is weak, and effective lesion area identification is insufficient), and local pathological patches extracted from the four general features on the right. The cellular pathological detail identification is weaker than that of the SPC branch. Overall comparison shows that SPC features can accurately anchor high-risk pathological lesion areas, preserve key diagnostic structures at the cellular level, and focus the thermal response on the core of the lesion, while the UNI general feature response is diffuse and the ability to extract detailed features is insufficient. This intuitively confirms the advantages of SPC in learning features specific to pathological structures, and is more suitable for the fine-grained feature capture needs of cervical cancer pathological diagnosis.

[0182] Figure 8 This is a comparison chart of the model performance of SPC features, UNI features, and ImageNet features on TransUNet in the segmentation task according to this embodiment. Specifically, the chart consists of two sets of bar charts, with the vertical axis uniformly representing the score (range 0-1). The evaluation metric for the left sub-chart is the Dice Score (segment overlap metric), and the evaluation metric for the right sub-chart is the IoU Score (Intersection over Union Ratio, segmentation region overlap metric). The horizontal axis is labeled with the corresponding evaluation metric name. The legend distinguishes three types of feature models: dark bar SPC, medium gray bar Uni, and light gray bar ImageNet, used to compare the accuracy performance of the three feature backbones in the pathological lesion segmentation task. Under the Dice Score metric, SPC scores the highest, Uni scores slightly lower than SPC, and ImageNet scores the lowest. Under the IoU Score metric, SPC scores the highest, Uni scores slightly lower than SPC, and ImageNet scores the lowest. The Score also shows a performance ranking of SPC > Uni > ImageNet. SPC has the best region segmentation overlap and contour matching accuracy, while general ImageNet pre-trained features perform the worst. Uni general pathological features are in the middle. This intuitively proves that the SPC feature extractor, which is self-supervised pre-trained for pathological data, has stronger region contour fitting and pixel-level lesion localization capabilities in lesion segmentation tasks compared to general Uni features and ImageNet natural image pre-trained features. It also leads in segmentation overlap and boundary matching accuracy.

[0183] Figure 9This is a visualization of the tumor region segmentation results according to this embodiment. Specifically, this figure is a 2-row, 4-column visualization comparison of cervical pathology whole slide lesion segmentation. The four horizontal columns are grouped to compare the segmentation effects of different feature schemes, and the two rows show the segmentation results of two independent pathology samples respectively. In each sample, from left to right, the original pathology slide image, the baseline Uni feature segmentation result, the ImageNet general pre-trained feature segmentation result, and the SPC-specific pathology feature segmentation result are shown. The dark area is the lesion positive area identified by the model. It can be seen intuitively that the Uni and ImageNet schemes do not fully cover the lesion area, have blurred edge contours, and are prone to missing small lesions or having missing areas. In contrast, the segmentation results corresponding to the SPC features can completely match the real lesion morphology, with clear lesion boundaries, complete lesion coverage, and accurate restoration of the cancerous tissue range. There are no obvious undersegmentation or oversegmentation problems. It intuitively shows that compared with general features, the SPC pathology-specific features have better lesion localization, edge contour restoration, and small lesion detection capabilities in pixel-level segmentation of whole slide lesions. The segmentation mask has a higher matching degree with the real pathological lesion area.

[0184] Figure 10This is a performance evaluation comparison chart of the domain transfer framework according to this embodiment in multiple test scenarios, including the performance comparison of the complete domain transfer framework, the baseline model, and ablation experiments. Specifically, the chart is divided into two comparison modules, each containing two confusion matrices on the left and a performance bar chart on the right, used to compare the cervical cancer pathology classification effects of RSDA (the method proposed in this paper, dark bars) and basic SPC (light bars) in cross-domain scenarios between the source and target domains. The axes are labeled TrueLabel (true label) and PredictedLabel (predicted label). The color bars of the confusion matrix represent the number of samples, with higher values ​​indicating darker colors. The first group at the top is a three-class classification task (labels 0, 1, 2). The first matrix on the left is the RSDA confusion matrix, with 84, 94, and 9 correctly predicted samples on the diagonal, indicating few misclassified samples. The adjacent matrix is ​​the SPC confusion matrix, with 15 samples misclassified from the second class to the third class, higher than RSDA's 11. The bar chart on the right compares Accuracy (ACC), Macro F1 (macro average F1 score), and Macro In terms of AUC (Area Under the Macro Average), RSDA outperforms SPC in all three metrics. The second group below shows a binary classification task (labels 0 and 1). The RSDA confusion matrix on the left shows 36 and 119 correctly classified samples diagonally, while SPC shows 25 and 117. SPC has a higher number of misclassified positive and negative samples. The bar chart on the right compares ACC and Macro F1, with RSDA still significantly outperforming SPC in both metrics. Overall, the confusion matrix clearly demonstrates that RSDA achieves fewer misclassifications and a higher number of correctly classified samples diagonally after cross-domain adaptation. The bar quantization further validates that the RSDA domain adaptation module effectively compensates for the data distribution offset problem of the source domain SPC in the target domain, improving classification accuracy, balancing classification precision, and global discrimination ability in cross-dataset scenarios. Its generalization performance is significantly better than the original SPC feature scheme.

[0185] Figure 11This is a schematic diagram illustrating the technical details of the domain migration self-supervised framework according to this embodiment. Specifically, this flowchart illustrates a cross-domain knowledge distillation and mask-based self-supervised learning architecture based on a dual-center dataset. First, the Ningbo slice Patch set is input, and the data is fed into a unified data stream, Data. Samples from the Ningbo dataset are first fed into a K-means clustering algorithm to complete feature clustering and filtering. After filtering, the samples are proportionally fused and stored in the Data buffer. The fused data in the buffer is fed in parallel into the Teacher model and the Student model. The Teacher model outputs features to calculate Loss_fd (feature distillation loss). The Student model aligns its features with the Teacher model to constrain Loss_fd, and simultaneously applies a 50% mask to its own input to perform a mask reconstruction task. Based on the mask reconstruction result, Loss_mim (mask imitation loss) is calculated. The two losses are combined to achieve lightweighting of the Student model and cross-domain feature alignment training. The overall process combines four modules: clustering data filtering, cross-domain hybrid buffer, teacher-student knowledge distillation, and mask-based self-supervised learning. This enables multi-center pathology data domain adaptation, knowledge transfer, and robust feature learning, adapting to feature generalization training under different hospital pathology data distributions.

[0186] To address the "domain shift" problem, this embodiment designs a self-supervised domain transfer framework, RSDA (Representative Sample Domain Adaptation), based on the Teacher-Student model. (Refer to...) Figure 3 (C) and Figure 11 .

[0187] (1) Framework architecture: The Teacher model loads mature SPC weights trained on the source domain and freezes them completely. The Student model has the same structure, its parameters are trainable, and the goal is to adapt to the data distribution of the target domain.

[0188] (2) Construction of K-means (K-means clustering) knowledge buffer: To prevent "catastrophic forgetting", SPC is first used to extract all image patch features of the source domain, and K-means clustering algorithm is used to select 50,000 of the most representative image patches to form a "knowledge buffer". During training, the representative samples of the source domain in the buffer are mixed with the unlabeled samples of the target domain at a ratio of 1:9 to retain the knowledge of the source domain and adapt to the distribution of the target domain.

[0189] (3) Composite loss function: The training of the Student model is performed by a composite loss function. Guided by modeling loss from masked image ( ) and characteristic distillation loss ( )composition:

[0190] Equation (8)

[0191] in, The mean squared error (MSE) loss is used to reconstruct image patches that are randomly occluded (75%), forcing the model to learn the intrinsic patterns of the target domain.

[0192] Equation (9)

[0193] Calculate the Student model for characteristic distillation loss. and Teacher Model The mean square error between output features for the same image patch is used to preserve source domain knowledge.

[0194] Equation (10)

[0195] The hyperparameter λ is set to 0.1 to balance the control domain migration robustness with source domain knowledge preservation.

[0196] To evaluate the actual diagnostic capability and clinical suitability of the CSCC-PAG framework in this embodiment, the system was comprehensively validated in a real pathological image environment, with reference to... Figure 10 and Figure 11 .

[0197] The system validation covers the following typical task types and datasets:

[0198] Internal source domain validation: Five-fold cross-validation is used to evaluate the performance of SPC on downstream typing, grading and segmentation tasks.

[0199] Cross-central domain migration validation (235 annotated WSI sheets): Evaluate the effectiveness of the RSDA framework in eliminating domain offsets and restoring model performance.

[0200] Third-party independent out-of-domain validation (TCGA-CESC, 189 WSI sheets): Evaluate the generalization ability of the model weights after RSDA transfer on completely unseen data.

[0201] Evaluation metrics include: for classification tasks (classification and grading), accuracy (ACC), balanced accuracy (BACC), macro average AUC (Area Under ROC Curve), and macro average F1 score are used; for segmentation tasks, intersection-over-union ratio (IoU Score) and Dice score are used.

[0202] Table 1 Model Training Parameter Configuration Table

[0203]

[0204] Table 2 Performance Comparison of Feature Extractors and Domain Transfer Strategies

[0205]

[0206] Note: The performance of the RSDA domain-transferred weights on the TCGA third-party validation set (ACC 0.820) is higher than that of the original SPC weights (ACC 0.751), verifying that the RSDA framework has a "generalization fine-tuning" effect.

[0207] The device includes the following functional modules:

[0208] Data preprocessing module: It receives raw whole-slice images from different medical centers, performs standardized downsampling (up to 20x magnification), extracts foreground regions of tissues, cropps patches (256×256 pixels, no overlap), and performs staining normalization to generate standardized image input data. It also supports the classification management of three dataset modes: unlabeled pre-training data, labeled downstream task data, and unlabeled domain transfer data.

[0209] Domain-Specific Pre-training Module: Employing a self-supervised learning method based on the Vision Transformer architecture, a domain-specific pre-training process is performed on a large-scale CSCC-specific dataset to construct an SPC feature extractor. This module learns CSCC-specific pathological patterns (keratinized beads, intercellular bridges, nuclear atypia, etc.) to obtain highly discriminative deep representations, supporting downstream multi-task recognition.

[0210] Multi-task analysis module: contains three parallel sub-modules: (1) a subtyping aggregator, based on the CLAM-MIL framework, which uses an attention mechanism to automatically focus on subtyping-related regions and outputs WSI-level subtyping classification predictions; (2) a grading aggregator, with an architecture similar to the subtyping aggregator, which predicts the degree of tumor differentiation; and (3) a segmentation model, based on the TransUNet architecture, which uses a combination of Transformer global modeling and UNet fine-tuning to achieve pixel-level tumor region depiction. Each sub-module is fine-tuned with SPC as the backbone network.

[0211] Self-supervised domain transfer module: The core module for realizing unlabeled cross-center model transfer, including: (1) K-means knowledge buffer manager, which is responsible for actively selecting 50,000 representative samples from the source domain feature library to build a buffer; (2) Teacher-Student training manager, which coordinates the joint training process of frozen Teacher model and trainable Student model; (3) Composite loss calculator, which calculates the weighted combination of mask image modeling loss (L_MIM) and feature distillation loss (L_FD).

[0212] The aforementioned device is widely applicable to intelligent analysis tasks of multi-center CSCC whole-slice images, and features accurate feature perception, strong cross-center generalization, and low deployment cost, significantly improving the standardization and intelligence level of CSCC pathological diagnosis.

[0213] In the above embodiments, when performing image analysis on whole-slice images, local pathological features of each image block are extracted after preprocessing the whole-slice image. Then, global pathological features are generated through feature aggregation. Based on the global features, the classification and grading results are output simultaneously. At the same time, the image blocks and local pathological features are combined to obtain fine segmentation results through a segmentation model. The three types of results are then integrated to comprehensively determine the final image analysis result. This avoids the shortcomings of traditional techniques that use a single classification model for analysis, which result in relatively one-sided image analysis results and low accuracy of whole-slice image analysis. This improves the accuracy of whole-slice image analysis.

[0214] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0215] Based on the same inventive concept, this application also provides an automated whole-slice image analysis apparatus for implementing the aforementioned automated whole-slice image analysis method. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more embodiments of the automated whole-slice image analysis apparatus provided below can be found in the limitations of the automated whole-slice image analysis method described above, and will not be repeated here.

[0216] In one exemplary embodiment, such as Figure 12 As shown, an automated image analysis device for whole-slice images is provided, comprising: an image processing module 1201, a feature extraction module 1202, a feature aggregation module 1203, a result determination module 1204, and a result analysis module 1205, wherein:

[0217] Image processing module 1201 is used to acquire the full slice image to be analyzed, preprocess the full slice image, and obtain the preprocessed full slice image.

[0218] The feature extraction module 1202 is used to perform feature extraction processing on image blocks in the preprocessed whole slice image through a pre-trained feature extractor to obtain the local pathological features of the image blocks.

[0219] The feature aggregation module 1203 is used to perform feature aggregation processing on local pathological features to obtain the global pathological features of the preprocessed whole slice image.

[0220] The result determination module 1204 is used to determine the typing and grading results of the preprocessed whole slice image based on global pathological features. It inputs the image block and the corresponding local pathological features into the trained segmentation model to obtain the segmentation results of the preprocessed whole slice image.

[0221] The results analysis module 1205 is used to obtain the image analysis results of the whole slice image based on the classification results, grading results, and segmentation results.

[0222] In an exemplary embodiment, the feature extraction module 1202 is further configured to perform feature extraction processing on image blocks in the preprocessed whole slice image using a pre-trained feature extractor to obtain keratin pearl features, intercellular bridge features, and nuclear atypia features of the image blocks; and obtain local pathological features of the image blocks based on the keratin pearl features, intercellular bridge features, and nuclear atypia features.

[0223] In an exemplary embodiment, the automated whole-slice image analysis device further includes an extractor training module, which is used to acquire sample image blocks in the whole-slice image of the sample, perform partial masking processing on the sample image blocks to obtain masked image blocks; input the masked image blocks into the feature extractor to be trained to obtain the sample local pathological features of the masked image blocks; perform image reconstruction processing on the sample local pathological features through a decoder to obtain reconstructed image blocks; and iteratively train the feature extractor to be trained based on the differences between the sample image blocks and the reconstructed image blocks to obtain a pre-trained feature extractor.

[0224] In an exemplary embodiment, the feature aggregation module 1203 is further configured to perform attention mechanism processing on local pathological features to obtain feature weights corresponding to the local pathological features; and perform feature aggregation processing on the local pathological features according to the feature weights to obtain the global pathological features of the preprocessed whole slice image.

[0225] In an exemplary embodiment, the result determination module 1204 is further configured to perform feature extraction processing on the image block to obtain the texture features of the image block; perform fusion processing on the texture features and the corresponding local pathological features to obtain the fused features of the image block; input the fused features into the trained segmentation model to obtain the sub-segmentation results of the image block; the sub-segmentation results are a probability map used to represent each pixel in the image block as an abnormal pixel; and perform splicing processing on the sub-segmentation results to obtain the segmentation results of the preprocessed full slice image.

[0226] In an exemplary embodiment, the image processing module 1201 is further configured to perform downsampling processing on the full slice image to obtain a processed full slice image; perform threshold filtering processing on the processed full slice image to obtain a filtered full slice image; perform color normalization processing and pixel value normalization processing on the filtered full slice image to obtain a normalized full slice image; and obtain a preprocessed full slice image based on the normalized full slice image.

[0227] Each module in the aforementioned automated whole-slice image analysis device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0228] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 13As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores data such as classification and grading results. The I / O interfaces are used for information exchange between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When executed by the processor, the computer program implements an automated full-slice image analysis method.

[0229] Those skilled in the art will understand that Figure 13 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0230] In one exemplary embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0231] In one exemplary embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above-described method embodiments.

[0232] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0233] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0234] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0235] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. An automated analysis method for whole-slice images, characterized in that, The method includes: Obtain the full slice image to be analyzed, and preprocess the full slice image to obtain the preprocessed full slice image; By using a pre-trained feature extractor, feature extraction is performed on image patches in the preprocessed whole-slice image to obtain the local pathological features of the image patches; The local pathological features are subjected to feature aggregation processing to obtain the global pathological features of the preprocessed whole slice image; Based on the global pathological features, the typing and grading results of the preprocessed whole slice image are determined. The image block and the corresponding local pathological features are input into the trained segmentation model to obtain the segmentation result of the preprocessed whole slice image. Based on the classification results, the grading results, and the segmentation results, the image analysis results of the whole slice image are obtained.

2. The method according to claim 1, characterized in that, The step involves using a pre-trained feature extractor to extract features from image patches in the preprocessed whole-slice image, obtaining the local pathological features of the image patches, including: By using a pre-trained feature extractor, feature extraction is performed on image patches in the preprocessed whole-slice image to obtain keratinized bead features, intercellular bridge features, and nuclear atypia features of the image patches; Based on the keratinized pearl features, the intercellular bridge features, and the nuclear atypia features, the local pathological features of the image patch are obtained.

3. The method according to claim 1, characterized in that, The pre-trained feature extractor is trained in the following manner: Obtain sample image blocks from the full slice image of the sample, and perform partial masking processing on the sample image blocks to obtain masked image blocks; The masked image patch is input into the feature extractor to be trained to obtain the sample local pathological features of the masked image patch; The local pathological features of the sample are reconstructed using a decoder to obtain reconstructed image blocks. Based on the difference between the sample image patch and the reconstructed image patch, the feature extractor to be trained is iteratively trained to obtain the pre-trained feature extractor.

4. The method according to claim 1, characterized in that, The step of performing feature aggregation processing on the local pathological features to obtain the global pathological features of the preprocessed whole-slice image includes: The local pathological features are processed using an attention mechanism to obtain the feature weights corresponding to the local pathological features. According to the aforementioned feature weights, the local pathological features are subjected to feature aggregation processing to obtain the global pathological features of the preprocessed whole-slice image.

5. The method according to claim 1, characterized in that, The step of inputting the image patch and the corresponding local pathological features into the trained segmentation model to obtain the segmentation result of the preprocessed whole-slice image includes: The image patch is subjected to feature extraction processing to obtain the texture features of the image patch; The texture features and the local pathological features corresponding to the image patch are fused to obtain the fused features of the image patch; The fused features are input into the trained segmentation model to obtain the sub-segmentation results of the image patch; the sub-segmentation results are a probability map representing the probability that each pixel in the image patch belongs to an abnormal pixel; The sub-segmentation results are stitched together to obtain the segmentation result of the preprocessed full-slice image.

6. The method according to any one of claims 1 to 5, characterized in that, The preprocessing of the whole slice image to obtain a preprocessed whole slice image includes: The full-slice image is downsampled to obtain the processed full-slice image; The processed full-slice image is subjected to threshold filtering to obtain a filtered full-slice image; The filtered full-slice image is subjected to color normalization and pixel value normalization to obtain a normalized full-slice image. Based on the normalized full slice image, the preprocessed full slice image is obtained.

7. An automated whole-slice image analysis device, characterized in that, The device includes: The image processing module is used to acquire the full slice image to be analyzed, and to preprocess the full slice image to obtain the preprocessed full slice image. The feature extraction module is used to perform feature extraction processing on image patches in the preprocessed whole slice image using a pre-trained feature extractor to obtain the local pathological features of the image patches; The feature aggregation module is used to perform feature aggregation processing on the local pathological features to obtain the global pathological features of the preprocessed whole slice image; The result determination module is used to determine the typing and grading results of the preprocessed whole slice image based on the global pathological features, and input the image block and the corresponding local pathological features into the trained segmentation model to obtain the segmentation result of the preprocessed whole slice image. The results analysis module is used to obtain the image analysis results of the whole slice image based on the classification results, the grading results, and the segmentation results.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images