A multi-scale pathological region automatic segmentation and labeling method and system based on digital pathological images

Abstract

The application discloses a kind of based on multi-scale pathological area automatic segmentation and marking method and system of digital pathological image, comprising: obtaining full field digital pathological image, and parsing multi-resolution pyramid hierarchy structure;At different magnification scales, image is blocked and cut, generate multi-scale image block set, and filter background area;Extract multi-dimensional features to each image block after filtering, and the multi-dimensional features extracted are fused into fusion feature vector;According to the fusion feature vector of each image block, the feature response score of the corresponding image block is calculated;According to the feature response score of each image block, the feature response score is screened according to the preset threshold, and the candidate image block set is constructed, and spatial aggregation processing is carried out;In the coordinate system of original full field digital pathological image, the marking information of each image region entity is generated, and is output in structured data format.The application realizes the efficient multi-scale segmentation of full field digital section, continuous region aggregation and standardized structured output.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and in particular relates to a method and system for automatic segmentation and annotation of multi-scale pathological regions based on digital pathological images. Background Technology

[0002] With the popularization of digital pathology technology, whole-slide images (WSI) have gradually replaced traditional glass slides, becoming the mainstream medium for pathological image storage and analysis. WSI can digitally preserve complete slide information and supports multi-resolution zoom browsing, laying the foundation for image processing, computer vision, and artificial intelligence algorithm research. Currently, automatic analysis and region segmentation methods for WSI images are mainly divided into two categories. One category is traditional image processing and machine learning methods, such as preliminary identification of tissue regions based on color thresholds, morphological operations, and manual features such as texture and color histograms. These methods typically first roughly locate the region of interest at low magnification, and then perform local fine analysis at high magnification. The other category is image patch-level classification methods based on deep learning, which cut the WSI into fixed-size image patches, use convolutional neural networks (such as ResNet and VGG) to classify or score the features of each image patch, and finally stitch the image patch-level results into a full-slide annotation.

[0003] However, the aforementioned existing technologies still have significant limitations when processing full-view digital slices. First, since the resolution of a single digital slice can reach hundreds of millions of pixels, the file size is typically between 1GB and 5GB, making direct processing of the entire image computationally extremely demanding. Although existing methods employ a block-based strategy, they often lack efficient multi-scale collaborative processing mechanisms, resulting in insufficient processing efficiency and failing to meet the demands of rapid processing of large-size images in practical applications. Second, WSI images exhibit distinct visual characteristics at different magnifications: low magnification focuses on macroscopic organizational structures, while high magnification emphasizes cellular-level texture details. Most existing methods only perform analysis at a single magnification, making it difficult to simultaneously capture macroscopic structure and microscopic texture information, thus affecting the accuracy and robustness of image region segmentation. Furthermore, block-by-block processing methods based on image patches generally lack effective spatial aggregation mechanisms after feature scoring, resulting in outputs that are discrete, fragmented sets of image patches, failing to form continuous regional entities with complete boundaries. This is detrimental to subsequent image annotation and visualization output. Furthermore, existing systems rarely provide structured annotation information at the output end, such as region bounding boxes, heatmap overlays, region priority ranking, and comprehensive feature scores. This makes it difficult to standardize and integrate the processing results with mainstream digital pathology image browsing systems (such as QuPath and SlideViewer), thus limiting the promotion and application of related technologies in actual business processes.

[0004] Therefore, there is an urgent need for an image processing method that can efficiently perform multi-scale block processing, multi-dimensional feature extraction, spatial region aggregation, and structured annotation output for full-view digital slices, in order to overcome the shortcomings of existing technologies. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides a method for automatic segmentation and annotation of multi-scale pathological regions based on digital pathological images, comprising: Acquire full-view digital pathology images and analyze the multi-resolution pyramid hierarchical structure of the full-view digital pathology images; Based on a multi-resolution pyramid hierarchical structure, the full-view digital pathological image is segmented into blocks at at least two different magnification scales to generate a multi-scale image block set, and the background area is filtered out. Multidimensional features are extracted from each filtered image patch, and the extracted multidimensional features are fused into a fused feature vector. The multidimensional features include color features, texture features, structural features, and depth features. Calculate the feature response score of the corresponding image patch based on the fused feature vector of each image patch; The feature response scores of each image block are filtered according to the preset first threshold and second threshold, and the image blocks with feature response scores higher than the second threshold are included in the candidate image block set. Spatial aggregation processing is performed on the candidate image block set to merge discrete image blocks into continuous image region entities; Annotation information for each entity in the image region is generated in the coordinate system of the original full-view digital pathology image, and the annotation information is output in a structured data format.

[0006] Optionally, acquiring a full-view digital pathology image and parsing the multi-resolution pyramid hierarchy of the full-view digital pathology image includes: The system reads full-view digital tile files through an interface compatible with open digital tile libraries. Supported file formats include SVS, TIFF, NDPI, SCN, and MRXS. The system parses the multi-resolution pyramid hierarchy structure from the read files and extracts metadata information, including scan magnification, physical pixel size, and tissue boundary information.

[0007] Optionally, the full-view digital pathology image is segmented at at least two different magnification scales to generate a multi-scale image patch set, and background areas are filtered, including: The full-view image is divided into blocks according to low-magnification, medium-magnification, and high-magnification scales, with low-magnification scales ranging from 2x to 5x, medium-magnification scales ranging from 10x, and high-magnification scales ranging from 20x to 40x. Each image block is identified by both spatial coordinates and scale level. For background areas or areas with tissue content below a preset threshold, the Otsu thresholding algorithm is used for filtering, and the filtered multi-scale image block set is output.

[0008] Optionally, extracting color features for each filtered image patch includes: Based on the hematoxylin-eosin staining distribution, the statistics of the hematoxylin color component and the statistics of the eosin color component are extracted. The statistics include the mean, standard deviation and histogram. Extracting texture features includes: calculating contrast statistics, correlation statistics, and energy statistics using the gray-level co-occurrence matrix, and extracting frequency domain power spectrum features using Fourier transform; Extracting structural features includes: extracting glandular structural parameters and cell arrangement features through morphological analysis. The glandular structural parameters include the ratio of glandular lumen area and glandular regularity. The cell arrangement features include nuclear density and cell polarity. Extracting deep features includes: inputting image patches into a pre-trained convolutional neural network, and extracting the high-dimensional semantic embedding vectors output by the convolutional neural network as deep features.

[0009] Optionally, the feature response score for each image patch is calculated based on the fused feature vector of each image patch, including: The fused feature vector is input into a pre-trained image processing scoring model, which outputs a feature response score between zero and one. The closer the score is to one, the more significant the feature response of the corresponding image patch. The image processing scoring model can be any one of a random forest model, a support vector machine model, or a neural network classifier.

[0010] Optionally, the feature response score of each image patch is filtered according to a preset first threshold and a second threshold, and image patches with feature response scores higher than the second threshold are included in the candidate image patch set, including: When the feature response score of an image patch is higher than the first threshold, the image patch is identified as a high-response region and included in the priority processing queue; when the feature response score of an image patch is higher than the second threshold but not higher than the first threshold, the image patch is identified as a medium-response region and included in the secondary candidate queue; wherein the first threshold is greater than the second threshold, and the first threshold and the second threshold can be dynamically configured.

[0011] Optionally, spatial aggregation processing is performed on the candidate image patch set to merge discrete image patches into continuous image region entities, including: A binary mask map is constructed for high-response image patches in the candidate image patch set, and an eight-connected component labeling algorithm is used to identify spatially adjacent image patch sets. For sparsely distributed candidate regions that cannot be merged by connected component analysis, a density-based spatial clustering algorithm is used for clustering. A graph structure is constructed with image patches as nodes and spatial adjacency relationships as edges. Image patches in the same continuous region are merged into independent image region entities by a graph merging algorithm. Each merged image region entity is represented by a rectangular bounding box, and the maximum value, average value, and proportion of high-response image patches of all image patch scores within the image region entity are recorded.

[0012] Optionally, annotation information for each image region entity is generated in the coordinate system of the original full-view digital pathology image, and the annotation information is output in a structured data format, including: The feature response score of each image patch is mapped to a color value and rendered on a thumbnail of the original full-view digital pathology image in a semi-transparent overlay to generate a heatmap. A colored bounding box is drawn for each merged image region entity, and the region number and comprehensive score are marked inside the box. The regions are sorted from high to low according to their comprehensive scores to generate a region index list. The heatmap, the colored bounding boxes, and the region index list are output in a structured data format, which contains the boundary coordinates, area, and feature response score information of each image region entity.

[0013] Optionally, the pre-trained convolutional neural network used to extract deep features is a residual network or a lightweight convolutional neural network.

[0014] This invention also proposes an automatic segmentation and annotation system for multi-scale pathological regions based on digital pathological images, used to implement the method, including: The digital slice acquisition and parsing module is used to acquire full-view digital pathology images and parse the multi-resolution pyramid hierarchical structure of the full-view digital pathology images. A multi-scale slicing processing module, connected to the digital slice acquisition and parsing module, is used to slice the full-view digital pathological image into blocks at at least two different magnification scales according to the multi-resolution pyramid hierarchical structure obtained by parsing, generate a multi-scale image block set, and filter out the background area in the multi-scale image block set. A multidimensional feature extraction module, connected to the multi-scale block processing module, is used to extract multidimensional features from each filtered image block and fuse the extracted multidimensional features into a fused feature vector. The multidimensional features include color features, texture features, structural features, and depth features. The image patch scoring calculation module is connected to the multi-dimensional feature extraction module and is used to calculate the feature response score of the corresponding image patch based on the fused feature vector of each image patch. The candidate region filtering module is connected to the image block scoring calculation module and is used to filter the feature response scores of each image block according to a preset first threshold and a second threshold, and to include image blocks with feature response scores higher than the second threshold into the candidate image block set. The spatial aggregation and region merging module is connected to the candidate region filtering module and is used to perform spatial aggregation processing on the candidate image block set, merging discrete image blocks into continuous image region entities. The annotation and structured output module, connected to the spatial aggregation and region merging module, is used to generate annotation information for each image region entity in the coordinate system of the original full-view digital pathology image, and output the annotation information in a structured data format.

[0015] Compared with the prior art, the present invention has the following advantages and technical effects: First, by acquiring and analyzing the multi-resolution pyramid hierarchical structure of full-view digital pathology images, and by segmenting and filtering background areas at at least two different magnification scales, the computational load of a single ultra-large slice can be significantly reduced, and the processing of invalid background areas can be avoided, thereby greatly improving the processing efficiency of full-view digital slice images and supporting the rapid analysis of GB-level image files.

[0016] Second, by segmenting the image into blocks at multiple magnification scales and extracting fused feature vectors of four dimensions—color, texture, structure, and depth—it is possible to simultaneously capture macroscopic tissue information at low magnification scales and cellular-level microscopic texture information at high magnification scales. This overcomes the problem of insufficient accuracy in single-scale analysis, thereby improving the accuracy and robustness of pathological image region segmentation.

[0017] Third, by calculating the feature response score for each image block and filtering out a set of candidate image blocks according to the preset first and second thresholds, and then performing spatial aggregation processing on the set of candidate image blocks to merge discrete image blocks into continuous image region entities, the fragmentation problem of segmentation results caused by block-by-block analysis can be effectively solved, forming continuous region entities with complete boundaries, which is convenient for subsequent visualization and application.

[0018] Fourth, by generating annotation information for each image region entity in the coordinate system of the original full-view digital pathology image and outputting it in a structured data format, it can provide standardized annotation results such as region bounding boxes, heat maps, and region indexes, and supports seamless interface integration with mainstream digital pathology image browsing systems, thereby improving the feasibility and compatibility of the technical solution of this invention in actual pathology business processes. Attached Figure Description

[0019] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a schematic diagram of the system functional module structure according to an embodiment of the present invention; Figure 2 This is a flowchart illustrating the overall processing of an embodiment of the present invention; Figure 3 This is a flowchart of the processing in Embodiment 2 of the present invention; Figure 4 This is a flowchart of the processing in Embodiment 3 of the present invention; Figure 5 This is a schematic diagram of a multi-scale pyramid segmentation according to an embodiment of the present invention; Figure 6 This is a schematic diagram of the spatial aggregation and region merging process in an embodiment of the present invention; Figure 7 This is a schematic diagram of the annotation and structured output processing procedure in an embodiment of the present invention. Detailed Implementation

[0020] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0021] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0022] Example 1 like Figures 1-2 As shown, this embodiment provides a method for automatic segmentation and annotation of multi-scale pathological regions based on digital pathological images, including the following steps: Acquire full-view digital pathology images and analyze the multi-resolution pyramid hierarchical structure of the full-view digital pathology images; Based on a multi-resolution pyramid hierarchical structure, the full-view digital pathological image is segmented into blocks at at least two different magnification scales to generate a multi-scale image block set, and the background area is filtered out. Multidimensional features are extracted from each filtered image patch, and the extracted multidimensional features are fused into a fused feature vector. The multidimensional features include color features, texture features, structural features, and depth features. Calculate the feature response score of the corresponding image patch based on the fused feature vector of each image patch; The feature response scores of each image block are filtered according to the preset first threshold and second threshold, and the image blocks with feature response scores higher than the second threshold are included in the candidate image block set. Spatial aggregation processing is performed on the candidate image block set to merge discrete image blocks into continuous image region entities; Annotation information for each entity in the image region is generated in the coordinate system of the original full-view digital pathology image, and the annotation information is output in a structured data format.

[0023] As a specific implementation method, the following steps are included: Step 1: Obtaining and parsing digital slices: The system reads WSI files through an interface compatible with the OpenSlide library, supporting the following formats: Aperio SVS (.svs), TIFF / BigTIFF (.tiff, .tif), Hamamatsu NDPI (.ndpi), Leica SCN (.scn), and MiraxMRXS (.mrxs).

[0024] During the reading process, the multi-resolution pyramid hierarchy of WSI is parsed to extract metadata, including scan magnification, physical pixel size, and tissue boundary information. Based on the parsed hierarchy and metadata, basic parameter configurations are provided for subsequent multi-scale slicing processing.

[0025] Step 2: Multi-scale dicing process: like Figure 5 As shown, considering the extremely large size of WSI, the full-view image is divided into fixed-size image patches according to three magnification levels. Each image patch is identified by a three-dimensional index consisting of spatial coordinates (i,j) and scale level k. Low magnification layer (k=1, 2×~5×): captures macroscopic tissue structure information for tissue region identification and background filtering; Medium magnification layer (k=2, 10×): Captures regional image features for preliminary screening of candidate regions; High magnification layer (k=3, 20×~40×): captures fine texture features at the cellular level for fine image region segmentation.

[0026] For background (blank areas) or low-tissue areas, the Otsu thresholding algorithm is used for filtering to avoid invalid calculations, and a multi-scale image patch set Patch(i,j,k) is output. The default image patch size is 512×512 pixels, which can be adjusted according to the actual scene.

[0027] Step 3: Multidimensional Feature Extraction For each image patch, features are extracted from the following four dimensions to form a fused feature vector: Color feature: Based on the HE staining distribution, the color component statistics (mean, standard deviation, histogram) of hematoxylin (blue-violet) and eosin (pink) are extracted to reflect the color distribution characteristics of the image patch; Texture feature: The gray-level co-occurrence matrix (GLCM) is used to calculate texture statistics such as contrast, correlation, and energy, as well as frequency domain features (Fourier transform power spectrum) to characterize the regularity of the microscopic structure of the image; Structure feature: Gland structure parameters (glandular cavity area ratio, glandular regularity) and cell arrangement features (nuclear density, cell polarity) are extracted through morphological analysis for the structural description of the image region; Depth feature: A pre-trained convolutional neural network (CNN, default is ResNet-50) is used as the backbone network for feature extraction to extract the high-dimensional semantic embedding of the image patch and capture complex image patterns that are difficult to describe with handcrafted features; The above four types of features are concatenated into a unified fusion feature vector Feature_Vector(i,j,k) as the input for the subsequent image patch scoring model.

[0028] Step 4: Image patch scoring calculation: Based on the pre-trained image processing model, calculate the feature response score for each image patch: Score(i,j,k)=f(Feature_Vector(i,j,k)) where f(·) is the trained scoring function, and the output value range is [0,1]. The closer to 1, the more significant the image feature response of the image patch. The scoring results are stored in the form of a heat map for use in subsequent steps. Supported model types include Random Forest, Support Vector Machine (SVM), and neural network classifier.

[0029] Step 5: Candidate region screening: Screen the feature response scores of all image patches according to the preset dual-threshold mechanism: Score>T1 (default T1 = 0.80): Determined as a high-response region and included in the priority processing queue; T2<Score≤T1 (default T2 = 0.50): Determined as a medium-response region and included in the secondary candidate queue; Score≤T2: Determined as a low-response region and not included in the candidate range for the time.

[0030] The thresholds T1 and T2 can be dynamically configured according to different image processing requirements. The candidate region set is defined as: RegionSet={R_n|Score(R_n)>T2} Step Six: Spatial Aggregation and Region Merging: like Figure 6 As shown, to integrate discrete candidate image patches into continuous image regions with complete boundaries, the following spatial aggregation operations are performed sequentially: Connectivity analysis: Construct a binary mask map for high-scoring image patches, and use the 8-connected component labeling algorithm (Two-Pass algorithm) to identify spatially adjacent image patch sets; DBSCAN Clustering: For sparsely distributed candidate regions that failed to be effectively merged by connected component analysis, density-based spatial clustering (DBSCAN) is used to complete the clustering by setting the neighborhood radius ε and the minimum number of samples MinPts parameter. Graph merging: Construct a graph structure with image blocks as nodes and spatial adjacency relationships as edges. Use the graph merging algorithm to merge image blocks in the same continuous region into independent image region entities (such as Region_A, Region_B). Each merged image region is represented by a rectangular bounding box, and the maximum, average, and percentage of high-response image patches within that region are recorded as comprehensive feature indicators for that region.

[0031] Step 7: Annotation and Structured Output: like Figure 7 As shown, in the original WSI coordinate system, the following three types of visual annotation information are generated and output in a structured manner: Heatmap generation: The feature response scores of each image patch are mapped to colors (blue→green→red gradient) and rendered on the WSI thumbnail in a semi-transparent overlay to intuitively present the feature response distribution of the entire image; Boundary box annotation: Draw a colored boundary box for each merged image region, with the color encoding the region type, and annotate the region number and overall score inside the box; Regional index list generation: Sort by regional comprehensive score from high to low and generate a numbered list; Structured output: The above annotation information is output in a structured data format, including the boundary coordinates, area and feature score information of each region, and supports interface integration with mainstream digital pathology image browsers (such as QuPath, SlideViewer, etc.).

[0032] On the other hand, this embodiment also provides an automatic segmentation and annotation system for multi-scale pathological regions based on digital pathological images. This system comprises a digital slice acquisition and parsing module, a multi-scale segmentation processing module, a multi-dimensional feature extraction module, an image block scoring calculation module, a candidate region screening module, a spatial aggregation and region merging module, and an annotation and structured output module. These modules work together to complete the entire process of segmentation and annotation of the full-view digital slice image. The digital slice acquisition and parsing module is used to read the full-view digital slice file through an interface compatible with the OpenSlide library, parse the multi-resolution pyramid hierarchical structure of the full-view digital slice file, and extract metadata information, including scanning magnification, physical pixel size and tissue boundary information, to provide basic parameter configuration for subsequent multi-scale slicing processing; The multi-scale slicing processing module, connected to the digital slice acquisition and parsing module, is used to slice the full-field digital pathological image into blocks according to low-magnification, medium-magnification, and high-magnification scales based on the parsed multi-resolution pyramid hierarchical structure, generating a multi-scale image block set Patch(i, j, k), and filtering the background area through the Otsu thresholding algorithm, wherein the low-magnification scale is two to five times, the medium-magnification scale is ten times, and the high-magnification scale is twenty to forty times. The multi-dimensional feature extraction module, connected to the multi-scale block processing module, is used to extract features from each filtered image block from four dimensions: color, texture, structure, and depth, forming a fused feature vector Feature_Vector(i,j, k); The image patch scoring calculation module is connected to the multidimensional feature extraction module. It is used to calculate the feature response score Score(i,j,k) for each image patch based on the pre-trained image processing model. The output value range is [0,1], and the scoring results are stored in the form of a heatmap. The candidate region filtering module is connected to the image block scoring calculation module and is used to filter the feature response scores of all image blocks according to a preset dual threshold mechanism, and output a set of high response and medium response candidate image blocks, RegionSet. The dual threshold mechanism includes a first threshold T1 and a second threshold T2. The spatial aggregation and region merging module, connected to the candidate region filtering module, is used to sequentially perform connected component analysis, DBSCAN clustering, and graph structure merging on candidate image blocks, integrating discrete image blocks into continuous image region entities with complete boundaries; the spatial aggregation and region merging module includes a connected component analysis unit, a DBSCAN clustering unit, and a graph structure merging unit; The annotation and structured output module, connected to the spatial aggregation and region merging module, is used to generate heatmap overlays, bounding box annotations, and region index lists in the coordinate system of the original full-view digital pathology image, and output them in a structured data format. The structured data format supports interface integration with mainstream digital pathology image browsers.

[0033] Compared to existing technologies, the multi-scale pathological region automatic segmentation and annotation method and system based on digital pathological images provided in this embodiment significantly improves the processing efficiency of ultra-large images by performing efficient multi-scale segmentation and background filtering on full-view digital slices; it effectively improves the accuracy and robustness of image region segmentation by extracting features at multiple scales, taking into account both macroscopic tissue structure and microscopic texture information; it solves the problem of fragmented segmentation results by integrating discrete image blocks into continuous regions through spatial aggregation algorithms; and it generates structured annotation output containing heatmaps, bounding boxes, and region index information, supporting seamless integration with mainstream image browsing systems. Furthermore, it reduces the dependence on large amounts of fine-grained annotation data throughout the processing, facilitating rapid deployment and application.

[0034] Example 2 like Figure 3 As shown, this embodiment is designed for standard image processing deployment scenarios. It uses ResNet-50 as the deep feature extraction backbone network and combines it with a random forest scoring model to achieve basic image region segmentation and annotation functions with relatively low computational resources. The specific implementation steps are as follows: Step 1: Digital Slice Acquisition and Parsing. WSI files are read using an interface compatible with the OpenSlide library. The multi-resolution pyramid hierarchy is parsed, and metadata information such as scan magnification, physical pixel size, and tissue boundaries is extracted.

[0035] Step 2: Multi-scale block processing. The full-view image is divided into blocks at three scales: low magnification (2×~5×), medium magnification (10×), and high magnification (20×). The default image block size is 512×512 pixels. Low tissue content areas are filtered by the Otsu thresholding algorithm to generate a multi-scale image block set Patch(i,j,k).

[0036] Step 3: Multidimensional feature extraction. For each image patch, extract color features (HE staining color component statistics), texture features (GLCM statistics and Fourier power spectrum), structural features (gland structure parameters and cell arrangement features), and depth features (ResNet-50 high-dimensional semantic embedding) in sequence, and concatenate them into a unified fused feature vector Feature_Vector(i,j,k).

[0037] Step 4: Image patch score calculation. Input the fused feature vector into the pre-trained random forest scoring model, calculate the feature response score Score(i,j,k) for each image patch, with an output value range of [0,1], and store the scoring results in the form of a heatmap.

[0038] Step 5: Candidate Region Filtering. Image patch scores are filtered based on a dual threshold mechanism (T1=0.80, T2=0.50), and a candidate image patch set, RegionSet, is output.

[0039] Step 6: Spatial aggregation and region merging. For candidate image patches, sequentially perform 8-connected component labeling (Two-Pass algorithm), DBSCAN clustering (ε and MinPts parameters configured with default values), and graph merging to generate continuous image region entities. Each region is represented by a rectangular bounding box, and comprehensive feature indices are recorded.

[0040] Step 7: Annotation and Structured Output. Generate a full-image heatmap overlay and bounding boxes for high-response regions. Output the boundary coordinates, area, and scoring information of each region in JSON format. Supports interface integration with mainstream digital pathology image browsers.

[0041] Example 3 like Figure 4 As shown, this embodiment introduces a more advanced feature extraction architecture based on Embodiment 2. It employs VisionTransformer (ViT-B / 16) as the deep feature backbone network, introduces a Feature Pyramid Network (FPN) to fuse multi-scale features, and strengthens the response of key regions through self-attention and cross-attention mechanisms to significantly improve the segmentation ability of small image regions and complex texture structures. The specific implementation steps are as follows: Step 1: Digital Slice Acquisition and Parsing. WSI files are read using an interface compatible with the OpenSlide library. The multi-resolution pyramid hierarchy is parsed, and metadata information such as scan magnification, physical pixel size, and tissue boundaries is extracted.

[0042] Step 2: Multi-scale block processing. The full-view image is divided into blocks at three scales: low magnification (2×~5×), medium magnification (10×), and high magnification (20×~40×). The image block size of the low-magnification and medium-magnification layers is 512×512 pixels, and the image block size of the high-magnification layer is adjusted to 256×256 pixels. Low tissue content regions are filtered out using the Otsu thresholding algorithm to generate a multi-scale image block set Patch(i,j,k).

[0043] Step 3: Multidimensional Feature Extraction. For each image patch, color features (HE staining color component statistics), texture features (GLCM statistics and Fourier power spectrum), and structural features (glandular structural parameters and cell arrangement features) are extracted. For depth feature extraction, VisionTransformer (ViT-B / 16) replaces ResNet-50 as the backbone network, introducing a global attention mechanism. Simultaneously, a Feature Pyramid Network (FPN) is introduced to fuse low-magnification, medium-magnification, and high-magnification layer features, and self-attention and cross-attention mechanisms are used to enhance the feature response of key regions. All the above features are then concatenated into a unified fused feature vector, Feature_Vector(i,j,k).

[0044] Step 4: Image patch score calculation. Input the fused feature vector into the neural network classifier to calculate the feature response score Score(i,j,k) and score confidence interval estimate for each image patch. The output value range is [0,1], and the score results are stored in the form of a heatmap.

[0045] Step 5: Candidate Region Filtering. Image blocks are filtered based on a dual threshold mechanism (T1=0.80, T2=0.50), and a candidate image block set, RegionSet, is output. The thresholds T1 and T2 can be dynamically adjusted according to actual needs.

[0046] Step 6: Spatial aggregation and region merging. For candidate image patches, sequentially perform 8-connected component labeling (Two-Pass algorithm), DBSCAN clustering (ε and MinPts parameters configured with default values), and graph merging to generate continuous image region entities. Each region is represented by a rectangular bounding box, and comprehensive feature indices are recorded.

[0047] Step 7: Annotation and Structured Output. Generate multi-scale heatmaps, hierarchical bounding boxes, and scoring confidence interval estimates. Output the boundary coordinates, area, and scoring information of each region in JSON format. Supports interface integration with mainstream digital pathology image browsers.

[0048] Example 4 This embodiment addresses edge device deployment scenarios with limited computing resources. It employs a lightweight MobileNetV3 as the deep feature extraction backbone network, compressing the number of parameters to one-tenth of ResNet-50. Combined with a lightweight random forest scoring model and a dynamic thresholding strategy, it achieves low-latency real-time image processing. The specific implementation steps are as follows: Step 1: Digital Slice Acquisition and Parsing. WSI files are read using an interface compatible with the OpenSlide library. The multi-resolution pyramid hierarchy is parsed, and metadata information such as scan magnification, physical pixel size, and tissue boundaries is extracted.

[0049] Step 2: Multi-scale block processing. The full-view image is divided into blocks at three scales: low magnification (2×~5×), medium magnification (10×), and high magnification (20×). The default image block size is 512×512 pixels. Low tissue content areas are filtered by the Otsu thresholding algorithm to generate a multi-scale image block set Patch(i,j,k).

[0050] Step 3: Multidimensional feature extraction. For each image patch, extract color features (HE staining color component statistics), texture features (GLCM statistics and Fourier power spectrum), and structural features (glandular structural parameters and cell arrangement features). For depth feature extraction, MobileNetV3 replaces ResNet-50 as the backbone network to extract high-dimensional semantic embeddings for the image patches. All the above features are then concatenated into a unified fused feature vector, Feature_Vector(i,j,k).

[0051] Step 4: Image patch score calculation. Input the fused feature vector into the lightweight random forest scoring model to calculate the feature response score Score(i,j,k) for each image patch. The output value range is [0,1], and the scoring results are stored in the form of a heatmap.

[0052] Step 5: Candidate Region Filtering. Image blocks are filtered based on a dual threshold mechanism (T1=0.80, T2=0.50), and the number of candidate regions is automatically adjusted according to the overall feature response density of the image (minimum 3, maximum 20), outputting a candidate image block set RegionSet; thresholds T1 and T2 can be dynamically configured.

[0053] Step 6: Spatial aggregation and region merging. For candidate image patches, sequentially perform 8-connected component labeling (Two-Pass algorithm), DBSCAN clustering (ε and MinPts parameters configured with default values), and graph merging to generate continuous image region entities. Each region is represented by a rectangular bounding box, and comprehensive feature indices are recorded.

[0054] Step 7: Annotation and Structured Output. Generate a simplified heatmap and bounding box list, outputting the boundary coordinates, area, and score information of each region in JSON format, suitable for edge inference environments with limited computing resources.

[0055] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for automatic segmentation and annotation of multi-scale pathological regions based on digital pathological images, characterized in that, include: Acquire full-view digital pathology images and analyze the multi-resolution pyramid hierarchical structure of the full-view digital pathology images; Based on a multi-resolution pyramid hierarchical structure, the full-view digital pathological image is segmented into blocks at at least two different magnification scales to generate a multi-scale image block set, and the background area is filtered out. Multidimensional features are extracted from each filtered image patch, and the extracted multidimensional features are fused into a fused feature vector. The multidimensional features include color features, texture features, structural features, and depth features. Calculate the feature response score of the corresponding image patch based on the fused feature vector of each image patch; The feature response scores of each image block are filtered according to the preset first threshold and second threshold, and the image blocks with feature response scores higher than the second threshold are included in the candidate image block set. Spatial aggregation processing is performed on the candidate image block set to merge discrete image blocks into continuous image region entities; Annotation information for each entity in the image region is generated in the coordinate system of the original full-view digital pathology image, and the annotation information is output in a structured data format.

2. The method according to claim 1, characterized in that, Acquiring full-view digital pathology images and analyzing the multi-resolution pyramid hierarchical structure of the full-view digital pathology images includes: The system reads full-view digital tile files through an interface compatible with open digital tile libraries. Supported file formats include SVS, TIFF, NDPI, SCN, and MRXS. The system parses the multi-resolution pyramid hierarchy structure from the read files and extracts metadata information, including scan magnification, physical pixel size, and tissue boundary information.

3. The method according to claim 1, characterized in that, The full-view digital pathology image is segmented at at least two different magnification scales to generate a multi-scale image patch set, and the background region is filtered, including: The full-view image is divided into blocks according to low-magnification, medium-magnification, and high-magnification scales, with low-magnification scales ranging from two to five times, medium-magnification scales ranging from ten times, and high-magnification scales ranging from twenty to forty times. Each image block is identified by both spatial coordinates and scale level. For background areas or areas with tissue content below a preset threshold, the Otsu thresholding algorithm is used for filtering, and the filtered multi-scale image block set is output.

4. The method according to claim 1, characterized in that, Extracting color features for each filtered image patch includes: Based on the hematoxylin-eosin staining distribution, the statistics of the hematoxylin color component and the statistics of the eosin color component are extracted. The statistics include the mean, standard deviation and histogram. Extracting texture features includes: calculating contrast statistics, correlation statistics, and energy statistics using the gray-level co-occurrence matrix, and extracting frequency domain power spectrum features using Fourier transform; Extracting structural features includes: extracting glandular structural parameters and cell arrangement features through morphological analysis. The glandular structural parameters include the ratio of glandular lumen area and glandular regularity. The cell arrangement features include nuclear density and cell polarity. Extracting deep features includes: inputting image patches into a pre-trained convolutional neural network, and extracting the high-dimensional semantic embedding vectors output by the convolutional neural network as deep features.

5. The method according to claim 1, characterized in that, The feature response score for each image patch is calculated based on the fused feature vector of that patch, including: The fused feature vector is input into a pre-trained image processing scoring model, which outputs a feature response score between zero and one. The closer the score is to one, the more significant the feature response of the corresponding image patch. The image processing scoring model can be any one of a random forest model, a support vector machine model, or a neural network classifier.

6. The method according to claim 1, characterized in that, Based on a preset first threshold and a second threshold, each image patch is filtered by its feature response score. Image patches with feature response scores higher than the second threshold are included in the candidate image patch set, including: When the feature response score of an image patch is higher than the first threshold, the image patch is identified as a high-response region and included in the priority processing queue; when the feature response score of an image patch is higher than the second threshold but not higher than the first threshold, the image patch is identified as a medium-response region and included in the secondary candidate queue; wherein the first threshold is greater than the second threshold, and the first threshold and the second threshold can be dynamically configured.

7. The method according to claim 1, characterized in that, Spatial aggregation processing is performed on the candidate image patch set to merge discrete image patches into continuous image region entities, including: A binary mask map is constructed for high-response image patches in the candidate image patch set, and an eight-connected component labeling algorithm is used to identify spatially adjacent image patch sets. For sparsely distributed candidate regions that cannot be merged by connected component analysis, a density-based spatial clustering algorithm is used for clustering. A graph structure is constructed with image patches as nodes and spatial adjacency relationships as edges. Image patches in the same continuous region are merged into independent image region entities by a graph merging algorithm. Each merged image region entity is represented by a rectangular bounding box, and the maximum value, average value, and proportion of high-response image patches of all image patch scores within the image region entity are recorded.

8. The method according to claim 1, characterized in that, Annotation information for each entity in the original full-view digital pathology image is generated in the coordinate system, and the annotation information is output in a structured data format, including: The feature response score of each image patch is mapped to a color value and rendered on a thumbnail of the original full-view digital pathology image in a semi-transparent overlay to generate a heatmap. A colored bounding box is drawn for each merged image region entity, and the region number and comprehensive score are marked inside the box. The regions are sorted from high to low according to their comprehensive scores to generate a region index list. The heatmap, the colored bounding boxes, and the region index list are output in a structured data format, which contains the boundary coordinates, area, and feature response score information of each image region entity.

9. The method according to claim 4, characterized in that, The pre-trained convolutional neural network used to extract deep features is a residual network or a lightweight convolutional neural network.

10. A multi-scale pathological region automatic segmentation and annotation system based on digital pathological images, characterized in that, For implementing the method according to any one of claims 1-9, comprising: The digital slice acquisition and parsing module is used to acquire full-view digital pathology images and parse the multi-resolution pyramid hierarchical structure of the full-view digital pathology images. A multi-scale slicing processing module, connected to the digital slice acquisition and parsing module, is used to slice the full-view digital pathological image into blocks at at least two different magnification scales according to the multi-resolution pyramid hierarchical structure obtained by parsing, generate a multi-scale image block set, and filter out the background area in the multi-scale image block set. A multidimensional feature extraction module, connected to the multi-scale block processing module, is used to extract multidimensional features from each filtered image block and fuse the extracted multidimensional features into a fused feature vector. The multidimensional features include color features, texture features, structural features, and depth features. The image patch scoring calculation module is connected to the multi-dimensional feature extraction module and is used to calculate the feature response score of the corresponding image patch based on the fused feature vector of each image patch. The candidate region filtering module is connected to the image block scoring calculation module and is used to filter the feature response scores of each image block according to a preset first threshold and a second threshold, and to include image blocks with feature response scores higher than the second threshold into the candidate image block set. The spatial aggregation and region merging module is connected to the candidate region filtering module and is used to perform spatial aggregation processing on the candidate image block set, merging discrete image blocks into continuous image region entities. The annotation and structured output module, connected to the spatial aggregation and region merging module, is used to generate annotation information for each image region entity in the coordinate system of the original full-view digital pathology image, and output the annotation information in a structured data format.

Images