Pathological image segmentation method and device based on category label and readable storage medium thereof

Abstract

The present invention proposes a pathology image segmentation method, device, and readable storage medium based on category labels. The method includes segmenting a full-field digital pathology (WSI) image into blocks, then extracting a feature map containing local texture and global semantics using ResNet, ViT, or a large pathology-specific pre-trained model, UNI; calculating the contribution of the image blocks to the classification result through an attention mechanism, and combining gradient analysis to extract the actual impact of features on the classification decision; adopting a sorting weighted strategy to fuse the two weights, and generating a lesion region mask through Gaussian blurring and Otsu's threshold segmentation. The mask is finally superimposed on the original image to output an interpretable segmentation result. The present invention only requires slice-level labels to achieve weakly supervised segmentation, significantly reducing labeling costs. The complementary fusion of attention and gradient information improves positioning accuracy, generates intuitive visualizations of lesion regions, enhances model transparency and clinical practicality, and is suitable for a variety of pathology image analysis scenarios.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and medical image processing, and in particular to a method and device for pathological image segmentation based on category labels and a readable storage medium thereof. Background Art

[0002] In the field of AI-assisted clinical diagnosis of pathology, deep neural networks have been able to achieve highly accurate pathology slide classification through strongly supervised learning. However, due to the "black box" nature of the model, clinicians find it difficult to understand the basis for their decisions, resulting in a lack of trust in diagnostic recommendations. Doctors not only need classification results, but also need the model to locate the lesion area and provide explainable visualization to verify the diagnostic logic and guide clinical decision-making. However, instance-level annotation of pathology slides (i.e., accurately labeling each lesion area) is time-consuming and labor-intensive and relies on professional pathologists. Such finely labeled data is generally lacking in practical applications. Existing technologies mainly rely on weakly supervised learning methods with slice-level labels to attempt to locate the lesion area, but there are two core problems:

[0003] 1. A single evaluation mechanism (such as relying solely on attention or gradient analysis) is unable to fully identify key lesion features, resulting in insufficient localization accuracy.

[0004] 2. Traditional weakly supervised algorithms have low sensitivity to low-saliency lesion areas in complex backgrounds of pathological images, and the generated positioning results have poor visualization effects, which cannot meet the strict clinical requirements for interpretability.

[0005] Therefore, how to achieve high-precision and explainable lesion area segmentation under weak supervision conditions that only require slice-level labels has become a key bottleneck in the implementation of pathology AI technology. Summary of the Invention

[0006] The embodiments of the present invention provide a pathological image segmentation method, device and readable storage medium based on category labels. They address the problems of weakly supervised pathological image segmentation methods in current technologies, such as reliance on a single evaluation mechanism or lack of effective feature fusion, resulting in insufficient accuracy in lesion area positioning and poor model interpretability, making it difficult to meet the clinical diagnosis requirements for transparent decision-making basis.

[0007] The core technology of this invention is to achieve accurate positioning and visual interpretation of the lesion area under the condition of only slice-level labels by fusing the regional importance weight calculated by the attention mechanism and the feature contribution extracted by gradient analysis, combining the sorting weighted strategy and Otsu threshold segmentation.

[0008] In a first aspect, the present invention provides a pathological image segmentation method based on category labels, the method comprising the following steps:

[0009] receiving a full-field digital pathology image and dividing it into a plurality of image blocks;

[0010] The local texture features and global semantic features of each image block are extracted through a pre-trained feature extractor to generate a feature map;

[0011] Based on the feature map, an attention score of each image block is calculated through an attention mechanism, where the attention score is used to measure the contribution of the image block to the classification result;

[0012] Based on the feature map, the gradient weight of each image block is calculated by gradient analysis, and the gradient weight is used to measure the actual impact of the image block feature on the classification result;

[0013] The attention score and gradient weight are fused through the sorting normalization strategy to generate the fusion weight;

[0014] The fusion weights are post-processed and the lesion area positioning results are output.

[0015] Furthermore, the image feature extractor includes ResNet, Vision Transformer or a pathology-specific pre-trained large model UNI.

[0016] Furthermore, the attention score is calculated through the attention mechanism, which includes:

[0017] An attention network based on multi-instance learning is adopted, with slice-level category labels as supervisory signals. The feature vectors of each image block are nonlinearly transformed and normalized through a learnable parameter matrix to obtain the attention score. The image attention score meets the normalization condition.

[0018] Furthermore, the gradient weight is calculated through gradient analysis, specifically including:

[0019] Based on the predicted score of the target category, the partial derivative of the feature dimension of the image block is calculated and average pooling is performed to obtain the gradient weight of the feature on the classification result.

[0020] Furthermore, the attention scores and gradient weights are normalized, specifically including:

[0021] The attention scores are normalized by min-max; the gradient weights are normalized according to the numerical order, and the gradient weights are converted into relative ranking weights in all image blocks.

[0022] Furthermore, the image fusion weight is obtained by multiplying the normalized attention score with the normalized gradient weight.

[0023] Furthermore, image post-processing includes:

[0024] Perform Gaussian blur denoising on the fusion weights;

[0025] Adaptively calculate the binarization threshold based on Otsu's method and generate a black and white mask;

[0026] The black and white mask is superimposed on the original pathological image to visualize the lesion area.

[0027] In a second aspect, the present invention provides a pathological image segmentation device based on category labels, comprising:

[0028] A block division module, used for dividing the full-field digital pathology image into multiple image blocks;

[0029] Feature extraction module, used to extract feature maps containing local texture and global semantic information from image blocks. Image feature extraction modules include ResNet, Vision Transformer, or the pathology-specific pre-trained large model UNI;

[0030] Weakly supervised localization module, including:

[0031] An attention calculation unit, which is used to calculate the attention score of each image block through the attention mechanism based on the image feature map;

[0032] A gradient calculation unit, configured to calculate the gradient weight of each image block through gradient analysis based on the image feature map;

[0033] The fusion unit is used to normalize the image attention score and gradient weight and fuse them to obtain the fusion weight;

[0034] The post-processing module is used to perform Gaussian blur processing on the image fusion weights, generate a lesion area mask based on the Otsu method, overlay it on the original pathological image, and output the segmentation result.

[0035] In a third aspect, the present invention provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the above-mentioned pathological image segmentation method based on category labels.

[0036] In a fourth aspect, the present invention provides a readable storage medium storing a computer program, wherein the computer program includes a program code for controlling a process to execute a process, wherein the process includes the above-mentioned pathological image segmentation method based on category labels.

[0037] The main contributions and innovations of the present invention are as follows:

[0038] 1. Reduced labeling costs: This technology breaks the reliance of strongly supervised learning on instance-level fine labeling and only requires slice-level labels for training, significantly reducing the time and labor costs of pathology data labeling and adapting to the actual needs of clinical pathology data labeling.

[0039] 2. Improved localization accuracy: Through dual-path evaluation and weighted fusion of the attention mechanism and gradient analysis, the model takes into account both the areas of "attention" of the model and the key features that actually affect classification, effectively identifying low-salience lesion areas, and significantly improving localization accuracy compared to a single mechanism.

[0040] 3. Enhance model interpretability: Generate a visual lesion area mask and overlay it onto the original pathology image, providing doctors with intuitive decision-making basis, solving the "black box" problem, and improving clinical trust in AI diagnostic recommendations.

[0041] 4. Optimize visualization effects: Gaussian blur is used to weaken the boundaries of image blocks, and Otsu's method of adaptive threshold segmentation is combined to form a continuous and accurate lesion area mask. The segmentation results are more in line with the visual habits of pathological diagnosis, significantly improving the efficiency of doctors' reading of images.

[0042] 5. Improved generalization capabilities: Compatible with multiple feature extraction architectures such as ResNet, ViT, and the pathology-specific pre-trained large model UNI, it supports analysis of pathology sections of different resolutions and staining types, adapts to multiple scenarios such as tumor typing and inflammation detection, and enhances the universality of the algorithm.

[0043] The above effects are achieved through the coordinated efforts of core innovations such as "gradient-attention fusion strategy", "ranking weighted normalization", and "Otsu threshold segmentation". From a technical perspective, they systematically solve the defects of existing weak supervision methods, combining technological advancement with clinical practicality.

[0044] The details of one or more embodiments of the invention are set forth in the accompanying drawings and the description below so that other features, objects, and advantages of the invention are more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0046] Figure 1 is a flow chart of a pathological image segmentation method based on category labels according to an embodiment of the present invention;

[0047] Figure 2 is a flow chart for highlighting key areas using the Otsu method according to an embodiment of the present invention;

[0048] Figure 3 is an algorithm effect evaluation diagram according to an embodiment of the present invention;

[0049] Figure 4 FIG. 4 is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0050] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The implementations described in the following exemplary embodiments are not intended to represent all implementations consistent with one or more embodiments of this specification. Rather, they are merely examples of apparatuses and methods consistent with certain aspects of one or more embodiments of this specification, as detailed in the appended claims.

[0051] It should be noted that in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the method may include more or fewer steps than those described in this specification. In addition, a single step described in this specification may be broken down into multiple steps for description in other embodiments, and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0052] Two core pain points in the field of pathology AI:

[0053] Lack of interpretability: While existing models have high classification accuracy, they lack the ability to locate lesions, making it difficult for doctors to trust these "black box" decisions.

[0054] High labeling cost: Pathology slide labeling requires manual delineation by professional physicians, and instance-level labeling is almost impossible.

[0055] Based on this, the present invention solves the problems existing in the prior art based on an innovative weakly supervised positioning algorithm.

[0056] Example 1

[0057] The present invention aims to propose a pathological image segmentation method based on category labels, specifically, referring to Figure 1 , the method comprising:

[0058] S1, receiving a full-field digital pathology (WSI) image and dividing it into multiple image blocks;

[0059] S2. Extract local texture features and global semantic features of each image block through a pre-trained feature extractor to generate a feature map;

[0060] In this embodiment, the feature extractor can use different deep neural network architectures for feature extraction, including three mainstream feature extractors: ResNet, VisionTransformer (ViT), and the pathology-specific pre-trained large model UNI.

[0061] Among them, the ResNet module effectively captures local cell morphological features through its residual connection structure; the ViT module uses the global self-attention mechanism to model long-distance histological structure associations; the UNI pathology large model is pre-trained based on one million H&E-stained pathology slides. The main architecture of the large model is ViT-Large with approximately 300 million parameters, and it is pre-trained using the DINOv2 self-supervised learning strategy. It can perform a variety of downstream pathology image analysis tasks with superior performance.

[0062] S3. Based on the feature map, the attention score of each image block is calculated through the attention mechanism. The attention score is used to measure the contribution of the image block to the classification result.

[0063] In this embodiment, this step innovatively proposes a weakly supervised localization algorithm based on category labels, which only requires the category labels of the slices to generate the localization information of the lesion area. The algorithm fuses gradient information and attention scores to achieve accurate localization of the lesion area, and adopts a dual-path mechanism to evaluate the lesion area: on the one hand, the importance weight of each area is calculated through the attention network, and on the other hand, the degree of influence of the features on the classification results is analyzed by back-propagation gradient. By fusing these two evaluation indicators, the algorithm can more comprehensively identify key pathological feature areas. The details are as follows:

[0064] In this step, the lesion localization algorithm based on category labels first uses an attention mechanism to evaluate the contribution of each local region to the classification result. Specifically, the input full-field digital pathology image is segmented into multiple patches, each of which is considered an instance. Under weak supervision with only slice-level labels, this system uses an attention-based multi-instance learning method to guide the model in learning the importance weight of each instance using category labels.

[0065] The feature extraction module (feature extractor) described above generates feature representations for each patch. These features contain local texture and global context information. Subsequently, the attention network uses the information of the category label to calculate the attention score (also called weight) of each instance. k It is used to measure the contribution of the small block to the final classification decision. Specifically, the attention score a k The calculation process is as follows:

[0066]

[0067] Where K is the number of instances, and is the learnable parameter matrix, is the feature vector of the kth instance, represents the hyperbolic tangent nonlinear activation function applied element by element. k The design of ensures that it satisfies the normalization condition (i.e. ).

[0068] For a WSI , which contains the feature vectors of K instances, then the global feature representation z at the WSI level can be formalized as:

[0069]

[0070] Finally, by constructing a supervised loss function on the WSI-level global feature representation z and its corresponding slice-level true label, end-to-end weakly supervised learning is achieved. Through the back-propagation algorithm, the attention score a of each instance can be obtained. k , that is, the importance weight of each region is calculated through the attention mechanism, that is, the attention score of each image block.

[0071] S4. Based on the feature map, the gradient weight of each image block is calculated through gradient analysis. The gradient weight is used to measure the actual impact of the image block feature on the classification result.

[0072] In this embodiment, this step is to obtain the classification score After that (comes from the classification prediction results of the model for the full-field digital pathology (WSI) image. Specifically, after the feature extraction module extracts the feature map of the image block through ResNet, ViT or the pathology-specific pre-trained large model UNI, the multi-instance learning method based on the attention mechanism will perform weighted aggregation on the features of each image block to form a global feature representation z at the WSI level. This global feature z is input into the classifier and calculated to generate a prediction score for the target category c, which is the classification score here. ), inspired by Grad-CAM, this paper uses a gradient-based method to calculate the contribution of features. The specific formula is as follows:

[0073]

[0074] in, Represents the gradient (feature contribution weight) of the i-th instance relative to the classification result c; Represents the value of the dth dimension of the i-th instance; represents the prediction score of target category c; D is the instance feature dimension; is the classification score The partial derivative of the feature dimension reflects the effect of this dimension on the classification result. direction and intensity of impact. right By taking the partial derivative, we can get the gradient of the target category c score relative to the d-th dimension of the i-th feature. Then, by averaging the feature gradients in the feature dimension, we can get the gradient of the feature pair classification result.

[0075] This gradient-based method can capture the importance weight of each instance feature to the final classification result. , that is, the importance weight of each region is calculated by feature gradient.

[0076] Grad-CAM (Gradient-weighted Class Activation Mapping) is a visualization technique for explaining the decision-making process of deep neural networks, proposed by R. Selvarajoo et al. in 2017. Its core concept is to calculate the gradient of the target class with respect to the convolutional layer feature map, generating a heatmap that visually displays the regions in the image that contribute significantly to the classification result. This approach addresses the "black box" problem of deep learning models and improves decision interpretability. Rather than directly using the convolutional layer feature maps (such as the spatial features in traditional CNNs), this method performs gradient analysis on the image patch feature vectors output by Transformer architectures such as ViT / UNI, adapting the feature representation of pathological images after block processing. This is then fused with the weights generated by the attention mechanism to achieve lesion localization under weak supervision. By combining the gradient analysis principle of Grad-CAM with the attention mechanism of multi-instance learning, this method constructs a dual-pathway evaluation system, achieving high-precision and interpretable lesion segmentation using only slice-level labels, representing a significant improvement over traditional weakly supervised learning methods.

[0077] S5. Fusion of attention scores and gradient weights through sorting normalization strategy to generate fusion weights;

[0078] In this embodiment, after obtaining feature gradient information (gradient weights) and the model's original attention scores, the key issue is how to effectively fuse these two types of information to enhance visualization. This step proposes a fusion strategy based on gradient information sorting weighted attention scores. This fusion strategy not only considers the important areas identified by the original attention mechanism, but also combines gradient-based saliency information to produce a more comprehensive and accurate heatmap representation. First, for each pathological slice instance, the present invention has obtained two types of weight information:

[0079] 1. Raw Attention Score , generated by the attention mechanism of the multi-instance learning model;

[0080] 2. Gradient Weight , obtained by the gradient calculation method described in the previous step.

[0081] In order to eliminate the difference in the numerical distribution of the two weights, the present invention first normalizes them separately. First, the original attention weight is normalized by "min-max" to obtain the normalized attention score :

[0082]

[0083] For the gradient weight, it can be found that its numerical distribution shows the characteristics of being dispersed at both ends and concentrated in the middle. The use of traditional normalization methods will result in the lack of discrimination of other values ​​except the maximum and minimum values, which will cause the weighted attention score to show the same trend. Therefore, the present invention adopts a normalization strategy based on sorting. Specifically, the present invention first sorts the original gradient information in ascending order, and then converts the gradient weight of each instance into its relative ranking among all instances to obtain the sorted gradient weight :

[0084]

[0085] in, represents the sorting function, For example, given n instances, if the original gradient information of an instance is the largest among all instances, according to the above formula, its Will be assigned a value of n, divided by n, and finally the gradient weight of this instance is normalized to 1. This sorting-based normalization method is insensitive to outliers and can better preserve the relative importance of gradient weights.

[0086] Then, the present invention adopts a simple and effective multiplication fusion strategy to multiply the normalized attention weights with the sorted gradient weights to obtain the fused instance weights. :

[0087]

[0088] Finally, the fused weights are normalized again to obtain the visualization score of each instance. :

[0089]

[0090] The advantage of this fusion strategy lies in the fact that gradient weights can identify regions that have a real impact on the model's decisions, while attention weights capture regions deemed important by the model. Combining these two approaches preserves the characteristics of the attention mechanism while enhancing the model's sensitivity to key regions through gradient information. This fusion approach demonstrates improved localization capabilities, particularly in areas where lesion features are less distinct but nonetheless have a significant impact on classification results.

[0091] S6. Post-process the fusion weights and output the lesion area positioning results (output the classification results with lesion area positioning, providing doctors with auxiliary diagnosis basis with both classification accuracy and positioning interpretability. The entire process only requires slice-level category labels to achieve accurate weak-supervised segmentation).

[0092] In this embodiment, if Figure 2 As shown in , after obtaining the visualization score corresponding to each instance, the generated original heat map is composed of several fixed-size image blocks, which will show obvious block boundaries when displayed directly. Figure 2 As shown in the upper left corner, the present invention uses Gaussian blur to weaken hard edges and make the transition between adjacent blocks more natural. After the above processing, the image cannot intuitively show the segmentation results. Therefore, the system uses the Otsu method to adaptively calculate the optimal threshold for binary segmentation, forming an accurate black and white mask image, which is then converted into a semi-transparent mask to color and cover the key areas in the original pathological image, realizing the automatic segmentation of abnormal areas in pathological tissue, greatly improving the efficiency and accuracy of doctors' diagnosis, and evaluating the base.

[0093] like Figure 3 As shown, the left side is the "true label" and the right side is the "segmentation result based on category label".

[0094] The "true label" on the left: Through manual and precise annotation, the lesion area is clearly marked in red, representing the exact location and range of the lesion area, and serves as a standard reference for comparison.

[0095] "Segmentation results based on category labels" on the right: Using the weakly supervised pathology image segmentation method of the present invention, segmentation is performed only relying on slice-level category labels, and the identified lesion area is presented in orange.

[0096] By comparison, it can be seen that the segmentation result on the right has a high degree of overlap with the lesion area of ​​the true label on the left, indicating that the method of the present invention can still locate the lesion area more accurately under the weak supervision condition of only using category labels (non-fine labeling), verifying the effectiveness and practicality of this method in pathological image segmentation, achieving lesion recognition that is close to the precise labeling effect, and reflecting the advantage of the invention in reducing labeling costs while ensuring segmentation accuracy.

[0097] Example 2

[0098] Based on the same concept, the present invention also proposes a pathological image segmentation device based on category labels, comprising:

[0099] A block division module, used for dividing the full-field digital pathology image into multiple image blocks;

[0100] A feature extraction module, configured to extract a feature map containing local texture and global semantic information from the image block, wherein the feature extraction module includes ResNet, Vision Transformer (ViT), or a large pre-trained model UNI for pathology;

[0101] Weakly supervised localization module, including:

[0102] An attention calculation unit, configured to calculate an attention score of each image block through an attention mechanism based on the feature map;

[0103] A gradient calculation unit, configured to calculate the gradient weight of each image block through gradient analysis based on the feature map;

[0104] A fusion unit, configured to normalize and fuse the attention score and the gradient weight to obtain a fusion weight;

[0105] The post-processing module is used to perform Gaussian blur processing on the fusion weights, generate a lesion area mask based on the Otsu method and overlay it on the original pathological image, and output the segmentation result.

[0106] Example 3

[0107] This embodiment also provides an electronic device, referring to Figure 2 , includes a memory 404 and a processor 402, wherein the memory 404 stores a computer program, and the processor 402 is configured to run the computer program to perform the steps in any of the above method embodiments.

[0108] Specifically, the processor 402 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits for implementing the embodiments of the present invention.

[0109] Memory 404 may include a large-capacity memory 404 for data or instructions. By way of example, and not limitation, memory 404 may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 404 may include removable or non-removable (or fixed) media. Where appropriate, memory 404 may be internal or external to the data processing device. In certain embodiments, memory 404 is non-volatile memory. In certain embodiments, memory 404 includes read-only memory (ROM) and random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically alterable ROM (EAROM) or a flash memory (FLASH), or a combination of two or more of these. In appropriate circumstances, the RAM may be a static random access memory (SRAM) or a dynamic random access memory (DRAM), wherein the DRAM may be a fast page mode dynamic random access memory 404 (FPMDRAM), an extended data output dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0110] The memory 404 may be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor 402 .

[0111] The processor 402 reads and executes computer program instructions stored in the memory 404 to implement any one of the pathological image segmentation methods based on category labels in the above embodiments.

[0112] Optionally, the electronic device may further include a transmission device 406 and an input / output device 408 , wherein the transmission device 406 is connected to the processor 402 , and the input / output device 408 is connected to the processor 402 .

[0113] Transmission device 406 can be used to receive or transmit data via a network. Specific examples of such networks may include wired or wireless networks provided by the electronic device's communications provider. In one embodiment, the transmission device includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, transmission device 406 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0114] The input / output device 408 is used to input or output information.

[0115] Example 4

[0116] This embodiment further provides a readable storage medium, in which a computer program is stored. The computer program includes a program code for controlling a process to execute the process. The process includes the pathological image segmentation method based on category labels according to the first embodiment.

[0117] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be repeated here.

[0118] In general, various embodiments may be implemented in hardware or dedicated circuitry, software, logic, or any combination thereof. Some aspects of the invention may be implemented in hardware, while other aspects may be implemented in firmware or software executed by a controller, microprocessor, or other computing device, but the invention is not limited thereto. Although various aspects of the invention may be shown and described as block diagrams, flow charts, or using some other graphical representation, it should be understood that, as non-limiting examples, the blocks, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuitry or logic, general-purpose hardware or a controller or other computing device, or some combination thereof.

[0119] The embodiments of the present invention may be implemented by computer software that is executable by a data processor of a mobile device, such as in a processor entity, or by hardware, or by a combination of software and hardware. Computer software or programs (also referred to as program products) including software routines, applets and / or macros may be stored in any device-readable data storage medium, and they include program instructions for performing specific tasks. A computer program product may include one or more computer executable components that are configured to perform an embodiment when the program is run. One or more computer executable components may be at least one software code or a portion thereof. In addition, it should be noted at this point that, for example, Figure 1 Any block of the logic flow in the program may represent program steps, or interconnected logic circuits, blocks and functions, or a combination of program steps and logic circuits, blocks and functions. The software may be stored on physical media such as memory chips or memory blocks implemented within the processor, magnetic media such as hard disks or floppy disks, and optical media such as, for example, DVDs and their data variants, CDs, etc. Physical media are non-transitory media.

[0120] Those skilled in the art should understand that the various technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the various technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0121] The above embodiments merely illustrate several embodiments of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art would be able to make various modifications and improvements without departing from the spirit of the present invention, all of which fall within the scope of the present invention. Therefore, the scope of the present invention shall be determined by the appended claims.

Claims

1. A pathological image segmentation method based on category labels, characterized in that: The following steps are involved: receiving a full-field digital pathology image and dividing it into a plurality of image blocks; Extract local texture features and global semantic features of each image block through a pre-trained feature extractor to generate a feature map; wherein the feature extractor includes ResNet, Vision Transformer or a large pre-trained model UNI for pathology; Based on the feature map, an attention score for each image block is calculated using an attention mechanism, including: using an attention network based on multi-instance learning, using slice-level category labels as supervisory signals, performing a nonlinear transformation on the feature vectors of each image block using a learnable parameter matrix, and then normalizing the transformed feature vectors to obtain an attention score; the attention score satisfies a normalization condition and is used to measure the contribution of the image block to the classification result; Based on the feature map, a gradient weight of each image block is calculated through gradient analysis, including: taking partial derivatives of the feature dimensions of the image block based on the predicted score of the target category and performing average pooling to obtain the gradient weight of the feature on the classification result; the gradient weight is used to measure the actual impact of the image block feature on the classification result; Perform min-max normalization on the attention score, normalize the gradient weight according to the numerical order, convert the gradient weight into the relative ranking weight in all image blocks, and obtain the fusion weight by multiplying the normalized attention score with the normalized gradient weight; Post-processing of the fusion weights includes: Gaussian blur denoising of the fusion weights; adaptive calculation of the binarization threshold based on the Otsu method and generation of a black and white mask; and superimposition of the black and white mask onto the original pathological image to visualize the lesion area. Output the lesion area positioning results.

2. A pathological image segmentation device based on category labels, characterized in that: include: A block division module, used for dividing the full-field digital pathology image into multiple image blocks; A feature extraction module, configured to extract a feature map containing local texture and global semantic information from the image block, wherein the feature extraction module includes ResNet, Vision Transformer, or a pathology-specific pre-trained large model UNI; Weakly supervised localization module, including: an attention calculation unit, configured to calculate an attention score for each image block based on the feature map using an attention mechanism, comprising: employing an attention network based on multiple instance learning, using slice-level category labels as supervisory signals, performing a nonlinear transformation on the feature vectors of each image block using a learnable parameter matrix, and then normalizing the transformed feature vectors to obtain an attention score; wherein the attention score satisfies a normalization condition; A gradient calculation unit is used to calculate the gradient weight of each image block through gradient analysis based on the feature map, including: taking partial derivatives of the feature dimensions of the image block based on the predicted score of the target category and performing average pooling to obtain the gradient weight of the feature pair classification result; The fusion unit is used to perform min-max normalization on the attention scores, normalize the gradient weights according to the numerical order, convert the gradient weights into relative ranking weights in all image blocks, and obtain the fusion weight by multiplying the normalized attention scores with the normalized gradient weights; The post-processing module is used to perform Gaussian blur denoising on the fusion weights, adaptively calculate the binarization threshold based on the Otsu method and generate a black and white mask; superimpose the black and white mask on the original pathological image to realize the visualization of the lesion area, and output the segmentation result.

3. An electronic device comprising a memory and a processor, characterized in that: The memory stores a computer program, and the processor is configured to run the computer program to execute the pathological image segmentation method based on category labels according to claim 1.

4. A readable storage medium, characterized in that: The readable storage medium stores a computer program, wherein the computer program includes a program code for controlling a process to execute a process, wherein the process includes the pathological image segmentation method based on category labels according to claim 1 .

Images