Pathological image universal segmentation method based on deep style features

Abstract

The invention discloses a general pathological image segmentation method based on depth style features. The method comprises the following steps: 1, preprocessing a pathological image through a preprocessing module; 2, inputting the preprocessed pathological image into a shared encoder module, and extracting multi-scale semantic features of the preprocessed pathological image to obtain a deep feature map; 3, inputting the deep feature map into a depth style feature extraction module, and outputting style features; 4, the task label and the learnable task embedding matrix are input into a task feature coding module, and dynamic task features are generated; 5, a task prompt generation module fuses the style features and the dynamic task features to generate a task prompt; 6, the task prompt is input into a dynamic convolution module, and dynamic convolution parameters matched with the current task are generated; and 7, inputting the shallow layer features and the dynamic convolution parameters output by the decoder into a multi-task dynamic head module, and predicting a segmentation mask. According to the invention, the multi-task segmentation precision and efficiency of the pathological image are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical image processing, and in particular is a general pathological image segmentation method based on deep style features. Background Art

[0002] Pathological images are remarkably complex and diverse, posing enormous challenges to the pathological analysis process, primarily in the following aspects: 1) Multi-task and multi-modal characteristics. Pathological diagnosis involves multiple organs (such as liver, lung, and breast) and multiple staining methods (such as hematoxylin-eosin staining (H&E) and immunohistochemistry staining (IHC). Even for the same tissue, multiple tasks such as nucleus segmentation, glandular typing, and cell membrane localization may be required. For example, in the pathological analysis of breast cancer, tumor cells, immune cells, and vascular structures must be identified simultaneously. Moreover, the target morphology and texture features of different tasks vary greatly; 2) Texture randomness and staining variability. Compared with radiological images (such as CT and MRI), the cell arrangement and tissue structure of pathological images are highly random, and the morphology of the same type of cells varies in different slices. It can reach 40%-60%. In addition, the staining process is affected by factors such as reagent concentration and staining time, resulting in fluctuations in image color intensity (such as the color of H&E-stained cell nuclei changes from dark blue to light purple). Moreover, the differences in staining methods between different tasks (such as the brown-yellow mark of IHC and the red-blue contrast of H&E) further aggravate feature confusion; 3) Traditional image segmentation models are relatively inefficient, such as NucleiSegNet and Hover-Ne, which are usually designed for a single task. Each task requires an independent training model. Taking 10 common pathology tasks as an example, the traditional method needs to maintain 10 independent models, and the total amount of model parameters exceeds 20GB, resulting in high training costs and difficulty in sharing cross-task features (such as the common morphology of cell nuclei in different tasks), resulting in resource waste and performance bottlenecks.

[0003] In order to solve the problems existing in traditional image segmentation models in multi-task segmentation, universal segmentation models have become an important research direction. The core idea is to achieve "one model for multiple tasks" by sharing network parameters and combining task-specific inputs. Typical technical routes include: 1) Dynamic parameter generation driven by task prompts, among which: DoDNet and its variant TransDoDNet generate dynamic convolution kernels through one-hot task labels to guide the shared network to focus on specific task targets; DoDNet fuses task labels with image features to generate convolution filters for the current task, which are applied to the segmentation head to adjust the decision logic; 2) Decoder guidance with learnable prompts, models such as UniSeg propose learnable task prompt vectors, and enhance the decoder's sensitivity to task-specific features through the attention mechanism. Such methods have achieved significant results in radiological images (such as CT multi-organ segmentation), with an average Dice coefficient of more than 0.85, but the performance drops sharply in pathological images. For example, UniSeg's Dice in the CoNSeP dataset is only 0.423, which is because the feature fluctuations of pathological images make the task prompts ineffective; 3) Migration application of basic models, SAM (Segment Anything The MedSAM model and its medical variant achieve zero-shot segmentation through large-scale pre-training. However, they rely on user-input bounding boxes or point prompts and cannot automatically handle multi-class segmentation tasks (for example, distinguishing seven cell types at the same time). In addition, the segmentation accuracy of cell boundaries is insufficient, with an average edge positioning error of 12 pixels, which makes it difficult to meet the clinical requirements of pathological diagnosis.

[0004] However, existing general segmentation models mainly focus on enabling a single model to handle multiple tasks through task prompt mechanisms, including dynamic parameter generation methods driven by task prompts (such as DoDNet and TransDoDNet) and decoder methods guided by learnable prompts (such as UniSeg, SAM and MedSAM). The core idea of this type of technology is to fuse image features and task features into task prompts, and guide the model to focus on specific task goals by dynamically adjusting network parameters or attention mechanisms. Among them: DoDNet uses one-hot task labels to generate dynamic convolution filters, which are applied to the last layer of the shared network to adapt to different tasks; UniSeg uses learnable task prompt vectors to enhance the decoder's attention to task-specific features; SAM and MedSAM rely on manually input point or box prompts, generate learnable prompt embeddings through position encoding, and after fusing with the features output by the image encoder, generate segmentation masks through the decoder.

[0005] Although the existing general segmentation models have achieved remarkable results in radiological images (such as CT multi-organ segmentation), the following technical difficulties still exist in the process of pathological image segmentation: First, the texture randomness and staining variability of pathological images lead to the instability of image features. Among them: at the texture level, unlike the regularity of organ structure in radiological images, the cell arrangement and tissue structure of pathological images are highly random. The morphology and texture of the same type of cells or tissues in different slices vary greatly, and there is no fixed arrangement pattern. The same type of tissue may appear in various forms such as "scattered distribution" or "dense clusters" or overlapping adhesions under high-power microscopes. The traditional feature extraction methods based on pixel intensity or local gradient (such as shallow features of U-Net) cannot stably represent the target. At the color level, the staining difference causes the color distribution of the same structure (such as cell nucleus) in different images to differ by more than 30%. For example, the brightness value of the cell nucleus stained by H&E fluctuates in the range of [50,150]. The positive cells stained by IHC show a change from light brown to dark brown due to different antibody labeling concentrations. Traditional normalization methods (such as instance normalization) are difficult to eliminate the impact of such differences on features. Second, the task hint mechanism fails to effectively separate image style from task semantics, resulting in cross-task The generalization ability is poor, which is specifically reflected in the following aspects: 1) Existing methods rely on the fusion of shallow image features (such as pixel values, low-dimensional semantics) and task labels to generate prompts, without distinguishing between the inherent style of the image (such as texture, color distribution) and task-specific semantics (such as the morphological definition of the target to be segmented). The style differences of pathological images (such as the global color shift caused by the staining method) and task semantics (such as the target definition of cell nucleus segmentation and gland segmentation) are intertwined, so that the task prompt vector contains both noisy style information and effective semantic information, resulting in feature confusion in the segmentation model when processing new tasks or new staining types. For example, UniSeg in Co The Dice coefficient for the NSeP dataset (multi-class cell segmentation) was only 0.423, reflecting that its hint mechanism was unable to cope with the dual interference of complex texture and staining variations. 2) Feature fluctuations in pathological images caused the hint vector to fail to accurately reflect the task objective. For example, when the nuclei of two H&E images appear different colors due to staining differences, the task hint of the traditional model may mistakenly identify them as different categories, resulting in segmentation errors. In particular, in the Her2 dataset (IHC staining), TransDoDNet achieved a Dice coefficient of only 0.413, significantly lower than the Dice coefficient of 0.00 for the single-task model.750, demonstrating that existing hinting mechanisms fail in cross-modal segmentation scenarios. 3) In actual pathology image analysis, pathological diagnosis often involves the simultaneous segmentation of multiple cell types (e.g., seven or more cell types) or substructures (e.g., nuclei and cell membranes), requiring models with sophisticated boundary perception and category discrimination capabilities. However, SAM and MedSAM cannot automatically handle multi-class segmentation tasks, and their accuracy and generalization capabilities for boundary segmentation of closely adjacent cells are insufficient, with an average edge positioning error of 12 pixels. 4) The fixed channel output design of existing general segmentation models (e.g., distinguishing only between background and a single target) makes it difficult to scale to multi-class scenarios, resulting in increased inter-class confusion. On the Kumar dataset, existing general segmentation models have a nucleus misclassification rate of 25%.

[0006] In short, the existing general segmentation models lack targeted design for the multi-class segmentation needs of pathological images, which is reflected in the following aspects: 1) The feature representation is not adapted to the characteristics of pathological images, and task prompts are generated only based on image pixels or shallow semantic features. There is no robust feature representation method designed for the texture randomness, staining variation and other characteristics of pathological images, and it is unable to cope with the texture randomness and staining variation of pathological images; 2) Task association modeling is insufficient, ignoring the common characteristics between different pathological tasks. For example, both cell nucleus segmentation and gland segmentation require the identification of cell boundaries, and fail to effectively utilize multi-task collaborative learning to improve performance, resulting in a mismatch between the task prompt mechanism and the multi-class segmentation requirements, poor cross-task generalization ability, and insufficient boundary positioning and category distinction capabilities; 3) The segmentation head design lacks flexibility, and the fixed channel output cannot adapt to the multi-class segmentation requirements, such as the 7 cell types in the CoNSeP dataset, resulting in redundant model parameters and low inference efficiency. Summary of the Invention

[0007] The purpose of the present invention is to provide a general pathological image segmentation method based on deep style features, which improves the accuracy and efficiency of multi-task segmentation of pathological images and has good generalization ability and strong applicability.

[0008] The present invention is achieved through the following technical solutions:

[0009] A general pathological image segmentation method based on deep style features includes the following steps:

[0010] Step 1: preprocessing the pathological image through a preprocessing module to obtain a preprocessed pathological image I;

[0011] Step 2: Using the U-Net architecture encoder as a shared encoder module, the preprocessed pathological image I is input into the shared encoder module to extract the multi-scale semantic features of the preprocessed pathological image, and obtain a deep feature map F containing deep semantic and structural information of the image;

[0012] Step 3: Input the deep feature map F into the deep style feature extraction module, and output the style feature G including the global texture and color distribution of the image. The specific process is as follows:

[0013] Step 3.1: Channel feature decomposition

[0014] Split the deep feature map F into C 2D matrices F by channel l , F l Each represents the spatial distribution of a semantic feature, F l ∈R H×W (l=1,2,...,C), C is the number of channels;

[0015] Step 3.2, Gram matrix calculation

[0016] Calculate the matrix F of channel i i and the matrix F of channel j j The global correlation of is expressed as:

[0017]

[0018] Where: G i,j is the inner product of the features of channel i and channel j, reflecting F i and F j The co-occurrence frequency in the image; h∈H, represents the height of the image; ω∈W, represents the width of the image;

[0019] By G i,j The C×C Gram matrix is constructed, where the diagonal elements represent the energy intensity of each channel feature, reflecting the dominant components of color and texture, and the off-diagonal elements represent the dependency between features, reflecting the complexity of the texture structure;

[0020] Step 3.3: Style feature encoding

[0021] Perform style encoding on the C×C Gram matrix to obtain the style feature G including the global texture and color distribution of the image;

[0022] Step 4: Input the one-hot encoded task label T and the learnable task embedding matrix Q into the task feature encoding module. Combine the one-hot encoded task label and the task embedding matrix through matrix multiplication to extract the semantic features of the task and obtain the original task feature vector V′. Normalize the eigenvalues of the original task feature vector V′ through the sigmoid function to generate the dynamic task feature V, which is expressed as:

[0023] V′=T·Q

[0024] V = sigmoid(V′)

[0025] Where T∈R1×N , Q∈R N×C , V′∈R 1×C , V∈R 1×C , N is the number of tasks, C is the number of channels;

[0026] Step 5: The task prompt generation module expands the style feature G and the dynamic task feature V into a 1×C×1×1 tensor, combines the style feature G and the dynamic task feature V through matrix multiplication, and generates a task prompt P that contains the task goal and the image's inherent style information, expressed as:

[0027] P=V·G

[0028] Where P∈R 1×C ;

[0029] Step 6: Input the task prompt P into the dynamic convolution module to generate the dynamic convolution parameters θ that are suitable for the current task;

[0030] Step 7: Input the shallow features and dynamic convolution parameters ω output by the decoder into the multi-task dynamic head module, use the dynamic convolution parameters θ to perform a 1×1 convolution operation on the shallow features to generate a multi-channel prediction map, and then dynamically filter out the required channels from the multi-channel prediction map according to the current task type. Then, use the softmax function to convert the predicted values of the required channels into pixel-level segmentation probabilities to predict the segmentation mask.

[0031] Furthermore, the specific process of step 1 is: by cropping or filling, the resolution of the pathological image is fixed to 512×512 pixels, and the original RGB color space of the pathological image is retained to obtain the preprocessed pathological image I, I∈R 512 ×512×3 .

[0032] Furthermore, the specific process of step 2 is as follows: first, the resolution of the preprocessed pathological image I is gradually reduced to 16×16 pixels through a four-layer downsampling module, and the number of channels is increased from 3 to 256, thereby proposing multi-scale semantic features, so that the feature map of the last layer contains the deep semantic and structural information of the image, that is, the deep feature map F, which is expressed as:

[0033] F∈R H×W×C

[0034] Where: H=W=16, C=256.

[0035] Furthermore, the downsampling module includes two 3×3 convolutions, instance normalization and ReLU activation.

[0036] Furthermore, the element range of the dynamic task feature V in step 4 is limited to (0, 1).

[0037] Furthermore, the specific process of step 6 is: transforming the task prompt P through the 1×1 convolution layer φ to generate the dynamic convolution parameter θ adapted to the current task, which is expressed as:

[0038]

[0039] The convolutional layer φ contains C×C weights, corresponding to the number of channels of the dynamic head. θ changes dynamically with the task prompt P, so that the segmentation operation can adapt to adjust the sensitivity to different targets.

[0040] The present invention has the following beneficial technical effects:

[0041] 1) Significantly improve the segmentation accuracy and stability of pathological images: First, common pathological features are extracted through a shared encoder, and then the Gram matrix is used to capture the correlation between feature map channels to encode the texture and color distribution of the image, forming a stable deep style feature. This overcomes the problem of unstable pathological image features caused by the randomness of pathological image texture and staining variability. The reason is that the staining process of pathological images is affected by multiple factors such as reagent concentration and staining time. The staining results of different samples vary greatly. Shallow image features are easily interfered by these factors, resulting in inaccurate segmentation results. In contrast, the deep style features extracted by the present invention show strong robustness to staining differences and can more accurately reflect the essential characteristics of the image. Therefore, the average Dice coefficient of the model of the present invention on 10 public data sets reaches 0.808. In particular, on the Her2 data set with significant staining differences, the Dice coefficient of the existing optimal method TransDoDNet is 0.413, and the Dice coefficient of the present invention is 0.755, an improvement of 34.2%. The accurate segmentation results provide clinicians with detailed information such as cell type and morphology for diagnostic decisions, and have significant clinical application value.

[0042] 2) Achieved efficient multi-task segmentation: A general image segmentation framework based on a dynamic hint mechanism fuses deep image style features with task features to form stable task hints, thereby driving the dynamic convolution module to adaptively generate task-specific segmentation parameters. It fully utilizes the synergistic effect between multiple tasks, realizes cross-task feature sharing, effectively separates the inherent image style from task semantics, and solves the interference of pathological texture randomness and staining variability on task hints. The model can efficiently handle a variety of different pathological image segmentation tasks under a single architecture, such as multi-staining modalities such as H&E and IHC, as well as complex segmentation tasks such as cell nucleus segmentation, gland localization, and multi-class cell classification. It has good multimodal compatibility and easy multi-task deployment. There is no need to train a separate model for each task, which reduces the training cost and model complexity, reduces computing resource consumption, and improves the model's inference efficiency. For pathological images with a resolution of 512×512, the inference speed can reach more than 10 images per second, meeting the needs of rapid clinical diagnosis.

[0043] 3) Strong generalization and adaptability: In clinical practice, pathological images come from a wide range of sources, and samples from different hospitals and laboratories may vary greatly. Traditional models often require a large amount of additional training data for adjustment when faced with these differences, otherwise the segmentation performance will drop significantly. However, the present invention combines deep style features and dynamic prompt mechanisms to have good generalization capabilities for different types of pathological images, reducing dependence on specific data sets. Whether it is pathological images of different organs or images processed with different staining methods, the model can accurately perform segmentation through its adaptive parameter adjustment mechanism. Specifically, by fine-tuning the task embedding matrix, it can quickly adapt to new data sets or new pathological tasks without retraining the entire model, greatly improving the application scope and flexibility of the model, and providing a new general framework for pathological image segmentation. It has significant technical foresight in promoting the development of large medical imaging models.

[0044] 4) Reduced model complexity and computing resource loss: The modules have a clear division of labor and work in coordination. In particular, common features are extracted by sharing the encoder module, which reduces the redundancy of feature extraction. The dynamic convolution module adaptively generates parameters according to task prompts, avoiding the limitations of fixed parameters in traditional methods. This makes the model parameter count relatively small, reduces computational complexity, reduces computing resource loss, reduces hardware requirements, and improves the practicality of the segmentation model.

[0045] 5) Provides rich clinical diagnostic information: Adaptive selection of output channels is achieved through the multi-task dynamic head module, supporting efficient switching across tasks and precise segmentation of multiple types of targets, and can achieve efficient segmentation of "one model, multiple tasks". By outputting various types of segmentation results, such as cell nucleus masks, gland contour maps, multi-class cell classification maps, etc., these segmentation results can intuitively display the cell and tissue structure, cell size and morphology, and distribution and status of immune cells in pathological images, providing doctors with rich diagnostic information, and then evaluating the grade and stage of tumors, and providing a basis for decision-making on immunotherapy. In addition, through statistical information on cell number and density, pathological diagnosis is further assisted; in short, the present invention can better meet clinical needs and improve the accuracy and efficiency of pathological diagnosis.

[0046] 6) Promote the standardization and automation of pathological image analysis: The universal segmentation framework and stable segmentation performance provide a basis for the standardization of pathological image analysis, reducing diagnostic errors caused by human factors and method differences in different clinical institutions. At the same time, there is no need for doctors to manually observe and analyze pathological images, and pathological image analysis can be completed quickly and accurately, reducing the workload of doctors, improving the accuracy and efficiency of diagnosis, and helping to promote the development of pathological image analysis towards standardization and automation, thereby improving the level of the entire pathological diagnosis field. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 This is a schematic structural diagram of a suitcase that facilitates remote monitoring according to the present invention;

[0048] Figure 2 It is a flowchart of the present invention;

[0049] Figure 3 The segmentation results of the pathological image using the existing segmentation models SAM, MedSAM and the segmentation model of the present invention;

[0050] Figure 4 The segmentation results of the segmentation model of the present invention and the existing model on different data sets. DETAILED DESCRIPTION

[0051] The present invention will be further described in detail below with reference to specific embodiments, which are intended to explain the present invention rather than to limit it.

[0052] refer to Figure 2As shown, this embodiment proposes a general pathological image segmentation model based on deep style features, including a preprocessing module, a shared encoder module, a deep style feature extraction module, a task feature encoding module, a task prompt generation module, a dynamic convolution module and a multi-task dynamic head module. After the pathological image to be segmented is preprocessed, the shared encoder module extracts a deep feature map, and the deep feature map passes through the deep style feature extraction module to output style features. The task prompt generation module fuses the style features and the dynamic task features output by the task feature encoding module into task prompts, driving the dynamic convolution module to generate adaptive segmentation parameters, which solves the defects of the inherent parameters of the existing model. The multi-task dynamic head module uses the adaptive segmentation parameters to achieve task-specific segmentation and outputs the segmentation results of the corresponding category, namely the segmentation mask, to achieve efficient switching of "one model multiple tasks" and avoid redundant calculations. In short, the workflow of the general segmentation model proposed in this embodiment can be summarized as a closed loop of "image preprocessing → feature extraction → dual feature fusion → dynamic parameter generation → task adaptive segmentation". The modules achieve collaborative processing through data transmission. For the specific process, see Figure 1 , it realizes the universal segmentation of pathological images and can efficiently handle complex scenes with multiple organs, multiple stains, and multiple types of targets. While keeping the model lightweight, it significantly improves the segmentation accuracy and generalization ability.

[0053] Figure 1 As shown in FIG, a general pathological image segmentation method based on deep style features includes the following steps:

[0054] Step 1: Preprocess the pathological image through the preprocessing module to obtain a preprocessed pathological image I. The specific process is as follows:

[0055] By cropping or padding, the resolution of the pathological image of any size is fixed to 512×512 pixels, and the original RGB color space of the pathological image is retained to obtain the preprocessed pathological image I, I∈R 512×512×3 ;

[0056] The preprocessing module supports multimodal staining types such as H&E and IHC. Considering the compatibility of subsequent modules, the sizes of pathology images of different sizes are adjusted to make the sizes of subsequent input images consistent. In order to preserve the integrity of the staining features, the original RGB color space of the pathology images is retained without additional normalization or standardization.

[0057] Step 2: Using the encoder of the U-Net architecture as the shared encoder module, the preprocessed pathological image I is input into the shared encoder module to extract the multi-scale semantic features of the preprocessed pathological image, and obtain a deep feature map F containing the deep semantic and structural information of the image. The specific process is as follows:

[0058] First, the resolution of the preprocessed pathological image I is gradually reduced to 16×16 pixels through a four-layer downsampling module, and the number of channels is increased from 3 to 256. In this way, multi-scale semantic features are proposed, so that the feature map of the last layer contains the deep semantic and structural information of the image, that is, the deep feature map F, which is expressed as:

[0059] F∈R H×W×C

[0060] Where: H = W = 16, C = 256;

[0061] The downsampling module includes two 3×3 convolutions, instance normalization and ReLU activation;

[0062] Step 3: Input the deep feature map F into the deep style feature extraction module, and output the style feature G including the global texture and color distribution of the image. The specific process is as follows:

[0063] Step 3.1: Channel feature decomposition

[0064] Split the deep feature map F into C 2D matrices F by channel l , F l Both represent the spatial distribution of a semantic feature, F l ∈R H×W (l=1,2,...,C), C is the number of channels;

[0065] Step 3.2, Gram matrix calculation

[0066] Calculate the matrix F of channel i i and the matrix F of channel j j The global correlation of is expressed as:

[0067]

[0068] Where: G i,j is the inner product of the features of channel i and channel j, reflecting F i and F j The co-occurrence frequency in the image; h∈H, represents the height of the image; ω∈W, represents the width of the image;

[0069] By G i,j The C×C Gram matrix is constructed, where the diagonal elements represent the energy intensity of each channel feature, reflecting the dominant components of color and texture, and the off-diagonal elements represent the dependency between features, reflecting the complexity of the texture structure;

[0070] Step 3.3: Style feature encoding

[0071] Perform style encoding on the C×C Gram matrix to obtain the style feature G including the global texture and color distribution of the image;

[0072] Step 4: Input the one-hot encoded task label T and the learnable task embedding matrix Q into the task feature encoding module. Combine the one-hot encoded task label and the task embedding matrix through matrix multiplication to extract the semantic features of the task and obtain the original task feature vector V′. Normalize the eigenvalues of the original task feature vector V′ through the sigmoid function to generate the dynamic task feature V, which is expressed as:

[0073] V′=T·Q

[0074] V = sigmoid(V′)

[0075] Where: T∈R 1×N ;Q∈R N×C , is a learnable parameter, which learns the deep features of different tasks through training; V′∈R 1 ×C ; V∈R 1×C , N is the number of tasks, for example, 10 data sets correspond to 10 tasks; C is the number of channels;

[0076] The dynamic task feature V is used to represent the target semantics of the current task, such as "nucleus segmentation" and "gland localization". In order to enhance feature stability, the element range of the dynamic task feature V is limited to (0,1);

[0077] Step 5: The task prompt generation module expands the style feature G and the dynamic task feature V into a 1×C×1×1 tensor, combines the style feature G and the dynamic task feature V through matrix multiplication, and generates a task prompt P that contains the task goal and the image's inherent style information, expressed as:

[0078] P=V·G

[0079] Where P∈R 1×C ,P serves as the guidance signal for dynamic convolution;

[0080] By replacing the original image features with style features G and fusing them with dynamic task features V, a stable task prompt P is formed. This effectively separates the inherent image style from the task semantics, and eliminates the interference of feature fluctuations on task prompts. This enables the model to robustly handle complex segmentation tasks such as H&E and IHC multi-staining modalities, as well as cell nuclei, glands, and multiple cell types.

[0081] Step 6: Input the task prompt P into the dynamic convolution module to generate the dynamic convolution parameter θ that is suitable for the current task. The specific process is as follows:

[0082] The task prompt P is transformed through a 1×1 convolutional layer φ to generate dynamic convolution parameters θ adapted to the current task, expressed as:

[0083]

[0084] The convolutional layer φ contains C×C weights, corresponding to the number of channels of the dynamic head. θ changes dynamically with the task prompt P, enabling the segmentation operation to adapt to adjust the sensitivity to different targets. For example, it can enhance the boundary recognition of IHC-positive cells, overcoming the defects of traditional risky fixed parameters.

[0085] Step 7: The multi-task dynamic head module predicts the segmentation mask. The specific process is as follows:

[0086] Step 7.1: Input the shallow features and dynamic convolution parameters ω output by the decoder into the multi-task dynamic head module, and then use the dynamic convolution parameters θ to perform a 1×1 convolution operation on the shallow features to generate a multi-channel prediction map;

[0087] Using shallow features for feature fusion can improve segmentation accuracy. The number of channels in the generated multi-channel prediction map can be set to 11, including: 0-background, 1-cell nucleus, 2-cell membrane, 3-gland, 4-7-multiple types of cells, etc.

[0088] Step 7.2: Dynamically select the required channels from the multi-channel prediction map based on the current task type. For example, for the CoNSeP dataset, 7 cell types and 1 background are required, for a total of 8 channels; for the GLAS dataset, only glands and background are required, for a total of 2 channels;

[0089] Step 7.3: Convert the predicted value of the desired channel into pixel-level segmentation probability through the softmax function and predict the segmentation mask;

[0090] By adaptively selecting output channels through a multi-task dynamic head module, it supports efficient switching across tasks and accurate segmentation of multiple classes of targets, thereby outputting segmentation masks, that is, task-specific segmentation results, such as: cell nucleus masks, gland outlines, or multi-class cell classification maps.

[0091] In order to verify the segmentation effect of the segmentation model proposed in this embodiment on pathological images, the existing segmentation model and the segmentation model of this embodiment are used to perform pathological image segmentation on different data sets. The results are shown in Table 1. Figure 3 and Figure 4 As shown:

[0092] Table 1 Segmentation results of different models on different datasets

[0093]

[0094] As can be seen from Table 1, the segmentation model of this embodiment achieves an average Dice coefficient of 0.808 on 10 public datasets. On the Her2 dataset with significant coloring differences, the Dice coefficient of the existing optimal method TransDoDNet is 0.413, while the Dice coefficient of this embodiment is 0.755, an improvement of 34.2%, indicating that this embodiment has more accurate segmentation results. In addition, the segmentation performance of the segmentation model of this embodiment on different datasets is verified when the one-hot encoded task label T, task embedding matrix Q, or style feature G are respectively removed. It can be seen that the one-hot encoded task label T, task embedding matrix Q, or style feature G are crucial to the performance of the segmentation model.

[0095] from Figure 3 It can be seen that the segmentation model of this embodiment has clearer boundaries for pathological segmentation than the existing models SAM and MedSAM.

[0096] from Figure 4 It can be seen that the segmentation results of the segmentation model of this embodiment are closer to the actual mask value than those of the existing models TransDoDNet, UniSeg, and DoDNet, indicating that the segmentation model of this embodiment has higher segmentation accuracy.

Claims

1. A general pathological image segmentation method based on deep style features, characterized by: The steps include: Step 1: preprocessing the pathological image through a preprocessing module to obtain a preprocessed pathological image I; Step 2: Using the U-Net architecture encoder as a shared encoder module, the preprocessed pathological image I is input into the shared encoder module to extract the multi-scale semantic features of the preprocessed pathological image, and obtain a deep feature map F containing deep semantic and structural information of the image; Step 3: Input the deep feature map F into the deep style feature extraction module, and output the style feature G including the global texture and color distribution of the image. The specific process is as follows: Step 3.1: Channel feature decomposition Split the deep feature map F into C 2D matrices F by channel l , F l Each represents the spatial distribution of a semantic feature, F l ∈R H×W (l=1,2,...,C), C is the number of channels; Step 3.2, Gram matrix calculation Calculate the matrix F of channel i i and the matrix F of channel j j The global correlation of is expressed as: Where: G i,j is the inner product of the features of channel i and channel j, reflecting F i and F j The co-occurrence frequency in the image; h∈H, represents the height of the image; ω∈W, represents the width of the image; By G i,j The C×C Gram matrix is constructed, where the diagonal elements represent the energy intensity of each channel feature, reflecting the dominant components of color and texture, and the off-diagonal elements represent the dependency between features, reflecting the complexity of the texture structure; Step 3.3: Style feature encoding Perform style encoding on the C×C Gram matrix to obtain the style feature G including the global texture and color distribution of the image; Step 4: Input the one-hot encoded task label T and the learnable task embedding matrix Q into the task feature encoding module. Combine the one-hot encoded task label and the task embedding matrix through matrix multiplication to extract the semantic features of the task and obtain the original task feature vector V′. Normalize the eigenvalues of the original task feature vector V′ through the sigmoid function to generate the dynamic task feature V, which is expressed as: V′=T·Q V = sigmoid(V′) Where T∈R 1×N , Q∈R N×C , V′∈R 1×C , V∈R 1×C , N is the number of tasks, C is the number of channels; Step 5: The task prompt generation module expands the style feature G and the dynamic task feature V into a 1×C×1×1 tensor, combines the style feature G and the dynamic task feature V through matrix multiplication, and generates a task prompt P that contains the task goal and the image's inherent style information, expressed as: P=V·G Where P∈R 1×C ; Step 6: Input the task prompt P into the dynamic convolution module to generate the dynamic convolution parameters θ that are suitable for the current task; Step 7: Input the shallow features and dynamic convolution parameters ω output by the decoder into the multi-task dynamic head module, use the dynamic convolution parameters θ to perform a 1×1 convolution operation on the shallow features to generate a multi-channel prediction map, and then dynamically filter out the required channels from the multi-channel prediction map according to the current task type. Then, use the softmax function to convert the predicted values of the required channels into pixel-level segmentation probabilities to predict the segmentation mask.

2. The general pathological image segmentation method based on deep style features according to claim 1 is characterized in that: The specific process of step 1 is: by cropping or filling, the resolution of the pathological image is fixed to 512×512 pixels, and the original RGB color space of the pathological image is retained to obtain the preprocessed pathological image I, I∈R 512×512×3 .

3. The general pathological image segmentation method based on deep style features according to claim 2 is characterized in that: The specific process of step 2 is as follows: first, the resolution of the preprocessed pathological image I is gradually reduced to 16×16 pixels through a four-layer downsampling module, and the number of channels is increased from 3 to 256, thereby proposing multi-scale semantic features, so that the feature map of the last layer contains the deep semantic and structural information of the image, that is, the deep feature map F, which is expressed as: F∈R H×W×C Where: H=W=16, C=256.

4. The general pathological image segmentation method based on deep style features according to claim 3 is characterized in that: The downsampling module consists of two 3×3 convolutions, instance normalization, and ReLU activation.

5. The general pathological image segmentation method based on deep style features according to claim 1, characterized in that: The element range of the dynamic task feature V in step 4 is limited to (0, 1).

6. The general pathological image segmentation method based on deep style features according to claim 1, characterized in that: The specific process of step 6 is: transform the task prompt P through the 1×1 convolution layer φ to generate the dynamic convolution parameter θ adapted to the current task, which is expressed as: The convolutional layer φ contains C×C weights, corresponding to the number of channels of the dynamic head. θ changes dynamically with the task prompt P, so that the segmentation operation can adapt to adjust the sensitivity to different targets.

Images